Resources2#15
Conversation
I didn't actually see this until after I went through your commit. I reject the notion regardless.
OMG... Don't flatter yourself. I reviewed the code, not you. Your last submission was written in a vacuum and needed a thorough code review to fully shake it out. There was a design or code issue in nearly every paragraph of code. Who that submission came from was only a side-note that affected the tone of a couple comments. Had I known the commit came from a GSoC student, for example, I would have toned those down. You, however, should know better. This submission is much much better. If I had to guess, I'd say it was roughly 1/3rd the size of the previous (now vaporized) submission, which would be pretty close to my estimate that the last submission was 300% of what it should have been. I think there are some areas that still need to be fleshed out and discussed, like whether we should allow arrays of urls in the values of the hash, or whether we should expose general access to the hash or only allow insertion via api... but all in all this is much more in tune with how rubygems works. |
|
As someone who likes to follow these tickets, and doesn't get the emails, holy crap it is next to impossible to follow these with certain members of the conversation deleting all their posts. If I may be so bold, trans, if you're going to go to the trouble of deleting all your posts, just delete the damned pull request. |
|
@ryan Fair enough. Let's let bygones be bygones, and move on from here. I made an update to the branch, that I think resolves the redundancy issue. I simply substituted I also converted to symbols as you suggested. The other details you mention are probably best hammered out by you and the rest of the RubyGems team. Note that I thought about a urls array too, but I think it might prove too problematic for effective utilization, eg. on the rubygems.org profiles. |
|
@erikh I tried to do exactly that, but could not find a way to delete the pull request. In a late night stupor of dismay I deleted my comments. I should not have done that. Sorry. I've since made a support request to remove the pull requests that have now been made redundant by new submissions. |
|
Looks like this is dead. |
When `bundle install` connects with a certification (CA) to a private RubyGems
HTTPS server emulated by WEBrick, the connection to the
https://localhost:18443/versions succeeded, but the connection to download gems
failed with the following error.
```
+ /home/jaruga/var/git/ruby/rubygems/bin/bundle config set --local ssl_ca_cert /home/jaruga/git/report-bundler-bundle-config-set-ssl_ca_cert/tmp/client/ssl/ca.crt
...
+ /home/jaruga/var/git/ruby/rubygems/bin/bundle install -V
Running `bundle install --verbose` with bundler 4.1.0.dev
Resolving dependencies because there's no lockfile
HTTP GET https://localhost:18443/versions
HTTP 206 Partial Content https://localhost:18443/versions
HTTP GET https://localhost:18443/versions
HTTP 200 OK https://localhost:18443/versions
Fetching gem metadata from https://localhost:18443/
Looking up gems ["hello"]
Resolving dependencies...
Using bundler 4.1.0.dev
1: bundler (4.1.0.dev) from /home/jaruga/var/git/ruby/rubygems/bundler/bundler.gemspec
Fetching hello 0.1.0
Retrying download gem from https://localhost:18443/ due to error (2/4): Gem::RemoteFetcher::FetchError SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)
Sleeping for 1.22 seconds before retry
Retrying download gem from https://localhost:18443/ due to error (3/4): Gem::RemoteFetcher::FetchError SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)
Sleeping for 2.26 seconds before retry
Retrying download gem from https://localhost:18443/ due to error (4/4): Gem::RemoteFetcher::FetchError SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)
Sleeping for 4.02 seconds before retry
Bundler::InstallError: Bundler::HTTPError: Could not download gem from https://localhost:18443/ due to underlying error <SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)>
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:406:in 'Bundler::RubygemsIntegration#download_gem'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:531:in 'block in Bundler::Source::Rubygems#download_gem'
/home/jaruga/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems.rb:1068:in 'Gem.time'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:530:in 'Bundler::Source::Rubygems#download_gem'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:459:in 'Bundler::Source::Rubygems#fetch_gem'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:443:in 'Bundler::Source::Rubygems#fetch_gem_if_possible'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:575:in 'Bundler::Source::Rubygems#rubygems_gem_installer'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:184:in 'Bundler::Source::Rubygems#download'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/installer/gem_installer.rb:29:in 'Bundler::GemInstaller#download'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/installer/parallel_installer.rb:148:in 'Bundler::ParallelInstaller#do_download'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/installer/parallel_installer.rb:132:in 'block in Bundler::ParallelInstaller#worker_pool'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:70:in 'Bundler::Worker#apply_func'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:65:in 'block in Bundler::Worker#process_queue'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:56:in 'Kernel#loop'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:56:in 'Bundler::Worker#process_queue'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:98:in 'block (2 levels) in Bundler::Worker#create_threads'
...
```
`Bundler::Fetcher` has the `#connection` calling `#bundler_cert_store` storing
`Bundler.settings[:ssl_ca_cert]` in the following part. It is used in some parts.
bundler/lib/bundler/fetcher.rb
...
def bundler_cert_store
...
ssl_ca_cert = Bundler.settings[:ssl_ca_cert] ||
(Gem.configuration.ssl_ca_cert if
Gem.configuration.respond_to?(:ssl_ca_cert))
...
end
```
However in the case of downloading gems in Bundler,
Bundler calls `Bundler::Source::Rubygems#download_gem`
calling `Bundler::Fetcher#gem_remote_fetcher`
calling `Bundler::Fetcher::GemRemoteFetcher`
extending `Gem::RemoteFetcher` managing `@cert_files` for RubyGems.
Therefore, the `Bundler::Fetcher::GemRemoteFetcher` needs to update the `@cert_files`
by adding the value of the `Bundler.settings[:ssl_ca_cert]`.
As `Gem::Request.configure_connection_for_https` is called, and gets the value of
`Gem.configuration.ssl_ca_cert` in the proess of downloading gems,
we don't need to add the value in the `Bundler::Fetcher::GemRemoteFetcher`.
The backtrace is below.
```
+ rdbg -c \
-e 'b Gem::Request.configure_connection_for_https' \
-e c \
-- /home/jaruga/git/ruby/rubygems/bin/bundle install -V
...
(rdbg) bt # backtrace command
=>#0 Gem::Request.configure_connection_for_https(connection=#<Gem::Net::HTTP localhost:18443 open=fal..., cert_files=["/home/jaruga/.local/ruby-4.1.0-debug-3e...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:71
ruby#1 Gem::Request::HTTPSPool#setup_connection(connection=#<Gem::Net::HTTP localhost:18443 open=fal...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request/https_pool.rb:7
ruby#2 Gem::Request::HTTPPool#make_connection at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request/http_pool.rb:43
ruby#3 Gem::Request::HTTPPool#checkout at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request/http_pool.rb:23
ruby#4 Gem::Request#connection_for(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:136
ruby#5 Gem::Request#perform_request(request=#<Gem::Net::HTTP::Get GET path="/gems/hel...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:194
ruby#6 Gem::Request#fetch at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:161
ruby#7 Gem::RemoteFetcher#request(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri..., request_class=Gem::Net::HTTP::Get, last_modified=nil) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:326
ruby#8 Bundler::Fetcher::GemRemoteFetcher#request(args=[#<Gem::Uri:0x00007f4e473adda0 @parsed_ur...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/fetcher/gem_remote_fetcher.rb:18
ruby#9 Gem::RemoteFetcher#fetch_http(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri..., last_modified=nil, head=false, depth=0) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:217
ruby#10 Gem::RemoteFetcher#fetch_path(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri..., mtime=nil, head=false) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:261
ruby#11 Gem::RemoteFetcher#cache_update_path(uri=#<Gem::URI::HTTPS https://localhost:18443..., path="/home/jaruga/var/git/report-bundler-bund..., update=true) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:302
ruby#12 block in Bundler::RubygemsIntegration#download_gem (2 levels) at ~/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:393
ruby#13 Bundler::SharedHelpers#filesystem_access(path="/home/jaruga/var/git/report-bundler-bund..., action=:write, block=#<Proc:0x00007f4e1d7dfad8 /home/jaruga/va...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/shared_helpers.rb:106
ruby#14 block in Bundler::RubygemsIntegration#download_gem at ~/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:392
ruby#15 Bundler::Retry#run(block=#<Proc:0x00007f4e1d7df9c0 /home/jaruga/va...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/retry.rb:48
ruby#16 Bundler::Retry#attempts(block=#<Proc:0x00007f4e1d7df880 /home/jaruga/va...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/retry.rb:38
ruby#17 Bundler::RubygemsIntegration#download_gem(spec=Gem::Specification.new do |s| s.name = "h..., uri=#<Gem::URI::HTTPS https://localhost:18443..., cache_dir="/home/jaruga/var/git/report-bundler-bund..., fetcher=#<Bundler::Fetcher::GemRemoteFetcher:0x00...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:384
ruby#18 block in Bundler::Source::Rubygems#download_gem at ~/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:531
ruby#19 Gem.time(msg="Downloaded hello in", width=0, display=true) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems.rb:1068
ruby#20 Bundler::Source::Rubygems#download_gem(spec=Gem::Specification.new do |s| s.name = "h..., download_cache_path="/home/jaruga/var/git/report-bundler-bund..., previous_spec=nil) at ~/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:530
...
```
According to the following logic, `@cert_files` is not nil,
as `Dir.glob(patterns` returns Array.
lib/rubygems/remote_fetcher.rb
```
def initialize(proxy = nil, dns = nil, headers = {})
...
@cert_files = Gem::Request.get_cert_files
...
end
```
lib/rubygems/request.rb
```
def self.get_cert_files
pattern = File.expand_path("./ssl_certs/*/*.pem", __dir__)
Dir.glob(pattern)
end
```
I might fit the Einsteinian definition of insane for this, given the previous engagement, but I deemed it rather wasteful to throw away the constructive aspects of the dialog on this feature, despite all the nonconstructive garbage. Since I had already endeavored at it once, no doubt its an easier matter for me to try again then for most others to come at it anew.
This rendition simplifies to whole matter to a simple Hash attribute in the gemspec itself. Resources can be assigned directly, e.g.
resources = { :type => uri, ... }or individually via the helper methodadd_resource(type, uri). For convenience of writing gemspecs, a few short terms for common resource types are translated to longer canonical names. This is simply for writing, not for reading. Validation of URIs is handled in thevalidatemethod.The hardest part of this was mapping the old
homepageattribute to the resource hash so as to maintain backward compatibility. This patch is close (and appears functional) but is not yet perfect in this regard. I lefthomepageas an attribute and rerouted the accessors toresources. I think this is probably the right way to do it and it seems to work fine. Butto_rubywill still add both thehomepage = ...andresources =entries (functional albeit redundant). For_dump, I putnilin the homepage slot and for_loadI skipped the homepage assignment (which I simply commented out at this point). Someone with a little more experience with this code can make the final decision on how best to handle this particular issue.With this patch I have taken the constructive parts of the critiques I received in my previous attempt and sought to address them while still providing some of convenience and greater functionality I originally had in mind to help improve the Gem::Specification.
As mentioned above, I do not consider this patch complete and ready to commit to master, but I think a few more tweaks should be enough to make it so (assuming of course that the overall approach is acceptable.)
[SIDE NOTE] I would like to make a personal request that Ryan Davis not critique this code (unless he wishes to apologize for past behavior). I don't find his comments very helpful and I believe he purposely attempts to insight me to anger (which he's pretty good at! ;-) I don't think it's an unreasonable request considering. Of course, feel free to ignore this push request altogether too, but I at least wanted to try make amends for my part in previous tensions and to produce something actually constructive out of it all.