RDoc Generation#16
Conversation
|
If this is never going to be used I will just delete my repo along with the branch. |
|
A user running For background, in master, documentation generation by RubyGems is a legacy feature. Instead, RubyGems depends on RDoc (or whichever other documentation tool) to use supply a callback to generate documentation. (RubyGems ships with a stub for users of old versions of RDoc that do not provide the necessary callbacks.) A future version of RDoc will be able to generate HTML documentation at runtime from the ri data. |
|
That's correct about the sudo, though it's become much less an issue in recent times due to the use of rvm and like. The only way to avoid it would be to generate the documentation to a location relative to user's home directory instead --which probably wouldn't be a very difficult adjustment. But in any case, I will close this pull request, as things seem to be moving in this direction anyway. |
When `bundle install` connects with a certification (CA) to a private RubyGems
HTTPS server emulated by WEBrick, the connection to the
https://localhost:18443/versions succeeded, but the connection to download gems
failed with the following error.
```
+ /home/jaruga/var/git/ruby/rubygems/bin/bundle config set --local ssl_ca_cert /home/jaruga/git/report-bundler-bundle-config-set-ssl_ca_cert/tmp/client/ssl/ca.crt
...
+ /home/jaruga/var/git/ruby/rubygems/bin/bundle install -V
Running `bundle install --verbose` with bundler 4.1.0.dev
Resolving dependencies because there's no lockfile
HTTP GET https://localhost:18443/versions
HTTP 206 Partial Content https://localhost:18443/versions
HTTP GET https://localhost:18443/versions
HTTP 200 OK https://localhost:18443/versions
Fetching gem metadata from https://localhost:18443/
Looking up gems ["hello"]
Resolving dependencies...
Using bundler 4.1.0.dev
1: bundler (4.1.0.dev) from /home/jaruga/var/git/ruby/rubygems/bundler/bundler.gemspec
Fetching hello 0.1.0
Retrying download gem from https://localhost:18443/ due to error (2/4): Gem::RemoteFetcher::FetchError SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)
Sleeping for 1.22 seconds before retry
Retrying download gem from https://localhost:18443/ due to error (3/4): Gem::RemoteFetcher::FetchError SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)
Sleeping for 2.26 seconds before retry
Retrying download gem from https://localhost:18443/ due to error (4/4): Gem::RemoteFetcher::FetchError SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)
Sleeping for 4.02 seconds before retry
Bundler::InstallError: Bundler::HTTPError: Could not download gem from https://localhost:18443/ due to underlying error <SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:18443 state=error: certificate verify failed (unable to get local issuer certificate) (https://localhost:18443/gems/hello-0.1.0.gem)>
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:406:in 'Bundler::RubygemsIntegration#download_gem'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:531:in 'block in Bundler::Source::Rubygems#download_gem'
/home/jaruga/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems.rb:1068:in 'Gem.time'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:530:in 'Bundler::Source::Rubygems#download_gem'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:459:in 'Bundler::Source::Rubygems#fetch_gem'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:443:in 'Bundler::Source::Rubygems#fetch_gem_if_possible'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:575:in 'Bundler::Source::Rubygems#rubygems_gem_installer'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:184:in 'Bundler::Source::Rubygems#download'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/installer/gem_installer.rb:29:in 'Bundler::GemInstaller#download'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/installer/parallel_installer.rb:148:in 'Bundler::ParallelInstaller#do_download'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/installer/parallel_installer.rb:132:in 'block in Bundler::ParallelInstaller#worker_pool'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:70:in 'Bundler::Worker#apply_func'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:65:in 'block in Bundler::Worker#process_queue'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:56:in 'Kernel#loop'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:56:in 'Bundler::Worker#process_queue'
/home/jaruga/var/git/ruby/rubygems/bundler/lib/bundler/worker.rb:98:in 'block (2 levels) in Bundler::Worker#create_threads'
...
```
`Bundler::Fetcher` has the `#connection` calling `#bundler_cert_store` storing
`Bundler.settings[:ssl_ca_cert]` in the following part. It is used in some parts.
bundler/lib/bundler/fetcher.rb
...
def bundler_cert_store
...
ssl_ca_cert = Bundler.settings[:ssl_ca_cert] ||
(Gem.configuration.ssl_ca_cert if
Gem.configuration.respond_to?(:ssl_ca_cert))
...
end
```
However in the case of downloading gems in Bundler,
Bundler calls `Bundler::Source::Rubygems#download_gem`
calling `Bundler::Fetcher#gem_remote_fetcher`
calling `Bundler::Fetcher::GemRemoteFetcher`
extending `Gem::RemoteFetcher` managing `@cert_files` for RubyGems.
Therefore, the `Bundler::Fetcher::GemRemoteFetcher` needs to update the `@cert_files`
by adding the value of the `Bundler.settings[:ssl_ca_cert]`.
As `Gem::Request.configure_connection_for_https` is called, and gets the value of
`Gem.configuration.ssl_ca_cert` in the proess of downloading gems,
we don't need to add the value in the `Bundler::Fetcher::GemRemoteFetcher`.
The backtrace is below.
```
+ rdbg -c \
-e 'b Gem::Request.configure_connection_for_https' \
-e c \
-- /home/jaruga/git/ruby/rubygems/bin/bundle install -V
...
(rdbg) bt # backtrace command
=>#0 Gem::Request.configure_connection_for_https(connection=#<Gem::Net::HTTP localhost:18443 open=fal..., cert_files=["/home/jaruga/.local/ruby-4.1.0-debug-3e...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:71
ruby#1 Gem::Request::HTTPSPool#setup_connection(connection=#<Gem::Net::HTTP localhost:18443 open=fal...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request/https_pool.rb:7
ruby#2 Gem::Request::HTTPPool#make_connection at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request/http_pool.rb:43
ruby#3 Gem::Request::HTTPPool#checkout at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request/http_pool.rb:23
ruby#4 Gem::Request#connection_for(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:136
ruby#5 Gem::Request#perform_request(request=#<Gem::Net::HTTP::Get GET path="/gems/hel...) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:194
ruby#6 Gem::Request#fetch at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/request.rb:161
ruby#7 Gem::RemoteFetcher#request(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri..., request_class=Gem::Net::HTTP::Get, last_modified=nil) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:326
ruby#8 Bundler::Fetcher::GemRemoteFetcher#request(args=[#<Gem::Uri:0x00007f4e473adda0 @parsed_ur...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/fetcher/gem_remote_fetcher.rb:18
ruby#9 Gem::RemoteFetcher#fetch_http(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri..., last_modified=nil, head=false, depth=0) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:217
ruby#10 Gem::RemoteFetcher#fetch_path(uri=#<Gem::Uri:0x00007f4e473adda0 @parsed_uri..., mtime=nil, head=false) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:261
ruby#11 Gem::RemoteFetcher#cache_update_path(uri=#<Gem::URI::HTTPS https://localhost:18443..., path="/home/jaruga/var/git/report-bundler-bund..., update=true) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems/remote_fetcher.rb:302
ruby#12 block in Bundler::RubygemsIntegration#download_gem (2 levels) at ~/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:393
ruby#13 Bundler::SharedHelpers#filesystem_access(path="/home/jaruga/var/git/report-bundler-bund..., action=:write, block=#<Proc:0x00007f4e1d7dfad8 /home/jaruga/va...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/shared_helpers.rb:106
ruby#14 block in Bundler::RubygemsIntegration#download_gem at ~/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:392
ruby#15 Bundler::Retry#run(block=#<Proc:0x00007f4e1d7df9c0 /home/jaruga/va...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/retry.rb:48
ruby#16 Bundler::Retry#attempts(block=#<Proc:0x00007f4e1d7df880 /home/jaruga/va...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/retry.rb:38
ruby#17 Bundler::RubygemsIntegration#download_gem(spec=Gem::Specification.new do |s| s.name = "h..., uri=#<Gem::URI::HTTPS https://localhost:18443..., cache_dir="/home/jaruga/var/git/report-bundler-bund..., fetcher=#<Bundler::Fetcher::GemRemoteFetcher:0x00...) at ~/var/git/ruby/rubygems/bundler/lib/bundler/rubygems_integration.rb:384
ruby#18 block in Bundler::Source::Rubygems#download_gem at ~/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:531
ruby#19 Gem.time(msg="Downloaded hello in", width=0, display=true) at ~/.local/ruby-4.1.0-debug-3ef48ef9c8-openssl-4.1.0-7194354488/lib/ruby/4.1.0+1/rubygems.rb:1068
ruby#20 Bundler::Source::Rubygems#download_gem(spec=Gem::Specification.new do |s| s.name = "h..., download_cache_path="/home/jaruga/var/git/report-bundler-bund..., previous_spec=nil) at ~/var/git/ruby/rubygems/bundler/lib/bundler/source/rubygems.rb:530
...
```
According to the following logic, `@cert_files` is not nil,
as `Dir.glob(patterns` returns Array.
lib/rubygems/remote_fetcher.rb
```
def initialize(proxy = nil, dns = nil, headers = {})
...
@cert_files = Gem::Request.get_cert_files
...
end
```
lib/rubygems/request.rb
```
def self.get_cert_files
pattern = File.expand_path("./ssl_certs/*/*.pem", __dir__)
Dir.glob(pattern)
end
```
Here is the on-demand RDoc generation patch, now in a separate branch.
To review, this patch makes it possible to run the Gem Server, click on the rdoc link of a gem that hasn't had rdocs previously generated, and the rdocs will be generated then and there.
The one caveat to this is that gems installed to system wide locations have permission restrictions that will prevent the rdocs from being generated unless the Gem Server is being run by a user with proper authority (e.g. sudo). When this arises a nice error message is displayed instead explaining the issue.