Skip to content

fix(parser): resolve Go and Ruby imports to files, not bare strings - #1027

Merged
tirth8205 merged 10 commits into
stagingfrom
fix/go-ruby-imports
Sep 18, 2026
Merged

tirth8205 merged 10 commits into
stagingfrom
fix/go-ruby-imports

Conversation

@tirth8205

@tirth8205 tirth8205 commented Sep 16, 2026 •

Copy link
Copy Markdown
Owner

What this fixes

_do_resolve_module had no go and no ruby arm. Both languages fell
through every branch to its bare return None, so every Go and Ruby
IMPORTS_FROM edge kept the raw import string as its target and matched no
node. importers_of and get_impact_radius therefore answered a confident
0 for packages with hundreds of real importers.

Verified against the diagnosis before touching anything, on a clean build of
each repository:

  • cli/cli: 8687 go import edges, 0 bound to anything in the repository.
    importers_of("pkg/iostreams/iostreams.go") returned 0 results for a
    package imported 435 times.
  • jekyll: 227 ruby import edges, 0 bound. 31 of them (14%) were not on a
    require line at all, and all 49 autoload statements produced no edge.
  • The grouped-block line defect: 8636 of cli/cli's 8687 import edges (99.4%)
    were stamped with the line of the import ( token.

Tests were written first and watched fail for exactly those reasons.

Go

The module path in the nearest go.mod is the anchor; there is no other way
to tell github.com/cli/cli/v2/pkg/cmdutil (this repository) from
github.com/spf13/cobra (a download). go.mod is read once per file, cached
on path/mtime/size like the existing composer.json reader.

An import equal to that module path, or prefixed by <module>/, names a
package directory. A Go import names a package, which is a directory of
files, so the edge target is that directory, tagged
extra.import_scope: package, and the read path expands the directory to its
member files. One import statement produces one edge, not one edge per member
file: fanning out makes the edge count grow with imports times package size,
and makes an incremental update disagree with a rebuild, because the target
set then depends on which files were in the package when the importing file
happened to be parsed. _test.go files are not importable from another
package, so they are never members.

  • Nested go.mod wins over its ancestor, and the ancestors stay in the chain,
    so a file inside cli/cli's nested module under .github/codeql still
    resolves imports of the outer module.
  • Local replace ... => ./dir (single and block form) is honoured.
  • vendor/<import path> is checked before an import is called external.
  • An import whose first path segment has no dot is the standard library and
    resolves to nothing. Resolving those by walking up from the caller looking
    for a matching directory name binds 149 of cli/cli's 3575 standard
    library imports to repository directories (context 133, embed 11,
    io 5), which is why the module prefix is the anchor and not a path search.
    That count is reproduced by script, not asserted.
  • Every specifier carries its own line.

Extraction parity checked spec by spec: all 8689 of cli/cli's (file, module)
pairs are accounted for after the change, 5194 still bare and 3495 bound,
0 dropped.

Ruby

The extractor now dispatches on the call's method name instead of testing
whether the node text contains the letters "require". That substring test is
what produced targets like Jekyll is a blog-aware, static site generator in Ruby (from a constant assignment) and local-require (from a Benchmark.ips
label), and it also returned "handled" for those whole call nodes, so every
call inside them was dropped from the call graph as well.

The form now survives to the resolver, because require and
require_relative take identical strings and resolve them by different
algorithms:

  • require_relative "x" -> against the requiring file's own directory.
  • require / load / autoload -> against load roots discovered from the
    repository: gemspec require_paths (defaulting to lib), lib, test,
    spec, the Rails app roots when the tree is Rails-shaped, and any
    directory a $LOAD_PATH.unshift in a repository-root script names.
  • autoload :Const, "path" resolves its second argument. This is how jekyll
    declares its entire internal module graph and it produced no edge at all.
  • require_all "dir" names a directory tree, so it emits one edge targeting
    that directory, tagged extra.import_scope: tree, expanded at read time to
    every .rb file below it. jekyll declares six subsystems this way
    (commands, converters, converters/markdown, drops, generators, tags).
  • A path built by an expression (require File.join(__dir__, "app", "helper"),
    require path, an interpolated string) emits no edge rather than a
    guessed one. The old regex took the first quoted string and emitted app.
  • A gem or stdlib feature matches no root and keeps its bare string.

Rails implicit autoloading (Zeitwerk) is explicitly out of scope, and the
code says so where a reader will look. There is no import statement to
resolve: the dependency is a bare constant reference (Jekyll::Drops::Drop)
that a runtime resolver maps to a file by naming convention. Supporting it
needs a constant-reference resolver plus autoload-root discovery from
config/application.rb, which is a different feature from module-to-file
import resolution.

Reused machinery, not a third resolver

  • The case-exact directory-listing probes are the existing ones, renamed from
    _python_file_exists / _python_dir_exists / _python_dir_entries /
    _python_repo_boundary to neutral names. Pure rename, no behaviour change,
    no call site outside the parser.
  • _path_is_within is the repository clamp, applied via one shared
    _parts_within helper: a go.mod replace => ../outside and a
    require_relative "../../../elsewhere" both name real files and neither
    becomes an edge target.
  • The edge shape is unchanged: IMPORTS_FROM, source file -> target, one row
    per import statement. Only the target spelling gained a directory form, and
    only behind an explicit extra.import_scope tag.

Measured, before -> after

Every number below was re-measured for this description. Baseline is
origin/staging at f0e4eb7; "after" is this branch with staging merged.
The two checkouts were built alternately from clean, on the same machine,
against the same clone of each repository: cli/cli 0cf1092 (943 .go),
spf13/cobra adbc881 (36), gin-gonic/gin 5c6a15f (99), jekyll 541d8b2
(162 .rb), rubocop 5bb0567 (1780).

Import edges that bind into the repository

Repo Lang Edges before Bound before Edges after Bound after
cli/cli go 8687 0 8689 3495
spf13/cobra go 190 0 190 12
gin-gonic/gin go 519 0 519 31
jekyll ruby 227 0 238 133
rubocop ruby 461 0 364 277

The edge count does not grow with resolution, because one import statement
still produces one edge. The unbound remainder is, top-first, fmt,
net/http, testing, strings, github.com/stretchr/testify/assert for Go
and pathname, fileutils, yaml, rainbow for Ruby: standard library and
gems, which is the correct answer. cobra and gin look low in absolute terms
because they barely import themselves; cobra has 12 in-repo import statements
and all 12 now bind. rubocop's ruby edge count falls because the false
positives are gone.

importers_of, hand-verified against the source

Repo File Before After Source
cli/cli pkg/iostreams/iostreams.go 0 424 435 import specs, of which 11 are inside the nested go.mod under .github/codeql and correctly bind to that module's own pkg/iostreams
cli/cli internal/ghrepo/repo.go 0 316 316 exact
cli/cli pkg/cmdutil/factory.go 0 422 422 exact
cobra command.go 0 11 11; a 12th file matches on grep but the hit is inside a Go source template string, not an import
cobra args.go 0 11 same 11
cobra doc/md_docs.go 0 1 1 exact
gin internal/bytesconv/bytesconv.go 0 8 8 exact
gin binding/binding.go 0 6 6 exact
gin render/render.go 0 3 3 exact
jekyll test/helper.rb 0 51 51 require "helper" lines
jekyll lib/jekyll.rb 0 11 11 require "jekyll" / require_relative "../lib/jekyll" lines, including extension-less scripts (exe/jekyll, script/console, benchmark/*)
jekyll lib/jekyll/filters.rb 0 1 lib/jekyll.rb
rubocop lib/rubocop.rb 0 5 5 exact
rubocop lib/rubocop/version.rb 0 2 2 exact
rubocop lib/rubocop/cop/base.rb 0 1 lib/rubocop.rb

Every edge behind these counts sits on a real
require/require_relative/autoload/load line, against 31 of 227 (14%) that did
not before.

get_impact_radius on one central file per repo

Repo File Impacted nodes before After
cli/cli pkg/iostreams/iostreams.go 0 1826 (100 nodes and 200 files returned, 11 files omitted, under the response ceiling)
cobra command.go 0 46 (11 files)
gin internal/bytesconv/bytesconv.go 0 40 (17 files)
jekyll lib/jekyll.rb 0 65 (62 files)
rubocop lib/rubocop.rb 0 8 (8 files)

Calls that now bind to an indexed node

This is the part worth having. Resolving an import gives the call resolver a
file to attribute a cross-file call to, so the gain is not confined to
IMPORTS_FROM.

Repo CALLS bound before CALLS bound after Of
cli/cli 5503 22564 67503
cobra 807 1047 4368
gin 1152 1222 9175
jekyll 934 1433 11306 -> 11649
rubocop 7405 7628 95886 -> 121103

Share of all edges naming nothing the build indexed

A directory target counts as named when the directory holds indexed files.

Repo Before After
cli/cli 70770 / 121396 (58.30%) 50216 / 121398 (41.37%)
cobra 3751 / 7911 (47.41%) 3499 / 7911 (44.23%)
gin 8569 / 18069 (47.42%) 8468 / 18069 (46.86%)
jekyll 10606 / 12969 (81.78%) 10328 / 13323 (77.52%)
rubocop 88942 / 107734 (82.56%) 113562 / 132891 (85.45%)

rubocop is the one row that gets worse, and it is worth reading rather than
hiding. Its ruby import edges went from 0 bound to 277 of 364; the overall
share rises because CALLS edges went 95886 -> 121103. Those 25217 calls
are the ones the old substring test was swallowing: rubocop's specs are full
of it 'registers an offense for EnforcedStyle: require_parentheses' do ... end, whose node text contains "require", so _extract_imports claimed the
whole block and the dispatcher never reached call extraction. More real edges,
a larger unresolved call target population, and a separate problem from the
one this PR fixes.

What it costs

Resolving imports that previously resolved to nothing costs build time and
disk. This branch is slower to build and produces a larger database than
staging on every repository measured. It is never faster.

Build time

Median of 5 alternating clean builds per side once the machine had settled,
with the observed range.

Repo Staging This branch
cli/cli 7.6s (7.0-7.8) 9.3s (8.6-9.6), +22%
cobra 0.8s 0.8s
gin 1.1s 1.2s
jekyll 1.1s 1.1s
rubocop 6.7s (5.7-8.4) 7.3s (6.5-7.8)

The three sub-2s repositories are noise-dominated and no difference should be
read into them. rubocop's two ranges overlap, so that row does not establish a
direction either. cli/cli is the smallest repository here where the two sets
of samples do not overlap at all.

Database

Identical on every repeat, so these are exact rather than median.

Repo Staging This branch
cli/cli 281.1 MB 325.9 MB (+16%)
cobra 19.2 MB 19.7 MB (+2.6%)
gin 42.6 MB 43.2 MB (+1.4%)
jekyll 30.9 MB 32.7 MB (+5.8%)
rubocop 285.2 MB 334.4 MB (+17%)

At scale: kubernetes/kubernetes

Measured isolated and alternating on one idle machine, two runs per side.

Staging This branch
Build 115.1s, 122.8s 148.7s, 146.5s (about 24% slower)
Database 4.41 GB 5.36 GB (21.5% larger)

An earlier revision of this description carried a kubernetes build row of
149.4s on staging against 129.0s here, which read as this branch being faster
than staging. That does not reproduce and it was wrong in the direction that
flattered the change. The table above replaces it.

Where the cost goes

Not the resolver. On cli/cli the extra work is downstream of parsing: the
go.mod reads and directory listings are cached, the edge count is unchanged
at 121398 against 121396, and 8689 import edges are still 8689. What grows is
what a bound import lets the rest of the pipeline do. 17061 more CALLS edges
now bind to an indexed node, and each resolution attempt that lands writes
target and candidate metadata that an unresolved edge never had: edges.extra
goes from 2.4 MB to 11.1 MB, and a bound target is an absolute file path
rather than a short module string, which grows the row and six indexes on it.

Going from 0 to 3495 bound imports and from 5503 to 22564 bound calls on a
943-file repository costs about 1.7s of build and 45 MB of database. That is
the trade being made, and it is visible here rather than buried.

The comparison that does hold against a pre-fix revision

The first revision of this branch emitted one edge per member file. Against
that revision, on kubernetes, the directory-scoped edge is a straight win:

Per-file fan-out Directory-scoped
Build 524.6s about 146s
IMPORTS_FROM edges 588,972 93,650
Edges naming an unindexed path 73,512 5
get_impact_radius on one file 189,163 tokens 23,669

Those four are the numbers this design change earned. They are not a
comparison with staging and are not offered as one.

Not regressing the other languages

Shared code was touched (the probe renames, _extract_imports,
_resolve_module_to_file), so the six other-language repositories from the
diagnosis were rebuilt on both sides and compared. Every graph is
identical
: edge totals, per-kind distribution, per-language import
resolution and database size all match exactly.

Repo Lang Import edges bound, before and after Total edges
google/gson java 390 / 2674 47564
square/okhttp kotlin 409 / 6781 119441
BurntSushi/ripgrep rust 182 / 799 29256
guzzle/guzzle php 369 / 819 43443
Newtonsoft.Json csharp 0 / 4991 76052
Alamofire swift 0 / 189 16632

One Python behaviour was nearly broken and caught by the existing guard: an
early self-edge filter dropped from . import runner inside
code_review_graph/eval/__init__.py, which legitimately resolves to the
package's own __init__.py. test_import_target_integrity.py failed, the
filter was removed, and the Python audit is back to 100%.

The same gap in other languages

C# and Swift are the other two zeros the diagnosis found, and both have the
same symptom (no branch in _do_resolve_module) but not the same shape,
so neither is fixed here rather than half-fixed:

  • C# (Newtonsoft.Json: 4991 edges, 0 bound). using X.Y; names a
    namespace, not a file, and C# imposes no filename convention. Resolving it
    needs a repository-wide namespace -> files index built before edges are
    written, which is a cross-file pre-pass, not a manifest read like go.mod
    or a gemspec. There is already a query-time compensation in tools/query.py
    that matches a file's declared csharp_namespaces against raw namespace
    edge targets, which is why importers_of on a C# file returns results even
    though no IMPORTS_FROM edge does; get_impact_radius, which reads the
    edge table, gets nothing.
  • Swift (Alamofire: 189 edges, 0 bound). Worse than Go and Ruby was:
    there is no _extract_import arm either, so the stored targets are raw
    statement text. Extracting the module name is cheap; resolving it is not,
    because a Swift import names a module whose files are listed by a
    Package.swift target declaration, a Swift DSL to parse rather than a data
    format to read. Extracting the name without resolving it would only make an
    unresolvable target look resolvable.

Two further gaps the diagnosis measured are real but are a different defect
(a branch that exists and is too narrow, not a missing branch), so they are
also left alone here:

  • Java and Kotlin in-repo resolution. Both walk up from the caller
    directory only, so gson/src/test/java/... never reaches
    gson/src/main/java/com/google/gson/Gson.java. The fix is sibling
    source-root discovery, which is cheap, but it is a separate change with its
    own before/after numbers.

Gates

Run on this branch with staging merged.

  • uv run --python 3.13 python -m pytest tests/ -q -> 4033 passed,
    772 skipped, 2 xfailed, 2 xpassed
  • uv run ruff check code_review_graph/ -> all checks passed
  • uv run --with mypy mypy code_review_graph/ --ignore-missing-imports --no-strict-optional -> no issues in 75 source files
  • bandit -r code_review_graph/ -c pyproject.toml -> 0 issues at every
    severity

Known limits

  • require File.expand_path("../lib/jekyll", __dir__) emits no edge.
    __dir__ is statically known, so this is resolvable in principle; the
    conservative "no guessed path" rule is applied instead. jekyll has 4 such
    lines (test/helper.rb, benchmark/schwartzian_transform.rb,
    script/stackprof, script/rubyprof). Nothing is lost against the old
    behaviour, which emitted an unresolvable bare string.
  • $LOAD_PATH.unshift is read from repository-root scripts, the gemspecs, the
    Rakefile, the Gemfile, spec/spec_helper.rb and test/test_helper.rb only.
    A push written elsewhere is not visible. The alternative is reading every
    file in the repository before resolving the first import.
  • Repository-internal wrappers around require
    (Jekyll::External.require_with_graceful_fail("kramdown"), 11 sites in
    jekyll) no longer emit an edge. Every one of them names a gem, so the edge
    they used to emit was an unresolvable bare string.
  • Go build constraints are not evaluated, so expanding a package target
    includes files excluded by //go:build on the current platform
    (pkg/iostreams/console_windows.go is a member of pkg/iostreams).
  • This branch is slower to build and larger on disk than staging, on every
    repository measured. See "What it costs".

🤖 Generated with Claude Code

Go and Ruby both reached `_resolve_module_to_file` and fell through every
arm of `_do_resolve_module` to its bare `return None`, so every import
edge kept the raw import string as its target. Measured on cli/cli: 8687
go IMPORTS_FROM edges, 0 matching a File node, 3493 of them naming a
directory that exists on disk; `importers_of("pkg/iostreams/iostreams.go")`
answered 0 for a package imported 435 times. On jekyll: 227 ruby edges, 0
matching a File node.

Go gets a go.mod anchored resolver. The module path declared in the
nearest go.mod is the only thing that tells this repository's packages
from its dependencies, so it is read (cached on path/mtime/size) and an
import under that prefix maps to the directory it names, then to the
non-test .go files in it -- a Go import names a package, which is a
directory of files, not one file. Nested modules win over their
ancestors while the ancestors stay in the chain, local `replace` targets
and `vendor/` are honoured, and an import whose first segment has no dot
is the standard library and resolves to nothing. Resolving those by
walking up from the caller looking for a matching directory name would
bind 149 of cli/cli's 3575 stdlib imports to repository directories
(context 133, embed 11, io 5), which is why the module prefix is the
anchor and not a path search.

Ruby gets load-path resolution. The extractor now dispatches on the
call's method name rather than testing whether the node text contains
"require": that substring test made 34 of jekyll's 227 edges (15%) false
positives, with targets like `Missing --ssl_cert or --ssl_key. Both are
required.` from a `raise`, and it swallowed those call subtrees whole so
their calls were dropped too. It never saw `autoload`, which is how
jekyll declares its entire internal module graph (49 statements). The
form now survives to the resolver, because `require` and
`require_relative` take identical strings and resolve them differently:
`require_relative` against the requiring file, `require`/`load`/
`autoload`/`require_all` against load roots discovered from the
repository -- gemspec `require_paths`, lib/test/spec, the Rails `app`
roots, and `$LOAD_PATH.unshift` in a repository-root script. A path built
by an expression emits no edge rather than a guessed one, and a gem
resolves to nothing.

Both reuse the existing machinery: the case-exact directory-listing
probes (renamed from `_python_*` to neutral names, no behaviour change),
`_path_is_within` as the repository clamp, and the same cache key shape
and eviction cap as `_resolve_module_to_file`. Every Go import specifier
also carries its own line now; 8648 of cli/cli's 8690 import edges were
stamped with the line of the `import (` token.

Measured, before -> after:

  cli/cli     go edges naming a file   0/8687 -> 17527/22721 (77.1%)
              importers_of(iostreams)  0 -> 424
              impact radius            0 nodes -> 500 (1764 total), 212 files
  cobra       go edges naming a file   0/190 -> 159/337 (47.2%)
  gin         go edges naming a file   0/519 -> 230/718 (32.0%)
  jekyll      ruby edges naming a file 0/227 -> 159/264 (60.2%)
              importers_of(test/helper.rb) 0 -> 51
  rubocop     ruby edges naming a file 0/458 -> 274/361 (75.9%)

Graphs for gson (java), okhttp (kotlin), ripgrep (rust), guzzle (php),
Newtonsoft.Json (csharp) and Alamofire (swift) are byte-identical before
and after.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown

code-review-graph review

Overall risk: 0.75 (HIGH) — 163 changed function(s)/class(es), 47 affected flow(s), 68 test gap(s)

Risk-scored changes

Risk Level Symbol Location Tested
0.75 high code_review_graph/tools/query.py::query_graph code_review_graph/tools/query.py:430 yes
0.65 medium code_review_graph/parser.py::_ImportSpec code_review_graph/parser.py:685 no
0.55 medium code_review_graph/graph.py::_impact_candidate_sql code_review_graph/graph.py:423 yes
0.55 medium code_review_graph/parser.py::CodeParser.add code_review_graph/parser.py:18031 yes
0.54 medium code_review_graph/main.py::get_impact_radius_tool code_review_graph/main.py:222 no
0.50 medium code_review_graph/tools/query.py::get_impact_radius code_review_graph/tools/query.py:204 yes
0.50 medium tests/test_import_scope.py::TestTraversalQueryPlan._plan tests/test_import_scope.py:383 yes
0.40 medium code_review_graph/graph.py::_edge_import_scope code_review_graph/graph.py:358 no
0.40 medium code_review_graph/graph.py::_parent_dir code_review_graph/graph.py:370 no
0.40 medium code_review_graph/parser.py::CodeParser._dir_entry_names code_review_graph/parser.py:14829 no

Affected execution flows

  • run — criticality 0.64, 28 node(s) across 3 file(s)
  • visit_FunctionDef — criticality 0.41, 55 node(s) across 1 file(s)
  • visit_AsyncFunctionDef — criticality 0.41, 55 node(s) across 1 file(s)
  • visit_ClassDef — criticality 0.41, 54 node(s) across 1 file(s)
  • query_graph_tool — criticality 0.36, 2 node(s) across 1 file(s)
  • ...and 42 more affected flow(s)

Test gaps

  • code_review_graph/graph.py::_edge_import_scope (code_review_graph/graph.py:358)
  • code_review_graph/graph.py::_parent_dir (code_review_graph/graph.py:370)
  • code_review_graph/graph.py::GraphStore._directory_member_files (code_review_graph/graph.py:1782)
  • code_review_graph/graph.py::GraphStore._resolve_bare_endpoints (code_review_graph/graph.py:1802)
  • code_review_graph/graph.py::GraphStore._impact_seed_qns (code_review_graph/graph.py:2267)
  • ...and 63 more without direct tests

Token savings: this graph-backed report used ~257,881 fewer tokens (~84%) than reading every changed file in full (estimated, chars/4 approximation).


Powered by code-review-graph — local-first analysis; no code leaves the CI runner.

tirth8205 and others added 5 commits September 17, 2026 00:11
A Go import names a PACKAGE, which is a directory of files, so resolving
one by emitting an edge per member file makes the edge count grow with
imports times package size. On kubernetes/kubernetes that was 588,972
IMPORTS_FROM edges against 93,624 import statements, a 6.5GB database,
and 524s of build time against staging's 149s. 73,507 of those edges
named a file under vendor/, which the default ignore patterns exclude, so
they were confident-looking paths that matched no node. And an
incremental update disagreed with a rebuild, because an edge's target set
depended on which files were in the package at the moment the importing
file happened to be parsed.

So the parser emits ONE edge naming the directory, tagged
extra.import_scope, and the read path expands a directory to its members:
importers_of matches edges targeting a file's own package, imports_of
reports import_target_kind, the impact traversal follows a package target
at every hop without spending one, and bare-call resolution expands the
members once per post-processing pass instead of storing them. Ruby's
require_all is the same shape with tree scope, because it loads every
file below the directory rather than the directory's own.

No resolver may name a path the build does not index, so an import into
vendor/ keeps its bare module string, which is visibly external.

get_impact_radius bounds every list by a fixed ceiling -- the same 100
nodes / 150 edges / 200 files get_review_context applies to the same
radius -- and reports what it dropped. edges and changed_nodes had no
ceiling and max_results was not exposed on the MCP tool, so one changed
file of cli/cli cost 138k tokens and one of kubernetes 189k, against a
documented 12k budget.

Measured on kubernetes/kubernetes, against the same branch before this
change: build 524.6s -> 144.4s (staging: 149.4s), database 6.50GB ->
5.07GB, IMPORTS_FROM 588,972 -> 93,650, import edges naming a path with
no node 73,512 -> 5, importers_of for apimachinery/pkg/runtime unchanged
at 2,682, get_impact_radius 189,163 -> 23,677 tokens and 6.5s -> 1.5s
warm.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The directory branch of one relaxation hop must be an index seek on
(target_qualified, kind). Left to itself SQLite drove it from the covering
index on kind alone and rescanned every IMPORTS_FROM row once per frontier
directory: 18 seconds of a 19-second kubernetes traversal, against 1.5
with CROSS JOIN and INDEXED BY. The same shape of mistake filled
_impact_frontier_dirs by scanning every File node in the graph instead of
the frontier.

Neither is visible in a token budget or on a small fixture, so the SQL
moves to module level where a test can EXPLAIN the text the traversal
actually runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tirth8205

Copy link
Copy Markdown
Owner Author

Correction to the scale table before merge.

The build-time row claimed kubernetes went from 149.4s on staging to 129.0s here, i.e. faster than staging. That does not reproduce. Measured isolated and alternating on one idle machine:

staging this branch
kubernetes build 115.1s, 122.8s 148.7s, 146.5s
database 4.41 GB 5.36 GB

So the branch is about 24% slower to build and 21.5% larger than staging, never faster. That is the real cost of resolving imports that previously resolved to nothing, and it is worth paying, but the table said the opposite.

What does hold, and is the point of the change: against the pre-fix version of this branch, kubernetes build time goes 524.6s to ~146s, IMPORTS_FROM edges 588,972 to 93,650 (one per import statement), edges naming an unindexed path 73,512 to 5, and get_impact_radius 189,163 tokens to 23,669.

The PR body and CHANGELOG entry need this row corrected before merge.

tirth8205 and others added 2 commits September 18, 2026 11:50
…this branch

An independent re-measurement found the kubernetes build-time row wrong in
the direction that favoured this change. It read 149.4s on staging against
129.0s here, which says this branch is faster than staging. Measured isolated
and alternating on one idle machine it is not: staging 115.1s and 122.8s
against 148.7s and 146.5s here, and 4.41 GB of database against 5.36 GB. The
branch is about 24% slower to build and 21.5% larger, never faster.

The CHANGELOG now says that plainly, alongside what the cost buys: on cli/cli
the CALLS edges bound to an indexed node go from 5503 to 22564 of the same
67503, because a bound import gives the call resolver a file to attribute a
cross-file call to.

The comparisons that do hold are against the pre-fix revision of this branch,
which emitted one edge per member file: 524.6s of build against about 146s,
588,972 IMPORTS_FROM edges against 93,650, 73,512 edges naming an unindexed
path against 5, and a 189,163-token get_impact_radius against 23,669. Those
are kept and labelled as what they are.

Also re-measured and corrected, on clean alternating builds of cli/cli,
cobra, gin, jekyll and rubocop against origin/staging:

- the grouped-block line defect is 8636 of cli/cli's 8687 import edges, not
  8648 of 8690
- jekyll's non-require import edges are 31 of 227 (14%), not 34 (15%)
- internal/ is 1317 of the 3495 cli/cli go import edges that bind in-repo,
  not "3493 of 8687"

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tirth8205

Copy link
Copy Markdown
Owner Author

Corrected in 2ccc937, and staging is merged in (5209f57, no conflicts).

The build-time row is gone and replaced with your measurement: kubernetes staging 115.1s and 122.8s against 148.7s and 146.5s here, database 4.41 GB against 5.36 GB, about 24% slower and 21.5% larger, never faster. The description says so in its own section and the CHANGELOG carries the same sentence. The four numbers that hold against the pre-fix revision (524.6s to about 146s, 588,972 to 93,650 IMPORTS_FROM edges, 73,512 to 5 unindexed-path edges, 189,163 to 23,669 tokens) are kept in a separate table labelled as a comparison with that revision and not with staging.

Everything else in the table was re-measured from clean alternating builds against origin/staging on cli/cli, cobra, gin, jekyll and rubocop, because the per-file fan-out had been replaced by directory-scoped edges after those numbers were taken. What changed:

  • The import-edge table was reporting fan-out counts (cli/cli 22721 edges, 17527 resolved). One import statement is one edge now: 8689 edges, 3495 bound, against 8687 and 0 on staging.
  • The paragraph explaining the slowdown blamed 14034 additional IMPORTS_FROM edges. That is not what happens; the edge count is 121398 against 121396. The real cause is that a bound import lets the call resolver attribute cross-file calls, so CALLS edges bound to an indexed node go 5503 to 22564 of 67503, and the resolution metadata and longer absolute-path targets are what grow the database. That is now a table of its own.
  • Diagnosis numbers: the grouped-block line defect is 8636 of 8687 (99.4%), not 8648 of 8690; jekyll's non-require import edges are 31 of 227 (14%), not 34 (15%).
  • Build time per repo is now median of 5 alternating clean builds with the observed range. cli/cli 7.6s to 9.3s (+22%) is the only row where the two sample sets do not overlap; cobra, gin and jekyll are sub-2s and noise-dominated, and rubocop's ranges overlap, so no direction is claimed for any of them.
  • Removed: the 64-file fan-out cap limit (there is no fan-out), the 28% larger database figure and the 2,648-row incremental disagreement figure (neither reproducible).

Reproduced unchanged and left alone: all 15 importers_of counts, the impact-radius counts for cobra, gin, jekyll and rubocop, the 149 standard-library collisions (context 133, embed 11, io 5), jekyll's 49 autoload statements, and the six other-language repositories, which are still byte-identical on both sides including database size.

Gates on the merged branch: 4033 passed, 772 skipped, 2 xfailed, 2 xpassed; ruff clean; mypy clean on 75 files; bandit 0 issues.

tirth8205 and others added 2 commits September 18, 2026 12:28
graph.py was the only overlap. Staging added a read-transaction helper
(contextlib.contextmanager, _active_read_txn, _read_transaction,
_get_stats_locked) and migration-safety retries; this branch added the
package-scoped import resolver and the pinned impact traversal, which
needs collections.abc.Iterable. The import block now carries both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
One conflict, in GraphStore.__init__ of code_review_graph/graph.py. Staging
wrapped the schema init and migration run in a guard that turns an unreadable
or wrongly shaped database into CorruptGraphDatabaseError; this branch added
the crg_parent_dir SQLite function that the directory-scoped import traversal
joins on. Both are kept: the guard keeps its original scope, and the function
is registered after it, because a failure to register a function is not a
corrupt file and must not reach the code path that deletes a database.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tirth8205
tirth8205 merged commit 9144206 into staging Sep 18, 2026
22 checks passed
tirth8205 added a commit that referenced this pull request Sep 18, 2026
One conflict, in GraphStore.__init__. Staging (via #1027) registers the
SQLite function crg_parent_dir after the open/migrate guard; this branch
ends that guard with a described GraphStoreError fallback. Kept both: the
fallback stays the last arm of the except, the registration stays after
it, so a failure registering the helper is still not mistaken for a
corrupt database.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tirth8205
tirth8205 deleted the fix/go-ruby-imports branch September 18, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant