Skip to content

test262 built-ins/RegExp — 89 (self-contained worklist) #5897

Closed

Description

@proggeramlug

Self-contained worklist — 89 failing. Snapshot 2026-07-03 full sweep (perry 0.5.1218, node v26.3.0). Everything you need is in this issue; no history to read, no re-measure to start.

Rust regex-crate impedance: \p{Script_Extensions=...} property escapes, the v-flag set notation (likely unimplemented), and exec/lastIndex internals. Crate: perry-runtime (regex).

Setup (any box)

git clone https://github.com/PerryTS/perry && cd perry
cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
git clone --depth 1 https://github.com/tc39/test262 vendor/test262   # checkout test-compat/test262/pinned-sha.txt
# node on PATH (differential oracle). See test-compat/test262/README.md.

Measure/verify: scripts/test262_subset.py --root vendor/test262 --dir built-ins/RegExp --jobs 8 (the .failures.txt sidecar lists failures; diff before/after for zero-regression).

Failure reasons

n reason
2 Uncaught exception: The value of __executed[index] is e
1 TypeError: Cannot read properties of null (reading 'len
1 Uncaught exception: __re.toString() must return "[objec
1 Uncaught exception: The value of RegExp.indicator is ex
1 Uncaught exception: Non WhiteSpace character, charCode:
1 Uncaught exception: Actual [a] and expected [a, undefin
1 Uncaught exception: The regex is expected to match the
1 Uncaught exception: '\p{Alphabetic}' should match U+000
1 Uncaught exception: '\p{Case_Ignorable}' should match U
1 Uncaught exception: '\p{Cased}' should match U+00A7CE (
1 Uncaught exception: '\p{Changes_When_Casefolded}' shoul
1 Uncaught exception: '\p{Changes_When_Casemapped}' shoul

All 89 failing cases (path — reason)

built-ins/RegExp/S15.10.2.5_A1_T4.js    Uncaught exception: The value of __executed[index] is expected to equal the valu
built-ins/RegExp/S15.10.2.8_A2_T1.js    TypeError: Cannot read properties of null (reading 'length')
built-ins/RegExp/S15.10.4.1_A6_T1.js    Uncaught exception: __re.toString() must return "[object "+"RegExp"+"]" Expected
built-ins/RegExp/S15.10.5_A2_T2.js    Uncaught exception: The value of RegExp.indicator is expected to be 1 Expected S
built-ins/RegExp/character-class-escape-non-whitespace.js    Uncaught exception: Non WhiteSpace character, charCode: 55296 Expected SameValue
built-ins/RegExp/lookahead-quantifier-match-groups.js    Uncaught exception: Actual [a] and expected [a, undefined] should have the same 
built-ins/RegExp/nullable-quantifier.js    Uncaught exception: The regex is expected to match the whole string Expected Sam
built-ins/RegExp/property-escapes/generated/Alphabetic.js    Uncaught exception: `\p{Alphabetic}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/Case_Ignorable.js    Uncaught exception: `\p{Case_Ignorable}` should match U+010EC5 (`𐻅`)
built-ins/RegExp/property-escapes/generated/Cased.js    Uncaught exception: `\p{Cased}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/Changes_When_Casefolded.js    Uncaught exception: `\p{Changes_When_Casefolded}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/Changes_When_Casemapped.js    Uncaught exception: `\p{Changes_When_Casemapped}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/Changes_When_Lowercased.js    Uncaught exception: `\p{Changes_When_Lowercased}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/Changes_When_NFKC_Casefolded.js    Uncaught exception: `\p{Changes_When_NFKC_Casefolded}` should match U+0000A0 (` 
built-ins/RegExp/property-escapes/generated/Changes_When_Titlecased.js    Uncaught exception: `\p{Changes_When_Titlecased}` should match U+00A7CF (`꟏`)
built-ins/RegExp/property-escapes/generated/Changes_When_Uppercased.js    Uncaught exception: `\p{Changes_When_Uppercased}` should match U+00A7CF (`꟏`)
built-ins/RegExp/property-escapes/generated/Diacritic.js    Uncaught exception: `\p{Diacritic}` should match U+0005C7 (`ׇ`)
built-ins/RegExp/property-escapes/generated/Emoji.js    Uncaught exception: `\p{Emoji}` should match U+01FAC8 (`🫈`)
built-ins/RegExp/property-escapes/generated/Emoji_Presentation.js    Uncaught exception: `\p{Emoji_Presentation}` should match U+01FAC8 (`🫈`)
built-ins/RegExp/property-escapes/generated/Extender.js    Uncaught exception: `\p{Extender}` should match U+011DD9 (`𑷙`)
built-ins/RegExp/property-escapes/generated/General_Category_-_Cased_Letter.js    Uncaught exception: `\p{General_Category=Cased_Letter}` should match U+00A7CE (`
built-ins/RegExp/property-escapes/generated/General_Category_-_Currency_Symbol.js    Uncaught exception: `\p{General_Category=Currency_Symbol}` should match U+0020C1
built-ins/RegExp/property-escapes/generated/General_Category_-_Decimal_Number.js    Uncaught exception: `\p{General_Category=Decimal_Number}` should match U+011DE0 
built-ins/RegExp/property-escapes/generated/General_Category_-_Letter.js    Uncaught exception: `\p{General_Category=Letter}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/General_Category_-_Letter_Number.js    Uncaught exception: `\p{General_Category=Letter_Number}` should match U+016FF4 (
built-ins/RegExp/property-escapes/generated/General_Category_-_Lowercase_Letter.js    Uncaught exception: `\p{General_Category=Lowercase_Letter}` should match U+00A7C
built-ins/RegExp/property-escapes/generated/General_Category_-_Mark.js    Uncaught exception: `\p{General_Category=Mark}` should match U+01E6E3 (`𞛣`)
built-ins/RegExp/property-escapes/generated/General_Category_-_Math_Symbol.js    Uncaught exception: `\p{General_Category=Math_Symbol}` should match U+01CEF0 (`𜻰
built-ins/RegExp/property-escapes/generated/General_Category_-_Modifier_Letter.js    Uncaught exception: `\p{General_Category=Modifier_Letter}` should match U+010EC5
built-ins/RegExp/property-escapes/generated/General_Category_-_Nonspacing_Mark.js    Uncaught exception: `\p{General_Category=Nonspacing_Mark}` should match U+011B60
built-ins/RegExp/property-escapes/generated/General_Category_-_Number.js    Uncaught exception: `\p{General_Category=Number}` should match U+011DE0 (`𑷠`)
built-ins/RegExp/property-escapes/generated/General_Category_-_Other.js    Uncaught exception: `\P{General_Category=Other}` should match U+01FAC8 (`🫈`)
built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Letter.js    Uncaught exception: `\p{General_Category=Other_Letter}` should match U+01E6FE (`
built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Punctuation.js    Uncaught exception: `\p{General_Category=Other_Punctuation}` should match U+010E
built-ins/RegExp/property-escapes/generated/General_Category_-_Other_Symbol.js    Uncaught exception: `\p{General_Category=Other_Symbol}` should match U+01FAC8 (`
built-ins/RegExp/property-escapes/generated/General_Category_-_Punctuation.js    Uncaught exception: `\p{General_Category=Punctuation}` should match U+010ED0 (`𐻐
built-ins/RegExp/property-escapes/generated/General_Category_-_Spacing_Mark.js    Uncaught exception: `\p{General_Category=Spacing_Mark}` should match U+011B61 (`
built-ins/RegExp/property-escapes/generated/General_Category_-_Symbol.js    Uncaught exception: `\p{General_Category=Symbol}` should match U+01FAC8 (`🫈`)
built-ins/RegExp/property-escapes/generated/General_Category_-_Uppercase_Letter.js    Uncaught exception: `\p{General_Category=Uppercase_Letter}` should match U+00A7C
built-ins/RegExp/property-escapes/generated/Grapheme_Base.js    Uncaught exception: `\p{Grapheme_Base}` should match U+011B61 (`𑭡`)
built-ins/RegExp/property-escapes/generated/Grapheme_Extend.js    Uncaught exception: `\p{Grapheme_Extend}` should match U+011B60 (`𑭠`)
built-ins/RegExp/property-escapes/generated/ID_Continue.js    Uncaught exception: `\p{ID_Continue}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/ID_Start.js    Uncaught exception: `\p{ID_Start}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/Ideographic.js    Uncaught exception: `\p{Ideographic}` should match U+016FF2 (`𖿲`)
built-ins/RegExp/property-escapes/generated/Lowercase.js    Uncaught exception: `\p{Lowercase}` should match U+00A7CF (`꟏`)
built-ins/RegExp/property-escapes/generated/Math.js    Uncaught exception: `\p{Math}` should match U+01CEF0 (`𜻰`)
built-ins/RegExp/property-escapes/generated/Script_-_Arabic.js    Uncaught exception: `\p{Script=Arabic}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/Script_-_Beria_Erfe.js    Uncaught exception: `\p{Script=Beria_Erfe}` should match U+016EA0 (`𖺠`)
built-ins/RegExp/property-escapes/generated/Script_-_Common.js    Uncaught exception: `\p{Script=Common}` should match U+01FAC8 (`🫈`)
built-ins/RegExp/property-escapes/generated/Script_-_Han.js    Uncaught exception: `\p{Script=Han}` should match U+016FF2 (`𖿲`)
built-ins/RegExp/property-escapes/generated/Script_-_Inherited.js    Uncaught exception: `\p{Script=Inherited}` should match U+001ACF (`᫏`)
built-ins/RegExp/property-escapes/generated/Script_-_Kannada.js    Uncaught exception: `\p{Script=Kannada}` should match U+000CDC (`೜`)
built-ins/RegExp/property-escapes/generated/Script_-_Latin.js    Uncaught exception: `\p{Script=Latin}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/Script_-_Sharada.js    Uncaught exception: `\p{Script=Sharada}` should match U+011B60 (`𑭠`)
built-ins/RegExp/property-escapes/generated/Script_-_Sidetic.js    Uncaught exception: `\p{Script=Sidetic}` should match U+010940 (`𐥀`)
built-ins/RegExp/property-escapes/generated/Script_-_Tai_Yo.js    Uncaught exception: `\p{Script=Tai_Yo}` should match U+01E6C0 (`𞛀`)
built-ins/RegExp/property-escapes/generated/Script_-_Tangut.js    Uncaught exception: `\p{Script=Tangut}` should match U+0187F8 (`𘟸`)
built-ins/RegExp/property-escapes/generated/Script_-_Telugu.js    Uncaught exception: `\p{Script=Telugu}` should match U+000C5C (`౜`)
built-ins/RegExp/property-escapes/generated/Script_-_Tolong_Siki.js    Uncaught exception: `\p{Script=Tolong_Siki}` should match U+011DB0 (`𑶰`)
built-ins/RegExp/property-escapes/generated/Script_-_Unknown.js    SyntaxError: Invalid regular expression: /^\p{sc=Unknown}+$/: invalid pattern
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Arabic.js    Uncaught exception: `\p{Script_Extensions=Arabic}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Beria_Erfe.js    Uncaught exception: `\p{Script_Extensions=Beria_Erfe}` should match U+016EA0 (`𖺠
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Common.js    Uncaught exception: `\p{Script_Extensions=Common}` should match U+01FAC8 (`🫈`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Han.js    Uncaught exception: `\p{Script_Extensions=Han}` should match U+016FF2 (`𖿲`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Inherited.js    Uncaught exception: `\p{Script_Extensions=Inherited}` should match U+000320 (`̠`
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Kannada.js    Uncaught exception: `\p{Script_Extensions=Kannada}` should match U+000CDC (`೜`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Latin.js    Uncaught exception: `\p{Script_Extensions=Latin}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Nandinagari.js    Uncaught exception: `\p{Script_Extensions=Nandinagari}` should match U+000951 (`
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Newa.js    Uncaught exception: `\p{Script_Extensions=Newa}` should match U+001CD5 (`᳕`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Sharada.js    Uncaught exception: `\p{Script_Extensions=Sharada}` should match U+001CEA (`ᳪ`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Sidetic.js    Uncaught exception: `\p{Script_Extensions=Sidetic}` should match U+010940 (`𐥀`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Syriac.js    Uncaught exception: `\p{Script_Extensions=Syriac}` should match U+000331 (`̱`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tai_Yo.js    Uncaught exception: `\p{Script_Extensions=Tai_Yo}` should match U+01E6C0 (`𞛀`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tangut.js    Uncaught exception: `\p{Script_Extensions=Tangut}` should match U+0187F8 (`𘟸`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Telugu.js    Uncaught exception: `\p{Script_Extensions=Telugu}` should match U+001CD8 (`᳘`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tifinagh.js    Uncaught exception: `\p{Script_Extensions=Tifinagh}` should match U+000323 (`̣`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tirhuta.js    Uncaught exception: `\p{Script_Extensions=Tirhuta}` should match U+001CD5 (`᳕`)
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Tolong_Siki.js    Uncaught exception: `\p{Script_Extensions=Tolong_Siki}` should match U+011DB0 (`
built-ins/RegExp/property-escapes/generated/Script_Extensions_-_Unknown.js    SyntaxError: Invalid regular expression: /^\p{scx=Unknown}+$/: invalid pattern
built-ins/RegExp/property-escapes/generated/Unified_Ideograph.js    Uncaught exception: `\p{Unified_Ideograph}` should match U+02B73A (`𫜺`)
built-ins/RegExp/property-escapes/generated/Uppercase.js    Uncaught exception: `\p{Uppercase}` should match U+00A7CE (`꟎`)
built-ins/RegExp/property-escapes/generated/XID_Continue.js    Uncaught exception: `\p{XID_Continue}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/generated/XID_Start.js    Uncaught exception: `\p{XID_Start}` should match U+00088F (`࢏`)
built-ins/RegExp/property-escapes/special-property-value-Script_Extensions-Unknown.js    SyntaxError: Invalid regular expression: /\p{scx=Unknown}/: invalid pattern
built-ins/RegExp/prototype/exec/S15.10.6.2_A1_T6.js    Uncaught exception: The value of __executed[index] is expected to equal the valu
built-ins/RegExp/prototype/exec/u-lastindex-value.js    Uncaught exception: Expected SameValue(«1», «2») to be true
built-ins/RegExp/prototype/global/S15.10.7.2_A8.js    Uncaught exception: __re.propertyIsEnumerable('global') must return false Expect
built-ins/RegExp/prototype/ignoreCase/S15.10.7.3_A8.js    Uncaught exception: __re.propertyIsEnumerable('ignoreCase') must return false Ex
built-ins/RegExp/prototype/multiline/S15.10.7.4_A8.js    Uncaught exception: __re.propertyIsEnumerable('multiline') must return false Exp

Workflow

Pick a coherent single-root subcluster, localize (cargo run --release -- f.ts --trace hir --focus <fn> / --trace llvm), fix, verify listed cases pass AND re-run the slice for zero regressions. Code-only PR (no Cargo.toml/Cargo.lock version bump, no CLAUDE.md Current-Version, no CHANGELOG — maintainer folds metadata). cargo fmt --all -- --check + bash scripts/check_file_size.sh must pass (files < 2000 lines). Reference this issue. Never reference internal infrastructure hostnames.

Activity

  1. added
    parityCompatibility gap with Node.js, ECMAScript, or the supported ecosystem
    on Jul 3, 2026
  2. proggeramlug commented on Jul 4, 2026

    @proggeramlug
    ContributorAuthor

    Triage notes after digging into the non-property-escape subclusters:

    Property-escapes (~70 of the 89) are a data/dependency limit, not a code bug. The generated property-escapes/generated/*.js cases fail on codepoints that were assigned in newer Unicode versions (e.g. U+00088F, U+01FAC8, the new Beria_Erfe/Tolong_Siki/Sidetic/Tai_Yo scripts) than the Unicode tables bundled in the pinned regex/regex-syntax (regex = "1"). \p{…} compiles against regex-syntax's embedded tables, so these only move when that dependency is bumped to a build whose Unicode DB matches the test262 pin — and test262 tracks the latest Unicode, so some lag is inherent. The Script=Unknown / scx=Unknown "invalid pattern" cases are the same root (the crate has no Unknown script alias). Recommend tracking these as the existing categorical-gap item rather than as individually-fixable bugs.

    propertyIsEnumerable('global'/'ignoreCase'/'multiline') (3 cases) is a real bug but systemic. RegExp.prototype (and every other builtin prototype) installs the Object.prototype methods as own no-op stubs via install_noop_proto_methods(proto, OBJECT_PROTO_METHODS) (object/global_this/proto_methods.rs). That makes RegExp.prototype.hasOwnProperty('propertyIsEnumerable') return true (should be false) and the stub returns the receiver ([object Object]) instead of a boolean. Fixing it means making those methods genuinely inherited (non-own, real impls) across all builtin prototypes — broad and regression-prone, out of scope for a focused RegExp PR; flagging for a dedicated builtin-prototype-inheritance change.

    The remaining handful (lastIndex/toString/S15.10.* exec cases) I'll assess separately. Two sibling test262 PRs from this batch already landed for #5905 (Proxy) and #5900 (Temporal).

  3. proggeramlug commented on Jul 13, 2026

    @proggeramlug
    ContributorAuthor

    Re-measured — the count and the earlier triage were both wrong

    Fresh sweep against the real corpus (test262 pinned 4249661, node v26.3.0, scripts/test262_subset.py --dir built-ins/RegExp):

    issue title measured before after #6358
    failures 89 11 6
    parity — 99.1% 99.5%

    Correction 1 — the property-escapes cluster IS fixed (the earlier triage comment is wrong)

    The existing triage comment on this issue concludes that property-escapes/generated/* (~71 cases) is an unfixable regex-syntax dependency/data limit and should be tracked as a categorical gap. That is no longer true and should be disregarded. They were fixed by #6097 and #6068, which bundle the Unicode 17.0 data directly rather than relying on the crate's tables. All of them pass.

    The 3 propertyIsEnumerable cases that comment lists are also already gone (fixed by db39c9b0d).

    Correction 2 — 2 of the 11 were never real failures

    property-escapes/generated/Alphabetic.js and Assigned.js were reported as compile-fail. They are the two largest generated files and were tripping the runner's 20s per-test timeout under --jobs 8, not failing. Standalone they compile in ~6s and run clean, and they did not recur on the re-measure. Worth remembering when reading this suite's output — the harness reports a timeout as a compile failure.

    Fixed in #6358 (3 real cases)

    • S15.10.4.1_A6_T1 — an own property must shadow the RegExp.prototype method. __re.toString = Object.prototype.toString; __re.toString() returned "/(?:)/". Two independent layers dropped the override: the runtime regex dispatch ran before any own-property check, and codegen's universal .toString() fold rewrites x.toString() straight into js_jsvalue_to_string_method(x), checking only user classes for an override and never runtime own properties. re.exec / re.test overrides were broken the same way and are fixed with it.
    • S15.10.5_A2_T2 — RegExp.indicator returned undefined after Function.prototype.indicator = 1. HIR member-lowering collapses <Builtin>.<prop> to globalThis.<prop> unless the property is on a per-builtin carve-out list, so an unrecognized property lost the receiver and never reached the constructor's prototype chain (Function.prototype IS RegExp's [[Prototype]]).
    • prototype/exec/u-lastindex-value — lastIndex was counted in Unicode scalars, not UTF-16 code units. /./ug.exec('𝌆') left lastIndex at 1 instead of 2. This was systematic: ~20 sites across exec / match / matchAll / replace / search converted byte offsets with chars().count(), so every index at or past an astral character disagreed with str.length / charAt on the same string.

    Remaining: 6

    4 cases — one root cause: ECMA-262 RepeatMatcher semantics

    S15.10.2.5_A1_T4, prototype/exec/S15.10.6.2_A1_T6, nullable-quantifier, lookahead-quantifier-match-groups

    These need two behaviours a finite-automata engine cannot express:

    1. Captures inside a quantified group reset on each iteration — /(z)((a+)?(b+)?(c))*/.exec("zaacbbbcac") must give undefined for the (b+)? group; Perry gives "bbb" (the value from an earlier iteration).
    2. An optional iteration matching the empty string is discarded and retried — /(a?b??)*/ must match all of "ab"; Perry gives "a".

    Perry compiles with the Rust regex crate, and the fancy-regex fallback delegates non-fancy patterns straight back to it — so neither engine path implements JS backtracking capture semantics. This is not a surface gap: closing it means writing a spec-compliant backtracking matcher. It is a coherent, self-contained piece of work and deserves its own issue rather than sitting on this worklist.

    2 cases — the documented lone-surrogate / WTF-8 categorical gap

    S15.10.2.8_A2_T1, character-class-escape-non-whitespace

    Both hinge on a lone surrogate (\uD800) surviving as a string value, which Perry's UTF-8 string representation cannot represent. This is already a standing limitation recorded in CLAUDE.md ("lone surrogate handling (WTF-8)"). These should be excluded from this worklist rather than counted against it — otherwise it can never reach zero.

    Suggested disposition

    With the WTF-8 pair excluded as a documented categorical gap, this worklist is down to one cluster (RepeatMatcher). I'd suggest:

    Also worth filing separately, surfaced while fixing S15.10.5_A2_T2: the receiver-drop in HIR member-lowering affects every built-in constructor (Array.indicator, Object.indicator, … all read globalThis.<prop>). #6358 only lifts it for RegExp, where nothing depends on the collapsed shape. For the others the collapsed shape is load-bearing — the intrinsic call/constant-fold paths for Array.isArray, Object.keys, Math.PI, … rely on it — so a general fix needs a complete per-builtin intrinsic-member table.

  4. added
    triagedMaintainer reviewed; type, scope, and next step are clear
    on Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    parityCompatibility gap with Node.js, ECMAScript, or the supported ecosystemtriagedMaintainer reviewed; type, scope, and next step are clear

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions