Skip to content

fix(computer): tolerate macOS AXDescription failure for verified controls - #1392

Merged
swjnxyf merged 1 commit into
yyjeqhc:mainfrom
krsnak:fix/macos-textedit-axdescription-20261107
Nov 8, 2026
Merged

swjnxyf merged 1 commit into
yyjeqhc:mainfrom
krsnak:fix/macos-textedit-axdescription-20261107

Conversation

@krsnak

@krsnak krsnak commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Summary

On an unlocked macOS desktop, TextEdit exposes an AXTextArea and an AXRadioButton whose optional AXDescription lookup returns AXError::Failure (-25200). This prevented deeper Computer Use Accessibility observation even though the window identity and other metadata were valid. The test reproduced the error with native macOS calls before the change.

This is independent of locked-screen behavior: the console was active and the GUI session was unlocked when the native failure was reproduced. Related to the broader Computer Use reliability tracker #1161.

Change

  • Interpret the native typed AXError::Failure as a missing optional AXDescription only for verified, non-protected AXTextArea / AXRadioButton elements.
  • Select that narrow policy after reading role, native protection and subrole metadata. The existing finish_ax_fingerprint secure-subrole and inherited-protection guard still prevents text reads.
  • Keep AXTitle, AXPlaceholderValue, AXValue, AXProtectedContent, all other AX roles and all other AX errors strictly handled.
  • Keep the default native attribute-reading path strict. Do not change any controls, input effects, window matching, permissions, or non-macOS platforms.

Validation

Base: current main@8fa239e3, isolated Mac worktree. Installed production Server/Runner was not modified.

  • cargo test --locked --offline -p webcodex-computer --lib -- --skip macos_live_tests --test-threads=2: 87 passed, 0 failed, 1 ignored.
  • Explicit read-only live smoke from the locally compiled patched source: cargo test --locked --offline -p webcodex-computer --lib unlocked_textedit_fingerprints_without_document_value_reads -- --ignored --nocapture: 1 passed.
    • Console present, user logged in, screen not locked.
    • Located TextEdit's exact AX window, checked 1 unprotected AXTextArea and 1 AXRadioButton, and independently observed 2 real AXDescription provider failures with native AXError::Failure.
    • The smoke deliberately never requests AXValue or the full accessibility_tree, to avoid reading document content; no UI actions were performed. Thus this validates the corrected native metadata path, not end-to-end full-tree output.
  • cargo fmt --all -- --check and git diff --check: pass.
  • Unit tests cover scope of the typed exception, protected and uncertain subroles, permission-denied propagation and unrelated error codes.

Only two source files are changed; no credentials, document contents, machine identifiers or screenshots are included.

@swjnxyf swjnxyf left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed full two-file macOS Accessibility change and native callers on mini's primary checkout. Narrow AXError::Failure fallback is permitted only for optional AXDescription on AXTextArea/AXRadioButton after role, native-protected and subrole checks; strict defaults remain for AXValue, AXTitle, AXPlaceholderValue, AXProtectedContent, other roles and other errors. Protected/inherited/uncertain subroles remain redacted and denied for native control. No changes to execution, authority, Window identity, input or other platforms. Verified 18/18 GitHub CI jobs passed; mini offline webcodex-computer unit suite: 87 passed, 0 failed, 1 intentionally ignored, 12 live test fixtures filtered; cargo fmt and diff check passed; remote-main merge-tree clean. PR author's opt-in TextEdit read-only smoke was not re-run, so this approval does not claim full-tree E2E coverage. No blocking findings or follow-up commit. Approved for user-authorized squash merge.

@swjnxyf
swjnxyf merged commit 3c3c216 into yyjeqhc:main Nov 8, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants