Repository navigation
fix(computer): tolerate macOS AXDescription failure for verified controls - #1392
Conversation
swjnxyf
left a comment
There was a problem hiding this comment.
Reviewed full two-file macOS Accessibility change and native callers on mini's primary checkout. Narrow AXError::Failure fallback is permitted only for optional AXDescription on AXTextArea/AXRadioButton after role, native-protected and subrole checks; strict defaults remain for AXValue, AXTitle, AXPlaceholderValue, AXProtectedContent, other roles and other errors. Protected/inherited/uncertain subroles remain redacted and denied for native control. No changes to execution, authority, Window identity, input or other platforms. Verified 18/18 GitHub CI jobs passed; mini offline webcodex-computer unit suite: 87 passed, 0 failed, 1 intentionally ignored, 12 live test fixtures filtered; cargo fmt and diff check passed; remote-main merge-tree clean. PR author's opt-in TextEdit read-only smoke was not re-run, so this approval does not claim full-tree E2E coverage. No blocking findings or follow-up commit. Approved for user-authorized squash merge.
Summary
On an unlocked macOS desktop, TextEdit exposes an
AXTextAreaand anAXRadioButtonwhose optionalAXDescriptionlookup returnsAXError::Failure (-25200). This prevented deeper Computer Use Accessibility observation even though the window identity and other metadata were valid. The test reproduced the error with native macOS calls before the change.This is independent of locked-screen behavior: the console was active and the GUI session was unlocked when the native failure was reproduced. Related to the broader Computer Use reliability tracker #1161.
Change
AXError::Failureas a missing optionalAXDescriptiononly for verified, non-protectedAXTextArea/AXRadioButtonelements.finish_ax_fingerprintsecure-subrole and inherited-protection guard still prevents text reads.AXTitle,AXPlaceholderValue,AXValue,AXProtectedContent, all other AX roles and all other AX errors strictly handled.Validation
Base: current
main@8fa239e3, isolated Mac worktree. Installed production Server/Runner was not modified.cargo test --locked --offline -p webcodex-computer --lib -- --skip macos_live_tests --test-threads=2: 87 passed, 0 failed, 1 ignored.cargo test --locked --offline -p webcodex-computer --lib unlocked_textedit_fingerprints_without_document_value_reads -- --ignored --nocapture: 1 passed.AXTextAreaand 1AXRadioButton, and independently observed 2 realAXDescriptionprovider failures with nativeAXError::Failure.AXValueor the fullaccessibility_tree, to avoid reading document content; no UI actions were performed. Thus this validates the corrected native metadata path, not end-to-end full-tree output.cargo fmt --all -- --checkandgit diff --check: pass.Only two source files are changed; no credentials, document contents, machine identifiers or screenshots are included.