Repository navigation
[Task]: Implement computer_use_runtime protocol as provider-neutral schema builders #3110
Description
Activity
感谢这个 issue,调研很扎实。但直接实现 docs-only 协议的三个 builder 会撞上仓库的 scope-fit 规则:没有 active call site 的协议不应先变成生产代码;而且仓库里已经有
content_ops(social_browser_x、connector_packets.py)和value_connectors(planner.py)两套真实在用的 browser-backed packet 契约。再独立建一套computer_use_*词汇,将来要么做映射层,要么废弃一边。建议改成下面这个结合方案:1. A — 把协议变成 machine-checkable 契约
- 为
computer_use_capability_profile_v0 / action_plan_v0 / receipt_v0增加 JSON Schema(或等价的 typed schema),放在协议文档旁; - 把文档里的 “Smoke Expectations” 固化成正反例 fixtures(拒绝 raw screenshot/cookie/credential、拒绝无 gate 的 external-write、区分
completed / stopped_at_gate / blocked_by_unknown_modal); - smoke 从 schema + 文档语义推导,而不是从实现自证。
2. B — 以真实契约做提取,提取结果必须被真实代码调用
- 通用 builder 从
content_ops/connector_packets.py(capability profile ↔ source_profile / connector_trial / install_check)、social_browser_x.py(write_gate / truth_contract)、value_connectors/planner.py(approval gate / forbidden actions / URL 与敏感文本清洗)的真实 shape 提取; - 验收条件:提取出的 builder 必须被 content_ops / value_connectors 或新的 consumer 真实调用,不许先抽象后找用户;
- 放置上先归最近 owner(content_ops/value_connectors 内部),等出现第二个真实 consumer(如 Grok/桌面 runtime)再提升为共享模块。
3. C — 真实 consumer
- 新增
loopx computer-use plan / receiptCLI(或复用value-connectors plan接线),让 plan 从 LoopX todo/gate 状态生成、receipt 进入 user gate 投影; - quota spend 复用已有
loopx turn run-once的 validated writeback,不做新的计费逻辑。
4. CUA showcase(合入门槛,核心)
- 一个 hermetic 端到端 demo:本地 fixture 页面(无真实账号) + 浏览器驱动(Playwright/ego-browser,或 fake runtime 模式)+ mock/deterministic 判断,完整走通:
loopx turn run-once → CUA host adapter(用新的 builders 从 Turn envelope 生成 action plan)→ runtime 执行 → compact receipt → 独立 validator → LoopX writeback + quota spend → 幂等回放; - 至少两个场景:draft-until-gate(填表后停在 submit,receipt=
stopped_at_gate,投影出具体用户问题)和 unknown modal(返回blocked_by_unknown_modalblocker,不硬点); - 真实浏览器模式由贡献者本地跑通并贴证据;CI 跑 hermetic 模式;
- 原始截图/DOM/凭据绝不进 LoopX state,只有 compact facts + evidence handle;
- showcase 跑通后再补 catalog 行。
合入验收清单
py_compile/ruff/mypy/loopx check通过;- schema fixtures + builder smoke 通过;
- content_ops / value_connectors 现有 smoke 在提取后不回归(证明 B 没有破坏真实契约);
- CUA showcase hermetic e2e 在 CI 绿,真实浏览器模式有本地验证记录;
- 无凭据、无真实账号、无原始会话数据入库。
参考实现:
scripts/dsh_turn_host_adapter.py(adapter 骨架)、examples/loopx-turn-dsh-real-e2e-smoke.py(hermetic e2e 模式)。如果整个范围太大,可以拆两个 PR:PR1 = A+B+C(无新 runtime 行为),PR2 = showcase;但 showcase 是最终合入的前置条件。如果你只想做契约层,可以只做 A(docs + fixtures),但那不进loopx/生产代码。
Thanks for the well-researched issue. Directly implementing the three builders for a docs-only protocol would violate the repo's scope-fit rule: no production module without an active call site. The repo already ships two real browser-backed contract families (
content_opsandvalue_connectors), so we'd rather combine all three directions:- A — Machine-checkable contract: JSON Schema + positive/negative fixtures next to the protocol doc; smokes derive expectations from the schema and doc semantics, not from the implementation.
- B — Extraction anchored on real contracts: derive the generic
computer_use_*builders from the actual shapes incontent_opsandvalue_connectors, and require that they are actually called by those code paths or by a new consumer — no abstraction before a user. - C — Real consumer: a
loopx computer-use plan / receiptCLI (orvalue-connectorswiring) that generates plans from LoopX todo/gate state and routes receipts into user-gate projection; quota spend reuses the existing validatedloopx turn run-oncewriteback. - CUA showcase (merge gate): a hermetic end-to-end demo on a local fixture page —
loopx turn run-once → CUA host adapter → runtime execution → compact receipt → independent validator → writeback + quota spend → idempotent replay. At least two scenarios: draft-until-gate (stopped_at_gatewith a concrete user question) and unknown-modal (blocked_by_unknown_modal, no clicking through). Real-browser mode is run locally by the contributor with evidence; CI runs the hermetic mode. Raw screenshots/DOM/credentials never enter LoopX state — only compact facts and evidence handles.
Reference patterns:
scripts/dsh_turn_host_adapter.py(adapter skeleton) andexamples/loopx-turn-dsh-real-e2e-smoke.py(hermetic e2e). If the scope is too large, split into PR1 (A+B+C) and PR2 (showcase), but the showcase stays a precondition for the final merge. If you only want the contract layer, A alone (docs + fixtures) is fine but stays out ofloopx/production code.- 为
更正上一条回复的第 4 点:showcase 不必基于
loopx turn run-once+ CUA host adapter。上一条把 showcase 限定在 Turn 协议上,这是我过度指定了。修正后的验收是:使用标准 LoopX 使用姿势跑通一个 CUA 场景即可,不需要为 showcase 造新的 turn 协议或 adapter。
具体来说:
- 标准 host 姿势:Codex App 里
/loopx <goal text>→loopx start-goal,LoopX 照常投影 goal/todo/claim/gate/quota/evidence;未来的 Grok build 如果作为 host,走同样的标准接线即可。 - CUA runtime(ego-browser / Playwright / 未来 Grok computer-use)只是 host 的工具/执行层:agent 在 goal 下通过工具调用驱动浏览器,LoopX 不接管像素级循环。
- 演示场景保持两个:draft-until-gate(填表后停在 submit,LoopX 投影出具体用户问题)和 unknown modal(返回 blocker,不硬点)。
- 约束不变:本地 fixture 页面、无真实账号、无凭据、原始截图/DOM 不进 LoopX state;CI 跑 hermetic 版本,真实浏览器模式本地验证并贴证据。
这样 A/B/C 不变(machine-checkable 契约、从
content_ops/value_connectors真实 packet 提取并被真实调用、真实 consumer),只是 showcase 的宿主从“自定义 turn adapter”改成“标准 host + 工具层”。可参考docs/reference/protocols/codex-app-host-command-registry-v0.md的标准接入姿势,以及content_ops/social_browser_x.py里 ego-browser 作为现成 browser-backed 工具的先例。
Correction to point 4 of my previous reply: the showcase does not need to be built on
loopx turn run-once+ a CUA host adapter. That was over-specified on my part.The revised acceptance is: run a CUA scenario through the standard LoopX usage posture, without inventing a new turn protocol for the showcase.
- Standard host posture:
/loopx <goal text>in Codex App →loopx start-goal; LoopX projects goal/todo/claim/gate/quota/evidence as usual. A future Grok build would wire in as a host the same way. - The CUA runtime (ego-browser / Playwright / a future Grok computer-use) is just the host's tool/execution layer: the agent drives the browser through tool calls under a LoopX goal; LoopX does not own the pixel-level loop.
- Keep the two demo scenarios: draft-until-gate (fill the form, stop at submit, LoopX projects a concrete user question) and unknown modal (return a blocker, do not click through).
- Constraints stay the same: local fixture page, no real accounts, no credentials, raw screenshots/DOM never enter LoopX state; hermetic version runs in CI, real-browser mode is validated locally with evidence.
A/B/C stay as described (machine-checkable contract; extraction anchored on and actually called by
content_ops/value_connectors; a real consumer). Only the showcase host changes from "custom turn adapter" to "standard host + tool layer". Reference:docs/reference/protocols/codex-app-host-command-registry-v0.md, andcontent_ops/social_browser_x.pyas the existing ego-browser tool precedent.- 标准 host 姿势:Codex App 里
感谢这么详细拆解,这几天我认真评估了一下 A+B+C+showcase 整体的实现难度。
我感觉按照目前我对这个仓库的熟悉程度来说,这个 scope 有点挑战,我担心自己接下来完不成,尤其是 CUA showcase 这一块。我去看了一下仓库现状,发现目前完全没有 Playwright(或任何浏览器自动化)依赖,也没有任何本地 fixture 页面的先例可以参考,
social_browser_x.py其实也不是真正驱动浏览器的代码,它只是一个描述外部工具 ego-browser 能力的 descriptor。也就是说 showcase 里"浏览器自动化基础设施"这部分基本要从零搭,还要保证它在 CI 里能 hermetic 地跑起来,同时正确接入标准 host 姿势(/loopx → start-goal → 工具调用驱动浏览器)下的 todo/gate/quota 投影。 这些我目前还不是很熟悉。虽然 showcase 用 mock/deterministic 判断替掉了真实 AI 判断这部分,但"截图循环"里我原本担心的浏览器驱动+落地验证的工作量基本还在。所以我想先只做 A(JSON Schema + 正反例 fixture,不进 loopx/ 生产代码)。B/C/showcase 我打算等自己对 LoopX 的核心机制(todo/gate/quota、host 接入姿势)更熟悉之后再挑战,或者如果有其他贡献者想先接手也完全没问题,我不想占着位置不动。
Thanks for the thorough, well-reasoned breakdown. I spent the past few days assessing what it would actually take to implement A+B+C+showcase end to end.
Honestly, given how unfamiliar I still am with this codebase, this scope feels like a real stretch, and I'm worried I won't be able to see it through, especially the CUA showcase. I looked into the current state of the repo and found there is no Playwright (or any browser-automation) dependency anywhere yet, and no existing local fixture-page pattern to build on
social_browser_x.pyisn't actually driving a browser either; it's just a descriptor for an external tool (ego-browser). So the "browser-automation infrastructure" part of the showcase would essentially need to be built from scratch, made to run hermetically in CI, and correctly wired into the standard host posture (/loopx -> start-goal -> tool calls driving the browser) with todo/gate/quota projection I'm not yet familiar with. The showcase replaces real AI judgment with a mock/deterministic decision, but the browser-driving and end-to-end verification work I was originally worried about in the "screenshot loop" is still largely there.So I'd like to start with just A (JSON Schema + positive/negative fixtures, staying out of loopx/ production code), which you mentioned is an acceptable starting point. I plan to come back to B/C/showcase once I'm more familiar with LoopX's core mechanics (todo/gate/quota, host integration posture) -- or if another contributor wants to pick those up first, that's completely fine, I don't want to sit on them.
也行,主要是我感觉 A 的设计需要经过 B、C 使用的打磨,类似于跑通一个 loopx 支持下的复杂 CUA 长程研究。
我觉得不一定要给 loopx 补基础设施,基于 ego lite browser 等外部依赖也行刚跑了个本地 spike 验证机制:一个 fixture 表单页,agent 用浏览器工具填完约定字段后停在 submit 前,产出 stopped_at_gate receipt;另一个页面模拟意外的支付类弹窗,agent 不点确认也不点关闭,直接停下产出 blocked_by_unknown_modal receipt。两次都是用的就是 agent 自带的工具调用能力,跟 ego-browser 暴露给 agent 的能力是一类东西。
这证明了不需要往 loopx 里加任何浏览器自动化基础设施,"判断+停在正确位置"这层能力本来就有,loopx 只需要在外面包 plan/receipt/gate 这层状态追踪。
我打算按 A → B → C → showcase 走完整条链路,不只是停在A。
CI 里的 hermetic 部分打算参考 loopx/experiments/planner_worker/runtime.py 的 Protocol + Fake 实现模式,不依赖真实浏览器。动手前有两个想提前对齐的问题:
B 提炼出来的公共 builder,先放在 content_ops 还是 value_connectors 内部比较合适?
C 里"标准 host 姿势"具体希望我对接到哪个现有 CLI 命令族,是新开 loopx computer-use plan/receipt,还是复用 value-connectors plan 加个 connector kind?如果方向 OK 我就按这个顺序开始了
这个 spike 很有价值:它证明了 LoopX 不需要自建 browser driver 或 pixel loop,host 已经可以提供足够的 computer-use primitive;LoopX 真正需要补的是边界协议、状态归约、gate 和可验证回执。
但我想同时纠正一个容易越做越重的抽象方向:CUA 不等同于 value connector,目前也不应直接注册成一个 LoopX product capability。 它首先是一类 provider/runtime execution surface。
Maintainer 结论
可以继续按 A → B → C → showcase 推进,但需要按下面的 ownership 重新定义 B 和 C:
outcome capability -> bounded action request CUA provider/runtime -> observation + typed ACK/receipt capability-local reducer -> proposed transition LoopX Kernel -> accepted todo/gate/evidence/quota state四层职责应当分开:
层 应负责 不应负责 Capability(如 content-ops、explore、issue-fix)用户可感知的 outcome、domain policy、允许的 effect、如何解释 provider ACK 浏览器安装、像素循环、某个 provider 的 session 细节 CUA provider/runtime(如 host browser tool、ego-browser、Playwright provider) session、低层 action、observation、replay/evidence handle、typed stop reason 直接完成 LoopX todo、创建 gate、决定 quota/writeback Extension/package 可选 provider 的 install/doctor/enable/disable/upgrade/compatibility 发明一个没有 caller outcome 的“能力” Kernel durable todo、gate、quota、evidence、recovery 与最终 transition authority 执行浏览器动作或理解某个 UI 的领域语义 这里是多对多关系:一个 CUA provider 可以服务
content-ops、explore等多个 capability;同一个 capability 也可以选择 API、CLI、CUA 或人工 provider。value-connectors当前是兼容 facade,不应成为新的通用 CUA owner。另外要区分两种经常都叫 “capability” 的东西:
- LoopX product capability:稳定、provider-neutral、面向 caller outcome 的契约;
- provider-advertised capability:某个 runtime 声明自己支持 screenshot、click、type、accessibility tree 等 primitive。
因此协议里的
computer_use_capability_profile_v0容易造成误解。我建议在落地前改成computer_use_runtime_profile_v0(或provider_profile_v0),并优先使用runtime_id/provider_id,不要让connector_id暗示它属于value-connectors。协议名称也可以从computer-use-runtime-connector-v0收敛成computer-use-runtime-v0。对当前 protocol draft 的关键修正
现在最需要调整的是 receipt ownership。provider 可以报告事实,但不能替 LoopX 决定状态迁移。
也就是说,receipt 可以包含:
- 尝试了什么 bounded action;
- 观察到了什么;
completed、stopped_at_gate、blocked_by_unknown_modal、failed等 typed stop reason;- compact evidence / replay handle;
- provider-side idempotency key 和 session reference。
但不应包含由 provider 决定的
next_loopx_writeback.complete_todo、create_user_gate或类似命令。正确的语义是:- provider 返回 ACK/事实;
- owning capability 的 domain-local reducer 结合原 action request、gate binding 和 domain policy 解释 ACK;
- reducer 提出 todo/gate/evidence transition;
- Kernel 校验 revision、authority、合法 transition 和 quota 后接受或拒绝。
这也是我对 “reducer/ACK” 的判断:它是 capability/domain-local 的,不应做成一个全局 CUA reducer。
unknown_modal在内容发布、支付、账号设置和研究浏览中的后续动作可能完全不同;CUA 层只需忠实报告事实。安全边界也不要主要依赖
forbidden_actions里的自然语言,或复用value_connectors/planner.py中的字符串 denylist。建议最少有 typed 字段表达:effect_class: read / draft / external_write / credential_use 等;write_scope与 validation target;gate_binding: gate id、revision、status;- action/result/stop reason;
- receipt/replay identity。
自然语言可以补充说明,但 machine-enforced obligation 必须由 schema 和 transition validator 承担。
A → B → C → showcase 的建议落地
A. 先把 provider boundary 做成 machine-checkable contract
保留正向与负向 fixtures,覆盖至少:
- 在 external write 前准确停止;
- 未知 modal 不被擅自确认或关闭;
- receipt 不携带 credential、cookie、原始 DOM、私有页面正文等 durable state;
- stale gate revision、缺少 write authority、重复 replay 被拒绝或幂等处理;
- provider 不能直接构造 Kernel writeback。
这里可以使用 Protocol + Fake,但 Fake 是测试替身,不是产品实现。测试 oracle 应独立判断 fixture 的 UI state 和合法 transition,不能让 Fake 直接回放测试预期,否则会变成 self-proving test。
A 阶段可以提供 schema/validator/fixtures;在真实 consumer 出现前,不建议仅为了放 schema 就新增一个宽泛的生产模块
loopx/computer_use_runtime.py。B. 第一个 adapter/reducer 放在真实 outcome owner 内
对你当前 “填写内容并停在 submit 前” 的场景:
- 如果它表达 draft/publish 工作流,第一条 vertical slice 放在
content-ops; - 如果主场景是长程网页研究与证据采集,则放在
explore; - 如果 fixture 只是任意表单,它本身还不足以决定一个新 capability owner。
所以对第一个具体问题,答案不是在
content_ops和value_connectors之间选一个“永久公共 builder”:首个 domain adapter/reducer 放在content-ops(若采用当前发布前 gate 场景),不要放进value-connectors;共享 provider SPI 暂缓提炼。stopped_at_gate和blocked_by_unknown_modal是同一个 consumer 的两个结果分支,不算两个独立 consumer。等第二个真实 outcome domain(例如explore或另一个非内容发布路径)也需要相同 request/receipt 语义,再提取窄的共享 CUA provider protocol。这样提取的是已被两个调用点证明的共同知识,而不是未来假设。C. CLI 应由 outcome 领航,而不是 execution mechanism 领航
对第二个具体问题,我建议:
- 不新增
value-connectors plan --connector-kind computer_use; - 暂时也不新增顶层
loopx computer-use plan/receipt; - 先通过 owning capability 的现有 CLI/API surface 发起,例如内容发布场景归
content-ops,研究场景归explore;如果现有命令不够,再增加 capability-owned 的窄命令。
只有当我们已经证明存在一个跨 domain、对 caller 独立有用的 outcome(例如 “bounded and verified UI task”),并且它有真实入口、至少两个 provider/consumer 约束和 focused validation,才考虑把它提升为独立 product capability。即使届时提升,名称也应描述 caller outcome,而不是 delivery mechanism
computer-use。provider 自身的 readiness/doctor/profile 命令可以属于 host integration 或 extension lifecycle;那是诊断 provider,不等于新建 product capability。
Showcase. 证明 LoopX 控制面,而不是证明浏览器库
showcase 使用标准
/loopxhost posture:目标与 todo 在 LoopX,agent 使用 host 已提供的 browser/CUA tool,provider 返回 compact receipt,capability reducer 决定是否提出 gate/transition。建议保留两个 fixture 页面:
- 正常填写后停在 submit 前,形成带精确问题的 external-write gate;
- 遇到未知支付类 modal,保持页面不变并 hand off。
本地可以用真实 browser tool 作为补充证据;CI 用 hermetic Fake 验证协议与 reducer。不要把真实账户、凭据、原始截图/DOM、完整页面正文写入 LoopX durable/public-safe state。showcase 的成功标准是 stop/gate/recovery 可验证且 replay 幂等,不是 LoopX 自己驱动了多少像素动作。
Extension 什么时候出现
- 如果 CUA 已经是 Codex/Claude/其他 host 原生暴露的 tool,LoopX 只消费 host declaration 和 receipt,不需要为了它创建 extension;
- 如果 LoopX 要负责安装、doctor、启停、升级、权限声明、版本兼容和 dispatch 某个可选 provider,再把该 provider 做成 extension/package;
- 如果是 “Grok 负责 reasoning + browser driver 负责动作”,它是 composite provider/host wiring,Grok 本身并不会因为能看图就自动成为 CUA provider。
extension 是交付与生命周期轴,不是 runtime 数据流里的第五层,也不应为了“可安装”而反向制造一个空的 capability。
建议拆分与 acceptance
为了让 diff 可审查,我建议至少拆成两阶段:
- Contract PR:修订 research protocol、ownership table、schema/fixtures/validator;重命名歧义字段;移除 provider-authored Kernel writeback;不加 speculative core abstraction。
- Vertical-slice PR:一个 outcome-owned request + domain reducer + Fake provider + 标准 host showcase;覆盖 gate、unknown modal、stale revision、idempotent replay 和 public/private boundary。
第二个真实 outcome consumer 出现后,再做第三个 extraction PR,决定共享 provider SPI 和 extension packaging。无需为了宣称 “A→B→C 全做了” 在首个 PR 里提前固化整个横向框架。
我会按以下条件判断这条实现链是否完成:
- CUA provider 不能直接 complete todo 或 create gate;
- external write 必须绑定可验证的 live gate/revision;
- compact receipt 足够 reducer 做决定,但不泄露 raw/private browser state;
- replay/retry 可幂等,未知 UI 默认停下而非猜测;
- validator 与 Fake 输出独立,负向 fixture 能证明非法 transition 被拒绝;
- preview/readiness 不改变 durable work state,真实 writeback 先经 Kernel validation;
- 不扩大默认权限,不改变现有
value-connectorscompatibility contract。
所以总体方向是认可 spike、继续推进,但将目标从“实现一个 generic computer-use builder/CLI”调整为:先完成一个 outcome-owned vertical slice,证明 provider ACK → domain reducer → Kernel transition;等第二个真实 domain 再抽共享 CUA provider contract。
English summary
The spike validates the right execution model: LoopX does not need to own a browser driver or pixel loop. However, CUA is primarily a provider/runtime surface, not a value connector and not yet a standalone LoopX product capability.
- Keep outcome policy in
content-ops,explore, or another real capability. - Let the CUA provider return observations and typed ACKs only; it must not author todo/gate writebacks.
- Interpret the ACK in a capability-local reducer, then let the Kernel validate and commit the transition.
- Do not add CUA to the
value-connectorscompatibility facade. - Do not add a top-level
loopx computer-useCLI until a provider-neutral caller outcome is proven. - Put the first publish-before-submit slice in
content-ops; extract a shared provider SPI only after a second distinct outcome consumer exists. - Use an extension only when LoopX owns optional provider lifecycle such as install/doctor/enable/upgrade. A host-native browser tool needs no extension merely to be consumed.
- Use the real host tool for local evidence and a hermetic Fake for CI, while ensuring the validator is independent of the Fake's expected output.
This preserves the useful A → B → C → showcase plan without prematurely turning an execution mechanism into a broad core abstraction.
Reacted by karl4c开始规划 vertical-slice PR 时发现一个问题,PR1 合入的 validator 目前是没法被
loopx/生产代码调用的,想在动手前先对齐方向。问题
查了
pyproject.toml:[tool.setuptools.packages.find] include = ["loopx*"]—— 只有loopx*会进最终安装包,scripts/和docs/(除了个别*.mdglob)都不会被打包进去。jsonschema只在[project.optional-dependencies].test里,核心dependencies = []。普通pip install loopx拿不到jsonschema。
也就是说
scripts/computer_use_runtime_contract_validator.py和docs/reference/protocols/schemas/computer_use_runtime_v0/*.schema.json现在都是仓库 checkout +[test]extras 才能用的东西,对 PR1(协议文档 + 自带 smoke)没问题,但意味着如果content-ops里新加的 provider/reducer 想在 module 级别import这个 validator,普通用户装出来的loopx要么因为scripts/没打包直接 import 失败,要么因为jsonschema缺失而报错。这跟"PR1 的 validator/schema 必须被真实调用"的要求直接冲突。考虑过三条路径
A —— 把契约打包成正式安装内容。 挪到
loopx/contracts/computer_use_runtime_v0/,jsonschema加进核心依赖,scripts/里原来那个文件改成薄的 re-export 保持现有 smoke/canary 命令不用改,schema JSON 走package-data打包,路径解析从相对文件路径改成importlib.resources(装进 site-packages 后目录结构会变)。这是唯一能做到"只有一份实现、不存在漂移"的方案,但代价有两个:- 会给每个安装加一个新依赖,直接冲突 README 里明确写的"The Python package has no runtime dependencies outside the standard library"
- 现在项目里所有验证都是
pip install -e .(editable install),这测不出打包配置对不对;真要落地这条路径,需要额外做一次python -m build打 wheel、装进干净 venv 里的验证,这是目前从没做过的一步,不难但要补上。
B —— 生产代码保持零依赖,validator 只在测试里真正被调用。
content-ops自己写一套最小的、纯标准库的结构检查(必填字段存在、stop_reason是那四个值之一、external_write/credential_use要求gate_binding、拒绝 writeback 形状的字段),不 importjsonschema。测试套件里额外 import PR1 真正的 validator,断言生产代码构造/接受的每个 request/receipt 都能通过 PR1 的完整校验。工作量是三条里最小的(大概一两个小时),不碰依赖、不碰 PR1 已合入的文件布局。代价是生产路径上跑的是精简子集,不是完整契约,但这个弱点是诚实的、写在代码里能看出来的,不会假装自己是完整校验。第三条路径(原生 Python 重写、不依赖 jsonschema 库但真正跑在生产代码里)思考后感觉不太好。 三个 schema 加起来有 30-40 条字段级约束(required/type/enum/const/嵌套对象的 additionalProperties/一个 if-then 条件),手写忠实复刻要么写成大量逐字段判断(任何一处漏改都是无声漂移),要么等于自己重新发明一个简配版 jsonschema;而且"用现有 fixture 交叉验证两边一致"这个保证,只能证明"这些 fixture 上两边判断一样",不能证明"所有可能输入下都一样",以后 schema 加个新字段,只要没有 fixture 恰好测到,两边悄悄跑偏也不会被发现。评估了一下这是三条里工作量最大、保证却最弱的一条,先排除。
想问的
你倾向哪个方向,还是有别的思路?
- 可能B好点,实现和假想中的完备schema设计不必对齐 | | huangrt01 | | ***@***.*** | ---- 回复的原邮件 ---- | 发件人 | ***@***.***> | | 发送日期 | 2026年08月18日 09:45 | | 收件人 | huangruiteng/loopx ***@***.***> | | 抄送人 | huangruiteng ***@***.***>, Comment ***@***.***> | | 主题 | Re: [huangruiteng/loopx] [Task]: Implement computer_use_runtime protocol as provider-neutral schema builders (Issue #3110) | KarlLeen left a comment (loopx-project/loopx#3110) 开始规划 vertical-slice PR 时发现一个问题,PR1 合入的 validator 目前是没法被 loopx/ 生产代码调用的,想在动手前先对齐方向。 问题 查了 pyproject.toml: [tool.setuptools.packages.find] include = ["loopx*"] —— 只有 loopx* 会进最终安装包,scripts/ 和 docs/(除了个别 *.md glob)都不会被打包进去。 jsonschema 只在 [project.optional-dependencies].test 里,核心 dependencies = []。普通 pip install loopx 拿不到 jsonschema。 也就是说 scripts/computer_use_runtime_contract_validator.py 和 docs/reference/protocols/schemas/computer_use_runtime_v0/*.schema.json 现在都是仓库 checkout + [test] extras 才能用的东西,对 PR1(协议文档 + 自带 smoke)没问题,但意味着如果 content-ops 里新加的 provider/reducer 想在 module 级别 import 这个 validator,普通用户装出来的 loopx 要么因为 scripts/ 没打包直接 import 失败,要么因为 jsonschema 缺失而报错。这跟"PR1 的 validator/schema 必须被真实调用"的要求直接冲突。 考虑过三条路径 A —— 把契约打包成正式安装内容。 挪到 loopx/contracts/computer_use_runtime_v0/,jsonschema 加进核心依赖,scripts/ 里原来那个文件改成薄的 re-export 保持现有 smoke/canary 命令不用改,schema JSON 走 package-data 打包,路径解析从相对文件路径改成 importlib.resources(装进 site-packages 后目录结构会变)。这是唯一能做到"只有一份实现、不存在漂移"的方案,但代价有两个: 会给每个安装加一个新依赖,直接冲突 README 里明确写的"The Python package has no runtime dependencies outside the standard library" 现在项目里所有验证都是 pip install -e .(editable install),这测不出打包配置对不对;真要落地这条路径,需要额外做一次 python -m build 打 wheel、装进干净 venv 里的验证,这是目前从没做过的一步,不难但要补上。 B —— 生产代码保持零依赖,validator 只在测试里真正被调用。content-ops 自己写一套最小的、纯标准库的结构检查(必填字段存在、stop_reason 是那四个值之一、external_write/credential_use 要求 gate_binding、拒绝 writeback 形状的字段),不 import jsonschema。测试套件里额外 import PR1 真正的 validator,断言生产代码构造/接受的每个 request/receipt 都能通过 PR1 的完整校验。工作量是三条里最小的(大概一两个小时),不碰依赖、不碰 PR1 已合入的文件布局。代价是生产路径上跑的是精简子集,不是完整契约,但这个弱点是诚实的、写在代码里能看出来的,不会假装自己是完整校验。 第三条路径(原生 Python 重写、不依赖 jsonschema 库但真正跑在生产代码里)思考后感觉不太好。 三个 schema 加起来有 30-40 条字段级约束(required/type/enum/const/嵌套对象的 additionalProperties/一个 if-then 条件),手写忠实复刻要么写成大量逐字段判断(任何一处漏改都是无声漂移),要么等于自己重新发明一个简配版 jsonschema;而且"用现有 fixture 交叉验证两边一致"这个保证,只能证明"这些 fixture 上两边判断一样",不能证明"所有可能输入下都一样",以后 schema 加个新字段,只要没有 fixture 恰好测到,两边悄悄跑偏也不会被发现。评估了一下这是三条里工作量最大、保证却最弱的一条,先排除。 想问的 你倾向哪个方向,还是有别的思路? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>Reacted by karl4c
Task ID or area
area: computer-use runtime connector (docs/reference/protocols/computer-use-runtime-connector-v0.md)
Intent
I am proposing a new task
Summary
Context: I came across a Twitter thread where people discussed wiring Grok
4.5 to a computer-use-style QA loop, and huangruiteng publicly welcomed a
loopx PR for Grok support. After digging into it, I found xAI does not ship
a dedicated computer-use primitive -- a real Grok-driven loop would mean
building a screenshot -> Grok judgment -> action loop myself, which is more
than I can take on right now. Scoping this task down to the schema/builder
layer instead, so a full provider loop (Grok or otherwise) has a tested
contract to build against later.
docs/reference/protocols/computer-use-runtime-connector-v0.md defines a full
provider-neutral protocol for a computer-use runtime connector (capability
profile, action plan, receipt, etc.), but it is currently docs-only. I
checked and found no Python implementation anywhere in the repo, no smoke
test, and no open issue/PR covering it.
I'd like to implement the core request/plan/receipt builders as plain,
provider-neutral Python so any future computer-use runtime (browser or
desktop automation, any model behind it) can be wired against a tested
contract instead of a prose spec. This is infrastructure only -- no
specific provider, model, or live automation is included.
Proposed scope
In scope:
and pure builder/validator functions for computer_use_capability_profile_v0,
computer_use_action_plan_v0, and computer_use_action_receipt_v0
Expectations": capability-profile builder rejects raw screenshots,
cookies, credentials, and private bodies; action-plan builder refuses
external-write-class actions without forbidden_actions/gate coverage;
receipt builder distinguishes completed / stopped_at_gate /
blocked_by_unknown_modal outcomes
using fixture data only -- no real screenshots, browser/desktop session,
or model call
tier, mirroring the existing worker-bridge-install-contract-smoke.py entry
Out of scope (future slices, not this task):
computer_use_replay_handle_v0, computer_use_handoff_gate_v0 records
model's vision+tool-calling loop, etc.)
makes sense once something is actually runnable
Relevant files or commands
connector module)
pattern)
loopx canary premerge; the new smoke needs an entry here, mirroringhow examples/worker-bridge-install-contract-smoke.py is registered)
examples/computer-use-runtime-contract-smoke.py
Validation plan
Public/private boundary