Skip to content

feat(action): report usage and extend integration attribution - #1703

Open
mldangelo-oai wants to merge 2 commits into
mdangelo/codex/shared-telemetry-contextfrom
mdangelo/codex/telemetry-integration-followups
Open

mldangelo-oai wants to merge 2 commits into
mdangelo/codex/shared-telemetry-contextfrom
mdangelo/codex/telemetry-integration-followups

Conversation

@mldangelo-oai

Copy link
Copy Markdown
Collaborator

Summary

The standalone GitHub and Azure workflow examples now identify their integration, and the GitHub Action summary shows the configured provider, token usage, and estimated cost. Missing measurements remain unavailable instead of appearing as zero.

Changes

  • Add integration tags through existing Codex configuration, preserving the examples' analytics settings and CLI version pins.
  • Display reported input, cached-input, and output tokens, plus available cost ranges. Token counts remain visible when pricing is unavailable.
  • Extend the existing packaged-CLI test with a local OpenAI/Bedrock request fixture that checks attribution and the analytics opt-out in the native header and body.

Testing

  • Action validation passed: 155 tests, TypeScript, metadata, documentation, and reproducible bundle checks.
  • Full CLI compatibility suites passed against the pinned 0.2.0 package and built 0.4.0 candidate, including the native request fixture and usage adapter assertions.
  • Both example YAML files parsed; all seven embedded shell scripts passed syntax checks; four full/diff invocations preserved the expected arguments.
  • The published 0.1.27 and 0.3.0 CLI parsers accepted the example tags with analytics unset, enabled, and disabled. These parser checks used each version's TOML dependency and shared the remaining imports from the installed Action runtime.
  • Three independent code reviews and a separate verification passed with no actionable findings.

Risk and rollout

Stacked on #1691; merge after that PR and retarget this follow-up to main.

No new CLI flags, Action inputs or outputs, dependency upgrades, or telemetry collectors. Existing analytics opt-outs remain unchanged. Request fixtures use synthetic credentials and localhost endpoints; they verify outbound attribution, not provider-side reporting or completed model scans.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 17, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-17T20:58:07.201955Z d3c875d New commits
🔒 Security Review ✅ Completed 2026-10-17T21:00:18.199979Z d3c875d New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2d89564450

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

const esc = (value: string, limit = 4000) => html(plain(value, limit));
const counts = result.counts ? Object.entries(result.counts).map(([level, count]) => `${level}: ${count}`).join(' · ') : 'unavailable';
const usage = result.usage;
const tokens = usage ? `${usage.input_tokens.toLocaleString('en-US')} input (${usage.cached_input_tokens.toLocaleString('en-US')} cached) · ${usage.output_tokens.toLocaleString('en-US')} output` : 'unavailable';

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Normalize usage before formatting cached tokens

When a successful CLI result contains valid usage without cached_input_tokens, as supported by the SDK's tokenUsage() normalizer (which defaults that optional field to zero), usage.cached_input_tokens is undefined here and .toLocaleString() throws. Because runAction() builds the summary before setting success, this turns an otherwise successful scan into a failed Action; normalize the raw usage or format a missing cached count safely.

AGENTS.md reference: AGENTS.md:L10-L10

Useful? React with 👍 / 👎.

@alandelong-oai alandelong-oai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reporting and attribution improvements. My owner-level merge-safety review still reproduces the existing cached-token finding at d3c875d27a91e7b0a71c8ffcaeb69f6ddb4cc325. A result serialized by ScanResult with {input_tokens: 1200, output_tokens: 50} is accepted as completed, and its cost calculation defaults cached input to zero, but the new summary calls toLocaleString() on the omitted cached count. Through runAction, that changes the job to failure and disables SARIF upload. Please normalize the optional cached count consistently before formatting, with a regression through the result adapter and Action entrypoint. I have not duplicated the existing inline comment.

All 155 Action tests and validation checks passed, including bundle reproducibility. Another 55 focused SDK/example/cost tests passed. I checked both examples' shell syntax and full/diff arguments, their pinned CLI parsers, and 16 local-only native request probes across the pinned and candidate Codex versions; attribution and explicit analytics settings were preserved in both request headers and bodies. The full local CLI integration suite stops at an existing trusted-owner environment restriction before the new fixture; the matching hosted CI run passed that suite. This review is against the current #1691 stack base; retain the documented merge order.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants