Repository navigation
feat(examples): add Invoice Desk fixture and OpenAI PR scans - #1144
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0e067a4ab6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1971a75c08
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
mldangelo-oai
changed the title
mldangelo-oai
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4d6c130b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4d6c130b9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01f25335ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
soyeon-oai
left a comment
soyeon-oai
left a comment
There was a problem hiding this comment.
Reviewed the fixture/scenario boundaries, protected-main dispatch, exact committed-blob extraction, post-approval eligibility check, checkout removal before inference, cache permissions and result reporting. No new actionable findings. Both hosted Node versions passed all 37 app/orchestration tests and node-ci passed. This is source review with hosted evidence, not independent execution; the protected inference/automatic-dispatch rollout still needs the post-merge validation described in the PR.
|
@codex review |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The fallback can miss token-created PRs and repeat paid scans for unchanged fallback targets.
Review effort: Balanced
Findings: 3
Open (3)
What changed in this PR
Adds Invoice Desk, a dependency-free security-scanning fixture with ten intentional vulnerabilities, behavioral QA, and protected OpenAI scan automation.
Changes:
- Adds the sample application, threat model, QA manifest, and tests.
- Adds trusted workflows for PR targeting, source extraction, caching, scanning, and reporting.
- Documents local use, CI operation, and scanner evaluation.
| File | Description |
|---|---|
examples/README.md |
Links the new sample. |
examples/invoice-desk/SECURITY.md |
Documents the threat model. |
examples/invoice-desk/README.md |
Provides setup and CI guidance. |
examples/invoice-desk/qa/README.md |
Defines QA and scoring practices. |
examples/invoice-desk/qa/expected-findings.json |
Records ten seeded findings. |
examples/invoice-desk/qa/app.test.mjs |
Tests scenarios and normal behavior. |
examples/invoice-desk/app/views.mjs |
Renders application pages. |
examples/invoice-desk/app/store.mjs |
Creates synthetic in-memory data. |
examples/invoice-desk/app/server.mjs |
Implements HTTP routing. |
examples/invoice-desk/app/README.md |
Documents the standalone application. |
examples/invoice-desk/app/public/styles.css |
Styles the interface. |
examples/invoice-desk/app/public/client.js |
Handles browser interactions. |
examples/invoice-desk/app/package.json |
Defines runtime metadata. |
examples/invoice-desk/app/invoices.mjs |
Implements invoice workflows. |
examples/invoice-desk/app/integrations.mjs |
Implements connector behavior. |
examples/invoice-desk/app/imports.mjs |
Implements imports and policies. |
examples/invoice-desk/app/accounts.mjs |
Implements authentication and profiles. |
examples/github-actions/README.md |
Documents OpenAI sample scanning. |
.github/workflows/invoice-desk.yml |
Runs tests and dispatches scans. |
.github/workflows/invoice-desk-scan.yml |
Performs protected source scans. |
.github/scripts/invoice-desk-target.test.mjs |
Tests target resolution. |
.github/scripts/invoice-desk-target.mjs |
Resolves eligible scan targets. |
.github/scripts/invoice-desk-source.test.mjs |
Tests safe source extraction. |
.github/scripts/invoice-desk-source.mjs |
Extracts committed application blobs. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d8b111e134
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
|
Codex Review: Didn't find any major issues. What shall we delve into next? Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It combines intentionally vulnerable code with privileged workflow dispatch, protected secrets, and hosted inference, warranting final human review.
Review effort: Balanced
Findings: None
Resolved since last review (1)
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 61bc0326e4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
Add Invoice Desk, a local application with synthetic data and ten intentional security scenarios for scanner QA. When GitHub delivers an eligible event, PRs targeting
mainqueue a protected OpenAI scan of the application's exact source revision. Maintainers can use the same protected workflow to dispatch a PR scan manually if automatic events do not arrive. Findings are report-only; scan failures fail the scan job.Changes
main, including a fallback for completed PR test runs. Reuse the same PR/head dispatch key so fallback runs can recognize an earlier scan.Testing
00a9608: 45 jobs passed and one was skipped.node --test .github/scripts/invoice-desk-source.test.mjs .github/scripts/invoice-desk-target.test.mjs: 18 passed, including repeated fallback dispatches.00a9608, 17 mocked workflow tests passed (eight new Invoice Desk cases and nine existing example cases). They execute the workflow scripts with synthetic CLI results to check arguments, exit/output handling, missing-key behavior, SARIF gating, metrics, and required artifacts. Types, formatting, andgit diff --checkpassed.git diff --checkpassed.00a9608completed with no findings.mainand protected environment approval; they have not been run as part of this cleanup.Risk and rollout
The application is deliberately insecure. Run it locally with synthetic data only; do not deploy it.
Configure
invoice-desk-inferencewith required reviewers, a deployment rule allowing protectedmain, and a dedicatedOPENAI_API_KEYenvironment secret. Review the selected source revision before approving inference. After merge, run a baseline twice to check reporting and cache reuse, then verify automatic dispatch from a PR.An approved scan already running can finish after the PR head changes; its reports identify the scanned revision. The scan runs in a separate Actions workflow. It does not create a PR status check, Code Scanning alerts, or PR comments. A completed scan does not prove all ten cases were found; review distinct root causes before setting recall thresholds. No public CLI, SDK, or plugin API changes.
Public disclosure review