[durgaprasadml]

Automate stable REST API permission reference doc generation.

This PR introduces an automated extractor for stable REST API permissions and generates a reference RST document directly from the source routes and permission declarations.

Current implementation:

• statically analyzes /api/v2 route definitions
• extracts requires_access_* permission metadata
• generates api_permissions_ref.rst
• includes regression and invariant tests
• integrates with existing prek-based generation workflows

This PR is intentionally opened as a draft to gather maintainer feedback on:

• AST-based extraction vs runtime introspection
• final file placement
• generation workflow integration
• committed generated artifacts

Closes: #10937

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[jscheffl]

It is missing to add it to .pre-commit-config.yaml

[durgaprasadml]

Implemented the requested changes:

• Added deterministic sorting for generated REST API permission entries (grouped by endpoint path, then HTTP method).
• Registered the generator in .pre-commit-config.yaml following existing Airflow prek hook conventions.
• Regenerated api_permissions_ref.rst with the stabilized ordering.

This ensures the generated permission reference stays automatically synchronized with route changes and prevents stale generated docs from passing CI.

[jscheffl]

Good for me, any other reviewer opinion? (besides CI needs to turn green...)

[Copilot]

Pull request overview

This PR automates generation of stable REST API permission reference documentation by statically extracting requires_access_* dependencies from /api/v2 route definitions and rendering a committed RST reference.

Changes:

• Adds a stdlib-only AST extractor and CLI for generating/checking API permission docs.
• Adds regression/invariant tests for route parsing, resource mapping, rendering, and check mode.
• Registers a prek hook and commits the generated api_permissions_ref.rst artifact.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
scripts/ci/prek/extract_permissions.py	New extractor and RST generator for stable REST API permissions.
scripts/tests/ci/prek/test_extract_permissions.py	Tests for extractor helpers, real route integration, rendering, and check mode.
airflow-core/docs/security/api_permissions_ref.rst	Generated permission reference documentation.
.pre-commit-config.yaml	Adds prek hook to regenerate the permission reference.

[durgaprasadml]

Thanks for the review and the detailed feedback.

Addressed the positional access_entity case for requires_access_dag by extending the AST extraction logic to support both:

• keyword usage (access_entity=...)
• second positional argument usage

Also added a regression test covering the positional form (DagAccessEntity.TASK_LOGS) and regenerated the permission reference docs.

Additionally, the generator hook is now registered in .pre-commit-config.yaml to ensure generated docs remain synchronized with route/security changes during CI and local prek runs.

[durgaprasadml]

@potiuk review

[durgaprasadml]

Good catch @Vamsi-klu , thanks for pointing this out.

You're right — the current extractor only emits routes declaring requires_access_* dependencies, while the generated page wording currently implies full /api/v2 endpoint coverage.

I’ll update the generation logic to include public endpoints without Airflow permission dependencies and render them explicitly as requiring no Airflow permission, so the generated reference remains complete and self-consistent.

[durgaprasadml]

Addressed the feedback regarding public /api/v2 endpoints being omitted from the generated reference.

Updates included:

• Public stable API endpoints are now extracted and rendered explicitly as requiring no Airflow permission.
• Added regression and integration coverage for:
  • /api/v2/monitor/health
  • /api/v2/version
  • /api/v2/auth/login
  • /api/v2/auth/logout
• Kept deterministic generation ordering and simplified the rendering pipeline so sorting happens only during normalization/extraction.
• Regenerated api_permissions_ref.rst.

CI is now green on the updated implementation.

@potiuk could you please take another look when you have time?

[potiuk]

@jscheffl's approval here is from May 29 (74ad787b0), and there have been 8 commits since — the mypy fix, the prek hook, positional access-entity extraction, more endpoints in the reference. Worth a re-review before merge so the approval reflects the current diff. @jscheffl, could you take another look?

---

Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting

[jscheffl]

Re-Review OK for me. CI should be green and like another reviewer prior merge.

[potiuk]

Really nice tooling — static AST extraction with a --check staleness gate is the right shape, and the resource mapping is thorough. One correctness bug to fix before this lands, since the output is a published security reference:

Wrong path for multi-router files. _extract_router_prefix() returns only the first AirflowRouter in a file, but dag_run.py defines two (dag_run_router → /dags/{dag_id}/dagRuns, dag_run_at_dag_router → /dags/{dag_id}). As a result POST /clearDagRuns is generated as /api/v2/dags/{dag_id}/dagRuns/clearDagRuns (line 270 of the generated api_permissions_ref.rst) — that path doesn't exist; the real one is /api/v2/dags/{dag_id}/clearDagRuns. Suggest keying the prefix off the router variable in each decorator (@dag_run_at_dag_router.post → look up that router's prefix) instead of a single file-wide prefix.

Secondary (non-blocking but security-relevant): the extractor only reads per-route decorator dependencies=[…]. It won't see permissions declared at the router level (AirflowRouter(dependencies=[…])) or via Security(…). None are used in the public routes today, so nothing is wrong now — but if a permission ever moves to the router level, those endpoints would silently render as "No Airflow permission required," which is the dangerous direction for this doc. A small regression test (or an explicit assertion/comment) guarding that assumption would make this robust.

---

Drafted-by: Claude Code (Opus 4.8); reviewed by @potiuk before posting

[potiuk]

You fixed exactly what I flagged, and then some:

• The multi-router prefix bug now keys off each decorator's router variable via
  _extract_routers(), so POST /clearDagRuns correctly resolves to
  /api/v2/dags/{dag_id}/clearDagRuns. Nice regression coverage in
  test_multiple_routers_in_same_file and test_clear_dag_runs_endpoint_prefix.
• Router-level permission dependencies now raise loudly ("Unsupported extraction
  semantics…") instead of silently rendering "No Airflow permission required" —
  which was the dangerous direction for a published security reference. Guarded
  by test_router_level_permission_dependency_raises_value_error.

CI green. Thanks for the thorough follow-up — LGTM.

---

This review was drafted by an AI-assisted tool and confirmed by an Airflow
maintainer. The maintainer approving this PR has read the findings and signed
off. If something feels off, please reply on the PR and a maintainer will
follow up.

More on how Airflow handles maintainer review:
contributing-docs/05_pull_requests.rst.

[github-actions]

Backport successfully created: v3-3-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
✅	v3-3-test