Skip to content

ref(auth): Stop falling back to scopes bitfield in get_scopes - #127282

Merged
skaasten merged 1 commit into
masterfrom
ref/remove-scopes-bitfield-fallback
Oct 14, 2026
Merged

skaasten merged 1 commit into
masterfrom
ref/remove-scopes-bitfield-fallback

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

HasApiScopes.get_scopes() and ApiGrant.get_scopes() fall back to the legacy scopes BitField when scope_list is empty. Nothing writes the BitField anymore. Every write path sets scope_list, and hybrid cloud replication copies only scope_list. scope_list was backfilled from the BitField in 2017 (#5139). This change makes scope_list the only source of truth for scopes.

This is the first step toward removing the scopes column (#50475 / ENG-3384). The field and its migrations are left untouched here, so this PR is code-only and easy to revert.

Two tests built tokens only through the BitField and are updated to use scope_list.

Prod data check: on the control DB, no rows have a nonzero scopes with an empty scope_list in sentry_apitoken, sentry_apikey, sentry_apiauthorization, sentry_apigrant, or sentry_sentryapp (0 rows each). Nothing relies on the fallback today. The cell-silo token/key replicas only receive scope_list through replication, so they never depend on the BitField.

via shaun.kaasten.

--

View Junior Session [Sentry]

Co-Authored-By: Shaun Kaasten <shaun.kaasten@sentry.io>
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Oct 13, 2026
@sentry-junior
sentry-junior Bot marked this pull request as ready for review October 13, 2026 18:05
@sentry-junior
sentry-junior Bot requested review from a team as code owners October 13, 2026 18:05

@wmak wmak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non blocking just curious

if self.scope_list:
return sorted(self.scope_list)
return sorted([k for k, v in self.scopes.items() if v])
return sorted(self.scope_list)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason this model returns them sorted, but the other doesn't?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No functional reason. The difference was already there before this PR. HasApiScopes.get_scopes() has always sorted, and its docstring promises alphabetical order. ApiGrant doesn't use the mixin and has its own get_scopes() that never sorted. I left both behaviors as they were so this PR only removes the fallback.

In practice the grant's order doesn't leak out. When a grant is exchanged, the resulting ApiToken goes through the pre_save hook, and add_scope_hierarchy sorts its scope_list. The seer agent-token callers turn the grant's scopes into sets. If we want the two models to match, sorting in ApiGrant.get_scopes() is a one-line change and would be easy to do in a follow-up.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gonna keep as is to avoid any unknown dependencies on that order.

@skaasten
skaasten merged commit a9546ce into master Oct 14, 2026
90 checks passed
@skaasten
skaasten deleted the ref/remove-scopes-bitfield-fallback branch October 14, 2026 15:53
sentry-junior Bot added a commit that referenced this pull request Oct 15, 2026
Now that get_scopes() reads only scope_list (#127282), drop the legacy
scopes BitField from HasApiScopes and ApiGrant. The columns are made
nullable and moved to pending deletion with SafeRemoveField; a follow-up
migration will drop them once this has deployed.

Refs #50475
Refs ENG-3384

Co-Authored-By: Shaun Kaasten <shaun.kaasten@sentry.io>

Co-Authored-By: shaun.kaasten <shaun.kaasten@sentry.io>
skaasten added a commit that referenced this pull request Oct 16, 2026
…127386)

Phase 1 of 2 for dropping the legacy `scopes` BitField columns.
Follow-up to #127282, which made `get_scopes()` read only `scope_list`.

Refs #50475, ENG-3384

**What changed**
- `src/sentry/models/apiscopes.py`: removed the `scopes` BitField and
`ScopesDict` from `HasApiScopes`, including the `assert` that kept the
bitfield keys in sync with `ApiScopes`. New scopes no longer need an
`AlterField` migration.
- `src/sentry/models/apigrant.py`: removed `ApiGrant`'s own `scopes`
BitField.
- `src/sentry/migrations/1215_remove_scopes_bitfield_pending.py`: makes
`scopes` nullable, then `SafeRemoveField(..., MOVE_TO_PENDING)` on
`apiauthorization`, `apigrant`, `apikey`, `apitoken`, `sentryapp`.
-
`src/sentry/hybridcloud/migrations/0044_remove_scopes_bitfield_pending.py`:
same for `apikeyreplica` and `apitokenreplica`.
- Tests: removed direct `scopes=` bitfield use from `test_apitoken.py`,
`test_apiapplication.py` (deletions), and `test_sentry_app_updater.py`.

**Deploy safety**
- The only SQL is `ALTER COLUMN "scopes" DROP NOT NULL` on each table.
`SafeRemoveField` with `MOVE_TO_PENDING` is a no-op in the DB, so the
columns stay in place and old pods that still write `scopes` keep
working during the rollout.
- Prod check from the #127282 thread: 0 rows with `scopes <> 0 AND
cardinality(scope_list) = 0` on `sentry_apitoken`, `sentry_apikey`,
`sentry_apiauthorization`, `sentry_apigrant`, `sentry_sentryapp`.
Replica tables never get `scopes` set; replication only copies
`scope_list`.

**Phase 2 (separate PR, after this deploys):** `SafeRemoveField(...,
deletion_action=DeletionAction.DELETE)` for the same 7 models to
actually drop the columns.

**Verification**
- `makemigrations sentry hybridcloud --check` reports no changes;
`sentry upgrade` applies both migrations.
- Ran these suites on a fresh test DB: models
(apitoken/apigrant/apikey/apiapplication), api token/authorization
endpoints, authentication, oauth authorize/token/device, hybridcloud
replica, backup exhaustive/imports/exports, sentry app creator/updater,
apiapplication deletions. 575 pass. Failures, none touching `scopes`: 3
`test_api_token_details.py::*::test_invalid_user_id`
(superuser/elevated-mode path) and 1 backup test that needs Snuba, which
isn't available in my sandbox.
- prek (ruff, mypy) passes on the changed files.

<!-- junior-request-attribution:start -->
via **shaun.kaasten**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AD0BAS2BU2TC%3A1791913665.641439 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AD0BAS2BU2TC%3A1791913665.641439)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AD0BAS2BU2TC%3A1791913665.641439/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: shaun.kaasten <shaun.kaasten@sentry.io>
skaasten added a commit that referenced this pull request Oct 19, 2026
Phase 2 of 2 for removing the legacy `scopes` BitField. #127386 (phase
1) removed the field from the models and moved the columns to pending
deletion. This PR drops the columns.

Refs #50475, ENG-3384. Earlier steps: #127282 (removed the
`get_scopes()` fallback), #127386 (phase 1).

**Phase 1 is fully deployed.** #127386 shipped as getsentry `910f30be`,
and GoCD shows it passed on all backend pipelines (S4S2, DE, US,
control, ST) on 2026-10-16. No running pod still writes `scopes`.

**What changed**
- `src/sentry/migrations/1218_drop_scopes_bitfield_columns.py`:
`SafeRemoveField(..., deletion_action=DeletionAction.DELETE)` on
`apiauthorization`, `apigrant`, `apikey`, `apitoken`, `sentryapp`.
-
`src/sentry/hybridcloud/migrations/0045_drop_scopes_bitfield_columns.py`:
same for `apikeyreplica` and `apitokenreplica`.
- `migrations_lockfile.txt`.

Migrations only; no code changes. Rebased on master and renumbered from
`1216` to `1218` (now depends on
`1217_grouprelease_add_new_release_id`), because master added its own
`1216` and `1217`.

**SQL**
```sql
ALTER TABLE "sentry_apiauthorization" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_apigrant" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_apikey" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_apitoken" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_sentryapp" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "hybridcloud_apikeyreplica" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "hybridcloud_apitokenreplica" DROP COLUMN "scopes" CASCADE;
```

**Verification**
- Before the rebase: `makemigrations sentry hybridcloud --check`
reported no changes; `sentry upgrade` applied both migrations locally;
181 targeted tests passed on a fresh test DB (apitoken, apigrant,
apikey, apiapplication deletions, sentry app updater, hybridcloud
replica, api tokens endpoint, oauth token).
- The rebase changed only the migration number, its dependency, and the
lockfile. CI re-validates the migration graph.
- No raw SQL references to these `scopes` columns in `getsentry/sentry`
or `getsentry/getsentry`.

<!-- junior-request-attribution:start -->
via **shaun.kaasten**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AD0BAS2BU2TC%3A1792158314.758249 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AD0BAS2BU2TC%3A1792158314.758249)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AD0BAS2BU2TC%3A1792158314.758249/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Shaun Kaasten <shaun.kaasten@sentry.io>
RudraPatel2003 pushed a commit that referenced this pull request Oct 19, 2026
…127386)

Phase 1 of 2 for dropping the legacy `scopes` BitField columns.
Follow-up to #127282, which made `get_scopes()` read only `scope_list`.

Refs #50475, ENG-3384

**What changed**
- `src/sentry/models/apiscopes.py`: removed the `scopes` BitField and
`ScopesDict` from `HasApiScopes`, including the `assert` that kept the
bitfield keys in sync with `ApiScopes`. New scopes no longer need an
`AlterField` migration.
- `src/sentry/models/apigrant.py`: removed `ApiGrant`'s own `scopes`
BitField.
- `src/sentry/migrations/1215_remove_scopes_bitfield_pending.py`: makes
`scopes` nullable, then `SafeRemoveField(..., MOVE_TO_PENDING)` on
`apiauthorization`, `apigrant`, `apikey`, `apitoken`, `sentryapp`.
-
`src/sentry/hybridcloud/migrations/0044_remove_scopes_bitfield_pending.py`:
same for `apikeyreplica` and `apitokenreplica`.
- Tests: removed direct `scopes=` bitfield use from `test_apitoken.py`,
`test_apiapplication.py` (deletions), and `test_sentry_app_updater.py`.

**Deploy safety**
- The only SQL is `ALTER COLUMN "scopes" DROP NOT NULL` on each table.
`SafeRemoveField` with `MOVE_TO_PENDING` is a no-op in the DB, so the
columns stay in place and old pods that still write `scopes` keep
working during the rollout.
- Prod check from the #127282 thread: 0 rows with `scopes <> 0 AND
cardinality(scope_list) = 0` on `sentry_apitoken`, `sentry_apikey`,
`sentry_apiauthorization`, `sentry_apigrant`, `sentry_sentryapp`.
Replica tables never get `scopes` set; replication only copies
`scope_list`.

**Phase 2 (separate PR, after this deploys):** `SafeRemoveField(...,
deletion_action=DeletionAction.DELETE)` for the same 7 models to
actually drop the columns.

**Verification**
- `makemigrations sentry hybridcloud --check` reports no changes;
`sentry upgrade` applies both migrations.
- Ran these suites on a fresh test DB: models
(apitoken/apigrant/apikey/apiapplication), api token/authorization
endpoints, authentication, oauth authorize/token/device, hybridcloud
replica, backup exhaustive/imports/exports, sentry app creator/updater,
apiapplication deletions. 575 pass. Failures, none touching `scopes`: 3
`test_api_token_details.py::*::test_invalid_user_id`
(superuser/elevated-mode path) and 1 backup test that needs Snuba, which
isn't available in my sandbox.
- prek (ruff, mypy) passes on the changed files.

<!-- junior-request-attribution:start -->
via **shaun.kaasten**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AD0BAS2BU2TC%3A1791913665.641439 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AD0BAS2BU2TC%3A1791913665.641439)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AD0BAS2BU2TC%3A1791913665.641439/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: shaun.kaasten <shaun.kaasten@sentry.io>
RudraPatel2003 pushed a commit that referenced this pull request Oct 19, 2026
Phase 2 of 2 for removing the legacy `scopes` BitField. #127386 (phase
1) removed the field from the models and moved the columns to pending
deletion. This PR drops the columns.

Refs #50475, ENG-3384. Earlier steps: #127282 (removed the
`get_scopes()` fallback), #127386 (phase 1).

**Phase 1 is fully deployed.** #127386 shipped as getsentry `910f30be`,
and GoCD shows it passed on all backend pipelines (S4S2, DE, US,
control, ST) on 2026-10-16. No running pod still writes `scopes`.

**What changed**
- `src/sentry/migrations/1218_drop_scopes_bitfield_columns.py`:
`SafeRemoveField(..., deletion_action=DeletionAction.DELETE)` on
`apiauthorization`, `apigrant`, `apikey`, `apitoken`, `sentryapp`.
-
`src/sentry/hybridcloud/migrations/0045_drop_scopes_bitfield_columns.py`:
same for `apikeyreplica` and `apitokenreplica`.
- `migrations_lockfile.txt`.

Migrations only; no code changes. Rebased on master and renumbered from
`1216` to `1218` (now depends on
`1217_grouprelease_add_new_release_id`), because master added its own
`1216` and `1217`.

**SQL**
```sql
ALTER TABLE "sentry_apiauthorization" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_apigrant" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_apikey" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_apitoken" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "sentry_sentryapp" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "hybridcloud_apikeyreplica" DROP COLUMN "scopes" CASCADE;
ALTER TABLE "hybridcloud_apitokenreplica" DROP COLUMN "scopes" CASCADE;
```

**Verification**
- Before the rebase: `makemigrations sentry hybridcloud --check`
reported no changes; `sentry upgrade` applied both migrations locally;
181 targeted tests passed on a fresh test DB (apitoken, apigrant,
apikey, apiapplication deletions, sentry app updater, hybridcloud
replica, api tokens endpoint, oauth token).
- The rebase changed only the migration number, its dependency, and the
lockfile. CI re-validates the migration graph.
- No raw SQL references to these `scopes` columns in `getsentry/sentry`
or `getsentry/getsentry`.

<!-- junior-request-attribution:start -->
via **shaun.kaasten**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AD0BAS2BU2TC%3A1792158314.758249 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AD0BAS2BU2TC%3A1792158314.758249)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AD0BAS2BU2TC%3A1792158314.758249/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Shaun Kaasten <shaun.kaasten@sentry.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants