Repository navigation
FEAT: Change saved registry entries through the API (part 21) - #4072
Open
varunj-msft wants to merge 671 commits into
Open
varunj-msft wants to merge 671 commits into
varunj-msft wants to merge 671 commits into
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved issues expose server paths during deletion failures, block recovery after editing unrestored entries, and lose keyboard focus after target or scorer deletion.
5 open issues
What changed in this PR
This PR extends PyRIT’s saved registry entries with version-checked API editing, building on the persistence and deletion work in #4071. Registry-tab editing remains out of scope.
Changes:
- Adds saved-entry edit request and response models, ownership metadata, and management-route integration.
- Carries forward persistence safeguards, restore recovery, saved-entry deletion controls, and version-checked preset and suite deletes.
- Expands regression coverage, isolates test storage, and documents saved-entry behavior.
| File | Description |
|---|---|
| tests/unit/setup/test_targets_initializer.py | Tests saved-target collision protection. |
| tests/unit/setup/test_scorer_initializer.py | Tests saved-scorer tagging protection. |
| tests/unit/setup/test_converter_initializer.py | Tests saved-converter collision protection. |
| tests/unit/setup/test_configuration_loader.py | Tests recipe storage configuration. |
| tests/unit/registry/test_scenario_suite_storage.py | Tests stale-version delete rejection. |
| tests/unit/registry/test_scenario_suite_registry.py | Tests version-checked suite deletion. |
| tests/unit/registry/test_scenario_preset_registry.py | Tests version-checked preset deletion. |
| tests/unit/registry/test_instance_registry.py | Tests saved-entry protection and origin metadata. |
| tests/unit/registry/test_converter_registry.py | Tests saved-name collision recording. |
| tests/unit/registry/test_attack_technique_registry.py | Tests saved-technique collision handling. |
| tests/unit/backend/test_technique_edit_service.py | Updates calls to keyword-only creation API. |
| tests/unit/backend/test_scenario_suite_service.py | Tests delete-version forwarding. |
| tests/unit/backend/test_scenario_suite_routes.py | Tests conflicts and sanitized storage errors. |
| tests/unit/backend/test_scenario_suite_execution_service.py | Tests saved-entry launch conflict messages. |
| tests/unit/backend/test_scenario_preset_service.py | Tests delete-version forwarding. |
| tests/unit/backend/test_scenario_preset_routes.py | Tests conflicts and sanitized storage errors. |
| tests/unit/backend/test_saved_entries.py | Tests temporary-reference save restrictions. |
| tests/unit/backend/test_mappers.py | Tests origin and ownership response mapping. |
| tests/unit/backend/test_initializer_service.py | Tests sanitized filesystem failures. |
| tests/unit/backend/test_auth_middleware.py | Tests authenticated object-ID lookup. |
| tests/unit/backend/test_analytics_benchmark.py | Tests benchmark recipe-storage isolation. |
| tests/unit/backend/conftest.py | Isolates saved-entry storage in backend tests. |
| tests/end_to_end/conftest.py | Gives launched backends temporary recipe storage. |
| test_support/analytics_benchmark.py | Configures isolated benchmark recipe storage. |
| pyrit/setup/initializers/tools.py | Rejects saved-entry collisions during tool setup. |
| pyrit/setup/initializers/targets.py | Rejects saved-target group collisions. |
| pyrit/setup/initializers/scorers.py | Prevents startup tags targeting saved replacements. |
| pyrit/setup/configuration_loader.py | Adds and validates recipe storage configuration. |
| pyrit/score/true_false/local_refusal_classifier_scorer.py | Declares model-backed save restrictions. |
| pyrit/score/true_false/audio_true_false_scorer.py | Declares audio scorer save restrictions. |
| pyrit/score/scorer.py | Defines scorer persistence-policy metadata. |
| pyrit/score/float_scale/azure_content_filter_scorer.py | Declares server credential fallback. |
| pyrit/score/float_scale/audio_float_scale_scorer.py | Declares audio scorer save restrictions. |
| pyrit/registry/scenario_suite_storage.py | Supports version-checked suite deletion. |
| pyrit/registry/scenario_preset_storage.py | Supports version-checked preset deletion. |
| pyrit/registry/components/scorer_registry.py | Reserves the scorer types route name. |
| pyrit/registry/components/scenario_suite_registry.py | Forwards suite deletion versions. |
| pyrit/registry/components/scenario_preset_registry.py | Forwards preset deletion versions. |
| pyrit/prompt_target/websocket_copilot_target.py | Declares account-dependent save restriction. |
| pyrit/prompt_target/prompt_shield_target.py | Declares credential and destination policies. |
| pyrit/prompt_target/openai/openai_target.py | Declares credential, header, and destination policies. |
| pyrit/prompt_target/litellm_chat_target.py | Declares API-key fallback policy. |
| pyrit/prompt_target/hugging_face/hugging_face_chat_target.py | Declares token and model-storage restrictions. |
| pyrit/prompt_target/http_target/http_target.py | Restricts saving raw HTTP requests. |
| pyrit/prompt_target/hack_a_prompt_target.py | Declares session and cookie fallback policies. |
| pyrit/prompt_target/github_copilot_target.py | Declares GitHub token fallback policy. |
| pyrit/prompt_target/common/prompt_target.py | Defines target persistence-policy metadata. |
| pyrit/prompt_target/azure_ml_chat_target.py | Declares credential and destination policies. |
| pyrit/prompt_target/azure_blob_storage_target.py | Declares token and container-address policies. |
| pyrit/prompt_target/a2a_target.py | Declares endpoint and agent-card destinations. |
| pyrit/models/catalog/target.py | Adds target origin and ownership fields. |
| pyrit/models/catalog/scorer.py | Adds scorer origin and ownership fields. |
| pyrit/models/catalog/__init__.py | Exports recipe and restore models lazily. |
| pyrit/converter/converter.py | Defines credential fallback metadata. |
| pyrit/converter/azure_speech_text_to_audio_converter.py | Declares Speech credential fallback. |
| pyrit/converter/azure_speech_audio_to_text_converter.py | Declares Speech credential fallback. |
| pyrit/backend/services/scenario_suite_service.py | Forwards suite deletion versions. |
| pyrit/backend/services/scenario_suite_execution_service.py | Exposes safe saved-entry conflict messages. |
| pyrit/backend/services/scenario_run_service.py | Reports launch-script saved-entry collisions. |
| pyrit/backend/services/scenario_preset_service.py | Forwards preset deletion versions. |
| pyrit/backend/routes/techniques.py | Adds persistence errors and caller ownership context. |
| pyrit/backend/routes/targets.py | Adds persistence errors and caller ownership context. |
| pyrit/backend/routes/scorers.py | Adds persistence errors and caller ownership context. |
| pyrit/backend/routes/scenarios.py | Documents saved-entry launch conflicts. |
| pyrit/backend/routes/scenario_suites.py | Adds conditional deletes and sanitized storage logging. |
| pyrit/backend/routes/scenario_presets.py | Adds conditional deletes and sanitized storage logging. |
| pyrit/backend/routes/restore_failures.py | Exposes restore failures and Retry. |
| pyrit/backend/routes/initializers.py | Sanitizes filesystem failure responses and logs. |
| pyrit/backend/routes/common.py | Shares ownership and save-refusal response helpers. |
| pyrit/backend/models/techniques.py | Adds persistence validation and ownership fields. |
| pyrit/backend/models/targets.py | Adds persistence validation and saveability metadata. |
| pyrit/backend/models/scorers.py | Adds scorer persistence validation. |
| pyrit/backend/models/scenario_suites.py | Documents delete-version usage. |
| pyrit/backend/models/scenario_presets.py | Documents delete-version usage. |
| pyrit/backend/models/saved_entries.py | Defines saved-entry inspection and editing contracts. |
| pyrit/backend/models/converters.py | Adds persistence validation and ownership fields. |
| pyrit/backend/middleware/runtime.py | Allows recovery and saved-entry management routes. |
| pyrit/backend/middleware/auth.py | Exposes the authenticated user’s object ID. |
| pyrit/backend/mappers/technique_mappers.py | Maps technique origin and ownership. |
| pyrit/backend/mappers/target_mappers.py | Maps target origin and ownership. |
| pyrit/backend/mappers/converter_mappers.py | Maps converter origin and ownership. |
| pyrit/backend/main.py | Registers recovery and saved-entry routers. |
| frontend/src/utils/registryOrigin.ts | Provides origin labels and ownership captions. |
| frontend/src/utils/registryName.ts | Generates valid, available registry names. |
| frontend/src/utils/registryName.test.ts | Tests name validation and suggestions. |
| frontend/src/test-utils/targetFixtures.ts | Supports origin and ownership fixtures. |
| frontend/src/services/errors.ts | Recognizes typed save refusals. |
| frontend/src/services/errors.test.ts | Tests save-refusal classification. |
| frontend/src/services/api.ts | Adds recovery, saved-entry, and conditional-delete calls. |
| frontend/src/services/api.test.ts | Tests delete-version request parameters. |
| frontend/src/hooks/useRuntime.styles.ts | Styles restore failure lists and actions. |
| frontend/src/components/Techniques/TechniqueEditor.test.tsx | Updates converter save-button expectations. |
| frontend/src/components/ScenarioSuites/ScenarioSuiteLibrary.tsx | Handles stale deletes and restores focus. |
| frontend/src/components/ScenarioSuites/ScenarioSuiteLibrary.test.tsx | Tests delete conflicts and focus recovery. |
| frontend/src/components/ScenarioPresets/ScenarioPresetLibrary.tsx | Handles stale deletes and restores focus. |
| frontend/src/components/ScenarioPresets/ScenarioPresetLibrary.test.tsx | Tests conditional deletes and focus recovery. |
| frontend/src/components/Registry/TechniqueRegistry.tsx | Adds origin, ownership filtering, and deletion. |
| frontend/src/components/Registry/ScorerRegistry.tsx | Adds origin, ownership filtering, and deletion. |
| frontend/src/components/Registry/ScorerRegistry.test.tsx | Tests ownership filtering and saved-row actions. |
| frontend/src/components/Registry/ScenarioPresetRegistry.test.tsx | Expects version-checked preset deletion. |
| frontend/src/components/Registry/SavedBy.tsx | Renders saved-entry ownership captions. |
| frontend/src/components/Registry/SavedBy.styles.ts | Styles ownership captions. |
| frontend/src/components/Registry/OriginBadge.tsx | Renders registry origin badges. |
| frontend/src/components/Registry/DeleteSavedEntryButton.styles.ts | Styles saved-entry deletion dialog content. |
| frontend/src/components/Registry/ConverterRegistry.tsx | Adds ownership filtering and saved-entry deletion. |
| frontend/src/components/Registry/ConverterRegistry.test.tsx | Tests origin, ownership, and deletion behavior. |
| frontend/src/components/Converters/SaveConverterDialog.tsx | Clarifies that converter copies remain temporary. |
| frontend/src/components/Converters/SaveConverterDialog.test.tsx | Tests temporary-copy messaging. |
| frontend/src/components/Config/TargetTable.tsx | Adds origin, ownership filtering, and saved deletion. |
| frontend/src/components/Config/TargetTable.styles.ts | Styles ownership captions and filter controls. |
| frontend/src/components/Config/TargetConfig.tsx | Refreshes targets after saved-entry deletion. |
| frontend/src/App.tsx | Moves the runtime banner inside the theme provider. |
| frontend/playwright.config.ts | Isolates seeded backend recipe storage. |
| frontend/e2e/fixtures/scenario_dataset_backend.py | Forwards delete versions through storage guards. |
| frontend/e2e/fixtures/manual_send_backend.py | Uses temporary saved-entry storage. |
| frontend/e2e/config.spec.ts | Tests target saving and ownership filtering. |
| frontend/e2e/_targets.ts | Adds origin and ownership mock fields. |
| doc/gui/0_gui.md | Documents saved entries, ownership, and deletion. |
| doc/getting_started/pyrit_conf.md | Documents recipe storage and restore recovery. |
| .pyrit_conf_example | Adds recipe storage configuration guidance. |
Review effort: Balanced
| status_code=status.HTTP_409_CONFLICT, | ||
| detail=f"Scenario preset '{name}' changed since it was read; re-read it and retry", | ||
| ) from None | ||
| except (AzureError, TimeoutError) as exc: |
Contributor
Author
| status_code=status.HTTP_409_CONFLICT, | ||
| detail=f"Scenario suite '{name}' changed since it was read; re-read it and retry", | ||
| ) from None | ||
| except (AzureError, TimeoutError) as exc: |
Contributor
Author
varunj-msft
requested review from
hannahwestra25 and
Richard Lundeen (richlundeen)
November 8, 2026 23:56
This was referenced Nov 9, 2026
varunj-msft
changed the title
Saved-entry storage failures now pass their traceback through redact_urls, and code scanning flags the scheme pattern as slow on a long run of letters. A scheme is short, so bounding it keeps the match linear.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This is one of the changes for saving Registry entries across restarts. Until the earlier PRs merge, the diff here also shows their changes.
Fixing a saved entry, including one that failed to restore, meant deleting it and saving it again under the same name.
GET /api/saved-entries/{kind}/{name}now returns the storedrecipeto a caller who may change the entry.PUT /api/saved-entries/{kind}/{name}takes it back, changed, with the version it read asexpected_version.persist. The name, the class or attack type, and who saved the entry stay the same, and the entry stays saved (422 otherwise).still_using_old. The old entry's resources are released only when nothing holds it and no work is in progress.runningis false); the next apply or restart restores it, and the backend becomes ready once every saved entry restores. A target'sauth_modecan change along with itsparams.What stays the same: creating, deleting and restoring work as before, and settings changed for one run in the Scanner are still never saved.
Not covered: