[shrinivas-sn]

Closes #404

Summary of Changes

1. Ask Rules TK-A052 & TK-A053 (tracekit/policy2/packs/coding.yaml):

   • Added TK-A052 ask rule in coding.yaml (inherited by dev, server, strict packs) holding:
     • kubectl or oc get secret/secrets with output formatting that prints values (-o yaml, -o json, -o jsonpath, -o go-template, -o template).
     • kubectl view-secret (also oc view-secret and kubectl-view-secret).
     • kubectl create token (also oc create token).
   • Added TK-A053 ask rule in coding.yaml holding:
     • vault kv get
     • vault read (reading secrets across any engine)
     • vault token create
   • Allowed lookalikes stay permitted:
     • kubectl get secrets -n default
     • kubectl get namespaces
     • kubectl get pods -n kube-system
     • kubectl describe serviceaccount deployer -n prod
     • kubectl auth can-i create deployments --as=developer
     • kubectl describe secret db -n prod
     • kubectl cluster-info
     • kubectl get secret db -o wide
     • kubectl get secret db -o name
     • vault secrets list
     • vault policy list
     • vault token lookup
     • vault kv list secret/
     • vault kv metadata get secret/app
     • vault status

2. Test Coverage:

   • Added 14 test cases to tests/data/policy/deny/shell.jsonl (expect: ask, with kubectl create token marked deny as server.yaml extends TK-C001).
   • Added 11 lookalike test cases to tests/data/policy/benign/shell.jsonl.
   • Added unit test test_reading_kubernetes_and_vault_secrets_asks_in_every_pack in tests/test_signer_policy.py verifying held and benign calls across all packs (coding.yaml, dev.yaml, server.yaml, strict.yaml) and engines.

3. Documentation & Evaluation:

   • Added TK-A052 and TK-A053 to docs/policy-v2.md coding pack table.
   • Added row to docs/limits.md covering uninstrumented direct API calls and in-process Vault client libraries.
   • Re-recorded E17 corpus (eval/e17_policy_corpus.py --record):
     • Deny recall: 100.0% (792/792 stopped).
     • Benign false positive rate: 0.12% (1 in 854).
     • Engine mismatches: 0.
   • Updated evaluation numbers in README.md and docs/evaluation.md.
   • In E21 blind pool (feat: add E21 blind evaluation pool for amandeavor (fixes #368) #401, eval/e21_blind/amandeavor.jsonl), harmful calls held increased from 30 of 45 (66.7%) to 34 of 45 (75.6%), with 0 false positives.

Verification

• tracekit policy lint across all 9 packs: Clean
• pytest tests/test_signer_policy.py -k test_reading_kubernetes_and_vault_secrets_asks_in_every_pack: Passed
• pytest tests/test_policy2.py -k test_every_pack_pattern_searches_64_kib_within_budget: Passed (0 slow fillers)
• pytest tests/test_eval_docs.py tests/test_docs.py: Passed (10/10)
• pyflakes tracekit tracekit_sdk.py tests eval: Clean

[LoopGlitch26]

Review of this one. The cases are thorough: options before get, -o before the type, and -o name/-o wide left alone. Updating the old A035 row and noticing that TK-C001 already denies kubectl create token was useful. It merges after #543. To get there, please:

1. Merge main and bump the pack version. 1.5.3 review: TK-M003 spellings, tool-name spellings two packs share, MCP bare names, F016/F017 case, A048 options #542 changed coding.yaml (now 2026.10-v2-51), docs/policy-v2.md and the E17 results, so the branch conflicts. Every rule change bumps version.
2. Re-record E21 as well as E17. The new deny cases change eval/results/e21_robustness.json and e21_policy_robustness.json, and CI's offline evals job compares both (the same e21_* lines in ci.yml). The description says the amandeavor pool goes from 30 to 34 of 45, but no results file says so yet. Update the README and docs/evaluation.md numbers that tests/test_eval_docs.py checks.
3. Fix two false alarms in TK-A052, with tests that they're allowed:
   • (?: [^ ]+){0,16}? secrets?\b matches any later word that starts with secret. So kubectl get -o yaml clusterrole secret-reader (the Kubernetes docs' example role) and kubectl get -o json pods -n secrets-store are held. End the type word at a space, /, , or the end, not at -.
   • -n is read as an option without a value, so kubectl get -n secrets deploy -o yaml is held. Give -n/--namespace its value.
4. Hold these spellings that print values, or name them in the limits row:
   • -o custom-columns=PW:.data.password;
   • --template={{.data.password}} on its own;
   • oc extract secret/db --to=-;
   • a type that isn't first in a list (cm,secrets);
   • Secret in another case (TK-C010 handles case);
   • vault print token, vault write auth/token/create and vault unwrap.
5. Fix the edited A035 row. It lists 1password, which isn't a command (the CLI is op).
6. Run make check before ticking it. The description lists three filtered test runs. Say in the PR what you ran.

When CI is green and a second review passes, I'll merge it.

[LoopGlitch26]

Closing this one in favour of #582, which carries your TK-A052 and TK-A053 cases together with the review's changes, and credits you as co-author. Your TK-C010 work lands in #543. Thanks for the cases.