Automate GCS bucket creation and split metadata generator from ASCII encoder #347

Activity

1. leekahung commented on May 12, 2026

   

   leekahung

   on May 12, 2026

   ContributorAuthor

   More actions

   @claude can you check what's already implemented in #345 and update the description accordingly?
2. claude commented on May 12, 2026

   

   claudeBot

   on May 12, 2026 – with Claude

   Last edited by claude

   Contributor

   More actions

   Claude finished @leekahung's task in 1m 44s —— View job

   ---

   Task progress

   • Check PR Include annotations for existing statutes and ASCII enforcement #345 state and what was merged/changed
   • Compare against issue Automate GCS bucket creation and split metadata generator from ASCII encoder #347 acceptance criteria
   • Report findings + propose updated description
   • Update issue description (requires repo write — see note below)

   ---

   What's in PR #345 today

   PR #345 is still open (not merged). Inspecting backend/scripts/generate_metadata_jsonl.py and the backend/Makefile target on that branch:

   Acceptance criterion from #347	Status in #345	Notes
   Bucket creation via google-cloud-storage, errors if bucket exists	❌ Not started	Script assumes the bucket already exists and only uses the name to build URIs. No google-cloud-storage import / create_bucket() call.
   Bucket name passed as a CLI flag (not env var)	✅ Done	generate_metadata_jsonl.py already takes --bucket as a required argparse argument. The Makefile target wraps it with GCS_BUCKET_NAME=… for ergonomics but the script's contract is the flag.
   Metadata generator moved into bucket creation flow	❌ Not started	Metadata generation and ASCII enforcement still live together in generate_metadata_jsonl.py. No separate bucket-creation entrypoint exists yet.
   ASCII encoder remains independently runnable	❌ Not split yet	enforce_ascii() runs in-place as part of the metadata script (make generate-metadata auto-converts known offenders). There is no standalone encoder script you can invoke without going through metadata generation.

   So one of the four boxes is effectively already satisfied by #345 once it merges; the other three are still real work for this follow-up.

   Proposed updated issue body

   Since I can't directly edit the issue body from here (the GitHub App permissions on this workflow allow comments but not issue edits — gh issue edit is blocked), here's a drop-in replacement you can paste over the current description:

   Click to expand proposed new body

   Follow-up from PR #345 review thread on `backend/scripts/generate_metadata_jsonl.py`.
   
   ### Context
   
   The current workflow for publishing law documents to Vertex AI RAG involves a manual GCS bucket creation step (see post-deployment notes on #345). During review, @yangm2 suggested two related changes that were deferred to keep #345 scoped to RAG document updates and ASCII enforcement:
   
   1. **Call Google's API to create the bucket programmatically.** Creation should fail if the bucket already exists rather than silently re-using it.
   2. **Split `generate_metadata_jsonl.py`** into:
      - A UTF-8 → ASCII encoder (stays with the ASCII enforcement work)
      - A metadata generator that lives alongside the bucket creation step, since the two are tightly coupled
   
   ### Status of acceptance criteria (after #345)
   
   Already addressed in #345:
   - [x] Bucket name is passed as a CLI flag (`--bucket`), not an env var — the Makefile sets `GCS_BUCKET_NAME` but forwards it to the script via `--bucket`.
   
   Still outstanding for this issue:
   - [ ] Bucket creation goes through `google-cloud-storage` (or equivalent) and errors on existing bucket
   - [ ] Split `generate_metadata_jsonl.py` so the ASCII encoder is a standalone, independently runnable script (currently `enforce_ascii()` only runs as a side-effect of `make generate-metadata`)
   - [ ] Move the metadata generator into the new bucket creation flow so the bucket + metadata steps are a single coupled command
   
   ### Links
   
   - PR: #345 (still open at time of writing)
   - Review thread: https://github.com/codeforpdx/tenantfirstaid/pull/345#discussion_r-bucket-creation (see comments by @yangm2 on `generate_metadata_jsonl.py:323`)

   Note on editing the issue directly

   I'm unable to edit the issue description itself — the gh issue edit command isn't in this action's allowed tool list, so I can only post via this comment. You can find more information and potential workarounds in the FAQ. If you'd like me to retry with broader permissions, re-trigger me with --allowedTools extended to include Bash(gh issue edit:*) (or just paste the body above yourself).
   · branch: claude/issue-347-20260512-0133
3. leekahung

   self-assigned this

   on May 12, 2026
4. leekahung

   closed this as completedin #349on Jun 2, 2026