Skip to content

JAMES-4231 Add S3 blob range reads - #3244

Open
HesandaLiyanage wants to merge 3 commits into
apache:masterfrom
HesandaLiyanage:JAMES-4231-range-reads
Open

HesandaLiyanage wants to merge 3 commits into
apache:masterfrom
HesandaLiyanage:JAMES-4231-range-reads

Conversation

@HesandaLiyanage

Copy link
Copy Markdown
Contributor

JAMES-4231 — Add S3 blob range reads

Scope

Adds the generic BlobStoreDAO.readRange contract and an S3 implementation that uses a single HTTP Range request. Range results retain the source blob metadata and expose the full object size in metadata, which lets later chunk readers interpret slot data safely.

Deliberately excluded

  • ChunkMarker and BloomFilter GC handling
  • BlobIdUpdater and backend metadata updates
  • Compaction format/algorithm and property-based tests
  • Guice wiring, WebAdmin routes, and user documentation

Design notes

  • Range bounds are inclusive. end = -1 reads through the object end; a negative start requests a suffix range.
  • Invalid ranges and S3 416 responses fail explicitly; no empty blob fallback is used.
  • S3 uses the existing request builder, so SSE-C headers and fallback namespace handling are retained for range reads.

Tests

Added a reusable BlobStoreDAORangeReadContract and a MinIO/Testcontainers S3 implementation test covering prefix, middle, suffix, open-ended, invalid-range, metadata, and total-object-size behavior.

Tests have not been run locally.

Follow-up

JAMES-4231 — ChunkMarker and BloomFilter GC exclusion for virtual chunk addresses.

Retriggers the PR build after an OpenSearch testcontainer
infrastructure failure unrelated to the change set.

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant