Skip to content

Import RLE segmentation masks as outlines - #1868

Merged
BryonLewis merged 8 commits into
mainfrom
dev/coco-rle-masks
Oct 6, 2026
Merged

BryonLewis merged 8 commits into
mainfrom
dev/coco-rle-masks

Conversation

@mattdawkins

Copy link
Copy Markdown
Member

COCO RLE masks were skipped on import with a warning, so masks from kwcoco and pycocotools — where RLE is the default mask encoding — were lost.

DIVE features store GeoJSON geometry, not rasters (rleMask exists only on the transient segmentation-service response, not the persisted schema), so a decoded mask is traced to its outline and imported as a polygon, the same shape a polygon segmentation takes today.

  • _decode_rle_counts reads both COCO spellings: the uncompressed list of run lengths, and the LEB128 string pycocotools writes
  • _rle_polygon_coords rebuilds the mask column-major (the COCO order), traces it with cv2.findContours, and returns image-space contours, largest first
  • An RLE mask can now supply bounds on its own, so an annotation carrying only a mask no longer needs a bbox
  • Undecodable masks are unchanged: skipped, with the warning; run lengths that do not fill the declared size are rejected rather than guessed at, and a mask above 64M pixels is refused rather than allocated
  • Holes are dropped — RETR_EXTERNAL — since a DIVE polygon cannot express them

The LEB128 decoder is checked against pycocotools' own rleToString algorithm reimplemented in the test, so it is verified against real encoder output rather than a fixture I wrote to match my decoder. Both spellings decode the same 6x6 square to x 3..8, y 2..7.

Server/web import only. The desktop importer still skips RLE and shows the warning: it has no opencv, so contour tracing there means hand-writing a tracer in TypeScript, which is a bigger change and not something I can test in this checkout. Flagged in the docs so the asymmetry is visible.

One pre-existing test changed: test_import_missing_bbox_raises_descriptive_error asserted the old error wording. The behaviour is identical — an undecodable mask with no bbox still raises — only the message now says an RLE mask supplies bounds when it can be decoded.

46 server tests pass locally. Not run: vitest, eslint, tsc — no node_modules here, though this PR touches no client code.

DIVE stores geometry rather than rasters, so a decoded mask becomes its
contour. Both counts spellings are read, the list form and the LEB128
string pycocotools writes; an undecodable mask still warns.
opencv is only a dev dependency, so the deployed server has numpy alone.
Moore-neighbour tracing walks the boundary instead, which costs the
perimeter rather than the area.
@BryonLewis

BryonLewis commented Aug 24, 2026 •

Copy link
Copy Markdown
Collaborator

Make a new issue for this to possibly revisit where the processing of input data comes in. It may be necessary to move all import processing into a task instead of being completed in the request thread.

@BryonLewis
BryonLewis self-requested a review October 6, 2026 14:54

@BryonLewis BryonLewis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This converts RLE encoded masks in coco json format into polygons that DIVE can display.

I've changed this so that instead of processing in the request thread it is now processed in a the conversion/transcoding task. I did this because for large datasets with thousands of Annotations/Masks this could become slow. Putting it in the async task reduces load on the server.

I also added the capability for the desktop to process Masks. This can be done differently because it's the electron app.

@BryonLewis
BryonLewis merged commit 8b23829 into main Oct 6, 2026
3 checks passed
@BryonLewis
BryonLewis deleted the dev/coco-rle-masks branch October 6, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants