Export Metadata in Standardized Format - P4

@csbrown-noaa

Is your feature request related to a problem? If so, Please describe.
Similar to priority 3, we also need a standardized format for our metadata or any metadata at all. Currently, we have to name our file the sitename in order to link our annotations to our database using a script in post. We need a better way to associate the metadata with our predictions without having to do so much legwork on the front and back end.

Describe the solution you'd like
Ability to define and export dataset metadata in a single agreed-upon common standard (e.g. Croissant).
Ability to associate metadata data with an image set (e.g. cruise ID, location and dive number) and display that info in the UI. Ability to add a comment field/other attributes. It would be helpful to have options to have this information displayed while annotating and also in the csv export files.
Just as an example but in Abi's current software (Red fish Blue fish) the following information is displayed with each image pair: image file name, timestamp, location (lat/long), water depth, AUV altitude off the substrate, image area. I'm not suggesting that you include all of these because this is a universal software program, but I am just giving you an example or maybe it could be customizable depending on what attributed you have in the first four columns of your metadata or something?

Describe alternatives you've considered
Setting aside the UI work if that becomes too much to include. We really just need to link the metadata to the predictions.

Additional context
Please work with Scott (@csbrown-noaa) on the common standard.

I think there are two different things here.
Global Dataset Metadata - Metadata about the Video or Image Sequence like location and other data
Then per-image or per-frame metadata.

I have another verison of OpenSource DIVE that is used for more medical-based visualization this has something called the DatasetInfo context view (a right side panel) that will display metadata and attribute/track attributes in the window.
https://digitalslidearchive.github.io/dive-dsa/UI-DatasetInfo/

I can import similiar functionality to this into DIVE and enhance it so we can have per-frame/image-sequence information as well.

adding this link as well: https://github.com/mlcommons/croissant

Add editable custom metadata to Dataset Info panel

Some updated notes about this issue:

Implement the UI Dataset Info tool from DIVE-DSA into this system
- Note this reads metadata info directly from the Folder Metadata with a key of datasetInfo
Maybe add a separate field for folder Metadata with a key of userInfo where it will create comments/text that can be tied to the user who made those changes.
- Using the username of the user there is a dictionary of arrays of comments and a user can edit/modify their own comments or look at comments from other users
Look into supporting the Croissant format. Maybe that is a separate metadata key on the DIVE Dataset

A separate thing is the per-frame metadata info. This would probably be separate task/PR.
One basic idea is to create a new Mongo datamodel for the mediaMetadata that can be stored alongside a video/image sequence. Then having a sidebar info panel that could read this in from some endpoint like /dive_dataset/{folderId}/media_metadata, then it would store that locally and remain in-sync with the current frame. This is similar to a detection attribute but will be attached directly to the media instead of a track on the system.

Sounds great Bryon. Since we're getting ready to implement our cloud infrastructure that will run everything in batch this will make things infinitely easier to link annotations to our database without having to rename all of our files. Please let me know if you need anything from me or have any questions.

@kelseylmartin

@kelseylmartin If you could provide a sample of your current structure/metadata, that would be really helpful.

The metadata will look something like this (attached). All we really need is the gfishsite_id column and then we can link it to all of our other files. I know Abi Powell might need other metadata included but the file structure should be the same.

metadata_06012026.csv

Add Dataset Info sidebar panel

Add editable custom metadata to Dataset Info panel

Widen datasetInfo metadata value type to allow non-string values

Hello there 🐣

Update on the dataset-metadata side of this (#1688, in review): you can now attach dataset-level metadata to an image set/video and it carries through to CSV export, so annotations could link back to your database by gfishsite_id without renaming files if that metadata is attached.

Adding the metadata. Two ways, both writing the same datasetInfo folder-metadata key:

With a script (best for batch): stamp it from your pipeline when the dataset is created, or run the same call later against datasets already in DIVE, driven from your metadata CSV:
```
gc.addMetadataToFolder(folder_id, {"datasetInfo": {"gfishsite_id": "2024TXN012", "cruise": "2403", "year": "2024"}})
```
(equivalently PUT /folder/{id}/metadata). This could fit the cloud/batch workflow: whatever owns the gfishsite_id stamps each dataset, and it flows through to export.
By hand: the new Dataset Info sidebar panel, for one-offs/corrections.

Getting it back out. On VIAME CSV export the whole datasetInfo object is written into the # metadata header line as one JSON entry, so gfishsite_id is right there for your re-link step.

Docs are in the PR: UI-DatasetInfo.md (the panel plus how to populate it) and the VIAME CSV section of DataFormats.md.

Question: Do you think this approach will help you assosiate gfishsite_id with predictions?

On the "standardized format" (Croissant) ask. We could emit a croissant.json sidecar so generic tools (HuggingFace, Kaggle, ...) can read the export without a NOAA-specific reader. The payoff is the typed annotation table (track id, frame, bbox, species, confidence, length), which DIVE owns the schema for. Your arbitrary datasetInfo keys are the catch: DIVE doesn't know them upfront, so they stay untyped text (no better than the CSV) unless you also declare their types alongside the values (simplest: store real numbers instead of strings, or stamp a small type schema next to them).

So the question: do you need these exports readable by standardized ML tooling (worth it mainly if you plan to publish or share the datasets), or is the CSV enough because only your own pipeline consumes them?

We are working up some ideas to support per frame metadata, stay tuned.

Add per-frame metadata design proposal

@csbrown-noaa

Hola @csbrown-noaa

DIVE now writes gfishsite_id into an exported VIAME CSV when that key lives in the datasetInfo folder metadata. That raises a question on the input and output side:

Input: Is it feasible for y'all to put gfishsite_id into the folder metadata, either from your batch pipeline (a one-line Girder call, ideally from whatever already owns the id at upload) or by hand in the new DIVE UI panel?

Output: Do you consume the VIAME CSV downstream, or a different export (DIVE JSON, COCO)? Today only the CSV carries the key, so knowing which format you actually read tells us whether the header is enough or we need to carry it elsewhere.

@csbrown-noaa

Hi Paul,

Input: I don't see any problem incorporating gfishsite_id into the folder metadata. Part of our pipeline is already extracting the public URLs from each data file in the GCP bucket. We can also include a step that ties each folder to it's respective metadata. Do you have any additional thoughts on this @csbrown-noaa?

Output: Moving forward, our standard output will be the DIVE JSON (KWCOCO).

Good info. #1704 will include datasetInfo in COCO JSON exports. Then you can do something like this to pass a gfishsite_id through DIVE to a COCO JSON export:

# 1. Stamp the site id onto the dataset folder (Girder's native metadata API)
gc.addMetadataToFolder(folder_id, {"datasetInfo": {"gfishsite_id": "2024TXN012"}})

# 2. Export KWCOCO — datasetInfo now rides along in the COCO `info` block
coco = gc.sendRestRequest("GET", "dive_annotation/export",
                          parameters={"folderId": folder_id, "format": "coco_json"})

# 3. Read the site id back out to join against your DB
coco["info"]["datasetInfo"]["gfishsite_id"]   # -> "2024TXN012"

Heya. I'm working up detailed proposal for per-frame metadata here #1725

I'm wondering if the design will support NOAAs workflow?

Current idea for how to load per frame metadata:

Make a delimited text file (.txt/.csv) with a header row and one row per image.
Give it a column of image file names — that column is what joins each row to its frame.
Drop the file next to the imagery (or, for multicam, at the parent root or in each camera folder).

This would pickup a .txt file like this:

image_file timestamp latitude longitude depth altitude
img_0001.tif 15:40:56 46.5759 -124.6031 192.8 2.78
img_0002.tif 15:41:04 46.5759 -124.6031 193.1 2.70

Followup up work could support EXIF metadata or KLV streams in videos.

@csbrown-noaa

Hi Paul. I could definitely see this being useful for roving deployments. I know one of our users was also interested in adding comments to a frame. Would this be feasible as well?

@csbrown-noaa how do you think this could be integrated into the GCS infrastructure?

Hi Kelsey. Whole frame comments/classifications would be nice. I consitered an approach that would have the "frame metadata" ride the DIVE Tracks code/data-model. Decided against that because it would lead to if track != is_frame_metadata checks in many of the places. I suggest we treat this frame metadata data model as an extra input channel assosated with media. For frame annotations and the whole frame classifcation use case, let us create a seperate issue (which would probably get implimented with DIVE Tracks.)

@kelseylmartin

Heyo @kelseylmartin and @csbrown-noaa,

We are working on per-frame metadata support in this PR stack. Do you think this approach will work for y'all?

Ingestion

Attach a metadata file to the dataset via the upload dialog
CSV and TXT files can be read as frame metadata
For Girder asset-store imports from cloud buckets, use one of three discovery paths:
- Use a reserved filename such as frame-metadata.* or frame_metadata.*
- For a video in a flat layout, use an asset-store paired sidecar: <video-stem>-metadata.* or <video-stem>_metadata.*
- Have a post-process set the Girder folder-metadata attachment pointer to the intended file

Joining rows to frames

Image datasets match rows by filename, DIVE frame number, or source image counter
Video datasets use a literal, zero-based frame number

Displayed in side panel and consumed by pipelines

Matched frame metadata rows appear in the Frame Metadata section of the Dataset Info side panel
The attachment remains available to downstream pipelines

Hi Paul,

Thanks for the update. This should work for us although I anticipate some tweaking might need to be made for other groups that have multiple images per site.

Navigation Menu

Repository navigation

Export Metadata in Standardized Format - P4 #1585

Ingestion

Joining rows to frames

Displayed in side panel and consumed by pipelines

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Export Metadata in Standardized Format - P4 #1585

Description

Activity

BryonLewis commented on Apr 22, 2026

BryonLewis commented on Apr 22, 2026

BryonLewis commented on Jun 1, 2026

kelseylmartin commented on Jun 1, 2026

BryonLewis commented on Jun 1, 2026

kelseylmartin commented on Jun 1, 2026

PaulHax commented on Jun 9, 2026

PaulHax commented on Jun 16, 2026

kelseylmartin commented on Jun 16, 2026

PaulHax commented on Jun 17, 2026

PaulHax commented on Jun 29, 2026

kelseylmartin commented on Jul 1, 2026

PaulHax commented on Jul 1, 2026

PaulHax commented on Aug 10, 2026

Ingestion

Joining rows to frames

Displayed in side panel and consumed by pipelines

kelseylmartin commented on Aug 15, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions