Skip to content

Add Dataset Info panel and metadata export - #1688

Merged
BryonLewis merged 5 commits into
mainfrom
dataset-info-panel
Jun 10, 2026
Merged

BryonLewis merged 5 commits into
mainfrom
dataset-info-panel

Conversation

@PaulHax

@PaulHax PaulHax commented Jun 8, 2026 •

Copy link
Copy Markdown
Collaborator

Towards #1585. Associate dataset-level metadata (e.g. gfishsite_id, station info) with a dataset, display it, and carry it out to export so it can be re-linked downstream.

  • Adds a datasetInfo field to the dataset metadata model (server + client).
  • New Dataset Info sidebar panel: manual key/value entry and display, round-trips on web and desktop, with save-error retry.
  • Passes datasetInfo into the VIAME CSV # metadata header as one nested JSON entry.
image

PaulHax added 3 commits June 8, 2026 15:27
Introduce a free-form datasetInfo key/value map on mutable dataset
metadata. Thread it through the shared apispec type and
DatasetMetaMutableKeys, the server MetadataMutable Pydantic model, and
desktop JSON metadata persistence so custom metadata round-trips on both
web and desktop.
Add a sidebar panel that displays core dataset metadata (name, type,
fps, created date, id) and supports editable custom key/value metadata:
inline editing, keyboard-driven entry, expand-to-dialog for long values
via a reusable DatasetMetaEditorDialog, save-error handling with retry
and permission detection, and a read-only mode. Register it in the
context component map.
Thread the dataset's datasetInfo metadata into the VIAME CSV export on both
the server and desktop paths, emitting it as one nested JSON entry on the
'# metadata' comment line when non-empty (omitted entirely when empty, so
existing exports stay byte-unchanged).

Server: read datasetInfo from folder metadata in get_annotation_csv_generator
and thread it into export_tracks_as_csv. The existing key: json.dumps(value)
header formatting serializes the dict, preserving numeric fields.

Desktop: writeHeader reads meta.datasetInfo (a DatasetMetaMutableKeys member
that exportDataset already loads) and pushes a 'datasetInfo: <json>' entry onto
the '# metadata' row, emitting the row when fps OR datasetInfo is present.
Empty-check uses lodash isEmpty. Unknown '# metadata' keys are already ignored
on parse.
@PaulHax
PaulHax requested a review from BryonLewis June 8, 2026 22:15
@BryonLewis

Copy link
Copy Markdown
Collaborator

This looks really good.

I would suggest updating some docs in the ./docs/DataFormats.md to mention that the DIVE Configuration JSON now has a new key 'datasetInfo' that can contain key/value pairs, make a note under the VIAME CSV section that metadata inside the datasetInfo will be included.

Possibily create a new ./docs/UI-Dataset-Info.md file explaining what the interface does, where data comes from and how to edit/modify it

Then update Context sidebar (Web) to have a link to the new ./docs/UI-DatasetInfo.md`

PaulHax added 2 commits June 9, 2026 14:21
Remove the StackedVirtualSidebarContainer wrapper from DatasetInfo. Its
overflow-hidden, max-height card forced a second scroll region inside the
shared SidebarContext .sidebar-content scroller, producing two scrollbars
on long content in short viewports. DatasetInfo never used the wrapper's
stacked-split slot data (enable-slot=false), so rendering directly under
SidebarContext leaves a single scroll region.

Also drop the now-dead width prop and .datasetinfo-content scroll CSS.
- New docs/UI-DatasetInfo.md: panel overview, the datasetInfo folder
  metadata key, the three ways to populate it (panel, pipeline stamp at
  upload, post-hoc script), and CSV export passthrough.
- DataFormats.md: document datasetInfo on DatasetMetaMutable and its
  emission on the VIAME CSV # metadata header line.
- Link the new page from the Context sidebar list, the UI overview, and
  the mkdocs nav.
@PaulHax

PaulHax commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator Author

Added some docs and fixed a double vertical scrollbar issue.

@BryonLewis BryonLewis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding the double scroll bar. I didn't think to add enough data for scrolling.
Looks good.

@BryonLewis
BryonLewis merged commit 13d271d into main Jun 10, 2026
3 checks passed
@BryonLewis
BryonLewis deleted the dataset-info-panel branch June 10, 2026 12:42
PaulHax pushed a commit to PaulHax/dive that referenced this pull request Jun 12, 2026
Design plan for per-frame/per-image metadata (issue Kitware#1585), building on
the dataset info work in Kitware#1688: field registry + sidecar-backed values,
import paths (CSV/JSON sidecar, column mapping, EXIF/KLV), Frame Info
panel and timeline/HUD display, and export integration (VIAME CSV
frm-atr columns, COCO image fields, Croissant).

https://claude.ai/code/session_01QPgk5fyYPuYYMHUX2dBchg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants