SignDATA: Data Pipeline for Sign Language Translation

   

A config-driven, modular pipeline for preprocessing multiple Sign Language datasets. Supports multiple extractors including MediaPipe Holistic, MMPose, MMDet, and YOLO. Supports two pipeline modes including Pose Landmarks and Video Clips.

---

Key Features

• Config-Driven — YAML job configs, experiment configs, and CLI overrides
• Multiple Extractors — MediaPipe Holistic, MMPose, MMDet, and YOLO
• Two Pipeline Modes — pose (landmarks) and video (clip extraction)
• WebDataset Output — sharded tar archives for efficient training data loading

---

Supported Datasets

Dataset	Venue	Description	License
YouTube-ASL	NeurIPS 2023	11,000+ videos, 73,000+ segments -- open-domain ASL-English parallel corpus	Apache-2.0
OpenASL	EMNLP 2022	Open-domain ASL-English translation dataset with official TSV and bbox metadata	CC BY-NC-ND 4.0
How2Sign	CVPR 2021	80+ hours of instructional ASL in a controlled studio environment	CC BY-NC 4.0
BOBSL	ICCV 2021	Broadcast subtitle-aligned BSL corpus with continuous subtitle segments and isolated-sign annotations	Dataset access required
BSL-1K	arXiv 2020	Compatibility lexical view over the public BOBSL release for isolated-sign style preprocessing	Follows BOBSL release terms
WLASL	WACV 2020	12,000+ isolated sign clips across 2,000 ASL glosses	Dataset site
MS-ASL	CVPR 2019	Large-scale isolated ASL dataset with signer-diverse lexical clips	Microsoft Download Center terms
AUTSL	ICCV 2021 challenge	Turkish Sign Language isolated-sign benchmark with RGB/depth clips and signer-independent train/val/test splits	Dataset access required
CSL	USTC release 2015	100 continuous Chinese sign sentences from 50 signers; RGB/depth/skeleton release with paper-aligned Split I / Split II evaluation	CSL release agreement
LSA64	CACIC 2016	3,200 isolated Argentinian Sign Language clips across 64 glosses	CC BY-NC-SA 4.0
SLoVo	ICCVS 2023	Russian Sign Language isolated-sign clips with official annotations.csv	Dataset license
RWTH-PHOENIX-Weather	CVIU 2015	German Sign Language weather corpus distributed as image sequences and annotations	Dataset access required

For paper-aligned preprocessing methodology, see Research-Aligned Preprocessing.

---

Installation

git clone https://github.com/balaboom123/signdata-slt.git
cd signdata-slt
python -m venv venv
source venv/bin/activate  # Linux/macOS — use venv\Scripts\activate on Windows
pip install -r requirements.txt

Optional: GPU-based Extractors (MMPose, MMDet)

MediaPipe and the shipped YOLO jobs work on CPU after the base install. MMPose and MMDet require a CUDA-capable GPU and additional dependencies -- see the Installation Guide for setup instructions.

---

Quick Start

# YouTube-ASL: download, extract MediaPipe landmarks, normalize, package
python -m signdata run configs/jobs/youtube_asl/mediapipe.yaml

# How2Sign: extract MMPose landmarks (CUDA required)
python -m signdata run configs/jobs/how2sign/mmpose.yaml

# BOBSL: validate local release, build subtitle-aligned manifest, extract MediaPipe landmarks
python -m signdata run configs/jobs/bobsl/mediapipe.yaml

# BSL-1K compatibility view: build isolated-sign manifest from the BOBSL release
python -m signdata run configs/jobs/bsl1k/mediapipe.yaml

# MS-ASL: validate local clips, extract MediaPipe landmarks, normalize, package
python -m signdata run configs/jobs/msasl/mediapipe.yaml

# AUTSL: validate local release, extract MediaPipe landmarks, normalize, package
python -m signdata run configs/jobs/autsl/mediapipe.yaml

# CSL: validate/materialize the continuous release, extract MediaPipe landmarks, normalize, package
python -m signdata run configs/jobs/csl/mediapipe.yaml

# LSA64: validate local clips, extract MediaPipe landmarks, normalize, package
python -m signdata run configs/jobs/lsa64/mediapipe.yaml

# Override config values from the command line
python -m signdata run configs/jobs/youtube_asl/mediapipe.yaml \
  --override processing.max_workers=8

Both modes produce WebDataset tar shards for efficient training data loading. See Pipeline Stages for detailed output formats and data shapes.

---

Documentation

• Installation Guide -- base setup and MMPose GPU dependencies
• Architecture -- system design, registry, pipeline flow
• Configuration -- job/experiment layout and CLI overrides
• Pipeline Stages -- recipe stages and optional stages
• Datasets -- setup for all built-in dataset adapters
• Contributing -- required dataset package structure and extension guide
• Research-Aligned Preprocessing -- paper-aligned preprocessing notes

Citation

If you use SignDATA in your research, please cite:

@Article{chen2026signdata,
    author  = {Kuanwei Chen and Tingyi Lin},
    journal = {arXiv:2604.20357},
    title   = {SignDATA: Data Pipeline for Sign Language Translation},
    year    = {2026},
}

License

The MIT license in this repository applies to the code and documentation in this project. Use of external datasets, research artifacts, and upstream repos referenced above must comply with their original licenses and usage terms.

MIT -- see LICENSE.