Specifically presented as taking the output of `distributed-downloader` and then does all these things to _create_ the TreeOfLife dataset. - [x] Rename repo: TreeOfLife-toolbox - [x] Rename under `src`: `TOL-toolbox` - [ ] Reorganize under `TOL-toolbox` to have submodules based on function (e.g., `filtering`, `data-transfer`) with sub-submodules (e.g., `research-filtering`) - [ ] Add from TreeOfLife-dev as submodules in `src/TOL-toolbox`: - [ ] webdataset creation - [ ] metadata transfer - [ ] lookup table creator - [ ] Add from TreeOfLife-dev at root-level `scripts/` directory all lookup table exclude creators (many of these are [here](https://github.com/Imageomics/TreeOfLife-dev/tree/main/src/processing)): - [ ] PDQ hash process - [ ] Image type separator (museum, camera trap, and citizen science categories) - [ ] Museum Specimen processing: label vs specimen filter - [ ] camera trap processing: megadetector & reducer - [ ] citizen science image processing - [ ] face detection - [ ] Add from TreeOfLife-dev at root-level `notebooks/` directory all lookup table exclude creators: - [ ] clustering and other EDA determinations - [ ] Add root-level `data/` folder with embeddings for museum support sets (relates to #22). This will be done following the merge of PRs #1 through #15.
Activity
src/TOL-toolbox: #21