feat(7sense): real-time animal sound identification and acoustic visualization

[ruvnet]

Context

The vibecast-7sense example advertises real-time species identification and a manifold visualization. An audit of the code found neither exists, and several documented entry points resolve to nothing:

• No audio capture dependency anywhere; ingestion is ingest_file(&Path) decoding the whole file into a Vec<f32>. The AudioStream::new() in the crate README is not a real type.
• /ws/stream is documented in two READMEs but no such route is registered, and the WebSocket receive task discards every inbound frame.
• No classifier, label set, or logits head exists. Perch 2.0 as integrated emits embeddings only, so POST /api/identify could not have been served.
• No frontend source of any kind. The sevensense-viz crate that ADR-009 specifies was never created, and the API's umap_url points at an unregistered route.
• sevensense-api declares all seven sibling crates as dependencies and imports none of them, reimplementing each stage as a stub that returns zero vectors and empty segments.

This PR closes those gaps. It is being built incrementally; the checklist below tracks progress.

Architecture decisions

Four ADRs record the design before the code lands.

ADR-010 — Real-time streaming ingestion. An AudioSource trait keeps cpal behind a non-default feature so WASM keeps building. A lock-free ring buffer with an explicit lossy-overwrite policy, because blocking a real-time audio callback causes device glitches and stale audio is worth less than current audio. An EMA noise floor with open/close hysteresis replaces global statistics, so a segment can close before the recording ends.

ADR-011 — Acoustic feature extraction. Descriptors are computed from a linear-frequency power spectrum rather than the existing mel path, because log-scaled perceptually-spaced bins do not yield a centroid in Hz.

ADR-012 — Manifold projection. Supersedes ADR-009's technology choices. PCA via randomized SVD is the default rather than UMAP: it is deterministic, so saved viewports stay valid, and it extends to new points in one matrix multiply, which is what streaming needs. UMAP remains opt-in, implemented over the existing HNSW graph rather than via the unmaintained umap-rs.

ADR-013 — Retrieval-based identification. k-NN against a labelled reference index instead of a classifier. Open-set by construction, so unknown species are detectable rather than forced into the nearest class; extensible by adding reference recordings rather than retraining; and every result carries the neighbours that produced it, which is what makes it auditable.

Progress

• ADR-010, ADR-011, ADR-012, ADR-013
• sevensense-audio::features — interpretable acoustic descriptors (ADR-011)
• sevensense-audio::streaming — live ingestion pipeline (ADR-010)
• sevensense-vector::projection — PCA / Poincaré projection (ADR-012)
• Retrieval-based identification (ADR-013)
• Wire sevensense-api to the real crates; add /ws/stream, /projection, /features
• Web UI — 3D manifold explorer with live feature panels
• Benchmarks and optimization pass

Changes so far

Acoustic features (ADR-011)

Spectral centroid, spread, skewness, rolloff, flatness, tonality, crest, entropy, slope, and a parabolically-interpolated dominant frequency per frame, plus amplitude and frequency modulation across frames. This populates CallSegment::spectral_centroid — a field whose builder previously had no callers anywhere in the workspace, leaving it permanently None.

Details worth review: flatness uses exp(mean(ln p)) because the product of ~1000 bin powers underflows f32; summary statistics cover voiced frames only, since averaging across silence pulls the centroid toward the noise floor; modulation returns None below 32 frames rather than a fabricated rate, and its band is capped at envelope Nyquist (50 Hz at the default hop), which corrects an overstatement in the first ADR draft.

Streaming ingestion (ADR-010)

RingBuffer is a lock-free SPSC queue that overwrites its oldest samples when the consumer falls behind and reports the gap. Only the producer stores write_pos and only the consumer stores read_pos, so no CAS is needed. Samples are held as AtomicU32 bit patterns — a torn read is inherent to an overwrite policy, and going through atomics makes that a stale value rather than undefined behaviour. The crate still contains no unsafe: Send/Sync are derived, with a compile-time assertion in place of a manual unsafe impl.

StreamSegmenter uses an EMA noise floor with open/close hysteresis. Only quiet frames update the floor — otherwise a long call drags the threshold up behind itself and closes its own segment. Pre-roll frames are retained so the opening transient is not clipped.

One design fix the tests forced: segments initially included the trailing hangover, which padded every segment with silence and let a 30 ms transient masquerade as a 300 ms call. Closing now trims the hangover, keeping one frame so natural decay survives.

Projection (ADR-012)

Randomized SVD that never forms the scatter matrix — scatter_multiply computes XᵀXv in O(n·d), so memory stays linear in the embedding dimension rather than quadratic. Two power iterations sharpen the subspace (acoustic embeddings have a slowly decaying spectrum); Gram-Schmidt reorthogonalizes twice, since one pass loses orthogonality on the nearly-dependent vectors power iteration produces.

Also puts the hyperbolic module to use — implemented and entirely unreferenced since the crate was written.

A bug the tests caught: tanh saturates to exactly 1.0 in f32 past ~9, landing distant points on the ball boundary, which is at infinite hyperbolic distance from everything. The radius is now clamped strictly inside.

Testing

186 tests passing, 0 failing across sevensense-audio and sevensense-vector.

Every test drives a signal or dataset whose correct answer is known analytically, so a failure points at the maths rather than at a fixture. The invariants that carry the most weight:

• The ring buffer's concurrent producer/consumer test asserts consumed + dropped == produced — no sample may vanish silently or be counted twice.
• Segmentation is asserted identical across push chunk sizes and source chunk sizes, since a live device delivers whatever it likes.
• A signal below the adapted noise floor must not open a segment.
• Single-point projection must agree exactly with the batch path, or streaming and batch views would disagree.
• Fitting at the real 1536-dimensional embedding size is time-asserted, which fails loudly if anyone reintroduces the full scatter matrix.

cargo test -p sevensense-audio -p sevensense-vector