Revision protocol

Corrected experiment protocol — review revision v1

This is the active pipeline for the September 2026 revision. Historical paper scores and the old experiment entry points remain archived in place; use the scripts below for corrected evaluation. Results are written to results_v2/ and never overwrite the old datasets or results.

Data and exclusions. Retain all 1,383 noise-filtered codas for unit and coda-type tasks. Parse dates explicitly as %d/%m/%Y. For identity tasks, exclude 620 unknown IDs, eight codas labeled 6070/6068, and the singleton 6059; 754 codas across 11 unambiguous, nonsingleton identity labels remain. The ambiguous ID is also ambiguous in the original Dominica metadata. It is not assigned arbitrarily to either whale. Excluded ID codas remain available for the other tasks. This changes the task population relative to the old paper.

Splits. Ordinary splits use seeds 0, 1, 2 and 64/16/20% train/validation/test membership, stratified by eligible identity and otherwise unit. All tasks inherit the same coda membership. The recording-date protocol uses the first three distinct outer folds of a fixed five-fold StratifiedGroupKFold partition (partition seed 0), with a grouped validation fold inside each training portion. These are three grouped folds, not three independent random test sets. No date crosses train, validation, and test within a run. Dates are a proxy for sessions, not verified session IDs. Manifests are fixed before any predictive results and include per-task class support. A fixed fold index is used; no fold search is performed for high scores.

Features. Resample each audio file to mono 48 kHz with soxr_hq. Compute 64-band mel power using a 2,048-sample Hann FFT window, a 1,152-sample hop (24 ms), center=True, and 0–8 kHz bands. Convert relative to each clip's peak to an 80 dB dynamic range. Retain 256 frames, pad missing frames at -80 dB, and apply the fixed transform db / 40 + 1. This is a roughly 6.1-second window; the audited corpus has no truncated clips. There are no population-fitted mel statistics. The 9-element ICI vector retains annotated intervals and zero padding; fit its scaler on training rows only. Temporal-mel floors use mean and standard deviation in four bins over valid frames, excluding padded frames. Frequency-band sensitivity is deferred.

Frozen embeddings. Reuse existing sample-local WhAM, AVES, and BirdNET caches. Check shape, finite/nonzero rows, and alignment; record SHA-256 hashes. The extraction code aligns WhAM by coda_id - 1, and AVES/BirdNET by the clean CSV order. These historical caches have not been re-extracted in this revision. In particular, freezing WhAM does not prove the original pretraining corpus excluded these codas; its repository describes DSWP species-specific fine-tuning. Pretraining overlap remains an explicit unresolved limitation.

Probes and layer selection. For each task, fit a train-only StandardScaler and balanced logistic regression (C=1, max_iter=2000, lbfgs). C is fixed in advance. For WhAM/AVES, select the layer by validation macro-F1, breaking ties by earliest layer; evaluate only the selected layer on test. BirdNET has one cached representation. Do not refit on train+validation. Log validation scores for every layer and convergence warnings. Neural representations are frozen and probed through the same harness.

Neural conditions. All initial neural conditions use unit-balanced sampling, so none is called label-free. Unit balancing is in expectation. Train-only inverse-frequency weights are used for classification losses. No test or validation identity labels enter representation training.

The dual encoder preserves the legacy two-layer GRU and three-block CNN design, with its spectral projection sized for 256 frames. The fusion is LayerNorm → Linear(128,64) → ReLU → Linear(64,64); normalization for NT-Xent occurs inside the loss. Supervised_Dual puts all losses on the fused representation so that fusion is actually trained; it matches inputs and label availability, but changes objective placement and is not a single-factor loss-location ablation. AVES adapters are 128-hidden/64-output MLPs with dropout 0.3. Their pretrained backbone stays frozen, so they match downstream labels, not full encoder fine-tuning. The AVES+ICI condition also receives the annotated timing information supplied to DCCE.

Whole-coda and swapped contrastive variants exclude self-pairs and draw partners only from training codas in the same unit. NT-Xent temperature is 0.07. Other same-unit samples can still be negatives; this is pairwise NT-Xent, not a multi-positive supervised contrastive loss. The initial grid does not test all alternate negative definitions.

Training budget. Every model uses 50 fixed epochs, batch size 64, AdamW learning rate 0.001 and weight decay 0.0001, with cosine scheduling. Loss weights, architecture, and preprocessing are fixed before viewing the corrected test results. Use the final epoch without test-dependent checkpoint selection. Train a new representation for each split/fold. Record seeds, configurations, histories, weights, embeddings, input hashes, and source snapshots. MPS runs are seeded but bitwise determinism across hardware/library versions is not claimed. A two-epoch smoke run is stored separately and excluded from all result summaries.

Metrics and uncertainty. Primary macro-F1 averages over true test classes that occur in training. Report the number of eligible and supported test codas, the evaluated classes, and excluded unseen-class codas. Also report macro-F1 over all eligible true test classes including unseen labels, which measures the closed-set classifier's failure on those labels. No unseen-identity recognition claim is made. This support-aware metric and the changed splits/ID population mean numerical comparisons to the original 0.834 are not direct before/after effect estimates. In date-grouped tests, candidate sets vary by fold, so raw grouped-versus-random ID differences conflate generalization and task composition. Compare models on paired folds, and state support beside results.

Report each run and mean ± sample standard deviation over three runs/folds. SD is not a 95% confidence interval. Overlapping training sets and sparse identities limit inference; no significance or population-generalization claim follows from three folds. Save per-coda predictions for future group-aware uncertainty analysis and per-unit/type breakdowns. A bootstrap of a single fitted model would not replace retraining variability.

Commands. Run from the repository root:

OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python scripts_v2/revision_data.py
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python scripts_v2/test_revision_pipeline.py
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python -u scripts_v2/revision_experiments.py --stage baselines
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python -u scripts_v2/revision_experiments.py --stage neural
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python scripts_v2/revision_diagnostics.py
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python scripts_v2/revision_report.py
OPENBLAS_NUM_THREADS=1 OMP_NUM_THREADS=1 .venv/bin/python scripts_v2/verify_revision_runs.py --require-complete

The experiment runner skips completed results only when their source, input-provenance, split hashes, and epoch counts match. It refuses stale resumes. A smoke run can use --stage neural --models DCCE_ID --seeds 0 --protocols random --epochs 2 --smoke. Default neural execution uses MPS; use --device cpu or --device cuda on other machines.

Remaining extensions include the full label-free/unit-only/type-only/ID-only objective matrix, standardized single-channel ablations, fully fine-tuned external encoders, frequency-band sensitivity, full five-fold grouped coverage, and external-population validation. They should follow the corrected first-round comparison rather than precede it.