Baseline Reframing Plan (Option A)
Goal: satisfy the professor's note — "add 1-3 strong baselines, if we can identify them from literature; weak baselines would not be worth doing" — by (a) fixing how we frame baselines, (b) adding strong baselines drawn from the bioacoustics literature, and (c) making the WhAM comparison fair.
Status: framing only. No new models have been run yet. Current numbers are from Phases 1-3; rows marked TBD are the new baselines to run once the model-weight download is unblocked.
1. The conceptual fix (what changed in our thinking)
Two confusions were baked into the current paper:
- "Baseline" was doing two jobs. (1) The conceptual status quo — prior work studied codas one channel at a time — which is motivation and belongs in the Introduction/Related Work. (2) Empirical baselines — concrete methods we run on our data and beat with real numbers — which is what the professor is asking for and belongs in Experiments. We had been answering (2) with mostly trivial methods.
- Baselines were conflated with ablations. Our DCCE variants (
rhythm_only,spectral_only,late_fusion,full) are ablations: they test which parts of our model matter (internal validity). They are not baselines. Baselines are external methods that test whether DCCE beats what already exists (external validity). The professor's note is only about baselines.
2. Baseline taxonomy for the paper
We re-organise every comparison row into four clearly-labelled roles:
| Role | Purpose | Members |
|---|---|---|
| Feature floors | what each channel gives with no learned encoder | Raw ICI, Raw Mel |
| Strong single-channel baselines | the spectral-channel paradigm done with a real learned encoder from literature | AVES, BirdNET/Perch, wav2vec2/HuBERT (TBD) |
| SOTA integrated baseline | the current best learned representation for this data | WhAM (compared fairly) |
| Ours | integrates both channels by design | DCCE variants |
Key point for the write-up: the "strong single-channel baselines" are literally the strong instantiation of our own "prior work was single-channel" motivation. Raw ICI / Raw Mel are the weak versions of that idea; AVES/BirdNET/Perch are the strong versions.
3. The strong baselines we will add (with literature justification)
Each line is the one-sentence justification that satisfies "from literature":
- AVES (Hagiwara 2023, ICASSP, arXiv:2210.14493) — a self-supervised animal-vocalization encoder (HuBERT-based); WhAM's own paper used AVES as a baseline, so it is the established category-peer for a learned representation of animal sound. (must-have)
- BirdNET or Perch embeddings (Kahl 2021, Ecol. Informatics; Ghani 2023, Sci. Rep., arXiv:2307.06292) — Tolkova et al. 2025 used BirdNET embeddings + a shallow classifier for cetacean individual ID (right whales, 11 individuals, small dataset) — almost exactly our task and regime. Strongest literature fit for our headline task. (needs TensorFlow/TFLite setup)
- wav2vec2 / HuBERT (Baevski 2020; Hsu 2021) — generic speech SSL; Sarkar & Magimai-Doss 2023 showed human-speech SSL features carry individual-caller information for animals. The "off-the-shelf foundation model" control. (trivial via torchaudio)
All run through the identical split + linear probe already validated in scripts/lib_eval.py (it reproduces our exact Phase 1 numbers), so they are directly comparable.
4. The fairness fix: tier the results by supervision
The single most important change. A strong baseline compared unfairly is still a weak result.
The problem: DCCE-full learns its representation with individual-ID supervision (the L_id auxiliary loss). WhAM, AVES, and the SSL models never saw ID labels. So "DCCE 0.834 vs WhAM 0.454 on individual ID" is largely supervised-on-the-task vs. never-saw-the-labels — not "domain prior beats scale."
The fix: split the individual-ID comparison into two tiers, and add DCCE without L_id so our architecture also appears in the label-free tier.
Individual ID (the headline task) — grouped by whether the representation saw ID labels
Tier A — label-free representations (the fair, apples-to-apples comparison):
| Model | ID macro-F1 | Source |
|---|---|---|
| Raw ICI | 0.493 | feature floor |
| Raw Mel | 0.272 | feature floor |
| WhAM L10 | 0.454 | SOTA generative, no labels |
| WhAM L19 | 0.363 | " |
| AVES | TBD | bioacoustic SSL |
| wav2vec2 / HuBERT | TBD | generic SSL |
| BirdNET / Perch | TBD | bird transfer (Tolkova-style) |
DCCE (no L_id) |
TBD | our architecture, label-free |
Tier B — ID-supervised representations:
| Model | ID macro-F1 | Source |
|---|---|---|
| DCCE-full | 0.834 | saw ID labels via L_id |
| Supervised CNN on mel | TBD | control |
| WhAM + supervised MLP head | TBD | control |
This makes the honest claim legible: the right comparison for our contribution is DCCE (no L_id) vs. WhAM/AVES in Tier A. If our architecture still wins there, the dual-channel prior matters; if it collapses toward WhAM, the win was the supervision. Either result is publishable and honest — and far stronger than the current framing.
(Unit and coda-type comparisons can stay as a single table; the supervision asymmetry mainly bites on individual ID.)
5. Honest reframing of the headline and the ablation story
The current draft says individual ID is "+83% relative" over WhAM and attributes it to the cross-channel novelty. The ablations actually say something different — and more defensible:
late_fusion→full(the actual cross-channel novelty) moves individual ID by only +0.009 (0.825 → 0.834).- The same change moves social unit by +0.222 (0.656 → 0.878).
spectral_only(CNN + ID supervision, no cross-channel) already reaches 0.787 on ID.
Honest reading: the cross-channel contrastive objective's real contribution is to the social-unit representation; the individual-ID strength comes from the supervised spectral encoder, not the novelty. We should state this plainly. It is a better paper to say "our novelty improves unit-level structure, and a supervised spectral encoder handles individual ID" than to over-claim the ID number.
6. Draft Related Work / baseline narrative (lift-able prose)
Prior computational work on sperm whale codas has treated the signal one channel at a time. The rhythm channel (inter-click-interval timing) has been modelled with sequence and clustering methods that recover coda type and clan structure (Gero et al. 2016; Leitão et al. 2025; Bermant et al. 2019). The spectral channel (within-click, vowel-like formant structure) has been characterised independently (Beguš et al. 2025, 2026). No prior model integrates the two. We therefore benchmark DCCE against the strongest available instantiation of each paradigm rather than against trivial features alone.
For the spectral/learned-audio paradigm, we evaluate self-supervised and transfer-learned encoders that the bioacoustics literature has established for animal-sound tasks: AVES (Hagiwara 2023), a self-supervised animal-vocalization encoder used as a baseline by WhAM itself; BirdNET/Perch embeddings (Kahl et al. 2021; Ghani et al. 2023), which Tolkova et al. (2025) applied to cetacean individual identification on a small dataset closely matching ours; and generic speech-SSL encoders (wav2vec 2.0, HuBERT), shown by Sarkar & Magimai-Doss (2023) to carry individual-caller information for animals. For the rhythm paradigm, a linear model on raw ICIs already classifies coda type near-perfectly (macro-F1 0.931), so it serves as a near-ceiling floor rather than admitting a stronger learned baseline. Finally, WhAM (Paradise et al. 2025) is the current state-of-the-art integrated representation for this data and our primary learned comparison; because WhAM is generative and label-free, we compare it against a label-free variant of DCCE to keep the comparison fair.
7. Factual corrections to apply in main.tex (from reading the source papers)
These are independent of the baselines but should be fixed in the same revision:
- Sharma et al. (2024) —
main.tex:91currently says the combinatorial vocabulary is "roughly 10× larger." The paper's actual claim is "≥143 frequently-realised combinations" and a doubling of per-coda information (~5→~10 bits), from rhythm/tempo/rubato/ornamentation. Fix the number. - Gero et al. (2016) — anywhere we imply broad individual recoverability: individual ID is reliable only within a unit, primarily on the
5R1coda type (73-83%); the shared1+1+3type classifies individuals near chance (33%). This actually supports our "individual ID is hard" framing. - Beguš — this is two papers (Open Mind 2025 + Proc. R. Soc. B 2026), and the vowel inventory is
a/i+ diphthongs, framed as a channel that interacts with (does not replace) rhythm. Update the citation and the claim.
8. What runs next (once unblocked)
| Step | Needs permission? | Output |
|---|---|---|
Retrain DCCE without L_id |
No (local training, ~1 min) | the Tier-A label-free DCCE number |
| Extract AVES + wav2vec2 + HuBERT embeddings | Yes (download weights + torch.load) |
Tier-A baseline numbers |
| Extract BirdNET/Perch embeddings | Yes + TF install | Tier-A bird-transfer number |
| Probe all (multi-seed) + build tiered table & figure | No (harness ready) | final results table + comparison figure |
Apply §6 prose + §7 corrections to main.tex |
No | revised paper |
Decision needed before step 2: green-light the model-weight download (and whether to include the TF-dependent BirdNET/Perch).