V2 — corrected experiment results
Completed 72/72 planned model/split runs. The initial experiment grid is complete.
All results below use the new shared split manifests, corrected dates and mel preprocessing, train-only representation learning, and validation-only layer selection. Historical scores are not directly comparable: identity eligibility, preprocessing, split sizes, and metric support have changed. The two-epoch smoke run is excluded.
Cells are mean ± sample SD over the available runs, with n=1 explicitly marked. The target is three ordinary random splits and three distinct grouped outer folds (the first three of a fixed five-fold partition). SD is not a confidence interval; no significance claim is made.
Ordinary splits — supported-class macro-F1
| Model | Unit | Coda type | Individual ID | Completed runs |
|---|---|---|---|---|
| Raw ICI | 0.570 ± 0.023 | 0.987 ± 0.009 | 0.439 ± 0.007 | 3/3 |
| Temporal mel | 0.773 ± 0.007 | 0.202 ± 0.025 | 0.429 ± 0.045 | 3/3 |
| ICI + temporal mel | 0.837 ± 0.013 | 0.804 ± 0.016 | 0.724 ± 0.039 | 3/3 |
| WhAM frozen | 0.906 ± 0.011 | 0.258 ± 0.046 | 0.450 ± 0.052 | 3/3 |
| AVES frozen | 0.910 ± 0.010 | 0.253 ± 0.028 | 0.491 ± 0.029 | 3/3 |
| BirdNET frozen | 0.887 ± 0.020 | 0.215 ± 0.053 | 0.480 ± 0.022 | 3/3 |
| DCCE, no ID loss | 0.907 ± 0.003 | 0.800 ± 0.026 | 0.627 ± 0.057 | 3/3 |
| DCCE, whole-coda + ID | 0.889 ± 0.004 | 0.722 ± 0.051 | 0.588 ± 0.079 | 3/3 |
| DCCE, swapped + ID | 0.879 ± 0.029 | 0.636 ± 0.042 | 0.641 ± 0.066 | 3/3 |
| Supervised dual encoder | 0.886 ± 0.017 | 0.762 ± 0.119 | 0.684 ± 0.081 | 3/3 |
| AVES supervised adapter | 0.933 ± 0.009 | 0.272 ± 0.027 | 0.479 ± 0.018 | 3/3 |
| AVES + ICI supervised adapter | 0.940 ± 0.009 | 0.452 ± 0.023 | 0.582 ± 0.078 | 3/3 |
Recording-date-disjoint folds — supported-class macro-F1
| Model | Unit | Coda type | Individual ID | Completed runs |
|---|---|---|---|---|
| Raw ICI | 0.540 ± 0.036 | 0.956 ± 0.036 | 0.379 ± 0.103 | 3/3 |
| Temporal mel | 0.569 ± 0.094 | 0.069 ± 0.012 | 0.130 ± 0.074 | 3/3 |
| ICI + temporal mel | 0.608 ± 0.150 | 0.668 ± 0.051 | 0.255 ± 0.062 | 3/3 |
| WhAM frozen | 0.667 ± 0.155 | 0.066 ± 0.016 | 0.101 ± 0.075 | 3/3 |
| AVES frozen | 0.700 ± 0.100 | 0.061 ± 0.022 | 0.074 ± 0.051 | 3/3 |
| BirdNET frozen | 0.612 ± 0.133 | 0.063 ± 0.011 | 0.085 ± 0.022 | 3/3 |
| DCCE, no ID loss | 0.687 ± 0.023 | 0.616 ± 0.066 | 0.281 ± 0.109 | 3/3 |
| DCCE, whole-coda + ID | 0.574 ± 0.074 | 0.502 ± 0.018 | 0.257 ± 0.063 | 3/3 |
| DCCE, swapped + ID | 0.611 ± 0.014 | 0.476 ± 0.084 | 0.219 ± 0.089 | 3/3 |
| Supervised dual encoder | 0.611 ± 0.080 | 0.675 ± 0.058 | 0.270 ± 0.095 | 3/3 |
| AVES supervised adapter | 0.646 ± 0.134 | 0.079 ± 0.029 | 0.059 ± 0.044 | 3/3 |
| AVES + ICI supervised adapter | 0.669 ± 0.119 | 0.183 ± 0.037 | 0.081 ± 0.072 | 3/3 |
Identity coverage — essential when reading grouped scores
| Protocol / run | Train / val / test codas | ID training classes | Supported / eligible ID test codas | Supported true ID test classes | Unseen test IDs |
|---|---|---|---|---|---|
| date_grouped / 0 | 828 / 227 / 328 | 9 | 186 / 213 | 3 | 5719, 5720 |
| date_grouped / 1 | 901 / 211 / 271 | 7 | 111 / 121 | 6 | 5973, 6069 |
| date_grouped / 2 | 883 / 239 / 261 | 7 | 145 / 145 | 6 | None |
| random / 0 | 884 / 222 / 277 | 11 | 151 / 151 | 11 | None |
| random / 1 | 884 / 222 / 277 | 11 | 151 / 151 | 11 | None |
| random / 2 | 884 / 222 / 277 | 11 | 151 / 151 | 11 | None |
Primary macro-F1 averages over true test classes also seen in training. Grouped folds have different identity candidate/support sets, so a higher grouped ID score does not establish better generalization than a random split. Every model is compared on the same supported samples within a fold. The next table includes the unseen-label test cases, which a closed-set classifier cannot identify.
| Model | Grouped ID F1, all eligible true test classes including unseen |
|---|---|
| Raw ICI | 0.290 ± 0.089 |
| Temporal mel | 0.095 ± 0.043 |
| ICI + temporal mel | 0.198 ± 0.064 |
| WhAM frozen | 0.086 ± 0.070 |
| AVES frozen | 0.061 ± 0.044 |
| BirdNET frozen | 0.067 ± 0.031 |
| DCCE, no ID loss | 0.215 ± 0.083 |
| DCCE, whole-coda + ID | 0.202 ± 0.073 |
| DCCE, swapped + ID | 0.153 ± 0.029 |
| Supervised dual encoder | 0.198 ± 0.063 |
| AVES supervised adapter | 0.050 ± 0.042 |
| AVES + ICI supervised adapter | 0.068 ± 0.059 |
Paired differences (first model minus second)
| Protocol | Comparison | Unit | Coda type | ID | Paired runs |
|---|---|---|---|---|---|
| random | DCCE, whole-coda + ID − DCCE, no ID loss | -0.017 | -0.078 | -0.039 | 3 |
| random | DCCE, swapped + ID − DCCE, whole-coda + ID | -0.010 | -0.086 | +0.053 | 3 |
| random | DCCE, whole-coda + ID − Supervised dual encoder | +0.004 | -0.040 | -0.096 | 3 |
| random | DCCE, whole-coda + ID − ICI + temporal mel | +0.052 | -0.082 | -0.136 | 3 |
| random | DCCE, whole-coda + ID − AVES + ICI supervised adapter | -0.051 | +0.270 | +0.006 | 3 |
| date_grouped | DCCE, whole-coda + ID − DCCE, no ID loss | -0.113 | -0.113 | -0.024 | 3 |
| date_grouped | DCCE, swapped + ID − DCCE, whole-coda + ID | +0.037 | -0.026 | -0.038 | 3 |
| date_grouped | DCCE, whole-coda + ID − Supervised dual encoder | -0.038 | -0.173 | -0.012 | 3 |
| date_grouped | DCCE, whole-coda + ID − ICI + temporal mel | -0.034 | -0.165 | +0.002 | 3 |
| date_grouped | DCCE, whole-coda + ID − AVES + ICI supervised adapter | -0.095 | +0.319 | +0.176 | 3 |
Diagnostics and interpretation limits
- Probe convergence warnings logged across all validation layer fits: 0. Inspect per-layer trials if nonzero; do not hide warnings.
- WhAM/AVES/BirdNET are frozen transfer references. The neural conditions have explicit label supervision. AVES adapters match downstream labels but do not constitute full backbone fine-tuning.
- Annotated ICI is extra task-informed input. AVES + ICI and the raw concatenation control help separate that input advantage from architecture.
- No condition here is fully label-free. No causal interpretation of channels, recording artifacts, or whale semantics follows from these classification scores.
- All dates parse: chi-squared = 530.0436 (df=6), Cramér's V = 0.43775. Recording date is only a session proxy, and external-model pretraining overlap remains unresolved.
- Per-unit/type/individual ID breakdowns are saved in
identity_strata.csv. They subset the global classifier's predictions; no within-unit classifier is refit. Tiny and single-identity strata are descriptive, not robust evidence. - This first grid does not complete the full single-channel/label-free/bandwidth ablation program or validate another whale population.
Artifacts
- Protocol and exact commands
- Per-run metrics
- Summary metrics
- Paired differences
- Identity strata
- Split coverage
- Test support for every task
- Comparison figure
- Corrected recording-year diagnostic
- Raw predictions, validation-layer profiles, checkpoints, embeddings, and training histories are under
results_v2/runs/.