Explicit year holdout diagnostic
This exploratory amendment was specified after partial V3 random-split results were visible, before any year-holdout execution. Each year is held out completely; training and validation use disjoint dates from other years. All six prespecified models and all four years are reported. Neural seed is fixed at 0, so these are four different populations, not four repeat-seed estimates.
Partition sizes
| Held-out year | Train codas | Validation codas | Test codas |
|---|---|---|---|
| 2005 | 687 | 328 | 368 |
| 2008 | 845 | 275 | 263 |
| 2009 | 967 | 321 | 95 |
| 2010 | 570 | 156 | 657 |
unit
| Model | 2005 | 2008 | 2009 | 2010 |
|---|---|---|---|---|
| Raw ICI | 0.224 (368/368 supported) | 0.425 (263/263 supported) | 0.541 (95/95 supported) | 0.476 (657/657 supported) |
| ICI + temporal mel | 0.649 (368/368 supported) | 0.360 (263/263 supported) | 0.778 (95/95 supported) | 0.327 (657/657 supported) |
| WhAM | 0.822 (368/368 supported) | 0.371 (263/263 supported) | 0.789 (95/95 supported) | 0.422 (657/657 supported) |
| AVES | 0.750 (368/368 supported) | 0.510 (263/263 supported) | 0.833 (95/95 supported) | 0.377 (657/657 supported) |
| DCCE 110 | 0.592 (368/368 supported) | 0.362 (263/263 supported) | 0.767 (95/95 supported) | 0.346 (657/657 supported) |
| DCCE 111 | 0.448 (368/368 supported) | 0.391 (263/263 supported) | 0.713 (95/95 supported) | 0.357 (657/657 supported) |
coda_type
| Model | 2005 | 2008 | 2009 | 2010 |
|---|---|---|---|---|
| Raw ICI | 0.957 (358/368 supported) | 0.908 (263/263 supported) | 1.000 (95/95 supported) | 0.965 (636/657 supported) |
| ICI + temporal mel | 0.589 (358/368 supported) | 0.607 (263/263 supported) | 0.977 (95/95 supported) | 0.427 (636/657 supported) |
| WhAM | 0.077 (358/368 supported) | 0.045 (263/263 supported) | 0.166 (95/95 supported) | 0.051 (636/657 supported) |
| AVES | 0.065 (358/368 supported) | 0.047 (263/263 supported) | 0.103 (95/95 supported) | 0.029 (636/657 supported) |
| DCCE 110 | 0.596 (358/368 supported) | 0.702 (263/263 supported) | 1.000 (95/95 supported) | 0.476 (636/657 supported) |
| DCCE 111 | 0.583 (358/368 supported) | 0.546 (263/263 supported) | 0.839 (95/95 supported) | 0.421 (636/657 supported) |
individual_id
| Model | 2005 | 2008 | 2009 | 2010 |
|---|---|---|---|---|
| Raw ICI | 0.404 (236/314 supported) | 0.542 (57/57 supported) | 0.412 (37/64 supported) | 0.232 (292/319 supported) |
| ICI + temporal mel | 0.353 (236/314 supported) | 0.491 (57/57 supported) | 0.410 (37/64 supported) | 0.128 (292/319 supported) |
| WhAM | 0.238 (236/314 supported) | 0.229 (57/57 supported) | 0.038 (37/64 supported) | 0.093 (292/319 supported) |
| AVES | 0.193 (236/314 supported) | 0.207 (57/57 supported) | 0.228 (37/64 supported) | 0.068 (292/319 supported) |
| DCCE 110 | 0.258 (236/314 supported) | 0.607 (57/57 supported) | 0.468 (37/64 supported) | 0.197 (292/319 supported) |
| DCCE 111 | 0.257 (236/314 supported) | 0.670 (57/57 supported) | 0.349 (37/64 supported) | 0.186 (292/319 supported) |
Full-eligible macro-F1 values (including unseen labels) are in year_holdout_metrics.csv. Conditional date-cluster intervals are in year_holdout_intervals.csv. Class support and composition differ across years. These tests remove shared years between train/validation and test, but performance differences still combine biological, recording and label-composition changes; they do not identify a causal recording artifact.