V3 supervision and implementation matrix
U/T/I refer to labels used during representation training. Every row later freezes the representation and trains a separate supervised linear probe. All V3 samplers are uniform; auxiliary class weights use training labels only.
| Model | U | T | I | Input | Pairing | Representation | Dimension |
|---|---|---|---|---|---|---|---|
| Dual_U0T0I0 | 0 | 0 | 0 | dual, 8 kHz | same-coda augmented views | trained, 50 epochs | 64 |
| Dual_U0T0I1 | 0 | 0 | 1 | dual, 8 kHz | same-coda augmented views | trained, 50 epochs | 64 |
| Dual_U0T1I0 | 0 | 1 | 0 | dual, 8 kHz | same-coda augmented views | trained, 50 epochs | 64 |
| Dual_U0T1I1 | 0 | 1 | 1 | dual, 8 kHz | same-coda augmented views | trained, 50 epochs | 64 |
| Dual_U1T0I0 | 1 | 0 | 0 | dual, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| Dual_U1T0I1 | 1 | 0 | 1 | dual, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| Dual_U1T1I0 | 1 | 1 | 0 | dual, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| Dual_U1T1I1 | 1 | 1 | 1 | dual, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| rhythm_UTI0 | 1 | 1 | 0 | rhythm, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| rhythm_UTI1 | 1 | 1 | 1 | rhythm, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| spectral_UTI0 | 1 | 1 | 0 | spectral, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| spectral_UTI1 | 1 | 1 | 1 | spectral, 8 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| concat_UTI0 | 1 | 1 | 0 | concat, 8 kHz | same-unit whole codas | trained, 50 epochs | 128 |
| concat_UTI1 | 1 | 1 | 1 | concat, 8 kHz | same-unit whole codas | trained, 50 epochs | 128 |
| DualswapUTI | 1 | 1 | 1 | dual, 8 kHz | same-unit swapped channels | trained, 50 epochs | 64 |
| Dual16kUTI0 | 1 | 1 | 0 | dual, 16 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| Dual16kUTI1 | 1 | 1 | 1 | dual, 16 kHz | same-unit whole codas | trained, 50 epochs | 64 |
| AVES_FT | CE | CE | CE | audio | none | transformer fine-tuned, 10 epochs; frontend frozen | 64 |
| AVESICIFT | CE | CE | CE | audio + ICI | none | transformer fine-tuned, 10 epochs; frontend frozen | 64 |
Single-channel auxiliary heads both use the available channel. Dual/concatenation type heads use rhythm and ID heads use spectrum. U=0 removes unit labels from representation training but not from evaluation partitioning/probes. Label-free objectives still use annotation-derived ICI inputs. AVES FT has a different pretraining history, parameter count and epoch budget from DCCE.
V2 benchmark: downstream whale-label access
These columns describe local downstream training, not all labels used in external pretraining. Every row uses a subsequently trained supervised probe.
| Model | Unit use | Type loss | ID loss | Unit-balanced sampler | Representation | Input |
|---|---|---|---|---|---|---|
| Raw ICI | none | no | no | no | fixed features | annotated ICI |
| Temporal mel | none | no | no | no | fixed features | audio |
| ICI + temporal mel | none | no | no | no | fixed features | annotated ICI + audio |
| WhAM | none | no | no | no | frozen pretrained; WhAM/AVES layer selected on validation | audio |
| AVES | none | no | no | no | frozen pretrained; WhAM/AVES layer selected on validation | audio |
| BirdNET | none | no | no | no | frozen pretrained; WhAM/AVES layer selected on validation | audio |
| DCCE no ID | whole-coda unit pairing | yes | no | yes | trained, 50 epochs | annotated ICI + audio |
| DCCE ID | whole-coda unit pairing | yes | yes | yes | trained, 50 epochs | annotated ICI + audio |
| DCCE swap ID | same-unit swapped channels | yes | yes | yes | trained, 50 epochs | annotated ICI + audio |
| Supervised dual | unit CE | yes | yes | yes | trained head/fusion; AVES backbone frozen in adapter rows | annotated ICI + audio |
| AVES adapter | unit CE | yes | yes | yes | trained head/fusion; AVES backbone frozen in adapter rows | audio |
| AVES + ICI adapter | unit CE | yes | yes | yes | trained head/fusion; AVES backbone frozen in adapter rows | annotated ICI + audio |
Additional frozen input-matched controls
WhAM + ICI, AVES + ICI, and BirdNET + ICI concatenate annotated timing with frozen audio embeddings. They use no local representation-training labels, losses, pairing, or sampler. Only the training-only scaler and supervised linear probe are fitted; WhAM/AVES layers are selected on validation data. External pretraining label/data access remains as described above. These controls are evaluated on all six random/date partitions and all four year holdouts.
The strict year diagnostic reuses the same definitions for Raw ICI, ICI + temporal mel, WhAM, AVES, DCCE 110 and DCCE 111. The latter two are trained afresh on each year partition with uniform sampling. Saved-branch probes are post-hoc diagnostics on fixed DCCE 110/111 checkpoints and do not replace the primary fused-representation results.