Two encoders. A shared representation.

The question behind the architecture

Sperm whales communicate through short click sequences called codas, whose structure includes the rhythmic pattern of inter-click intervals and the spectral composition of the clicks. We propose the Dual-Channel Contrastive Encoder (DCCE), which explicitly encodes rhythm and spectral information with separate encoders before fusing them.

Our guiding question is: does encoding rhythm and spectral texture as separate input channels, rather than learning implicitly from one input representation, improve what a model can learn from sperm whale codas? We measure information relevant to social unit, coda type, and individual identity using lightweight linear probes.

Work on coda structure and identity motivates this question, including Gero et al., Sharma et al., and studies of spectral patterns. These precedents do not establish that each target belongs exclusively to one channel, or that learned branches are statistically independent.

Rhythm encoder

The rhythm of a coda is captured by its sequence of inter-click intervals: the elapsed times, in seconds, between consecutive clicks. A coda of k clicks yields k − 1 ICIs. Since the codas in our data contain up to ten clicks, each coda has at most nine ICIs.

Annotated ICIs are zero-padded on the right and standardized using training examples only. A two-layer GRU with hidden size 64 and inter-layer dropout 0.2 reads the sequence one ICI at a time. A linear projection and layer normalization map its final hidden state to a 64-dimensional rhythm vector, r. We choose a compact recurrent encoder for these short sequences and the small dataset.

Spectral encoder

Each recording is converted to mono and resampled to 48 kHz. A short-time Fourier transform uses a 2,048-sample Hann window and a 1,152-sample hop, mapped onto 64 mel bands spanning 0–8 kHz. Log power is peak-relative, clipped to 80 dB, padded at the minimum level to a fixed 64 × 256 representation, and transformed by x_dB/40 + 1. No mel input is truncated.

Three convolutional blocks (Conv2d–BatchNorm–ReLU–MaxPool, with 16, 32, and 64 channels) use 3 × 3 kernels and 2 × 2 pooling to reduce the input to a 64 × 8 × 32 feature map. The flattened output passes through dropout 0.3 and a two-layer MLP with layer normalization to a 64-dimensional spectral vector, s.

The full-coda spectrogram retains temporal structure. This branch is therefore not restricted to within-click identity cues, and “spectral” should not be read as “timing-free.”

Fusion and objectives

We concatenate r and s, then apply LayerNorm, Linear(128,64), ReLU, and Linear(64,64) to obtain the fused vector z. Embeddings are normalized inside the contrastive loss. Primary probes use the saved fusion output; separate branch probes are post-hoc diagnostics.

The training objective combines NT-Xent with optional weighted cross-entropies for type on the rhythm branch and identity on the spectral branch:

The switches a_T and a_I are binary. Inverse-frequency class weights use training counts only; ineligible identity labels are masked. The contrastive temperature is 0.07, with cosine similarity and a symmetric average over positive pairs. Assigning different tasks to the branches encourages discriminative features but does not enforce independence.

Positive pairs and augmentation

The primary construction pairs distinct training codas A and B from the same social unit: f(r_A, s_A) and f(r_B, s_B). This uses unit supervision. Actual channel swapping instead pairs f(r_A, s_B) and f(r_B, s_A). Without unit pairing, positives are independently augmented views of the same coda.

Other batch views act as negatives, including possible same-unit examples. The objective is not a full supervised-contrastive treatment of all same-unit codas.

Each view masks a random four-bin frequency band and eight-frame time band. Gaussian noise with SD 0.01 is added to standardized ICI entries corresponding to nonzero raw intervals. Padded intervals retain their standardized values. Current DCCE conditions use uniform shuffled sampling and 50 epochs of AdamW, learning rate 0.001, weight decay 0.0001, cosine decay, and batch size 64. These settings were fixed for the exploratory batch, not validation-tuned.

Supervision is explicit

U/T/I denote unit pairing, type loss, and ID loss. The main comparison, 110 versus 111, differs only in ID loss. Both models are task-informed. Absence of identity supervision does not mean absence of all supervision.

The 000 representation objective and sampler use no task labels, but its ICI inputs still come from annotations and its downstream probes are supervised. Rhythm-only and spectrum-only controls place both auxiliary heads on the available branch. Joint concatenation retains 128 dimensions and removes the fusion MLP; it is not independently trained late fusion.

Reference models and input access

Raw ICI provides a timing reference. Temporal mel features comprise means and standard deviations within four valid-frame bins, giving 512 dimensions. Concatenating nine ICI entries gives a 521-dimensional reference. Temporal binning retains coarse timing that a single time-averaged mel vector would discard.

WhAM, AVES, and BirdNET provide pretrained audio representations. Each is evaluated alone and with appended ICI. The latter controls match access to annotated timing, with only downstream probes trained locally.

Native frontend procedures are retained: AVES at 16 kHz over the full segment, WhAM at 44.1 kHz with mono/loudness processing, and BirdNET at 48 kHz with a three-second window. Three available waveforms exceed that BirdNET window.

Evidence: code package, supervision matrix, and evaluation protocol. The methodological descriptions on this page retain substantial wording from the final manuscript.