# Numerical audit of the conference manuscript

Date: 2026-09-20. Manuscript: `main.tex` and its two included result tables.

## Outcome

No numerical discrepancies were found in the checks below. The manuscript's experimental scores are traceable to saved experiment records; the main tables are independently reproducible from saved predictions. This is evidence of numerical consistency, not a guarantee that every upstream processing decision or scientific interpretation is correct.

No models or probes were rerun. No manuscript, table, figure, or experimental result was changed by this audit. The existing PDF remains five pages (four content pages plus references).

## What was checked

- **792 task-prediction CSVs across 264 result records:** independently calculated macro-F1 using class counts, without importing the experiment's scoring function. Both supported-example primary scores and full-eligible secondary scores match their saved metrics to an absolute tolerance of 1e-12.
- **426 summary mean/SD pairs:** independently recomputed three-run means and sample standard deviations from the prediction-derived scores. All match the saved summaries.
- **96 displayed mean ± SD entries:** all entries in Tables 1 and 2 match the independently recomputed results at the printed three-decimal precision. Current DCCE entries use the matched uniform-sampling runs, not pooled historical regimes.
- **24 Figure 2 data points:** recorded plotting values match the recomputed means and SDs.
- **27 additional numerical prose values:** full-eligible type scores, branch diagnostics, pairing/swap/bandwidth contrasts, and eight DCCE year-holdout scores match the underlying predictions. Main results repeated in the prose agree with the tables.
- **Dataset accounting:** 1,501 files minus 118 noise records gives 1,383 clean codas; three units and 22 types. Identity eligibility gives 754 codas from 11 identities; exclusions are 620 unknown, eight ambiguous, and one singleton.
- **Identity support:** grouped folds contain 186/213, 111/121, and 145/145 supported/eligible examples, with three, six, and six scored identities, as stated.
- **Year analysis:** reconstructed the year-by-unit contingency table from labels. Pearson chi-square is 530.0435927400224, and uncorrected Cramér's V is 0.43775358015833493, correctly printed as 530.04 and 0.438. Unit F has 821/1,383 examples, correctly printed as 59.4%. Frozen AVES exceeds both reported DCCE variants on unit classification in all four year holdouts.
- **New type diagnostic:** independently reproduced the `1+1+3` identity scores 0.000/0.233/0.234, counts 52/53/34, and identity counts 3/3/4.
- **Partition integrity:** saved train/validation/test indices are disjoint and cover the dataset; grouped partitions do not share dates; year holdouts exclude the test year from training and validation. Prediction IDs and true-class encodings match the reconstructed label ordering.
- **Training and selection records:** recorded encoder-training IDs match the training partitions; checkpoint files and finite epoch histories exist. Applicable current DCCE runs record uniform sampling and 50 epochs; AVES adaptation records 10 epochs. Selected layers maximize the saved validation scores with earliest-layer tie breaking. The checked validation trials record no convergence warnings. Recorded split hashes and the checked experiment entry-point source hashes match the current files.

## What “rerun” means here

The retained archive contains 240 primary/control result records and 24 separate branch-probe records. Among the 240, 158 have local neural-training records and 82 use frozen or fixed representations. These totals include historical comparison runs and year holdouts; they are not 240 models in the final paper.

- DCCE and adaptation variants have saved local training histories, checkpoints, and held-out predictions.
- Frozen WhAM, AVES, and BirdNET evaluations reuse pretrained embedding caches and evaluate downstream probes. They do not constitute retraining those foundation models.
- Raw ICI and temporal-mel controls use fixed features with trained probes.
- Branch diagnostics probe already-trained representations; they do not retrain the encoders.
- This audit recalculated metrics from saved predictions. It did not regenerate embeddings, run checkpoints on audio, or refit any probe.

## Results and interpretation

- DCCE U+T's random-split unit/type/identity means of **0.901/0.865/0.597** are supported by the saved predictions. U+T+I gives **0.907/0.735/0.581**. Adding the ID loss does not improve the reported mean fused-ID result, although its spectral branch improves from **0.367 to 0.443**. These are different outputs and the values are consistent.
- Raw ICI's **0.987** random type score is real in the saved evaluation. Because type categories reflect timing structure, it should be read as a strong annotation-based reference rather than evidence of waveform-only type recognition.
- The simple ICI + temporal-mel identity reference scores **0.724**, above both main DCCE variants. The revised paper correctly does not claim overall DCCE superiority.
- Lower date-grouped scores are consistent across the four plotted models. Identity class support also changes between protocols, so the gaps cannot be interpreted as a pure recording-context effect.
- Reported uncertainty is **sample SD across three runs**, not a confidence interval or a significance test. Rounded means do not establish reliable superiority between close models.

## Verification limits

- The separate **within-unit** probes retain run-level scores, but no per-example predictions were located. Their three-run averages reconcile exactly: raw ICI **0.384**, DCCE U+T **0.267**, and DCCE U+T+I **0.238**. These three means have weaker verification than the main tables: their underlying macro-F1 values were not independently recomputed. Verifying those more deeply would require rerunning probes, which was outside the authorized scope.
- Training logs and checkpoints support provenance, but this audit does not independently replay training or establish that each checkpoint generated each saved prediction.
- Checks against the reconstructed labels do not independently validate the original audio-to-annotation mapping. External foundation-model pretraining membership also remains unresolved.
- Literature numbers belong to the separate citation audit; they are not outputs of our experiments.
- This audit covers result arithmetic, provenance records, and the stated dataset accounting. It is not a fresh implementation audit of every architecture or preprocessing hyperparameter.

## Reproducible audit artifacts

- `qa/audit_saved_numbers.py`: read-only audit of experiment artifacts; writes only its audit JSON.
- `qa/numerical_audit.json`: detailed prediction/table ledger, source paths, full-precision numbers, support counts, and manuscript SHA-256.
- Run from the project root: `python3 paper_conference/qa/audit_saved_numbers.py`.

Manuscript SHA-256: `9d34e66d40ae29ad88cdc9379972ca0a50e95f26115b42aa4c0f50212075ed8c`.