Executed revision notebook: saved outputs; no cells run by this page. Read the research record · All notebooks

V2 — data audit and shared splits

New notebook for the corrected revision. Original notebooks remain unchanged. These cells inspect the completed artifacts and run read-only regression checks; they do not regenerate features or train models.

from pathlib import Path
import json, os, subprocess, sys
import pandas as pd
from IPython.display import display, Image, Markdown
ROOT = Path.cwd()
if not (ROOT / 'scripts_v2').is_dir():
    ROOT = ROOT.parent
assert (ROOT / 'scripts_v2').is_dir(), 'Open this notebook from the project or notebooks_v2 folder.'
RESULTS = ROOT / 'results_v2'
PYTHON = ROOT / '.venv' / 'bin' / 'python'
sys.path.insert(0, str(ROOT / 'scripts_v2'))
ENV = dict(os.environ, OPENBLAS_NUM_THREADS='1', OMP_NUM_THREADS='1')
def command(script, *args):
    result = subprocess.run([str(PYTHON), str(ROOT / 'scripts_v2' / script), *args],
                            cwd=ROOT, env=ENV, capture_output=True, text=True)
    print(result.stdout)
    if result.stderr:
        print(result.stderr)
    result.check_returncode()
print('Canonical results:', RESULTS)
print('Training is disabled by default; existing artifacts are read below.')

Canonical results: /Users/joaoquintanilha/Downloads/data_297_final_paper/results_v2
Training is disabled by default; existing artifacts are read below.

Data eligibility

Ambiguous IDs and the singleton are excluded only from identity training/scoring. All 1,383 clean codas remain available for unit/type tasks.

audit = json.loads((RESULTS / 'data_audit.json').read_text())
print('Clean codas:', audit['clean'])
print('Eligible identity codas:', audit['id_eligible'])
print('Identity classes:', len(audit['classes']['individual_id']))
display(pd.Series(audit['id_exclusions'], name='codas').to_frame())

Clean codas: 1383
Eligible identity codas: 754
Identity classes: 11

	codas
eligible	754
unknown	620
ambiguous	8
singleton	1

Corrected year counts

Explicit day/month/year parsing retains all records. Association motivates controlled evaluation; it does not prove that models rely on recording artifacts.

display(pd.read_csv(RESULTS / 'year_unit_counts.csv', index_col=0))
print('Chi-squared:', audit['chi_squared'], 'df:', audit['chi_squared_df'])
print("Cramér’s V:", audit['cramers_v'])

	A	D	F
year
2005	10	0	358
2008	27	171	65
2009	30	18	47
2010	174	132	351

Chi-squared: 530.0435927400224 df: 6
Cramér’s V: 0.43775358015833493

Shared split membership and identity support

Three ordinary splits and the first three distinct folds of one fixed five-fold recording-date partition. A date is a session proxy. Every task inherits the same coda membership. Unseen test identities are reported explicitly.

display(pd.read_csv(RESULTS / 'split_support.csv'))
display(pd.read_csv(RESULTS / 'task_support.csv'))

	protocol	seed	train	validation	test	id_train_classes	id_test_eligible	id_test_supported	id_test_supported_classes	unseen_ids
0	date_grouped	0	828	227	328	9	213	186	3	5719, 5720
1	date_grouped	1	901	211	271	7	121	111	6	5973, 6069
2	date_grouped	2	883	239	261	7	145	145	6	NaN
3	random	0	884	222	277	11	151	151	11	NaN
4	random	1	884	222	277	11	151	151	11	NaN
5	random	2	884	222	277	11	151	151	11	NaN

	protocol	seed	task	n_test_eligible	n_test_supported
0	date_grouped	0	unit	328	328
1	date_grouped	0	coda_type	328	326
2	date_grouped	0	individual_id	213	186
3	date_grouped	1	unit	271	271
4	date_grouped	1	coda_type	271	271
5	date_grouped	1	individual_id	121	111
6	date_grouped	2	unit	261	261
7	date_grouped	2	coda_type	261	261
8	date_grouped	2	individual_id	145	145
9	random	0	unit	277	277
10	random	0	coda_type	277	276
11	random	0	individual_id	151	151
12	random	1	unit	277	277
13	random	1	coda_type	277	277
14	random	1	individual_id	151	151
15	random	2	unit	277	277
16	random	2	coda_type	277	277
17	random	2	individual_id	151	151

Feature provenance

New mel arrays live in results_v2/features. Original feature arrays in datasets are untouched.

features = json.loads((RESULTS / 'features' / 'config.json').read_text())
print(json.dumps(features['config'], indent=2))
print('Truncated clips:', features['truncated'])
print('Native sample rates:', features['rates'])

{
  "sample_rate": 48000,
  "n_fft": 2048,
  "hop_length": 1152,
  "n_mels": 64,
  "fmin": 0,
  "fmax": 8000,
  "n_frames": 256,
  "center": true,
  "window": "hann",
  "top_db": 80.0,
  "padding_db": -80.0,
  "res_type": "soxr_hq"
}
Truncated clips: 0
Native sample rates: {'44100': 360, '48000': 780, '96000': 41, '120000': 202}

Regression checks

These checks reject overlapping memberships, verify distinct grouped folds, enforce training-only partners, and demonstrate that altering test labels/features cannot affect layer selection.

command('test_revision_pipeline.py')

test_ambiguous_and_singleton_id_are_not_targets (__main__.RevisionEvaluationTests.test_ambiguous_and_singleton_id_are_not_targets) ... ok
test_dates_are_not_silently_lost (__main__.RevisionEvaluationTests.test_dates_are_not_silently_lost) ... ok
test_every_task_inherits_disjoint_manifest (__main__.RevisionEvaluationTests.test_every_task_inherits_disjoint_manifest) ... ok
test_grouped_runs_hold_out_distinct_dates (__main__.RevisionEvaluationTests.test_grouped_runs_hold_out_distinct_dates) ... ok
test_layer_selection_cannot_read_test_labels_or_features (__main__.RevisionEvaluationTests.test_layer_selection_cannot_read_test_labels_or_features) ... ok
test_overlap_is_rejected (__main__.RevisionEvaluationTests.test_overlap_is_rejected) ... ok
test_partners_stay_in_training_unit_and_exclude_anchor (__main__.RevisionEvaluationTests.test_partners_stay_in_training_unit_and_exclude_anchor) ... ok

----------------------------------------------------------------------
Ran 7 tests in 0.212s

OK

The registry snapshot is in registry_v2/pre_revision_materials/. Run/source hashes and exact training code are under results_v2/. See README_v2.md for the compatibility aliases retained after moving the completed batch.