Swaraga

Empirical benchmark evaluating Google DeepMind EmbeddingGemma 2 (onnx-community/embeddinggemma-2-ONNX, Gemma 4 architecture, selective 570M Text 270M + Audio 300M in q4) across 9 architectural configurations on N = 46 audio recordings (18 human vocal clips across 3 tonic keys + 28 multi-harmonic performances across canonical Sa = D3 and +600¢ tritone-shifted Sa = G#3, including 3 shared-swara sibling pairs) against both the 14-Raga Concert Pool and the Full 962-Raga Catalog on Apple M5 (arm64, 10 cores).

SWARAGA FULL PIPELINE (YIN F0 + JEV)
93.5% Top-1 (14) · 84.8% (962)
95% CI [87.0%, 100%] · Macro-F1 95.6% · 367 ms p50

EMBEDDINGGEMMA 2 AUDIO k-NN (300M)

58.7% 75.0% Canonical → 46.2% Shifted

−28.8% Tonic-Shift Drop · 94.4% Same-Key Vocal · 2,087 ms p50

EMBEDDINGGEMMA 2 HYBRID MRL (768d / 512d)

47.8% Top-1 · 73.9% Top-3 Recall

512d MRL 100% Lossless · −3.8% Tonic Drop · 624 ms p50

EMBEDDINGGEMMA 2 ZERO-SHOT (AUDIO→TEXT)

6.5% Top-1 (14) · 0.0% (962)

Random Chance = 7.1% · No Sargam Interval Isomorphism

FOCUS SLICE:

Engine & Architecture	Modality & Params	Top-1 (14) & 95% CI	Top-3 (14)	962-Raga (Top-1 / Top-5)	Macro-F1	Canonical Sa	Transposed Sa (Δ)	Sibling Pairs	Latency p50

LIVE SIDE-BY-SIDE RAGA PROBE · EMBEDDINGGEMMA 2 COSINE VS. SYSTEM ONE

Inspect Real Cosine Similarities, MRL Truncation & Bi-Encoder Negation Collisions

01 TONIC INVARIANCE ABLATION (Δ = −28.8%)

Why Raw Audio Embeddings Break Under Key Shifts

Gemma4AudioFeatureExtractor tokenizes 16 kHz audio via 128-bin HTK mel-spectrograms (0–8,000 Hz, 20 ms frame, 10 ms hop, 25 tokens/s). In Indian classical music, a raga is defined by relative frequency ratios to Adhara Shadja (Sa), not absolute pitch. Shifting tonic by a tritone (D3 = 146.8 Hz → G#3 = 207.7 Hz) translates every harmonic by √2 across mel bins, dropping EmbeddingGemma 2 Audio k-NN from 75.0% → 46.2% (−28.8%), whereas YIN circular cent folding 1200 log₂(f₀/f_Sa) mod 1200 retains 100% → 88.5%.

02 THE BI-ENCODER NEGATION PARADOX

Why "Omits M2 & N3" Attracts M2 & N3 Queries

In a dense 768-d mean-pooled bi-encoder, listing forbidden Varjya swaras in a raga's text card (e.g., "Raga Bhupali omits Teevra Madhyam M2 and Shuddha Nishad N3") injects positive token embeddings for M2 and N3 into Bhupali's vector—causing Bhupali (0.8516) to outscore Yaman (0.8487) on heptatonic Yaman queries. Removing negated Varjya text and encoding only positive spelled-out swaras lifts Yaman to #1 (0.8784), proving why Swaraga enforces an algebraic −1.15 × VarjyaLeakage penalty.

03 MATRYOSHKA MRL & L2 RENORMALIZATION

768d → 512d Lossless · 256d (1:3) · 128d (1:6)

EmbeddingGemma 2 is trained with Matryoshka Representation Learning (MRL). Slicing a unit vector v[0:d] breaks unit norm (‖v[0:d]‖₂ < 1), so L2 renormalization v̂ = v[0:d] / ‖v[0:d]‖₂ is mandatory before dot-product scoring. Empirically, 512d (1:1.5 compression) is 100% lossless on the 14-raga pool (47.8% Top-1, 73.9% Top-3, 0.626 MRR), 256d (1:3) retains 67.4% Top-3 recall (91.2% retention), and 128d drops to 56.5% Top-3.

04 HONEST ARCHITECTURAL TIERING

Bi-Encoder Pruner (73.9% Top-3) vs. System One Executor (93.5%)

Shared-swara sibling ragas—Bhupali vs. Deshkar (both S R2 G3 P D2, Vadi G3 vs. D2) and Miyan ki Todi vs. Multani (both S r1 g2 M2 P d1 N3)—score 40.0%–53.3% in EmbeddingGemma 2 because single-vector pooling compresses sequential phrase order and dwell hierarchy. By pairing Stage 1 deterministic pruning with Stage 2 TypeSafe System One (choice, score, noul), Swaraga resolves sibling pairs at 93.3% Top-1 in 367 ms.

PROGRAMMABLE MUSICOLOGY · TYPESAFE SYSTEM ONE

Drop an MP3 or sing a Raga.

Upload a vocal clip, record 8 seconds from your microphone, or pick a classical recording below. Each analysis card renders progressively.

Drop an .mp3 / .wav recording here, or sing into the microphone

Decoded locally at 16 kHz via Web Audio API · YIN F0 pitch contour + 10 parallel Jev judgments

OR SELECT A CLASSICAL VOCAL RECORDING:

Sa Tonic:

01 IDENTIFIED RAGA AUTO_VERIFIED

HINDUSTANI & CARNATIC DUAL EQUIVALENCE

—

Carnatic Equivalent: —

PROBABILITY —

CONFIDENCE —

BEAM SCORE —

PARENT THAAT —

JATI & VADI / SAMVADI —

PRAHAR (SAMAY CHAKRA) —

AROHA / AVAROHA —

02 F0 PITCH CONTOUR & SWARASTHANA TRAJECTORY 0 voiced frames

DETECTED AROHA: —

DETECTED AVAROHA: —

TOP N-GRAM MOTIFS: —

03 12-SWARA DWELL HISTOGRAM (VADI / SAMVADI / VARJYA) Vadi: —

04 CANDIDATE RAGA PROBABILITIES choice()

Calibrated probability distribution over candidate ragas combined with hierarchical parent Thaat beam score √(P(Thaat) × P(Raga)).

05 MUSICOLOGICAL RUBRICS & GRAMMAR VERIFIERS score() + noul()

Continuous Rubric Expectations (0.00 – 3.00)

Calibrated Grammar Verifiers (0.00 – 1.00)

06 INSPECT RAW SYSTEM ONE JSON PAYLOAD & RAGA GRAMMAR CATALOG Expand ▾

POST https://api.typesafe.ai/v1/systemone (model: "jev-latest")

REQUEST: STATE + 10 QUESTIONS

Loading...

RESPONSE: CALIBRATED ANSWERS

Loading...