XB XKCD-Bench v1
BLIND HUMOR & REVERSE COMIC BENCHMARK

Can AI actually get the joke, or just recite the punchline?

We gave 15 frontier models raw XKCD strips and hover text with zero spoilers, then graded their explanations against human ExplainXKCD contributors using TypeSafe System One (Noul). Compare how Gemini 4 Argon, Claude Opus 5.5, Claude Fable 5.1, Gemma 4 31B, and GPT-6 Luna break down the humor, or test reverse comic search across 891 strips.

HF Corpus 891 2,630 ExplainXKCD × AltTitle
Live Blind Evals 120 15 Models × 8 Comics + 30 Queries
Top Explain (#1) 70.5% Gemini 4 Argon
Reverse Hit@1 93.3% 30 Queries / 891 Strips
#327 Exploits of a Mom
Computer Science & Security Standard
xkcd.com
XKCD Comic
Official Title-Text (Mouseover Alt)
Her daughter is named Help I'm trapped in a driver's license factory.
Select Evaluated Model Judge: TypeSafe Noul + ExplainXKCD Reference
Paste or Write an Explanation to Grade Against ExplainXKCD
73.8
Composite
Gemini 4 Argon
Humor & Punchline 84.2%
Domain Accuracy 71.3%
Visual Grounding 60.1%
Alt-Text Synthesis 59.2%
Canonical Concept Recall 5/6 Concepts Captured (83%)

Track 1: LLM Comic Explanation Leaderboard (Explain-Bench)

LIVE BLIND PNG + ALT-TEXT EVAL

Real empirical outputs across 15 frontier models judged by TypeSafe System One (Noul) + ExplainXKCD ground-truth reference across 8 canonical strips. Click any model row to inspect its explanations in the Arena.

Rank Model Modality Composite Score Humor & Punchline Domain Depth Visual Grounding Alt-Text Synthesis Concept Recall Avg Latency

Per-Comic Live Evaluation Matrix (Blind Multimodal Runs)

Composite scores per canonical benchmark strip across all 6 technical domains. Click any cell to inspect that model's exact blind explanation & rubric breakdown in the Explanation Arena.

Model #327Bobby Tables #2347Dependency #2630Shuttle Skel #927Standards #386Duty Calls #1838Machine Lrn #882Significant #1597Git MeanComposite

Track 2: Reverse Comic Search Leaderboard & Live Evaluation Sweep

Evaluated across 30 ground-truth queries (including 10 hard zero-title-word & alt-text paraphrase queries) over the 891-comic cross-joined HF corpus.

Rank Retrieval Engine Architecture Hit@1 (Top-1) Hit@3 Hit@5 MRR@10 Calibration ECE P50 Latency
Showing 891 comics
SYSTEMS ARCHITECTURE BLUEPRINT · XKCD-BENCH V1

Two-Track Blind Evaluation & Tri-Modal Retrieval Topology

How 891 cross-joined Hugging Face comics flow through zero-spoiler multimodal prompting, hybrid TypeSafe System One (Noul) + deterministic ExplainXKCD grading, and tri-modal Reciprocal Rank Fusion.

XKCD-Bench Two-Track Evaluation and Retrieval Architecture Architecture diagram showing 891 Hugging Face XKCD comics feeding Track 1 blind multimodal LLM evaluation judged by a hybrid TypeSafe Noul and ExplainXKCD rubric, and Track 2 reverse comic search powered by BM25F, 64-bit dHash, and TypeSafe Jev reranking. 01 · CORPUS & BLIND INPUTS 02 · EVALUATION & RETRIEVAL CORE 03 · TELEMETRY & WORKSPACES PNG + ALT EXPLAIN REF 30 QUERIES 4 PILLARS HIT@1 · MRR BLIND INPUT Raw Strip + Hover Text 8 Canonical Comics · No Spoilers HF CORPUS 891 Cross-Joined Strips ExplainXKCD × AltTitle Topics SEARCH EVAL 30 Retrieval Queries Memory · Concept · PNG Upload VISION + TEXT GEN 15 Frontier Multimodal & Reasoning Models Gemini 4 Argon · Opus 5.5 · Fable 5.1 · Gemma 4 · 4096 Tok HYBRID JUDGE CORE 40% TypeSafe Noul + 60% Deterministic Rubric Humor 30% · Domain 30% · Visual 25% · Alt 15% · Stem Recall TRI-MODAL SEARCH BM25F Lexical + 64-Bit dHash + TypeSafe Jev Field-Weighted Index · Hamming Similarity · Zero-Shot LLM STATE STORE Live Eval Telemetry live_eval_results.json · 120 Runs TRACK 1 VIEW Leaderboards & Arena 6 Domains · SVG Medals · Matrix TRACK 2 VIEW Reverse Search Lab 100% Hit@1 · Live 30-Query Sweep LEGEND Synchronous Eval / Search Pipeline Focal Hybrid Rubric Judge (Noul + Deterministic) Telemetry Persistence & Hydration
TRACK 01 · EXPLAIN-BENCH PROTOCOL

Blind Multimodal Prompting & Hybrid Rubric

Every model is evaluated blind: vision models receive only the raw comic .png bytes and official hover alt text (max_tokens=4096, temperature=0.2), with zero access to ExplainXKCD ground truth.

Composite Score Formula:
0.30 × Humor + 0.30 × Domain + 0.25 × Visual + 0.15 × Alt-Text
Per-Pillar Calibration:
40% TypeSafe System One (Noul) + 60% Deterministic Reference (Stem Concept Recall + Lexical F1 @ 115 tokens)
TRACK 02 · REVERSE COMIC SEARCH RRF

BM25F + 64-Bit dHash + TypeSafe Jev Rerank

Reverse search resolves vague human memory queries ("Bobby Tables SQL injection", "whale bones inside space shuttle") and cropped comic panels across all 891 strips in three fused stages:

  • 1. BM25F Index: Field-weighted lexical retrieval (title ×4.0, concepts ×3.5, alt ×2.5, transcript ×2.0, explanation ×1.2).
  • 2. 64-Bit dHash: 9×8 LANCZOS grayscale luminance gradient hashing with normalized Hamming similarity (1 − dH / 64).
  • 3. TypeSafe Jev + RRF: Parallel System One binary judgment reranking fused via Reciprocal Rank Fusion (k = 60), achieving 100.0% Hit@1 across all 30 queries.
CORPUS & TAXONOMY · 891 STRIPS

Cross-Joined Hugging Face Datasets

Built by cross-joining olivierdehaene/xkcd (2,630 ExplainXKCD crowd-annotated strips) with devparagiri/dataset-xkcd (curated topic tags and hover titles) into 891 verified local comics partitioned across 6 technical domains:

CS & Security
188 strips · #327, #1597
AI & ML
65 strips · #1838
Physics & Space
169 strips · #2630
Math & Stats
99 strips · #882
Workflow
121 strips · #2347, #927
Science & Culture
249 strips · #386