Benchmark

Measured Run 20261105-0038-d71e · dataset k8s-eval-v0: 55 questions, 55 human-reviewed · snapshot k8s-20261030-878ad3 · generation gpt-6-luna · judge gpt-6-luna. Differences show paired bootstrap 95% confidence intervals.

MetricnBaselineIntermediateAdvancedIntermediate − BaselineAdvanced − Intermediate
Hit Rate@5460.8040.8480.935+0.043 (-0.065, 0.174)+0.087 (0.022, 0.174)
Recall@5460.7500.8010.902+0.051 (-0.065, 0.174)+0.101 (0.029, 0.188)
Recall@5 (multi-section)90.7780.6480.833-0.130 (-0.317, 0.000)+0.185 (0.000, 0.436)
Full Coverage@5 (multi-section)90.5560.4440.667-0.111 (-0.375, 0.000)+0.222 (0.000, 0.556)
MRR@10460.6360.6510.870+0.015 (-0.093, 0.122)+0.219 (0.134, 0.312)
nDCG@10460.5720.5960.735+0.024 (-0.047, 0.098)+0.139 (0.082, 0.201)
Answer Correctness460.8040.8040.913+0.000 (-0.136, 0.136)+0.109 (0.022, 0.208)
Faithfulness440.9920.9921.000-0.000 (-0.023, 0.022)+0.008 (0.000, 0.025)
Citation Correctness (provisional)250.6800.6430.821-0.037 (-0.273, 0.195)+0.179 (-0.101, 0.444)
Uncited Claim Rate680.6320.6000.611-0.032 (-0.149, 0.082)+0.011 (-0.117, 0.140)
Correct Abstention Rate90.8891.0001.000+0.111 (0.000, 0.364)+0.000 (0.000, 0.000)
False Abstention Rate460.0650.0650.022+0.000 (-0.087, 0.085)-0.043 (-0.132, 0.043)
Faithful but incorrect430.0230.0000.000-0.023 (-0.075, 0.000)+0.000 (0.000, 0.000)
Context tokens (mean)551,6841,1241,230-560.218 (-621.183, -500.273)+106.000 (46.343, 165.545)
Latency p50 (ms)551,9281,7403,385
Latency p95 (ms)554,6483,80368,473
p50 latency (ms)Recall@5baselineintermediateadvanced

Answer metrics are LLM-judged and validated against human labels on a sample: correctness 94% agreement (n=18), faithfulness 100% agreement (n=28), citation 32% agreement (n=28).