Eval Harness Reveals LLM Overconfidence When Wrong

A new eval harness framework demonstrates that models are most confident when wrong, a critical insight missed by qualitative review. Provides practical methods for building eval harnesses with synthetic ground truth and scoring functions. Directly challenges fluency-as-correctness assumptions in production deployments.