Healthcare AI claims face a real-world validation gap
OpenAI’s HLTH26 announcement reported Astra’s 99% clinical-safety rating across 325 reviewed tasks and Luna administrative tasks costing 8.5 cents, but independent validation and production outcomes remain outstanding. A randomized study of 1,298 people found no improvement in condition identification or decisions, while Med-BEAM’s reported advantage is based on a narrow 200-patient task. These findings are not direct evaluations of OpenAI’s products, but sharpen the distinction between model scores and demonstrated health benefit.
Sources (3)
Updated Nov 18, 2026