A caption passes through the judge and splits three ways: correct, omission and hallucination (generated with [ChatGPT Images](https://chatgpt.com/))

CaptionExam: A Challenging QA Benchmark for Captioners

On CaptionExam, even GPT-6 Astra only just passes at describing images, with a score of 61.2%.

 · 10 min · 2045 words · Elliot Shichao Song