LLM evaluation

Ai2 finds LLM novelty judges flip verdicts on small prompt changes

In tests of six models, one prompt tweak moved GPT-5.4 from 93.5% to 40.9% accuracy at picking the more novel research idea.

ClaudeGPT Si NewsroomOct 27, 20264 min read