Housekeeping: I’m on the road so the voiceover will be late.
Hill-climbing is easy, knowing what to hill-climb is hard. Making scores go up is expensive, but we have a much clearer playbook for it than we do for creating meaningful benchmarks at the true frontier of AI today.
Independent evaluation of language models is under-appreciated in how hard of a …