Terminal-Bench-Science: Evaluating AI agents on scientific research workflows

4 comments

• akshay_akula 57 minutes ago
  Evals on actual research workflows is the right direction, most agent benches are toy tasks.
  [-]
• mlmonkey 5 minutes ago
  Sad to see no mention of Gemini ...
• rubslopes 1 hour ago
  I'm glad to see that GPT Sol beats Opus at least in Mathematical Sciences, because that's my need right now, and I much prefer GPT's prose style.
• vatsachak 53 minutes ago
  Damn. These things aren't AGI... but I don't care.

  Luna is good enough for me to give a parser spec and have it write one.