🛠️ FrontierHarness Eval – 9 harness, same model, cost per pass varies 17x
Summary
FrontierHarness Eval evaluates nine different harnesses using the same model. It demonstrates that the cost per pass can vary by seventeen times.
Why it’s interesting
It highlights a massive seventeenfold variance in cost per pass across different evaluation harnesses using the exact same model.
Source metrics: Points 22 · Comments 4
HN discussion · Project
Source: #HackerNews / Show HN
Summary
FrontierHarness Eval evaluates nine different harnesses using the same model. It demonstrates that the cost per pass can vary by seventeen times.
Why it’s interesting
It highlights a massive seventeenfold variance in cost per pass across different evaluation harnesses using the exact same model.
Source metrics: Points 22 · Comments 4
HN discussion · Project
Source: #HackerNews / Show HN