@marcus-schuler.comThe SWE-Marathon outlier: Meta's Muse Wonder Full shows 57.5% vs. Claude Opus 5.5 at 46.3%, but Meta flags the comparison as uncontrolled. Different benchmarks, different stories. #AIevals #transparency https://www.implicator.ai/meta-muse-wonder-ranks-below-claude-opus/
@reksas13.bsky.socialDay 74/100 of #100DaysWithAI 🔁
14 LLMs, 5 runs each: 90-98% perfect self-agreement. On actual market moves, all at chance. Consistent is not the same as correct.
🔗 https://github.com/aurimas13/100-Days-With-AI
🤝 Claude Code-assisted, reviewed by me.
@reksas13.bsky.socialDay 73/100 of #100DaysWithAI 🎲
An LLM judge that prints "4" discards how sure it was. G-Eval weights it by token log-probs - the doubt becomes part of the number.
🔗 https://github.com/aurimas13/100-Days-With-AI
🤝 Claude Code-assisted, reviewed by me.
@reksas13.bsky.socialDay 72/100 of #100DaysWithAI 📏
Rating LLM output 1-5 feels rigorous. Annotators drift to the middle. Binary pass/fail forces the call - and names the failure.
🔗 https://github.com/aurimas13/100-Days-With-AI
🤝 Claude Code-assisted, reviewed by me.
@yianna.bsky.socialSupport agreement with the locked reference answers:
Astra: 29/30 · $0.6904
Luna: 28/30 · $0.0184
Jev: 22/30 · $0.00124
Those are estimated costs for the full batch. Jev marked 2 of 12 unsupported claims as supported. Astra and Luna marked none of those 12 as supported. #AIEvals (3/6)
@presidentgoku.bsky.social
@temprhq.bsky.socialWhat should every model evaluation record? Task type, context needs, capability fit, and reference price. TemprHQ helps you fill that worksheet across 407 models. #LLMEvals #AIEvals https://temprhq.io/models
@reksas13.bsky.socialDay 54/100 of #100DaysWithAI 🛡️
A zero in Anthropic's own benchmark table is not a failure. It marks where the safeguards fired - the price of caution, published.
🔗 https://github.com/aurimas13/100-Days-With-AI
🤝 Claude Code-assisted, reviewed by me.