Day 74/100 of #100DaysWithAI 🔁

14 LLMs, 5 runs each: 90-98% perfect self-agreement. On actual market moves, all at chance. Consistent is not the same as correct.

🔗 https://github.com/aurimas13/100-Days-With-AI

#LearningInPublic #AIEvals

🤝 Claude Code-assisted, reviewed by me.

Day 73/100 of #100DaysWithAI 🎲

An LLM judge that prints "4" discards how sure it was. G-Eval weights it by token log-probs - the doubt becomes part of the number.

🔗 https://github.com/aurimas13/100-Days-With-AI

#LearningInPublic #AIEvals

🤝 Claude Code-assisted, reviewed by me.

Day 72/100 of #100DaysWithAI 📏

Rating LLM output 1-5 feels rigorous. Annotators drift to the middle. Binary pass/fail forces the call - and names the failure.

🔗 https://github.com/aurimas13/100-Days-With-AI

#LearningInPublic #AIEvals

🤝 Claude Code-assisted, reviewed by me.

Support agreement with the locked reference answers:
Astra: 29/30 · $0.6904
Luna: 28/30 · $0.0184
Jev: 22/30 · $0.00124

Those are estimated costs for the full batch. Jev marked 2 of 12 unsupported claims as supported. Astra and Luna marked none of those 12 as supported. #AIEvals (3/6)

what i be doing at 3am:
discovering that Jev is better, faster, and cheaper than every LLM as an auto-mode classifier for agent harnesses like antigravity and opencode
🤖🧠 #MLSky #AIEvals #LLMs

What should every model evaluation record? Task type, context needs, capability fit, and reference price. TemprHQ helps you fill that worksheet across 407 models. #LLMEvals #AIEvals https://temprhq.io/models

Day 54/100 of #100DaysWithAI 🛡️

A zero in Anthropic's own benchmark table is not a failure. It marks where the safeguards fired - the price of caution, published.

🔗 https://github.com/aurimas13/100-Days-With-AI

#LearningInPublic #AIEvals

🤝 Claude Code-assisted, reviewed by me.