HakemBench is a new open Turkish benchmark for typed decisions, with 2,346 items and over 4,275 questions across seven tracks including fact-check triage, moderation, and customer support. It scores decision quality,…

#HakemBench #NLP #AIbenchmarks #LLMeval
https://arxiv.org/abs/2610.02293

An evaluation of TypeSafe AI's System One model Jev benchmarks it on 37 datasets across classification, routing, NLI, moderation, and rubric scoring, with the authors noting under USD 10 in API cost for 346,009…

#AIbenchmark #LLMeval #datainfrastructure #mlresearch
https://arxiv.org/abs/2609.37647

Research shows Deepseek-R1-Distill and similar open-source reasoning models can swing benchmark results significantly based on subtle evaluation conditions, making claimed gains hard to reproduce. The authors call for a…

#OpenSourceAI #LLMEval #AIResearch #DeepSeek
https://arxiv.org/abs/2506.04734