A new benchmark called VisionQ evaluates vision-language models on criterion-conditioned visual discrimination, using 1,800+ figures from peer-reviewed CVPR and ICCV papers to ground judgments in…

#VisionLanguageModels #ComputerVision #Benchmark #AIResearch
https://arxiv.org/abs/2610.00666

Incident-Arena is a new benchmark of 20 tasks designed to evaluate AI coding agents on real-world production incident response using Kubernetes clusters and sustained load. The authors highlight limitations in current agentic SRE…

#AI #SRE #AIAgents #Benchmark
https://arxiv.org/abs/2610.00648

A new VAmoS Energy benchmark stress-tests voice agents across 100 utility billing calls with multiple requests, background speech, and verification hurdles. Fourteen stacks scored 17.3%–44.7% completion, with Grok Voice leading.…

#VoiceAI #LLMAgents #Benchmark
https://arxiv.org/abs/2609.38512

More from today ❤️ #RamsdenClough #HolmeMoss
#BenchMark

Gemini 4 Argon wins benchmarks—but employees say it struggles in real-world coding. #AI #Gemini4 #Benchmark #Coding #OpenAI #Tech https://thedailytechfeed.com/gemini-4-argon-draws-congrats-and-concerns-over-real-world-results/

DexHoldem is a new real-world benchmark using a ShadowHand to test dexterous manipulation in Texas Hold'em, with 1,470 teleoperated demonstrations across 14 primitives and an agentic perception task for…

#Robotics #DexterousManipulation #EmbodiedAI #Benchmark
https://arxiv.org/abs/2605.18727

AI Overviews : leur présence triple sur les requêtes de marque DemandSphere observe que les AI Overviews ont triplé sur les requêtes de marque suivies en septembre.

https://positionzero.net/ai-overviews-leur-presence-triple-sur-les-requetes-de-marque/

#Overviews #Benchmark

MM-FinEval is a new benchmark that tests multimodal LLMs on financial forecasting using 2,045 S&P 500 earnings call transcripts, presentation slides, and audio from 2019 to 2022 across 12 financial tasks.

#AI #DataInfrastructure #FinAI #Benchmark
https://arxiv.org/abs/2609.38523

LLMs can rival expert humans on a new model-training intuition benchmark, but still lag in architecture-specific questions and lack structured reasoning.

#AIresearch #LLMs #MachineLearning #Benchmark
https://arxiv.org/abs/2609.39714

A new benchmark called DoGBENCH evaluates whether agents can produce user-facing software documentation that experienced technical writers would accept, covering open source projects such as Helm, PostHog, and Mautic.

#opensource #AI #documentation #benchmark
https://arxiv.org/abs/2609.39909

New arXiv benchmark Zero2Repo asks coding agents to build whole repositories from a PRD, interface contract, and empty workspace, using language-agnostic tasks derived from real version-pinned open-source projects with hidden…

#opensource #AI #devtools #benchmark
https://arxiv.org/abs/2609.38269

A new benchmark called RobotValues introduces 8K value-conflict household scenarios to evaluate how robot planners prioritize competing values like autonomy, efficiency, and social appropriateness. Testing 10…

#Robotics #AI #MachineLearning #Benchmark
https://arxiv.org/abs/2606.03312

CTR Google : le desktop recule, le mobile progresse au deuxième trimestre Au deuxième trimestre 2026, le taux de clic des deux premières positions Google recule sur ordinateur…

https://positionzero.net/ctr-google-le-desktop-recule-le-mobile-progresse-au-deuxieme-trimestre/

#Overviews #Benchmark

A new arXiv benchmark tests whether AI agents can tell apart laws they inferred from evidence versus ones they merely recognize, with early results showing a split between predictive success and mechanism recovery.…

#AIResearch #Benchmark #AIInfrastructure #GPUs
https://arxiv.org/abs/2609.36726

BIABench introduces a 16-task benchmark for evaluating AI agents on end-to-end bioimage analysis, drawn from published biological studies with paired raw images and peer-reviewed ground truth. The benchmark spans modalities from…

#AI #Benchmark #Bioimaging #arXiv
https://arxiv.org/abs/2609.34274

DISCERN is a new arXiv benchmark testing whether AI agents can vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence across three workflow levels. It aims to evaluate automated research…

#AI #DataScience #Research #Benchmark
https://arxiv.org/abs/2609.33357

arXiv:2609.32020 introduces an NGSS-aligned benchmark for evaluating LLMs on middle and high school science content, tested across nine open-weight models. Several smaller, locally deployable LLMs performed strongly across diverse…

#AI #EdTech #LLMs #Benchmark
https://arxiv.org/abs/2609.32020

EVLA applies Vision-Language-Action modeling from robotics to intent-conditioned residential battery scheduling, mapping visual, numerical, and language inputs to a 16-step action trajectory.

#EVLA #EnergyAI #Robotics #Benchmark
https://arxiv.org/abs/2609.31648

JRDB-AVR introduces a benchmark for active visual reasoning in embodied agents, requiring them to request specific timestamped observations and viewing angles rather than relying on passive input. This shifts…

#Robotics #EmbodiedAI #ActiveReasoning #Benchmark
https://arxiv.org/abs/2609.35032