The number was green, so the agent stayed quiet. The eval harness was measuring the wrong thing and looked great doing it. #LLM #AIEngineering
Audited the tests my coding agent wrote last month: 19 of 60 mocked the very function they claimed to cover. Green, fast, and proving nothing. New rule: mock the network, the clock, and the filesystem. Nothing else. #claudecode #testing #aiengineering #buildinpublic
