gekro
GitHub LinkedIn
News

AI News

Real-World Coding Agent Benchmark Reveals 74% Human-Cleanup Rate; Google Releases Spec-First Agent Plugin

Alibaba's Co-Design Bench shows even top AI coding agents fail to fully solve three out of four real-world tasks; Google AI releases Elephant-Goldfish plugin for spec-driven code review.

1 min read 3 sources

Alibaba’s new Co-Design Bench coding-agent benchmark tested AI agents on real user requests that had previously caused failures in production. (The Decoder) Even GPT-6 Astra, the top performer, fully solved only 26.4 percent of tasks - a stark contrast to models scoring above 99 percent on older benchmarks. The gap highlights the persistence of real-world complexity in software development that standardized eval suites do not capture, suggesting that human oversight remains critical for production agent deployments.

Google AI released the Elephant-Goldfish Model plugin for coding agents, which enforces a spec-first workflow: agents draft specifications before generating code, then run five zero-context review stages to validate implementation. (MarkTechPost) Separately, Nous Research released Arete-27B, an Apache 2.0 fine-tune of Qwen3.8-27B that reduced propaganda output from 22 percent to 0.9 percent on an 803-prompt evaluation while maintaining general capability benchmarks. (MarkTechPost)

Compiled automatically from the linked sources and published without manual editing - a neutral summary of third-party reporting, for information only. Every claim links to its origin. Not original reporting.