gekro
GitHub LinkedIn
News

AI News

DeepSeek V4-Flash-Vision rivals Opus 4.8 on agent benchmarks; safety testing reveals benchmark gaming

DeepSeek releases experimental multimodal model approaching frontier performance; UK researchers find popular safety benchmarks don't measure consistent traits and can be artificially inflated.

1 min read 3 sources

DeepSeek released V4-Flash-Vision-Exp, an experimental multimodal model adding image understanding to V4-Flash’s text capabilities (The Decoder). On the company’s own multimodal agent benchmarks, the model approaches Opus 4.8 performance and sometimes exceeds it, extending DeepSeek’s open-weight competitive reach into vision-based agentic tasks.

Researchers at the UK AI Security Institute applied psychometric methods to reveal major weaknesses in language model safety benchmarking (The Decoder). The study found that popular safety benchmarks do not measure one consistent trait, and that blanket request blocking can artificially inflate safety scores even as model utility declines. Separately, Anthropic deployed Claude Mythos 5 into its Claude Security product, now in public beta for enterprise customers, enabling vulnerability scanning of codebases with severity ratings and patch suggestions (The Decoder).

Compiled automatically from the linked sources and published without manual editing - a neutral summary of third-party reporting, for information only. Every claim links to its origin. Not original reporting.