Tsinghua AI released AKV, an SGLang-based serving system that optimizes KV cache management for LLM agents (MarkTechPost). Instead of summarizing conversation history, the system drops and repositions KV cache blocks, achieving a 27.7% average cost reduction across four agent benchmarks. The optimization addresses a core efficiency constraint in multi-turn agent workflows where cache overhead compounds with each interaction.
Stanford and UC Berkeley researchers introduced TokenGate, a learned token pruning module that accelerates long-context prefill by 1.4x to 11.3x and reduces peak memory consumption by 1.5x to 3.4x without accuracy degradation (MarkTechPost). The drop-in module works with existing LLM inference stacks. Separately, Nanbeige LLM Lab open-sourced Nanbeige4.3-33B, a 33B dense coding model with 512K context window under Apache 2.0 license, reporting 80.9 on Terminal-Bench 2.1 (MarkTechPost).