#推理

We love papers that go deeper down the stack. This is a good example that shifts from agentic context engineering to agentic cache engineering. They also apply recursive self-improvement to cache policies to further reduce cost by 7.1% over expert design. Specifically, this work focuses on KV cache management in long-running agents. When an agent summarizes or trims its history, the prefix change invalidates the cache after it, and the retained context must be prefilled again. Cache engineering gives the agent two operations on its own KV cache. One removes entries from future attention, and the other shifts the positions of the keys that remain. The trajectory text stays unchanged. This essentially provides a programmable interface for agents to define and refine their own cache engineering strategies. A rolling drop of old tool responses scores 1.7 points higher and costs 27.7% less than threshold-triggered summarization. Their AKV serving engine adds up to 1.2x throughput and 1.5x faster P95 latency. https://arxiv.org/abs/2610.28577 Chat with Paper: https://academy.dair.ai/papers/cache-engineering-for-agents-2610.28577

Our latest post explores on-policy distillation, a training approach that unites the error-correcting relevance of RL with the reward density of SFT. When training it for math reasoning and as an internal chat assistant, we find that on-policy distillation can outperform other approaches for a fraction of the cost. https://thinkingmachines.ai/blog/on-policy-distillation/