# Amazon SageMaker AI real-time endpoints now support KV cache aware routing

Source: [Artificial Intelligence](https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-real-time-endpoints-now-support-kv-cache-aware-routing/) · 2026-11-17 · AI
Page: https://themorning.dev/story/amazon-sagemaker-ai-real-time-endpoints-now-support-kv-cache-aware-routing-1183c101fd5882a0

Today, Amazon SageMaker AI introduces KV cache-aware routing (KV-aware routing), a new routing strategy for real-time endpoints. It uses the cache state that the model container reports. It sends each request to an instance that already holds a long cached match for its prompt…

## Coverage

- [Amazon SageMaker AI real-time endpoints now support KV cache aware routing](https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-ai-real-time-endpoints-now-support-kv-cache-aware-routing/) — Artificial Intelligence, 2026-11-17

---

Brief generated by AI (–) from the reporting listed under Coverage. Headlines belong to their publishers. The Morning Dev – https://themorning.dev
