Topic
Inference
3 stories about Inference that passed the news check, newest first.
Security · High impact ·
SGLang RCE Exploit Breaches NEAR AI, Steals Data From 18 Agents and About 100 Users
An attacker exploited a remote code execution flaw in SGLang, an open-source inference engine.
The Hacker News
Release ·
Google claims its new AI software cuts inference latency by over 90% — but there’s a catch
Google announced TPU Sync, an open-source system for moving KV cache and model weights on TPU, alongside its eighth-generation TPU announcement.
The New Stack · 2 sources
Release ·
SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
Amazon SageMaker Inference now supports NVIDIA Multi-Instance GPU (MIG) on model endpoints.
Artificial Intelligence