Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI
AWS · By Dmitry Soldatkin ·
Serving a model from more than one AWS Region is a well-established pattern for highly available inference: when one Region runs short of GPU capacity, requests move to another. The pattern is simple to describe and hard to operate. Until now, if you served models on Amazon…