International
The Morning Dev

Only the news worth knowing

‹
 

Launch

AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU

InfoQ · By Steef-Jan Wiggers ·

AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU

The Brief

  • AWS Lambda now runs Event Functions on EC2 G-series GPU instances through Lambda Managed Instances.
  • Up to 48 concurrent invocations can share one model per GPU.
  • vLLM and SGLang base images need no handler code, and asynchronous invocations get 90-minute timeouts.

Why it matters

Developers can run GPU-backed model inference on Lambda with shared models and concurrency, without writing a custom handler when using the vLLM or SGLang base images.

Who’s affected

  • Teams running GPU inference on AWS Lambda

AWS Lambda now runs Event Functions on EC2 G-series GPU instances through Lambda Managed Instances, with up to 48 concurrent invocations sharing one model per GPU, vLLM and SGLang base images that need no handler code, and 90-minute asynchronous timeouts. Billing is the instance…

How do you feel about this?

0 Comments