Opinion
Where should AI workloads run? Probably next to the data.
The New Stack · By Shubham Thakur ·
The Brief
- AWS raised prices on EC2 Capacity Blocks, its reserved GPU capacity, by about 15% in January and about 20% in July.
- The article's example is a training job whose bill climbed after hundreds of terabytes of sensor data were copied across regions to reach the GPUs.
- The article argues that AI workloads should generally run next to the data where possible.
Why it matters
GPU capacity costs are rising, and moving large datasets across regions adds cost on top of compute, so where a workload runs affects its bill.
Who’s affected
- Platform teams running AI training workloads on AWS GPU capacity
- Teams moving large datasets across regions to reach GPUs
AWS raised prices on EC2 Capacity Blocks , its reserved GPU capacity, by about 15% in January and about 20% in July . Picture a platform team watching a training job’s bill climb after copying a few hundred terabytes of sensor data across regions just to reach the GPUs, then…