All tech news

HardwareInfoQ · 12 h ago

AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU

Photo: InfoQ

AWS Lambda now runs Event Functions on EC2 G-series GPU instances through Lambda Managed Instances, with up to 48 concurrent invocations sharing one model per GPU, vLLM and SGLang base images that need no handler code, and 90-minute asynchronous timeouts. 75 percent fee; first hands-on tests report an eight-minute publish and no response streaming. AWS has added GPU support to AWS Lambda: Event Functions running on Lambda Managed Instances can now run on EC2 G-series instances (G4dn, G5, G6, G6f, Gr6, Gr6f and G6e at launch), deployed as container images, and triggered by the same event sources as any other function.

The feature is generally available in the commercial regions where both Managed Instances and G-series instances exist.

Excerpt from InfoQ, in their own words. Bonzai did not write this story; the full version is on the publisher’s site.