|
Immediately, we’re asserting the overall availability of Amazon Elastic Compute Cloud (Amazon EC2) G7e situations that ship cost-effective efficiency for generative AI inference workloads and the best efficiency for graphics workloads.
G7e situations are accelerated by the NVIDIA RTX PRO 6000 Blackwell Server Version GPUs and are effectively fitted to a broad vary of GPU-enabled workloads together with spatial computing and scientific computing workloads. G7e situations ship as much as 2.3 occasions inference efficiency in comparison with G6e situations.
Enhancements made in comparison with predecessors:
- NVIDIA RTX PRO 6000 Blackwell GPUs — NVIDIA RTX PRO 6000 Blackwell Server Version GPUs supply two occasions the GPU reminiscence and 1.85 occasions the GPU reminiscence bandwidth in comparison with G6e situations. Through the use of the upper GPU reminiscence provided by G7e situations, you’ll be able to run medium-sized fashions of as much as 70B parameters with FP8 precision on a single GPU.
- NVIDIA GPUDirect P2P — For fashions which can be too massive to suit into the reminiscence of a single GPU, you’ll be able to break up the mannequin or computations throughout a number of GPUs. G7e situations cut back the latency of your multi-GPU workloads with help for NVIDIA GPUDirect P2P, which permits direct communication between GPUs over PCIe interconnect. These situations supply the bottom peer to see latency for GPUs on the identical PCIe change. Moreover, G7e situations supply as much as 4 occasions the inter-GPU bandwidth in comparison with L40s GPUs featured in G6e situations, boosting the efficiency of multi-GPU workloads. These enhancements imply you’ll be able to run inference for bigger fashions throughout a number of GPUs providing as much as 768 GB of GPU reminiscence in a single node.
- Networking — G7e situations supply 4 occasions the networking bandwidth in comparison with G6e situations, which implies you should utilize the occasion for small-scale multi-node workloads. Moreover, multi-GPU G7e situations help NVIDIA GPUDirect Distant Direct Reminiscence Entry (RDMA) with Elastic Cloth Adapter (EFA), which reduces the latency of distant GPU-to-GPU communication for multi-node workloads. These occasion sizes additionally help NVIDIA GPUDirectStorage with Amazon FSx for Lustre, which will increase throughput by as much as 1.2 Tbps to the situations in comparison with G6e situations, which implies you’ll be able to rapidly load your fashions.
EC2 G7e specs
G7e situations function as much as 8 NVIDIA RTX PRO 6000 Blackwell Server Version GPUs with as much as 768 GB of complete GPU reminiscence (96 GB of reminiscence per GPU) and Intel Emerald Rapids processors. Additionally they help as much as 192 vCPUs, as much as 1,600 Gbps of community bandwidth, as much as 2,048 GiB of system reminiscence, and as much as 15.2 TB of native NVMe SSD storage.
Listed below are the specs:
| Occasion title |
GPUs | GPU reminiscence (GB) | vCPUs | Reminiscence (GiB) | Storage (TB) | EBS bandwidth (Gbps) | Community bandwidth (Gbps) |
| g7e.2xlarge | 1 | 96 | 8 | 64 | 1.9 x 1 | As much as 5 | 50 |
| g7e.4xlarge | 1 | 96 | 16 | 128 | 1.9 x 1 | 8 | 50 |
| g7e.8xlarge | 1 | 96 | 32 | 256 | 1.9 x 1 | 16 | 100 |
| g7e.12xlarge | 2 | 192 | 48 | 512 | 3.8 x 1 | 25 | 400 |
| g7e.24xlarge | 4 | 384 | 96 | 1024 | 3.8 x 2 | 50 | 800 |
| g7e.48xlarge | 8 | 768 | 192 | 2048 | 3.8 x 4 | 100 | 1600 |
To get began with G7e situations, you should utilize the AWS Deep Studying AMIs (DLAMI) in your machine studying (ML) workloads. To run situations, you should utilize AWS Administration Console, AWS Command Line Interface (AWS CLI) or AWS SDKs. For a managed expertise, you should utilize G7e situations with Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Parallel Computing Service (AWS PCS). Help for Amazon SageMaker AI can also be coming quickly.
Now out there
Amazon EC2 G7e situations can be found in the present day within the US East (N. Virginia) and US East (Ohio) AWS Areas. For Regional availability and a future roadmap, search the occasion sort within the CloudFormation assets tab of AWS Capabilities by Area.
The situations could be bought as On-Demand Cases, Financial savings Plan, and Spot Cases. G7e situations are additionally out there in Devoted Cases and Devoted Hosts. To study extra, go to the Amazon EC2 Pricing web page.
Give G7e situations a strive within the Amazon EC2 console. To study extra, go to the Amazon EC2 G7e situations web page and ship suggestions to AWS re:Publish for EC2 or via your traditional AWS Help contacts.
— Channy