Amazon EC2 G5 Instances

Amazon EC2 G5 instances are NVIDIA GPU-based EC2 instances built on the AWS Nitro System for graphics-intensive applications and machine learning inference and training, featuring up to 8 NVIDIA A10G Tensor Core GPUs, up to 192 vCPUs, up to 100 Gbps network bandwidth, and up to 7.6 TB of local NVMe SSD storage.

Official product page

1published use cases

1industry

1countries on record

Large Language Models

top AI capability

Evidence mix: High 1 · Medium 0 · Low 0 — bands are computed from each record's evidence signals.

LogisticsLarge Language Models

Delhivery achieves 160 ms latency for high-precision geocoding using Amazon EKS

Delhivery, a logistics provider in India, implemented a fine-tuned open-source Llama 3.2 1B large language model on Amazon EKS to support high-volume geocoding of pickup and drop-off addresses. The system processes up to 8,000 requests per minute at 160 milliseconds latency using NVIDIA A10G GPU-backed G5 Xlarge instances and the vLLM framework. Delhivery cut model-serving costs by approximately 80 percent and accelerated prototyping cycles from two days to under six hours, working with the AWS Prototyping and Cloud Engineering (PACE) team.

Delhivery· IndiaLlama 3.2 1B · Amazon Elastic Kubernetes Service · Amazon EC2 G5 Instances +3