Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2

Heidi Health, AWS, and NVIDIA collaborated to reduce automatic speech recognition (ASR) inference costs by 75% on Amazon EC2 using NVIDIA CUDA Multi-Process Service (MPS) and Triton Inference Server. This approach increased GPU utilization from 15-20% per request to supporting 92.1 requests per second per GPU, cutting required GPU instances from 16 to 4 while maintaining sub-second latency.