Accelerating decode-heavy LLM inference with speculative decoding on AWS Trainium and vLLM

AWS researchers have implemented speculative decoding to speed up inference of large language models on AWS Trainium chips using the vLLM framework, achieving faster and more efficient processing.