Checked for new stories 9m ago

Updates on Inference Optimization

Every AI story we track on Inference Optimization — 14 stories so far, each summarized in our own words and linked back to the publisher that reported it.

Pulled from 123 sources

This month

LLM & Text Generation7 min read

Speed Up LLM Inference with DSpark Speculative Decoding

KDnuggets10d ago

Machine Learning16 min read

Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2

AWS Blog14d ago

AI Research5 min read

Z.AI's Use of Chinese Chips for New Model is About Optimization

AI Business14d ago

Machine Learning6 min read

Up to 3.2x Faster Inference with LFM2.5-DSpark

Covered by 3 sources21d ago

LLM & Text Generation18 min read

Introducing cross-Region inference for OpenAI GPT-5.6 models on Amazon Bedrock

AWS Blog21d ago

Dev4 min read

NVIDIA Releases TensorRT Model Connect in Public Preview: Hugging Face Checkpoint to Native C++ Inference in Two Commands

MarkTechPost23d ago

AI Research4 min read

IBM bets $240m on cheap, open-source inference to take on the hyperscalers

Covered by 2 sources30d ago

Dev29 min read

Tiered KV cache for large LLMs on Amazon SageMaker HyperPod with Curvine

AWS Blog29d ago

LLM & Text Generation2 min read

Arbitrage: Efficient Reasoning via Advantage-Aware Speculation

Apple Research BlogAug 7

LLM & Text Generation

Predictive Speculative KV Replication for Bursty LLM Inference

Hacker NewsJul 31

Dev10 min read

Launching UI for generative AI inference recommendations in Amazon SageMaker AI

AWS BlogJul 13

Machine Learning11 min read

Streaming benchmark and recommendation results to MLflow with Amazon SageMaker AI

AWS BlogJul 6

LLM & Text Generation

Parallelize speculative decoding with P-EAGLE on Amazon SageMaker AI

AWS BlogJun 16

Machine Learning

Amazon SageMaker AI now supports optimized generative AI inference recommendations

AWS BlogApr 22

That's everything we have on Inference Optimization right now