NVIDIA makes US$20b Groq deal

NVIDIA has agreed to acquire AI chip startup Grog’s key assets and license inference technology for US$20 billion.

Structured as a non-exclusive licensing agreement rather than a full company buyout, the move includes hiring Groq’s Founder Jonathan Ross and top executives to integrate their language processing unit (LPU) tech into NVIDIA’s ecosystem.

“While we are bringing in top talent and licensing Groq’s intellectual property, we are not acquiring the company Groq,” NVIDIA Founder and CEO Jensen Huang was quoted as saying in an internal email.

Announced around December 24, the all-cash transaction targets Groq’s proprietary LPU architecture which is designed for ultra-low-latency AI inference, while GroqCloud continues independently under new leadership.

Groq’s LPUs claim 10x faster inference speeds and lower energy use compared to NVIDIA GPUs for tasks such as LLMs, addressing GPU limitations in the shift from model training to deployment.

By absorbing this tech, NVIDIA bolsters its GPU dominance with specialised inference capabilities with potential yield of 2-10x throughput gains in real-time AI applications. This fusion enhances NVIDIA’s end-to-end AI stack, from training on GPUs to inference on hybrid LPU-augmented systems.

Following this move, enterprises should gain access to faster, cost-efficient inference for NLP, chatbots and autonomous systems via NVIDIA’s expanded portfolio, improving scalability for high-volume deployments.

Groq’s low-latency tech targets real-time enterprise AI to potentially cut costs and boost performance in data centres across sectors such as finance and healthcare.