Indian Startup Ziroh Labs Unveils Kompact AI to Run Large Language Models Without High-End GPUs

Ganpat Singh Chouhan

In a significant breakthrough for the artificial intelligence landscape, Indian startup Ziroh Labs has launched Kompact AI, a system that can run large language models (LLMs) on standard CPUs instead of costly, high-performance GPUs. Developed in collaboration with the Indian Institute of TECHNOLOGY (IIT) Madras, this innovation could democratize access to AI capabilities globally — especially in regions where advanced computing infrastructure is limited.

Kompact AI: Bringing Powerful AI to Everyday Devices

Kompact AI is designed to perform AI inference — the process of running trained models to generate responses or predictions — using the central processing units (CPUs) found in most consumer-grade laptops and desktops. In a live demonstration, the platform successfully ran models like Meta’s Llama 2 and Alibaba’s Qwen2.5 on a laptop equipped with an off-the-shelf Intel Xeon processor, bypassing the need for GPUs like Nvidia’s H100 or A100.

Huge Discounts Available on These Maruti Suzuki Cars — Check Latest Offers

📅 9 months ago

This technology could be a game-changer, especially in emerging markets, where access to premium GPUs is often hindered by cost and supply constraints. As the AI race intensifies globally, innovations like Kompact AI open the door for broader participation beyond the handful of companies and nations with deep resources.

Backed by Industry and Academia

Ziroh Labs’ CPU-first approach has reportedly been tested by leading chipmakers, including Intel and AMD, further lending credibility to the system’s performance claims. The startup’s strategic advisor William Raduchel, a former executive at Sun Microsystems, praised the platform’s disruptive potential during the launch event.

Realme 15 5G and Realme 15 Pro 5G Set for India Launch Soon — AI Cameras, Big Battery & Flagship-Level Features Teased

📅 8 months ago

“It’s going to have a very profound market impact in the years ahead,” Raduchel said.

V. Kamakoti, Director of IIT Madras, echoed the sentiment: “The AI divide is because only those with high-end, expensive GPU-powered resources can access, develop and deploy powerful AI. We’re demonstrating that you don’t need a revolver to kill a mosquito.”

Global Implications: Tackling the AI Divide

As tech giants like Google, OpenAI, and Meta continue to compete over AI superiority, concerns around resource disparity and “AI colonialism” have surfaced. GPUs — essential for training and running large-scale AI models — are both expensive and in limited supply, exacerbating the gap between well-funded labs and emerging developers.

Nothing Phone 3a and Phone 3a Pro: Key Features Leaked Ahead of March 4 Launch

📅 1 year ago

Ziroh Labs’ innovation provides a potential solution to this problem by enabling cost-effective, scalable AI deployments. The startup’s mission aligns with a broader global push to make AI development more inclusive, sustainable, and locally driven.

What’s Next?

The debut of Kompact AI comes at a time when energy-efficient, hardware-agnostic AI solutions are in high demand. With further refinement and enterprise adoption, Kompact AI could pave the way for a new generation of lightweight, accessible, and decentralized AI deployments — not only in India but across the globe.

Apple Breaks into India’s Top Five Smartphone Brands: A Major Milestone

📅 1 year ago

Author Profile

Ganpat Singh Chouhan: My name is Ganpat Singh Choughan. I am an experienced content writer with 7 years of expertise in the field. Currently, I contribute to Daily Kiran, creating engaging and informative content across a variety of categories including technology, health, travel, education, and automobiles. My goal is to deliver accurate, insightful, and captivating information through my words to help readers stay informed and empowered.