French Startup ZML Unveils Free Tool to Accelerate AI Inference Across Multiple Chips

Nvidia’s era of unchecked market supremacy may be winding down, yet competitors and alternatives are emerging from every angle.

ZML, a prominent French AI startup backed by Turing Award laureate Yann LeCun, has unveiled inference-performance software enabling various open-source large language models to operate across diverse hardware, including Nvidia, AMD, Google TPUs, Apple Metal, and Intel Arc.

Through ZML/LLMD, its newly launched LLM inference server, the company aims to dismantle existing silos and maximize the speed of different chips for AI applications, often matching or exceeding current benchmarks, according to ZML founder Steeve Morin.

As AI integrates deeper into professional and daily routines, optimizing inference—processing user prompts—has become increasingly critical, yet it remains fragmented due to software and architectural barriers that frequently result in vendor lock-in, Morin noted.

While delivering peak performance across multiple chip types is a significant technical achievement, it also poses a potential market disruption amid growing concerns over rising AI costs.

ZML intends to offer enterprises and cloud providers the flexibility to utilize a hybrid mix of chips, potentially reducing costs and energy consumption. “Our goal is to empower users to build their own systems and achieve tangible efficiency gains, facilitating broader AI adoption,” Morin explained.

This software support could benefit emerging AI chipmakers, particularly those based in Europe, Morin pointed out, listing companies such as Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. However, he emphasizes that ZML’s ability to collaborate on unprecedented innovations matters more than geographic origin.

Morin does not view Nvidia negatively, largely due to its established supply chain. He told TechCrunch that ZML maintains a strong relationship with the AI hardware giant, which is actively preparing for the growing demand for inference capabilities.

Inference has attracted massive investment, sparking what some call the “inference gold rush.” Consequently, ZML faces competition from players like Baseten, recently valued at $13 billion; Inferact, spun out from the creators of vLLM; and RadixArk, the commercial entity behind SGLang.

While vLLM and SGLang offer partial overlap with LLMD, Morin envisions ZML covering a wider scope. “We have reached a stage where we are co-designing silicon,” he stated. He also credited ZML’s lean team of 20 employees for the Paris-based startup’s rapid development pace, with more updates planned.

The small team is also well-capitalized. Leveraging his background as VP of engineering at Zenly, which Snapchat acquired for nine figures in 2017, Morin secured $20 million from investors including 20VC, >commit, AALVC, Drysdale Ventures, Kima Ventures, Kindred Capital, LocalGlobe, and Puzzle Ventures.

Unlike ZML’s initial open-source inference framework released in 2024 and updated in March, ZML/LLMD is not open source. However, it is launching as a free tool to gather usage data. “I prefer to measure impact and generate revenue where it matters most, avoiding unnecessary growth hindrances caused by premature monetization,” Morin said.

It remains unclear when ZML/LLMD might transition to a paid model or how widely it will be adopted. Nevertheless, the startup’s investor list signals strong interest from industry leaders, including Docker founder Solomon Hykes, Hugging Face’s Clément Delangue and Julien Chaumond, and LeCun, now at AMI Labs. This reinforces the argument that European AI startups can thrive locally. “I couldn’t have built ZML anywhere but in Paris,” Morin concluded.