Apple and LM Studio Unveil Groundbreaking Partnership as Four Mac-Based Studios Deploy Trillion-Parameter Large Models

Breakthrough Demonstration at WWDC: Running a Trillion-Parameter AI Model on Apple Hardware

At the recent WWDC, the AI software platform LM Studio collaborated with Apple to present a groundbreaking achievement in artificial intelligence technology. The team successfully ran Moonshot AI’s flagship model, Kimi K2.6, using a cluster of four Mac Studios. This demonstration clearly illustrated the substantial capabilities of Apple Silicon architecture when it comes to processing ultra-large-scale AI models.

The Kimi K2.6 model is built around an advanced MoE architecture, featuring a total of up to one trillion parameters. While the model’s dynamic expert scheduling system reduces the computational load during inference by activating only around 32 billion parameters at a time, loading the entire model still poses a significant memory challenge. When processed in FP16 precision, it requires approximately 2TB of memory. In conventional data center setups, meeting this requirement typically involves using server clusters with 8 to 16 high-end GPUs, which can cost millions of dollars.

Yet the demonstration overcame this limitation through an innovative technical solution. The four Mac Studios equipped with M3 Ultra chips were connected using Thunderbolt 5 interfaces, taking advantage of RDMA-over-Thunderbolt technology available in the latest macOS versions. This approach enabled direct memory sharing between the devices, effectively merging their combined 2TB of unified memory into a single logical memory pool. As a result, the trillion-parameter model could be loaded without issue. During the live demo, the cluster delivered strong performance, generating around 28 tokens per second while using far less power than traditional GPU-based computing centers.

Alongside this demonstration, LM Studio also introduced LM Link as part of the collaboration. Built on the Tailscale Mesh VPN architecture, this tool allows users to securely access the local Mac Studio cluster from anywhere over end-to-end encrypted connections. Users can leverage the cluster’s computing power for inference using either a MacBook or iPhone, without needing to be physically present at the host device. All sensitive data remains processed locally within a secure loop, avoiding any transfer through third-party cloud servers.

This demonstration served more than just as a technical showcase; it also sent a clear message to the industry. Apple Silicon, with its unified memory design and efficient multi-device connectivity features, is emerging as a viable option for deploying large AI models. For organizations that need to run large model inference tasks frequently and on a long-term basis, this solution eliminates the high costs associated with cloud services by replacing them with hardware investments, resulting in significant cost savings over time.

As the performance of consumer-grade hardware clusters continues to improve, barriers to adopting AI technologies are further decreasing. This achievement suggests that the source of cutting-edge AI innovation will no longer be confined to a handful of tech giants with large supercomputing facilities. A decentralized computing landscape is likely to see new opportunities for development in the near future.