Can Brain Waves Unlock Physical AI?

The frontier of physical AI resembles a high-stakes game of Jenga inside a San Leandro, California warehouse.

Encord operates within this space, providing data tooling for AI model training. Andrew Ceja, a "pilot" (Encord’s term for robotic trainers), carefully extracts wooden blocks from an unstable tower while wearing a camera-equipped headset. This setup is standard for robot training data collection, but Ceja’s headset also features sensors that record his brain waves as he dismantles the structure.

Encord is among a growing number of startups betting that the primary bottleneck for humanoid and warehouse robotics is not model architecture, but the scarcity of real-world physical training data. Instead of merely managing existing data, Encord is building a business model around generating the data that robotics companies lack.

The brain wave headset Ceja wears was developed by Zander Labs, a German neuroscience startup. They believe that measuring brain activity to infer mental states—such as error, intent, or surprise—can generate more valuable datasets for training models. Encord’s collaboration with Zander is currently a trial; the goal is to create an initial brain wave-tagged dataset, test it against customer robotics models, and evaluate performance improvements before scaling.

Lucas Gehrke, a Zander neuroscientist overseeing the project, notes that the level of brain activity during specific tasks provides clues for model developers regarding when to deploy high-effort models.

Vineeth Velmurugan, Encord’s head of robot learning, describes this as the “bleeding edge” of solving the robotics data bottleneck. A veteran of OpenAI’s robot lab and Berkshire Grey, a warehouse automation firm, Velmurugan joined Encord to establish its internal data-creation team.

Encord was originally founded to assist companies building machine-vision applications with data annotation and model evaluation. As customers—many leading robotics firms, though unnamed by Velmurugan—began applying end-to-end learning to robotic manipulation, executives realized they needed to generate training data rather than just manage it. “The data simply does not exist,” Velmurugan stated.

The assumption that generative AI can revolutionize robotics as it did for chatbots continues to hit the same wall. LLMs were trained on the entirety of internet text and more. Acquiring equivalent raw materials to teach neural networks about physical manipulation is difficult: self-driving car companies collect their own data, but this is hard to scale. While training from video is possible, it lacks the fidelity of real-world data. Velmurugan estimates that a dataset five times the size of YouTube’s video corpus is needed to break through—explaining why data generation has become a standalone business rather than just a research challenge.

Feed your egocentric data needs

Robotics developers are now relying on two primary sources: “egocentric” video captured by workers wearing cameras (often supplemented with additional angles and metrics) and data from remotely operated robots. Encord utilizes both, gathering egocentric data from factories globally and using its San Leandro facility to experiment with new modalities, such as brain waves, or to collect datasets focused on specific skills for fine-tuning.

During a TechCrunch visit, pilots used leader-follower rigs—paired robotic arms where one is controlled directly by a human and the other mimics its movements—to generate data for tasks like pouring coffee (which is prone to sloshing) and stacking poker chips. “Every humanoid company has requested these datasets,” Velmurugan noted.

Storage racks held cartons of fake flowers in vases, books, plastic vegetables, kitty litter trays and scoops, bags, and bundles of wires—essential props for training manipulators for household tasks.

At one station, another pilot, Sofia Infante, maneuvered robotic arms to plug and unplug ethernet cables from server backs—a task data center operators would love to automate, provided robots could manipulate them with sufficient precision. Operating the controls myself revealed why this remains out of reach: robotic pincers are far less dexterous than human fingers and lack the degrees of freedom we take for granted.

Encord is also developing a new data modality using sensors strapped to the forearm to detect electrical muscle signals. Since video of human hands often fails to capture the entire hand, Velmurugan aims to create a 3D representation of hand position based on arm sensor data, providing models with a more robust understanding.

Encord’s datasets are annotated with physical descriptions of video content—such as “right hand tightens bolt”—to help LLM-based models comprehend actions. Velmurugan estimates that this dense annotation is 100 times more valuable than “junky ego data” for training specific tasks, costing only 20 times more to produce—a favorable trade-off on paper.

However, “20 times more” represents real money, which is the catch: scraping internet text to build LLMs from sources like Stack Overflow cost frontier labs almost nothing. Generating physical training data does not, highlighting the limit of comparing physical AI to LLMs. This data must be manufactured, not just collected, fundamentally changing the economics of model development.

Velmurugan believes progress is being made. With Encord’s visibility into industry programs, he observes startups and frontier labs determining what works to improve physical AI models. This vantage point—positioned between multiple robotics companies—is part of Encord’s value proposition, allowing them to identify data techniques gaining industry-wide traction before any single customer does.

This will keep the dozen or so pilots at Encord’s facility busy. Both Infante and Ceja are part of a growing workforce developing neural network building blocks; they previously worked at Scale, another AI data annotation firm, before joining Encord.

Ceja previously worked at a waste management company, where his interest in technology led him to maintain a robotic trash sorter. Now, as the Jenga tower collapses, he enjoys the challenge of solving training tasks for robots: “It’s something new every day!”