Ex-Meta Researchers Aim to Bring Visual AI to Factory Floors

Artificial intelligence is reshaping our world, yet it has mostly stayed within digital boundaries. Now, a new wave of startups is bringing these capabilities into the physical world.

Perceptron, founded by two ex-Meta researchers, is leading this shift. Established in November 2024, the company builds advanced vision models that enable machines to interact more effectively with their physical surroundings.

This week, Perceptron introduced Isaac 0.5, a model designed to give machines the ability to “perceive, reason, and act” within industrial environments. The software empowers vision-guided robots to navigate complex spaces like factories and warehouses, while also allowing companies to extract valuable visual insights from footage captured by these robots.

Isaac 0.5 is available as an open-weight model, making its parameters and training data accessible for public inspection.

Co-founded by Armen Aghajanyan and Akshat Shrivastava, both former members of Meta’s Fundamental AI Research (FAIR) team, Perceptron recently secured $21 million in funding led by Bessemer Venture Partners. The founders envision their software as the next step in industrial automation.

“Current Physical AI presents a false dilemma: generalist foundation models requiring multiple dedicated cloud GPUs per instance, or narrow models that handle either perception or control, but rarely both,” the company explains.

Aghajanyan and Shrivastava argue that their tool differs from existing solutions by being general-purpose. Rather than being optimized for a single repetitive task, the model is built to adapt flexibly to various environments and situations.

In an interview, Shrivastava prompted a reflection on simple physical tasks, such as organizing boxes: “Consider a robot deployed to sort packages. What specific tasks must it perform?”

This seemingly straightforward process involves multiple complex steps. The robot must first read package labels, perform spatial analysis to locate boxes, and determine which item to pick. When handling multiple boxes, it must also plan the sequence and order of operations.

Perceptron’s software assists robots in navigating each of these steps. While other industry tools can handle individual tasks, few offer the flexibility required for dynamic, real-world applications.

So, where does the data for this algorithmic innovation come from?

Models like Isaac 0.5 acquire operational skills by processing massive volumes of video training data. Perceptron states that its new model was trained on one million hours of general video to recognize settings, visuals, and scenarios. The company also utilized ego video—footage captured from a first-person perspective, often via wearable cameras—and UMI video, which records repetitive human movements to teach AI systems physical actions.

Although Perceptron has not disclosed specific data sources, Shrivastava noted that the company has “internally built petabyte-scale datasets spanning multiple modalities, including images, text, video, and robotic trajectories.”

The potential for software that enables robots to operate efficiently in warehouses is immense. Perceptron believes it is well-positioned to drive this automation wave, offering its software to various vendors for integration across diverse industries.

Target sectors include manufacturing, logistics, warehousing, security, mobility, and media and entertainment.

“Nothing like this currently exists in the market,” Aghajanyan stated. “We are truly excited about the possibilities.”