Robots 'see' touch via color-changing tactile sensor

A robot touches a coin, sensing its fine details. Credit: Queen Mary University of London

Engineers at Queen Mary University of London have developed a new color-changing tactile sensor that enables robots to "see" and feel touch in real time. The concept was created by Giacomo Sasso, a postdoctoral researcher at the School of Engineering and Materials Science, and works by converting invisible forces into dynamic color patterns. This allows high-resolution maps of contact, strain, and pressure to appear instantly.

The research is published in the journal Science Advances.

When pressure is applied to a soft sensing surface, the material produces spatially varying structural colors that a standard camera can capture immediately, eliminating the need for complex reconstruction algorithms.

From factory floors to surgery

This technology enables a robotic gripper to assemble microscale components with the precision required in manufacturing, where every subtle change in force becomes visible in real time. It also has potential in healthcare, where an external prosthetic limb could gain a richer sense of touch during delicate daily or clinical tasks.

At the same time, surgical systems could distinguish healthy from abnormal tissue by reading fine pressure signatures directly through the material's color response.

Sensing built into the material

Unlike traditional tactile sensors, this new system embeds sensing directly into the material itself. Mechanical interactions are transformed into color fields that a low-cost USB camera can read in real time. The work has already produced results demonstrating the first real-time solution in this field.

"You wouldn't guess how much information is generated when your finger presses a light switch. A human hand contains more than 10,000 mechanoreceptors to do the job, yet touch sensing remains one of the major challenges in robotics," said Sasso.

"We were pleased to capture the finger ridges, as no existing technology can reproduce such sensor density at a comparable scale and simplicity. The key idea behind this project was to think outside the box: instead of embedding dense and overengineered sensor arrays, sensing is moved into the material itself, where mechanical cues are directly transformed into color fields and captured using a simple, low-cost USB camera." This produces rich pressure maps while simplifying the system architecture.

Co-authors from the University of Florence, University of Trieste, and University of Trento in Italy agree that "What is particularly powerful is that the information is already in the light signal. You are no longer reconstructing touch—you are observing it directly," said Professor James Busfield.

Solving a long-standing trade-off

The idea emerged from the need to overcome a persistent trade-off in vision-based tactile sensing: High-resolution systems typically require heavy computational pipelines to reconstruct contact geometry, introducing latency, while faster systems often sacrifice spatial detail.

The collaboration between Professor Federico Carpi from the University of Florence and Busfield merges the research worlds of soft robotics and materials science. Building on years of work on stretchable sensors and polymer characterization, the team has progressively advanced the ability to interface mechanical compliance with functional sensing.

Within this framework, mechanochromic materials represent a new direction: Instead of relying on highly engineered microelectronics to interpret deformation (taxels), the material itself becomes the sensing medium, directly encoding mechanical interaction into visible optical signals.