Relaxation-Aware Multimodal Sensing of Soft Gripper Driven by Structure-Perception-Learning

AI summary

Overview

Research area: Soft robotics — variable-stiffness soft grippers, multimodal perception, and physics-informed learning for grasp force regulation.

Technical level: Advanced. The paper assumes familiarity with viscoelastic material modeling (generalized Maxwell–Wiechert formulations), shape memory polymers, closed-loop force control, and hybrid physics-plus-learning inference.

Scope: The paper integrates a thermally tunable, bio-inspired soft gripper with vision, infrared thermography, and a temperature-coupled learning model to hold a target grasp force despite viscoelastic stress relaxation.

What This Paper Is About

Soft grippers contact objects gently because their polymer bodies conform to irregular shapes, but that same compliance causes the grasp force to decay over time through stress relaxation, so a held object can gradually slip. The authors build a two-finger soft gripper whose stiffness can be switched by heating and cooling shape memory polymer beams, then use cameras plus a physics-informed learning model to continuously reconstruct the interaction force and compensate the relaxation-induced decay during long holds. The goal is stable, sustained grasping that mimics how a human hand regulates stiffness by phase while continuously sensing contact.

Key Contributions

1. A bio-inspired multi-mode soft finger that unifies passive compliance, thermally tunable stiffness (via embedded shape memory polymer beams), and switchable thermally activated adhesion pads in one structure, enabling compliant, stiff, and adhesive grasping modes without changing external geometry or requiring mechanical reconfiguration.

2. A non-invasive multimodal perception architecture with sensors mounted on the gripper base rather than embedded in the soft body: an RGB camera tracking a single ArUco marker for pose and deformation, and an infrared thermal camera monitoring the temperature field, fused with a model-based inference module.

3. A temperature-coupled reduced viscoelastic force representation that separates interaction force into an instantaneous elastic term and a relaxation-induced decay term described by an effective magnitude K(T) and time constant τ(T), paired with a physics-constrained learning pipeline: an Instantaneous Contact Force Network (ICFN) and a Relaxation Compensation Force Network (RCFN, implemented as two lightweight Ridge regressors).

4. Experimental validation of long-duration force regulation, including a closed-loop framework where the host PC fuses RGB and infrared data, reconstructs force, and drives a PID controller through an MCU while a separate temperature loop regulates the heating film via an L298N driver.

Main Findings

• Temperature strongly softens the finger. Increasing temperature from 27 °C to 60 °C produced a 1.6× to 1.7× increase in deformation, corresponding to a 35%–45% reduction in local stiffness k₈₀ (measured at x = 80 mm as F/z₈₀) across force levels in the 2–8 N range. Global deformation area A_z at 8 N rose from 361 mm² to 626 mm², a +73% increase, indicating system-level rather than localized softening.

• Instantaneous force estimation is accurate but not perfect. On the held-out test set, ICFN achieved an MAE of 0.651 N (6.51% full-scale error over the 0–10 N range), RMSE of 0.886 N, and R² = 0.848. Median absolute errors were 0.489 N (0–1 N), 0.297 N (1–2 N), 0.393 N (2–5 N), and 0.695 N (5–10 N).

• Relaxation reconstruction is reliable across temperatures. On the RCFN test set, NRMSE had mean/median 0.160/0.118 and AUC_rel was low at 0.065/0.049, with a mean curve-level correlation of 0.990.

• Repeatability holds but spreads at high temperature. Across six randomly sampled sequences per temperature group, ~40 °C gave NRMSE = 0.091 ± 0.040, AUC_rel = 0.029 ± 0.018, Corr = 0.997 ± 0.005 (ground-truth std at t_end = 0.041 N); ~50 °C gave NRMSE = 0.143 ± 0.068, AUC_rel = 0.042 ± 0.015, Corr = 0.991 ± 0.010 (GT std 0.040 N); ~60 °C showed larger spread (NRMSE = 0.140 ± 0.099, AUC_rel = 0.067 ± 0.042, GT std 0.121 N) but correlation remained high at 0.991 ± 0.013.

• The proposed method substantially reduces holding-force error. In a 280 s grasp-and-hold task with a 5 N target, the proposed ICFN+RCFN method achieved a mean absolute error of 0.066 N (1.32% of target, interquartile range [0.055, 0.076] N), compared with 0.333 N (6.66%) for the fixed-aperture baseline and 1.327 N (26.54%) for the instantaneous-only baseline — reductions of approximately 80.2% and 95.0%, respectively.

• Error statistics favor the proposed approach. Over the steady-state interval of 100–280 s, the proposed method showed negligible bias (mean signed error −0.036 N, median −0.041 N) and low dispersion (RMSE 0.102 N). The fixed-aperture baseline showed mean 0.606 N and RMSE 0.643 N, while the instantaneous-only baseline suffered severe overestimation (mean 1.019 N, RMSE 1.035 N).

• Stiffness-dependent sensing trade-off. Softer configurations enhanced sensitivity to weak interactions (detectable signals from a feather and a brush) but stiffer configurations favored generating and maintaining larger forces. Instantaneous force estimates remained stable across all tested stiffness states despite differing deformation patterns.

• Task-appropriate stiffness selection works in closed loop. High stiffness sustained a 5 N target while lifting a hammer, intermediate stiffness regulated force for an orange at a 6 N target, and a highly compliant configuration regulated sub-newton force (0.3 N target) to grasp a balloon without visible deformation.

• Integrated demonstration spans incompatible requirements. In a multi-stage task, infrared thermography non-contactly distinguished a room-temperature cup from a hot-water cup, the gripper poured water from a paper cup at a 1 N target in a low-stiffness regime, then transitioned to a stiff regime at a 6 N target to hand a glass cup to a human. The dotted line marks a glass-transition temperature T_g ≈ 48 °C, around which SMP stiffness changes rapidly.

• Labeled-mode contrast. The stiff mode could lift a 1 kg weight by fingertip pinch whereas the compliant mode could not; conversely, the compliant mode grasped an empty aluminum can without visible deformation while the stiff mode crushed it.

• Inference latency is low. RGB images update at approximately 30 Hz and infrared at approximately 1 Hz. ArUco pose extraction takes approximately 3.5 ms and ICFN inference approximately 1.3 ms, giving instantaneous force latency of approximately 4.8 ms; RCFN steady-state inference adds approximately 0.14 ms for a relaxation-compensated latency of approximately 4.94 ms.

Methodology in Plain English

The researchers started from a physical model rather than a purely data-driven one. They described the soft finger as a spring in parallel with several spring-and-dashpot branches (a generalized Maxwell–Wiechert model), where every stiffness and damping parameter depends on temperature. During holding, the displacement is nearly constant, so each branch's contribution decays exponentially, and the authors collapse the many branches into a single effective relaxation magnitude K(T) and time constant τ(T). This reduces the complicated material behavior to two learnable numbers.

The force is then written as the instantaneous contact force minus a relaxation loss term. Rather than trying to identify all the viscoelastic parameters, the authors learn the reduced quantities directly. One network (ICFN) estimates the instantaneous force at the moment of contact from ArUco-derived pose, squared marker translation, beam temperatures, and engineered thermal features such as temperature statistics and spatial gradients. A second module (RCFN, implemented as two Ridge regressors) predicts K and the logarithm of τ during holding from current temperature, the contact-onset pose, the current pose, and pose velocity.

Hardware choices support this sensing scheme. The finger has an open skeletal architecture of axially aligned load-bearing beams joined by compliant joints, which both enables large passive deformation and provides an unobstructed sensing corridor so a base-mounted camera can see the marker even under large deformation. Keeping all sensors off the soft body preserves compliance and avoids the integration problems of embedded sensors. A host PC runs perception and regulation, a PID controller sends velocity commands through an MCU to the gripper motor, and a separate temperature loop drives the heating film to set stiffness.

Validation proceeded in stages: a UR5e robot applied controlled normal indentation to characterize temperature-dependent deformation; ICFN was trained on 8,258 samples spanning 25–60 °C and forces up to 8 N; RCFN was trained on 810 relaxation sequences of 100 s each under fixed contact across four contact locations and multiple temperature–force conditions; all data were split 70%/15%/15% into training, validation, and test sets. Finally, the full loop was tested on a spherical force measurement unit with an internal pressure sensor providing ground-truth normal force during a 280 s hold.

Why This Matters

Impact on research. The paper argues for a "mechanism–AI co-design" view: the mechanical structure defines which interactions are feasible, and learning compensates the uncertainty that remains in the viscoelastic dynamics. It also contributes a reduced, temperature-coupled force representation that makes long-horizon force drift observable, learnable, and compensable online without embedded force sensors — an approach the authors state remains largely unexplored for variable-stiffness soft grippers. Details of SMP dynamic mechanical analysis, finger fabrication, feature definitions, hyperparameters, and an ablation study are deferred to the Supplementary Materials.

Real-world applications:

• Food and fragile-object handling, where stiffness must drop to avoid crushing items like balloons or deformable containers.
• Liquid handling and pouring tasks, demonstrated with a paper cup at a 1 N target force.
• Human–robot handover, demonstrated by passing a glass cup to a person under a stiff 6 N regime.
• Non-contact material or object state identification, demonstrated by using infrared thermography to distinguish cups by water temperature.

Industry relevance. The work targets long-duration manipulation reliability, which matters for logistics, assembly, and service robotics where grasps must persist for minutes rather than milliseconds. Because the sensing hardware is mounted on the base and no force sensor is embedded in the soft body, the design is potentially easier to manufacture and integrate, and the reported inference latencies of approximately 4.8 ms and 4.94 ms are compatible with real-time control.

Future Directions

• Extending the thermo-coupled relaxation compensation formulation to broader soft-hand morphologies beyond the two-finger design presented.
• Further investigating the underlying material mechanisms of thermally coupled relaxation, which the authors list as an open question.
• Addressing the increased variability seen at approximately 60 °C, where NRMSE spread widened (0.140 ± 0.099) and ground-truth end-force standard deviation rose to 0.121 N, to determine how far up the thermal range the method remains dependable.
• Reducing the reliance on the instantaneous force estimate, given that ICFN's MAE of 0.651 N over a 0–10 N range is substantially larger than the 0.066 N holding error achieved after relaxation compensation.

Target Audience

This paper is most useful to soft robotics and robotic manipulation researchers, particularly those working on variable-stiffness grippers, viscoelastic modeling, and force control without embedded sensors. It also suits engineers designing grasping end-effectors for delicate or long-duration tasks, and machine learning researchers interested in physics-informed or physics-constrained models that combine a reduced analytical representation with learned regressors. Readers without a background in continuum viscoelasticity or thermal actuation will find the modeling section demanding.

Authors’ abstract

Achieving stable, sustained grasping with soft robotic hands remains a fundamental challenge. Compliance enables safe and adaptive contact, yet the intrinsic viscoelasticity of soft polymers leads to stress relaxation and a continuous decay of grasping force during holding. Inspired by human grasping, which combines phase-dependent stiffness regulation with continuous sensing and feedback, this paper presents an integrated structure--perception--learning framework. We develop a variable-stiffness soft gripper that uses onboard vision and infrared thermography to track deformation and the temperature field in real time, preserving continuous tracking of the interaction state. To mitigate relaxation-induced force decay, we propose a temperature-coupled viscoelastic force representation, together with a physics-informed learning model, to reconstruct the force trend and provide explicit compensation during holding. Experiments show that, in a 280s force-controlled grasp-and-hold task, the proposed method maintains the desired force with a mean absolute error of 0.066N, outperforming fixed-aperture and instantaneous-only baselines by 80% and 95%, respectively. Overall, the results support a mechanism--AI co-design view: mechanisms shape feasible interactions, while learning compensates remaining uncertainty in viscoelastic dynamics, together enabling stable, sustained grasping.