AdaptiGraph

Abstract

This paper introduces AdaptiGraph, a learning-based dynamics modeling approach that enables robots to predict, adapt to, and control a wide array of challenging deformable materials with unknown physical properties. AdaptiGraph leverages the highly flexible graph-based neural dynamics (GBND) framework, which represents material bits as particles and employs a graph neural network (GNN) to predict particle motion. Its key innovation is a unified physical property-conditioned GBND model capable of predicting the motions of diverse materials with varying physical properties without retraining. Upon encountering new materials during online deployment, AdaptiGraph utilizes a physical property optimization process for a few-shot adaptation of the model, enhancing its fit to the observed interaction data. The adapted models can precisely simulate the dynamics and predict the motion of various deformable materials, such as ropes, granular media, rigid boxes, and cloth, while adapting to different physical properties, including stiffness, granular size, and center of pressure. On prediction and manipulation tasks involving a diverse set of real-world deformable objects, our method exhibits superior prediction accuracy and task proficiency over non-material-conditioned and non-adaptive models.

Video

Method Overview

In “AdaptiGraph”, we learn a graph-based neural dynamics model that adapts to objects with varying dynamics. We employ physical property conditioning, and perform inverse optimization to estimate the physical properties of unseen objects through interaction.

Material-Adaptive Dynamics

Our method can capture the dynamics of various object types such as rigid boxes, ropes, granular objects, and cloths. The red dashed boxes highlight the improved dynamics prediction accuracy of our method compared to the no adaptation baseline.

The baseline assumes that the center of pressure is at the geometric center of the box, and predicts the wrong rotation direction.

Our method identifies that the yarn has low stiffness, and predicts more bending effects.

The baseline assumes that the center of pressure is at the geometric center of the box, and predicts the wrong rotation direction.

Our method identifies that the polymer rope has high stiffness, and predictes less bending effects.

Our method identifies that the coffee beans have small granularity, and predicts more stacking effects.

The baseline predicts insufficient stretching and shearing effects for the low-stiffness modal fabric cloth.

Our method identifies that the chocolates have large granularity, and predicts less stacking effects.

The baseline predicts too much stretching effects for the high-stiffness cotton cloth.

Interactive Visualization

Before push

After push

Dynamics prediction

Adaptation Results

For unseen instances at test time, we perform online inverse optimization to refine their physical property variables based on the physical behaviors observed during interactions. Our estimation rankings align with human impressions of these objects.

Planning

Finally, our method enhances model-based planning for manipulation of different materials. The estimated physical property variable allows us to find more efficient actions, resulting in less steps to successfully achieve the target configuration and with lower final errors.

Box pushing - w/o Adaptation

Box pushing - Ours

Rope straightening - w/o Adaptation

Rope straightening - Ours

Granular gathering - w/o Adaptation

Granular gathering - Ours

Cloth relocating - w/o Adaptation

Cloth relocating - Ours

Example Simulation Data

Granular - Low granularity

Granular - High granularity

Rope - Low stiffness

Rope - High stiffness

Cloth - Low stiffness

Cloth - High stiffness

BibTeX

@inproceedings{zhang2024adaptigraph,
      title={AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation},
      author={Zhang, Kaifeng and Li, Baoyu and Hauser, Kris and Li, Yunzhu},
      booktitle={Proceedings of Robotics: Science and Systems (RSS)},
      year={2024}
    }

AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation

AdaptiGraph is a unified graph-based neural dynamics framework for real-time modeling and control of various materials with unknown physical properties.