ASBI: Leveraging Informative Real-World Data for Active Black-Box Simulator Tuning

AI summary

Overview

Research area: Robotics — simulation parameter tuning / sim-to-real calibration, combining Simulation-Based Inference (SBI) with active data collection.

Technical level: Advanced. The paper assumes familiarity with Bayesian inference, Shannon entropy and information gain, Neural Posterior Estimation (NPE), mixture-density networks, and Bayesian experimental design. The framing is accessible, but the method section is mathematically dense.

Scope (one sentence): The paper proposes ASBI, a framework in which a robot actively selects actions that maximize expected information gain so that a black-box simulator's parameters can be estimated accurately from a small number of real-world executions.

What This Paper Is About

Black-box simulators in robotics expose sophisticated dynamics models but not their likelihoods, so tuning simulator parameters to match the real world is hard. Standard Simulation-Based Inference (SBI) handles this by learning a posterior from offline real observations plus forward simulations, but in black-box settings nobody knows in advance which observations carry information about which parameters. ASBI solves this by letting a robot interact with the real world online and pick, at each round, the action expected to reduce uncertainty about the parameters the most.

Key Contributions

1. The ASBI framework itself: an online, active parameter-estimation framework for black-box simulators in which a robot sequentially collects real-world observations and refines the posterior round by round, rather than relying on a fixed offline dataset.

2. An action-extended Neural Posterior Estimator: a straightforward extension of NPE in which the neural network takes a pair (observation, action) as input and outputs the parameters of a k-Mixture of Gaussians over the simulation parameters, approximating p(θ | x, ξ) without a likelihood.

3. A likelihood-free action optimizer based on information gain: because analytic information gain requires the inaccessible likelihood, ASBI approximates the utility U(ξ) using the trained posterior estimator plus forward simulations, and selects the maximizing action (a myopic, one-step-ahead optimization).

4. Empirical validation including a real robot: three sim-to-sim experiments (a numerical model, a box-collision task, and a particle-parameter estimation task) plus a real-to-sim experiment estimating cubic-particle simulation parameters for two real materials, beads and gravel, using a bucket-pouring action on a robot manipulator.

Main Findings

• ASBI beats random-action and likelihood-based baselines on the numerical model. Across ten runs of a 4-round estimation, the Reproduction Error (RepErr) at ξ = −3.0 was 34.5 ± 10.7 for ASBI, 127 ± 76 for NSBI, 940 ± 556 for ALHI, and 895 ± 264 for SeqBED. At ξ = 0.0: 1.95 ± 0.49 (ASBI), 6.35 ± 3.74 (NSBI), 46.7 ± 27.6 (ALHI), 44.4 ± 13.2 (SeqBED). At ξ = 3.0: 1.60 ± 0.36 (ASBI), 2.01 ± 1.56 (NSBI), 4.35 ± 1.32 (ALHI), 2.92 ± 1.44 (SeqBED). Lower RepErr means better calibration.

• Action selection adapts to the current uncertainty. In the numerical model the optimal actions were (ξ₁*, ξ₂*, ξ₃*, ξ₄*) = (5.0, 0.5, 0.0, −0.5). The first round with ξ₁* = 5.0 mainly reduced uncertainty in θ₂, while the later actions near zero reduced uncertainty in θ₁ — the sequence changes as the prior (belief) changes.

• Posteriors concentrate around the true parameters. Using the informative observation from the optimal action at each round, ASBI produced a posterior sharply peaked around θ_true = (−3, 1) in the numerical model.

• Likelihood-free approximation outperformed surrogate-likelihood approaches. ALHI, which trains a surrogate likelihood via Neural Likelihood Estimation (NLE) to compute information gain analytically, performed substantially worse than ASBI in the numerical study, which the authors attribute to the difficulty of likelihood modeling when simulator outputs are high-dimensional.

• SeqBED was only tested on the toy model. The authors restricted comparison with Sequential Bayesian Experimental Design (SeqBED) to the numerical model because it requires a prohibitively large number of forward simulations for robotics applications.

• Real-robot applicability demonstrated. The paper reports a practical real-to-sim application estimating cubic-particle simulation parameters corresponding to beads and gravel with a bucket-pouring action. Quantitative results for the box-collision and particle-parameter estimation tasks, and the specifics of the real-to-sim results, are not included in the truncated content provided here.

• Assumptions stated explicitly. ASBI assumes the simulator and the real world share the same dynamics model with no noise or model mismatch, and it uses myopic (one-step-ahead) optimization rather than long-horizon planning.

Methodology in Plain English

The robot faces a simulator whose internal model it cannot inspect. Instead of guessing which real-world measurements would reveal the simulator's parameters, the robot runs a loop of rounds.

Step 1 — Retrain the estimator. In each round, the team samples candidate parameters from the current belief (the prior), selects actions uniformly at random, runs forward simulations, and produces a training set of (parameters, simulated observation, action) triples. A neural network is trained on these triples so that, given an observation and an action, it outputs the parameters of a mixture of Gaussians describing which simulation parameters are plausible.

Step 2 — Pick the best next action. The question "which action will teach me the most?" is normally answered by computing information gain, the expected drop in uncertainty from prior to posterior — but that calculation needs the likelihood, which the black-box simulator does not provide. ASBI substitutes the trained posterior estimator into the information-gain formula and approximates the resulting integral by Monte Carlo sampling: sample parameters from the prior, simulate observations for a candidate action, evaluate the learned posterior, and average the log ratio of posterior to prior. The action with the highest score wins.

Step 3 — Act, observe, update, repeat. The robot executes the chosen action in the real world, obtains an observation, and feeds that observation and action into the trained estimator to get an updated posterior, which becomes the prior for the next round.

The numerical testbed used x = f_ξ(θ) = θ₁ exp(3 − ξ) + θ₂ ξ + ε with standard normal noise, a two-dimensional parameter θ = (θ₁, θ₂) each in [−5, 5] under a uniform prior, a one-dimensional action ξ on the discrete grid {−5.0, −4.5, …, 4.5, 5.0}, a network with two hidden layers of size (128, 128) outputting 5-MoGs, trained for 2000 iterations with batch size 50, and M = 1000 samples for the utility approximation. Evaluation used both the log-probability of the true parameter at each round and RepErr, computed by sampling N = 10⁵ parameter values from each estimated posterior.

Baselines: NSBI (ASBI without action optimization — random actions), ALHI (surrogate likelihood for information gain, via NLE), NLHI (ALHI without action optimization), and SeqBED.

Why This Matters

Research impact. The paper targets a gap the authors identify explicitly: likelihood-free parameter estimation has advanced, but no effective framework existed for adaptively selecting informative actions in an online setting, as likelihood-based approaches do. ASBI brings information-gain-driven experimental design into likelihood-free, black-box robotics simulators, and it differs from prior online SBI work in robotics (online BayesSim, Neural Posterior Domain Randomization) by strategically choosing actions rather than only collecting data.

Real-world applications (as motivated or demonstrated in the paper):

• Simulation calibration for granular and deformable materials the paper names as hard to model with rigid-body simulators — cloth, ropes, cables, powders, and soils — with beads and gravel demonstrated concretely.
• Sim-to-real transfer for robot manipulators, where accurate simulator parameters reduce the cost and safety risk of real-world experiments.
• Automated tuning of commercial or third-party black-box simulators that expose advanced models as black-box features.
• Robotic pouring and material-handling tasks, where the estimated cubic-particle parameters for beads and gravel matter directly.

Industry relevance. Robotic simulation is widely used because real experiments are expensive and risky, and the paper cites black-box simulators (e.g., Mittal et al. 2023; Algoryx Simulations 2025) as tools now actively used in robotics. A method that reaches accurate parameters from a small number of costly real executions speaks directly to industrial engineering workflows, where engineer time spent hand-tuning simulator parameters is a bottleneck.

Future Directions

• Long-horizon planning. ASBI assumes myopic optimization — the robot optimizes only one step ahead. Extending to multi-step or non-myopic planning is an open question.
• Relaxing the simulation assumptions. The framework assumes the simulator and real world have identical dynamics with no noise or model mismatch. Robustness to model mismatch and real-world stochasticity is unaddressed.
• Scaling the empirical validation. Results for the box-collision and particle-parameter estimation tasks, and for a broader set of real materials beyond beads and gravel, would show how far the method generalizes; SeqBED's restriction to the toy model due to simulation cost also leaves a comparison gap in the robotic setting.
• Scaling to higher-dimensional parameter and observation spaces, and reducing the computation required both for training the estimator and for evaluating the utility at each round.

Target Audience

Robotics researchers working on sim-to-real transfer and simulator calibration; practitioners who need to tune black-box or commercial simulators for manipulation and granular-material tasks; and machine-learning researchers interested in Simulation-Based Inference, Neural Posterior Estimation, and Bayesian experimental design who want to see information gain used without an explicit likelihood. Readers without a background in Bayesian inference and probabilistic modeling will find the method section demanding.

Authors’ abstract

Black-box simulators are widely used in robotics, but optimizing their parameters remains challenging due to inaccessible likelihoods. Simulation-Based Inference (SBI) tackles this issue using simulation-driven approaches, estimating the posterior from offline real observations and forward simulations. However, in black-box scenarios, preparing observations that contain sufficient information for parameter estimation is difficult due to the unknown relationship between parameters and observations. In this work, we present Active Simulation-Based Inference (ASBI), a parameter estimation framework that uses robots to actively collect real-world online data to achieve accurate black-box simulator tuning. Our framework optimizes robot actions to collect informative observations by maximizing information gain, which is defined as the expected reduction in Shannon entropy between the posterior and the prior. While calculating information gain requires the likelihood, which is inaccessible in black-box simulators, our method solves this problem by leveraging Neural Posterior Estimation (NPE), which leverages a neural network to learn the posterior estimator. Three simulation experiments quantitatively verify that our method achieves accurate parameter estimation, with posteriors sharply concentrated around the true parameters. Moreover, we show a practical application using a real robot to estimate the simulation parameters of cubic particles corresponding to two real objects, beads and gravel, with a bucket pouring action.