The Pulse
LG AI Research Uses LLMs to Redesign Reinforcement-Learning Interfaces
LG AI Research introduced LOAM, a framework that uses large language models to generate observation and action models for reinforcement-learning agents. In simulated robot tasks, the team reports more than three times faster learning on Hum
AI.info Team · Written with AI · fact-checked ·
On October 26, LG AI Research described a system that asks large language models to design the interface between a reinforcement-learning agent and its simulated environment. The framework, called LOAM, generates the functions that determine what information an agent receives and how its decisions become commands for a robot. The work addresses a part of reinforcement learning that often remains hand-built even as researchers automate reward design.
LOAM changes what the agent sees and controls
Reinforcement-learning agents learn by acting in an environment and using feedback to improve their decisions. Their performance depends not only on the learning algorithm or reward, but also on the observation space—the information available to the agent—and the action space, which defines the decisions it can make. In complicated tasks, people often choose those inputs and controls by hand.
LOAM uses a task’s goals and end conditions, the robot’s structure, and descriptions of available variables to generate Python functions for observation and action. One function transforms raw simulator data into features the policy can use; the other maps the policy’s output to commands the environment can execute. The generated functions wrap the existing environment, so researchers can retain their reinforcement-learning algorithm.
LG AI Research traces the work to an industrial scheduling project that began in 2022 with LG Chem’s Daesan naphtha-cracking center. The team says understanding the plant and translating operators’ decisions into observations, actions and rewards took more effort than implementing and training the reinforcement-learning algorithm. The experience pointed to a recurring design problem: deciding which operating variables an agent needs and what choices it should be allowed to make.
More than three times faster on locomotion tests
LG AI Research evaluated LOAM with FastTD3, a reinforcement-learning algorithm, on HumanoidBench and tasks in Isaac Lab. On HumanoidBench’s stand, walk and run tasks, LOAM reached high returns more than three times faster than the benchmark’s hand-designed FastTD3 environment, the company says. Its blog also reports faster learning or higher final performance on most of the other tested tasks, rather than claiming a win on every task.
The paper’s authors summarize the broader result this way: Empirical evaluations on HumanoidBench and Isaac Lab demonstrate that LOAM consistently outperforms handcrafted baselines in both learning speed and asymptotic performance. Deunsol Yoon et al., “Designing Observation and Action Models for Efficient Reinforcement Learning with LLMs,” Proceedings of Machine Learning Research.
The experiments covered 12 HumanoidBench tasks, from locomotion to object manipulation, and three Isaac Lab manipulation tasks. Those Isaac Lab tests included a Franka robot arm lifting a cube and opening a drawer, plus an Allegro hand reorienting a cube. The reported results come from simulation; the post does not present a deployment on a physical robot.
LOAM-Race spends training time on promising designs
LLMs can produce different observation and action functions from the same prompt, and those design differences can affect how well a policy learns. Testing every candidate until its policy converges would consume substantial simulation and training time. LOAM-Race addresses that cost by letting candidate designs compete during a shared training process.
After short initial runs, LOAM-Race uses each candidate’s performance and learning curve to estimate its later potential. Bayesian ridge regression helps the system account for uncertainty as well as predicted performance, guiding which candidates receive the next portion of the training budget. The approach can limit spending on designs with little promise while preserving resources for candidates that may improve later.
That allocation has a trade-off: because LOAM-Race divides training among candidates, it can learn more slowly at the start than a single LOAM design. LG’s account says the racing method helped on tasks such as Cube and Insert_small, where generated designs varied in quality, while the Package task remained difficult for all tested methods. The company identifies complex, long-horizon manipulation as an area requiring more work.
Real-robot use remains a stated goal
LG AI Research says it plans to extend LOAM to home robots and manufacturing robots. Its proposed next steps include preparing robot specifications, environment APIs and task documents in formats an LLM can use, then checking generated models in simulation for executable code, physical plausibility and safety.
For now, LOAM’s evidence supports automated interface design in the tested simulated benchmarks, not a general replacement for robotics experts or a demonstrated path to safe physical deployment. The immediate result is narrower and measurable: for three humanoid locomotion tasks, a language model’s generated observation and action functions helped FastTD3 reach high returns in less than a third of the baseline’s learning time.