Structural Plasticity as Active Inference: A Biologically-Inspired Architecture for Homeostatic Control

AI summary

Overview

• Research area: Computational neuroscience and biologically-inspired machine learning, specifically neural network architectures grounded in the Free Energy Principle and active inference (cs.NE).
• Technical level: Intermediate. The paper assumes familiarity with concepts such as prediction error, Hebbian plasticity, and reinforcement learning benchmarks, but the architecture itself is described in fully explicit algorithmic steps.
• Scope: The paper introduces and empirically tests a single architecture — the Structurally Adaptive Predictive Inference Network (SAPIN) — on one benchmark task (Cart Pole), with a follow-up "locking" experiment and a comparison of punishment conditions.

What This Paper Is About

Most neural networks learn using global backpropagation, which is widely considered biologically implausible because it requires error information that is not available locally at each synapse. This paper proposes SAPIN, a model in which units on a 9x9 grid learn only from local prediction errors, and in which cells can additionally physically move across the grid to find more predictable positions. The goal is to test whether the intrinsic drive to minimize local prediction error — with no external reward signal — is enough to solve a control task.

Key Contributions

1. A dual-plasticity architecture. SAPIN combines a local, Hebbian-like synaptic rule (updating directional strengths s_i and a homeostatic expectation E_i) with a structural plasticity rule in which processing cells migrate across a 2D grid based on long-term average prediction error ("desire").
2. A spatially flexible propagation scheme. Instead of fixed layers or a fixed spiking threshold, activation propagates by a winner-takes-all order — the un-activated cell with the highest absolute activation |V_j| propagates next — with distance decay and angular weighting determining how signals reach neighboring cells.
3. A reward-free demonstration on Cart Pole. The paper shows the architecture can solve the CartPole-v1 task using only prediction-error minimization, and reports that a punishment mechanism based on the DishBrain experiment made no noticeable difference.
4. A locking experiment addressing stability. Freezing all learning after the first 500-step success produced a stable policy, with locked networks maintaining an average 82% success rate over 100 post-locking episodes, repeated across 100 successful agents.

Main Findings

• Cart Pole was solved, often quickly. The network frequently achieved success (500 steps) within the first 10 episodes, and its behavior showed clear corrective actions to balance the pole.
• Success was unstable under continual learning. A network that reached 500 steps might fail after only 10 steps in the very next episode. The paper attributes this to the learning rules not guaranteeing convergence, so a good policy could be forgotten as the network continued to explore its state space.
• Bad initial positions could stall learning. Poor random initial positions for the processing cells could prevent the agent from succeeding for 100 episodes or more.
• Locking produced robust performance. When the agent first achieved 500 steps, a global flag disabled all synaptic plasticity and structural plasticity. Evaluated over 100 episodes and repeated for 100 different successfully trained agents, locked networks maintained an average 82% success rate. The paper describes this as a computational analogue to synaptic consolidation.
• Punishment made no meaningful difference. Three conditions were compared — punishment on catastrophic failure only, punishment on catastrophic failure plus probabilistic punishment during poor performance, and no punishment at all — and all three produced very similar results. The network learned to balance the pole even when never punished for failing.
• An alternative movement rule was worse. A variant in which cells moved toward the smallest variation in error rather than to reduce error magnitude was less successful, suggesting that matching a homeostatic set-point is a more effective drive than seeking predictability alone.

Methodology in Plain English

SAPIN is laid out on a 9x9 grid containing 4 input cells (fixed in the leftmost column at (0,1), (0,3), (0,5), (0,7)), 2 output cells (fixed in the rightmost column at (8,2) and (8,6)), and 30 processing cells placed at random unoccupied coordinates. Most numerical values — inputs, activations, weights — are kept in the range [-1, 1].

At each environment timestep, the normalized 4-dimensional Cart Pole state vector is loaded into the four input cells, and a wave of activation propagates across the grid. The next cell to fire is whichever un-activated cell has the highest absolute activation. How much signal a receiver gets depends on the Manhattan distance to the sender (decay of 1.0 at distance 0, 0.75 at distance 1, 0.25 at distance 2, and 0.0 at distance 3 or more) and on an angular weighting that interpolates the sender's four directional strengths using sine and cosine terms. Nonlinearity enters through tanh(V). The action is chosen from the two output cells: action 0 if V_o0 > V_o1, otherwise action 1.

After each action, each processing cell computes a local error, error_i = V_i − E_i, the gap between its actual activation and its learned expectation. With learning rate η = 0.02, it updates its expectation by (η/2)·error_i and updates its directional strengths using the directional proportions of its influx multiplied by the same error, clipping the strengths to [-1, 1]. A per-cell lock flag skips this update entirely when engaged.

Every macro-episode — defined as 4 full environment episodes — the network considers structural plasticity. Each cell's "desire" is the absolute difference between its long-term average activation and its expectation, and only cells with desire at or above θ_D = 0.1 (or a small random chance of random() < 0.025) are candidates. Candidates are sorted by desire, and each picks a movement axis either randomly with probability ε_rand = 0.05 or by a weighted choice over its average directional influx. A cell that is over-activated moves away from its dominant signal source; an under-activated cell moves toward it. Occupied squares block movement.

CartPole-v1 from Gymnasium was used, with the state normalized using the environment's standard bounds (cart position ±2.4, cart velocity ±4.0, pole angle ±0.209 rad, pole angular velocity ±4.0). An episode counted as successful if the pole was balanced for 500 consecutive timesteps.

The punishment mechanism, inspired by the DishBrain experiment's use of unpredictable stimuli, worked two ways. On catastrophic failure (pole falls), 10 epicenters were created at random grid locations, each emitting a random value p in [-1, 1]. During non-terminal states with a high pole angle (between 4 and 12 degrees), there was a 1-10% chance — scaling with the angle — of a punishment event using 1-30% of the 10 epicenters. In both cases the punishment values triggered a propagation wave whose resulting activations drove a synaptic update.

Why This Matters

The paper is a proof-of-concept that an agent can acquire a workable control policy from purely local, homeostatic error signals, without external reward, and that the same framework can learn where to place its computational resources rather than only how to weight connections. It sits at the intersection of active inference theory, biologically plausible credit assignment, and structural plasticity, and it offers one concrete mechanism — freezing plasticity after success — for the classic stability-plasticity dilemma.

Real-world applications implied by the work:

• Homeostatic control systems where maintaining a stable state (temperature, pressure, balance, power draw) is the objective, and where labeled rewards are hard to define.
• Continually adapting robotics, where sensors and actuators may be repositioned and the controller must relearn based on where its resources sit.
• Neuromorphic and edge hardware, where local update rules avoid the need for global error broadcast and centralized backward passes.
• Adaptive sensor placement, since the structural plasticity mechanism is explicitly a rule for relocating units to improve the predictability of their inputs.

Industry relevance centers on any setting where defining a reward function is awkward but "keep things in the normal range" is a natural specification. The paper's own caveat is that the approach suits homeostatic problems and is less obviously suited to tasks requiring long-term planning.

Future Directions

• Deep active inference. The paper notes the agent currently minimizes immediate prediction error and suggests learning a temporal model that minimizes expected future free energy, allowing short-term homeostasis to be sacrificed for longer-term goals.
• Explain why punishment failed. Future work should investigate whether the local homeostatic updates are simply stronger than the punishment-driven updates, or whether the random punishment signal was too noisy to give a useful learning gradient.
• Better initial configurations. The authors propose using a genetic algorithm to evolve optimal initial positions for the processing cells, which the plasticity mechanisms would then fine-tune.
• A continuous rather than discrete system. Moving to continuous movement is expected to decrease the step size when cells move and greatly stabilize the model. They also propose richer cell state representations (larger look-up tables with bins connecting inputs to values), grid configurations wrapped around a cylinder rather than flat, and placing the cells directly onto the pole so that cell movement physically affects the environment.

Target Audience

Researchers and graduate students working on biologically plausible learning rules, active inference, predictive coding, and structural plasticity, as well as engineers interested in reward-free or homeostasis-driven control. The paper is most useful to readers who want an explicit, implementable architecture rather than a high-level theoretical argument, and who are comfortable with reinforcement learning benchmarks such as Cart Pole.

Authors’ abstract

Traditional neural networks, while powerful, rely on biologically implausible learning mechanisms such as global backpropagation. This paper introduces the Structurally Adaptive Predictive Inference Network (SAPIN), a novel computational model inspired by the principles of active inference and the morphological plasticity observed in biological neural cultures. SAPIN operates on a 2D grid where processing units, or cells, learn by minimizing local prediction errors. The model features two primary, concurrent learning mechanisms: a local, Hebbian-like synaptic plasticity rule based on the temporal difference between a cell's actual activation and its learned expectation, and a structural plasticity mechanism where cells physically migrate across the grid to optimize their information-receptive fields. This dual approach allows the network to learn both how to process information (synaptic weights) and also where to position its computational resources (network topology). We validated the SAPIN model on the classic Cart Pole reinforcement learning benchmark. Our results demonstrate that the architecture can successfully solve the CartPole task, achieving robust performance. The network's intrinsic drive to minimize prediction error and maintain homeostasis was sufficient to discover a stable balancing policy. We also found that while continual learning led to instability, locking the network's parameters after achieving success resulted in a stable policy. When evaluated for 100 episodes post-locking (repeated over 100 successful agents), the locked networks maintained an average 82% success rate.