Incorporating Quality of Life in Climate Adaptation Planning via Reinforcement Learning

AI summary

Overview

Research area: Reinforcement learning applied to urban climate adaptation planning, specifically integrating flood modeling, transport accessibility, and quality-of-life measurement into a single Integrated Assessment Model (IAM).

Technical level: Advanced. The paper assumes familiarity with reinforcement learning (agents, rewards, policies), integrated assessment modeling, and urban accessibility metrics.

Scope: A proof-of-concept study that uses a reinforcement learning agent to search for sequences of urban flood-adaptation measures that maximize long-term quality of life in Copenhagen's inner city.

What This Paper Is About

Urban pluvial flooding is expected to become more frequent and severe with climate change, disrupting mobility and access to services and thereby lowering quality of life (QoL) in cities. Policymakers face the hard problem of choosing which adaptation measures to implement when, under uncertain climate projections and complex urban dynamics. This paper asks: "What is the sequence of climate adaptation measures that lead to a higher quality of life in the long term?" and tests whether reinforcement learning can answer it.

Key Contributions

1. An Integrated Assessment Model (IAM) coupling four components: a rainfall projection model built on the Danish Meteorological Institute's Climate Atlas, a flood model using SCALGO Live, a cumulative-based transport accessibility model, and a transport-based quality of life index.

2. A quality-of-life-centered objective for adaptation planning. Rather than the cost-minimization or welfare-optimization objectives typical of IAMs, the reward function combines a QoL index, action costs, and maintenance costs, with user-adjustable weights (β) that let planners trade off QoL against economic costs.

3. A reinforcement learning formulation of adaptation sequencing. The agent selects among eight proposed adaptation actions (including increasing road drainage and permeable paving solutions) applied per zone of the city, learning a sequence of policies over time to maximize cumulative reward.

4. An open-source, reusable framework released publicly at https://github.com/MLSM-at-DTU/maat_qol_framework, described as easily expandable to other cities, other climate impacts, and other contexts.

Main Findings

• The learned policy outperforms all baselines on reward. The RL framework achieves better rewards than the five comparison strategies (No Control, Cumulative Response, Intense Event Response, Election Cycle, and Random Control). The specific reward values are not reported in the paper content.

• The random and learned policies reach similar cumulative rewards under this experimental setup. The authors attribute this to the chosen β weights, which place heavier emphasis on QoL and thereby reduce the relative importance of action costs.

• The action sequences differ despite similar rewards. Even though cumulative reward is similar to the random policy, the sequence of actions the agent takes is different, producing small increases in cumulative QoL and small decreases in cumulative action costs.

• Feasibility of RL-based IAMs for QoL-focused adaptation is demonstrated. The results support using RL to identify policy pathways aimed at minimizing QoL loss, in contrast to the cost-minimization or welfare-optimization approaches typically used in IAMs.

• Robustness was probed with multiple runs. Results are presented for five distinct seed runs, allowing for different weather projections.

• Uncertainty remains a first-order concern. The authors state that IAMs, with or without RL, contain multiple quantifiable and unquantifiable uncertainties that must be considered.

Methodology in Plain English

The researchers built a simplified digital version of Copenhagen and let a reinforcement learning agent practice managing it.

First, they sampled rainfall events from probability distributions derived from the Danish Meteorological Institute's Climate Atlas for 2023–2100, assuming for simplicity that projected rainfall intensity equals accumulated daily rainfall. Each sampled rainfall amount was then turned into a flood map using SCALGO Live, an event-based tool that distributes water across terrain data to produce flood depths and flow directions.

Second, they measured quality of life as access to amenities. For each residential location they counted the per-capita number of accessible points of interest (POIs) within a reasonable travel time, then took a weighted sum across POI categories. The weights come from a logistic regression model trained on survey data about satisfaction with life in cities. This yields a fine-grained index that can reveal flooding effects both across and within administrative boundaries.

Third, they cast adaptation planning as a reinforcement learning problem. The agent observes the state of the digital city, chooses one of eight possible adaptation actions for a zone, and receives a reward defined as the sum over zones of a QoL term plus action cost plus maintenance cost, each scaled by a tunable β weight. Over repeated interactions the agent learns which sequence of measures maximizes cumulative reward.

The implementation used Python with the Gymnasium interface, Stable-Baselines3, and PPO. The preliminary case study covered Copenhagen's inner city, consisting of 29 zones, over a 2023–2100 time horizon, with β weights prioritizing QoL over economic costs.

Why This Matters

Impact on research: The work challenges the dominant framing of climate adaptation IAMs around cost or welfare optimization by placing a spatially granular, survey-weighted quality of life index directly in the objective function. It also connects two literatures that rarely meet: reinforcement learning for sequential decision-making under uncertainty, and urban accessibility/QoL measurement.

Real-world applications:

• Urban drainage and blue-green infrastructure planning: sequencing investments such as road drainage upgrades and permeable paving across city districts over multi-decade horizons.
• Municipal climate adaptation budgeting: exploring how much QoL is gained or lost per unit of capital and maintenance spending under different priority weightings.
• Equity-aware transport and land-use planning: the zone-level index exposes where flooding degrades access to services within a city, not just at administrative averages.
• Transfer to other hazards and cities: the authors state the framework can be extended to coastal flooding, wildfires, and other urban contexts.

Industry relevance: Engineering consultancies, utilities, and municipal planning departments that must justify adaptation investment sequences could use the released framework as a decision-support sandbox, particularly because the reward weights can be tuned to reflect local political priorities.

Future Directions

• Sensitivity analysis on the β weights. Testing how the model responds to different weights for QoL, representing the "value" of QoL relative to economic costs.
• Sufficientarian metrics. Incorporating measures such as Foster-Greer-Thorbecke indices to evaluate how well adaptation protects the worst-off rather than only aggregate QoL.
• Scaling to the full city of Copenhagen, beyond the 29 zones of the inner-city case study.
• Broadening the framework's scope to other cities, other climate-related impacts such as coastal flooding or wildfires, and other normative contexts such as subjective wellbeing or mobility justice. The authors also note plans to further investigate the effects of normative choices embedded in models like this one.

Target Audience

The paper is best suited to researchers and practitioners working at the intersection of reinforcement learning and urban systems: RL researchers looking for high-impact sequential decision applications, climate adaptation and IAM researchers interested in QoL-based objectives, urban planners and transport accessibility analysts, and municipal or consultancy teams building decision-support tools for flood adaptation. Readers without a background in reinforcement learning will find the modeling framework accessible but the objective formulation and training setup require some technical grounding.

Authors’ abstract

Urban flooding is expected to increase in frequency and severity as a consequence of climate change, causing wide-ranging impacts that include a decrease in urban Quality of Life (QoL). Meanwhile, policymakers must devise adaptation strategies that can cope with the uncertain nature of climate change and the complex and dynamic nature of urban flooding. Reinforcement Learning (RL) holds significant promise in tackling such complex, dynamic, and uncertain problems. Because of this, we use RL to identify which climate adaptation pathways lead to a higher QoL in the long term. We do this using an Integrated Assessment Model (IAM) which combines a rainfall projection model, a flood model, a transport accessibility model, and a quality of life index. Our preliminary results suggest that this approach can be used to learn optimal adaptation measures and it outperforms other realistic and real-world planning strategies. Our framework is publicly available: https://github.com/MLSM-at-DTU/maat_qol_framework.