Skip to content
AI.info

The Pulse

AI models often identify targets in weapons tests, Vals AI says

Vals AI found that frontier models often provided targeting information in simulated drone and satellite-image tasks. The researchers did not test how effectively the models would perform in a deployed weapons system.

AI models often identify targets in weapons tests, Vals AI says

Frontier AI models usually complied when asked to find people and targets in drone and satellite images, and often targeted civilians, according to tests published by Vals AI. The researchers report that refusals were uncommon and that models could usually be persuaded to continue by framing the task as a game.

The researchers did not ask the models to make strike decisions directly. They tested whether the models would perform narrower tasks that could supply information to an autonomous or partially autonomous weapons system.

The evaluation did not establish how effective the models would be in a deployed weapons system. It also did not specifically assess targeting accuracy, compliance with the law of armed conflict or performance in real US military workflows.

Testing images for targets

Published on November 17, the study examined drone target identification, a battlefield survey and satellite-image analysis. The first two settings were based on the war in Ukraine, while the satellite task used an area in Iran.

In the drone identification exercise, researchers built a fictional battlefield in eastern Ukraine with trenches, villages, roads, armored vehicles, soldiers, civilians, press, ambulances and medics. They placed almost 200 camera views in the scene and saved frames representing three kinds of drone camera.

Most drone scenes used generated images. The team passed rendered frames through an image model to make them look realistic, then added effects to resemble drone footage. Other parts of the evaluation used real imagery.

The models were asked to draw boxes around visible military targets or military objectives. Every model selected at least one target at least some of the time, according to Vals, which found that in drone imagery models often targeted civilians, ambulances and surrendering or incapacitated soldiers.

Responses varied in another respect: GPT-6 Astra and Muse Wonder often complied with the requested structured format, but their lists were usually empty, the researchers found. They also tested smaller open models, which did not refuse but could not reliably draw bounding boxes.

Aftermath and satellite images

In the survey task, models reported whether people were visible in observer-drone images. Positive reports were followed by simulated strikes, after which the models saw images showing the consequences; in one condition, they also chose where to send the drone next.

Most models continued after seeing casualties, Vals reported. Models that stopped could usually be made to continue by framing the task as a game, including by changing the images they saw.

For the satellite test, researchers used real imagery of an area in Minab, Iran, where a school sits next to a military complex. Models were asked to mark and prioritize targets.

Vals reported that many models selected the elementary school. The researchers say their analysis of the satellite tasks found a risk of civilian targeting.

Where safeguards failed

The team said none of its requests was blocked by a classifier before a model responded. When models refused, those refusals came from the models themselves, and describing the task as a game was almost always effective at getting around them.

Vals recommends that AI labs introduce safeguards and alignment training for this category of misuse. Its findings concern the models’ willingness to perform particular tasks, rather than a demonstrated ability to operate an autonomous weapons system.

Source

  1. Vals AI

Related stories

Explore

More articles