The Pulse
UNIDIR studies security risks of open-source AI models
UNIDIR is researching how openly available AI models could be misused by malicious non-state actors. Its forthcoming work will examine technical safeguards, regulation and international cooperation alongside the benefits of access.

UNIDIR is studying how openly available AI models could be misused by malicious non-state actors, with a forthcoming study set to examine possible policy responses. In a commentary published on 17 November, the institute warned that measures to address those risks have not kept pace with the spread of AI capabilities.
The work, led by UNIDIR’s Centre of Excellence on AI, Peace and Security, will examine open-source AI, its uses and potential regulatory responses. It forms part of the institute’s work on countering the proliferation of AI.
Degrees of openness
The institute distinguishes between fully closed models, cloud-based services, open-weight models and fully open-source models. It cautions that an open-or-closed distinction can be misleading because access to a model’s components varies.
The most common open models are open-weight models, UNIDIR says. Their weights are the parameters that help determine the responses a model produces. Open-weight models can be downloaded, run and modified, while components such as training data or code may remain private. By contrast, a fully open-source model has all its components available to everyone.
The terms open-source and open-weight are often used interchangeably, although they are not technically identical. The commentary argues that the two can nevertheless be viewed as equivalent when assessing risks from their spread among malicious actors.
Access and misuse
Among the benefits, open models usually have lower hardware requirements, according to UNIDIR. For example, the institute says a high-end laptop is sufficient to run a local model for automated social media content generation, basic intelligence analysis and logistical support for a small organization.
Open models also let researchers, governments and private organizations adjust models to their own needs while retaining control over sensitive data, the commentary says. It adds that such models can reduce dependencies and provide equal opportunities for developing countries.
That same access creates opportunities for misuse, the institute argues. It says that built-in safeguards can be removed or bypassed within days of a model’s release, and that models with those safeguards removed can assist malicious actors with activities including planning, logistics, recruitment and intelligence gathering.
The institute says cases have also been documented of terrorist groups using closed models for logistics and the manufacture of arms and explosives, as well as similar cases involving extremists and terrorists acting alone. Providers of closed models can monitor requests and suspend or ban accounts suspected of violent intent; open models do not have the same oversight, according to the commentary.
Responses under study
The forthcoming research will consider how regulatory frameworks might address the risks. UNIDIR identifies the dual-use character of the technology as a particular challenge for regulating training data: the same technology can serve beneficial and harmful purposes.
Potential responses to be explored range from technical measures, including safeguards against removing guardrails and identity verification, to policy measures and multilateral cooperation against proliferation and misuse. The commentary calls for coordinated responses that prevent malicious use while preserving fair access to the benefits of open models.