Skip to content
AI.info

The Pulse

OpenAI's MedChemBench tests AI reasoning on drug chemistry

OpenAI has introduced a benchmark with 99 medicinal-chemistry problems drawn from published research. Its tests found stronger results in reconstructing molecules than in selecting compounds or predicting their properties.

OpenAI's MedChemBench tests AI reasoning on drug chemistry

OpenAI's new MedChemBench tests AI models on 99 medicinal-chemistry problems that ask them to interpret molecular structures and experimental evidence. The company reported that GPT-6.2 Astra earned 84.5% of available credit at the xhigh reasoning setting, while GPT-6 Astra earned 83.3%.

What the benchmark tests

Introduced on November 20, MedChemBench draws on 117 papers and patents and includes 761 distinct molecular structures. For most problems, a model receives the information available at a particular point in a research program, including the biological target, the researchers' goals, compounds already studied and selected experimental findings.

The benchmark grades molecular reconstruction, scientific explanations, compound ranking and selection, and predictions of properties. Structure scoring checks atom connections and specified stereochemistry; explanations are assessed against the evidence supplied. Property predictions are checked against numerical values and categories in the source material, including measured and calculated properties.

The tasks address tradeoffs that medicinal chemists face when developing compounds. A molecule that acts strongly against a disease-related protein, for example, might be poorly absorbed or break down too quickly in the body.

Results across models

OpenAI evaluated eleven GPT models at different reasoning-effort settings. At the shared xhigh setting, GPT-6.1 Sol earned 82.0% of available credit across the benchmark, GPT-6.1 Terra earned 75.1%, and GPT-5.6 Sol earned 58.2%. At its max effort setting, GPT-6 Astra reached 83.8%.

For GPT-6 Astra, the strongest areas were reconstructing structures and explaining scientific evidence, according to OpenAI. At xhigh, it earned 95.4% of the available structure credit but only about half the available credit for selecting compounds and predicting numerical properties. The categories use different tasks and scoring rules.

Two problems in detail

One problem concerns inhibitors of PRMT5, a protein studied in cancer drug discovery. Given structures and results from earlier experiments, the model must predict how three related compounds behave: how strongly they block PRMT5 in cells, how readily they cross a cell barrier and how quickly liver enzymes break them down in a laboratory assay.

In results withheld from the model, compound 23 blocked PRMT5 in cells at a lower concentration. Compound 24 crossed the cell barrier more readily and broke down more slowly in the liver-enzyme assay. Predictions receive partial credit based on their closeness to the measured values, while measurements already included in the prompt must also be reproduced correctly.

Another problem asks models to assemble a molecule from a central structure and two attached groups while preserving an 18-atom ring. OpenAI reported that GPT-6 Astra reconstructed it correctly in all ten responses at xhigh. GPT-5.6 Sol and GPT-5.6 Luna failed across all ten responses; illustrated answers showed a smaller ring from the former and disconnected fragments from the latter.

Limits of the results

The company cautioned that the results do not establish whether models can discover new drugs or improve laboratory decisions. Prior exposure to the published sources cannot be ruled out, and each response was a single attempt without tools, feedback or new experiments. The benchmark evaluates reasoning from historical evidence rather than real-world drug discovery.

Sources

  1. Original source OpenAI openai.com

Companies in this story

In the tracker

Related stories

Explore

More articles