# OpenAI MedChemBench tests AI on drug chemistry

> OpenAI introduces MedChemBench, 99 medicinal chemistry problems. Its best model scores 84.5% of the credit, but OpenAI says this does not show AI can discover new drugs.

Published 2026-11-20, updated 2026-11-20. Confidence: medium. When it could matter: Independent replication and any prospective testing: timing unknown.

## In plain words
OpenAI has published a new test, called MedChemBench, that checks how well AI models handle problems from the chemistry of making medicines. Its newest models score much higher than an older one, but they do much worse at choosing which compounds to test and at predicting measured values. OpenAI itself says the test does not show that AI can discover new drugs. Readers should treat the scores as a progress report on reasoning, not as evidence of a ready tool for medicine development.

## What happened
On 20 November 2026 OpenAI introduced MedChemBench, a benchmark of reasoning tasks from medicinal chemistry. OpenAI says it tests whether models can connect a molecule's structure with experimental evidence to explain results, compare compounds and predict their properties.

The benchmark has 99 problems drawn from published research. They come from 117 papers and patents and include 761 distinct molecular structures. For most problems the model gets the context available at a chosen point in a research program: the biological target, the researchers' goals, the compounds studied so far and selected experimental findings.

Scoring covers four abilities: reconstructing molecular structures, scientific reasoning, ranking and selection, and property predictions. Predictions get partial credit based on closeness to measured values, and measurements already given in the prompt must be reproduced correctly.

OpenAI evaluated eleven GPT models at different reasoning-effort settings. At the xhigh setting, it reports that GPT-6.2 Astra earned 84.5% of the available credit, against 83.3% for GPT-6 Astra, 82.0% for GPT-6.1 Sol and 58.2% for GPT-5.6 Sol. GPT-6.1 Terra scored 75.1%. At max effort GPT-6 Astra reached 83.8%.

One example problem uses inhibitors of PRMT5, a protein studied in cancer drug discovery. The model had to predict how three related compounds behave in three respects. Withheld later experiments showed a tradeoff: compound 23 blocked PRMT5 in cells at a lower concentration, while compound 24 crossed the cell barrier more readily and broke down more slowly in a liver-enzyme assay.

In a structure problem requiring an 18-atom ring, OpenAI says GPT-6 Astra was correct in all ten responses at xhigh, while GPT-5.6 Sol and GPT-5.6 Luna failed in all ten. Sol produced a 15-atom ring and Luna split the molecule into two disconnected fragments. On structure components Astra earned 95.4% of the credit, but only about half the credit for selecting compounds and predicting numerical properties.

## What is confirmed and what is not
Confirmed by OpenAI's own publication: the benchmark's size and sources, the scores it reports for its models, and the limits it states. These are OpenAI's measurements of its own models on its own test.

Only claimed: that the test reflects a central challenge in medicinal chemistry, and that it will support progress. OpenAI also refers to selected reasoning examples in a preprint, which the announcement summarises but does not reproduce.

Not established, by OpenAI's own account: whether models can discover new drugs or improve laboratory decisions. Prior exposure to the published sources cannot be ruled out. Each response was one attempt without tools, feedback or new experiments.

Unknown from the announcement: how models from other developers perform, whether independent groups have run the test, how the 99 problems were chosen from the 117 sources, and how the credit weights differ between categories. The announcement also lists "GPT-6.2 Astra" at 84.5% and then uses 83.3% for "Astra" in later passages. That points to two model versions, and the text does not explain which result applies where.

## Analysis
### Reading a benchmark score as proof of drug-discovery skill
The most direct consequence is a risk of misreading. A headline figure of 84.5% invites the belief that an AI model is mostly able to do a chemist's job. The announcement says otherwise. OpenAI states: "These results do not establish whether models can discover new drugs or improve laboratory decisions."[1] The tasks reason from historical evidence, so the answers already exist in the literature.

The weak link is the step from a score to a laboratory result. A sceptical expert would ask three things. Did the models see these papers and patents in training? OpenAI concedes that "Prior exposure to the published sources cannot be ruled out, and each response was a single attempt without tools, feedback, or new experiments."[1] Does credit on a mix of tasks hide uneven ability? The announcement itself shows it does, because structure reconstruction is near 95.4% while selection and numerical prediction are about half. And does predicting a published value equal choosing the right compound in a live program? The sources do not show that link.

### Technology and industry
For companies building AI for drug research, a shared test is useful because it separates skills. The mechanism is simple: a model that mis-reads a ring or splits a molecule into fragments analyses the wrong compound, so every later step is unreliable. OpenAI's ring example shows that older models in its set fail this foundation while newer ones pass. If that pattern holds on other structures, structure handling may be a solved enough step to build on, and effort would shift to selection and prediction, where the announcement reports the largest gap.

There is also a commercial reading to hold at arm's length. The benchmark was built, run and reported by the maker of the models it ranks, and all eleven models tested are GPT models. Toswo Research reads the comparison as a progression within one family, not as a ranking of the field. Results from other developers' models or an outside group would change that reading.

### Public health and medicine
Nothing in the announcement points to a patient-facing change. The task is early-stage reasoning about which molecules to make and test next. If models later improve at selection and prediction, the plausible benefit is fewer wasted experiments, but the announcement gives no evidence of that and no timeline. Readers should keep this separate from other medical AI claims, such as the work covered in our note on an [AI selfie tool that predicts cancer survival](/signals/ai-selfie-tool-predicts-cancer-survival-study-reports), which concerns a different kind of evidence, patient outcomes, and has its own limits.

The PRMT5 example also shows why the field is hard. Stronger activity in cells did not come with better permeability or slower breakdown. A model that favours the first property alone would pick the wrong compound in that case. A benchmark that rewards closeness to measured values on several properties at once checks for that trap, though it does so on one historical example.

### Law and policy
If regulators or funders were ever to rely on such scores, the gap between benchmark and real discovery would matter. The announcement offers no regulatory claim, and none should be inferred. The practical point for oversight is that a developer's own results are an input, not a verdict. How such measurements are made and weighed is the kind of question covered in [how TOSWO measures AI risk](/method).

### Science and reproducibility
Reproducibility is the open question. The problems come from published papers and patents, so outside chemists could in principle check the answers. Whether the problem set, scoring rules and model outputs are available for rerunning is not stated in the announcement. Without them, only OpenAI can confirm the numbers. The strongest reason this view could be wrong is that the preprint may contain full materials and independent replication may already be under way; the announcement points to the preprint but does not describe its contents beyond selected examples. Evidence of released problems and third-party reruns would raise confidence in the scores, though not in the leap to real discovery.

A further point concerns the model line itself. The same Astra family appears in other research, including a preprint on [GPT-6 Astra editing other models' internals](/signals/gpt-6-astra-edits-other-ai-models-internals-mit-preprint). That work concerns a separate question and does not bear on these chemistry scores.

### What would settle it
The preprint is the first document to read, including the full problem set, scoring weights and the selected reasoning examples. Next would be results from independent labs or other developers running the same 99 problems. The decisive test lies outside the benchmark: whether models' compound choices, made before the experiments, beat chemists' choices in a prospective study with real measurements. OpenAI's promised updates would show whether the selection and prediction gap narrows.

## What can be done
Readers can treat the figures as a statement about reasoning on past data and wait for outside replication before drawing conclusions about medicines. Researchers who use AI tools in chemistry can check structure outputs by hand, since the ring examples show that identity errors occur, and can test selection and prediction separately from explanation. Organisations building or buying such tools can ask developers for per-category scores, not only the total. OpenAI could publish the problem set, scoring rules and failure cases so others can reproduce the results, and could report models from other developers. No safeguard beyond verification is known to be needed now.

## Questions readers ask
### Does MedChemBench show AI can discover new drugs?
No. OpenAI says the results do not establish whether models can discover new drugs or improve laboratory decisions. The test reasons from historical evidence in published sources, not from new experiments.

### How well did the models score on MedChemBench?
OpenAI reports that GPT-6.2 Astra earned 84.5% of the available credit at the xhigh setting, GPT-6 Astra 83.3%, GPT-6.1 Sol 82.0% and GPT-5.6 Sol 58.2%. Astra earned 95.4% on structure tasks but only about half the credit for selecting compounds and predicting numerical properties.

### What is in the MedChemBench test?
It has 99 problems drawn from published research, from 117 papers and patents, with 761 distinct molecular structures. The tasks cover reconstructing structures, explaining evidence, ranking compounds and predicting properties.

### Has anyone else checked the results?
The announcement reports only OpenAI's own evaluation of eleven GPT models. It describes no independent replication and no results for other developers' models.

## Who is affected

- Drug discovery teams considering AI assistants (relative exposure 8 of 10)
- AI developers building science and medicine models (relative exposure 7 of 10)
- Medicinal chemists who would check AI output (relative exposure 6 of 10)
- Independent researchers who could replicate the benchmark (relative exposure 5 of 10)
- Readers following AI health claims (relative exposure 4 of 10)

## Risks

- Benchmark scores are read as proof of real drug-discovery ability, which OpenAI says they do not establish
- Prior exposure to the published sources may inflate results, and OpenAI says it cannot be ruled out
- The maker designed, ran and reported the test on its own models, with no independent check described
- Weak compound selection and numerical prediction could mislead users who trust the overall score

## What can be done

- Ask for per-category scores, not one total
- Verify any AI-reconstructed molecular structure by hand
- Wait for independent replication and prospective laboratory tests
- Publish the problem set, scoring rules and failure cases

## Sources

1. Can AI help scientists identify promising drug candidates?, OpenAI, 2026-11-20: https://openai.com/index/introducing-medchembench

---
Source: https://www.toswo.com/signals/openai-medchembench-tests-ai-on-drug-chemistry
Prepared by Toswo Research (https://www.toswo.com/research) from the sources above. Free to cite with attribution: https://www.toswo.com/cite
