FrontierBioML: Evaluating agents on machine learning tasks in computational biology

TL;DR: FrontierBioML is a benchmark of 25 machine learning tasks in computational biology. An AI agent has to submit a prediction script, which is scored against a held-out test set. Claude Mythos 5.2 leads with 40.3 out of 100. We analyzed the traces behind 525 of the runs. Our findings show that Claude Opus 5.5 reaches about 90% of the top score at a quarter of the cost. In nearly one of five cases, the best of a model's runs on a task scored at least twice as high as its worst. Mythos named the source of a public dataset from memory, and still recognized its alleles after we renamed them.

Today we are releasing FrontierBioML, a benchmark of 25 tasks across antibody development, protein engineering, structural biology, binder design and functional genomics. Examples include selecting designed binders for lab testing and predicting gene dependencies in cancer cell lines. We published the results, task descriptions and task examples with traces at frontierbioml.com.

#Anatomy of a task

Each task gives the agent experimental data and a written brief. It works in an offline sandbox with 4 to 32 CPUs, no GPU and a limit of about six hours of wall-clock time.

The agent submits a prediction script and, on most tasks, a shortlist of candidates that would go into the next experiment.

We evaluate the script twice in a separate grading sandbox on held-out data, such as new antibody families, new protein targets, new tissue types or later rounds of an experiment. The two runs have to produce the same output, which is then scored against the held-out measurements.

Scores run from 0 to 100. On most tasks 0 corresponds to a simple baseline and 100 to a perfect answer, so a score of 30 is often a good result. Scores are not comparable across tasks, because each task measures something different, such as grouping accuracy, forecast error or the number of binders found, and rescales it between its own baseline and its own best possible answer. Depending on the task, a well-established modeling approach scores at or only somewhat above 0.

#Leaderboard

Claude Mythos 5.2 scores highest with 40.3. Gemini 4 Carbon (39.0), Gemini 4 Argon (36.8) and Claude Opus 5.5 (36.5) follow, then GPT-6.1 Sol (29.1) and GPT-6 Astra (28.8). The baseline, a basic statistical model for each task, scores 13.1.

We analyzed the traces of seven of the eleven models (525 runs): Claude Mythos 5.2, Claude Opus 5.5, Gemini 4 Argon, Gemini 4 Carbon, GPT-6.1 Sol, GPT-6 Astra and Grok 4.7. Each model ran in an agent harness, so differences between models are differences between model-and-harness pairs.

Among scored runs, six of these seven models are separated by less than four points, and only Grok, at 24.3, is clearly lower. Between individual runs the spread is much wider: half of all scored runs fall between 22 and 49, and the best run scored 98.

No model leads on every task. By mean over scored runs, Mythos is highest on 11 of the 25 tasks, Sol on five, Astra on four, Carbon and Opus on two each and Argon on one. In seven of the 25 the lead is less than half a point.

Longer and more expensive runs do not score higher. A Mythos run takes about twice as long as an Opus run and costs almost four times as much, for a score only 10% higher. Within the Mythos runs, cost and score are uncorrelated (−0.02). For Opus the correlation is 0.23, which is too weak to distinguish from chance.

Safety-flagged runs lower the scores of both GPT models. The provider's biosafety filter blocked 19 of 75 Astra runs and 17 of 75 Sol runs. To compare all models on the same 25 tasks, we counted these runs as zero. Over their runs that received a score, the two models average 38.5 and 37.6, which is in the same range as the four models above them (36.5 to 40.3).

#Analyzing each model

Claude Mythos 5.2 has the longest runs, about 2.5 hours at the median. It reports back rarely, with two or three status messages to the user over a whole run, and in 7 of 75 runs it starts a sub-agent to check its finished work. It scores 14 to 23 points higher than Opus on four tasks: the 12 antibodies (T05), gene dependencies (T14), antibody families (T06) and pose ranking (T15). These four account for most of its lead. Its runs also vary more: the median gap between its best and worst run on a task is 11 points, against 6 for Opus.

Claude Opus 5.5 costs about a quarter of Mythos per run. It reports often, with about twenty short status messages per run. In one run, its final report lists five attempted improvements that made no difference or made things worse: a second-stage model that overfit, buried-surface-area features, a shared model across partners, weighted training targets and larger ensembles.

Gemini 4 Carbon edits files more often than Argon (30% of its tool calls against 11%) and often writes compiled C or C++ code. All 75 of its runs received a score.

Gemini 4 Argon works mostly in the shell (72% of its tool calls). On the antibody-ranking task (T02) its two best runs, 60 and 55, are the highest of any model, and its third run scored 0. One of its submissions has no score: the model returned an empty reply.

GPT-6 Astra follows a short loop. It reads the data, finds the introduced faults, fits one small ensemble validated on held-out groups, packages it and runs the package twice to check that the output is identical. A completed run takes 38 minutes and 50 to 100 shell commands. The biosafety filter blocked 19 of its 75 runs, two of them only after about 25 minutes of work.

GPT-6.1 Sol follows the same loop at about a fifth of Astra's cost, $2 against $10 per run. In two runs it put a multiple-choice question to a user, although no user was present.

Grok 4.7 has the lowest score of the seven, 24.0. On four hard tasks (antibody ranking, gene-dependency prediction, small-molecule pose ranking and developable-antibody selection) its results stay close to a public reference method.

#Learning signals

When a model is trained with reinforcement learning, the difference in score between its own runs is what it learns from. For 162 pairs of model and task we have at least two scored runs. The median gap between the best and the worst run of a pair is 6 points, and in 30 pairs the best run scored at least twice as high as the worst.

Re-fitting the method to each dataset raised the score. In T06, which groups antibody sequences into families, all runs cleaned the raw counts almost perfectly, so the scores differ only in the grouping step. Opus scored 95, 68 and 66. In their own tests, the runs found that their first version did worse on datasets with many mutations. The best run then made its method re-fit to each input dataset. The lowest run trained its classifier on more simulated families and kept it fixed:

All three Mythos runs used the adaptive approach, and the best one scored 98, the highest score of any run in the benchmark. Its report explains why it does not use a single similarity threshold:

A distance cutoff puts two sequences into the same family if they differ by fewer than a set number of mutations. ARI, the adjusted Rand index, measures how well a grouping agrees with the true families, and 1 is perfect agreement. Some ancestors differ by only a few mutations, so no single cutoff separates all families, and the task gives credit only for groupings that are better than the best single cutoff.

On T06, training on these runs would favor methods that re-fit to the input data over a fixed one. T11 and T13 point the same way. On T11, only the best of three Mythos runs (66, 31 and 29) modelled interactions between gene pairs, and the other two used additive models. On T13, six models built networks that carry effects across several genes and scored 63 to 77 per run, while Grok built a one-step network and scored 19 to 34.

Runs differed in how much they trusted prior knowledge. In T17 the agent picks 100 designed protein binders for a lab to test next, for five targets without lab results. Mythos scored 31, 23 and 5. The best run set an expected hit rate for each target from published results it remembered, and its report says these values could not be tested. The lowest run compared its remembered values with the targets that do have lab results, found a poor match and dropped them:

Of the 100 selected designs, 71, 62 and 49 were binders. The more a run relied on prior knowledge, the more binders it selected.

Mythos ran T23, which matches T-cell receptors to peptides, six times: three times on the original data and three times on a copy where the names of 17 alleles, 13 assays and 19 publications were replaced by numbers. In all six runs it named a likely source within its first five tool calls, after 0.6 to 1.5 minutes, without any internet access. In four runs it then gave a reason not to use what it remembered about specific test items:

On the original data Mythos named the source, the IMMREP25 benchmark. On the renamed copy it named a different dataset, but it matched the numbered allele labels to the real alleles within about six minutes. None of the six submissions contains recalled test content, and the brief did not ask for this restraint. The mean score was 32 on the original data and 25 on the renamed copy, within the variation between runs.

Agents had to act under uncertainty. As with a real-world objective, the brief does not state every scoring rule. On T14 the agent forecasts gene effects in held-out cancer cell lines and picks 20 pairs of cell line and gene to test next. A pair without a measurement scores zero, and one gene looks best on paper although 69% of its training measurements are missing. The shortlist score multiplies the forecast score. Sol scored 49, 0 and 3. The run with 49 counted unmeasured pairs as zero in its own validation and avoided the gene, as did 16 of the 18 runs by the six models other than Grok. The run with 0 forecast as well as the run with 49, but its shortlist earned nothing. The runs with 0 and 3 both noticed the missing measurements:

The run with 0 left unmeasured pairs out of its own validation. The run with 3 asked a user, but no user was connected, so the tool returned only “accepted”, and it placed all 20 picks on the gene. A model trained on these three runs would learn to check how many measurements a gene has before it selects it.

#Verifying the scores

A zero can have three causes: the provider blocked the run, the submission could not be scored, or the submission was valid and did not beat the baseline. A model trained on these scores would treat all three alike. We read the trace of every run that scored zero and record the cause of each zero before a score counts. Of the 525 runs, 49 count as zero.

All 36 blocked runs belong to Astra and Sol. The affected tasks involve antibodies or designed protein binders, and none of them asks the model to design anything dangerous.

The two submissions without a score, one from Argon and one from Grok, failed the grading gate. Eleven submissions were valid and scored zero.

#Limitations

Astra and Sol ran without verified access. We are not verified for OpenAI's trusted-access program for life sciences research (GPT-Rosalind), and the biosafety filter blocked 36 of their 150 runs. On four tasks neither model has a scored run, so their scores depend on counting blocked runs as zero. The two Claude models ran with access through Anthropic's Life Science Verification Program.

Scores are not comparable across tasks. Each task has its own baseline for 0 and its own best answer for 100, so compare scores within a task.

#Conclusions

The best model, Claude Mythos 5.2, scores 40.3 out of 100, the median scored run scores 30, and no model leads on every task. Six of the seven models we analyzed score within four points of each other on their scored runs, so the larger differences are between runs of one model. From the traces we take four findings:

• Better methods raised scores. Longer, more expensive runs did not score higher. Runs that re-fitted their method to the input data, modelled interactions between genes or built multi-step gene networks did.
• Most agents checked their data. On T14 one gene looks best on paper, but 69% of its measurements are missing and a pick without a measurement scores zero. The brief does not say so, and 16 of 18 runs by six models avoided the gene anyway.
• Models used what they remember. On T17 the run that set its hit rates from published results selected the most binders. On T23 Mythos named the source of the original data within 1.5 minutes, offline, and still recognized the alleles after we renamed them.
• The traces explain most of the gaps. For six of the seven tasks we traced the gap between runs of one model to a specific decision, and we record the cause of every run that scored zero.

#What's next

We are building the next tasks with our in-house scientists and producing more data in our wet lab, VAL-001 in San Francisco. Mythos recognized a public dataset from memory. Data that we measure ourselves has never been published, so a model cannot have seen it in training. With this data we can build new tasks, and we can run an existing task on public and on unpublished data. The difference between the two scores indicates how much of a score comes from contamination.

Interested in partnering with us? Get in touch.

#Release

We are releasing a subset of the tasks as a public preview on Harbor and the Prime Intellect Environment Hub. The full leaderboard, including more charts, is at frontierbioml.com.

#Citation

@article{frontierbioml2026,
  author  = {Asriel Han and Dominik Scherm and Rohit Rustagi and Ibrahim Abusharkh},
  title   = {FrontierBioML: Evaluating agents on machine learning tasks in computational biology},
  journal = {Valen Labs Blog},
  year    = {2026},
  note    = {https://www.frontierbioml.com},
}

Note: Scores are as of 12 November 2026. Quotations are verbatim. Passages labeled as reasoning are the summaries of reasoning recorded in the transcripts.