About 30% of newborns with a suspected rare disease receive a diagnosis. We think that number should be much higher.
At Gamow Labs, we’re building AI systems to make state-of-the-art neonatal genomic care more accessible and to push the field toward better diagnostic rates.
When you’re working with AI, progress depends on having a strong metric to optimize against. There needs to be a standardized way to measure whether your model or harness is actually getting better.
That’s why we are building RareBench. As it exists today, it’s a 122-case benchmark for rare genetic disease diagnosis. We see this as an early step toward giving the field a better way to measure how well AI can solve one of medicine’s hardest diagnostic problems.
Why we are sharing RareBench
By regularly publishing RareBench results, we’re hoping to encourage the frontier AI labs to make progress on this problem too. More hands on it means more babies get diagnoses.
Right now we are using RareBench at Gamow Labs to understand the capabilities of different agentic systems at genomic variant interpretation generally, and to optimize the performance of George-0.1, our own rare disease diagnosis agent, specifically.
Rare disease variants have many of the properties we look for in a task that can benefit from reinforcement learning and increasingly capable agents.
- It’s verifiable: there is a causal variant or ground truth diagnosis, so the agent’s response either matches the answer key or it doesn’t.
- It’s agentic: diagnosing a rare genetic disease requires searching databases, inspecting variant annotations, reasoning about inheritance, checking population frequencies, comparing phenotypes, examining genomic regions, validating hypotheses, and revisiting earlier assumptions. The agent may need to make dozens of tool calls before arriving at an answer.
- It’s unsaturated: genomic interpretation remains difficult even for highly capable systems. There is substantial room for better models, better harnesses, and better tool use.
- It’s meaningful: a better answer here is not just a higher benchmark score. It can mean a child and their family finally receiving a diagnosis.
What RareBench measures
Given a patient’s phenotype and genome, RareBench attempts to identify the causal variant or variants responsible for the disease.
We measure whether the correct causal variant appears among the system’s top hypotheses. Additionally, we separately track cases where the system identifies the correct variant but does not rank it highly enough to make the top-k cutoff.
This distinction is useful because it separates two different failure modes. An agent might completely miss the relevant biology, or it might find the right variant somewhere in a large search space but fail to prioritize it correctly. For an agent designed to assist with diagnosis, both are important.
This is how we built our scoring framework for the eval: a top-k recall, a detection that fell short of top-k, or a miss.
The latest results
We ran the full 122-case benchmark against frontier models and established a leaderboard. The initial results are encouraging, although there is clearly a long way to go.
Here are some things that stood out to us.
Grok 4.6 takes the lead
The best performing model from this run is Grok 4.6.
Grok 4.6 solves 35% of RareBench cases, putting it narrowly ahead of Claude Opus 5 at 34%. What makes the result particularly interesting is the cost. Our production runs put Grok 4.6 at approximately $1.98 per case, compared with $9.35 per case for Opus 5.
So on this benchmark, Grok is achieving slightly higher diagnostic performance at roughly one-fifth the cost.
This is one of the reasons we think task-specific evaluation is so important. General benchmarks give us useful information about frontier models, but they don’t necessarily tell us how a model will behave when asked to investigate a rare genetic disease over a long sequence of tool calls.
DeepSeek V4 Flash gives us a good margin
Another noteworthy discovery from this run was seeing the performance-to-cost ratio of DeepSeek V4-Flash. It scored about 20% for just $8.55 to run the entire benchmark. This, compared to Anthropic and OpenAI models on the order of $1,000 for 34% accuracy is interesting.
For such a gap in cost for a meaningful diagnosis rate, it begs the question of what we should be indexing for. If the goal is making this technology available to every NICU rather than only the ones that can afford the most expensive option, that cost curve matters as much as leaderboard position.
GLM-5.2 and Kimi K3
GLM-5.2 currently lands at 14%. That is a meaningful result on a difficult benchmark, but it trails Kimi K3 at 28% despite the two models attracting considerable attention for similar coding and agentic use cases.
GLM had roughly 4% additional cases where it detected the correct variant but failed to place it in the top-k. Kimi, by contrast, had only about 2% in this category while achieving twice the top-k recall.
This is another example of why we think task-specific evaluation matters. General impressions of a model’s capabilities, and even performance on adjacent benchmarks, do not necessarily predict how well it will perform when asked to investigate a rare genetic disease.
The potential impact of this work
The leaderboard is becoming a useful measurement of something we care deeply about: how effectively an AI system can actually reason through a patient’s genome and find the variant responsible for their disease.
The frontier is moving quickly. We intend to keep measuring it.
This eval is the first step in a long journey toward George, our agentic system for diagnosing rare genetic disease. A benchmark like RareBench doesn’t diagnose anyone on its own. What it does is give us, and the rest of the field, a shared, quantitative target: a way to know whether a change to a harness, prompt, or tool integration is actually making the system better.
We believe AI will revolutionize genomics, and we hope to play a small part in distributing better healthcare to every NICU baby who needs a diagnosis. We’ll keep publishing RareBench results as we iterate, including where we fall short.
If working on agentic systems, largely harness and tool engineering, for a deeply mission-driven company at the intersection of AI and biology resonates with you, we’re hiring.