Why This Dataset Exists
Lots of great work exists already tracking LLM progress in mathematics. Why introduce another set of problems?
Given the recent breakthroughs on some of the hardest problems in mathematics, more compute is being spent on open problems now than ever before. This introduces two failure modes in the existing literature:
- Many existing datasets (like VibeMathed or MathDB) don’t have a fixed list of problems which are causally unrelated to progress in AI. Instead, as more and more mathematicians use these tools, these trackers collect the successful applications comprehensively but have no method of tracking failures. For this, much like in the social sciences, we need a “pre-registered” list of problems to use as comparison, a denominator.
- Given the pace of progress on the frontier, even strong existing benchmarks may be near saturation within a few model releases.
Epoch’s FrontierMath benchmarks are the go-to for tracking progress in mathematics, and deliberately mitigate the issue of selection bias by curating a frozen set of challenging problems. These are tremendously strong benchmarks, especially FrontierMath Erdős and FrontierMath Open Problems, due to the effort and expertise invested in curation and formalization (such that they can be verified automatically).
But these datasets suffer from their own modest selection effects: Erdős problems are sampled from the work of a single mathematician, and the Open Problems benchmark is limited to problems with computationally verifiable answers. Depending on the pace of progress, these subsets could quickly become unrepresentative of the true mathematical frontier, as LLMs continue to conquer the most difficult areas of math.
Hence, the ideal question set would be a large set of problems representing the peak difficulty of open problems: diverse enough to avoid selection effects, unsaturated to allow for continued relevance, and fixed in time to avoid polluting our base rates with selection bias. We think Wikipedia’s list of unsolved problems, while far from flawless, captures the necessary conditions nicely: more than 450 problems, across a range of mathematical subfields, notable enough to have a Wikipedia entry (and optionally filtered to have their own Wikipedia page), and fixed in time to a particular point.
Crucially, this is a strong list of problems, which has only become a viable bellwether with the most recent state-of-the-art models. Given the breadth and strength of the list, saturation (or near-saturation) would be extremely strong evidence of AI possessing superhuman mathematics capacity.
How to Read the Data
We mark problems that have been resolved, are claimed to have been resolved, have been partially resolved, or have seen notable progress. Much of this required judgement calls, by both human and LLM reviewers. See the limitations section for more information.
- Resolved
- A proof or counterexample that the community has broadly accepted, or that is machine-verified and unchallenged.
- Resolution claimed
- A preprint or announcement not yet accepted, or two conflicting claims on the table.
- Partially resolved
- A listed sub-question or a major special case settled, with the general problem still open.
- Notable progress
- A new bound or an instructive incident; the problem itself is still open.
When we reference “Progress” throughout this site, we merge “Partially resolved” and “Notable progress”. Separately, each result is marked by who did the mathematics, based on the authors’ own account:
- AI-generated
- An AI system produced the proof, construction or bound, and humans checked it.
- AI-assisted
- Human-led work in which an AI system produced part of the mathematics in the result (e.g. an argument, a construction, or a case).
- No AI involvement
- No AI system was used, or it only checked, proofread, searched the literature, debugged code or wrote a formal (Lean) proof of the humans’ argument.
These categories may need to become more granular in the future, as these tools are used to solve more and more problems.
The Wikipedia List of Unsolved Problems Snapshot
The list of problems is Wikipedia’s List of unsolved problems in mathematics as it stood on 26 January 2024 — the last edit that month, and earlier than every event recorded here. That revision carries 462 open bullets across twelve field sections, as well as five Millennium Prize Problems named only in the introduction; ten of those 467 bullets are headings the article uses to group the problems beneath them, yielding the problems in this tracker.
This site includes the option to filter the problems based on whether or not they have (or had, at the time of the snapshot) a dedicated Wikipedia page. This is to help mitigate overinclusion on the list. By default, all problems in the snapshot are included.
You can explore the data and results in Explore All Problems.
Collecting the Data
Every status on this site was last reviewed on 28 September 2026.
Every problem was researched on its own by Claude Sonnet 5 (medium effort) with web search enabled, given that one problem and a fixed written protocol. Each agent searched for anything that had happened to its problem since the snapshot, opened the primary sources — arXiv preprints, journal pages, the authors’ own announcements, lab posts, Lean repositories — and returned a structured record: the status, the dated event, author credit, the AI involvement level, whether the result is a counterexample, and supporting links. Agents used about nine sources per problem on average.
Initially, we had Sonnet do a slimmer triage pass (with fewer suggested searches) before doing a deep dive, but many events were missed using this approach. We opted instead to give Sonnet the triage results while still completing a full search for every problem.
The records were then checked automatically for internal contradictions (manually by humans and using Claude Opus 5.5 with low effort). This step is to ensure the data gathered by Sonnet was consistent with the metadata summary, which often requires judgement calls. Every hand correction was kept in the record, with the underlying reasoning.
Two smaller passes, using Claude Haiku 4.5, added dates: the year each problem was first posed, read from the Wikipedia article where it states one, and otherwise researched in the literature; and the release date of each AI system credited.
Next Steps and Limitations
This dataset is a first version, and there are many improvements left to make. As we make changes, we’ll include versioning to ensure full reproducibility. But our next steps are roughly as follows:
- As mentioned above, although Wikipedia tends to be a reliable source, its curation of open problems errs on the side of being overly inclusive. Currently, we control for this with the option to filter problems based on the presence of a dedicated Wikipedia page, but this is still an imperfect proxy of importance. We plan to introduce additional proxies to contextualize these results: total Wikipedia edits, mentions in the scientific literature, subjective LLM or expert ratings of importance.
- The agents involved used nine sources per problem, on average. While this number might be robust for some problems, more ambiguous cases may have sources missing given this limitation. Running the research pass with a more capable model and a larger search budget is a high priority.
- Some of the problems in the dataset have more than one event, and we currently prioritize resolutions over progress, and AI-assisted progress over human progress. As more progress gets made, a more substantial portion of the problems will contain numerous breakthroughs, and our categorization and presentation will have to change.
The repository for this project can be found here.
If you have any questions about the dataset, please reach out to Alex Fogelson (fogelson - at - mit - dot - edu).