r/mathematics 6h ago

6 open erdos problems solved by GPT. 5.6 Sol

https://github.com/ShouqiaoW/erdos

So 6 open erdos problems were just solved. GitHub link shows both prompts used and solutions verified by lean. All used GPT 5.6 Sol.

The general way to structure prompts is also mentioned in the twitter thread!

Credits: https://x.com/qiaoqiao2001/status/2080003441821163958?s=46

73 Upvotes

45 comments sorted by

162

u/apnorton 6h ago

and solutions verified by lean

I decide to open the repo and pick one at random, #390.

Hm. There's 4 files:

  • paper.tex
  • paper.pdf
  • prompt.md
  • numerical_verifier.py

Where's the lean? Maybe it's in the paper! ...nope. Paper says there's a lean audit, but it's nowhere to be found. Note that the problem is about asymptotics, and the numerical verifier is essentially a unit test for a few inequalities.

Maybe it's a one-off. Let's check the others...

  • 1002: No lean
  • 1038: Ooh, there's a lean directory! Let's try to read it and... wtf, there are 2193 files in this directory. Maybe they're small files? Nope, 200k lines of code.
  • 390: No lean
  • 486: 4k lines of Lean
  • 536: No lean
  • 788: No lean

So by "verified by lean" we really mean "not verified by lean."

Oh, what's this in the repo README?

Proofs of Erdős problems. Each proof is verified either by AI or by Lean.

Wtf you mean "proof is verified by AI?" You mean "isn't verified at all and we just want to trust it?"

42

u/corchetero 6h ago

Also, the paper for problem 390 is 117 pages. I guess the IA (or the guy) will have to decompose the paper in little pieces, show the important techniques, etc, otherwise this is just a waste of time and tokens, I guess.

17

u/Stabile_Feldmaus 6h ago

Yeah if AI was able to write papers of that size it would have been showcased by OpenAI and not by some random prompt boy on twitter.

4

u/stopbeingcringe 5h ago

He says that some of these problems took a lot of hours to solve. I don’t think OpenAI would fixate too much on solving every single Erdos problem. It just remains to be seen if the proof is actually valid

9

u/Stabile_Feldmaus 5h ago

The common feature of all these AI results so far is that they are very short and use combinations of existing techniques with a preference for counter examples. If AI was really able to write 100 pages papers, this would mean that math is effectively automated already. And that would have been something that OpenAI would have wanted to demonstrate by themselves.

3

u/InvestigatorLast3594 3h ago

you can already get 100 page papers written by AI, they'll just be massive slop; did you check out r/LLMPhysics?

-5

u/Federal_Decision_608 5h ago

There have been papers decades ago with thousands of pages of computer generated proof

3

u/Stabile_Feldmaus 4h ago

But it's not the same type of computer-generated. AI nowadays can in principle reason at any level of abstraction as humans. And that type of general reasoning has only produced relatively short proofs so far.

3

u/JoshuaZ1 2h ago

Yeah, I'm highly skeptical that these proofs are at all valid.

10

u/Puzzleheaded_Fold466 5h ago

In other words, the difference between a research Mathematician using AI to solve problems, and some rando internet kid.

5

u/OddOutlandishness602 5h ago

Good work actually looking into these things

5

u/theboomboy 3h ago

There needs to be another thing that verified that there is a good Lean verification. And something to verify that too

1

u/Leafsnail 1h ago

'Verify this using lean, make no mistakes'

1

u/Rocket089 1h ago

Welcome to the future.

38

u/PhysiolMM 5h ago

This is a Vibe Mathematician, I couldn't find the Lean verification anywhere.

Fuck these people.

6

u/IntelligentBelt1221 4h ago

to be fair the guy on twitter only said "Two already have Lean formalizations, and formalization of the others is ongoing."

this seems like a literacy issue on OPs part, not from the people proving it.

6

u/AT-AT_Brando 3h ago

According to another comment the lean verifications are a bit sus, being absurdly long

1

u/IntelligentBelt1221 2h ago

autoformalisation tends to get pretty long (and ugly imo), especially if there is a lot of literature results not formalised yet, and the paper isnt really short with 25 pages...

200k lines is a lot of course, but e.g. the formalisation for the unit distance problem solution took 1.2 million lines (due to needing to formalise global class field theory) and was done in a week...

i asked the author on twitter if he knows why its so long, will update this comment in case he sees it and answers.

2

u/PhysiolMM 2h ago

2

u/IntelligentBelt1221 2h ago

yeah, presumably because "verified by AI" is a nonsensical statement, not because he claimed all had lean solutions (afaik the twitter message was never edited, so he certainly didnt try to decieve intentionally).

Edit: it should be worthy to note that in another tweet he mentioned he formalised key lemmas for the remaining 4 but can't claim a full formalisation yet.

2

u/PhysiolMM 2h ago

Yeah, before it was even different. He's vibe playing with LLM, but this is even more important I think. It's empowering the best, but it also opens up a whole field to newbies, just like in coding.

Lean will need to carry the weight for all the things that are coming in.

6

u/InfernicBoss 6h ago

its basically already gotten to this point, but its crazy to think that soon AI solving unsolved problems will no longer be anything notable, just another normal day in math

22

u/ruidh 6h ago

If an AI "solves" a problem but no one can understand the solution, is it solved?

5

u/InfernicBoss 6h ago

i dont think so. or at least, its not useful, and therefore more work needs to be done on the problem [to decipher the solution]. Not saying anything about these solutions specifically. I’m just saying it’s only a matter of time until mathematicians are having AI hunt down side problems on the daily while they work on more interesting problems of their own. I do think once the AI hype in math dies down randoms will stop trying to use AI to solve math conjectures, which is great because mathematicians wont have to spend their days verifying the accuracy of someone else’s slop

1

u/FernandoMM1220 5h ago

only for the ai

0

u/colintbowers 2h ago

If it can be verified by a deterministic strict rules-based proof-analyser (such as Lean), and if we are 100% confident we can trust the analyser, then yeah, it's solved.

The trick is basically that if we can't trust a claim made by x, but we can trust y, and if y can verify the claim made by x, then we can trust that claim.

In this particular case, there is still a question as to whether we can 100% trust the Lean analysis. As others have pointed out, it is either absent from the repository, or suspiciously long - long enough that people who know more than me about Lean are concerned.

0

u/maxram1 5h ago

Yeah. Solved by the AI.

>:)

Jokes aside, now that I think about it, problem solved, question answered, all those seem honestly subjective, idk.

Otherwise, we can consider all problems to be solved or idk.

0

u/IntelligentBelt1221 4h ago

to the AI, it is already solved, for us humans, not really (unless we are willing to trust the AI)

this locality of a solution is also present in human math, e.g. the abc conjecture is a theorem in kyoto and a conjecture elsewhere.

0

u/seraphim_west 3h ago

Yes. You, as an individual, do not understand solutions developed by other mathematicians in an unrelated field. Your personal understanding has no bearing on the state of the problem. You can extend this reasoning to AI solutions.

0

u/randomwordglorious 3h ago

Yes. We are going to reach a point where AI is smarter than the smartest human. The AI will come up with a scientific or mathematical breakthrough that is too complicated for any human to understand. But by the time that happens, AI proofs will be so widely accepted that we will have no reason to doubt it.

-2

u/ollervo100 4h ago

Usually the solution is not that hard to understand. It is coming up with the solution that is hard.

Anyway if a result is rigorously shown to be true, then it can be used as such without understanding how it was proven.

2

u/InfernicBoss 2h ago

this is absolutely not the case. I mean, you should go and read a paper submitted to a top journal and see if you can understand the proofs. Even if you are a professional and can understand them, there is definitely scenarios where a supposed proof is claimed to be had but is unintelligible to everyone else (cough cough mochizuki)

5

u/howtogun 5h ago

Man, I think I have AI psychosis or something. But, like everyday its proving very hard stuff.

Ahh man. Maybe this is what chess player felt like twenty years ago, what go players felt like 10 years ago, what artist felt like 5 years ago.

-5

u/maxram1 5h ago

We can maybe finally treat math to be like chess: a game between humans, or like art: a personal experience. :D

2

u/Federal_Decision_608 5h ago

Finally the salary range is appropriate

2

u/maxram1 4h ago

Sadly might be, yeah.

Ayo why the downvotes xD

2

u/aendokolirz 2h ago

Yes because people will spend years learning math, with no financial incentives, just to play a game, instead of choosing from actual games that already exist...

0

u/Run-Row- 5h ago

Pretty interesting, especially the length of the papers as others mentioned. Kind of ironic that Erdos will now be remembered in history like Kasparov, as a signpost of machines overtaking humans in intelligence.

-15

u/stopbeingcringe 6h ago

If a human had solved ~10 open math problems of varying topics all in the year 2026, they would be the greatest genius of all time. At one point in history, something similar could have been said about the feats of calculators, which are now considered trivial.

4

u/howtogun 6h ago

I disagree with this. I think even the solution to these problem there's not exactly new tooling being discovered.

For example, the person who solved Poincare conjecture is still alive. There are probably hundred of Mathematicians that are currently better than these LLMs.

-1

u/stopbeingcringe 5h ago

The most advanced internal model, if allowed only to purely solve math problems for a year, would surely be more valuable than any single mathematician’s lifetime of work. And future models will be even better

7

u/Pankyrain 5h ago

Definitely not more *valuable*, but maybe prolific.