VALUE: After Hours (S07 E14): Gödel’s Theorem, AI Limits, and Machine Logic

During their recent episode, Taylor, Carlisle, and Bill Brewster discussed Gödel’s Theorem, AI Limits, and Machine Logic. Here’s an excerpt from the episode:

Jake: All right. So, for today’s veggies, we’re traveling back in time to 1931 to meet a quiet genius who reshaped how we understand certainty itself. His name, Kurt Gödel. He was not exactly the most flashy figure. In fact, he was pretty reserved and very introspective. Austrian mathematician born in 1906 in what’s now the Czech Republic. As a child he, asked a lot of questions. In fact, his nickname was Herr Warum, which is Mr. Why, because he was always asking why things were the way they were.

Fast forward to his early 20s, and Gödel’s at the University of Vienna. He’s immersed in this philosophical group called the Vienna Circle. These were thinkers that were obsessed with logic and language. This dream of reducing the universe into this very clean, logical system of rules is very utopian. The idea was that one day, everything could be fully explained if you had the right system.

So, Gödel actually didn’t agree. And at the age of 25, he published what would become one of the most important theorems in the history of math, and it’s called the incompleteness theorem. He showed that any formal system that’s powerful enough to express basic arithmetic, let’s say like one plus one equals two, there will always be true statements that can’t be proven within that system.

So, let me see if I can give a little bit of a– Well, and let me cover too that he has a second– There’s a Gödel second incompleteness theorem. It’s basically that you need a bigger frame of reference to confirm the rules of the current one. So, this is all like gobbledygook right now, but let me just see if I can make this less abstract.

Think of this classic paradox. This statement is false, okay? Now, if it’s true, it’s then false. But if it’s false, it’s then true. So, you end up in this self-referential loop that a system can’t resolve. Gödel basically used a very similar move, except instead of using it in words, he encoded in logic as numbers. He assigned these logical statements that this is actually a technique now called Gödel numbering, which allowed him to build math that could bizarrely talk about itself.

So, let me give another concrete example that might help with this bring this home. Imagine that you’re creating the ultimate encyclopedia and your goal is to include every single fact, every true fact in the universe. No exceptions. Okay. But now consider this sentence that you might write in the encyclopedia, and that is quote, “This sentence does not appear in this encyclopedia.” All right.

If you include that sentence, then your encyclopedia now contains a falsehood because the sentence explicitly says it does not appear, so your encyclopedia fails. If you exclude the sentence completely, then you’ve missed a truth because the sentence correctly describes reality, it indeed doesn’t appear. So, your encyclopedia is now incomplete. So, you either have not true or incomplete that you can choose from.

So, you end up in these weird reductionist absurdities and contradictions. But either way, perfection is mathematically out of reach for these systems. Now, why bring this up now nearly 100 years later? Well, I think it’s interesting to think about AI and GPT, specifically LLMs. And in someways, it’s actually quite instructive for what are really modern Gödel machines.

So, LLMs work by taking language, converting it into numbers, processing those numbers through a bunch of mathematical computations and probabilities and then spitting out more numbers which are finally decoded back into words. So, in a sense, it’s like words into numbers, numbers into math, math into numbers, numbers back into words. It’s actually quite eerily close to what Gödel did. He’s turning language into numbers, running logical operations and then translating back to something that’s human readable.

So, just like Gödel’s formal system could create statements about its own limits, LLMs can also generate statements about themselves. They can tell you things like, “As an AI, I don’t possess consciousness, or I was trained on a data set up until 2023.” To us, this feels like self-awareness, but it’s really not. The model doesn’t know anything. It’s not reflecting. It’s simply predicting. So, when it says “I am not conscious,” it’s not verifying that against some internal truth. It’s just offering the most statistically likely sentence in response to the prompt that you gave it. So, LLMs, they can simulate this self-reference, but they don’t have access to any actual self-knowledge.

So, this is why sometimes you see the LLMs hallucinate. They produce these confident sounding, but completely false information. It’s not that they don’t know that they’re hallucinating, because they don’t know they’re doing anything. They’re just following these patterns.

So, it’s interesting to think as a thought experiment as AI becomes more powerful and it’s more able to reflect reason and simulate logic, the more it enters Gödel’s world, because the more expressive it becomes, the more it resembles a formal system, and Gödel’s theorem applies even more strongly. So even if we had a hypothetical super intelligent AI, like a God mode type of AI, it was still going to run into these limits.

So, Gödel tells us that it can’t know everything and stay consistent. So, just to close this up, it means that a perfect AI is mathematically impossible. Gödel’s theorem isn’t just a piece of math, it’s a philosophical mirror held up to every system that tries to explain the world. And now, we’re building AI systems which are living proof of this.

So, I would say Gödel didn’t really close doors, he opened them. As humans building machines in our image, it’s important to remember that perhaps these limits aren’t walls, but they’re invitations to explore beyond there. So, a little bit of historical trivia of Gödel and this incompleteness theorem and how that relates to AI and where that’s going.

Tobias: What’s a perfect AI?

Jake: Well, assuming that it had all of the truths of the world gathered together and could provide them, reference them, sort through them, create logic.

Tobias: This is the problem that we have, isn’t it, that we’ve fed everything that humans have written down since the Epic of Gilgamesh through to whatever anybody was writing on to–

Jake: The Acquirer’s Multiple?

Tobias: Yeah, that’s right. That’s eight years old now. There’s some stuff that’s been written since then.

Jake: Fair enough.

Tobias: It’s still not able to reason at a– It hasn’t reached that– I forget what the test is, but it’s not yet reached that level yet. So, it clearly has access to a wider information than we do, like why does it not, why is it struggling?

Jake: I don’t know. It gets like graduate level answers right at this point.

Tobias: I still want to go mid to– [crosstalk]

Jake: It’s smarter than most of us already in a lot of ways. Although there is some question marks about whether a lot of the test materials were actually in the training data, so it basically turns it into an open book test. [laughs] Yeah. It didn’t even really have to look anything up, it just already had the answer in the book.

Tobias: But how would a human learn the answer to that inferentially? Doesn’t the human have to have seen the kind of analogy for it, at least?

Jake: Yeah, pattern–

Bill: But a legal problem, you need to reason your way through. That would be different than a multiple-choice test that you were fed the information. But yeah, you would think AI should be able to– I’d be interested to read a legal answer from AI on a test.

Tobias: It should do pretty well, shouldn’t it?

Bill: Yeah, you would think so.

Tobia: Like, it should be come up precedent.

Bill: Could do the issue, rule, analysis, conclusion thing. Pretty okay, I would think.

Tobias: [crosstalk] How funny.

Bill: Yeah, Iraq, baby.

Jake: I’d have to defer to the two lawyers on the program.

Tobias: I think I’d get there. I saw that it got admitted to Mensa in the Netherlands or something like that. Maybe they got a lower threshold there. Sorry [unintelligible 00:44:03]

Jake: [laughs] Probably, rightfully so. What do you guys think about AI and creativity? Does it enhance creativity or does it make us all dumber because we’re just asking the AI how to do things?

Tobias: I think it enhances it. I think you can use it like, you can pose it questions with things that are similar but not the same and have it answer what are the similarities between these two and get some interesting abstraction out of it when it does that.

I find it useful. I feed stuff into it just to see what it says. I think it comes up with some good stuff. I also think it can’t count. I’ll say, “Give me 20 examples,” and it’ll give me one through 15 and then go to 19 and then 20 and I say, “You’ve missed a few there.” So, it says, “I’m sorry, I missed.” It’ll do a count. “I missed 16, 17, and 18.” And then it’ll do it again and it’ll just miss some different numbers. I find it bizarre.

Jake: Yeah, there are little super annoying things.

Bill: Yeah. I have that with Gemini.

Jake: I was making a little GPT, and I fed it a bunch of material and it was working really well. And then, I just wanted to have the little picture of what it shows for the model. It was giving me a man and I wanted like, “Okay, don’t put a beard on this man.” [Tobias laughs] It was like, “Okay, I’m removing the beard.” No joke. Like, I tried 15 different times prompts to like, “Get this goddamn beard off of this guy in this little picture.” And it could not do it. It just would not remove the beard. It would tell me it did. It would give me like, “Oh, you should see clean jawlines.” I’m like, “No you don’t at all.”

[laughter]

Tobias: Everybody looks better in a beard.

Jake: It could do all this magic, but yet somehow the little things like it’s not– [crosstalk]

Bill: I do like how productive it helps being. I’ve been using like FinChat. I signed up, I got a 14-day trial. I like going through that. AlphaSense gives me access to their stuff. So, I’ll be using that.

Jake: Yeah. What’s the use cases that you’re doing right now?

Bill: Well, if you were to just–

Jake: Just [crosstalk] on a company.

Bill: Yeah. If you were to just like go through a checklist, it can just get you the information much faster than I would be able to go out and retrieve it. And then, you still have the onus on you to like click citations or something. Google Deep Research will return you a pretty decent report and not that long. You’ve got to verify it. It’s not like, just because it wrote it doesn’t mean you understand it because you read it once. But it’s a lot better than me taking three days to put it together and then still not knowing it, you know?

Tobias: Yeah, it’s excellent for research. I agree. It gives you the line and then the citation, so you can click on the citation. I think that’s super useful.

Bill: Yeah. It’s how I wish people wrote.

Tobias: Yeah.

Bill: It’s like, “Okay, then I can actually see what you’re talking about.”

Tobias: Yeah, that thing where you can see it reasoning is great.

Jake: What percent of people are going to do the extra step of verifying the references and going through the logic that it was using to do those things. How many people are going to hit send?

Bill: Most people are in [crosstalk] Yeah.

Jake: That’s fair. I kind of meant more outside, not necessarily investing only, but just everything that’s going to come out now is how well referenced will it be.

Tobias: I think it should be– [crosstalk]

Bill: Yeah. Well, I think to your point– Sorry, Toby, go ahead.

Tobias: No, no, I think you should have to argue like that. I think you should have to put your sources in your arguments.

Bill: Yeah.

Tobias: Footnote it. Footnote each sentence.

Bill: It’s because you were trained.

Tobias: Because many of these arguments that I like, the current one that everybody’s having about the tariffs, like I have no idea whether tariffs are a good thing or a bad thing. I’ve read lots and lots of arguments, but I’ve seen nobody– I don’t know, everybody’s speculating, but– [crosstalk]

Bill: Yeah, I don’t think anybody does. Speaking of AlphaSense, they actually had a tariff thing. It’s like four parts. I’ll drop it in the show notes. It’s reasonably good. But it’s four hours of your time and you’re not going to get the answer. It’s just a way to think about it. So, I don’t know.

Jake: I’ll skip that and go to the answer– [crosstalk]

Bill: The hard thing about the tariff thing is you need– Like, to have the whole picture come together, I think you need tax policy to come together, and I do think you need Powell to cooperate and the sequencing of those things. There’s a lot of variables that are moving around that I don’t think anybody knows how this all plays out. I know that if the market goes up, people will say, “You always buy the dips. And if you didn’t buy the dip, you’re a moron.” And that’s what you have to do.

Tobias: [crosstalk] Yeah, that’s right.

You can find out more about the VALUE: After Hours Podcast here – VALUE: After Hours Podcast. You can also listen to the podcast on your favorite podcast platforms here:

VALUE: After Hours (S07 E14): Gödel’s Theorem, AI Limits, and Machine Logic

Read next