In episode 40, Patrick and Cyprian speak with Rob Freeman, language and artificial intelligence enthusiast. The team discuss linguistics, machine learning, and defining grammar across all languages from first principles.

Welcome to Entangled Things, your quantum computing podcast hosted by Patrick and Cyprian.

Hey, Cyprian, how you doing?
Hi, Patrick. Very well.
Ready for another great episode of Entangle Things. I think we're going to have exactly that today. We're joined by Rob. Rob, do you mind introducing yourself to our audience?
Okay, so I'm... My name is Rob Freeman. I have an interest in language, initially primarily, linguistics, and then from that leading on to artificial intelligence and a somewhat unique perspective on that whole area, I think.

We're very interested in that perspective. We've recently talked to a few people who have shown us that there is a link between quantum and linguistics. And is that a burgeoning field that we just didn't know about until recently?
Not at all. Unfortunately, I mean, burgeoning is relative. In fact, I'm on the podcast now because I heard an interview you had with Bob Coecke. I don't know exactly when it broadcast. And just so happens that I think he's one of the very few people who's looking at that aspect of the problem. And So, I mean, I like that very much, though I have a slightly different perspective to him. But no, I think that it's actually very much an ignored area of AI. And it's one I'm trying to draw attention to from all aspects. And, you know, in particular, my own perspective, which possibly is unique.
That sounds interesting. So AI is definitely in Cyprian's wheelhouse as well. It's something I know about tangentially. But how does your, what's the flavor of what you're thinking? Well, that firstly, intelligence, so artificial intelligence, but intelligence has a quantum aspect to it.

So, I mean, I'll give a bit of history. So I first came to this. I was working, worked in machine translation early in the 90s. I was working in Japan on machine translation companies. and then subsequently at Fujitsu actually in Japan and then subsequently I was working on a project to for the characterization of grammar in in Hong Kong and It, I mean, so I sort of stumbled into the field of machine learning back in the 90s. So I was trying to find the grammar from first principles. And so what I found was that you could learn the grammar, but it had a kind of a... a contradictory character and and it was actually... having a like a undergraduate degree background in physics... um it had uh it it felt very much like a a kind of an uncertainty principle actually of grammar is the way i the way i saw it um and um so Yeah, so I mean, I sort of stumbled onto this idea that there was something like a quantum character to grammar, in fact, to language very early on.

Regardless of the language, for all languages that you've looked at,

Well, absolutely. I mean, it just comes from like so all languages have grammar, right? So, I mean, it's been a basic problem and this is what I found in working on machine translation in Japan as well that I mean, like the project I worked on in Japan, this is a fifth generation project. So this was going through the 80s. It was a massive project that Japan was going to leapfrog its way ahead of the West and take the lead in computing. And the way they were going to do it was human factors computing. And... So they invested a massive, massive amount of money and effort in, in human factors computing, natural language interface. And I sort of, I joined that project right at the end. Um, of that was through the 80s. I joined it in 1989. So it's actually kind of, it was a very privileged way to join it. I got to arrive just at when the effort had been made, I got to see the end result. And the end result was that you reach a certain, every day we would look at the results of the machine translation system. We would edit it and we would correct the grammar and try and and correct the errors so that the next day you would have fewer and fewer errors. The idea was that you would take off with your errors. But what we found is, in fact, is that you got to a certain point where you would correct the errors and, yeah, the next day those would be fixed so that there'd be other errors. and then you correct the new errors, and then you come back to the original errors. So you reach a sort of a certain steady state where you weren't improving. You were tapering off. You were asymptotic. And you could not capture the last 10, 20% of the grammar. There was this sort of, once again, this kind of indeterminacy between one way of looking at things and the other way of looking at things. And it wasn't that either one was correct. It was just that they were in... Neither was a perfect fit. Neither was a perfect fit for the waveform, basically. Neither was exactly. It was like a wave where you could see the... You could measure one thing about the system. You could measure another thing about the system, but you couldn't catch the whole thing at the same time. So, yeah, I saw this kind of... I mean, actually, it was kind of an epiphany for me because, I mean, having done the undergraduate degree in physics, and I found quantum fascinating and as a very nice perspective of the world. But I mean, most people, I mean, that's the sort of thing which you learn, but you don't really understand. I mean, intuitively how it's just very bizarre that things behave that way. You don't understand why they behave that way. But this was a fascinating perspective because I was actually sort of coming at it from the other direction. I was coming at it from underneath. I had the system, which was well, language. And you can look at, you can treat language as a data, and then you do machine learning on it. So you're going to learn the patterns in the language. And But as you learned the patterns, actually, you got something like quantum mechanics appearing. So you were coming up, you were coming at a sort of a quantum mechanical behavior, but you're coming at it from the bottom. It wasn't something that you were hypothesizing and then testing. It was something which emerged naturally from the process of trying to learn grammar. So actually, it was a fascinating perspective because it was like, wow. So it's like, yeah. quantum-like behavior emerges naturally from assemblies of elements of data. And I was thinking, well, so, I mean, it made me think, well, gee, I thought, well, it's an, you know, why would I have this perspective? It seemed like a very explanatory perspective. And I thought, well, it must just be random. I can't imagine that has any relevance to true physics, but it certainly seemed to have relevance to language and grammar. So I thought that was interesting. Yeah. And I just thought it was, it felt kind of explanatory to me from the perspective of physics, but I didn't really go into that any further. Recently, there's been some articles about, I don't know whether they say there's proof, but there's indications that consciousness is quantum mechanical. and that there's things that we can't explain about the nature of consciousness. And language certainly springs from consciousness. And so maybe you just saw that connection very early. Well, I mean, so for me, the first, yeah, I mean, I was seeing something happening with language. I thought that the relevance to physics would be a separate issue. I mean, cognition and physics was just a chance that they perhaps acted the same. There's two sides to it. So there's consciousness, but actually in the physics side as well, I was... delighted to find a few years later that I came across an article in a new scientist by a guy named Robert Lockland. And he's a solid state physicist. And he was talking about the quantum-like character of superfluids. And So you was talking about macro quantum. So, I mean, this was a wonderful vindication for me, actually, because he was saying that you can look at actual quantum mechanics as having some kind of emergence from assembly perspective as well. And... So there's that direction there seems to be, and there's been other vindications of that as well. So that's for the pure physics that does seem to be some kind of emergent assembly aspect to actual subatomic particles to quantum physics. So I don't know if you have you heard of this guy, Robert Lockland, either you or Siprin perhaps? I have not. Siprin? So he wrote it. I've read some of, yeah, I've read some of the very interesting theories. So he wrote a book called, which the most memorable title anyway, is Reinventing Physics from the Bottom Down. So, I mean, it's a nice play on words. So he's looking, he's saying that we have to refocus. Instead of looking, trying to borrow down to greater and greater abstraction, we have to look at the way things act together. as an important quality in science and physics in particular. And to your point, what I find fascinating, like, looking in general, right? Quantum computing is a very kind of specific problem of this. But in general, I see this huge gap, like between kind of analyzing and trying to understand phenomena at the discrete kind of quantum level, like studying the behavior of a photon or the spin of an electron or whatever, right? And then, like, to me, it feels like there's a gap. And then we're getting into the realm of understanding, like, complex structures of matter and things like that. There seem to me like a hiatus here where we don't really understand how these discrete phenomena, right, the discrete quantum phenomena are essentially kind of building up into the more complex one. And to me, this is exactly the... kind of the equivalent of the big problem that fundamental physics has today of reconciling, right, between the current kind of standard modeling physics and quantum, uh, mechanics and gravity, right? Because at the end of the day, right, if you look at gravity, gravity is something that you kind of feel and it happens at like macro levels, right? That's fascinating to me. But I love what you said about, right, the connection of quantum and language and patterns, because I think There's a very interesting connection at the end of the day in terms of the mathematical models that would describe the results, right? We use a certain mathematical model to describe the world of quantum, and then we use various mathematical models to build and describe like machine learning processes that have on language. I feel like at some point, right, there are synergies between those mathematical models, like. right absolutely you know i mean there's different uh directions you can you can take this but um i mean one one of those directions that occurred it's occurred to me is that you know people are building these uh these parallel processing uh chips uh now a lot a lot of the large companies are starting to explore this ibm had this chip this uh true north chip they had it it's like they had a 10 years ago or more like 15 years ago they they built this chip called true north which was uh i think it's actually a spiking uh parallel processing chip um and they so they had this thing and they were trying to figure out how to use it nobody nobody knew nobody knew how to use it and still nobody really knows how to use it so and intel recently also they have uh this loihi chip you've heard of that one that's um It's also a massively parallel spiking processing chip. And so they've got that in like limited beta with different research institutes. But it's also a question that people people are sure that these kinds of chips must have lots of potential, but nobody really knows how to use them anymore. We're still very much stuck in this whole sort of von Neumann logical way of programming. And we think, gee, this thing must work, but I don't really know how to use it. But I'm sure that we figure out that it might be some use for it. And I mean, it has occurred to me that because of this connection I see between what I believe is being called macro quantum actually. So this is quantum behavior at a macro scale. So it's quantum behavior between assemblies, like not at the subatomic level, but at an assembly level. So and super fluids and super conductors and that sort of thing. It's like between like when you've got very, very low temperatures and you start to get quantum behavior with, with between vast numbers of electrons rather than sort of within an electron. Um, So this quantum-like behavior of assemblies of things might be, it might mean that one application of the current work which is happening with quantum computing might be as a mathematical formalism to use some of these new macro architectures, some of these new parallel processing architectures like, you know, True North or Intel though easy. So match may be the same. Yeah, sorry, Patrick. Did you want to say something? I was just asking, are we thinking that this is, optimization and you'll give the system. I mean, as with most things, most of the language processing, if we try to put this into an application, would be classical compute. You'd take in the data. You'd parse the data. But then you might get a series of five words that. has an idiom to it and that may have multiple meanings. And maybe you'd put it into a system to read its Hamiltonian, or you'd put it into a system to figure out its lowest energy state to figure out what its most likely meaning is. Is it that kind of application, or are you thinking in a more holistic way? I was thinking more generally, actually, that problems which are currently solved using quantum computing might be also solved with quantum computing, but it could be done at a macro scale so that you could do your quantum computing, but do it on top of a massively parallel processor where the quantum behavior would be, you would actually be have the quantum behavior emerging from the the the distribution of processes on the chip. But you would be corolling, you would be... guiding that processing as quantum computing is done today. So it would be quantum computing, but it would be macro quantum computing for Lang. And I don't know how that would happen because actually I'm not an expert on how quantum computing itself works. I'm only I only see this parallel between what emerges from machine learning of large assemblies of data and language in particular. As to how it would happen with language, I mean, there's also, you can also apply, you know, the quantum abstractions to the language problem. And that, as I understand it, is what Bob Koch is doing, for instance. So he's analyzing language using quantum abstractions. So he has different, I think, what, gates and different... abstractions for grammar. And he's analyzing it top down in that way. For language specifically, I have a very particular interpretation of how I think that processing should be applied. Okay. And that comes from this perspective that I've had that the whole thing emerges simply from trying to do machine learning of grammar. And the way you do that, the way you do machine learning of grammar is you look at distributions of word sequences. And if a word has associated with this, you get like a vector or a distribution of context. And then you find that those different words have similar vectors. You can cluster those and then you can spec, you can hypothesize that's a grammatical category. And you can, and that's your grammar. But so from my particular perspective on that is that because those categories are actually have a kind of a quantum quality, you can never get a single category. You always have sometimes it acts in like one category and then you collect its distribution of behavior. It is probabilistic, but it's more than... I mean, it's fuzzy. The way I see it, the way I've always explained it was, imagine you've got a group of people I mean, this is what you have with language. You have language, you have a big, like, a bunch of sentences, words and word sequences. But you imagine it's like a group of people and people have different attributes. You can take their height, perhaps, and say their IQ. Okay? So now, you want to order this group of people. So you say, okay, we'll order them. We'll order them according to their IQ, right? So you've got going from, you know, whatever 70 up to 140, whatever, 90, 100, 120. But their height is going to be completely mixed up. Right. It's the primary key thing. Sorry? It's like it's in data science, there's the primary key. You can only physically order data in one way because the odds of two things lining up is nil. Exactly. And similarly, if you say, okay, well, order them according to height, then the IQs are going to be completely mixed as well. So you can never order something in every way at once. So if grammar is, if grammar behaves like that, then the only way you're going to be able to get the complete grammatical information is that you have to be able to order on the fly. You have to be able to do it dynamically. You have to actually sort of collapse the data. I mean, if you think of the data as a kind of a wave function, you have to collapse the data either in terms of IQ or in terms of hype. You can't do both at once. So I'm going to, data scientists, please speak. I don't know. So I think that's kind of like a very interesting view in a sense that What you're really saying here is that you look at, let's say, language, you look at grammar, and there are possibilities of having simultaneously, like, multiple interpretations of what's there, like multiple, multiple models. And that to me sounds like very, very interesting. And I would just like to... to take 30 seconds for our listeners who are not really into like machine learning and language and just highlight the fact that the way to do we do language processing today is we essentially take natural language we transform it into a mathematical model which is a fancy name for a bunch of numbers like lots and lots of numbers and and vectors and then Machine learning operates on top of those, right? And the kind of the name of the game is how do you learn those vectors? How do you kind of do this what we call word embedding and sentence embedding and everything that derives from that, right? So this is where we are today. And while I think that natural language processing has some remarkable results, I think we're still very, very far from completely cracking the problem of language because when we get into kind of meaning, nuances, interpretations, things like that, we're still quite far from kind of having like a compelling solution or approach. Or for that matter, a single clear strategy to deal with that. Absolutely. I mean, the big success of the last five years has been the was the GPT series from OpenAI with these Transformers. And they've done amazing, astonishing things with very human-like sequences of, apparently very human-like sequences of words, which they produce from these numerical models. And... I mean, famously, I'm sure you know they learn the structural parameters of a language. And they've gone, I think, as far as the GPT-3 learned 175 billion grammatical relevant grammatical parameters. And there's already word about moving towards the trillion barrier. So well, Jeff Hinton joked at the time when GPT-3 came out. He said, look, you know, okay, so you got 175 billion. Where's it going to stop? He said, I expect that the final answer he said would be two to the power of 42. Yeah. because 42 and I would highlight 42 out of that 42 must be the answer so he's like you know okay well 175 billion is not enough you know where's it going to stop so I mean for me what it's saying is that you I mean this is grammar you're learning grammar and you're getting how many parameters you're going to have and um i mean they spent 12 and 175 billion it costs them 12 million dollars in processing time to generate them all right i mean it's like burning a hole in the planet to to generate all these damn things because there's so many of them and um and it's a major barrier for anybody else to get involved i mean nobody no nobody can compete with these major corporations because nobody's got 12 million dollars that's He's got $12 million to train the model, even if you release the source code. You know, I mean, who's going to spend the money to? So, and yeah, so this is the way modern language processing is being done. And it's exactly the same as the sort of stuff I was trying to do in the 90s. I didn't have the, I didn't have the, you know, the sort of the super computing capacity or the millions of dollars to find them all. But we were also trying to find the grammar. And I mean, I think what they're doing with these systems like GPT-3 is, I mean, yeah, 175 grammatical parameters. I think you can go on to infinity. I mean, because it's all just different orderings of the data. It's different orderings of the data. And I think actually the way the orderings go is that you can essentially generate an infinite number. The other thing about what these transformers do is that they generate billions of parameters, but they don't reveal any structure. There's actually and and this is why they have great difficulties with with meaning because they they produce plausible sequences of words, but they don't they don't actually reveal what the The structures are which are leading to these These plausible sequences of words And that's that's the the missing link for meaning as far as I'm concerned, that they're not giving you the structure. And I think the reason that they're not revealing the structure is because you've got this tension between different structures. You cannot learn. You cannot. They're trying to learn the structures, but they can't learn them because there's multiple ways of doing it. You know what I like to say about deep learning, I've been working for quite some time now with it. Usually people ask me like, wow, deep learning is so powerful, right? And sometimes people ask me like, what do you think are the weaknesses or the weakness of deep learning, right? Which, by the way, deep learning is one of the paradigms that we use. It's actually used by GPT-3 to train those models, right? Yeah. One of the big problems and drawbacks of deep learning is that it has absolutely no capabilities of superposition, right? Which is kind of like, right, one of the things that because at the end of the day, no matter how large, no matter how big the neural network is, right, At the end of the day, the whole process is fairly deterministic. It will learn at the end of the day a function, a super complex function, unbelievably complex function with potentially trillions of parameters, right? But that particular function has absolutely no ways of kind of, let's say, living in alternate worlds. And by the way, one of the things that people ask me about they hear, they look at the news and things like that. Right. They're like, wow, Cyprian, we saw this GPT, whatever, open AI stuff. And we heard Elon Musk saying that singularity is coming upon us. And wow, like what is what is happening? Did we crack the human brain? Right. And then I always kind of try to make this parallel between what we have today and the human brain. Like, yeah, we used to think the human brain has a hundred billion neurons. Now we're kind of more down to 86 billion. That's the kind of like the common knowledge right now with probably close to maybe several hundred trillion connections between them. Even if we get with neural networks to that complexity, there's one thing for me that keeps us kind of like very, very far from getting there is the connections themselves. In machine learning, in deep learning neural networks, those connections are essentially real numbers. In the real world, those connections are chemical networks. processes that at the end of the day are governed by the laws of quantum physics. So there is the distinction between having a connection described by a single number and a connection described by a complex quantum process of chemistry and kind of fundamental physical interactions, which is why I believe, including with language, which by the way, for language we use a very... not a very limited portion of our brain of those 86 billion neurons, right? But still, we're very far from being able to, let's say, duplicate what's happening inside our brains. So, I mean, we're far from being able to duplicate it. But, you know, for me, actually, the micro quality of the neurons is not... I mean, I think, sure, there's something that we need to learn more about that. But for me, what is... still interest, what is interesting for me is just the connectivity because I think even the connectivity, we really don't grasp what the potential of the connectivity is. One thing which I always like to bring up in the whole sort of deep learning context is why is the question, why has deep learning... dominated, why is it supplanted the sort of, I mean, so AI in the earlier on there was the symbolic AI and then in the 90s symbolic, certainly for language, it was symbolic grammars. And then it was supplanted by statistical AI. And so it's like, I mean, I mean, I don't see it being addressed. I don't see people addressing why deep learning works. It's like it's this wonderful thing. It's this black box that you apply the problem. But nobody, I don't see good answers from people as to why it's better than logic and symbolism. I mean, I have an answer, but I mean, I don't see that answer being, I don't see people addressing that question elsewhere. Yeah. I mean, we're it's very interesting, right? In a sense, it's like very similar to what happens in quantum, right? We understand some of the consequences. Like we understand, for instance, how does entanglement look? But we have absolutely no idea at this moment why entanglement. entanglement works the way it works, right? There's where we're like very far from an explanation. It's pretty much the same with neural networks, right? You see them work, but if I go into the, let's say, the GPT-3 model, I take, one of those 175 billion parameters, and I change it, and then I challenge anyone to explain or to calculate or to evaluate in an a priori mode the impact of that change, right? It's virtually impossible at this point. I think the secret of why deep learning has dominated symbolic approaches is that... It's because it is less structured. So we're seeing that these networks, they have different ways of implicitly or sort of they have a de facto different ways of organizing information, but it's hidden from us. We're still trying to use them to learn things, but... Actually, we're sort of, I mean, I see the move from symbolic to probabilities to deep learning as being a process of moving towards learning less. We've been, so it's the different ways that the elements can be combined together in deep learning are, are much greater than they were in symbolic systems. So the, the, I said as being all, it's, it's about these, I find different ways of ordering data that we've actually been moving towards a less ordering of data. And the less we order the data, the more results we get. I'll take a, an analogy. Why has... Why did Google dominate Yahoo as a search engine? So, you know, in the 90s, we had different search engines and Yahoo was yet another hierarchical ordering Oracle or something like that. And they attempted to make a directory of the Internet and they were going to have hierarchies of categories of things and you would find it by going through this directory. But the one which succeeded was actually Google, which is indexed search. And so, I mean, what's index search? Index search is actually a sort of an ad hoc ordering of data. There is no fixed order in index search. Everything is done ad hoc. And I think that... That is telling us that actually there are no single ways of ordering the world, that we have to embrace this idea that there are multiple ways of ordering the world. And I think what GPT is finding with all these parameters is it's just trying to list an infinity. I mean, if there's an infinite way, a number of ways of ordering the world, then... you know, 175 billion is one step on the way to that infinite noise of ordering. It relates to quantum in the sense that so quantum is also says that there is no single way of ordering the world. It says that there's always going to be a tension between different perspectives and you need the observer to collapse the wave function in one way or another depending on the context and the observation. But if you want to have a complete ordering, then it's always going to be a distribution or a tension between different orderings. And I think that... I think that quantum today, at least whatever we understand out of quantum, right, is kind of capable of managing in a hidden way for now to us, right, multiple orderings or multiple states at the same time, right? Exactly. I think that's why quantum kind of is so promising. provided that we will be able to properly tap into that hidden complexity, right? Because one of the things that we're constantly kind of reminding to our listeners is that as of today, that fabulous complexity, right, collapses to some very simplistic values as soon as you attempt to measure it, which is kind of like the way Mother Nature seems to... protect its powers from us. I think this is it. I think that this is a fundamental insight about the world is that we've got to get a greater appreciation for the importance of being able to order things in different ways. And that's relevant for quantum. And it's also relevant for learning grammar. And so when you try and learn grammar, you get a sort of a macro quantum because you've got to take into account this possibility to order things in different ways. And I think that ordering things in different ways is still being ignored in machine learning. and artificial intelligence. We still are not really addressing this whole ability to order things in different ways. I think that deep learning has succeeded in the future. over the prior symbolic processing because implicitly in a deep network, you still leave a certain ability to order things in different ways. We don't know that's why it works. We just know that you throw this black box at the thing and you get better results out of it. I think the reason it's working is because of the ordering into things in different ways. So you just reminded me of something that one of my mentors in business has had always told me. He says that most people never look for an alternative solution once they found one because they're just so happy to have found one. Yeah, well, yeah, I mean, so maybe... Maybe the application here is right now we have machine learning and we come up with an answer and we desperately want that to be the answer. But maybe like the lessons of quantum, we have to try multiple times and then come up with a probabilistic answer. answer of what the most likely answer would be. Yes, and look, the beauty of doing it at a macro level, so this whole macro quantum idea, is that actually you don't have to guess. The system is probabilistic in the sense that you can't say that it's one way or the other because there's multiple ways of ordering. But with a macro quantum system like language, You have access to all the parameters. You have access to the data. So you can order it in the way which is most relevant to whatever problem you have. So you can... Context. There's actually a deterministic system. Well, we're starting to run low on time, though not on topic. So I will ask that you and Cyprian make sure we round out this conversation so we don't leave anybody hanging. What would we want to leave in the last few minutes? I would definitely like to put, like, because this conversation has gone like in a very nice way from my point of view, back and forth between quantum and machine learning in general and deep learning in special in particular. So I would like to ask you, like, do you see a possible kind of convergence at some point or a possible, let's say, greater synergy between machine learning and deep learning in particular and and and quantum. Because as of today, we know that one of the big problems that prevents us even thinking about this, right, is the fact that the current paradigms of quantum computing are not very friendly when it comes to even the small amounts of data, not to mention like huge amounts of data, the ones that are typically involved in deep learning workloads. So where do you see this kind of going? Or if you don't want to do like a bat on the future, tell us where do you want this to go? Or what are your hopes for? I think... I think there's a tremendous and extremely easy synergy. I think that the problem with machine learning and artificial intelligence at the moment is that it does not take into account this possibility of ordering things in different ways. And all we have to do is to imagine that the data might be better ordered in order. in different ways that there might be this sort of quantum tension between different ordering so that we cannot find a perfect ordering. And we have to let the system order itself in the way which is appropriate to the problem at the time you have the problem. And then I think that there will be a solution to the problems that we have with artificial intelligence at the moment. And it will be a quantum solution. And it will also solve the problems of structure because it will be a deterministic system. And we can get this deterministic system simply by forgetting the idea that we might find a single complete ordering and trying to list billions and billions of global orderings. And simply, so all we have to do to solve the problems, I believe, in artificial intelligence now, in machine learning, artificial intelligence as machine learning from data, is to accept that there could be multiple orderings in a quantum-like way. And that could also be a... I mean, it would be a quantum solution in the mathematical sense because it would have a top-down description in terms which would be quantum. And it might also imply some kind of application for quantum computing done at a macro scale on parallel processes. Measure twice, cut once. it's an old measure an infinite amount of times patrick cut once measure as many times as you can measure measure and and for me it's it's cut at the at the time cut the cut cut the way you want to at the at the time you need the uh the answer because i think that there's no there's no single answer that um if we simply accept that there's no single answer for these machine learning problems and that we have to, um, and we have to allow, we have to accept that the system has multiple orderings and there's a quantum like tension between different orderings. Um, I think that we can get structure and solve AI. The analogy, I love analogies. That's, that's my whole thing. But it sounds like, like for example, if, if I was doing negotiations with a head of state from another country that I didn't speak the language and I have no basis for the language, not that that cut is struck from the headlines right now. The, I wouldn't realize. rely on a single translator, no matter how reliable that translator were. I might have four of them. I might have five of them. And then I would come up with, okay, because each person would translate, they would have a different order, like you say, in their mind. And then I would get a consensus. And that consensus would always be better than relying on a single ordering, as you would say. Do you think it would be, though? I'm not sure. I mean, I think that possibly, it's like saying make good poetry by having a committee vote on the... Well, actually, I thought I was supporting your assertion. I thought that I thought my analogy supported your assertion that you should try different orderings because each person would have a different perspective on what was being said. It's not a consensus. It's a subjective take. So every person will have a different take on it and all of the takes will be true. But we need to have the data and we have to allow it to be ordered in the different ways in order to get all the different perspectives. There's no single perspective. If you try a single perspective, then you get a wave function, which is probabilistic. Not deterministic, not precise. So maybe a better analogy is have somebody listen to the words, somebody look at the body language, someone else look at the setting they picked. You're looking at different reads. I'm trying to get an analogy that our audience can buy into so they understand what we're trying to espouse as far as what's a better approach. And I thought I had it, but maybe I don't. It's that we need to... We need to be able to order things in the way that makes sense. Understanding there's multiple ways to order them. Just understand there's multiple ways to order things. Yeah, because I mean, look, everything in machine learning is trying to learn. It's trying to learn a single abstraction which will capture all of the relevant information. It assumes that there is such an abstraction. It assumes that there are a fixed number of parameters which will parameterize all of the meaning in the world, right? Whether it's 175 billion or 2 to the power of 42. But maybe there are multiple ways of ordering. I mean, actually, it gives you all sorts of interesting things. possible solutions to other problems in artificial intelligence. For instance, if you can order things in multiple ways, you have a single system which can be ordered twice, right? And then say each of those orders can be perhaps ordered in different way too. You've got a system which actually grows in complexity. It becomes larger than itself. And so, I mean, so a system which becomes larger than itself, well, I mean, that's the sort of thing you're probably going to need if you're ever going to have a theory of consciousness. Right. Or if you're going to have a theory of creativity, you need to have something which can get bigger, right? I mean, otherwise, what? Creativity is like somebody wrote a science fiction story once about when all the stories had been written. It wasn't possible to write a new story because they'd all been written already, right? That's right. We're coming up on that. I think Netflix has that problem. Yeah, maybe. So we're hitting some deep stuff and I think we're out of time for real this time. It's been wonderful talking to you. I know we could talk about this for another six hours, but let's not try right now. But is there anything else? Last words, anything works that you've read or you want to point people to? Okay. Well, I mean, look, I mean, like a big motivation for me is that, you know, we're sort of taking different limbs, different parts of the elephant and trying to get a hold. But I have a particular experiment, which I'm very keen to... to try. And that is a structuring language in this context dependent way. So this multiple orderings way. And I would like to try that experiment, but I need access to parallel hardware. So if somebody wants to experiment with a different approach to ordering language, one which will simply change the current paradigm from deep learning to actually dynamic ordering, then I want to try the experiment. Love to see that happen. That would be interesting. Get in touch with me and we'll try the experiment. Thank you very much for joining us. I hope somebody reaches out to you on this, and if they do, they can go through us for it. But we're out of time. Thanks again for talking to us. Thanks. It's been an absolute pleasure. It's been fun talking to you guys. Thank you. All right. Thanks, everybody. See you soon.