Your brain is a simulation machine.

I think there are compelling reasons to dispense with the "statistical parrot" hypothesis for LLMs: 1. As LeCun et al. showed --- and discussed on MLST -- LLMs exclusively extrapolate, never interpolate. Even in compressed latent spaces, the curse of dimensionality makes interpolation virtually impossible. 2. LLMs have remarkable zero-shot & few-shot capabilities. They can learn new skills on the fly, often faster and better than humans. 3. LLMs can solve a unique problem that you construct from a combination of many skills. The combinatorial math here pretty quickly requires that folks who believe LLMs are hash maps to also believe in miracles. (A more formal sketch: make a massive, arbitrary set of know LLM skills S = { bunch of ~disjoint skills }, e.g. S = { "rapping", "C# unit testing", "US tax law recall", "solving 1D kinematics problems", etc }. Compose a question using a handful of such skills X ⊂ S. For set sizes s = |S| and x = |X|, the chance of an LLM seeing/solving a similar question before is ~ 1 / C(s, x). That is, it's inversely proportional to s choose x. And those initial astronomical odds are just for starters.) 4. There are truly open-source models with corresponding data; OLMo being a recent example. Conversely, researchers & companies must still carefully attend to training splits, contamination, etc. in their private data. There's little reason to expect some smoking gun to reveal that model reasoning is merely 3 interpolation algorithms in a trench coat or whatever. Taking a broader perspective, dismissive skeptics often seem tunnel visioned on rationalizing their position rather than informing it. Their confidence feels misplaced beside our modest scientific reality. We don't understand how our own minds work, let alone how other minds could. We can't agree on the definition of intelligence, let alone its objective measure or mechanisms or detection. Ditto for consciousness, reasoning, agency, etc. I reflect on the transformations of this past year. The hundreds of millions of people helped, and billions in market value. The immense help I've received. The crisis in universities & schools. The untold thousands of people now saved from the trauma of moderating content of the most unspeakably heinous acts of (in)humanity. The ingenuity of the impaired who are taking their lives back. The productivity multiplier now augmenting engineers, lawyers, healthcare workers, data analysts, teachers, etc. The nuance and intentions unlocked in communications the world over. The expressions of artistic creativity. The deep societal anxiety. The leading machine learning experts, academics, and millions of people globally who believe that these systems exhibit genuine intelligence or reasoning... Those who look at all of this and (often instantly) categorically dismiss these LLMs as basically handy autocorrect that has fooled the world...it's difficult to see that as anything but naked bias. A "real" vs simulacrum distinction usually asserted without testable, falsifiable predictions of real-world consequence, i.e. pseudoscience. Metaphysical hairsplitting. And hey, maybe these systems are convincing impostors. Or maybe the origin of intelligence is disappointingly prosaic; a pedestrian byproduct of unceremoniously throwing scale at any sufficiently general problem. And expecting otherwise was just anthropocentrism, human presumption that led to Copernican humblings -- learning that humans occupy no special place at the center of the universe, the solar system, the tree of life, or (finally) on the spectrum of intelligence. I don't know. I guess what bothers me is that nobody else does, either. So anyone pronouncing otherwise ironically sounds less authentic to me than anything said by GPT-4.

Great teaser! When will full episode be released?

IMO we just get really good at constraining hallucinations as we age. This is useful as it allows our simulations to be more realistic, but if you've ever talked to a 4yo it's amazing how much more vivid and creative their simulations/hallucinations are without all those constraints.

I'm a sucker for your book recommendations. I just purchased it this morning.

Already looking forward to the full episode!

Good job on the book Brodie, I loved it

Less is more with editing

5:40 a) sensory experience bombards the brain with a shipload of data, it's only logical that the phenomenology is more intense if compared to a local to the brain, internally induced simulation b) it makes no evolutionary sense to waste a lot of energy to generate all the finest details when reproducing a situation in our imagination, when a sketch is more than enough to evaluate pros and cons There you have it Max!

love these short form videos

The end was epic 😂

I've been on a Donald Hoffman binge lately so this is perfect.

The intelligence I am interested in is the kind that are trained while deployed without catastrophic forgetting.

A social calculator i called it.

Book is EXCELLENT Wonder if Patreon chat says what Max thinks In Breakthrough #6

Is this ai biased interpretation of brain ?

Talk to Bernardo Kastrup!

"from a phenomenology point of view"

Tim, since last almost 3 years you try to figure out the diffdrences between human and machine intelligence. Therefore you started your channel mlst to have the opportunities to interview brilliant researcher in ML, CS and neuroscience. End the day we can conclude that the machine is machine and the flows in system can not be fixed. Now, the question what is the human intellegence in era of ai? Is iq tests, universities exams, degrees or number of scientific publications? What is exactly and the human intellegence should defined? Every expert and researchers have different answers. Here is simple definition of human intelligence in era of ai: if you can tasks and reasoning the ai-system can not, you are intelligent. The reason: if the ai system can solve 95% of problems where the system trained on, because the correlations are easy to detect. If you are capable to solve problem in rest of 5%, you are smarter than most people on this planet. The universities do not prepare the students the ai-culture schock, after they are joining the workforce - next generation will be hopeless lost and no chances to win tge competition against ai-system.

Well this all sounds fine and dandy but how do blind people experience a simulation ? Words of the week: diodastic corpus dispersed Introspections recapitulating inductive biases stochastic infosphere I'm just a simple guy living my simple life maybe I should incorporate some of these words into my everyday vocabulary to understand simulation 😁

Internal and an external simulation