Why AI is Doomed to Fail the Musical Turing Test

I am a computer scientist, and the category error problem constantly annoys me. We find a problem that requires a lot of intelligence in humans, like playing chess or go at a grandmaster level, and declare that AI is therefore "intelligent". For some reason, it's only AI that we use this kind of language about. The best human weightlifter is easily outcompeted by a small forklift, but we don't call the forklift "strong". The best human sprinter is outcompeted by a locomotive, but we don't call the train "fit". Hell, computers have been beating humans at mental arithmetic for ages, and that's even a marker of human intelligence. To quote the great computer scientist Edsger Dijkstra, "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." One of the under-appreciated aspects of the Turing test is that it was an activity that humans should find easy, not something like playing chess which humans find difficult. It's these "easy for us" problems where AI tends to fail, partly because they are the problems that machines find very hard, and partly because you can't get money to solve unspectacular problems. I want a machine to do things that I find easy but tedious, like cleaning my bathroom.

0:00 "What would it take for a machine to jam?" Very little, actually, my printer jams all the time.

This is so interesting. And such an honor to be featured! I feel like you and Jack could nerd out for days on AI (and music).

Adam!! Drive-by praise during the sponsor read, I was not prepared!

I think the Red Lobster thing isn't proof that AI is now good at producing music, but rather an indictment of how bad music in advertising is.

Things that will stop AI at jam sessions: “the usual key”

I’m about to start randomly yelling out “BLUES IN E FLAT” At the jam from now on. I’m sure everyone will love that.

Imagine having to prove you're not a robot by playing free bird on a fucking hurdy gurdy

One might even call it a Touring Test

It doesn't need to pass a musical turing test to anihilate 90% of media composer jobs.

I really very much like the concept of "musicking", it reminds me of a quote from someone I very much respect a few years back, they were talking about NFTs and the commoditisation of the visual medium, but this hit me so much I don't think I'll ever forget it: "Art is a verb. It's a process. It's an act of communion. What hangs on the wall is a fucking collectible. What you and the artist communicate across centuries is the art." Obviously Small's book and concept well predate this quote, but it honestly changed my perspective on why art and music mean so much to me, and what's truly valuable and meaningful to me as someone who appreciates these things.

I gotta say: I'm glad no human being was forced to record those Red Lobster lines...

I always forget that the guy from pomplamoose is the one who invented patreon, and every time I'm reminded of it I'm blown away all over again

Someone clearly hasn't been jamming out to 'I Glued My Balls to My Butthole Again' as much as we all are.

Professor of computer science & amateur musician from the Netherlands here. First, thanks for all the thorough and well-researched videos, always a joy to watch. "AI cannot do X" arguments are, in my opinion, always tricky: AI has surprised all of us, even researchers in the field, with its incredible progress. In particular, I am not convinced about the interaction argument. Reinforcement learning is branch of AI that is specifically tailored to interacting agents learning behavior in a dynamic environment. Amazing progress by companies like Boston Dynamics has enabled robots run and do back flips. I see no reason why in several years this technology would not be able AIs to play in jam sessions. Sure, there are challenges, like there were in AI before. So the real question is how should we musicians, writers, scientists and all other relate ourselves to AI? This video makes a good contribution to that debate and the various aspects.

The soccer analogy is the best way I’ve heard to show the difference between the end product and the process/creators behind it. An AI could very feasibly be trained to generate HD video of a full sports game. But I’d be very interested to ask sports fans if they’d be invested in the outcome of a game that wasn’t physically played and where none of the players are real people. Would they watch excited to find out who will win, worry about their team making a mistake, believing that any outcome is possible? Would they wear the jersey of their favourite AI player? Art isn’t art without artists the same way that sport isn’t sport without athletes.

Hi Adam, this is one of the best "AI will never be able to do " argument I've ever seen ! For context, my current job is about GenAI (for instance I know exactly the ElevenLabs settings you used for Xenakis), and I started doing machine learning 20 years ago. So I've heard my share of silly "AI will never be able to do " arguments, that proved to be wrong in the end. I'm also an amateur musician who knows what jamming is, and the very subtle body language communication it can involve. I was very skeptical at the beginning of the video, but this is maybe the first time I hear such a compelling argument against AI taking the place of humans. Two thoughts to slightly moderate your argument : * I do think that machines will be at some point able to pass your musical Turing test, but it might require a big paradigm shift in architecture, training procedure, etc. (a shift that will not likely happen soon for the reasons you describe around economic incentives). * In a sense I feel that that Turing test is a bit unfair in the sense that it involves doing a thing well "with humans", and not "between AIs". It is like saying "Oh but humans will never pass the spider test which consists in being able to get offsprings with spiders". I'm sure that AIs will be able to jam "between them" because they will find digital ways to replace the subtle body language and so on. (It reminds me the "Hanabi" card game test : it is a cooperative game that AI can do very well when they are playing together, but do badly if they try to play with humans.) Keep up the good work !

As a researcher on social interaction (including social and distributed cognition), I love that you did a deep dive into cognition beyond the computational model that is so prevalent in everyone's imagination at this point. I hope folks learned something from it. (I'm also a little jealous that you explained so clearly in a few minutes what took me years of PhD study to understand. Bravo!)

Around 25 years ago, I attended an informal live musical Turing test (the Output kind) hosted by Douglas Hosftadter at the University of Rochester, co-sponsored by the Computer Science department and the Eastman School of Music. Hofstadter was showcasing what was then a state-of-the-art generative music composition AI. I'm pretty sure it was a version of EMI by David Cope, but my memory of the event has faded with time and details are scarce on the internet. I've found articles about Hofstadter's work with music AI around this time and articles about David Cope, but nothing about any live experiments or presentations like the one I remember attending. The key element was that the AI was trained on a corpus of a real composers work; a large set of similar compositions (I think Bach's chorales). The idea being that there were enough that even the musically literate in the audience wouldn't know each one by ear. A skilled live pianist played us half a dozen pieces, some from the original set and some from the software, and the room voted which we thought were originals and which were computer generated. The audience was a split between people there because of the music connection and people there because of the computer science connection., and did merely an OK job identifying the real compositions. At least one of the computer generated pieces was obvious, but several were good enough to split the room, and at least one was only differentiated because some of the room knew enough of the chorales to know it wasn't one of them. Earlier in his career, Hofstadter made a list of ten things he thought computers and artificial intelligence could never do, and one of the things he said at this presentation was that the work he was doing had led him to cross "create beautiful music" off of that list. And this was 25 years ago. I think that one day, computers will be able to jam with you. I agree with most of the video about why it will be hard, and why there isn't money in developing it. I don't think it will be easy, or it will happen soon. But I think that eventually, after one or two or three more generational paradigm shifts, the ability to do so will fall out of some other profound advancement practically for free, like how so many basically economically worthless but really fun roleplaying and other conversational abilities have been achieved for free with the profound paradigm shift of the GPT-based chatbot advancement.

Transformers are really good at one thing and that's pretending to be something. I'm convinced that if ample training data of jam sessions, with specific instruments removed and added to the mix (using simple music editing programs) is provided to an AI, we could have a real time musical jam AI, similar to how we have real time voice changing AI now.