"So that is my call to arms. If we want the full power of AI scientists, then we should share the goals with them so they can create, evaluate, discover, and in these ways fully participate in achieving the goals. Let’s be bold! Let’s fully automate Creativity and Discovery!"
Should we automate exercise and play as well? How about learning?
The machine didn't have a soul, so we donated ours.
I don't quite follow his point. Is it: a) that we need a new foundational algorithm that integrates a goal (one with "taste") directly into the training step, or b) that we need to point trained models towards goals as they iterate?
If it's a), he doesn't propose such an algorithm, and I don't know how you'd do it at such a low level because how do you quantify abstract goals? Did he suggest such an algorithm and I misread? If it's b), that already exists, see AlphaEvolve or any number of things he said. Or, to be a bit of a smart-ass, just type /goal and let it rip ...
I also think he's just categorically wrong that LLMs cannot do good and novel things. And if it can, then you could just say "well that's not novel, that's derivative". A simple example, if I make up a programming language with an LLM and it works well for my purposes, then is that not novel and good? I mean, is any language other than FORTRAN not novel?
Everything is derivative and you can put an LLM in a loop to evaluate LLMs trying things. I must be misunderstanding because he's too smart to be this wrong.
LLMs possess the map but are unable to discern fertile from barren ground. For instance: how does Anthropic's new model generate promising 'medications'? Because, beyond the knowledge embedded within the model, it has assimilated AlphaFold's reasoning paradigm. By itself, Claude would be incapable of engineering a protein analysis method
I think the variation, evaluation, and selection idea is a good, if not the only, way do do creative work.
I don't think I would attribute anything in that process that I would consider an AI to be incapable of.
The characterisation of variation like this would seem to rest on the same 'random but directed' crutch that some free will arguments rest upon.
There is no random but directed of course, there is random and there is caused, and there are things that use both as components, but the random remains wholly random, and the caused remains entirely deterministic.
I think there is a good case to say that, in many fields, AI is better than humans at evaluation.
To find avenues to consider, I'm not entirely convinced that human innovation is more than a heuristic that appears more chaotic by virtue of a inconsistent and opaque formulation.
Many aspects of ideas com from noting how some two things are different and then considering that axis of difference when applied to another thing.
The possibilities thrown up by this extremely simple method are vast enough to require multiple layers of evaluation, most could be dismissed out of hand by a quick 'This is nonsense' check that I suspect people do so often and at a rate that it wouldn't even rise to the level of consciousness.
Yes, the guy with a PhD in Machine Intelligence, co-author of Reinforcement Learning: An Introduction, which is universally considered the bible of the field, recipient of the AAAI fellowship award and the Turing Award, and the inventor of Temporal Difference Learning doesn't know what he's talking about.
I don't completely disagree but its worth noting how new a lot of the empirical evidence in favour of LLMs are, so its not impossible to be a tad ignorant of the present
"If an elderly but distinguished scientist says that something is possible, he is almost certainly right; but if he says that it is impossible, he is very probably wrong." Arthur C. Clark
Should we automate exercise and play as well? How about learning?
The machine didn't have a soul, so we donated ours.
Eureka! My AI found it!
If it's a), he doesn't propose such an algorithm, and I don't know how you'd do it at such a low level because how do you quantify abstract goals? Did he suggest such an algorithm and I misread? If it's b), that already exists, see AlphaEvolve or any number of things he said. Or, to be a bit of a smart-ass, just type /goal and let it rip ...
I also think he's just categorically wrong that LLMs cannot do good and novel things. And if it can, then you could just say "well that's not novel, that's derivative". A simple example, if I make up a programming language with an LLM and it works well for my purposes, then is that not novel and good? I mean, is any language other than FORTRAN not novel?
Everything is derivative and you can put an LLM in a loop to evaluate LLMs trying things. I must be misunderstanding because he's too smart to be this wrong.
AlphaGo uses discovery when it evaluates potential moves and iterates.
Claude Code uses discovery when it generates a script and the evaluates whether it works or not.
He’s saying we need to allow ai systems to do the evaluation and iteration themselves for science and engineering the same way we do for code.
Basically, harness engineering for engineering.
https://youtu.be/ThFq87Rp21s?si=SrKj72_X8bjnB6ED
Around 35min mark
I don't think I would attribute anything in that process that I would consider an AI to be incapable of.
The characterisation of variation like this would seem to rest on the same 'random but directed' crutch that some free will arguments rest upon.
There is no random but directed of course, there is random and there is caused, and there are things that use both as components, but the random remains wholly random, and the caused remains entirely deterministic.
I think there is a good case to say that, in many fields, AI is better than humans at evaluation.
To find avenues to consider, I'm not entirely convinced that human innovation is more than a heuristic that appears more chaotic by virtue of a inconsistent and opaque formulation.
Many aspects of ideas com from noting how some two things are different and then considering that axis of difference when applied to another thing.
The possibilities thrown up by this extremely simple method are vast enough to require multiple layers of evaluation, most could be dismissed out of hand by a quick 'This is nonsense' check that I suspect people do so often and at a rate that it wouldn't even rise to the level of consciousness.
Best thing about nerds is watching them try and build frameworks and formulas for the creative act. Like a metronome trying to compose a symphony.
That contradiction kind of says he doesn't know what he's talking about.