← Back to videos

【BRUTAL】Elon Musk: "People who only ask questions to AI are going to get completely left behind,

Jyoti

Jyoti

@ai_with_jyoti

November 19, 2026· 47:25

【BRUTAL】Elon Musk: "People who only ask questions to AI are going to get completely left behind," his statement is insane hahaha Employee: "What's the right way to use AI?" Elon: "Most people use it wrong." Employee: "Huh?" Elon: "You ask a question, read the answer, and that's it, right?" Employee: "Isn't that how it's normally done?" Elon: "That's wasting it." Employee: "So what do you have to do?" Elon: "Give the AI 50 subordinates." Employee: "What? hahaha" Elon: "First, an AI Chief of Staff plans the day." Employee: "Okay." Elon: "Then, an AI project manager assigns the work." Employee: "I see." Elon: "And more than 50 AI agents do the work." Employee: "50?! hahaha" Elon: "And on top of that, they can work 24 hours a day." Employee: "But isn't that like a human company?" Elon: "Exactly. AI manages AI." Employee: "So what do humans do then?" Elon: "They set the goals and make the important decisions." Employee: "……" Until now, AI was a human asking a question and getting an answer. But with AI agents, you can divide the work, distribute it among multiple AIs, and even set up a system to review the results. Of course, human supervision and quality control are still needed. Even so, we've moved from the era where AI was just a simple assistant to the era where you direct an entire team of AIs. From now on, those who know how to delegate work to AI will earn more than those who know how to use it.

Transcript

It's, Grok 4 is, is smarter than almost all graduate students
0:07
in all disciplines simultaneously
0:10
Like, it's actually just important to appreciate the
0:14
like, that's, really something
0:19
and, the
0:21
the reasoning capabilities of Grok are incredible
0:25
so there's some people out there who
0:26
who think AI can't reason
0:28
and it, it, it can reason at superhuman levels
0:33
so, yeah, and frankly, it
0:35
it only gets better from here
0:38
So we'll, we'll take you through
0:41
the Grok 4 release
0:43
and
0:45
yeah
0:47
share you back the
0:48
the pace of, pace of progress here
0:52
like, I guess the first part is like
0:54
in terms of the training
0:56
we're going from Grok 2 to Grok 3
1:00
to Grok 4
1:00
we've essentially increased the
1:02
training by an order of magnitude in each case
1:05
So it's, you know, a hundred times more training than
1:08
than Grok 2
1:11
and, and that, that's only gonna increase
1:15
so it's, yeah, frankly, I mean, I don't know, in some ways a little terrifying
1:20
but
1:23
the, the growth of intelligence here is
1:24
is remarkable. Yes
1:27
It's important to realize there are two types
1:29
of training compute. One is the pre-training compute
1:31
that's from Grok 2 to Grok 3
1:34
but for, from Grok 3 to Grok 4
1:37
we're actually putting a lot of compute in reasoning
1:40
in RL
1:42
Yeah, and, just like you said
1:44
this is literally the fastest moving field
1:46
and Grok 2 is like the high school
1:48
student by today's standard
1:49
If you look back in the last twelve months
1:52
Grok 2 was only a concept
1:54
Within even have Grokk2
1:56
twelve months ago
1:58
and then by training Grokk2
1:59
that was the first time we scale up
2:00
like the pre-training
2:02
We realized that
2:03
if you actually do the data ablation really carefully
2:06
and the infra, and also the algorithm
2:08
we can actually push the pre-training
2:10
quite a lot by amount of 10x
2:13
to make the model
2:14
the best pre-trained based model
2:16
And that's where we build classes
2:18
the world's supercomputer with hundred thousand H100
2:22
And then with the best pre-trained model
2:25
and we realize if you can collect these verifiable outcome reward
2:29
you can actually train this model to start
2:30
thinking from the first principle
2:31
start to reason, correct its own mistakes
2:33
and that's where the Grokk2 reasoning comes from
2:36
And today we ask the question
2:38
what happens
2:39
if you take the expansion of the classes
2:41
with all two hundred thousand GPUs
2:43
put all these into IL
2:45
10x more compute than
2:48
any of the models out there on reinforcement learning
2:51
on precedent scale, what's gonna happen?
2:53
So this is the story of
2:56
Grok4 and, you know, Tony, share some
3:00
insight with the audience
3:01
Yeah, so yeah
3:03
let's just talk about how smart Grok4 is
3:06
So I guess
3:07
we can start discussing this benchmark called Humanity's Last Exam
3:11
and this, this benchmark is a very
3:14
very challenging benchmark
3:16
Every single problem is curated by
3:19
subject matter experts
3:22
it's in total 2,500 problems
3:25
and it consists of many different subjects
3:27
mathematics, natural sciences, engineering, and also all the humanity subjects
3:32
So, essentially when, when it was first released
3:37
actually like earlier this year
3:39
most of the models out there
3:41
can only get single digit accuracy on this benchmark
3:46
Yeah, so we can
3:47
we can look at some of those examples
3:49
you know, so there, there is this mathematical problem
3:54
which is about natural transformations in category theory
3:58
and there's this organic chemistry problem that talks about
4:02
electro, cyclic reactions
4:05
and also there's this linguistic problem
4:07
that tries to ask you about
4:09
distinguishing between closed and open syllabus
4:12
from a Hebrew, Hebrew source text
4:15
So you can see also
4:17
it's a very wide range of problems
4:20
and every single problem is PhD
4:23
or even advanced research level problems
4:26
Yes, I mean, these
4:28
there are no humans that can actually answer these
4:30
can get a good score
4:31
I mean, if you actually, so like any given human
4:35
what
4:36
like what's the best that any human could score?
4:39
I mean, I'd say maybe five percent
4:42
optimistically
4:45
So this, this is much harder than
4:47
than what any, any human can do
4:49
It's, it's incredibly difficult
4:51
And you can see from the
4:52
the types of questions
4:53
like you might be
4:54
incredible in linguistics
4:56
or mathematics or chemistry or physics or
4:58
any one of a number of subjects
4:59
but you're not going to be
5:01
at a postgrad level in everything
5:04
And Grok four is a postgrad level in everything
5:08
Like it's, it just
5:09
some of these things are just worth repeating
5:11
Like Grok four is postgraduate
5:15
like PhD level in everything
5:18
Better than Ph, like most PhDs would fail
5:21
So it's better that said
5:22
I mean, at least with respect to academic questions
5:25
it
5:25
I want us to just emphasize this point
5:28
With respect to academic questions
5:30
Grok four is better than PhD level in every subject
5:34
no exceptions
5:37
now this doesn't mean that it's
5:40
You know, times it may lack common sense
5:42
and it has not yet invented
5:44
new technologies or discovered new physics
5:49
but that is just a matter of time
5:51
if it, I, I think it may discover new technologies
5:56
as soon as later this year
6:00
and I
6:00
I would be shocked if it has not
6:01
done so next year
6:03
So I would expect Grok to
6:05
yeah, literally discover new
6:07
new technologies that are actually useful no later
6:09
than next year and maybe end of this
6:10
year
6:12
and it might discover new physics next year
6:15
and within two years
6:16
I'd say almost certainly
6:19
Like, so just let that sink in
6:25
Yeah
6:29
So, yeah, how, okay
6:32
so I guess we can talk about the
6:34
the, what
6:34
what's behind the scene of Route 4
6:36
As Jimmy mentioned
6:38
we actually throwing a lot of compute
6:41
into this training
6:43
You know, when it started, it's only also a single digit
6:46
sorry, the previous slide, sorry
6:49
Yeah, it's only a s- s- single digit
6:52
number
6:54
But as you start putting in more and more training compute
6:57
it started to gradually become smarter and smarter
7:01
and eventually solved
7:03
a quarter of the HRE problems
7:06
And this is without any tools
7:08
The next thing we did was to adding
7:11
tools capabilities to the model
7:14
And unlike Grok 3
7:16
I think
7:17
Grok 3 actually is able to use tool as well
7:19
but
7:19
here we actually make it more native in
7:21
the sense that
7:22
we put the tools into training
7:25
grocery was only relying on generalization
7:28
Here, we actually put the tools into training
7:30
and it turns out
7:31
this significantly improves the model's capability of using those tools
7:36
Yeah, I remember we had like deep search
7:39
back in days. So how is this different?
7:42
Yeah, exactly. So deep search was exactly the grocery reasoning model
7:47
but without any specific training
7:49
but we only
7:50
asked it to use those tools
7:52
So compared to this
7:54
it was much weaker
7:55
in terms of its tool capabilities
7:57
And unreliable
7:59
And unreliable, yes, yes. And to be clear
8:01
like, these are still, I'd say, fairly
8:03
this is still a fairly primitive tool use
8:06
If you compare it to
8:07
say, the tools that are used at Tesla or SpaceX
8:10
where you're using, you know
8:13
finite element analysis and computational flow dynamics
8:17
and, and you're You're able to run
8:20
or say like Tesla does like
8:22
crash simulations where the simulations are so close
8:25
to reality that if the test doesn't match the simulation
8:28
you assume that the test article is wrong
8:31
That's how good the simulations are
8:32
So Grok is not currently using
8:34
any of the tools
8:35
the, the really powerful tools that a company would use
8:38
but
8:39
but that is something that we will
8:41
provide her with later this year
8:43
So we'll have the tools that
8:45
that a company has
8:48
and, and have very accurate physics simulator
8:51
ultimately the
8:53
the thing that'll make the biggest difference is
8:54
being able to
8:56
interact with the real world via humanoid robots
8:58
So we combine sort of Grok with
9:00
with Optimus
9:00
and it can actually
9:02
interact with the real world and figure out if
9:04
if it's hypo, if it has
9:06
if it's
9:07
formulate an hypothesis and then confirm if that hypothesis is
9:12
is true or not
9:14
so we're really, You know
9:17
I think about like where we are today
9:19
We're
9:19
we're at the beginning of an immense intelligence explosion
9:23
We're in
9:24
we're in the ex-intelligence big bang right now
9:29
and the mo-
9:30
we're at the most interesting time
9:32
to be alive of any time in history
9:37
Yeah
9:38
Now, that said
9:39
we need to make sure that the AI is
9:42
a good AI
9:45
good Grok
9:46
and the
9:48
the thing that I think is most important for AI safety
9:50
at least my biological neural net tells me
9:53
the most important thing for
9:55
AI is to be maximally truth-seeking
9:58
So this is, this is a very fundamental
10:02
that you can think of AI as
10:04
this super genius child that ultimately will outsmart you
10:08
but you can still in
10:10
you can instill the right values
10:13
and encourage it to be sort of You know
10:19
truthful, I don't know, honorable, you know, good, good things, like
10:28
the values you'd want to instill in
10:30
in a child that
10:32
that, that grow, that would grow up
10:34
ultimately grow up to be incredibly powerful
10:41
yeah
10:43
So, yeah, so these, this is really, I say
10:46
we, we say, we say tools
10:47
These are still primitive tools
10:50
not the kind of tools that
10:52
that, serious commercial companies use
10:56
But we, we will provide it with those tools
10:58
and, I think it will be able to solve
11:00
with those tools, real world technology problems
11:02
In fact, I'm, I'm certain of it
11:03
It's just a question of how long it takes
11:06
Yes
11:07
Yes, exactly
11:09
so is it just compute all you need
11:11
Tony?
11:13
Is it just compute all you need at this point?
11:16
Well, you need compute plus
11:18
plus, the right tools, and
11:22
and then ultimately to be able to interact
11:24
with the physical world
11:27
and then
11:30
I mean, we'll effectively have an economy that is
11:34
well
11:34
ultimately an economy
11:36
that is thousands of times bigger than our current economy
11:39
or maybe millions of times
11:42
I mean, if you
11:42
if you think of civilization as
11:45
percentage completion of the Kardashev scale
11:48
where
11:49
Kardashev one is using all the energy output of a planet
11:53
and Kardashev two is
11:55
using all the energy output of a sun
11:57
and three is all the energy output of a galaxy
12:00
we're, we're only, in my opinion, probably like, closer
12:04
closer to one percent of Kardashev one
12:08
than we are to ten percent
12:10
So, like maybe a point
12:13
one, one or two percent
12:14
Of Kardashev One. So
12:18
we, we will get to
12:22
most of the way
12:23
like eighty, ninety percent of Kardashev One
12:25
and then hopefully, if civilization doesn't self-annihilate
12:29
and then Kardashev Two
12:32
like it's, the, the actual notion of
12:34
of a human economy
12:35
assuming civilization continues to progress
12:37
will seem very quaint
12:40
in, in retrospect
12:42
it will, it will seem like
12:45
sort of cavemen throwing sticks into a fire
12:49
level of economy
12:51
compared to what the future will hold
12:55
I mean, it's very exciting
12:56
I mean, I, I, I've
12:57
I've been at times kind of worried about like
13:00
well
13:02
you know, is this, this seems like
13:05
it's
13:05
it's somewhat unnerving to have intelligence
13:08
created that is far greater than our own
13:13
And
13:14
will this be bad or good for humanity?
13:18
it's like I, I
13:20
I think it'll be good
13:21
Most likely it'll be good
13:25
yeah
13:28
Yeah
13:29
But I somewhat reconciled myself to
13:31
the fact that e- even if I
13:34
even if it wasn't gonna be good
13:36
I'd at least like to be alive to
13:37
see it happen
13:40
So, yeah
13:44
So, actually, one, one, yeah
13:47
Yeah. I think one
13:48
one technical problem that we still need to solve
13:51
besides just compute, is how do we unblock the data
13:55
data, bottleneck?
13:57
because, when we try to scale up the RLL
14:01
in this case, we did invent a lot of new techniques
14:05
innovations
14:06
to
14:07
allow us to figure out how to find a lot of
14:10
a lot of challenging
14:11
RLL problems to work on
14:13
It's not just the problem itself needs to be challenging
14:15
but also it needs to be
14:18
you, you also need to have
14:19
like, a reliable, signal to tell the model
14:22
you did it wrong
14:23
you did it right
14:24
This is the sort of the principle of
14:25
reinforcement learning
14:26
And As the models get smarter and smarter
14:30
the number of
14:31
cool problem or challenging problems will be less
14:33
and less
14:35
So
14:36
it's gonna be
14:36
a new
14:37
type of challenge that we need to surpass besides just compute
14:40
yeah
14:41
Yeah, we, we actually are running out of
14:44
of actual test questions to ask
14:46
so there's like even ridiculously
14:49
questions that are ridiculously hard
14:50
if not essentially impossible for humans that are written down questions
14:56
are, becoming, swiftly becoming trivial for
15:00
for AI
15:02
so then there's
15:05
but then, you know, the, the one thing that is
15:07
an excellent judge of things is reality
15:10
So, cause physics is the law
15:13
ultimately, everything else is recommendation
15:14
You can't break physics
15:17
so the, the ultimate test, I think
15:18
for whether an AI is
15:21
the, the ultimate reasoning test is reality
15:24
So you invent a new technology
15:26
like say improve the design of a
15:28
car or a rocket or
15:31
create a new medication
15:34
that, and, and, and does it work?
15:37
does, does the rocket get to orbit?
15:39
Does the, does the car drive?
15:41
Does the medicine work? Whatever the case may be
15:45
reality is the ultimate judge here
15:48
so it's
15:48
it's going to be a reinforcement learning closing
15:50
loop around reality
15:56
We asked the question
15:57
how do we even go further?
15:58
So, actually, we are thinking about now
16:03
with single agent
16:04
we're able to solve forty percent of the
16:05
problem
16:07
What if we have multiple agents
16:09
running at the same time?
16:11
So this is what's called test and compute
16:14
And as we scale up the test and compute
16:16
actually
16:17
we are able to solve almost
16:19
more than fifty percent of the
16:21
tax-only subset of the HRA problems
16:24
So it's a remarkable achi-achievement
16:26
I think. You know
16:27
this is, this is, this is insanely difficult
16:30
These are, it's, it's
16:31
so what we're saying is like a majority of the
16:33
of the, of the text-based, of humanities, you know, scarily named humanities last exam
16:41
Grok four can solve
16:42
and you
16:43
and you can try it out for yourself
16:45
and the, with the Grok four heavy
16:47
what, what it does is it spawns multiple agents in parallel
16:50
and, all of those agents do
16:53
do work independently, and then they compare their work
16:56
and they, they decide which one
16:59
like it's like a study group
17:02
and it's, it's not as simple as
17:03
majority vote, because
17:05
often only one of the agents
17:07
actually figures out the trick or figures out the solution
17:12
and, and, and, but once they share the
17:14
the trick or, or, or figure out what
17:16
what the real nature of the problem is
17:18
they share that, solution with the other agents
17:22
and then they compare
17:22
they essentially compare notes and then
17:25
and, and then yield a
17:27
yield an answer
17:28
So that's, that's the, the heavy part of Grokful is
17:31
is where we
17:32
you scale up
17:33
the test time compute by roughly an order of magnitude
17:36
have multiple agents, tackle the task
17:40
and then they compare their work
17:42
and they
17:43
they
17:44
put forward what they think is the best
17:45
result
17:47
Yeah
17:48
So We will introduce Guafo and Guafo Happy
17:51
Sorry, can you click the next slide?
17:53
yeah
17:54
Yes
17:55
So, yeah, so basically Guafo is a single version
17:59
a single agent version
18:00
and Guafo Happy is the multi-agent version
18:05
So
18:05
let's
18:06
take a look how they actually do
18:08
on those exam problems and also some real
18:10
real life problems. Yeah
18:12
So
18:13
we're going to start out here and we're
18:14
actually going to look at
18:15
one of those HLE problems
18:17
This is
18:18
actually one of the easier math ones
18:20
I don't really understand it very well
18:22
I'm not that smart
18:23
but
18:23
I can launch this job here and we
18:25
can actually see how it's going to go
18:27
through and start to think about this problem
18:29
while we're doing that
18:30
I also want to show a little bit
18:31
more about like what this model can do and launch a
18:34
Grok four heavy as well
18:36
So everyone knows Polymarket
18:39
it's extremely interesting. It's the
18:40
you know, C curve truth
18:42
It aligns with what
18:43
reality is most of the time
18:45
And with Grok
18:46
what we're actually looking at is being able
18:48
to see how we can try to take
18:50
these markets and see if we can predict the
18:53
the future as well
18:54
So as we're letting this run
18:56
we'll see how, Grok four heavy goes about
18:59
predicting the, you know
19:01
the world series odds for like the current
19:03
teams
19:04
And they'll be
19:05
And while we're waiting for these to process
19:07
we're going to pass it over to Eric
19:08
and he's going to show you an example
19:10
of his
19:14
Yeah
19:15
So, I guess one of the coolest things about
19:19
Grok4 is its ability to
19:22
understand the world and to solve hard problems by leveraging tools
19:26
like Tony discussed
19:27
And I think one kind of cool example of this
19:31
we asked it to generate a visualization
19:33
of two black holes colliding
19:37
and of course, you know, it took
19:39
some, there are some liberties
19:41
it's in my case actually pretty clear in
19:43
its thinking trace about what these liberties are
19:46
for example, in order for it to actually be visible
19:48
you need to really exaggerate the
19:50
the scale of the
19:51
you know, the, the, the waves
19:57
And yeah, so here's like, you know
20:00
this kind of inaction
20:03
it exaggerates the scale in like multiple ways
20:06
it drops off a bit less in terms of amplitude
20:10
a over distance, and, but yeah, we can kind of see
20:16
the basic effects that
20:17
you know, are actually like, you know, correct
20:20
It starts with the inspiral
20:22
it merges, and then you have
20:25
the ring down, and like this is Basically
20:30
largely correct
20:33
yeah, modulo some of the simplifications that need to do
20:37
you know, it's, it's actually quite explicit about this
20:40
you know, it uses like post
20:42
post-Newtonian approximations instead of actually like
20:45
computing the general relativistic effects of like
20:49
near the center of the black hole
20:50
which is, you know, incorrect, and, you know, will lead to, you know, some incorrect results, but the overall, you know, visualization is
20:59
yeah, is basically there
21:02
and you can actually look
21:03
at the kinds of resources that are references
21:06
So here it, it actually, you know, it obviously uses search
21:11
it gathers results from a bunch of links
21:13
but also reads through a undergraduate text in analytical
21:19
analytic gravitational wave models
21:22
it's yeah, it, reasons quite a bit about
21:29
the actual concepts that they should use
21:31
for a realistic simulation
21:34
it references, I guess, existing real world data
21:38
and yeah, I, it, yeah, it's a
21:42
it's a pretty good model
21:45
yeah
21:46
Yeah. But like, actually going forward, we can
21:48
we can, we can plug
21:49
we can give it the
21:50
same model that physicists use
21:52
so it, it can run the
21:53
the same level of compute that
21:56
so leading physics researchers are using and
21:59
and give you a physics accurate
22:01
black hole simulation. Exactly
22:04
Just right now is running a new browser
22:05
so
22:06
Yeah, this is just running in your browser
22:07
Yeah, exactly. Pretty simple
22:09
So swapping back real quick here
22:11
we can actually take a look
22:12
The math problem is finished
22:14
the model was able to
22:16
let's look at its thinking trace here
22:19
so you can see how it went through
22:20
the problem
22:21
I'll be honest with you guys
22:22
I really don't quite fully understand the math
22:25
but what I do know is that I
22:26
looked at the answer ahead of time
22:29
and it did come to the
22:30
the correct answer here in the final part
22:32
here
22:33
we can also come in and actually take
22:35
a look here at our
22:37
our World Series prediction
22:39
and it's still thinking through on this one
22:41
but we can actually try some other stuff
22:43
as well. So we can actually
22:44
like
22:45
try some of the
22:46
X integrations that we did
22:47
So we worked very heavily on working with
22:50
all of our
22:50
X tools and building out a really great
22:52
X experience
22:54
So we can actually ask
22:55
you know, the model, you know
22:57
find me the XAI employee that has the
22:58
weirdest profile photo
23:01
so that's gonna go off and start that
23:02
and then we can actually try out
23:04
you know, let's, create a timeline based on X post
23:08
detailing the, you know, changes in the scores over time
23:11
and we can see
23:11
you know
23:12
all the conversation that was taking place at
23:14
that time as well
23:15
so we can see who are the
23:16
you know, announcing scores and, like
23:18
what was the reactions at those times as
23:20
well
23:21
so we'll let that go through here and process
23:24
and if we go back to
23:27
this was the, Greg Yang photo here
23:30
so, if we scroll through here
23:32
whoops, so Greg Yang, of course, who has his favorite
23:37
photograph that he has on his account
23:40
that's actually not how he looks like in real life
23:41
by the way, just to hit where
23:44
but it is quite funny
23:45
But it had to understand that question
23:46
Yeah, yeah. Which is the
23:48
that's the wild part
23:48
is like, it understands what is a weird photo
23:52
what is a weird photo
23:53
Yeah
23:53
What is a less or more weird photo
23:56
It goes through, it has to find all the team members
23:58
it has to figure out who we all are
24:00
it, you know, searches X
24:01
Without access to the internal XAI personnel logs
24:04
it's literally looking at the
24:05
just at the internet. Exactly
24:07
So you could say like weirdest of any
24:09
company. Yeah
24:10
To be clear
24:11
Exactly
24:12
And, we can also take a look here at the
24:16
question here for the humanities last exam
24:18
So it is still researching all of the historical scores
24:22
but it will have that final answer here
24:23
soon. But we can
24:24
while it's finishing up
24:25
we can take a look at one of
24:26
the ones that we set up here a second ago
24:29
and we can see like
24:30
you know
24:30
it finds the date that like Dan Hendricks
24:32
had initially announced it
24:33
We can go through
24:34
we can see, you know, OpenAI announcing their score back in
24:38
February, and we can see
24:40
you know, as progress happens with like Gemini
24:42
we can see like Kimmy
24:44
and we can also even see
24:45
you know, the leaked benchmarks of
24:48
what people are saying is
24:49
you know, if it's right
24:49
it's going to be pretty impressive
24:51
So, pretty cool
24:54
so yeah
24:54
I'm looking forward to seeing how everybody uses
24:56
these tools and gets the most value out of them
24:58
but, yeah, it's been great
25:01
Yeah, and we're, we're
25:02
we're gonna close the loop around usefulness as well
25:04
so it's like, it's not just
25:06
book smart, but actually practically smart
25:09
Exactly
25:18
And we can go back to the
25:19
slides here, yeah, so
25:21
Cool
25:23
so the
25:24
so we actually evaluate
25:26
also on the multimodal subset
25:28
So on the full set
25:30
this is the number on the HLE exam
25:33
it
25:34
you can see there is a little dip
25:36
on the numbers
25:37
this is actually something we're improving on
25:39
which is the multimodal
25:41
understanding capabilities
25:42
But I do believe
25:44
in a very short time
25:45
we're able to
25:47
really improve and got much higher numbers on this
25:51
even higher numbers on this benchmark here
25:54
Yeah, this is the, we, we saw like, so like, what
25:57
what is the biggest weakness of Grok currently is that it's
26:00
it's sort of partially blind
26:02
It can't, its, its image understanding, obviously, and its image generation
26:08
need to be a lot better
26:09
and that, that, that's actually, being trained right now
26:14
So
26:16
Grok
26:17
Grok four is based on version six of our foundation model
26:20
and we are training version seven
26:23
which we'll complete in a few weeks
26:26
and, that, that'll address the
26:29
weakness on the vision side
26:31
Just to show off this last here
26:33
So the, the prediction market finished
26:36
here with a heavy
26:37
and we can see
26:39
here
26:39
we can
26:39
see all the tools and the process it used to actually
26:42
go through and find the right answer
26:45
So it browsed a lot of odd sites
26:47
it calculated its own odds comparing to the market
26:50
the market to find its own alpha and
26:51
edge
26:52
It walks you through the entire process here
26:54
and it calculates the odds of the winner being like the
26:58
the Dodgers, and it gives them a 21.6% chance of
27:04
winning, this year
27:06
So, and it took a process
27:08
approximately four and a half minutes to compute
27:11
Yeah, that's a lot of thinking
27:12
Yeah
27:19
We can also look at all
27:21
the other benchmarks besides HIE
27:24
As it turned out
27:25
Grok4
27:26
excelled on all the reasoning benchmarks that people usually test on
27:31
including GBQA, which is a PhD level
27:35
problem sets
27:36
that's easier compared to HLE
27:40
on AIME 25, American Invitation Mathematics exam
27:44
we, with Grok4 heavy
27:46
we actually got a perfect score
27:49
also on some of the coding benchmark called Live Coding Bunch
27:53
and also on, HLMTcovered math, MIT
27:58
exam and also USAMO
28:00
you can see actually
28:02
on all of those benchmarks
28:05
we often have a very large lead against the second best
28:09
model out there
28:12
Yes
28:12
I mean, really, we're gonna get to the point where
28:15
it's gonna get
28:16
every answer right in every exam
28:19
and where it doesn't get an answer right
28:20
it's going to tell you what's wrong with
28:21
the question
28:22
Or if the question
28:24
is ambiguous, disambiguate the question into answers A
28:27
B, and C, and tell you what
28:29
what answers A, B
28:30
and C would be with a disambiguated question
28:33
so the only real test then will be
28:35
reality
28:36
can it make useful technologies
28:39
discover, new science?
28:42
That'll actually be the
28:43
the only thing left
28:44
because human tests will simply not be
28:47
meaningful
28:49
Well, we need to make a update to HR very soon
28:52
given the current rate of progress
28:54
So yeah, it's super cool to see
28:55
like
28:56
multiple agents that collaborate with each other solving
28:58
really challenging problems
29:00
so we're going to try this model
29:02
so it turned out it's available right now
29:05
if we advance to the next slide
29:08
where there is a super Grok heavy tiers that we're introducing
29:13
where you're able to access to both Grok
29:15
4 and Grok 4 Heavy
29:16
where you're actually gonna be the task master
29:18
of a bunch of little Grok research agent to help you
29:21
you know, become smarter, do all the little research
29:23
and save hours of times of
29:25
you know, going through mundane tasks
29:28
and it's available right now
29:30
so, yeah
29:31
So, We, we, we did limit usage during the
29:36
the demo, so we didn't, it didn't break the demo
29:38
because all these, this, all this stuff is happening live
29:40
so it's not, there's nothing canned about the
29:43
any of the tests that we're doing
29:45
so, after the, after the, demo is done, we'll, we'll allow
29:49
we'll enable more, subscribers for SuperGrok
29:53
So if you can't subscribe right now
29:54
just try and half an hour
29:55
it should work
29:58
so, and now let's, let's
30:00
let's get into voice
30:03
Great
30:05
So many of you have been enjoying our voice mode
30:07
and we've been working hard to improve the
30:09
experience over the past couple months
30:12
we have cut latency in half to make it much snappier
30:16
and today we're excited to announce a set
30:18
of new voices
30:19
that have exceptional naturalness and prosody
30:23
You might have noticed the
30:25
movie trailer voice that opened up the live
30:27
stream. That is one of our new voices
30:29
Sal, who's got That epically deep tone
30:33
and we're also excited to introduce Eve
30:36
a beautiful British voice who's capable of rich emotions
30:41
Aman, would you like to introduce Eve?
30:43
Absolutely. Yeah, let's get into the demo
30:48
Hey Eve
30:49
millions of people are watching us right now
30:51
Are you excited?
30:53
Oh, I'm thrilled, really
30:55
It's like being on stage at the Old
30:57
Vic with a sea of faces out there
30:59
Just imagine all those eyes and ears tuned
31:02
in
31:02
It's a proper buzz
31:03
isn't it?
31:04
What do you reckon we should chat about
31:06
to keep this lot entertained?
31:08
You know, we have a lot to talk about
31:09
but I'll be honest
31:10
I'm feeling a little nervous
31:11
Can you whisper something soothing to calm me
31:13
down?
31:15
Take a deep breath
31:17
love
31:18
You've got this
31:19
It's just you and me
31:21
having a quiet chat
31:23
like we're tucked away in a cozy corner
31:25
of a Yorkshire The world's just a murmur out there
31:30
feel that calm wash over you
31:32
Let's keep talking, nice and easy
31:35
What's on your mind?
31:38
You know what, I'm actually feeling quite relaxed to try this
31:41
something crazy. So, can you give us an opera on
31:44
Diet Coke?
31:46
Oh, an opera on Diet Coke
31:49
that's a riot
31:51
Picture this, a shimmering silver cannon
31:55
glistening under stage lights
31:58
Oh, Diet Coke, thou elixir divine
32:03
with bubbles that dance in a sparkling line
32:07
Thy crisp, cool kiss, on lips so fine
32:12
Oh, aspartame and brosia, forever mine
32:18
How's that for a mad little aria?
32:21
Want me to belt out another verse or
32:23
switch up the tune?
32:24
Okay
32:25
let's try switching it up and just start singing
32:29
Right then, let's have a go
32:32
Oh, silver can, you gleam in the light
32:35
diet coke, my heart's delight
32:37
Okay, I won't touch it
32:38
It's very, very poetic
32:41
yeah, thank you. Feeling quite nice now
32:44
Cool
32:45
so that's Eve. And
32:47
so something else we do here is
32:49
is a little test each day
32:51
to check on the natural flow
32:54
of our conversations against some of our competitors
32:58
So we figured why not try that live
33:00
on the stream today
33:04
Should I start?
33:08
Hey, I'm gonna say a number
33:10
and can you say the same number back
33:11
to me? One. One
33:12
got it
33:14
This is like a fun little game of
33:16
echo. What's next? Two. Sorry
33:22
you said two?
33:25
Three
33:29
Four
33:30
Nice and steady. What's next?
33:34
five. Five. Nailed it. This is pretty fun
33:39
actually
33:40
Feel free to keep going or switch it
33:42
up if you like
33:47
Hey, Grok. Hey, Grok
33:53
Hello there. How can I help you today?
33:55
We're gonna do a little test
33:56
Can you repeat after me?
33:57
One
33:59
One
34:00
All right. What's next? Two
34:03
Two
34:04
What's on your mind? Three
34:07
Three
34:08
Need anything else? Four
34:10
Four
34:11
How can I assist you?
34:12
Five
34:15
Five
34:16
What's next?
34:18
So as you can see
34:19
Grok was, snappier
34:21
It didn't interrupt me
34:22
and the prosody
34:23
is we have made different design choices
34:26
I think we were shooting for something more calm
34:27
smooth, more natural versus something that's more poppy or artificial
34:31
so we'll keep improving on these ones
34:33
Thanks, guys
34:34
Yeah. Yeah
34:40
So since the launch of the voice model
34:44
we actually see the 2x faster end-to-end latency
34:48
in the last 80 weeks
34:50
five different voices
34:51
and also 10x the active user
34:53
So Grok voice is taking off
34:55
Now, if you think about releasing the models
34:59
this time we're also releasing
35:01
Grok 4 through the API at the same
35:03
time
35:04
So if we go to the next two slides
35:07
So, you know, we're very excited about
35:10
you know, what all developers out there is gonna build
35:12
So, you know, if I think about myself as a developer
35:15
what the first thing I'm gonna do
35:16
when I actually have access to the GraphQL
35:18
API?
35:19
Benchmarks
35:20
So we actually asked around on the X platform
35:23
what is the most challenging benchmarks out there that
35:25
you know
35:26
is considered the holy grail
35:27
for all the AGI models?
35:29
so turn up, AGI is in the name
35:32
RKGI. So, the last twelve hours
35:36
you know
35:36
kudos to Greg over here in the audience
35:39
so who answered our call
35:41
take a preview of the GraphQL API and independently verified
35:45
you know, the GraphQL's performance
35:47
So initially we thought
35:48
hey, GraphQL is just, you know, we think it's pretty good
35:50
it's pretty smart, it's our next-gen reasoning model
35:53
spend 10x more compute
35:55
can use all the tools
35:56
right?
35:56
But turned out
35:58
when we actually verify
36:00
on the private subset of the RKGI v2
36:04
It was like
36:05
the
36:05
only model in the last three months that
36:07
breaks the ten percent barrier
36:08
and in fact was so good that actually
36:10
gave us sixteen percent
36:11
well, fifteen point eight percent
36:13
accuracy, two X of the second place
36:16
that is the Car4 Opus model
36:20
and it's not just about performance
36:22
right? When you think about intelligence
36:24
having the, an APM model drives your automation
36:27
it's also the intelligence per dollar
36:29
right? If you look at the plots over here
36:31
the Grok is just
36:32
four inches in the league of its own
36:35
all right, so enough of benchmarks
36:37
over here, right? So what can Grok do
36:39
actually, in the real world?
36:41
So, we actually, you know, contacted the folks from
36:47
Andon Labs
36:49
who, you know, were, you know, gracious enough to, you know
36:52
try to Grok in the real world to
36:53
run a business
36:54
Yeah, thanks for having us
36:56
So I'm Axel from Andon Labs
36:58
and I'm Lucas
36:58
and we tested Grok 4 on vending bench
37:01
Vending bench is an AI simulation of a business scenario
37:06
where
37:06
we thought what is the most simple business
37:08
an AI could possibly run
37:10
and we thought vending machines
37:12
so in this scenario
37:13
the, the Grok and other models need to do stuff like
37:18
manage inventory, contract, contact suppliers, set prices
37:22
All of these things are super easy
37:23
and all of, they, like
37:25
all the models can do them one by one
37:27
but when you do them over very long horizons
37:30
most models struggle
37:32
but we have a leaderboard
37:33
and there's a new number one
37:34
Yeah
37:35
so we got early access to the Grok
37:36
4 API
37:37
we ran it on the landing bench
37:39
and we saw some really impressive results
37:41
So it, ranks
37:43
definitely at the number one spot
37:45
It's even double the net worth
37:47
which is the measure that we have on
37:48
this event. So it's not about the percentage on a
37:52
or score you get
37:52
but it's more the dollar value in net
37:54
worth that you generate
37:56
So we were impressed
37:57
by Grok. It was able to formulate a
37:59
strategy
38:00
and adhere to that strategy over a long period of time
38:03
much longer than other models that we have tested
38:05
other frontier models
38:07
So it, managed to run the
38:09
simulation for double the time and score
38:11
yeah, double the net worth
38:12
And it was also really consistent
38:15
across these runs
38:15
which is something that's
38:17
really important when you
38:18
want to use this in the real world
38:20
And I think as we give more and
38:21
more power to AI systems in the real world
38:25
it's important that we test them in
38:27
scenarios that either mimic the real world or
38:29
are in the real world itself
38:32
Because otherwise we, we fly blind into some
38:35
some things that, that might not be great
38:38
Yeah, it's
38:39
it's great to see that we've now got
38:40
a way to pay for all those GPUs
38:42
We just, need a million vending machines
38:45
and, we could make, I don't know
38:47
four point seven billion dollars a year with
38:49
a million vending machines
38:51
That can be epic
38:52
vending machines
38:54
All right
38:55
We are actually going to install vending machines here
38:58
like a lot of them
38:59
We're happy to supply them
39:00
All right, thank you
39:02
All right, yeah
39:03
I'm looking forward to seeing what amazing
39:05
things are in this vending machine
39:07
That's, that's for, for you to decide
39:09
All right. Or tell the AI
39:11
Okay, sounds good
39:14
All right. Yeah. I mean
39:15
so
39:16
we can see like Grok is able to
39:17
become like the co-pilot of
39:19
the business unit. So
39:20
what else can Grok do?
39:21
So we're actually releasing this Grok
39:23
If you want to try it
39:24
right now to evaluate
39:25
run the same benchmark as us
39:27
it's on API, has 256K contact lens
39:33
So
39:33
we already actually see some of the really
39:35
early adopters to try Grok 4 API
39:38
So, our Palo Alto neighbor
39:40
ARC Institute, which is a leading
39:42
biomedical research
39:44
center, is already using, seeing like
39:46
how can they automate their research flows
39:49
with Grok 4?
39:50
it turned out it performs
39:52
it's able to help the scientists to sniff through
39:54
you know, millions of experiments logs
39:56
and then, you know
39:57
just like pick the best hypothesis
40:00
within a split of seconds
40:02
we see this as being used for their
40:03
like the CRISPR, research, and also
40:07
you know
40:07
Grok 4
40:08
independently evaluate scores as the best model
40:10
to examine the chest X-ray
40:13
who would know?
40:14
And, on the, in the financial sector
40:17
we also see, you know
40:18
the GraphQL with access to all the tools
40:20
real-time information is actually
40:22
one of the most popular AIs out there
40:24
So, you know
40:26
our GraphQL is also gonna be available on
40:28
the hyperscalers. So the xAI enterprise sector is only
40:33
you know, started two months ago
40:35
and we're open for business
40:39
Yeah, so, the other thing, we talked a lot about
40:42
you know, having Grok to make games
40:44
video games
40:45
so Danny is actually a
40:47
video game, designers on X
40:49
So, you know, we mentioned, hey
40:51
who want to try out some
40:53
Grok for, preview APIs, to make games
40:56
and Danny answered the call
40:58
so this was actually just made
41:00
first person shooting game
41:01
in the span of four hours
41:04
so, some of the, actually
41:06
the unappreciated hardest problem of making video games
41:09
is not necessarily encoding the core logic of the game
41:12
but actually go out
41:14
source all the assets
41:15
all the textures, files, and, and, you know, to create a
41:19
a visually appealing game
41:21
So
41:21
one of the core aspect Grok for does
41:23
really well with all the tools
41:25
there's actually able to automate these
41:27
like, asset sourcing
41:29
capabilities. So
41:30
the developers can just focus on the core development itself
41:34
rather than, like, you know, so now you can run a
41:36
you know, entire game studios with One
41:39
with, per, like one person, and then
41:42
you can have Grok four
41:43
to go out and source all those
41:45
like, assets to automate tasks for you
41:47
Yeah, the, now the next step
41:50
obviously, is for Grok to
41:53
play, be able to play the game
41:55
So it has to have very good video understanding
41:56
so it can play the games and interact
41:58
with the games and actually assess what
42:01
whether a game is fun
42:02
and, and
42:03
and actually have good judgment for
42:04
whether a game is fun or not
42:07
so with the, with version seven of our foundation model
42:10
which finishes training this month
42:12
and then we'll go through post-training RL and whatnot
42:16
that, that will have excellent video understanding
42:19
and with the, with the video understanding and the
42:22
and improved tool use
42:24
for example, for video, for video games, you'd want to use
42:27
you know, Unreal Engine or Unity or one of the
42:29
one of the, the main graphics engines
42:33
And then gen-generate the
42:35
generate the art, apply it to a 3D model
42:39
and then create an executable that someone can
42:41
run on a PC or
42:43
or a console or a
42:44
or a phone
42:47
like, we, we expect that to happen probably this year
42:52
and if not this year
42:53
certainly next year
42:55
so that's, it's gonna be wild
42:59
I
42:59
I would expect the first
43:01
really good
43:02
AI video game to be next year
43:08
and probably the first
43:11
half hour of watchable TV this year
43:16
and probably the first watchable AI movie next year
43:20
Like, things are really moving at
43:22
an incredible pace
43:24
Yeah, when Grok is connecting the world economy with vending machines
43:28
which is create video games for human
43:31
Yeah, I mean
43:31
it went from not being able to do any of this
43:34
really even six months ago to
43:37
to what you're seeing before you here
43:38
and, and from, from very primitive a year ago
43:43
to making a three
43:46
a sort of a 3D video game with
43:49
with a few hours of prompting
43:51
Yeah
43:53
I mean, yeah, just to recap, so in today's live stream
43:56
we introduced the most powerful
43:58
most intelligent
43:59
AI models
44:00
out there that can actually reason from the first principle
44:02
using all the tools
44:03
do all the research
44:04
go on a journey for ten minutes
44:05
come back with
44:06
the most correct answer for you
44:09
so it's kind of crazy to think about
44:12
just like four months ago
44:13
we had Grok 3
44:15
and now we already have Grok 4
44:16
And we're going to continue to accelerate as a company
44:18
XAI
44:18
we're going to be the fastest moving
44:20
AGI companies out there
44:22
So what's coming next is that we're going to
44:25
you know, continue developing the model that's not just
44:28
you know, intelligent, smart, think for a really long time
44:31
spend a lot of compute
44:33
but
44:33
having a model that actually both fast
44:36
and smart is going to be the core focus
44:38
right? So
44:39
if you think about what are the applications
44:40
out there that can really benefit from all those very intelligent
44:44
fast and smart models
44:45
then coding is actually one of them
44:48
Yeah
44:48
so the team is currently working very heavily
44:50
on coding models
44:52
I think
44:53
right now the main focus is
44:55
we actually trained recently a specialized coding model
44:58
which is going to be both fast and
45:00
smart
45:02
and
45:03
I believe we can share with that model with you guy
45:05
with all of you
45:06
in a few weeks
45:07
Yeah. Yeah
45:09
Yeah, that's very exciting. And
45:11
you know
45:11
the second after coding is
45:13
we all see the weakness of Grok 4 is
45:18
the multimodal capability. So in fact
45:21
it was so bad that
45:22
you know, Grok effectively just like looking at the world
45:25
squinting through the glass
45:26
and like see all the blurry
45:28
you know
45:29
features and trying to make sense of it
45:32
the most
45:33
immediate improvement we're gonna see
45:34
with the next generation pretty much model is
45:36
that we're gonna see a step function improvement
45:38
on the model's capability in terms of image understanding
45:40
video understanding, and audios, right?
45:43
It's now the model's able to
45:45
hear and see the world just like any of you
45:47
right? And now with all the tools at its command
45:51
with all the other agents it can talk to
45:52
you know
45:54
so we're gonna see a huge unlock
45:57
for many different application layers
46:00
after the multimodal agents
46:01
was gonna come after is the video generation
46:04
and I would believe that
46:06
you know, at the end of the day
46:07
it should just be
46:08
you know, pixel in, pixel out
46:11
and, you know
46:13
imagine a world where
46:15
you have this infinite scroll
46:17
of content in inventory on the X platform
46:20
where not only you can actually watch these
46:22
these generate videos, but able to intervene
46:26
create your own adventures
46:28
The future's gonna be wild
46:30
and we, we expect to be training our video model with
46:33
over a hundred thousand GB200s
46:36
and, to begin that training within the next
46:39
three or four weeks
46:41
So we're, we're confident it's gonna be pretty spectacular in
46:44
in video generation and video understanding
46:49
So, let's see
46:52
So that's, anything, anything you guys wanna say?
46:58
Other than that I guess that's it
47:01
yeah it's it's a good model sir
47:03
it's a good
47:04
yeah well we're very excited for you guys
47:06
to try Grok four yeah all right thanks
47:09
everyone thank you good night

Watch these videos on your TV

XTTV brings video content from X to your living room. Download now and start watching on the big screen.