← Back to videos
Sam Altman (CEO of Open AI): "You no longer need to write prompts." In just 38 minutes, he explain

Ram Singh Verma
@RamSingh_369
November 19, 2026· 26:31
Sam Altman (CEO of Open AI): "You no longer need to write prompts." In just 38 minutes, he explains how to use ChatGpt at a level that most people can't even imagine. It's talk he gave to stanford students. A friend sent me the recording last night. After watching it, I realized I was only taking advantage of about 15% of what this tool can really do. Watch it in full and then read the guide l leave below on how to create a system that prompts itself.
Transcript
- You, with like an affordable amount of spend on tokens
- 0:05
- you can do what a hundred person
- 0:07
- incredibly great engineering team would do as a startup
- 0:11
- and that was just totally impossible
- 0:12
- That was like not in the
- 0:14
- set of options for a startup
- 0:17
- and now it is. So
- 0:18
- so I think what you can take on
- 0:20
- the level of ambition you can have
- 0:21
- the speed at which you can move
- 0:23
- the amount of stuff you can do at once
- 0:25
- is just totally different
- 0:27
- And
- 0:27
- does that change the
- 0:28
- shape of the problems you feel like
- 0:30
- you'd
- 0:31
- assign at the end of the class for people to attack
- 0:34
- you know
- 0:34
- at the end of that
- 0:35
- quarter if you were teaching the game?
- 0:37
- I don't think assigning
- 0:39
- problems to attack ever works because if you
- 0:41
- like If I can think of a problem
- 0:43
- if I can think of like a really great startup idea
- 0:46
- if it's like obvious enough to me
- 0:48
- then
- 0:49
- it's probably obvious to a lot of people
- 0:51
- When we started OpenAI
- 0:52
- we were, we were like the
- 0:54
- you know
- 0:54
- one of maybe generously speaking four AGI efforts in the world
- 0:58
- right? And you want to find something like
- 1:00
- that. And I'm sure
- 1:01
- that there exists something today
- 1:03
- that just wasn't possible at all pre like automated coding era
- 1:08
- that is totally non-obvious that will be
- 1:10
- you know, a multi-trillion dollar market soon
- 1:13
- and that only four companies are working on
- 1:15
- right now
- 1:16
- But I don't know what that is
- 1:16
- It's much more likely
- 1:17
- you all know what that is than I
- 1:19
- know what that is. I just
- 1:20
- you know, my brain is like taking over my OpenAI
- 1:23
- but, you know
- 1:24
- the kind of idea someone can assign you
- 1:26
- to work on is Probably not what you want
- 1:28
- Yep
- 1:30
- okay, so that, that's fair
- 1:32
- but I, I think it would be helpful
- 1:33
- since this is a systems class
- 1:35
- to maybe
- 1:36
- reason about a particular problem that you have to reason through
- 1:39
- so that they can then apply
- 1:42
- the shape of the techniques used to break
- 1:43
- down from a systems perspective that problem into
- 1:45
- solutions to their own problem
- 1:47
- Yeah
- 1:48
- and a, and a concept that
- 1:50
- you had started to tease in the class
- 1:51
- you know, back in 2014, and then
- 1:54
- clearly you've talked about publicly over the years is
- 1:57
- scale
- 1:58
- Right? Scale is its own beast
- 2:00
- It's, it's, it's, you know, quantity is its own quality
- 2:03
- Whatever
- 2:03
- scale as a concept has been something it seems like you've
- 2:08
- empirically investigated in all kinds of ways over
- 2:10
- the last 10 years. So
- 2:12
- could you help on
- 2:13
- help us first unpack
- 2:14
- like, what you mean by scale now
- 2:16
- 10 years later? How would you deconstruct that
- 2:18
- as a systems design
- 2:20
- attribute to apply, whether it's a, as, as a tool?
- 2:23
- can, can we start there?
- 2:25
- Yes
- 2:27
- So I don't
- 2:28
- know why the following observation is true
- 2:32
- I offer no theory that I find satisfying to explain it
- 2:37
- and that makes me
- 2:38
- a little bit nervous to suggest you follow it
- 2:42
- but I'm going to anyway
- 2:43
- because empirically it does seem to be true
- 2:45
- which is
- 2:46
- all of the most interesting things I have
- 2:48
- observed in my career in watching other things happen
- 2:53
- all of the most interesting ones
- 2:56
- have had something to do with
- 2:59
- emergent properties that scale
- 3:00
- or
- 3:01
- scale continuing to provide returns far beyond what
- 3:04
- the consensus thinks will work
- 3:06
- And this obviously happens with like scaling laws for AI models
- 3:11
- but this happens with
- 3:14
- you know
- 3:14
- getting more smart people together to think about
- 3:16
- one problem in a research setting
- 3:19
- This happens with
- 3:22
- Companies and the sort of economy of scale
- 3:24
- you can get in all these different ways
- 3:26
- I really learned this at Y Combinator
- 3:28
- when it became clear to me that everybody was saying
- 3:32
- oh, Y Combinator's gotten too big
- 3:33
- it should shrink, we should fund less companies per batch
- 3:35
- you know
- 3:36
- the best times of Y Combinator
- 3:37
- when it was like 10 companies per batch
- 3:39
- And a lot of like very smart people
- 3:41
- were saying this
- 3:45
- And it was like tempting because it would
- 3:46
- have been like much less work
- 3:47
- And the theory was that
- 3:48
- you know
- 3:48
- the best companies
- 3:50
- are always kind of obvious and then you
- 3:51
- fund the rest and it's not as helpful
- 3:53
- But a huge part of the magic of
- 3:55
- what made YC work were
- 3:58
- was the sort of the network effects inside
- 4:00
- of the batch
- 4:01
- And that was an emergent property at scale
- 4:03
- that just hadn't been discovered before
- 4:04
- No one had tried to fund startups
- 4:07
- at scale in the same way
- 4:09
- And thus no one had ever
- 4:11
- happened upon this observation of when you do
- 4:13
- that
- 4:15
- There's
- 4:17
- There's something important that happens that just didn't
- 4:19
- exist at all at the one-tenth or one-one-hundredth
- 4:21
- of a scale
- 4:23
- There's a bunch of other examples like this
- 4:26
- I
- 4:30
- and I'll skip them in the interest of time
- 4:31
- but I, I would say, again
- 4:34
- I offer no explanation for why
- 4:35
- but empirically speaking
- 4:37
- when you find a time that you can push on
- 4:41
- you can push something to a scale people
- 4:42
- have not tried before
- 4:43
- and it's already working
- 4:45
- in some interesting way at the smaller scale
- 4:47
- more often than not
- 4:48
- that seems to be a good idea
- 4:50
- And it also seems to be
- 4:51
- something that
- 4:54
- most people don't do enough
- 4:56
- And I don't offer an explanation for this either
- 4:58
- but like in, you know, when we were like
- 5:01
- we're really going to scale AI models
- 5:04
- All of the like geniuses in the field
- 5:05
- most of them were
- 5:06
- oh, this isn't really working
- 5:07
- You know, that's barely a scientific result
- 5:09
- It's not interesting that it gets better at
- 5:10
- scale. You've already shown that
- 5:12
- Why keep scaling it?
- 5:13
- I mentioned the YC example
- 5:15
- I've seen a lot of startup founders where they're like
- 5:19
- well, you know
- 5:21
- there might be something interesting that would happen
- 5:22
- if I scaled this up
- 5:23
- but
- 5:24
- I'm a little worried about it for nonspecific reasons
- 5:27
- And again
- 5:27
- looking back at like a huge
- 5:30
- data set of people that have scaled
- 5:33
- their companies in all these different ways
- 5:34
- there's almost always interesting stuff there
- 5:36
- So I think directionally
- 5:37
- that's like an interesting thing to
- 5:40
- push on and severely underexplored
- 5:45
- On the systems design part of that
- 5:50
- I think one reason people
- 5:51
- don't do it as much
- 5:54
- is Stuff breaks
- 5:56
- at an accelerating rate and in an unpredictable
- 5:59
- way as you scale it
- 6:00
- And if you are going to really scale something
- 6:05
- it's always like a little bit broken
- 6:07
- There are always like very smart people who say
- 6:10
- why you shouldn't do this?
- 6:11
- You know, don't get too ambitious
- 6:12
- don't get too big
- 6:13
- let's try this smaller
- 6:14
- And
- 6:15
- so breaking that down is a systems problem
- 6:17
- I use the
- 6:18
- thing of when we were like scaling up AI models
- 6:20
- there was
- 6:21
- technically can we do this at all?
- 6:22
- This seems crazy. Like no one had ever
- 6:24
- thought about trying to do a run across
- 6:25
- 10,000 or 100,000 GPUs and that was going
- 6:27
- to require stacks of engineering talent
- 6:30
- There was the
- 6:31
- capital requirements and what was going to take
- 6:33
- to do this and like
- 6:34
- how is there ever going to be a
- 6:35
- business? How can you think about taking this
- 6:37
- risk?
- 6:38
- There was the sort of like cultural stuff of researchers saying
- 6:40
- well, if we're going to get all this compute
- 6:42
- why do we put it all into this
- 6:43
- one project? We're not going to learn something
- 6:44
- Why not divide it up among
- 6:46
- all these projects?
- 6:47
- And
- 6:47
- this also happens in kind of every area I've looked at
- 6:51
- almost every area for scale
- 6:52
- And breaking it down into the
- 6:55
- Sort of
- 6:56
- each difficult area
- 6:58
- or each reason not to do it and
- 6:59
- trying to address them one at a time
- 7:01
- Yeah. That's been really important
- 7:04
- I'm going to push on that a little
- 7:05
- bit because
- 7:07
- there's very few people who've been able to
- 7:08
- sort of
- 7:10
- repeatedly scale new products and
- 7:12
- systems the way
- 7:13
- the OpenAI team has over the years
- 7:15
- But it seems like one of the
- 7:17
- issues is
- 7:18
- there are all these prior conditioning
- 7:21
- sort of mental models and expectations humans have
- 7:24
- And you said things break
- 7:25
- And one of the things it seems often breaks
- 7:29
- the hardest to refactor is human
- 7:32
- the human side of the system's design
- 7:35
- right? Wherever there's human implementers or there's
- 7:38
- human participants in that. And so what have
- 7:40
- you learned about humans at scale
- 7:41
- like organizing humans at scale to participate in
- 7:43
- a system that may not be
- 7:46
- like
- 7:46
- just a redo of some past system that
- 7:48
- they get naively on at
- 7:51
- a priority on first blush?
- 7:55
- I think like clear
- 7:57
- a clear goal, a clear plan to get there
- 8:01
- and like a clear
- 8:06
- answer to the way that you're going to
- 8:08
- get there and kind of how you're going
- 8:09
- to make decisions along the way
- 8:10
- that's, that's very important. So
- 8:13
- you know
- 8:14
- if we go back to the example of
- 8:15
- when we decided to scale up models
- 8:17
- there were a lot of people who were like
- 8:18
- this isn't really going to work
- 8:19
- it's going to have these problems
- 8:21
- it's also not, you know, we need a more diversified portfolio
- 8:23
- but once we say no
- 8:24
- we're going to make a bet on scaling deep learning
- 8:26
- like that's our thing. If we're wrong
- 8:28
- we'll fail, but we're going to do that
- 8:30
- Here's why we're going to do that
- 8:31
- here's what we believe about what the state
- 8:32
- of the world can be like if we
- 8:33
- get there
- 8:35
- that's very powerful
- 8:37
- And then
- 8:40
- for whatever reason
- 8:41
- we did not
- 8:42
- evolve to be good at thinking about exponentials
- 8:47
- People have a hard time imagining that
- 8:50
- Scaling laws are going to continue exponentially
- 8:52
- that revenue will grow exponentially
- 8:54
- that an organization can take on exponential complexity
- 8:57
- And in my experience
- 8:59
- it takes a lot of time
- 9:01
- to really reason through first principles with people about why
- 9:04
- why that can happen
- 9:05
- Can we take two examples
- 9:07
- to walk through that?
- 9:09
- The first being ChatGPT and the second being Codex
- 9:11
- You know, both of these have transformed
- 9:14
- Can
- 9:14
- can everyone hear? I'm going to try to
- 9:15
- project it. Yeah. Okay
- 9:18
- so let
- 9:19
- let me put in
- 9:20
- a frame and you can challenge both the
- 9:21
- assumption and then
- 9:22
- we can hopefully reason through
- 9:23
- example of what happened
- 9:25
- In the case of ChatGPT
- 9:25
- you know, for a long time in scaling of models
- 9:28
- a big mental block that
- 9:30
- Seemed to be prevalent in the space is
- 9:32
- what
- 9:33
- what are these things going to be useful for?
- 9:35
- This is, you know, it's a research, sort of solution, solution chasing a problem
- 9:39
- research first approach. It's not a product
- 9:42
- and then, you know
- 9:43
- ChatGPT came out and it proved to the world that
- 9:46
- you know
- 9:47
- that chat experience was a killer app for general models
- 9:51
- at scale for consumers
- 9:53
- And then a couple of years later
- 9:55
- yes
- 9:55
- it's clear that coding has been the killer enterprise app
- 9:59
- So, what
- 9:59
- how would you compare and contrast the systems
- 10:01
- you guys used to discover those use cases
- 10:03
- ship them, scale them, monetize them?
- 10:06
- Any, any salient learnings from those two systems?
- 10:09
- Yes
- 10:12
- so we had made GPT-3 and
- 10:16
- we
- 10:16
- needed to make money because we wanted to
- 10:18
- go scale up to
- 10:19
- you know
- 10:19
- a billion and multi-billion dollar computers and we
- 10:21
- had GPT-3 and it was kind of interesting
- 10:22
- it was a cool demo
- 10:24
- but we couldn't figure out
- 10:25
- a product to build around it
- 10:27
- And we had been thinking
- 10:27
- thinking, we just couldn't do it
- 10:28
- We had tried a few things
- 10:29
- they, they hadn't worked
- 10:31
- And so we knew the models were going to get better
- 10:33
- but we also wanted to like
- 10:35
- start a revenue engine sooner
- 10:37
- And we said, well
- 10:38
- since we can't figure out what product to build
- 10:40
- we're just going to put this into an
- 10:41
- API
- 10:42
- And we're going to hope that somebody else
- 10:44
- can figure out what product to build
- 10:45
- And so we launched in like
- 10:47
- I don't know, something in the summer of 2020
- 10:50
- the GPT-3 API
- 10:52
- And
- 10:53
- initially it kind of got no
- 10:55
- traction at all
- 10:57
- And then about a month later
- 10:59
- randomly, as far as we can tell
- 11:01
- it went viral on Twitter
- 11:03
- On the same day
- 11:04
- a few different developers kind of found
- 11:06
- got it to do something cool
- 11:07
- posted it, other people started trying
- 11:09
- And
- 11:10
- and then like
- 11:11
- a lot of people started trying the API
- 11:14
- But it was shockingly bad
- 11:16
- If you go back and use GPT-3 or 3.5
- 11:21
- you will be astonished at how bad
- 11:23
- The models were then
- 11:24
- relative to the
- 11:25
- amount of excitement they generated at the time
- 11:28
- So people tried all of these things
- 11:29
- and really the only business that people got
- 11:31
- to work
- 11:33
- in a significant way with GPT-3 was copywriting
- 11:37
- And that was like not that great and not that exciting
- 11:39
- and we were kind of like
- 11:40
- you know, oh
- 11:41
- it's just going to have to wait for
- 11:42
- a better model
- 11:43
- But although that was the only business that was working
- 11:47
- developers had figured out how to like put
- 11:49
- in a prompt and get
- 11:50
- and be able to chat with it
- 11:52
- And we saw this a lot
- 11:54
- like more people were using
- 11:57
- they couldn't get the API to work for their business
- 11:59
- but they were using their API key to
- 12:01
- just chat. And we said
- 12:02
- well, we can build a good chatbot
- 12:03
- People clearly want that
- 12:05
- And we had a new model
- 12:07
- We actually had GPT-4 done
- 12:08
- but we had a
- 12:09
- new model we were ready to release in
- 12:10
- between called 3.5
- 12:12
- And we had figured out
- 12:13
- a new kind of post-training
- 12:14
- where we could get the models to do
- 12:15
- like a good job with instruction following
- 12:17
- so it could make it easier to chat
- 12:18
- with
- 12:19
- And we said, well, you know
- 12:22
- The API is not working great
- 12:24
- Maybe it was like a 10 or a
- 12:25
- 20 million dollar run rate kind of business
- 12:28
- But there is this thing that people love
- 12:30
- and
- 12:31
- under the YC principle of
- 12:33
- see what your users love and do that
- 12:34
- we said, well
- 12:34
- we'll build a chatbot around it
- 12:36
- And we put that out
- 12:37
- and we still didn't think it was going
- 12:38
- to do that well
- 12:39
- There was
- 12:40
- it was really meant as like a research
- 12:42
- demo
- 12:43
- to
- 12:43
- convince other people that they should build chat-like
- 12:45
- products and pay us for the API
- 12:48
- But that went like crazy viral
- 12:50
- And another thing I had learned from YC
- 12:52
- is when something really starts growing and it's not very good
- 12:56
- you have like a guaranteed hit on your
- 12:57
- hands
- 12:58
- And so we had like five days
- 13:00
- where the traffic would shoot up
- 13:02
- fall off, and everybody would be like
- 13:03
- well, that was just a hype cycle
- 13:04
- But then the next day it would get
- 13:05
- to a higher peak
- 13:06
- fall off again later in the day
- 13:07
- people would say that's a hype cycle
- 13:10
- By the fourth or fifth day
- 13:11
- I was like, I know how this works
- 13:13
- I know what's going to happen
- 13:13
- Like we have the potential here
- 13:16
- at a killer product
- 13:19
- And
- 13:19
- we knew we could make it much better
- 13:21
- We knew we could
- 13:22
- we knew we had GPT-4
- 13:23
- we knew we could keep scaling
- 13:26
- but by that fifth day
- 13:29
- we got everybody together and said
- 13:31
- this is an emergency
- 13:32
- this is the good kind of emergency
- 13:33
- but we have to build a company and
- 13:35
- a product all at once
- 13:38
- we then had like
- 13:39
- two months of crazy scaling
- 13:42
- and then we said
- 13:42
- you know
- 13:43
- we have to figure out a business model later
- 13:45
- For now
- 13:45
- we're just going to charge people so that
- 13:47
- we don't like run out our compute bills
- 13:49
- but that's obviously not the long-term answer
- 13:51
- That also turned out just to work
- 13:54
- and that was the story of ChatGPT
- 13:55
- And then there was so much utility
- 13:57
- that people just had not
- 13:59
- gotten over the activation energy of find
- 14:00
- that that has worked really well
- 14:04
- and then Codex Actually
- 14:06
- the plan before ChatGPT was that we were
- 14:08
- going to go all in on code
- 14:10
- We knew these models could write code
- 14:12
- We knew that they could be really
- 14:15
- and we knew that that would be like a valuable area
- 14:17
- but then we had this incredibly exciting thing
- 14:19
- happen
- 14:20
- But our kind of internal belief at the
- 14:21
- time was that coding
- 14:24
- was how these models would control things on computers
- 14:27
- and robots were how these models would
- 14:30
- control things in the physical world
- 14:31
- And if you made a smart enough model
- 14:33
- that had sort of the actuators of writing
- 14:35
- code and robot and driving a robot
- 14:38
- you could then kind of
- 14:39
- actually get this intelligence to do stuff for
- 14:41
- you in the world
- 14:42
- So
- 14:43
- then it took us a while to get
- 14:44
- there
- 14:45
- And then
- 14:46
- I think Codex got really good by early this year
- 14:49
- but with 5.5 is when we saw this
- 14:52
- real inflection point where people are now like
- 14:54
- doing just incredible things with it
- 14:57
- And, you know
- 14:59
- early in the class we've talked about how
- 15:01
- the capabilities pipeline
- 15:03
- is starting to look
- 15:05
- It's starting to become somewhat more
- 15:06
- legibly standard across different research groups
- 15:09
- You've got, you know, pre-training, mid-training, post-training
- 15:12
- then you've got the RL and supervised feedback loop
- 15:14
- Is
- 15:15
- do you think that's roughly like the shape
- 15:17
- of the pipeline that allowed Codex to
- 15:19
- you know
- 15:19
- go through a capability jump and that will
- 15:21
- basically stay stable now and consistent
- 15:23
- or are we going to go through a
- 15:23
- major rewrite of that pipeline?
- 15:25
- I think that is definitely the current pipeline
- 15:28
- I expect we will go through a major
- 15:29
- rewrite. I don't know when it'll happen or exactly how
- 15:32
- but
- 15:34
- it is a little odd to me that
- 15:35
- it's so
- 15:37
- It happens
- 15:38
- as a pipeline and it doesn't quite feel
- 15:40
- like the optimal solution
- 15:44
- what would be an optimal solution
- 15:46
- in your head? I think that's a research
- 15:47
- problem for the AIs to figure out
- 15:49
- I think we're at a point where
- 15:51
- and we've set this goal that by September of this year
- 15:54
- we will use 500,000 A100 equivalent GPUs
- 15:57
- like a lot of computing power
- 15:58
- as an AI research intern
- 16:00
- and by March of 2028 that we will
- 16:02
- have a full end-to-end
- 16:04
- very talented researcher
- 16:05
- like figuring out complete new architectures
- 16:09
- so I think we are going to get like
- 16:11
- with the current pipeline
- 16:12
- the current architectures
- 16:13
- I think we're going to get over the
- 16:14
- line of when AIs can do incredible
- 16:17
- incredible work
- 16:19
- you know
- 16:20
- one of the things that you just described there
- 16:25
- Is, you, you
- 16:27
- we've been talking a lot in the class
- 16:28
- about systems frameworks and analogies to make
- 16:30
- concepts from one domain legible to other people
- 16:32
- who may not have all the
- 16:33
- context in another
- 16:35
- And that sometimes because of the translation problem
- 16:39
- you know
- 16:39
- reasoning by analogy is not helpful
- 16:41
- because then errors compound
- 16:43
- right there you said
- 16:44
- you know
- 16:44
- our goal is to try to use it
- 16:46
- as an AI intern
- 16:47
- which obviously is a very useful metaphor within the context of
- 16:50
- you know, Silicon Valley
- 16:51
- a class that understands how these pipelines work
- 16:53
- and so on. And then as
- 16:54
- as you scale actually that metaphor globally
- 16:57
- people who might not have all that context
- 16:58
- go
- 16:59
- start analogizing these models in ways that they shouldn't be
- 17:01
- Like, how should we think about the limits of
- 17:04
- of that, of, of, what are the limits to scale of
- 17:08
- what are the product analogies
- 17:10
- the research analogies you find most useful
- 17:12
- within the Valley? And which one of the
- 17:14
- what have you found about
- 17:16
- found about the limits of those analogies scaling?
- 17:19
- And now how do you navigate between those
- 17:20
- two problems?
- 17:23
- I've been very interested in studying how
- 17:26
- like
- 17:28
- I think what is happening is
- 17:30
- we are in the process of creating a
- 17:31
- new utility
- 17:32
- This doesn't happen very often
- 17:33
- You know, electricity is a utility
- 17:34
- internet's a utility, water, I guess
- 17:37
- there's not a lot of these
- 17:38
- And so there are not a lot of
- 17:39
- examples that we can study
- 17:41
- for good
- 17:42
- metaphors or learnings about how to
- 17:45
- explain this to the world
- 17:47
- But I was recently
- 17:49
- looking at
- 17:49
- what happened when electricity became a utility
- 17:53
- And it's a good analogy for many reasons
- 17:55
- It's imperfect, of course, too
- 17:57
- But the electricity companies
- 17:59
- at least the ones I could find information about
- 18:01
- they didn't talk about selling electricity because no
- 18:03
- one knew what that was or why they
- 18:04
- wanted it. It sounds like very scary
- 18:05
- It's this thing that's like
- 18:06
- going to come into your house and it
- 18:07
- could kill you in this like gruesome way
- 18:09
- And, you know
- 18:10
- it feels sort of like very different than
- 18:12
- the world before
- 18:15
- And
- 18:15
- maybe they tried to sell electricity or market
- 18:17
- electricity first. I don't know
- 18:19
- But in any case
- 18:20
- that didn't work
- 18:21
- And then what they started Marketing
- 18:23
- selling to people was light at night
- 18:25
- You know, we are gonna
- 18:27
- what you are getting from us is not electricity
- 18:29
- it's light at night. By the way
- 18:31
- you can use the same thing that lets
- 18:32
- you get light for all these other things
- 18:34
- but people are like
- 18:34
- well, why would I want that?
- 18:35
- And they're like, well, you know
- 18:37
- it'll wash your clothes for you someday
- 18:38
- you know, no, no it won't, I can't
- 18:40
- that's too far of a jump for me
- 18:41
- Right
- 18:42
- so I don't know what our analogy for this should be
- 18:48
- but I suspect that even if Even if we're totally right
- 18:52
- and intelligence is going to become this new
- 18:54
- utility that every company
- 18:56
- every customer
- 18:58
- every government just
- 18:59
- needs access to and is going to use
- 19:01
- in all sorts of incredible ways
- 19:02
- and you will have like a
- 19:04
- OpenAI token subscription that you will plug into
- 19:06
- everything and use to access everything
- 19:08
- and
- 19:08
- you have running for you all the time
- 19:09
- and doing this amazing stuff
- 19:11
- I kind of don't think
- 19:13
- at least right now
- 19:14
- the right way for us to analogize that
- 19:15
- is we're selling intelligence
- 19:17
- because people are just like
- 19:19
- Somehow not resonating
- 19:20
- I don't know what our equivalent of
- 19:23
- we're selling you light at night is going to be
- 19:25
- but I think if we're going to become a new utility
- 19:28
- we need to find a way to explain
- 19:29
- to the world what it means to have
- 19:31
- this like
- 19:32
- intelligence pipe
- 19:33
- that you can just do whatever you'd like with
- 19:35
- It, so, one question that has emerged
- 19:40
- an emergent property of this class is of
- 19:42
- of having a
- 19:43
- diversity of different speakers is that
- 19:45
- the utility analogy has come up several times
- 19:47
- but in reference to different things
- 19:49
- So Jensen likened like compute to a utility
- 19:54
- and why there should be access and so on
- 19:55
- and talked about how Stanford should pool
- 19:57
- budget and so on and
- 19:58
- and
- 19:59
- and procure that as a utility for everybody
- 20:01
- on campus
- 20:01
- Whereas you just likened
- 20:03
- the intelligence part to a utility
- 20:04
- Are both of these things true?
- 20:06
- Is one of them true?
- 20:06
- One, is one more likely to be true?
- 20:08
- How should people reason about compute as a
- 20:09
- utility versus tokens as a utility?
- 20:11
- And, and by compute
- 20:12
- I mean here chips versus tokens
- 20:14
- Does that, does that make sense?
- 20:16
- I think as a consumer
- 20:18
- as like a business or an individual
- 20:22
- you will think in something closer to tokens
- 20:24
- or probably even one level up from tokens
- 20:27
- I don't think you'll care very much about
- 20:29
- you know, where the hardware is
- 20:31
- what particular chip it is
- 20:32
- what's powering it. I think that stuff will
- 20:34
- be abstracted out
- 20:35
- And what you will care about is when
- 20:38
- you're interacting with the system
- 20:41
- can you use it a lot?
- 20:42
- Is it cheap? Is it doing a good
- 20:43
- job?
- 20:45
- so right now it's like tokens
- 20:47
- It may get
- 20:48
- as we move into a world where
- 20:50
- we all just have like this constant agent running for us
- 20:53
- being useful to us all of the time
- 20:55
- you may think about it
- 20:56
- as even one level up
- 20:57
- But yeah, my, my guess is, is you
- 21:00
- when you like pay for your cell phone bill
- 21:02
- you're like, all right
- 21:03
- I'm buying access to airtime and some number of gigabytes and
- 21:08
- you know
- 21:08
- it's going to do all these things and
- 21:09
- I'll use all these apps and whatever else
- 21:10
- But like what you think about
- 21:12
- paying for the kind of internet utility in
- 21:14
- this case is just like
- 21:16
- Access to the whole system and
- 21:19
- the particular hardware at the base station
- 21:21
- now it connects to the internet
- 21:22
- You don't think about that as much
- 21:25
- I know
- 21:26
- I could nerd out about utility infrastructure for a long time
- 21:28
- but I want to make sure we switch
- 21:29
- a little bit to being relevant for the
- 21:31
- students. Usually we have
- 21:32
- questions, but we're not hearing those today
- 21:34
- unless you're comfortable. Oh
- 21:36
- okay, great. How about that?
- 21:37
- Improv. Okay
- 21:40
- so
- 21:40
- one final question to start getting the creative juices flowing is
- 21:43
- the final project for this class
- 21:46
- Orlin 5 by 183
- 21:47
- is the one person frontier lab
- 21:49
- So everybody here is working on projects
- 21:51
- where they're simulating being an individual
- 21:54
- as a lab with access to all the
- 21:55
- right tools. They've got hundreds of thousands of
- 21:57
- dollars of credits from CloudFlare
- 21:59
- I think we've got some OpenAI tokens maybe
- 22:01
- but there's a bunch of compute at their
- 22:02
- disposal
- 22:04
- what would you, if you were in the class
- 22:06
- what would you be working on for your
- 22:08
- one person frontier lab project?
- 22:10
- First of all
- 22:10
- I think that's an awesome project
- 22:15
- I think this is top of mind because
- 22:18
- you, we, we were just like talking about utility frame
- 22:21
- frameworks
- 22:22
- I think there's a lot of very smart people working on
- 22:26
- great training ideas
- 22:28
- and we're going to have incredible models
- 22:30
- No matter what you all do
- 22:31
- we're going to have incredible models
- 22:32
- I promise you, like pretty quickly
- 22:35
- But I
- 22:37
- I think we have not invested enough in
- 22:39
- being able to deliver
- 22:41
- at scale huge amounts of cheap intelligence
- 22:43
- So maybe I would go work on like
- 22:44
- the inference part of the stack
- 22:46
- And how are we going to get
- 22:48
- this incredible intelligence to be cheap and abundant?
- 22:51
- I think that's underinvested
- 22:53
- and, and
- 22:53
- and
- 22:54
- I think all of the frontier labs are
- 22:55
- going to have to become
- 22:57
- inference companies to a significant degree
- 23:00
- okay
- 23:02
- It might be too late to pivot your projects
- 23:04
- but better late than never
- 23:05
- Work on whatever you want to work on
- 23:08
- okay, let's start taking questions
- 23:09
- and I'm gonna moderate and try to be not
- 23:12
- you know, please try to be productive and not spicy
- 23:14
- etc.
- 23:15
- Remember, it's a CS class
- 23:16
- but up to you
- 23:17
- Sam, if you're gonna ask this one
- 23:19
- Oh, we've got questions. Oh
- 23:20
- perfect. All right, first one
- 23:22
- The question's about your views on
- 23:24
- Yann LeCun's view that
- 23:25
- LLMs are a dead end
- 23:29
- first of all, in terms of achieving human-level intelligence
- 23:33
- these models have already far surpassed human intelligence in some ways
- 23:37
- and then they're wildly worse than others
- 23:39
- Like, for example
- 23:40
- they seem much worse than people are
- 23:42
- at very long horizon
- 23:46
- kind of high judgment signal and tasks
- 23:51
- on the other hand
- 23:52
- yesterday, we had one of our models
- 23:55
- discover, disprove a conjecture
- 23:58
- one of the Erdős problems that had
- 24:00
- Smart people had worked on for a long time
- 24:02
- and a lot of people
- 24:04
- a lot of smart scientists
- 24:05
- I don't know if Lacoon was one of them or not
- 24:06
- had
- 24:07
- even quite recently said something like that was
- 24:09
- not going to happen
- 24:11
- And then like the model just did it
- 24:13
- And you know, now you have all these mathematicians saying like
- 24:16
- is math over? What does this mean for
- 24:17
- our field?
- 24:18
- So clearly LLMs are capable of figuring out new knowledge
- 24:23
- and clearly they are capable of doing some things that
- 24:26
- some intelligence tasks that humans just can't do
- 24:30
- They're going to scale much further
- 24:31
- So how much better
- 24:32
- and what distribution of the tasks they can
- 24:34
- do better than humans
- 24:35
- we'll find out, but I suspect it's a lot
- 24:37
- And the, you know
- 24:38
- in terms of this like
- 24:40
- lack of a belief in the exponential we
- 24:42
- were talking about earlier
- 24:44
- I think the field was
- 24:45
- honestly held back by
- 24:47
- a
- 24:47
- generation of scientists who just were way too
- 24:50
- certain on what wouldn't
- 24:51
- what scaling was not going to produce
- 24:53
- And then some people just looked at the graphs and said
- 24:55
- well, it looks like it's continuing beautifully
- 24:57
- Let's keep going
- 25:01
- I think world models are clearly important and to
- 25:06
- we'll need that for things like robotics
- 25:08
- but betting against LLM scaling at this point
- 25:15
- feels quite misguided to me
- 25:19
- Does it get annoying to be the I
- 25:20
- told you so guy? No
- 25:21
- I mean
- 25:24
- there are these like Twitter trolls that
- 25:26
- you know, for years have just been like
- 25:28
- it's not gonna work
- 25:29
- it's not gonna work
- 25:29
- this is so dumb
- 25:30
- like, you know, this is fraud, this company is gonna fail
- 25:32
- this research approach is gonna fail
- 25:34
- and I used to get more bothered by them
- 25:36
- but I don't even like feel the I
- 25:37
- told you so at this point
- 25:38
- it's like, you're still going on about it
- 25:41
- like, the data is quite strong on our side
- 25:48
- In
- 25:48
- I don't think it'd be that fun to
- 25:49
- say I told you so
- 25:50
- and also the fact that you're like still
- 25:51
- saying we're wrong doesn't really bother me
- 25:53
- I think there's that kind of move on
- 25:55
- There's that saying that like insanity is
- 25:57
- doing the same thing over and
- 25:58
- over again when presented with data that is not working
- 26:00
- and if they keep repeating that
- 26:01
- in a sense it's
- 26:02
- it's, it's a form of insanity
- 26:04
- I think. I, I think there's something that happens
- 26:07
- which is if you make your identity
- 26:09
- about a particular thing is going to work or not work
- 26:14
- and you associate yourself with that belief
- 26:17
- and then the science or the empirical results disprove you
- 26:21
- and you're like too hung up on your identity
- 26:23
- you can't let it go
- 26:24
- you can't see the truth
- 26:26
- And I think this is like a important
- 26:27
- reminder in both directions. Yeah
Watch these videos on your TV
XTTV brings video content from X to your living room. Download now and start watching on the big screen.