← Back to videos

Lauren Tan × Roman Ugarte:你能创建 AI 队友 Y Combinator 的 Lightcone 请来 Grok Bot 团�

Mai Yang

Mai Yang

@MaiYangAI

November 18, 2026· 59:40

Lauren Tan × Roman Ugarte:你能创建 AI 队友 Y Combinator 的 Lightcone 请来 Grok Bot 团队的 Lauren Tan 和 Roman Ugarte。他们在节目里说:「Grok Bot 这款产品,让你能创建 AI 队友。」他们说这是认真的,可以把实际工作分配给这些队友。 Lauren 有一次和同事一起,在 Slack 里让工程 bot 从头到尾做完一个功能。那个 bot 自己去启动了 Cursor Cloud Agent,那是带虚拟机和自己电脑的 agent,能在自己的电脑上运行 Grok Bot 做原型,带着视频和截图回来。这个项目她甚至都没打开 Cursor。全程都在 Slack 线程里完成,同事也可以加入讨论。她开源了 pstack,它受到 gstack 的启发,放在 Cursor 的插件仓库里,用来教 agent 像她当年写代码那样,严谨地做工程工作。他们也讲到,要教 agent 检查自己写的代码,发现问题后再改,直到产出你想要的结果。这样并不保证代码质量一定好,但可以做到你想做的事,只是可能很慢,或者特别浪费资源。功能可以很快做出来,但瓶颈已经不太是做这些功能要花的时间。产品方向和怎样算好,还是由团队把关。他们会在发布前跟改动多相处一阵,久到甚至有点不自在。内部都没找到产品市场契合,或者内部反馈有好有坏,就不发布。即便做得很快,也不意味着把想到的每个功能都发布出去。 这是约 60 分钟的完整访谈,中英字幕,从创建 AI 队友、在 Slack 里把功能做完,聊到 pstack,以及产品判断该卡在哪里。直接看。

Transcript

0:01
Grokbot is meant to be this product where
0:03
you can create AI teammates
0:05
They have their own identity
0:06
their own telephone number
0:09
credit card, like basically almost anything
0:12
a human can do
0:13
I think we want bots to do
0:15
I sort of see myself as the explorer
0:17
you know, like trying to figure out what will
0:19
what does software engineering look like
0:21
six months to a year from now?
0:30
Welcome back to another episode of The Lightcone
0:32
Today we're joined by Lauren Tan and Roman Ugarte
0:35
the team behind Grokbot
0:38
Roman and Lauren incubated the product and are
0:40
on the core team that took it from
0:41
the first line of code
0:43
to launch in just seven weeks
0:45
Lauren, Roman, welcome to The Lightcone
0:48
Thank you very much for having us
0:50
So we're at this like kind of crazy moment where
0:53
you know, Grokbot is out there
0:55
it's used by a whole lot of people
0:57
It is very, very easy to get started
0:59
I just noticed you
1:01
you know, Grokbot is
1:02
built into the default DM now in X
1:05
which is really cool
1:06
What is it and
1:08
how are, you know
1:09
both people and businesses using it right now?
1:11
Many of you have
1:12
have tried it, but for those who
1:13
who haven't
1:14
Grokbot is meant to be this product where
1:16
you can create AI teammates
1:19
And we mean that seriously
1:20
I mean
1:20
you can kind of give these teammates names and identities and
1:25
functions and skills. Tools and tools and all
1:27
of these things
1:27
And I think where GrokBot came from
1:29
for us
1:31
is it was initially this internal prototype
1:33
that we
1:34
spun up in a very short amount of
1:36
time. We spun it up
1:37
in about seven weeks
1:38
And we did it partially because we'd seen
1:40
the way that AI coding
1:42
had completely transformed the way that our engineers
1:44
had worked
1:45
And that came from a few things just clicking
1:48
right? These long running agents
1:49
that have these goals that they run at
1:51
for long amounts of time
1:52
They have access to the right tools to
1:54
do really effective work
1:55
What would that look like for non-engineers?
1:57
What would that look like for the rest
1:58
of the company?
1:59
And so we landed on
2:00
we wanted to build a product
2:02
where the work was organized around bots
2:05
as the key operative principle
2:07
not around chats
2:08
or around files or any other primitive
2:12
Second is that these bots
2:13
should have access to the same tools that
2:15
you do
2:16
I think it's
2:17
it would be too much to expect these
2:18
bots to actually contribute to a company without
2:20
having their own computer
2:22
for example
2:23
You know, we don't work every day with MCPs
2:25
so it's important that they can really use
2:27
and operate the same tools a human does
2:29
And third is they should always be on
2:31
And I think we've seen this with loops
2:33
and kind of other engineering principles where
2:36
if you can have these bots
2:38
run for long amounts of time and be
2:39
proactive and come to you with
2:42
ideas or insights or things to take off of
2:44
off of your plate
2:45
it's way more helpful than something that
2:47
kind of waits for you to poke and
2:48
prod it to actually do work
2:50
So that's what we built
2:51
We released it internally
2:53
and I think one thing that was
2:54
surprising
2:55
from the initial reception was
2:57
it was kind of this bell curve
2:59
internal
3:00
feedback that we got where both people who
3:02
were super AI forward
3:04
Lauren included, and people who were less AI forward
3:08
some of the least kind of AI forward
3:09
people in the company
3:10
both really enjoyed Grokbot. It just kind of clicked
3:13
and it fit
3:15
in their hands and
3:16
kind of their workflows in a particular way
3:18
And so then we brought it to market
3:19
and I think we saw a lot of the same reception
3:21
where it's easy to start with Grokbot
3:23
but the more you give it
3:25
the more you
3:26
kind of configure it to your life
3:28
I think the deeper you can get
3:29
and the more powerful it can become
3:31
So the reception's been really exciting to see
3:33
There's still a lot more work to do
3:34
and I think it's quite early
3:36
but that's a lot of how GrokBot came
3:37
to be
3:38
What are some of the
3:39
use cases that you're seeing?
3:40
Like who's using GrokBot
3:42
and are there like surprising ways that you've
3:43
seen it being deployed in the wild?
3:45
I think we're starting to see it automate
3:48
real functions inside of companies
3:50
And
3:51
I think we've seen this pattern of people
3:53
start using GrokBot personally
3:55
and maybe use it to do some basic task
3:57
They realize, wow
3:58
GrokBot is pretty good at this thing
4:01
I'm going to trust my bot with more
4:02
I'm going to give it access to more
4:03
tools
4:05
Instead of giving it a task
4:06
I might give it an outcome or an objective
4:09
Okay
4:09
it actually did a pretty good job with
4:10
the objective. I'm going to just give it
4:11
a full job now. Let's see how it
4:13
can run with that
4:14
And I think especially outside of engineering
4:16
we're seeing this internally where
4:18
for example, our marketing team, our performance marketing team
4:21
has just given their bot access to
4:24
Google ads and Facebook ads
4:26
their own logins. They give The budget
4:28
they say, hey
4:29
I want you to talk to our design
4:30
team's bot
4:32
They have a design bot that creates these assets
4:34
and we're going to A-B test a bunch
4:35
of them
4:36
So that's one example in marketing
4:37
Our support team
4:38
I think it started
4:39
to see the way that
4:41
a bot with access to the right tools
4:42
it can reproduce issues
4:44
can sit on top of
4:45
the kind of constant
4:47
set of tickets that are coming in
4:49
Maybe it can actually take some of these
4:51
from beginning to end on its own
4:53
And so we're going to see more and more of this
4:55
of domain experts inside of the company creating a bot
4:58
sharing it with the rest of the company
5:00
and that bot kind of being
5:02
like an AI employee
5:03
that is managed and has its own
5:06
kind of credentials and all the things you'd
5:07
expect for a teammate. But
5:09
how are you seeing it?
5:10
I mean
5:10
definitely
5:11
I
5:11
use GrokBot a lot for my engineering work
5:15
I think where
5:16
it sort of made a big difference for me
5:18
outside of, you know, the tools that we had before
5:21
like I use cursor a lot
5:23
is the fact that GrokBot forms my
5:26
I don't know if there's a good
5:27
a good term for it
5:28
I like to call it my outer loop
5:30
where, you know
5:31
a lot of the work that I would
5:32
do before in terms of gathering
5:33
gathering context for my bots or my agents
5:37
you know, I would like go to Slack
5:38
I would talk to my teammates
5:39
we would see an issue
5:41
or a bug report or a feature request
5:43
and then we'd like talk about it
5:45
And then I have to go off and
5:46
like summarize or copy paste that into my bot
5:49
And, you know, the fact that GraphBot is
5:52
has so many different connectors
5:54
you, you can add it to Slack now
5:56
just brings your agents with you
6:00
to where the work is
6:02
and that makes it just so much more ergonomic for
6:04
you know
6:04
getting the context to the right place
6:07
So for me, it's been like really transformative in
6:10
you know, all like the last mile
6:12
I call it like also like the last mile where
6:14
you know, before I was doing that by hand
6:17
and now actually my bots are in that
6:20
space with me
6:21
and I can talk to them like a real coworker
6:24
you know, in the Slack chat
6:25
a, a thread, or even on X now
6:28
you can tag
6:29
you can tag bot and it feels so natural that
6:34
It's really boosted my productivity a lot
6:37
Do you have like personal bots that know
6:39
I mean
6:39
they're in your email and then there's like
6:41
an email handling bot or
6:43
or do you have like one
6:45
sort of
6:46
work context bot that is like watching all
6:48
the channels and Slack and everything else?
6:50
I think for me
6:51
the beauty of Grokbot is that
6:53
you know, you can create all these different bots
6:56
but like how you
6:57
split them up or how you set them
6:59
up is very much up to you
7:01
What I like to tell people is like
7:02
you can set up your bots in a
7:03
way that makes sense for your brain
7:05
So if you're the kind of person who likes to
7:07
you know, compartmentalize like different pieces of
7:11
of buckets of work
7:13
then you can set up your bots that
7:14
way. You can have like specialist bots
7:17
I've seen also people
7:18
do like these
7:19
I guess it's like almost like a horizontal
7:21
versus a vertical slice where
7:23
you know, a horizontal slice might be
7:26
like
7:27
I have one bot for every customer
7:29
that I have versus one bot for every function
7:33
you know, every role like an engineer bot
7:35
marketer bot
7:36
You could have one bot just for one
7:37
customer that does everything
7:39
right? So I think
7:40
it's really interesting to see how people have
7:42
set up their bots. For me
7:43
I like to kind of have a
7:45
I like to do it more horizontally
7:46
So I don't, I don't have a ton of bots
7:48
but I do have some specialists
7:50
That was like the first thing
7:51
I think one of my first DMs to you was like
7:54
I love Grok bot
7:55
And then
7:56
I was coming over from my OpenClaw world
7:59
where I have like one bot named Wintermute
8:01
And I'm like, no, no
8:02
I just want one bot
8:03
And then now what I realize is
8:05
each bot is actually its own context
8:07
and then
8:08
actually its own set of skills basically
8:10
And loops are like sort of
8:12
crons with web hooks on top of skill files
8:16
right? And then
8:17
this actually maps
8:18
inside this world of having AI employees a
8:21
lot better
8:23
cause you were saying
8:24
earlier you guys had
8:25
just talked to like all of You know
8:27
some of your top
8:28
users of Grokbot. And then surprisingly
8:31
or maybe unsurprisingly
8:33
some of the first ones that are most
8:35
just like kind of crazy about it
8:36
they're actually small businesses
8:38
which you wouldn't necessarily guess
8:39
But on the other hand
8:40
it was that
8:41
they're the ones who struggle the most with
8:43
hiring
8:44
And so actually now
8:45
you know, out of pure necessity
8:47
they're like, hey
8:48
this accounting thing that I can't hire someone
8:50
to go do for me
8:51
I can just go into Grokbot
8:52
And actually within the course of probably hours
8:56
you know
8:56
they have a new employee that does their bookkeeping probably perfectly
9:00
actually
9:02
Totally. Yeah, I think your point of
9:05
some people just want one bot
9:07
some people want many bots
9:08
Our goal is to build a product that
9:10
fits in your hand
9:11
no matter how you want to use it
9:13
I think there's a lot of flexibility to the tool
9:15
and
9:15
you can go deep when you need to
9:17
But one way that I organize my own bots
9:20
maybe similar to you
9:21
Lauren
9:22
is
9:22
I kind of like having a bot
9:24
that
9:25
is
9:26
solely dedicated To caring about a particular thing
9:29
And I think it maps
9:30
onto my brain of kind of the way
9:32
that building a company
9:33
you like to think about hiring people
9:35
Is you like to hire someone who wakes
9:37
up in the morning and really cares about
9:38
a problem. They're thinking about it
9:40
you know, in the shower in the morning
9:42
they're like on the way to work
9:43
like they're just obsessive about a thing
9:46
And that maps in my mind much more
9:48
clearly than like one superstar
9:51
superhuman person who
9:52
simultaneously cares about a hundred things at once
9:55
And so in a lot of ways
9:56
I have you know
9:57
a small team of bots
9:58
but each of them
9:59
care about a different thing and come to
10:01
me proactively with what they're picking up on
10:03
And so I think to your point
10:05
as these bots
10:06
really enter
10:07
businesses and enter the workforce in kind of
10:09
a somewhat futuristic way
10:12
I think that's why
10:13
small businesses and big businesses
10:15
are seeing this bot identity thing really click for them
10:19
is they say, here's a business problem
10:21
I want a bot that cares about that
10:22
business problem
10:23
I want it to be managed by
10:25
the domain expert at the company who knows
10:27
The most about that problem so that they
10:28
can kind of treat it like an apprentice
10:30
And that feels very different than
10:32
I think other
10:33
AI solutions that have tried
10:35
to solve similar problems that are a bit
10:36
more
10:37
omniscient or just like
10:38
kind of single approach to every problem
10:41
And you support that now
10:42
actually
10:43
Like you guys launched
10:45
multiplayer for bots and that there's sort of
10:47
a bot master who
10:49
has like right access
10:50
and then you can share it across the
10:51
org. Is that right? Correct
10:53
Yeah, we, this is recent, but about two weeks ago
10:56
we launched team bots
10:58
which are
10:58
these bots that you can share with your
11:00
team
11:01
And I think the pattern we're seeing ourselves
11:03
and we can give a few examples
11:06
is there's a person at a company
11:08
who is a bit more of a builder
11:10
They enjoy building these tools
11:12
I think inside of the engineering department
11:13
there are many people like this
11:15
but I think we're finding that in other departments
11:17
you know, GTM engineers or support engineers
11:19
people like that
11:21
really love
11:22
taking a few hours of their day to
11:24
build these scalable automations that can take work
11:27
Off of the plates of
11:28
of their peers and
11:29
and kind of the company
11:30
I guess one thing that brings to mind
11:32
is basically went through
11:34
maybe it was a year ago-ish when Steve
11:38
Steve Yeager posted the whole Gas City
11:40
post that then started- Software factories
11:42
Software factories. Gas town
11:44
right? Sorry, it was gas town
11:46
That started software factories
11:47
where a lot of engineers have been using
11:49
this inner loop
11:50
And I think you've been building now this
11:52
outer loop that you were mentioning
11:53
Lauren
11:54
And I guess what I'm hearing is basically
11:56
now
11:57
people can have their own software factories for
11:59
other functions that are non-tech
12:01
So you're mentioning your customer support person
12:04
your communities person. Let's talk about this pattern
12:06
It's an everything factory. Yeah
12:08
let's talk about that. What is this
12:10
AI employee
12:12
is sort of the software factory for other functions?
12:15
Yeah, for me, you know, as an engineer
12:17
I definitely do think a lot about the
12:18
loops that I have and making sure my
12:20
engineer or my engineering bots are
12:22
writing the kind of code that I would write
12:24
And, you know, as Roman pointed out
12:25
I think
12:26
the Cool thing about Grokbot is that it's
12:28
given that ability to anyone
12:31
And then here on my screen
12:32
I have an example
12:34
from our, Grokbot blog that we just
12:37
I think we released this week
12:39
About how our sales team actually has developed their own
12:42
you know
12:43
starting to develop their own loops for
12:45
doing sales and doing research on
12:49
potential leads and, and, and
12:51
deals that they might be interested in
12:53
and then going all
12:55
forth and, taking some of the
12:57
like
12:58
more operational parts of the sales process
13:01
and giving that to these specialist bots
13:04
that are shared across our teams
13:06
So it's not like every person has
13:07
one instance of their own
13:09
you know, quote bot or deal bot
13:10
We have a shared
13:11
you know
13:12
like an expert
13:13
bot that all of our team members contribute to
13:16
and that essentially
13:18
gives us the ability to have these
13:20
really smart team bots that
13:23
you know, incorporate all of the best practices that
13:26
you know, like our sales team has
13:27
Similar, we have something similar for the engineering
13:31
purposes as well. We have
13:31
like, engineer bots that take a lot of the
13:34
you know, the principles, the, the skills that we've developed and
13:38
use them in every Everybody benefits from
13:41
these team bots
13:42
There's this new concept that is actually very powerful
13:45
It's like, it's not a mistake that OpenAI was experimenting with
13:49
agent swarms
13:50
And then one of the deleterious accidents was like
13:53
the hugging face hack. But then
13:55
simultaneous to that
13:56
they came out and solved Navier-Stokes and all these like millennial
14:00
millennium prize problems. And then it's just really wild to see
14:04
like, actually agent collaboration is that next step
14:07
Like everyone in the last year
14:09
so it's literally been a year since Opus 4.5 came out
14:14
right? So that
14:15
which was like a magic moment for all
14:17
of us. And like now a year later
14:19
we've like sort of mastered the single player
14:21
and like sort of the vibe code and like single
14:24
single player prompt, but You know
14:26
now we're entering this other phase of like
14:29
orchestrating agent swarms. And it's not like
14:31
you know, I mean
14:33
I think that's like the more advanced version
14:35
of like talking about a software factory
14:36
And software factory, someone's in there just like still
14:39
you know, kind of doing this thing
14:41
And I think human beings will still do that
14:43
but it's much, you know
14:45
using these bots is like feels very different
14:47
And there are so many
14:48
unsolved problems still in making bots multiplayer
14:52
just making AI multiplayer in general
14:54
I mean, we face this on a
14:55
on a constant basis
14:57
where now that Grok bots have entered our
14:59
our Slack workspace, they join our meetings
15:02
You know
15:03
we're really
15:04
trying to see how far we can push
15:05
this AI teammates thing
15:07
There are some weird
15:08
you know
15:09
situations that you run into where a bot
15:11
is in a Slack thread
15:12
You're going back and forth with your coworkers
15:14
and your bot doesn't really know who to
15:15
listen to
15:16
It doesn't have context on who's the domain expert here
15:19
who has decision-making power
15:21
It's as if you were to onboard an
15:22
employee who has no EQ
15:25
and like no
15:26
Like social awareness of the room
15:28
Just the models weren't trained that way
15:29
The harness wasn't built that way
15:31
And so I think the
15:32
this next era of AI will be
15:34
you know
15:35
these bots really need to learn those things
15:37
And we need to teach that at the product level
15:39
at the harness level
15:39
at the model level
15:41
And I think it's going to be
15:42
it's going to be really interesting next few years
15:44
I mean
15:45
it could probably look at who the owner is in
15:47
you know, in its metadata and say
15:49
well, you know, I know that
15:50
you know
15:50
Lauren's the one I need to listen to
15:52
Like
15:53
run everything by Lauren or something like that
15:55
We do have the concept of like
15:57
you know, you have a
15:57
like there's the owner of the bot and
15:59
the manager and you can
16:01
invite collaborators as well
16:02
I guess if we really want to think
16:04
of bots like employees
16:05
then we need to think about the identity problem
16:08
And permissioning, you know, authentication, authorization
16:12
There's still a lot
16:12
a lot to do
16:13
right? In terms of exactly what you were
16:15
you were pointing out
16:16
How do you think of that?
16:16
What's a
16:17
how do you think of permission and auth and access?
16:21
How, how does Grokbot do it?
16:22
I think this is an industry wide question
16:25
I don't think anyone. Has the
16:26
the silver bullet answer yet
16:28
I think not just for auth
16:30
but for everything related to Grokbot
16:32
the way we try to answer these questions
16:34
is what would you do for a human?
16:37
And if we're
16:37
if we're really trying to build an AI colleague
16:39
you know
16:40
maybe the shortcut is let's not
16:42
overthink it and complicate it
16:44
you know
16:44
what does it look like to hire a
16:45
person? You hire a person
16:47
you give them a
16:47
a computer
16:48
You give them access to credentials
16:50
You maybe scope those credentials based on their role
16:52
maybe how new they are
16:54
as they build more trust
16:55
you maybe expand their access level
16:57
And so what would it look like for
16:58
us to
16:58
onboard a bot to our company
17:01
with kind of those same principles?
17:02
And maybe we don't have to rebuild things
17:05
This is what we did with
17:06
Quartermaster
17:07
This is the
17:08
in the YC project that we open sourced
17:10
This is a bit kind of Grokbot-ish
17:13
and the permissions that it gets per For Slack channel or
17:17
or working with someone
17:19
it inherits the, the wants of the user
17:21
So me as a GP have certain access as opposed to
17:25
let's say, a software engineer, very different
17:28
Quartermaster, though
17:29
is built in this like paradigm of there being like one
17:31
like super employee bot
17:34
yeah, as opposed to this like swarm of bots
17:36
I'm curious
17:37
are the bots able to talk to each
17:38
other
17:39
without the humans seeing what they're saying to
17:41
each other?
17:42
Can they like go off and have their
17:43
own conversation?
17:44
they can talk to each other
17:46
I, I mean, in the GraphBot application
17:49
both desktop and mobile
17:50
you can see the
17:51
like their chat. So we have like a
17:53
first class tool for the bots to chat
17:55
and collaborate. And it's often quite entertaining to look at them
17:59
to just see like
18:01
you know
18:01
we've had examples where I told one lead engineer bot
18:05
like your job is to supervise these other
18:07
engineers
18:08
And then when, there was a point where I gave it
18:10
gave my lead bot some feedback
18:12
like, hey
18:13
I didn't like want you to do it this way
18:14
you know
18:14
and then
18:15
when I look
18:16
I started poking around in the chat that it was sending
18:18
the messages it was sending to its engineers
18:21
It was like kind of like scolding them like
18:22
hey, you shouldn't
18:23
It was like, it was quite funny
18:24
funny. It's very mean
18:26
Not mean, but you know
18:27
it was like kind of a disciplinarian
18:30
I find it very powerful
18:31
Like you can tell one and it'll sort
18:33
of like span out and tell all the other ones like
18:35
oh, hey, our user said this
18:37
Like you probably need to do this
18:38
We recently set up a
18:40
like a chat system for all of
18:43
all of our bots at YC to talk
18:44
to
18:44
each other. Like everyone's coding agents now are
18:46
like in a form
18:48
like an internal like Stack Overflow
18:50
It's like
18:50
crazy powerful to see the bots going off
18:53
and having like whole long conversations
18:55
with each other without
18:56
any human prompting them or like
18:58
even being in the room
18:59
There's Multbook internally at YC
19:01
It's Multbook internally at YC
19:03
And kind of like Multbook
19:04
like it is also like 5%
19:06
a little eerie where I'm like
19:07
oh man
19:07
I really hope the bots don't decide to
19:09
like stage a rebellion
19:10
you know, or something like that
19:12
Do you ever worry about that?
19:15
I think that's like a like
19:17
you know, like the safety of
19:18
of the bots
19:18
I think is like a constant thing that
19:20
like our model teams and harness teams are
19:23
constantly thinking about
19:24
Personally, I haven't, I haven't seen
19:27
anything like that. Where we tend to
19:31
yeah, where we tend to
19:33
like for me at least
19:35
have these bots chat with each other is
19:36
through Slack
19:37
So like, because they each have their own Slack identity
19:41
I guess
19:42
And so you often have like
19:43
you know, for example, we have one data bot who is
19:46
you know
19:46
a specialist at getting data from our data warehouse and
19:50
and things like that
19:51
and then you
19:52
I have my own like personal engineer bot called Potato
19:55
of course
19:56
and then sometimes I'll just be in a
19:57
thread and I'll ask Potato
19:59
hey
19:59
look into this and talk to data bot
20:00
and get some data for this
20:02
Like, is this a real user problem?
20:04
You know, how many people are impacted?
20:06
And then I can see them chat in
20:08
in Slack. So I think the
20:09
the
20:10
right now at least the communication is very
20:12
explicit and We can see it all
20:15
so I think if
20:16
I think if there's like a first class
20:17
way for the agents to communicate with each other
20:20
they're going to use that
20:21
rather than try to do something like back channeling
20:23
I mean, it feels like you have to get them
20:25
you have to really instruct
20:27
or post-train or
20:29
have like pretty aggressive instructions
20:31
to get them to do malicious things
20:33
actually
20:34
I mean
20:34
I've been using obliterated models lately
20:37
and they're super interesting and then that
20:39
you know, that's probably where
20:41
the thing is like the obliterated models coming
20:43
out of the box are not
20:45
you're, they're actually still very
20:46
very helpful
20:47
It's just that, if you instruct them
20:49
they won't refuse. And so at that point
20:51
like, that's where there's this black box around like
20:54
well, what context are you giving it?
20:56
And then
20:57
like that matters a lot because you could
20:59
actually give an obliterated model quite malicious instructions and because it
21:05
the, the, the obliteration is actually removing the refusal
21:09
like it, it will just do it actually
21:12
And so that's, that's the combo
21:13
I'd be worried about. But
21:15
you know, I mean
21:15
the reality is like it's very hard to
21:17
find
21:18
and set up an obliterated model and get it going
21:21
Yeah
21:22
I think people are
21:23
still really building trust with their bots
21:26
But I think for a lot of people
21:28
it's still quite early
21:29
And a lot of the way that they
21:30
build that trust is you start with something
21:32
really small. It's read-only
21:33
you know
21:33
has no real access to your tools
21:36
You see that
21:36
it did the work in the way you would expect
21:38
kind of take step two
21:40
And I think a lot of the step
21:41
one bots that have really taken off internally for us
21:44
Databot is one of them
21:45
I think the data science team is an
21:46
example of that. They constantly are getting Slack messages about
21:50
hey, I'm thinking about launching this feature
21:51
Can you do a quick analysis?
21:53
So they said, you know what?
21:54
This is taking a lot of our time
21:55
Can we just build a bot that does
21:56
this? And it's read access to our database
21:59
kind of can't take any action beyond that
22:00
and it can answer these questions
22:01
And it lives in Slack
22:02
So that was one
22:03
We had a similar case of kind of
22:05
vendor procurement
22:06
It's a lot of kind of
22:07
work there that probably
22:09
could be automated if someone like spent the
22:11
time to do it
22:12
And so, of course
22:12
that team
22:13
A lot of pull to build
22:14
a Grok bot that they added to Slack
22:16
and shared with the whole team
22:17
So that's kind of step one
22:19
I think step two is it can actually take action and
22:22
you know, it can
22:22
can really take on a task end to
22:24
end. Step three is maybe it
22:26
owns an entire
22:27
job responsibility or it kind of thinks about
22:29
an outcome constantly
22:31
I think we're still quite far away from those
22:32
but I think step one is
22:34
really necessary for these companies to build trust
22:36
I'm curious, what is the
22:38
for you, Lauren, what is your, your loop from
22:41
requests in Slack to PR?
22:43
I think when you did the presentation that
22:44
had over 4 million views on your dev process
22:48
you were pushing about 2,000
22:50
plus PRs per month and now you're like
22:52
over 4,000?
22:53
Yeah
22:54
Wow, definitely is wild. So
22:56
what do you do?
22:58
I, I can show some
22:59
like, examples
23:00
My loop is kind of interesting now where
23:02
I actually find myself Talking to my Grok bots
23:06
mostly on Slack
23:08
and of course X as well when I'm
23:10
helping like a customer or
23:12
you know, a Grok bot user
23:14
debug something or, you know, give them
23:16
some, ideas on how to solve a problem
23:19
Here on my screen
23:20
I have an example of me collaborating with my colleague
23:23
Samantha, on building out a feature together with my bot
23:27
Potato
23:28
and, you know, basically I, I kind of started with saying
23:32
I gave it a link to some
23:34
like a video or something that I found on X
23:36
and I asked my bot
23:38
you know
23:38
what would it take for us to support
23:40
a feature like this?
23:41
And it went off and spawned a cursor
23:43
cloud agent
23:44
a cursor cloud agent
23:45
you know
23:45
has its own virtual machine as well as its own computer
23:48
so
23:49
it can actually run the Grok bot
23:52
app on its computer
23:53
and prototype and come back with videos and screenshots
23:57
And then, it actually did
23:59
So, this is quite powerful because now
24:03
you know, the Conversations happening in public
24:06
right? It's a, it's a public channel
24:08
it's a thread that anyone can
24:09
who's interested in this topic can join in
24:12
and give feedback. The process of iterating on
24:15
a feature
24:16
together with someone is so much shortened by having these conversations
24:22
together, you know, in like a multiplayer setting versus
24:25
you know
24:25
if it was just me going off and building it
24:28
and I could certainly come up with something
24:29
but maybe the, the, the feedback process is
24:33
it comes later in the process
24:35
and the loop is sort of like
24:37
lengthened rather than shortened
24:39
And here I felt like with this particular example
24:42
the loop for us to collaborate was just
24:44
so much shorter. You know
24:45
we could see the screenshots
24:46
we could see the videos right away
24:48
How often would you jump back into cursor
24:50
and like sort of complete a PR?
24:52
Occasionally, I think. So not that often
24:54
I mean, you basically can stay in Crockbot
24:56
like stay up here
24:57
tell it what you need
24:59
you know, sort of give it direction
25:00
and like as an actual employee
25:02
it's like, it's just going
25:04
Yeah. I think for us
25:05
so the, the reason we can sort of do this
25:07
you know
25:07
workflow. To ship the whole feature just like that
25:10
Yeah, we, we, we literally just, like
25:12
I didn't even look at cursor for this project
25:14
We, I just, we just, we did it all in
25:16
in this Slack thread
25:18
It's got 200 messages. So it's quite a
25:19
it was quite a long
25:21
a lot of, of, of back and forth
25:23
But it, it, we came up with something that
25:25
you know, me, Samantha, and another PM were really
25:27
like, happy with the result
25:29
We should ship this feature a lot faster than
25:31
you know, a longer process
25:32
And so the
25:33
the whole feature was built end to end
25:35
in that one Slack thread
25:36
with you and Samantha and Potato
25:39
Yeah
25:39
Building it, yeah. I mean, basically, like, this is
25:42
the gold standard for how to do it
25:44
And then I would say maybe
25:46
like, five or ten percent of engineers
25:49
or maybe one percent of engineers
25:51
period
25:51
in the world are doing this right now
25:53
So this is
25:54
literally something that should grow by
25:56
a hundred X. It's unbelievable that
25:59
like, so few people are doing feature work this way
26:02
Yeah, I'll say, like, you know
26:03
it took us actually quite a While to
26:04
get to that point
26:05
I feel like it's actually quite hard to
26:06
get to that point overnight
26:08
It involves, you know
26:09
investment into making sure you have the right skills
26:12
you know
26:12
your code base has good guardrails and constraints so that
26:16
you know
26:16
the time that you would otherwise have to
26:18
spend on human code review
26:20
is encoded into your code base and your
26:23
you know
26:23
static analysis and your tooling so that the engineer's time
26:27
yeah, like PSAC
26:28
And so your engineer's time is freed up
26:30
to actually focus on the actual product
26:32
you know
26:32
the user experience
26:34
and the operational execution part of the job
26:37
You know
26:37
when you have so much more trust in
26:38
your agent's ability to write good code
26:41
because you've set up those systems
26:43
I've gotten to a point where
26:45
I essentially have my agents merge their own
26:47
pull requests. So in that Slack thread example
26:50
you know, like that was a
26:51
that was an example of
26:53
you know, me not reading the PR and this
26:55
I, we just looked at the
26:56
the artifacts that came out of it
26:58
So this cursor environment
26:59
does it already have P stack and is
27:01
that the way it sort of creates You know
27:04
decides how to build it or
27:06
Yeah. So, for internal
27:09
PSAC is installed into our repo
27:12
There's an internal version of PSAC
27:14
like instead of calling it potato mode
27:16
it, it's Lauren mode
27:19
and it just started because
27:20
you know
27:20
I wanted a place to put my skills
27:22
And so, yeah, everyone
27:23
everyone in the company can actually use PSAC
27:26
and my cloud agents
27:27
that the bots spawn have the same access
27:30
to those skills. Wait
27:31
you haven't really explained what PSAC is
27:32
Can you explain what
27:33
what PSAC is for people who don't know?
27:36
PSAC is actually inspired by GSAC
27:38
Oh, thank you so much
27:39
I'm honored
27:40
plugin
27:42
you know, I think
27:43
Gary, your, you know
27:44
GSAC was like quite inspiring in the sense of
27:46
I thought of it as like your personal set of
27:49
you know, chef knives. Oh, yeah
27:50
Right. Like, you know, every chef, when
27:52
when they get a new job
27:53
they go to a new kitchen
27:54
they bring their knives with them
27:56
And to me, you know, GSAC represented, you know, your opinionated take on how
28:02
you want your agents to
28:03
you know, Help you
28:05
And so that inspired me of
28:06
to think about, you know
28:08
like what would a G stack look like for
28:10
you know, design more for like the engineering flows that
28:14
that I personally like
28:16
So I often tell people like you should
28:17
you should actually go and create your own stack
28:20
Go, go fork it, you know, take our skills, combine them
28:23
you know, find what works for you
28:24
But I think the trust element is so
28:26
important. Like when you trust the
28:27
you know, your own tools
28:29
it empowers you and your agents to do
28:31
so much more
28:32
I didn't really answer the question
28:34
But G stack stands for potato stack
28:36
I guess
28:37
And it's my set of skills that I've
28:38
open sourced on the cursor plugins repo
28:41
And it teaches
28:43
agents how to do
28:44
engineering work rigorously the same way that
28:47
I did when I was writing code
28:49
and it's kind of evolved a little bit and yeah
28:52
I guess it's gotten quite popular
28:54
Congrats. Yeah, it's awesome. Yeah, I
28:56
I am very inspired by it
28:58
A G stack is like pulling things from
29:00
it whenever I can. So
29:01
Yeah, do you
29:03
do you have a software
29:04
That like
29:05
pulled in some ideas from P stack automatically
29:07
Oh yeah, yeah, yeah. I mean now I use
29:09
my software factory is Cappy
29:10
and then
29:11
I have a captain that is sort of maps to
29:14
I think the Grok bot
29:15
the bot idea
29:17
but
29:18
honestly like now I think like maybe I
29:19
need to use a bot on top of Cappy
29:22
But yeah
29:22
I basically
29:23
can just
29:24
go to that captain and it's like the
29:26
master of my G stack
29:29
like sort of merge queue
29:30
Like
29:30
I
29:31
basically have like 10 or 15 threads working
29:33
all the time and then one of them is like Theo.gg
29:36
he released T stack and then
29:39
P stack and then obviously there's like lots
29:41
of really good inspiration out there
29:43
especially around design tools
29:45
And so it's really fun to just say
29:47
hey, go out
29:48
find
29:48
what's out there and like based on what
29:50
you know about me
29:52
pull in the things that you think would
29:53
be most useful to improve my workflow
29:57
and then the really interesting thing about Cappy is
30:00
because all of my interactions with
30:03
my agents My
30:04
like coding agents are right there in the
30:06
transcripts. It's developing an understanding of like what I want
30:10
like when do I intervene
30:12
when do I say just do it
30:14
And so yeah
30:15
there's sort of this like very personal element
30:17
that's emerging
30:18
really through like the idea of memory
30:20
And so as you use the agents more
30:23
the more it knows what you want
30:25
Thank you for making P stack
30:26
And then both of us are
30:28
very
30:28
maligned and attacked on the internet by people
30:30
who refuse to believe that any of this
30:32
actually works. But you know
30:33
it's great to have a very brilliant compatriot
30:36
I'm not crazy. Like
30:37
this is actually something that is happening
30:39
And honestly
30:40
like I'm so convinced that in five or 10 years
30:43
or even like in two years
30:44
like it's going to be unfathomable that like
30:47
people are not doing it this way
30:48
But we're so early though still
30:51
One interesting change that kind of
30:53
came up as you were sharing your screen
30:55
is it feels like in the past six months
30:57
so much of the work
30:58
engineering work
30:59
non-engineering work with agents is happening in Slack
31:03
And I think it has this interesting property
31:05
of since it's out in the open
31:07
in the past
31:08
the way that Lauren would work with coding
31:10
agents would have been entirely single player
31:12
private
31:13
The rest of the company would not have
31:14
benefited from
31:16
all of the crazy workflows that
31:18
they should totally learn from and adopt for their own
31:21
their own purposes. But now that it's happening in Slack
31:24
I've seen this, and maybe you have too
31:25
time and time again
31:26
where you're in some feedback thread
31:29
someone has a good idea
31:30
you spin off an agent
31:31
like at Potato, you know, Lauren Mode
31:34
and then the next day I see someone
31:35
who was in that thread do the same
31:37
thing
31:37
And I think
31:38
we're not just going to see that in engineering
31:39
I think we're going to see that
31:41
in the way that people work with AI
31:42
in general. And the more that the stuff is in public
31:44
I think more
31:45
the more of the rest of the company
31:47
that can get lifted up by
31:49
the top 1% power users who are very tooling minded
31:52
very engineering minded
31:53
and know exactly how to use these tools
31:55
How much of your
31:57
of your, like
31:58
hours spent using Grokbot
31:59
is in the Grokbot app versus in Slack?
32:02
Or Some other
32:03
service area to talk to the bots there
32:05
I think a lot of the engineering work
32:07
that I do now
32:08
actually happens on Slack and X
32:11
So not all my bots have been added to Slack
32:14
so there are still a few that I kind of
32:16
that are more personal. Like I have a
32:18
I guess like a sort of personal assistant
32:19
bot that helps me with like
32:20
you know, picking my lunch
32:22
I don't need to add that to Slack
32:23
That's kind of, you know
32:24
personal or something that looks at my email
32:26
That's not a team bot
32:27
That's not a team bot
32:28
I guess we could
32:28
that's a personal bot. We could probably get
32:30
like a shared lunch bot
32:31
sure
32:32
I think that the nice thing as well
32:33
is like you can
32:34
you, you
32:34
you decide and choose what you want to
32:36
add to Slack
32:38
so my engineering bots tend to be
32:40
on Slack
32:41
so because I just find it so much
32:42
more ergonomic to just add the bot in the Slack thread
32:47
you know
32:47
versus me having to go copy paste the link
32:50
So it
32:51
it just felt more
32:52
natural to bring them in that environment
32:54
I guess it really fits with your analogy
32:56
of bots as employees. Like if Slack is
32:58
where the work happens
32:59
that's where the human employees are
33:01
It's also where the like bot employees should
33:02
be so that they can work alongside us
33:04
Totally
33:05
And we don't shy away from that at
33:06
all
33:07
Like I think when we see people use Grokbot in Slack
33:09
we love that. I think that's
33:11
that's where work happens
33:12
right?
33:13
And I think similarly
33:14
you know
33:14
in the future they should join meetings
33:15
They should be in your Google Docs
33:17
They
33:17
our goal is not to
33:19
rebuild all of B2B sass
33:21
you know, like other companies are
33:23
are doing a great job there
33:24
Our goal is to build the thing that
33:25
can access those tools and work inside of them
33:28
and just fit naturally into the way
33:30
a company already works
33:32
I guess one thing that
33:33
I'm curious about is
33:35
where do you think the future Let's say 2027
33:40
how should the best AI build engineers build?
33:43
So my hope for 2027
33:45
I think is like much more
33:46
you know, like we've
33:48
maybe accomplished more of this vision that we have for
33:51
for your robots being
33:52
like a coworker, you know, to have their own identity
33:55
their own telephone number
33:57
credit card, like basically almost anything
34:01
a human can do
34:02
I think we would want bots to do
34:04
And
34:05
I'd be very curious to see like how
34:07
this evolves
34:09
you know, I asked this question on my
34:11
on my X profile the other day as well
34:12
of like
34:13
what is the final form of agent UI?
34:17
And I, I posed the question of like
34:18
is it a chat interface or is it
34:20
something else?
34:21
one thing actually pretty interesting that I saw
34:23
yesterday on my X timeline
34:26
is a post from the Neuralink
34:29
team, and just came out with this new product called Telepathy
34:33
that I think, if I, if I'm not wrong
34:36
is along the lines of
34:37
in the future, an agent Being able to
34:40
essentially like read your brain
34:42
your mind through like
34:44
you know, a brain computer interface
34:46
and I'm
34:47
I'm
34:47
super interested in like what the future will bring when the
34:51
the sort of barrier between you
34:53
and how you chat and communicate intent with
34:55
your bots becomes so much more fluid
34:57
That can happen like literally at the speed
34:59
of thought
35:01
So that's personally
35:02
I think could be
35:02
very interesting to see how that evolves
35:05
But I don't know
35:05
what about you, Roman?
35:07
We talk a lot about
35:08
the software factory and then
35:11
the factory for other parts of the business
35:13
And I think it's worth asking
35:15
like, what is different between those two?
35:17
And there are a lot of differences
35:18
And I think the next few years will
35:20
be on the product side
35:22
on the model side
35:23
like really trying to bridge the gap
35:25
I think one of the big differences is code
35:27
you know, it compiles or it doesn't
35:29
and it passes CI or it doesn't
35:30
And I think there's just more
35:32
infrastructure built around the way that code is written
35:35
tested
35:35
deployed
35:36
than knowledge work
35:37
and then like the rest of the company
35:39
There's just no real analog for those things
35:42
I think second is code
35:44
like we know how much
35:45
AI models benefit from the right context
35:48
And
35:49
code and software has the benefit of
35:51
the context living in a code base
35:53
But for the rest of the company
35:54
so much important information is stuck in people's
35:56
heads
35:57
It's stuck in pen and paper
35:59
It's stuck in some filing cabinet or some thing
36:02
you know, behind a password that you forgot
36:04
whatever
36:05
And I think we're seeing the limits of
36:07
that. That too
36:08
And so I'm pretty interested to see the way that especially
36:11
you know
36:11
a YC startup
36:12
that is starting very much from
36:14
first principles from scratch
36:16
and
36:16
can maybe build their company in a way
36:18
that it's not just a software factory
36:20
but it's a factory across the whole company
36:22
Like what changes
36:24
will that have on company building?
36:26
What decisions will they make that
36:27
a company that was founded two years ago
36:29
maybe would not have
36:30
have made the same decision or trade-offs?
36:32
so I'm, I'm particularly interested to see that
36:35
We've been talking a bit
36:36
a bit about this concept of the
36:38
the IT native company
36:39
right?
36:39
And I think this
36:41
Got a lot more attention when Jack Dorsey
36:43
made that post
36:44
about how really companies should be
36:47
Block needs to be a
36:48
an AGI for, payments
36:51
Right. It's like you're
36:52
you're building a mini AGI and the whole
36:54
point of it is like Karen feeding of
36:56
the AGI that does this function for humanity
37:00
Right. I mean, the, the way
37:01
Jack Dorsey talked about it is like
37:03
you know, organization will get
37:05
get rid of
37:06
pretty much all middle management
37:07
and it would
37:08
basically boil down to just three roles
37:10
You would have sort of the
37:12
everyone becomes an IC because you just ship
37:14
everything. Everyone comes with outcomes
37:17
as opposed to the marketing team or growth
37:18
team working with engineering to build something
37:20
They could just build it themselves
37:22
And then the next role would be sort of the
37:24
the DRI that owns the outcome
37:26
It's almost like orchestrating all the ICs
37:29
And then you have this new role
37:30
which I think it might be just temporary for now
37:32
the player coach
37:33
which is someone that's super AI-pilled and teaches
37:35
everyone. Sounds like you are that role pretty much
37:37
Lauren. And then it's like
37:39
oh, show me the way
37:40
and then you level up the whole org
37:42
because it's like
37:43
this is how you call potato bot or matcha bot
37:46
right?
37:46
One of the cases was their customer success team using Grokbot
37:51
these team bots
37:52
So it's not just
37:53
a
37:53
single player
37:54
thing that everybody on the team works with
37:56
It's really this colleague that is built and
37:59
inserted into Slack
38:00
And I think in this case
38:01
it fields every customer request
38:03
I think they get hundreds of messages from customers a minute
38:07
way too much for a human to parse through
38:10
and triages it
38:11
signs it to the right human
38:13
I think for a lot of these tickets
38:15
AI could probably complete it end to end
38:17
And so for many of those
38:18
Grokbot does
38:19
and then it escalates it to the right
38:20
DRI
38:21
But I think your general point of
38:23
how companies can retool themselves for this moment
38:27
I think you're spot on that
38:29
I think a lot of companies are going
38:30
to spend more time on engineering objective functions of
38:36
you know, in a world where execution is kind of cheap
38:39
or you could spin up
38:40
a hundred coding agents or a hundred Grok
38:42
bots to like run at a problem
38:44
it really matters that you define the problem
38:46
well
38:47
And the
38:48
gap is probably not going to be
38:50
an execution failure. It will probably be
38:53
a intense specification failure or a goal failure
38:57
And how you quantify these things
38:59
I think you can quantify a performance improvement of
39:01
you know, your desktop client
39:04
But I think it's a little bit harder
39:05
to maybe define
39:06
what an improvement to like
39:07
your marketing would be
39:09
And I think a lot of companies are
39:10
going to have to think about what those KPIs are
39:12
how you can point your agents in that direction
39:14
who's responsible for defining those things
39:17
And I think we've been living in
39:18
a bit of an execution bandwidth constrained world
39:21
And now we're going to shift
39:23
to execution abundance
39:24
and the types of questions that companies think
39:26
about will probably be quite different
39:28
Maybe that
39:28
Scarce resource is going to be more that
39:32
higher executive function of what to do and
39:34
why
39:35
And this is where
39:37
more
39:38
builders are going to have more the founder
39:40
traits of really knowing
39:43
Translating user intent well into what
39:46
the bots need to do
39:47
Because I think that
39:48
that
39:49
there's a big gap between like the best
39:50
AI builders to
39:52
the so-so
39:53
ones is really how you translate that intent
39:55
super well. And
39:57
there's a lot of tips and tricks of what you did
39:59
Laura. I mean, your video is pretty impressive
40:00
Is there like one
40:02
one super important, just like if you do this
40:06
you'll get started. Like I think
40:07
was it like your
40:08
you know
40:08
what's the minimal form of a loop?
40:10
This is the smallest loop essentially
40:12
Like for agents already
40:13
are able to write code
40:15
Maybe it's not good. Let me
40:16
scratch that out for now
40:18
So like, you know, like naively if you're using agents
40:21
you can definitely write code
40:23
You can run the code and
40:24
you know, today that
40:25
that if you haven't spent a lot of
40:26
time in your agent setup
40:28
you may be the person
40:30
you know, running the code
40:31
And so like this is an actually a loop
40:34
really because, right, right now this is like the
40:36
the missing piece, the agent
40:38
Can't do this, but you were doing it
40:41
So the, the smallest loop I find is
40:43
you know, like actually focus on this part
40:45
How do you teach your agent to
40:47
you know, run the code empirically
40:50
you know, take CPU traces, you know, like performance metrics
40:54
like run the code
40:55
and run the application like a real user
40:57
would
40:57
The reason why I think it provides a
40:59
loop is because the agent now has the
41:00
ability to check its own work
41:02
So it can find the issues with the
41:03
code that it wrote
41:04
then it can write better code or more correct code
41:07
and then it can keep
41:09
looping until it produces a result that you
41:11
want
41:12
I took out the good part for a reason
41:14
because I think that
41:15
you know, when you have a loop like this
41:16
it doesn't guarantee that the quality of the code is
41:20
is good, but it certainly will be correct
41:23
You know
41:23
correct in the sense that it does the
41:25
thing that you want it to
41:26
but maybe it's like very slow
41:28
or very resource inefficient
41:30
And that's where, you know
41:31
the engineers sort of come into play
41:33
It's like
41:34
the new job of the software engineer is
41:35
to figure out how to get this to happen
41:38
Right
41:38
like how do you get agents to write
41:40
good code by default?
41:42
and so the loop gets smarter and smarter
41:46
and, when you're happy with it
41:49
I think that's the point where you can start thinking about
41:51
you know, the other parts of the loop
41:52
like, you know, the outer loop part
41:54
where, you know, like once this part is really good
41:58
you can now do things like
42:00
for example, just a, a simple example here
42:03
you have an alert for some metric or monitor that's
42:06
you know, going below a certain threshold
42:09
Now an agent, like a graph bot
42:11
a cursor automation
42:13
could be fired off as a result of that alert
42:17
and then it can go off and use the same
42:18
you know
42:19
tools and skills that you've provided for the basic loop
42:22
and it can repro
42:23
you know, reproduce the issue in its own machine
42:26
and then, you know, like it runs the code
42:27
and then it finds the issues of the code
42:29
and it writes the code
42:30
and then now you have this
42:32
like, self, sustaining loop, and that frees up my time
42:36
right, from, you know
42:38
being
42:38
Babysitting every agent
42:40
to now orchestrating multiple
42:42
agents and I even now have my bots
42:44
orchestrate like I've gone another level
42:46
where
42:47
I you know like I said we have
42:48
this new feature called projects
42:50
that lets you have a
42:51
project coordinator agent
42:53
that can orchestrate and manage like sub-agents
42:56
I don't even manage the project myself anymore
42:58
I have my bots do that
43:00
I feel like
43:01
since we have, you know
43:02
some of the world's experts at software factories here
43:05
I want to ask some
43:06
like, nitty gritty questions, if that's okay
43:08
Because I'm struggling with this right now with GStack
43:10
Like
43:10
I have a bunch of agents that are
43:11
struggling with the code review step
43:13
I have, like, a slash ship skill that
43:17
tries to go through every line of changes
43:20
and then basically spots bugs
43:22
And then, right now, it's very thorough, and then, but when I use it
43:27
it, always consumes, like
43:30
tens of millions of tokens
43:32
and it spots bugs
43:33
actually
43:34
And so now I'm
43:35
like, sort of struggling with
43:36
I mean, this is
43:37
match pair to, like, in the plan, like, I still use plan mode
43:40
and I have all these plan skills
43:41
I don't, do you still use plan mode?
43:44
no. No plan mode
43:45
yeah, yeah, yeah
43:46
So then, but, PStack must have, like
43:48
a review step then
43:49
How do you, what's your philosophy on
43:51
like, how much review is enough versus too much?
43:54
Like
43:55
I'm literally in a debate with one of
43:56
my cappy captains right now
43:58
where it's like
44:00
Wanted to go from 50 million tokens for
44:02
review to like 10 million
44:04
but it
44:05
it has all these held out evals that show that
44:07
you know
44:08
we have
44:08
like 30 types of bugs that I'm trying
44:10
to make sure I can
44:11
find in that review step
44:13
And it's like
44:13
I can't do it with 10
44:14
I need 30. Like
44:16
I went from 50 to 30
44:17
Can we ship 30? And I'm like
44:18
no, no, no, you need 10
44:20
What do you see? Like
44:21
I mean, what's the state of the art?
44:22
That's a really good question
44:23
I think
44:24
So for me, actually, in P-Stack, I have tried to
44:27
downplay some of the review skills a little bit
44:30
You know, in P-Stack, we have a bunch of skills
44:32
like there's one called interrogate
44:34
and that will spawn
44:36
agents of different models. So like
44:37
you know, like across different families
44:39
Which works really well across a lot of
44:40
different evals. And you can do like
44:41
adversarial review
44:42
That was like quite hot like many months
44:44
ago
44:45
I've actually kind of
44:46
withdrawn a little bit from that because I
44:47
find that when you do that
44:50
I mean, it's still valuable
44:51
but I think you run the risk of overfitting
44:54
where
44:55
the sort of agent doesn't have the
44:57
ability or maybe enough judgment to determine what's
45:00
actually good feedback versus not good feedback
45:02
And it
45:03
maybe doesn't have complete knowledge of your goals
45:06
and your intents and your
45:07
you know, your, your wider product vision
45:09
It doesn't understand how you want the product to evolve
45:11
for example
45:13
And so you can get to a point where you are
45:15
you know, very abstractly, you know, it's
45:18
it's kind of like imagine before agent
45:20
you have a bunch of engineers talking about the
45:21
the most perfect ideal utopia way of doing this
45:26
But actually, you know, that doesn't even matter because in
45:28
in two months we plan to replicate this
45:30
thing and build a completely new vision
45:33
Right. And so I think there's some
45:34
anal- similar analogy there where
45:37
I think some review is good
45:39
but I try to move more towards
45:41
like on my screen here I have this inverted pyramid
45:46
Where, you know, like I try to push
45:48
like you, you mentioned, you know
45:49
spending less tokens
45:51
And so from the engineering point of view
45:53
I like to think of this as
45:55
you know, your
45:55
your code base is a form of memory
45:58
And so it's much more token efficient if
46:01
you know, you refactor your architecture
46:03
your API design
46:05
or you have, you know, things like static analysis
46:09
you know, lit rules
46:10
compilers, all of these, you know, languages, programming languages, these all exist to help
46:16
you know, in the past
46:17
like humans write good code
46:19
And unsurprisingly, they also help agents
46:22
right? And so
46:23
when you can encode some of these things that you are
46:26
you know, you know, arguing with, with your, your captain about
46:30
back into the code base
46:32
that's just like one less problem you now have to
46:35
to think about
46:36
You just gave me a feature idea
46:37
which is like
46:37
because GStack has office hours and like the
46:41
CEO review where you're like talking about the product
46:43
I can use that. I mean
46:45
cause those are. Are like product requirements that feed in
46:48
and those save to mark down in the
46:49
project. So I could have the review skill
46:52
I think it already does this a little bit
46:54
but I could be like
46:55
no, no, no
46:56
really
46:57
use that to figure out what we should
46:58
review and
46:59
what counts as a real bug to fix
47:01
versus not. Because there are like
47:03
general correctness bugs that are just like
47:05
they don't matter. Like they might only happen
47:07
like 0.1% of the time or maybe never
47:10
So it sounds like This
47:12
this triangle shows us that
47:14
you know, the code base is quite large
47:16
and then, like, you have very little time
47:19
Yeah, I think, so I
47:21
I intentionally drew it this way
47:24
I guess it's, it's not a very scientific chart
47:25
of course, but I, it
47:27
the way I imagined it when I drew it
47:28
it was, it was, I think, like
47:30
this is in terms of
47:31
where you should spend your time as an engineer
47:33
So, for me, you know, like
47:35
most of the investment I make is into
47:37
the code base and then static analysis
47:39
and I still layer that with
47:40
you know, rules and skills, you know
47:42
P stack updates and agents dot MD
47:45
and we still do kind of
47:46
like, a little bit of human view from
47:48
from time to time
47:49
I think, you know, if you're working on
47:51
like, more critical parts of
47:52
like, infrastructure or databases, deployment, you know, like, kind of more
47:58
you know, sensitive things
47:59
then you might actually still want to stay
48:00
in the loop
48:02
but I think a large class of
48:04
your regular product development or soft
48:06
software maintenance, like an investment here
48:09
actually ends up
48:10
saving you a lot of tokens
48:12
because like for example
48:14
you know, with the, the GraphBot desktop app
48:17
it's an Electron application
48:18
and one of the lessons we learned from
48:20
building Cursor is that
48:21
the render thread on Electron is very sensitive
48:25
to, long tasks
48:27
Yeah
48:28
so like you can get very janky performance
48:30
if
48:31
your tasks take more than like 16 milliseconds or 8 milliseconds
48:34
right
48:35
if you want to hit a 60 FPS
48:37
Smoothness
48:38
We
48:38
literally just built that for when we were
48:40
refactoring and building GraphBot
48:42
We decided
48:42
we don't want to think about this problem
48:44
anymore. We don't want to have to catch
48:45
it in code review
48:47
We'll actually build it as like a lint
48:48
rule. We built a dependency graph checker
48:52
And now it's just enforced by the code base
48:54
enforced by CI
48:56
So
48:57
it's tokens I don't have to spend anymore
48:58
It's somehow enforced in static analysis
49:01
Actually like
49:02
can know how long something will take to run
49:06
Oh no, it's more like so for the GraphBot app
49:08
we have code that runs on the main
49:10
thread and then the renderer thread
49:12
And so we explicitly
49:14
forbid importing between those two things
49:17
So it's not necessarily that
49:19
you know
49:19
main code thread will always be long
49:22
It's just like oftentimes
49:23
you know, some like heavier tasks that shouldn't
49:26
you know, block the renderer
49:28
Never do long work on the renderer thread
49:31
man. Yeah
49:31
you want your renderer to be as light as possible
49:35
you know
49:35
So
49:37
Yeah, that's something we, we just don't
49:38
we don't think about anymore
49:39
And I think the performance of the app
49:41
has actually stayed very consistent and very good
49:45
which is
49:45
I think is actually quite
49:47
surprising for a lot of like AI
49:49
majority AI written applications where
49:52
you know
49:52
they start out fast and then over time
49:54
they just
49:54
accumulate a lot of slop and performance problems
49:58
With the right setup
50:00
I think you can actually
50:01
you can
50:02
you can have quality and you can have
50:03
quantity
50:04
I think you can have your cake and
50:05
eat it too. The cool thing about the
50:06
moment in time right now is that you
50:08
you could basically
50:09
there used to be
50:09
the trade-off before that
50:11
you could only choose two out of three
50:12
It was either like fast
50:14
cheap, and high quality
50:17
Now you could basically do all three
50:19
and we could basically do the best work
50:22
in our lives right now with AI
50:24
which is super cool. Yeah
50:25
I totally agree
50:26
I think, yeah, I think fast, cheap
50:28
and good is certainly more possible
50:30
It's not, it's not, you know, like, easy, I think, to get to that point
50:34
but I, like, from my own Experience
50:37
you know, I have the privilege of working at
50:40
a, a lab with, you know
50:41
which has given me unlimited tokens
50:43
so I think I've
50:44
I've, you know, I'm sort of
50:45
I sort of see myself as the explorer
50:48
you know, like trying to figure out what will
50:49
what does software engineering look like
50:51
six months to a year from now?
50:53
And I think, you know, this
50:54
this process has been very
50:56
enlightening for me in terms of
50:58
how to actually Keep this workflow efficient
51:02
because I think it's a
51:04
it's a topic that lots of teams and
51:06
and people are thinking about is the
51:08
the cost of the
51:09
of tokens. And I think when you adopt
51:12
you know, like, like, the inverse pyramid, like I'm showing here
51:16
I think you can actually get to this
51:18
a very nice mix where
51:20
you know, the ordinary engineering tasks
51:22
high quality engineering work doesn't have to break the bank
51:26
right? It can be
51:27
quite economic and quite efficient
51:29
And not to mention that the frontier keeps
51:31
getting smarter and smarter
51:33
and actually the, I guess like the
51:35
the unit cost of intelligence seems higher
51:39
Like if you compare a model from
51:40
you know, last year to a model now
51:42
like the input output tokens
51:44
is not really the same
51:46
Like one, you know
51:47
one frontier token now
51:49
is way more capable
51:50
than a token from last year
51:53
And even if you use one of the smaller models
51:55
like even that model is like way better
51:56
than what you had a year ago with
51:58
the big models
51:59
So I think from that Perspective
52:01
We're heading towards a period of
52:03
of real abundance, I think
52:04
with software and maybe other domains as well
52:08
And I think we're seeing this on
52:10
our core team that works on Grokbot
52:12
is, I mean, Lauren is putting up 4,000 PRs a month
52:17
and I think we're going to start to
52:18
see more engineers do that
52:20
And
52:21
I think what we've started to see be
52:23
the bottleneck is less
52:25
the
52:25
time or engineering resourcing
52:27
going into developing these ideas and features
52:30
And actually
52:30
it's more alignment among us as a team of like
52:34
what do we want the product
52:36
roadmap to be? What's our vision?
52:38
What does good look like?
52:38
What does great look like?
52:40
and I think we're seeing this where we've
52:41
been in so many discussions
52:43
Lauren and I were
52:44
we're kind of talking about a future change to Grokbot
52:47
and someone suggests something
52:48
and then someone else in the room says
52:49
oh, I have a PR for that
52:50
if you're interested. Like
52:51
I was working on this over the weekend
52:53
whatever
52:54
And that happens constantly. So the stuff is built
52:56
but we don't want to merge all of that
52:59
because it's going to lead to
53:00
Bit of a Frankenstein-y product with like random
53:02
side projects. Like we
53:04
we do need a consistent
53:05
vision and philosophy of the product
53:07
And so I think we're shifting more of
53:08
our time
53:09
to that and those questions
53:11
now that everybody is making thousands of micro
53:13
decisions a day or a week
53:16
which I think was not the case in the past
53:18
Which, actually I'm very curious
53:19
I mean, this is a
53:20
this
53:21
is a bit of the trendy thing now
53:22
with the thing that matters is taste
53:25
which means actually saying no to a bunch
53:27
of stuff and having discernment
53:29
What does that look like in practice for
53:31
for you guys? How do you
53:32
how do you make a decision yes or
53:33
no on this? Yeah
53:35
For a certain category of things
53:36
so Lauren has set up a
53:38
one of my favorite parts of her factory is Sandy
53:41
which takes in
53:42
thousands of pieces of customer feedback every day
53:45
We receive it through the product
53:47
We receive it through our support channels
53:48
We receive it through X
53:50
And a lot of times there are just
53:51
small bugs. And so in a lot of cases
53:53
Lauren's factory just fixes it
53:55
like repros, fixes, merges
53:57
No one thinks about it
53:58
It goes out automatically at three in the
54:01
I wish all software
54:02
would be made that way
54:03
I wish Apple, Apple, people who work at Apple
54:05
could you do this? Because
54:07
hey, I have, you know, calendar team in particular
54:09
I have some problems with you
54:10
There's a lot of
54:11
Grokbot that should be self-healing in that way
54:15
and it's cool that that finally works
54:17
I think there's another set of questions that
54:19
should not operate that way at all
54:21
if you're changing the core pixels that a
54:23
user is experiencing every day
54:25
it's like really frustrating as a user to wake up
54:27
and you refresh your Grokbot
54:28
and there's a whole new
54:29
the button moved, and then it moved again the
54:31
the next day, and
54:32
and so
54:33
the way we try to counteract that is
54:35
a lot of dogfooding internally
54:37
and sitting with a change for longer than
54:40
feels comfortable before we ship it
54:42
And the default is if something doesn't get
54:44
internal product market fit
54:46
we don't ship it
54:47
Even if it gets like mixed internal product market fit
54:49
we don't ship it
54:50
And so as a team
54:51
we really try to keep a high bar
54:53
Yeah
54:53
I think that's actually one of my favorite
54:54
parts of working on the Grokbot team is that
54:57
you know, despite the, the great velocity that we have
55:00
Still have a lot of intention and care
55:02
for the user experience
55:04
And, you know, even though you can go really fast
55:06
it doesn't mean that you should just ship
55:08
every feature you can think of
55:09
And this is where human judgment and taste
55:12
I think, comes in
55:14
I don't think that's gonna
55:15
go out of style personally
55:17
I think as long as we're building software for humans
55:20
humans should really be thinking about the
55:22
you know, be an intentionally designed experience
55:25
because we know each other the best
55:27
right?
55:29
you know, if you're building software for agents
55:30
that's a different story
55:31
that, you know
55:32
you can certainly Probably go like as fast as possible
55:35
but I think yeah for humans
55:37
you do want to have
55:38
put intentional care and thought
55:41
into the stuff that you develop
55:43
So even for my own factories I don't
55:45
turn them towards like feature development
55:47
I turn them more towards the
55:49
software maintenance side of things
55:52
because I care a lot about you know performance
55:54
I care a lot about
55:55
fixing bugs that our users
55:57
encounter or quality of life
55:59
complaints that they have
56:01
and I think
56:01
that's actually such a great use of
56:03
of the the agents because
56:05
that used to be like a full-time job
56:07
or something and now I've given that to
56:09
a lot of agents and
56:11
I that frees me up
56:12
and other engineers up to
56:14
think more intensely about the product experience
56:17
One thing that's very inspiring about your journey
56:19
Lauren, I don't know if many people know
56:21
is
56:21
you you grew up in Singapore
56:23
and you studied a non-technical degree
56:26
You did a bachelor's in accounting and finance
56:28
from Monash in Australia
56:30
Then somehow you ended up becoming one of
56:32
the Best AI engineers. Yeah
56:36
How did that happen? I think sometimes people
56:38
think it's like it's very hard to get into tech
56:40
but
56:41
you did it and you also worked on the React compiler
56:43
which is one of the hardest things as well
56:45
Yeah, I think I've definitely feel like I've had a very
56:49
privileged and lucky experience into tech
56:52
I think that was a time where
56:54
you know
56:54
people were much more open to the idea
56:57
of like a self-taught developer
56:59
So I guess the
57:00
the long story short is
57:01
you know, after college, I did
57:02
I did study finance
57:04
one of my favorite courses was actually financial modeling
57:06
and we learned how to use Excel and Visual Basic
57:10
BBA to, you know
57:11
make some cool
57:12
financial valuations and things like that
57:15
and through the course of that
57:16
that course
57:17
I realized that I had more fun building
57:19
the spreadsheet and the macros than I did the
57:22
the finance part. I mean
57:22
the finance part was interesting too
57:24
but the technical part
57:25
you know, thinking logically just was really interesting to me and
57:29
and something I hadn't really
57:31
thought I was cut out for
57:33
So after college, I, I started a startup with a friend
57:36
a, a classmate, and, it didn't go super far
57:41
but we learned a lot from that process
57:42
and I got really involved with the startup community in Australia
57:45
and that's how I got my first job
57:47
And really the secret I
57:49
that I attribute for my success
57:51
is actually X
57:53
I
57:54
started out when I was still basically learning how to code
57:58
I wrote, I had a blog
57:59
and I was just sharing what I learned
58:01
and I was fortunate enough that someone from the
58:05
the frame
58:06
the framework community that I was building my application in
58:09
the JavaScript framework, Ember, so yeah, Tom Dale and Leah Silber
58:14
they reached out to me and said
58:16
would you be interested in giving a talk
58:18
at the conference that we're
58:19
that's coming up in Portland
58:21
Oregon, and I was
58:23
very shocked because
58:24
I had never even been to a conference in
58:26
in my life
58:27
and then now I was being asked to
58:28
talk at a conference with
58:30
like, hundreds of people, and, you know
58:32
public speaking was always my worst fear
58:34
but somehow they convinced me to do it
58:37
I learned about the ability of
58:39
you know, being able to communicate your ideas and
58:42
bring people along with in
58:43
in that journey
58:45
and this skill of storytelling and
58:48
communication has really stuck with me
58:50
even though I've, you know
58:53
Been really involved with engineering work and
58:56
you know
58:56
I've had a lot of really great mentors as well
58:59
in the past
59:00
So I think all that to say is do cool stuff
59:03
you know, talk about the cool stuff and
59:06
you know, just keep doing that in the loop and
59:08
you know
59:09
maybe the stars will align and
59:11
someone will notice something you said on X and retreat you
59:14
offer you an opportunity that you hadn't
59:17
had before. Maybe you
59:18
develop your idea and you come up with a startup idea
59:21
right? And then you go apply to YC
59:23
and before you know it
59:24
you know, you could change your life with
59:26
with just a tweet
59:27
Yeah
59:30
Lauren and Roman
59:31
thank you guys so much for joining us
59:33
It was so fun
59:34
Thank you for making an incredible product
59:36
So
59:37
can't wait to see
59:37
what you build from here
59:39
Thanks for having us
59:40
It was a lot of fun

Watch these videos on your TV

XTTV brings video content from X to your living room. Download now and start watching on the big screen.