← Back to videos
It's been nearly 6 months since I posted a 'watch me work' deep dive on my skills. Since then, they

Matt Pocock
@mattpocockuk
November 18, 2026· 69:29
It's been nearly 6 months since I posted a 'watch me work' deep dive on my skills. Since then, they've become the most-starred skills repo in the world. So, here's 70 minutes of me building a 21,000-line PR, especially focusing on: - DDD with /grill-with-docs - Improving your codebase with /retro - How to do human review effectively Enjoy!
Transcript
- You've been asking again
- 0:01
- and again and again for me to
- 0:03
- show you what my development process looks like
- 0:06
- I've done a tiny little bit of it on this channel
- 0:08
- but not nearly enough. And since I did it
- 0:11
- I've come into possession of
- 0:12
- one of the most starred repos in the world
- 0:15
- which is my skills repo
- 0:16
- and certainly the most starred skills repo
- 0:18
- in the world. And so I do think
- 0:19
- I owe it to you to
- 0:20
- show off how I use these skills
- 0:23
- in something more than a tutorial
- 0:25
- in a real walkthrough. And in this walkthrough
- 0:27
- we're going to
- 0:28
- focus especially hard on two skills
- 0:30
- The first one is going to be the
- 0:31
- grill with docs skill. And
- 0:34
- I think grilling is really much more of
- 0:36
- an art form than most people assume
- 0:38
- In this walkthrough
- 0:39
- I really focus a lot more on the
- 0:40
- domain modeling aspects of it
- 0:42
- In other words
- 0:43
- the kind of classic DDD skills
- 0:45
- that allow you to basically say
- 0:47
- okay
- 0:47
- I've figured out the language of my application
- 0:49
- And so once I've figured out the language
- 0:51
- then
- 0:51
- I've actually figured out the app and figured
- 0:53
- out what I want to build
- 0:54
- In this video
- 0:55
- you see me go through that process on
- 0:57
- a real feature that I will
- 0:59
- use from now on. Even to build the
- 1:00
- thumbnail for this video. And the next skill
- 1:02
- I go deep on is the retro skill
- 1:04
- which is we look back on the huge chunk of sessions
- 1:07
- I think we do like 30 sessions in this video
- 1:10
- and we suggest a bunch of improvements
- 1:12
- I'm going to walk through a real huge
- 1:14
- I think 25,000 line PR
- 1:16
- I'm going to do some real human review
- 1:17
- on it and show you how that human
- 1:19
- review
- 1:20
- can then go back and influence and impact
- 1:22
- the system going forward. Now
- 1:23
- if you're interested in this stuff
- 1:24
- then the AI coding crash course that I
- 1:26
- have is
- 1:27
- absolutely the place that you need to be
- 1:28
- if you want a more structured approach
- 1:30
- But with that, let's get cracking
- 1:32
- Let's dive in
- 1:33
- So this is where all of my courses get created
- 1:36
- all of my videos get created
- 1:37
- It's a video editor
- 1:38
- a thumbnail editor, a AI writer
- 1:41
- It helps me schedule things
- 1:43
- It
- 1:43
- creates shorts. It does all sorts of stuff
- 1:45
- And the bit that I'm particularly interested in
- 1:47
- right now is this little thumbnail creator down
- 1:50
- here. This is where I can post the
- 1:51
- video that I'm creating and editing right now
- 1:54
- and I can create a new thumbnail
- 1:56
- And this is essentially a process where I capture my face
- 1:59
- and then And my face goes into the
- 2:01
- thumbnail and I can create a diagram with it
- 2:03
- For instance
- 2:04
- I just stopped recording so that I could
- 2:05
- take this still. This allows me to move
- 2:07
- this left and right
- 2:08
- And then in here in my diagram creator
- 2:11
- I can essentially screenshot this part of the
- 2:14
- window here
- 2:15
- And what that will do is I can
- 2:16
- then copy and paste it into the thumbnail
- 2:18
- So if I press control V here
- 2:20
- you can see it gets pasted in
- 2:22
- and then the cutout automatically gets cut out
- 2:24
- by this nice API
- 2:26
- I can position these and I have my
- 2:27
- thumbnail. Now this is pretty nice
- 2:29
- but what I would really like to do
- 2:31
- is remove a bit of friction here
- 2:32
- It's annoying having to swap over to the
- 2:34
- diagram view and like have my
- 2:37
- you know, it's like an annoyingly manual process
- 2:40
- especially if I get things wrong
- 2:41
- I can't quite position them correctly
- 2:43
- I have to go back
- 2:44
- I have to screenshot it again and try it again
- 2:47
- Essentially, I don't have enough flexibility here
- 2:49
- What I really want is to be able
- 2:50
- to draw on this surface here
- 2:52
- In other words
- 2:53
- have the diagram kind of integrated
- 2:55
- with the picture of me there
- 2:57
- I really don't have a clue on how
- 2:59
- I'm going to Build this
- 3:00
- I just know that it's something I want
- 3:02
- And so I thought it would make a
- 3:03
- good candidate for like a new feature or
- 3:05
- an improvement and to sort of show you
- 3:07
- the whole process end to end
- 3:08
- And so when I enter into a new piece of work
- 3:11
- I start thinking through this decision tree
- 3:14
- which is essentially what skill from my skills
- 3:16
- am I going to use
- 3:17
- In other words, you have essentially a new session here
- 3:19
- you want to create a new piece of work
- 3:21
- you don't know what to choose
- 3:22
- You first think through
- 3:23
- okay
- 3:23
- what is the thing I'm trying to create?
- 3:25
- And the thing I'm trying to create is
- 3:27
- essentially a diff
- 3:28
- The diff between the old code base and
- 3:31
- the new code base
- 3:32
- And if that diff is really small
- 3:33
- then you can probably just one-shot the thing
- 3:36
- So you don't need to worry about grilling
- 3:38
- before you continue. But in this case
- 3:39
- I know that the thing we're creating is
- 3:41
- pretty big. And so
- 3:42
- I'm going to go with grill with docs
- 3:45
- From there
- 3:45
- if it ends up being a lot larger than I thought
- 3:48
- that can always bail out into Wayfinder
- 3:51
- which is what I use for turning initial grilling sessions
- 3:55
- running them over multiple sessions
- 3:57
- developing this huge, great, big concept
- 4:00
- Of what we're trying to build
- 4:01
- and then going until we get there
- 4:02
- But I prefer starting with grill with docs
- 4:05
- because if it ends up being smaller than I thought
- 4:07
- then it means I don't have all the
- 4:08
- overhead of running a wayfinder session
- 4:10
- The place I'm going to run this is
- 4:11
- inside my personal wiki
- 4:13
- which is a home repo
- 4:15
- If you don't know what a home repo is
- 4:17
- then check out my accompanying video on that
- 4:19
- The agent I'm going to use is Claude
- 4:21
- Code because that's the one I have a subscription to
- 4:22
- that's what I'm set up for
- 4:24
- and that's kind of what I like
- 4:25
- I am on the Claude Ultra plan
- 4:27
- I have Opus 5.5
- 4:28
- and I'm running it on medium effort
- 4:30
- And of course, I have all of my skills installed
- 4:32
- and I also have a bunch of custom
- 4:33
- skills that are inside my home repo
- 4:35
- My home repo is not open source
- 4:37
- but most of the skills you're going to
- 4:38
- see are open source
- 4:40
- and they're at AIHero.dev forward slash skills
- 4:43
- So we're going to start with grill with docs here
- 4:45
- and then I'm going to explain my problem through dictation
- 4:48
- and I'm using WhisperFlow for that
- 4:50
- In the course video manager
- 4:51
- we have a thumbnail viewer
- 4:53
- and the thumbnail creator is okay
- 4:57
- I like that it asks me
- 4:58
- and it sort of
- 5:00
- Me to put my face onto the screen
- 5:02
- but what I don't like is the
- 5:04
- sort of rigmarole I have to go through
- 5:05
- for attaching a diagram
- 5:07
- Most of my diagrams are built in TLDraw
- 5:10
- and using the custom diagram playground that we have
- 5:13
- but I don't have a
- 5:15
- what I would really like is for the
- 5:16
- diagram to be able to be editable
- 5:18
- in the scene that I'm viewing
- 5:20
- so in the context of the thumbnail
- 5:22
- In other words
- 5:22
- I want my face visible and then
- 5:25
- to have the
- 5:26
- thing just editable next to me
- 5:28
- I really don't know
- 5:29
- UI-wise how we're going to
- 5:31
- handle that and what that's going to look like
- 5:33
- so I think there are a lot of
- 5:34
- questions to answer there. I want you to
- 5:35
- notice a couple of things about how I'm
- 5:37
- prompting there. The first thing is I'm thinking
- 5:39
- a lot about the why
- 5:41
- I'm thinking a lot about the problem that
- 5:43
- I'm trying to get the agent to solve
- 5:45
- I'm not necessarily giving it a
- 5:48
- you know
- 5:48
- decent sort of
- 5:50
- like idea of what exactly to do
- 5:53
- If I give it the why
- 5:54
- then the agent
- 5:55
- feels a lot more empowered to go away
- 5:57
- and actually produce a solution itself
- 6:00
- For maybe have a discussion about the solution
- 6:02
- Whereas if you come to it with a what
- 6:04
- saying I want to do X
- 6:05
- Y, Z
- 6:06
- then it will often not necessarily
- 6:09
- come up with that creative an answer
- 6:11
- And so we can see here it's come
- 6:12
- back with a bunch of grilling questions here
- 6:15
- It's asking first, what is the rigmarole?
- 6:17
- What is the part of attaching a diagram
- 6:19
- that hurts most? I would say that A
- 6:21
- leaving the thumbnail creator is annoying
- 6:23
- and B is also annoying
- 6:26
- and C is also annoying
- 6:28
- So I think you've put your finger on all three there
- 6:31
- to be honest. Is the thumbnails diagram the
- 6:32
- same as the videos diagram?
- 6:34
- When you edit the diagram inside the thumbnail
- 6:36
- should the changes go back to the diagram
- 6:37
- the video
- 6:38
- uses? This is kind of a question about
- 6:41
- where do we want this diagram stored
- 6:43
- you know
- 6:43
- like should it be inspired from existing diagrams?
- 6:46
- Because in the application
- 6:47
- you can see if I do a search here
- 6:49
- then I can actually search for existing diagrams
- 6:51
- So I've found like this Wayfinder diagram
- 6:54
- by just sort of like
- 6:55
- oh, I've got to search
- 6:56
- haven't I? So go to diagram
- 6:58
- Wayfinder, and then search all the diagrams
- 7:00
- Diagrams that I view. So this one
- 7:01
- for instance, is good. This one's another one
- 7:03
- So it's kind of asking
- 7:04
- how do we
- 7:06
- like associate these thumbnail diagrams with this list?
- 7:09
- And I'm not quite sure there
- 7:11
- With Q2, I'm not sure
- 7:13
- I'm kind of not sure of the trade-offs there
- 7:15
- really. I suppose the question is
- 7:16
- do I want the diagram to be
- 7:19
- sort of searchable inside the main list?
- 7:22
- Do I want it associated?
- 7:23
- I'm not quite sure what the use case is there
- 7:25
- Q3, starting from scratch
- 7:27
- should you be able to draw a
- 7:28
- new diagram straight in the thumbnail without any existing diagram?
- 7:31
- Q3, yes, I agree. That sounds good
- 7:33
- With Q3 too
- 7:34
- I kind of maybe want to be able
- 7:36
- to
- 7:36
- search existing diagrams. I like the ability to
- 7:40
- add components that we've already got
- 7:42
- maybe searching for stuff that we've already got
- 7:44
- or maybe I could just copy stuff in
- 7:46
- from one TL drawer board to another
- 7:48
- I want you to notice at this point
- 7:49
- how loose the language is that we're using
- 7:51
- to describe the feature
- 7:53
- It doesn't feel like we are particularly sort of
- 7:57
- we haven't figured out the right names for things yet
- 7:59
- and so I'm having Having to use a
- 8:00
- lot of weird verbosity and shortcuts
- 8:03
- it's not quite clear in my head still
- 8:05
- As we go through this grilling process
- 8:06
- the
- 8:07
- idea of what we're building is going to
- 8:08
- become clearer and clearer in my head
- 8:10
- until we
- 8:11
- end up with a nice set
- 8:13
- of words for the things we're building
- 8:14
- And once we're there
- 8:15
- we've kind of built the feature
- 8:17
- So Q4 says
- 8:18
- should there be
- 8:18
- one editing mode or two in the thumbnail?
- 8:20
- Should there be a single edit the diagram
- 8:22
- mode and a separate arrange mode?
- 8:24
- For Q4
- 8:25
- I think the process should probably be
- 8:27
- that we take a photo of my face first
- 8:29
- then that face gets arranged onto the canvas
- 8:33
- and then we, once my face is there
- 8:36
- then we can sort of build the diagram
- 8:37
- around my face and I should have the
- 8:39
- option of retaking it. For Q5
- 8:41
- is the face fixed? For Q5
- 8:44
- I think I should have the option of
- 8:45
- moving the face horizontally and From there
- 8:50
- that would just give me a bit more flexibility
- 8:52
- Q6, is this only about diagrams?
- 8:55
- Yeah, Q6
- 8:55
- let's make it diagram only for now
- 8:58
- I can't think of a use case where
- 8:59
- I want to attach other stuff inside the
- 9:01
- thumbnail
- 9:01
- We don't want to build a second Figma
- 9:03
- We just want to make this particular flow easier
- 9:05
- Okay, so while we were answering those questions
- 9:06
- the agent was going off
- 9:08
- spinning up a background agent
- 9:09
- and going to find the
- 9:11
- supporting information that we needed here
- 9:13
- So it's actually going ahead and changing two
- 9:16
- of the questions here. And then Q7 says
- 9:18
- should edit the diagram in place mean a
- 9:20
- live TL draw editor clipped to the
- 9:22
- thumbnail frame with the camera locked so the
- 9:25
- area you see is the thumbnail
- 9:26
- For Q7
- 9:27
- I'm concerned that
- 9:28
- cramping us into that smaller space
- 9:30
- will mean that editing the thing
- 9:33
- the thumbnail, actually becomes kind of painful
- 9:36
- And so
- 9:36
- I think we might need to prototype a
- 9:37
- few different approaches there. So that's a very
- 9:39
- detailed first response. Let's fire it off
- 9:41
- Now you might be looking at this approach
- 9:43
- and you might be thinking
- 9:44
- okay
- 9:46
- what benefits this have over approaches that I'm
- 9:48
- taking over like plan mode
- 9:49
- Let's say
- 9:50
- what this
- 9:51
- skill does and what this approach does is
- 9:53
- we align on the thing that we're building
- 9:55
- before we go ahead and build it
- 9:57
- This means that we're going to get a
- 9:57
- much more predictable output
- 9:59
- I'm going to have much stronger
- 10:01
- understanding over the thing that we're building
- 10:04
- and like while it's building
- 10:05
- I'm going to let it just sort of run by itself
- 10:08
- but this is the moment early on when
- 10:10
- I get to
- 10:10
- maybe with my teammates align on what we're building
- 10:14
- figure out what it actually is
- 10:15
- what the words we're going to use about it
- 10:17
- and then everything from that point all the
- 10:19
- way down to human review at the end
- 10:21
- is kind of going to be fairly on rails
- 10:23
- So Q2, it's reframing it, it's saying
- 10:25
- is the thumbnails diagram a real diagram?
- 10:28
- And at this point we should probably talk
- 10:29
- about what
- 10:30
- it actually means by a real diagram
- 10:32
- This is inside the course video manager repo
- 10:35
- and this is a glossary.md
- 10:37
- that is my pride and joy
- 10:39
- When I talk about language inside a project
- 10:41
- what I'm really talking about is the words
- 10:43
- that you're using to describe certain things
- 10:46
- And in this pretty massive file here
- 10:48
- which is about five 560 lines
- 10:52
- we define all of the main concepts
- 10:54
- that sit inside this course
- 10:57
- or sorry
- 10:57
- that sit inside this repository that are responsible
- 11:00
- for this project. And the stuff that we're
- 11:02
- really talking about here is a diagram
- 11:04
- and so there's a whole section on diagrams
- 11:06
- So a diagram is really a group of diagram snapshots
- 11:10
- which is a copy of the diagrams drawing
- 11:12
- at the moment that a clip is filmed
- 11:15
- In other words, in my video editor
- 11:16
- it's linked up to the diagram
- 11:18
- and so I can see what diagram I
- 11:20
- was using at each point in the video
- 11:22
- There is the active diagram
- 11:24
- so the diagram that the author is drawing
- 11:26
- on now. We have an action called restore to head
- 11:29
- when we
- 11:29
- load an older diagram snapshot back as the
- 11:32
- active diagram. There are preserved snapshots as well
- 11:34
- which are diagram snapshots kept in its diagrams timeline
- 11:38
- even when no clip uses it
- 11:39
- You can start to see that these definitions
- 11:41
- form a really nice picture of the feature
- 11:45
- And what we're kind of doing here is
- 11:46
- we are trying to add to this
- 11:48
- trying to figure out where our new
- 11:50
- Feature lives in this existing landscape
- 11:52
- So by asking what is a thumbnails diagram a real diagram
- 11:57
- we're kind of asking
- 11:58
- okay, it's a diagram
- 11:59
- but maybe it's private to the thumbnail
- 12:02
- For Q2
- 12:02
- I think it makes sense to have
- 12:04
- our existing setup here
- 12:06
- But what I'm concerned about is that
- 12:09
- the thing we're putting into the thumbnail is
- 12:11
- not really a diagram
- 12:12
- it's just a single snapshot
- 12:14
- In our model
- 12:15
- a diagram is really a grouping of different
- 12:18
- snapshots of that diagram as they change through a clip
- 12:21
- whereas this is more like a diagram snapshot
- 12:23
- So now that
- 12:24
- probably sounds like garbage or jargon to you
- 12:27
- if you don't understand the language of the project
- 12:29
- but I've been able to communicate something pretty
- 12:32
- rich
- 12:32
- that the model understands because the glossary.md is
- 12:35
- in its context. Q8 sounds good
- 12:37
- all three of those sound really valuable
- 12:39
- Q9, yes, when we move the face
- 12:43
- the diagram should just stay put
- 12:44
- Q10, yes, when we retake
- 12:46
- then we do the background removal and we
- 12:48
- replace the photo. Okay
- 12:49
- it's Now asking whether it should prototype a
- 12:52
- few different editing services
- 12:53
- and
- 12:54
- I love this. That's exactly what I want
- 12:55
- it to do
- 12:56
- The tricky thing about Q11 here is that
- 12:58
- it's really about a question about feel
- 13:01
- How does it look? How does it feel
- 13:03
- to use?
- 13:03
- And
- 13:04
- I won't really know the trade-offs until I
- 13:06
- see the differences all laid out in front
- 13:09
- of me. In other words
- 13:09
- until I have a go at it
- 13:11
- And so
- 13:11
- I have a prototype skill that gives you
- 13:13
- multiple options
- 13:14
- that you can use to make these kinds of decisions
- 13:17
- and that's what the agent is trying to do here
- 13:19
- which ones should it prototype
- 13:21
- Q11, yes, that sounds great to me
- 13:23
- The benefit of having multiple prototypes to look at is
- 13:26
- especially if it builds something that you don't like
- 13:29
- that often tells you more
- 13:31
- than what you might get from something you
- 13:33
- do like
- 13:34
- So actually exploring lots of different options
- 13:37
- can give the agent a really rich view
- 13:39
- over what you actually want to build
- 13:41
- And there we go
- 13:42
- We now have the glossary
- 13:43
- The CVN glossary has no entry for the thumbnail
- 13:46
- and it's proposing two. So it's proposing a thumbnail
- 13:49
- the image...for video on YouTube or a cutout
- 13:52
- or rather and a cutout
- 13:54
- the author's face with the background removed
- 13:55
- placed over the diagram
- 13:57
- and then it's also saying there should be
- 13:58
- a third term thumbnail diagram for Q12
- 14:02
- Yes, that sounds great. All right
- 14:04
- let's send this off
- 14:05
- and now again
- 14:06
- we should be getting closer and closer as
- 14:08
- we go
- 14:09
- Lots of folks ask me like
- 14:11
- you know, why
- 14:13
- what happens if you have a grilling session
- 14:15
- that runs for like 500 questions or something?
- 14:19
- How do you know when to stop grilling?
- 14:22
- And you should, like, I should be driving this conversation
- 14:26
- you know what I mean?
- 14:26
- Like
- 14:27
- you should be the one in charge of
- 14:28
- it. The agent is not grilling you
- 14:31
- you are in a conversation with it
- 14:33
- And so you should feel free to just go
- 14:34
- no, we've talked enough
- 14:36
- let's go and actually create some code now
- 14:37
- Okay
- 14:38
- we can see it's updated the section on
- 14:40
- thumbnails inside the glossary
- 14:41
- that's really good. And now it's doing something cool
- 14:43
- which is it's pinging off a sub-agent to
- 14:45
- go and prototype thumbnail diagrams
- 14:48
- So that's great
- 14:48
- we're able to continue grilling while it goes
- 14:51
- off and does some work in the background
- 14:53
- Okay
- 14:53
- and now we're doing some hardcore language stuff
- 14:55
- The thumbnails drawing is neither a diagram nor
- 14:58
- a diagram snapshot. You're right that it isn't a diagram
- 15:00
- and a diagram snapshot doesn't fit either
- 15:02
- The glossary defines a snapshot as a copy
- 15:04
- at the moment a clip was filmed
- 15:06
- So it's now proposing a new term
- 15:08
- a thumbnail drawing. The drawing that belongs to one thumbnail
- 15:12
- you add it in place and we can
- 15:13
- start from a diagram
- 15:14
- diagram, snapshot, or another thumbnail's drawing
- 15:16
- It's actually saying to avoid thumbnail diagram here
- 15:20
- Now I know this glossary pretty well
- 15:21
- right? And the better I know the glossary
- 15:23
- the more able I'm able to
- 15:25
- make smart contributions to the application
- 15:28
- the better I know
- 15:29
- understand the application
- 15:30
- And so I know there's this thing called a preserve snapshot
- 15:32
- which is the diagram snapshot kept in its diagrams timeline
- 15:35
- even when no clip uses it
- 15:37
- So I can say to the agent
- 15:39
- doesn't it make sense to have a
- 15:42
- thumbnail represent a preserved snapshot?
- 15:46
- We don't
- 15:47
- always define snapshots as the moment that a
- 15:49
- clip was filmed. I'm going to show you
- 15:51
- a technique that
- 15:52
- I sometimes use when I feel like I'm
- 15:53
- at the crux of something
- 15:55
- or when I just want to talk about
- 15:56
- one thing for a minute
- 15:57
- which is I'm just going to say
- 15:59
- for now, let's just focus on this question
- 16:01
- Ask me
- 16:01
- one question at a time until I say
- 16:03
- otherwise. This is another way that you can
- 16:05
- direct the grilling if you're kind of getting a bit scatterbrained
- 16:08
- if you just want to focus
- 16:09
- On the thing that you actually care about
- 16:11
- and don't care about all the small stuff
- 16:13
- then you can just say
- 16:14
- now just go
- 16:14
- one question at a time for now
- 16:15
- One other thing it's doing here is it's
- 16:17
- reading an architectural decision record
- 16:20
- that we made on snapshots and preservation
- 16:22
- This is another thing we do during grilling
- 16:24
- We can take a look at ADR 003
- 16:27
- just up in here
- 16:28
- diagrams as first class entities with per clip
- 16:31
- snapshots
- 16:31
- So this is where we actually made the
- 16:33
- decision
- 16:34
- about the glossary. So the glossary kind of
- 16:36
- describes the application as it is now
- 16:38
- as does the code
- 16:40
- but the ADRs are here to explain why
- 16:42
- we made those decisions. In other words
- 16:44
- we are interacting with a part of the
- 16:46
- application that's relatively old
- 16:47
- relatively mature, and if we didn't write this stuff down
- 16:50
- then we would be guessing
- 16:51
- This is a classic Chesterton's fence issue
- 16:54
- right?
- 16:54
- There exists in such a case a certain institutional law
- 16:57
- let us say, for the sake of simplicity
- 16:59
- a fence or gate erected across a road
- 17:01
- The more modern type of reformer goes gaily
- 17:03
- up to it and says
- 17:04
- I don't see the use of this
- 17:05
- let us clear it away
- 17:07
- To which the more intelligent type of reformer
- 17:09
- will do Well to answer
- 17:10
- if you don't see the use of it
- 17:12
- I certainly won't let you clear it away
- 17:14
- Go away and think
- 17:15
- Then when you come back and tell me
- 17:17
- that you do see the use of it
- 17:19
- I may allow you to destroy it
- 17:20
- In other words
- 17:21
- do not remove a fence until you know
- 17:23
- why it was put up in the first
- 17:25
- place. So the ADRs
- 17:26
- the architectural decision records
- 17:27
- help us keep track of why these fences
- 17:30
- have been put up. So it's come back
- 17:31
- and of course it's saying
- 17:32
- yes, you're absolutely right
- 17:34
- I was wrong to rule it out
- 17:35
- And so it's saying
- 17:36
- which diagram does a thumbnail's snapshot belong to?
- 17:39
- Yeah
- 17:39
- the thumbnail pins a snapshot of some diagram's
- 17:42
- head. The question is which head you're editing
- 17:44
- in the thumbnail creator. This is a really
- 17:46
- tricky one because
- 17:47
- we're basically
- 17:48
- deciding how the thing is going to work now
- 17:51
- We basically, in this mode
- 17:53
- we might have to choose a diagram
- 17:55
- so that we can create a snapshot inside it
- 17:58
- or we choose an existing snapshot that we
- 18:00
- then edit
- 18:01
- It all feels a little bit
- 18:03
- clunky and weird. And what it's kind of
- 18:05
- suggesting is that we might need a diagram
- 18:07
- of its own for each thumbnail
- 18:09
- Now
- 18:09
- it keeps the teaching diagrams head safe while
- 18:11
- you simplify for the thumbnail
- 18:12
- and you still get to
- 18:14
- start from the teaching version
- 18:15
- I really don't know here
- 18:17
- I really don't know. I'm just going to
- 18:18
- sort of spill out what I'm thinking about
- 18:20
- I think. So what I'm thinking about in
- 18:21
- my head
- 18:22
- is the decision between whether we
- 18:24
- kind of tie into the existing diagram
- 18:27
- diagram snapshot stuff, which is mature
- 18:29
- there's a lot of code in there
- 18:30
- or whether we
- 18:31
- sort of extend that work
- 18:34
- and instead apply it to a different data model
- 18:37
- call it a different thing
- 18:38
- and
- 18:39
- it's essentially whether to mix the streams or
- 18:41
- not mix the streams. I think probably
- 18:44
- what I'm
- 18:44
- most inclined to do here is not mixing the streams
- 18:48
- but what I'm scared about is later down
- 18:49
- the line we might wish that we had
- 18:52
- so that these models could be linked somehow
- 18:54
- But
- 18:55
- I can't see the concrete use case for
- 18:57
- why we would want to
- 18:59
- still hang on to the existing model
- 19:01
- You can see that I'm struggling here
- 19:02
- right? Like I, this is a hard thing to think about
- 19:05
- and if I had colleagues
- 19:06
- this would be the moment when I would
- 19:07
- reach out to them and go
- 19:08
- help me
- 19:09
- I'm not quite sure what we do here
- 19:11
- This feels like quite a large consequential decision
- 19:13
- for this feature. So it's saying
- 19:14
- I would go with your instinct and keep
- 19:16
- them separate. The fear of wanting them linked
- 19:17
- later is mostly covered by two facts
- 19:20
- So both sides store the same scene format
- 19:23
- and in other words
- 19:24
- the way they store it in the database
- 19:25
- is the same. Copying between them is trivial in either direction
- 19:28
- so start from a diagram or snapshot
- 19:30
- works without the models being joined
- 19:31
- The link you're likely to want is where
- 19:33
- a drawing came from
- 19:34
- not one model
- 19:35
- Which thumbnails were made from which diagram is
- 19:37
- answered by one nullable started from snapshot reference
- 19:39
- recorded when you copy. So
- 19:41
- the reason we're so obsessed with language at
- 19:43
- this point is because the language is going
- 19:45
- to form the data model
- 19:46
- that everything else fits on top of
- 19:48
- Once we understand the language
- 19:49
- once we understand the names for these things
- 19:51
- we then have names for database tables
- 19:53
- We understand how they reference other parts of the system
- 19:58
- and so we can create tables from that
- 19:59
- create modules, and set it up
- 20:02
- Okay, so I think I've decided
- 20:04
- Okay, that suggestion sounds good
- 20:07
- I
- 20:08
- like that we have a lot of flexibility
- 20:09
- Ability not to quite decide the data model now
- 20:12
- and
- 20:13
- I like your suggestion of tracking the
- 20:15
- snapshot that it came from
- 20:17
- in case that becomes relevant later down the
- 20:19
- line and we properly want to link it
- 20:21
- I suppose I don't necessarily matter how we
- 20:23
- model this as long as
- 20:25
- the way we're modeling it is not lossy
- 20:27
- and we preserve all of the links
- 20:28
- Let's just see what it says to that
- 20:30
- I'm not necessarily giving it a great answer
- 20:32
- but
- 20:32
- maybe that's enough for it to go on
- 20:34
- At its best
- 20:34
- grilling really does feel like rubber ducking with someone
- 20:38
- with a really respected colleague
- 20:40
- someone that understands the codebase really well
- 20:42
- and you're kind of just able to talk
- 20:44
- out your thoughts or type out your thoughts
- 20:47
- and By talking them through
- 20:49
- you get to a more concrete solution even
- 20:51
- before you've written any code
- 20:52
- Okay, it's gone a bit nuts here
- 20:54
- which is it's actually gone ahead and written
- 20:56
- an entry for thumbnail drawing
- 20:58
- which I don't like. I don't like having
- 20:59
- the concept of drawing here
- 21:01
- Let's take a step back
- 21:01
- I really don't like the
- 21:03
- wording of thumbnail drawing
- 21:05
- I hate the fact that we've got a
- 21:06
- drawing and a diagram
- 21:08
- Those feel like they conflict really hard
- 21:11
- I think thumbnail diagram is just fine
- 21:13
- Can you also run me through the decision
- 21:15
- to create an ADR here?
- 21:16
- Why did you decide to go for
- 21:18
- a new ADR? So it sort of eagerly
- 21:20
- create an ADR without asking me
- 21:22
- which is not
- 21:23
- how I usually like to do it
- 21:24
- I like to
- 21:25
- have that decision be something I do during
- 21:27
- the grilling session. Okay
- 21:28
- he says, he says, the agent says
- 21:31
- I've renamed it to thumbnail diagram everywhere
- 21:33
- The glossary, the ADR, and the ADR's file name
- 21:36
- Yeah
- 21:36
- it's kind of coming back at me a
- 21:37
- little bit on here saying
- 21:39
- once more catch to be aware of
- 21:40
- an active diagram is a diagram
- 21:42
- but a thumbnail diagram is not
- 21:44
- So maybe the word diagram isn't right here
- 21:47
- So now it's saying
- 21:48
- why did it suggest an ADR?
- 21:49
- Why did it feel the need to
- 21:51
- write down this non-obvious decision?
- 21:54
- So it's saying it is non-obvious
- 21:55
- right? It is surprising without context
- 21:57
- There is a real trade-off here
- 22:00
- and so that's another criteria
- 22:01
- and it is strong
- 22:02
- and it's hard to reverse
- 22:04
- Or rather
- 22:04
- it's saying it's not hard to reverse
- 22:06
- We could potentially walk this back
- 22:08
- So now it's spelled it out
- 22:09
- I do agree with its reasoning as to
- 22:11
- why it suggests an ADR
- 22:12
- And by the way
- 22:13
- I'm investing
- 22:14
- time into this
- 22:15
- so that future runs understand the context for
- 22:17
- every decision
- 22:18
- so that I continue getting better grilling sessions
- 22:21
- as I continue working on this project
- 22:23
- Without this, or being lazy here
- 22:25
- would mean that my next run
- 22:26
- ends up being a lot worse because I
- 22:29
- don't have as good context as I need
- 22:31
- Okay, I agree with your decision on the ADR
- 22:34
- but
- 22:34
- and I also agree That
- 22:36
- thumbnail diagram is a little bit awkward
- 22:38
- Can you give me some options here
- 22:40
- and let's do some scenario testing to see
- 22:42
- how they match up against diagram versus diagram
- 22:44
- snapshot
- 22:45
- I'm sort of leaning towards thumbnail diagram snapshot
- 22:48
- even though that's a little bit long
- 22:49
- but might work. Scenario testing is a really
- 22:51
- great way of Kind of testing your domain language
- 22:55
- testing the language that you're trying to use
- 22:57
- What it does is it gives you a
- 22:58
- table of different options
- 23:00
- different sentences you could use to describe the system
- 23:04
- and then it checks out how
- 23:06
- easy it would be to describe in different
- 23:08
- forms of language. So
- 23:09
- here we can see we have a candidate
- 23:11
- a thumbnail diagram snapshot versus a thumbnail diagram
- 23:14
- versus a thumbnail scene. So in the first scenario it says
- 23:17
- I started the thumbnail from the agent loop
- 23:18
- snapshot used in clip 14
- 23:20
- then I simplified it
- 23:21
- All three there are probably fine
- 23:24
- You open the agent loop in the playground
- 23:25
- is the thumbnail in its timeline?
- 23:27
- Yeah
- 23:28
- that's an interesting one because if it's saying it's a snapshot
- 23:31
- then you would kind of expect it to
- 23:33
- be in there or
- 23:34
- you know
- 23:34
- expect it to be in there somewhere
- 23:36
- If it's a thumbnail diagram
- 23:37
- then you might try to look at it
- 23:39
- on the top level. And so yeah
- 23:41
- by calling it a diagram or a diagram snapshot
- 23:43
- it sort of makes it feel like it's
- 23:45
- linked in with the other staff
- 23:47
- It says here, I dragged an arrow in the diagram snapshot
- 23:51
- but that's not good because snapshots Change
- 23:54
- You tell an agent to tweak the thumbnails
- 23:56
- diagram snapshot
- 23:58
- Doesn't really work
- 23:59
- but I'm starting to see
- 24:01
- why it thinks about thumbnail scene and why
- 24:03
- it thinks it's good. We can see in
- 24:05
- all of the tables here that thumbnail scene
- 24:07
- is basically
- 24:08
- fine. Apart from
- 24:10
- Matt says it allowed while working
- 24:12
- In other words
- 24:12
- does it match my actual mental model?
- 24:14
- For thumbnail diagram snapshot
- 24:16
- it's a bit too long
- 24:17
- a bit too wordy
- 24:19
- Thumbnail diagram seems fine because it is at
- 24:21
- what I naturally say
- 24:22
- and thumbnail scene is not
- 24:24
- a word that I would use
- 24:26
- It's more sort of coding vocabulary
- 24:27
- In other words
- 24:28
- scene is not something that we sort of
- 24:30
- have throughout the application
- 24:34
- Maybe it is actually
- 24:34
- maybe we have like a scene layer
- 24:36
- possibly. I'm just going to search in the
- 24:37
- glossary for scene here
- 24:39
- and yeah
- 24:40
- we do have a concept of a scene
- 24:43
- And diagram we've also got
- 24:45
- okay
- 24:46
- that's not good either because we've got a
- 24:48
- scene in there too. So let's read its
- 24:49
- kind of final part here
- 24:52
- Yeah, even though C has all the ticks
- 24:54
- It actually does weight number seven really high
- 24:57
- And it says C is the clearest technically
- 24:58
- but it's not how you talk
- 25:00
- And it puts a code word in the
- 25:01
- domain language. B has one weakness
- 25:03
- It reads as a kind of diagram
- 25:06
- And so, yeah
- 25:07
- I
- 25:07
- do think it's right that thumbnail diagram makes
- 25:09
- sense as long as we have an ADR
- 25:11
- to say it's a diagram
- 25:12
- but not
- 25:13
- a diagram
- 25:14
- And I can see why it went for
- 25:15
- thumbnail drawing as well
- 25:17
- but I just don't like that word
- 25:18
- It doesn't feel like it rolls off the tongue
- 25:20
- Okay, thumbnail diagram is the right thing
- 25:22
- I agree that thumbnail scene is poor
- 25:25
- especially because we do actually have a concept
- 25:27
- of a scene elsewhere in the glossary
- 25:28
- I also agree that having the ADR there means that
- 25:32
- yeah, we have a concept of a diagram
- 25:33
- but we've also got a thumbnail diagram
- 25:35
- And
- 25:35
- I think that's okay to have that distinction
- 25:37
- Can you now widen things out?
- 25:39
- I feel like we've understood the crux
- 25:41
- of what we're building
- 25:42
- Let's now have multiple questions at once
- 25:44
- I think this kind of ceremony feels
- 25:46
- really off to a lot of people
- 25:48
- A lot of people go
- 25:48
- why are you spending so much time thinking
- 25:50
- about language? And sort of like
- 25:53
- you know, About the words that you're using
- 25:55
- why is that so important to you?
- 25:58
- And I think
- 25:59
- the reason it's so important is what you've just seen here
- 26:02
- what we've just done
- 26:04
- We have, from almost nothing
- 26:06
- now we have a
- 26:07
- fully realized concept of the thing that we're
- 26:09
- building and the data model that's going to
- 26:11
- go into it. We now know that we
- 26:12
- have a thumbnail and a thumbnail diagram
- 26:15
- That thumbnail diagram is not technically a diagram
- 26:18
- it's more like a scene
- 26:19
- but we can't use the word scene because
- 26:20
- we've got that elsewhere. And then this is
- 26:22
- going to then go
- 26:23
- into the rest of this grilling session
- 26:25
- Q2C says starting from a diagram's head
- 26:27
- When you start from a diagram's head
- 26:28
- should the CVM save that head as an
- 26:30
- unpreserved diagram snapshot? Q2C
- 26:32
- that is absolutely correct. What the start from
- 26:34
- searches and in what order?
- 26:36
- I think it's got this idea that I
- 26:38
- want a UI where we like start from an existing diagram
- 26:41
- but I think
- 26:42
- I always want an empty diagram to start
- 26:44
- and then be able to search my existing
- 26:46
- diagrams from there. I think what I want
- 26:48
- from the Q14
- 26:51
- is
- 26:51
- I don't necessarily want to go from
- 26:53
- A start from list of potential different diagrams
- 26:57
- I want to start with an empty canvas
- 26:58
- and then
- 27:00
- maybe search for existing diagrams and plonk them on
- 27:03
- Q15 says, should we have one thumbnail diagram per thumbnail?
- 27:07
- Absolutely, Q15 is correct. Q16
- 27:11
- no user images in the thumbnail diagram
- 27:13
- Absolutely agree with Q16. Q17
- 27:15
- it's asking about a separate concept called a pitch
- 27:19
- whether a pitch has a YouTube thumbnail description
- 27:21
- with arrow text field. Q17
- 27:22
- let's not worry about pitches
- 27:24
- They're kind of soft deprecated
- 27:25
- I don't use them very much and so
- 27:27
- let's not worry about integrating with them
- 27:29
- Q18, yes, let's keep existing thumbnails as they are
- 27:32
- Q19
- 27:33
- let's give the thumbnail editor
- 27:36
- a parity with the diagram editor in terms of saving components
- 27:40
- inserting components
- 27:41
- We may even want to consider
- 27:43
- a shared component there. We're seeing something a
- 27:46
- little bit awkward actually in our domain language
- 27:47
- for component
- 27:49
- We can see that inside the diagram sections
- 27:51
- of the glossary we have this concept of
- 27:53
- a
- 27:53
- Component
- 27:54
- This is something I'm actually pretty proud of
- 27:55
- in my user interface
- 27:57
- which we have diagram components here
- 27:59
- which are reusable diagrams that I can plug
- 28:02
- into any diagram that I want to
- 28:05
- But calling them just raw components in the
- 28:07
- glossary means that they actually sort of
- 28:09
- bump up against the concept of components in React
- 28:13
- and that's what I was just talking about
- 28:15
- We need to have a shared component
- 28:17
- that shows a list of components
- 28:19
- It doesn't make sense
- 28:20
- A quick note on the component part of the glossary
- 28:23
- which is
- 28:24
- I'm finding it a little bit awkward
- 28:26
- with the word component
- 28:28
- in there because that bumps up against React
- 28:30
- components
- 28:31
- So I think we probably need to prefix
- 28:33
- it with the word diagram
- 28:34
- so a diagram component. Okay
- 28:36
- we are cooking with gas now
- 28:37
- so
- 28:38
- I'm starting to get a real sense for what we're building
- 28:40
- much, much more than when we started
- 28:42
- I'm not feeling the need to
- 28:43
- end the grilling session just yet
- 28:46
- I'm feeling pretty happy with where we are
- 28:48
- These are still high quality questions
- 28:50
- Okay
- 28:50
- it's now asking whether I want to change
- 28:51
- the way the search Q20
- 28:54
- no, I disagree
- 28:55
- I still want the way
- 28:59
- I want it to work in the diagram
- 29:00
- playground the exact same
- 29:02
- as it works in the thumbnail editor
- 29:04
- I like
- 29:05
- that components have one search and then
- 29:08
- diagrams and diagram snapshots have their own
- 29:09
- If we make two subtly different diagram editors
- 29:12
- that will be very
- 29:13
- very annoying and users
- 29:16
- i.e. me
- 29:16
- will paper cut themselves all the time on it
- 29:18
- Q21, what an insert brings in
- 29:20
- should an inserting a diagram or snapshot bring
- 29:22
- in the whole drawing?
- 29:23
- Q21, yes. Paste counts as a source too
- 29:27
- So what it's saying is if we
- 29:29
- copy one diagram and then paste it in
- 29:33
- should we track the snapshot that it came from?
- 29:35
- Q22
- 29:36
- that sounds like a lot of complexity
- 29:38
- for a feature that I'm maybe not going
- 29:40
- to use that often
- 29:41
- So no
- 29:42
- I don't think I care about it
- 29:44
- One diagram component library
- 29:45
- one
- 29:46
- UI
- 29:47
- Should a diagram component save from the thumbnail
- 29:49
- show up in the playground and the reverse?
- 29:51
- Q23, yes, absolutely. Next, deleting things that are a source
- 29:56
- Today
- 29:56
- deleting a video cascades and hard delete siths
- 29:58
- thumbnails
- 29:59
- If next week's thumbnail diagram copy shapes from this week's
- 30:02
- deleting this week's would break that link
- 30:04
- which breaks the not lossy rule
- 30:06
- Q24, yes
- 30:07
- thumbnails should be archived instead of deleted
- 30:09
- We kind of want soft delete on everything
- 30:11
- should be the rule. For those who don't know
- 30:12
- soft delete, Instead of deleting something from a database
- 30:16
- which would be a hard delete
- 30:18
- you instead mark it with a
- 30:20
- little marker like archive true or something
- 30:23
- and then it doesn't show up in the UI
- 30:25
- or you build your application
- 30:26
- so that it doesn't show up
- 30:27
- Soft deletion is
- 30:28
- incredibly useful because it means that you can
- 30:31
- go back and retrieve that data
- 30:32
- if you want to
- 30:34
- Whereas if you hard delete
- 30:35
- then you're kind of screwed because it's literally been gone
- 30:38
- gone from the system. Because this data is
- 30:40
- really valuable to me
- 30:41
- I pretty much always go with soft delete
- 30:43
- and I don't think I have any cleanup
- 30:45
- mechanism for like cleaning up old data
- 30:47
- I just keep it around
- 30:48
- But in an application with lots of users
- 30:49
- then I would want to make sure I'd eventually
- 30:52
- let's say on a 30-day cycle or something
- 30:55
- deleted the stuff that people had soft deleted
- 30:57
- Okay, it's asking about
- 30:58
- should parity come from extracting the playground editor
- 31:00
- into one shared diagram editor component?
- 31:02
- Q25, yes, absolutely
- 31:04
- let's have a single source of truth for this
- 31:06
- Q26
- 31:06
- it's saying your step of searching for a
- 31:08
- diagram or snapshot or dropping it in doesn't
- 31:10
- exist yet
- 31:11
- This feels to me like it's actually untrue
- 31:14
- It's actually not noticed something about the code
- 31:17
- which is a little annoying
- 31:18
- Q26, I don't think that's true
- 31:20
- We already have a
- 31:22
- like, find diagram thing, right?
- 31:25
- So we would just reuse that
- 31:27
- wouldn't we?
- 31:28
- Q27, navigation results in the thumbnail editor
- 31:31
- Q27, yes, I agree, looks great
- 31:33
- Q28
- 31:34
- soft delete everywhere is a separate piece of
- 31:35
- work. It's now taking my
- 31:38
- opinion and kind of running with it
- 31:40
- But I'm pretty sure we already have soft delete everywhere
- 31:43
- Q28, yeah, let's not overscope this
- 31:45
- I'm pretty sure we already have
- 31:47
- soft delete most places that I want it
- 31:49
- This is an important part of grilling too
- 31:50
- is like when you're talking about your system
- 31:53
- you will often find little
- 31:54
- ways to improve it that might lead to
- 31:56
- issues that you create on your backlog
- 31:58
- that you eventually
- 31:59
- maybe grill about or maybe just like slam
- 32:02
- through. And so grilling is not only about
- 32:03
- the feature you're on
- 32:04
- but it's also about
- 32:06
- staying in touch with the code base
- 32:07
- It's a ritual
- 32:09
- a ceremony for thinking about the code
- 32:11
- This is why I spend so long on it
- 32:12
- Q26, yes
- 32:14
- let's add a new action to the existing
- 32:15
- search. That sounds great. Okay
- 32:17
- it's now saying, this is a good one
- 32:19
- So
- 32:19
- should past thumbnail diagrams show up in the
- 32:22
- same search? So that's actually pretty neat idea
- 32:24
- because
- 32:25
- we probably do want the ability to
- 32:28
- Search through old thumbnails that we created
- 32:31
- since they might be useful in diagrams
- 32:33
- and
- 32:33
- just in the same way that diagrams are
- 32:35
- useful for thumbnails. This is again exactly the
- 32:37
- kind of thing that grilling is really
- 32:39
- really good at pulling together
- 32:40
- Q29, yes, that is a really smart idea
- 32:44
- You should be able to search for thumbnails from
- 32:47
- or thumbnail diagrams from the diagram playground
- 32:50
- and you should be able to search for
- 32:51
- diagrams and diagram snapshots from the thumbnail editor
- 32:54
- So that is an important
- 32:56
- thing that we need to think about in
- 32:57
- the data model that we weren't really valuing before
- 33:00
- but I think our
- 33:01
- our current model gets us there
- 33:03
- Because it
- 33:04
- thumbnail diagrams are probably going to be stored in a separate
- 33:07
- database table
- 33:09
- or they might just be an entry on
- 33:10
- the existing thumbnail
- 33:12
- So, yeah, interesting one
- 33:15
- Okay, while we've been talking
- 33:16
- the prototype has been built
- 33:18
- So let's check it out
- 33:19
- Okay, here we go
- 33:21
- This is pretty surreal
- 33:23
- It's quite surreal editing a thumbnail for my
- 33:25
- own face. This is very cool
- 33:27
- This is pretty
- 33:29
- similar to what I was imagining here
- 33:31
- We can move my
- 33:32
- face like left and right here
- 33:33
- It's a little bit fast there
- 33:35
- but I mean it is a good one
- 33:38
- And then in the top left
- 33:39
- you can actually see a preview of the
- 33:41
- composited photo there. That's nice
- 33:43
- And this is only variant A
- 33:45
- right? We've got a couple more variants
- 33:46
- So we now have a face inside the
- 33:48
- canvas. Whoa. This is again pretty trippy
- 33:51
- We
- 33:52
- this is actually worse than I thought it
- 33:53
- was going to be. It's a bit too
- 33:54
- large. I really hate this like
- 33:57
- bar over the left here with the view
- 33:59
- This face thing again is just way too
- 34:01
- Okay
- 34:03
- B is really not as good as I
- 34:04
- thought it was going to be
- 34:05
- And then what's this? A split view
- 34:08
- Okay
- 34:08
- the split view is garbage because I have
- 34:09
- to like
- 34:10
- look over on the left here and
- 34:12
- then see it update on the right
- 34:14
- You can see I'm moving the
- 34:15
- blue box around
- 34:17
- and then I have to like manually line
- 34:19
- it up with my finger and I can't
- 34:20
- see my finger. This is bad
- 34:21
- So
- 34:22
- I guess A
- 34:23
- is the one that we're kind of going for here
- 34:25
- Oh, look at this feature
- 34:26
- Look
- 34:27
- it has a ghost face
- 34:28
- while pointer is in the frame
- 34:30
- Ghost used as a verb there
- 34:32
- So look, I can like
- 34:33
- when the pointer is in the frame
- 34:35
- my hand disappears. That's so freaky
- 34:38
- It's annoying that it's showing get license for production there
- 34:40
- because I do actually have a license
- 34:41
- so something's going wrong there
- 34:42
- So I'm going to dictate out all the
- 34:44
- things that I love and hate about this
- 34:47
- and we're going to iterate with the agent
- 34:49
- I really love A
- 34:50
- I think you've really nailed something there
- 34:53
- I especially like the ghost face thing
- 34:55
- That's a nice feature
- 34:57
- The moving me left and right
- 34:59
- so moving the face left and right
- 35:00
- is a little too sensitive
- 35:02
- so just bump that down a little bit
- 35:03
- There appears to be
- 35:05
- in the composited diagram a little white
- 35:08
- line just to the left of
- 35:10
- or sorry, to the
- 35:11
- on the outside of the main block there
- 35:14
- which is not good. I also want the
- 35:16
- composites to display below the thumbnail editor
- 35:21
- We're also missing a lot of the stuff
- 35:23
- that we've grilled about
- 35:24
- such as being able to insert components
- 35:27
- and being able to drop in stuff
- 35:28
- And I'm pretty sure we don't currently have
- 35:30
- parity with the other
- 35:33
- with the diagram playground
- 35:34
- right? So we probably need a version of
- 35:36
- it like that
- 35:37
- So let's abandon the three variants now
- 35:39
- Let's just go with A
- 35:40
- and let's continue to iterate on it
- 35:41
- It's also got a couple of questions for
- 35:43
- me after I've tried it
- 35:44
- So it's saying which surface
- 35:45
- I think we've added that
- 35:47
- With Q31, yeah
- 35:48
- I want it to live inside the thumbnail
- 35:50
- page
- 35:51
- And
- 35:52
- I
- 35:52
- think we probably need to see that in
- 35:53
- the next prototype so that we can arrange
- 35:55
- it on the page properly
- 35:56
- Should moving the face and retaking be on
- 35:58
- the same undo history as drawing?
- 36:01
- Oh, yeah. Q32, that is genius
- 36:04
- Really, really smart. Definitely. That's really cool
- 36:07
- Cause I don't have an undo history in
- 36:08
- my current thumbnail editor
- 36:09
- so I'm kind of screwed there
- 36:10
- But we kind of get that for free because TLDraw
- 36:13
- which is the tool that I'm using
- 36:14
- bakes it into its data model
- 36:16
- Very, very, very smart. Okay
- 36:17
- so it's now gone and sent the prototype
- 36:19
- agent back. To it saying
- 36:21
- okay, drop B and C
- 36:22
- put A inside the real thumbnail page
- 36:24
- blah, blah, blah, blah, blah
- 36:25
- In other words
- 36:26
- it's confirming with me all the things that
- 36:27
- I gave it as feedback
- 36:29
- and it's now implementing all of those
- 36:30
- This prototyping phase is so
- 36:32
- so important because we really get to stamp
- 36:34
- how we think the product is going to
- 36:36
- look
- 36:37
- before we then
- 36:39
- like actually schedule an agent to build it
- 36:41
- In other words
- 36:41
- this time human in the loop with the
- 36:43
- agent is so essential because
- 36:46
- we're really going to like
- 36:47
- reduce the decision space for the agent
- 36:49
- We're going to say
- 36:49
- no, it has to look exactly like this
- 36:51
- We know exactly the data model
- 36:53
- All the agent then has to do
- 36:55
- once we run it AFK
- 36:57
- is plug those two things together
- 36:58
- I find it pretty hard to like
- 37:00
- not get excited when I see a prototype
- 37:02
- when I see things starting to come to life
- 37:04
- you know
- 37:05
- We've done this grilling
- 37:06
- which feels kind of a little bit onerous
- 37:08
- sometimes a little bit annoying
- 37:09
- but when you actually see the thing coming together
- 37:11
- you see how it's going to feel to use
- 37:13
- and I'm really doing this
- 37:15
- This is personal software for me
- 37:16
- you know. It's just
- 37:17
- something to make my life more delightful
- 37:20
- And Seeing that actually coming together is so exciting
- 37:22
- Okay, we are back
- 37:24
- and we have a new version here inside
- 37:26
- the thumbnail editor. So we've got the YouTube
- 37:28
- previews down the bottom
- 37:31
- that's nice. The face is in frame here
- 37:33
- I love that I can resize this
- 37:35
- actually. That's something I've never had before
- 37:37
- Let's switch me around a little bit
- 37:39
- That is really nice and smooth
- 37:41
- Yeah, this really, really rocks
- 37:44
- I now have access to a command palette as well
- 37:46
- so I can insert components
- 37:48
- Let's take a look at the list item here
- 37:50
- Yes
- 37:51
- that's looking excellent
- 37:52
- I should be able to search for other diagrams
- 37:55
- so I can insert a diagram
- 37:57
- So let's just go for a wayfinder here
- 38:00
- And then would you look at that
- 38:01
- we've got all of that working too
- 38:03
- So let's take a look at the composites
- 38:05
- We can see
- 38:06
- still looks like there's that little white bit
- 38:08
- just there. Let's move me slightly this way
- 38:11
- move this so it covers the whole screen
- 38:13
- Yeah, that does, that's really cool
- 38:15
- Oh
- 38:16
- I love it. I can still see the
- 38:16
- little white bar on the right of the export
- 38:20
- But overall, this is looking a lot better
- 38:22
- It's also asked me some questions here
- 38:24
- Q34, I agree. Q35, I agree
- 38:28
- Who owns the face is an excellent AI sentence
- 38:31
- very good
- 38:33
- Q36, I agree. Q37, I agree
- 38:36
- Q38, I agree. So at this point
- 38:40
- the agent is pretty much aligned with us
- 38:42
- We now have
- 38:44
- just sort of one problem to solve in the prototype
- 38:46
- and
- 38:47
- I can feel we're nearly at the end
- 38:48
- of the grilling session. Like once I just start going
- 38:50
- yes, yes, yes, yes, yes
- 38:52
- we are aligned
- 38:53
- right? The agent is making decisions as if it were me
- 38:56
- and it's making decisions
- 38:57
- understanding what I care about
- 38:59
- So this
- 39:00
- little white bar thing is actually showing something
- 39:03
- pretty nice here. This is essentially a bug in the prototype
- 39:07
- and this is a bug that could have
- 39:08
- gone into production if we hadn't spotted it
- 39:10
- but we have spotted it
- 39:11
- So what it's
- 39:12
- trying to do here is it's trying to
- 39:13
- verify its own findings. It's trying to construct
- 39:16
- a harness around the problem
- 39:19
- in other words
- 39:19
- a And then it's going to try to fix it
- 39:23
- whatever the issue is. In other words
- 39:25
- it's going to reproduce the bar in its own browser first
- 39:27
- then read pixel colors along the right edge
- 39:29
- of the export and of each preview
- 39:31
- That should tell us where the white comes
- 39:32
- from. This generation of agents appear to be really
- 39:35
- really good at this. In other words
- 39:37
- verifying their own work really solidly
- 39:39
- There might be stuff in my home repo
- 39:41
- like steering files, that tell it to do this
- 39:43
- but I'm not quite sure
- 39:46
- I think it is just doing this off
- 39:47
- its own back. We can see
- 39:48
- too, that it's opened up the existing subagent
- 39:51
- so this one that's currently on 260k
- 39:54
- tokens here
- 39:55
- So this is already punching pretty deep into
- 39:58
- its dump zone. I would probably have preferred
- 40:00
- it to kick off a new subagent with
- 40:02
- just specifically this problem
- 40:04
- I think that probably would have been better
- 40:06
- But I mean
- 40:07
- let's just see how it does and see
- 40:09
- if it manages to find the solution
- 40:10
- If it doesn't manage to find the solution
- 40:11
- then I will punch it again with
- 40:15
- a new subagent with a clean context
- 40:17
- Everyone's always asking me
- 40:18
- by the way, oh
- 40:19
- do you still stand by the smart Zone
- 40:20
- dumb zone thing
- 40:22
- You know, this is my contention
- 40:24
- and other people have had this contention as before
- 40:26
- it's not original to me
- 40:27
- Dex Hordy from HumanLayer and now Factory came up with it
- 40:30
- which is that if
- 40:32
- when your agent gets to around 150k tokens
- 40:36
- it starts acting a little bit stupider
- 40:38
- and it starts getting slowly dumber and dumber
- 40:41
- And if you think of what smart zone
- 40:42
- versus dumb zone actually means
- 40:44
- like
- 40:45
- what we're talking about there is that the
- 40:46
- risk of hallucination slowly raises
- 40:49
- the more tokens you add to the context
- 40:51
- window. That's because if you imagine each one
- 40:53
- of these boxes is an individual token
- 40:55
- then the
- 40:56
- model needs to keep track of all of
- 40:58
- the relationships between every single token
- 41:01
- And it's not the fact that the number of tokens increases
- 41:04
- it's the number
- 41:04
- of
- 41:05
- relationships that the model needs to keep track
- 41:07
- of just expand quadratically
- 41:09
- and you end up with this enormous amount of memory needed
- 41:14
- and amount of work needed
- 41:15
- to hear the signal in the noise
- 41:17
- However
- 41:17
- making the models better at this is essentially an engineering Problem
- 41:21
- right? Some kind of model design problem
- 41:23
- that I'm sure people are trying to figure
- 41:25
- out. And the question everyone always asks me is like
- 41:27
- what number does the dumb zone begin at?
- 41:30
- When should I clear the context?
- 41:32
- And my answer is becoming
- 41:33
- okay, in previous generations of models
- 41:35
- sure, okay
- 41:36
- you pretty much had to clear the context at 120k sometimes
- 41:39
- even like 80k tokens for Sonnet 3.5 or something
- 41:43
- You know
- 41:43
- the number of token relationships would just go
- 41:45
- nuts and you would end up with
- 41:47
- loads and loads of factuality errors
- 41:50
- But now it's more like
- 41:52
- it's a kind of cost and optimization thing
- 41:55
- really
- 41:55
- You know
- 41:56
- that your agent will be smarter
- 41:58
- when there are fewer tokens here
- 42:00
- So it's not necessarily
- 42:02
- about the risk of hallucinations
- 42:04
- It's more like optimizing your agent to produce
- 42:06
- the best work. So for me
- 42:08
- it's not so much that the agent produces
- 42:10
- hallucinations in the dumb zone
- 42:11
- It's more
- 42:12
- that it just produces worse work and I
- 42:14
- end up with better results when I do
- 42:16
- my work in the smart zone
- 42:17
- Plus
- 42:17
- operating at like 750k tokens in your Context window is very
- 42:22
- very expensive because even like when your cache runs out
- 42:26
- all of those previous tokens are going to
- 42:28
- be run through again and that's going to
- 42:29
- be billed again. Okay
- 42:30
- the next version of the prototype is
- 42:32
- ready and
- 42:33
- while it still has this little white thing in it
- 42:37
- it actually says that it's not an issue
- 42:39
- The little bar actually comes from the browser
- 42:41
- at Windows 125% scaling
- 42:44
- Chrome shrinking 120 wide image into the previews
- 42:47
- left the last pixel column only partly painted
- 42:50
- There's some other stuff here like some hazards
- 42:53
- I agree with Q39. Q40
- 42:55
- yes, drawing edit should auto save the thumbnail
- 42:57
- And it's now asking whether I want to
- 42:59
- write it up
- 43:00
- So I'm going to be careful here
- 43:01
- I'm going to go back to the start
- 43:03
- of this little text input
- 43:05
- and I'm going to say to spec
- 43:06
- let's write
- 43:08
- it
- 43:08
- up
- 43:09
- I have to go back to the start
- 43:10
- because to spec is a user-invoked skill
- 43:14
- so I can invoke it essentially
- 43:16
- and
- 43:17
- I have to have it at the start
- 43:18
- when I'm using Claude Code
- 43:20
- I can't just sort of scatter it throughout the text input
- 43:22
- which is a bit annoying and something I've
- 43:24
- been wanting them to change for a long time
- 43:27
- To spec
- 43:27
- what it's going to do is it's going
- 43:28
- to create a
- 43:29
- specification. It's going to ask me a couple
- 43:31
- of questions about where we're going to test
- 43:35
- and then
- 43:37
- from there we'll be able to implement it
- 43:38
- I already have a few quite robust reducers
- 43:41
- and reducers are essentially places in the front
- 43:43
- end where you can contain lots of state together
- 43:46
- and I find them really
- 43:48
- really good with agents because they can just
- 43:49
- kind of extend existing ones
- 43:51
- All of those three scenes look good
- 43:52
- I'm then going to
- 43:54
- go for the to ticket skill as well
- 43:56
- I'd like you to add some tickets to
- 43:59
- let's just do these in one batch
- 44:00
- Now what spec
- 44:01
- and
- 44:01
- to tickets do
- 44:02
- is they essentially give the agent everything it
- 44:04
- needs to run a long running session
- 44:07
- The spec acts like a destination document
- 44:10
- So we're essentially running work over multiple sessions
- 44:13
- and the spec is going to say
- 44:14
- okay, once we get to the end of these sessions
- 44:16
- this is where we want to end up
- 44:18
- And then the tickets
- 44:19
- are essentially about scoping each individual session
- 44:22
- So each sub-agent we're going to run
- 44:24
- is going to be handed a single ticket
- 44:26
- and that's going to be the scope of
- 44:27
- its
- 44:28
- piece of work. So here it's given us
- 44:29
- the proposed breakdown of all of the sessions
- 44:32
- So the first one is like a pre-factor
- 44:34
- like
- 44:35
- extracting the shared diagram editor into a shared component
- 44:39
- Really useful
- 44:40
- that's going to be very handy later
- 44:42
- Each ticket says what it's blocked by
- 44:44
- and I'm not going to like review this
- 44:46
- too much
- 44:47
- Nine seems fine to me
- 44:49
- I'm just going to say
- 44:50
- yep, looks good to me
- 44:51
- Now the plan is that once this is done
- 44:53
- I'm then going to compact
- 44:55
- So I'm going to take that entire session
- 44:57
- the grilling session we've just done
- 44:58
- compact it down
- 45:00
- and Then we're going to run a skill called implement spec
- 45:03
- which is going to essentially AFK just
- 45:06
- create all of this stuff and merge it
- 45:08
- into a single PR. And you think of
- 45:09
- how easy that job is going to be for the agent
- 45:12
- right? We are going to have an updated
- 45:13
- glossary by that point
- 45:15
- We are going to have
- 45:16
- an entire spec of the thing it needs to build
- 45:19
- all of the tickets
- 45:20
- and how it's going to get there
- 45:21
- And we've also got a prototype
- 45:23
- which it essentially just needs to copy
- 45:25
- in terms of the UI
- 45:26
- So we've already got a bunch of code
- 45:27
- written too. By the time I get to
- 45:29
- the spec and the tickets
- 45:30
- by the time I get to implementation
- 45:32
- like I'm really, you know
- 45:34
- trying to give the agent no
- 45:35
- decision space in which to operate
- 45:37
- no nuance that it can go
- 45:39
- oh yeah
- 45:39
- I'll just sort of fiddle around with this
- 45:41
- Because the more nuance you give to an agent
- 45:43
- the worse it's going to perform
- 45:44
- because
- 45:45
- as soon as you ask an agent to make a decision
- 45:47
- it might make a bad decision
- 45:49
- So the more decisions you give it
- 45:50
- the more decision space it has
- 45:53
- the more likely it is to choose a
- 45:54
- bad one. So I tend to like giving the agent very
- 45:57
- very few options when it comes to implementation
- 45:59
- when it comes to the decision
- 46:00
- Decisions of the features that I want
- 46:01
- Okay, we now have our spec
- 46:04
- and we can see that this is
- 46:05
- in GitHub. So this is a GitHub issue
- 46:08
- I have a bunch of user stories here
- 46:10
- around 47 of them
- 46:12
- a bunch of implementation decisions
- 46:14
- a very, very rich document
- 46:15
- including stuff that's out of scope
- 46:17
- We can see it's also referencing the prototype
- 46:19
- on the local branch
- 46:20
- fantastic. And it then has a bunch of sub-issues
- 46:23
- each of which are the individual tickets
- 46:26
- So all of these are individual tasks
- 46:27
- and crucially, these can be parallelized
- 46:29
- And so what I'm going to do is
- 46:30
- I'm going to compact
- 46:32
- the current conversation and the words just sort
- 46:34
- of
- 46:34
- take
- 46:34
- all of the context that we've had so
- 46:36
- far and squash it down
- 46:38
- And then I'm going to say implement spec just on the
- 46:41
- essentially just on the
- 46:42
- spec that it's just created
- 46:44
- I've started using compact a lot more
- 46:45
- Actually, compact has become a lot more reliable
- 46:48
- a lot nicer to use
- 46:50
- I still don't think it's a perfect thing
- 46:53
- like auto compact is still a little bit dodgy
- 46:56
- because the timing sometimes just absolutely whomps you
- 46:59
- Something important will be Taken out of context
- 47:01
- But compact, especially in this situation
- 47:03
- when we've taken the conversation
- 47:05
- and just turned it into a document
- 47:07
- is so, so safe. It's almost free
- 47:09
- It's just a kind of like
- 47:11
- a minute it spends just compacting this conversation
- 47:13
- and bam, we're back down to essentially our starting context
- 47:16
- Now
- 47:16
- implement spec is a skill that I've had
- 47:19
- for a little while now
- 47:19
- I think it came out in V1.3
- 47:21
- And this is really
- 47:23
- it just takes the spec and the tickets
- 47:25
- and it implements them inside our local setup
- 47:28
- And when it finishes
- 47:29
- which for a piece of work like this
- 47:31
- might be like an hour or 45 minutes maybe
- 47:33
- then we're going to end up with a
- 47:35
- nice PR that
- 47:37
- essentially implements the entire thing that is ready
- 47:39
- to merge. So this is the point here
- 47:40
- where I would step away
- 47:42
- where I would trust it to run AFK
- 47:44
- It's going to run as much in parallel
- 47:46
- as it possibly can. You can see it's
- 47:47
- starting the three tickets that can begin now
- 47:50
- and it's able to pass them very rich
- 47:51
- context about what needs to happen
- 47:53
- because it's just a compacted version of our
- 47:56
- original conversation. So it knows
- 47:58
- exactly the parts of the code base that
- 47:59
- matter
- 48:00
- Knows the spec, knows the prototype too
- 48:04
- So I'm going to walk away
- 48:04
- and when I come back
- 48:05
- I should have working code
- 48:07
- Okay, it's been a little while
- 48:08
- We now have a massive PR
- 48:11
- and we have a code review that has
- 48:13
- run and found some violations
- 48:15
- We can see the pull request inside the
- 48:17
- PRs here for edit a thumbnail diagram in
- 48:19
- place beside the face. We can see we've
- 48:21
- added 21,000 lines of code
- 48:25
- I'm looking forward to reviewing that
- 48:26
- And the way the implement spec works is
- 48:28
- it goes away and reviews everything with an agent
- 48:31
- and then it checks against some existing coding
- 48:34
- standards. You should take a look at the
- 48:35
- code review skill if you want to see more of that
- 48:37
- but basically
- 48:38
- it found one hard violation
- 48:40
- and a couple of cases of scope creep
- 48:43
- and then it's fixing them up right now
- 48:45
- I'm going to try something I don't usually try here
- 48:48
- which is I think
- 48:49
- I do want to review this with you guys watching
- 48:52
- and so
- 48:53
- I think I want to try
- 48:55
- a
- 48:55
- different way of breaking down this massive PR
- 48:57
- into smaller chunks. I don't have a skill
- 48:59
- for this yet. This isn't something Something that I do regularly
- 49:01
- but for large chunks of work like this
- 49:03
- that I do want to review
- 49:04
- this makes sense. Once the review is finished
- 49:07
- I think I want you to break it
- 49:08
- into smaller chunks using stacked divs
- 49:12
- These smaller chunks ideally should be reviewable on their own
- 49:15
- and
- 49:16
- what that means
- 49:18
- is that I want them to be
- 49:19
- each a bunch of two-way doors
- 49:21
- and finish in a final one-way door
- 49:24
- You can look for the definitions of two-way
- 49:25
- door and one-way door inside the PR skill
- 49:27
- One-way doors and two-way doors are
- 49:29
- it's a really nice analogy
- 49:30
- It basically means that
- 49:32
- when you have a PR that is a two-way door
- 49:33
- it means you can merge it and then
- 49:35
- revert it anytime. So you can
- 49:37
- walk through the door and then walk back
- 49:39
- through the door. Whereas a one-way door is
- 49:41
- a PR that is merged and then
- 49:43
- is very difficult to unmerge
- 49:45
- In other words, it might mess with some data
- 49:47
- it might speak to an external service
- 49:50
- it might expose like a security window that's
- 49:52
- hard to walk back. And agents are pretty
- 49:54
- good at breaking up massive PRs into small
- 49:56
- PRs. So it's given me this
- 49:58
- chunk of different PRs here
- 49:59
- I can tell Although that some of them
- 50:01
- are smaller than others
- 50:02
- it feels like this last one is kind
- 50:03
- of the big one. Can you give me
- 50:05
- an estimate of the lines of code that
- 50:07
- would be in each PR
- 50:07
- since that's really what I want to do
- 50:09
- here? I'm also wondering whether we could utilize
- 50:11
- feature flags to
- 50:12
- make the thumbnail diagram available
- 50:15
- and then pull it back later
- 50:17
- In this situation
- 50:17
- I'm kind of cosplaying as if this is a production system
- 50:20
- you know, I'm kind of pretending as if
- 50:22
- okay
- 50:23
- I want to break this huge PR down
- 50:24
- into reviewable chunks
- 50:26
- that we can ship to production and not
- 50:28
- have
- 50:29
- like this big risk when we merge a
- 50:30
- single PR. Usually when I merge to this repo
- 50:33
- because it's only personal software
- 50:34
- I don't have to care that much and
- 50:36
- I can just go
- 50:37
- all right, bam, let's stick it in
- 50:38
- But because if I'm imagining I've got
- 50:41
- you know, hundreds of thousands, hundreds of thousands of users
- 50:43
- like
- 50:44
- We really want to make sure production is
- 50:46
- as close to the change as possible
- 50:48
- and then once we finally flick it on
- 50:51
- it should be relatively seamless
- 50:52
- In other words, we're taking this huge chunk of
- 50:55
- or taking this
- 50:56
- a single door and turning it into lots of doors
- 50:58
- which we can revert as we need to
- 51:01
- and then finally we're going to flick it
- 51:02
- on at the end. Okay
- 51:03
- so now it's taken that last PR
- 51:05
- the massive one
- 51:06
- and turned it into four smaller ones
- 51:08
- The upshot of this is it would basically
- 51:10
- run both implementations side by side for a bit
- 51:13
- and then when we are ready
- 51:14
- we would merge the final PR
- 51:15
- and then that would turn the flag on
- 51:17
- Okay, let's go with this version
- 51:18
- Sounds good to me
- 51:19
- but only do this once the review has
- 51:21
- finally fixed its things. Make sure you delegate
- 51:23
- this work to sub-agents
- 51:25
- since you're a little bit constrained in terms
- 51:26
- of context window. So this is exciting
- 51:28
- This is kind of an experiment
- 51:29
- Hopefully these are going to be easy to merge individually
- 51:32
- Okay
- 51:32
- so we have our first PR of
- 51:34
- seven here. The first one is literally just the glossary terms
- 51:37
- diagram component, and ADR 060
- 51:40
- If we take a little more detailed look at this
- 51:43
- this all looks fine. Fine
- 51:45
- And so as this is
- 51:46
- this can just go in
- 51:47
- I'm happy to enable an auto merge here
- 51:49
- I can press confirm auto merge
- 51:51
- and that will now go in as soon
- 51:53
- as the CI checks pass
- 51:54
- And there we go
- 51:55
- It has gone in and merged
- 51:56
- Beautiful. Next
- 51:57
- we have a new PR coming in saying
- 51:59
- extract the shared diagram editor from the editor
- 52:02
- or diagram playground. So just a few files changed here
- 52:05
- Now, while we're here
- 52:05
- we might as well talk a little bit
- 52:06
- about human review and like what the purpose
- 52:08
- of this exercise is. For me in this situation
- 52:11
- I'm going to do this human review because
- 52:13
- I know that you're watching
- 52:15
- and you're interested in how I do human
- 52:17
- review. And so
- 52:17
- I'm going to review every single line
- 52:19
- But do
- 52:20
- am I saying that you need to
- 52:22
- do every single line yourself or review every single line?
- 52:26
- No, I don't think you do
- 52:27
- When I'm reviewing code like this
- 52:28
- obviously
- 52:29
- I want to make sure that it works
- 52:30
- And so if this was going out to a production system
- 52:33
- then I would clone it locally
- 52:35
- or I would get some kind of playground
- 52:37
- where I could
- 52:37
- walk through it and check that it works
- 52:39
- What I'm actually going to do is I'm
- 52:40
- going to merge everything and then because it's
- 52:42
- I'm only, I'm the only user
- 52:44
- I'm Just then going to review at the
- 52:45
- end. But the point of checking the code
- 52:47
- here is that I'm not just interested in
- 52:49
- like, what the code does
- 52:51
- I'm also interested
- 52:53
- in the system that produced this code
- 52:55
- In other words
- 52:56
- if
- 52:56
- I notice that something is going wrong inside
- 52:59
- this code or there's a pattern that I don't like
- 53:01
- then what I'd like to do is create a
- 53:05
- like
- 53:05
- a lint rule or something
- 53:07
- to make sure it doesn't happen again
- 53:09
- And by don't like
- 53:10
- I mean that
- 53:11
- I want to have a hypothesis around that
- 53:14
- thing that I want to change
- 53:15
- that it is affecting my
- 53:18
- environment my agent operates in in a negative
- 53:20
- way. In other words
- 53:21
- I can see in this file here
- 53:23
- like the command palette
- 53:24
- I can see there's a huge comment right
- 53:26
- next to it saying the command K
- 53:27
- command palette mounted inside the active diagrams
- 53:30
- or rather the shared diagram editor over its
- 53:32
- canvas. I'm kind of not happy
- 53:34
- about these large comments and they are kind
- 53:37
- of everywhere in my code base
- 53:39
- because it's more surface that the agent needs
- 53:41
- to update. And so I'm going to add
- 53:43
- a comment here. I'm going Say
- 53:45
- I'm not really a big fan of these
- 53:46
- comments because they leave a huge surface for
- 53:48
- the agent to update
- 53:49
- and they don't really add a lot that
- 53:50
- isn't already explained by the code
- 53:52
- I think I'd like you to add something
- 53:53
- to coding standards to make these comments much less prevalent
- 53:58
- and I think we should have a conversation
- 53:59
- about maybe banning
- 54:00
- comments in certain parts of the code base
- 54:02
- So I'm going to comment here
- 54:03
- and then
- 54:04
- I'm going to show you what I'll do
- 54:05
- with those comments later on
- 54:07
- For instance, here
- 54:07
- I really don't like the as
- 54:09
- keyword in TypeScript. It can be used to cheat the TypeScript
- 54:12
- which I hate
- 54:13
- So I want to say a helper function
- 54:14
- is needed here. I want to ban all
- 54:16
- uses of as not inside shared utilities
- 54:18
- This is just something I know because I am a
- 54:21
- you know, I suppose an ex-TypeScript expert
- 54:24
- I made a course on TypeScript
- 54:25
- basically. Again
- 54:26
- these docs here just sort of
- 54:28
- feel pretty gnarly. I don't like this
- 54:31
- You can see how much the diff as
- 54:32
- we're going through is literally just code comments
- 54:35
- It really is brutal. But overall
- 54:36
- this review didn't take too long
- 54:38
- and I've noticed a couple of things that
- 54:39
- we might need to improve
- 54:41
- So next, we go to the next PR in the list
- 54:43
- which is just A relatively small change again
- 54:46
- We've got some reducer testers
- 54:47
- which I really, really like
- 54:49
- If you've never used a reducer before
- 54:51
- it's just so nice
- 54:53
- so easy to work with
- 54:54
- Again, more comments that I just want to kill
- 54:56
- man. This repo is full of comments
- 54:58
- There's also another TypeScript thing that I really don't like
- 55:00
- which is using this pick and this
- 55:02
- and here. Prefer using interface extends to
- 55:05
- using intersections. See if you can find a
- 55:07
- lint rule for this that we can ratchet
- 55:09
- That phrasing, by the way, like lint rule
- 55:10
- it's asking it to find a deterministic rule
- 55:13
- that we're not necessarily going to impose on day one
- 55:16
- but we're going to ratchet up over time
- 55:18
- Again, so many comments here to review
- 55:20
- It's just, it's getting pretty disgusting
- 55:22
- I have to say. Another one here is
- 55:23
- that even in code that is not actually
- 55:25
- touched by the PR
- 55:25
- I'm noticing it's calling fetch and then ignoring
- 55:28
- the result
- 55:29
- And I say it feels like a reusable
- 55:30
- function is needed here. I'm being pretty harsh in these reviews
- 55:33
- right? Like I'm
- 55:33
- I'm trying to force it
- 55:35
- to design APIs that remove all of the nuance
- 55:39
- all of the decisions from the agent
- 55:40
- Another one here is I can see we
- 55:42
- have a use search params hook
- 55:44
- and we're Wrapping things using magic strings here
- 55:47
- it would be very
- 55:47
- very easy to
- 55:49
- just change the name of this highlight or this snapshot even
- 55:52
- and we would get no type errors from
- 55:54
- it
- 55:54
- Find a more type-safe design for the way
- 55:57
- that we access search parameters
- 55:58
- since this feels like a magic strings style
- 56:01
- hole that we could potentially fall through
- 56:03
- Create a new lint rule banning the use
- 56:05
- of use search params
- 56:07
- and then let's create a strategy to ratchet it up
- 56:09
- Again, I'm being super ambitious here
- 56:11
- I'm saying, okay, everything that we find in here
- 56:14
- I never ever want to see this again
- 56:16
- never ever want to loot
- 56:17
- like
- 56:18
- I wanted to lint it out of existence
- 56:20
- That's my plan. Next PR again
- 56:22
- we're up to 500 lines here
- 56:24
- Got to say that apart from the comments here
- 56:25
- this is a clean
- 56:27
- clean PR
- 56:28
- We can see it's taken a bunch of
- 56:29
- like magic conditionals here
- 56:31
- and it's replaced them with a single source
- 56:33
- of truth for is diagram search
- 56:35
- Phenomenal. Again, just way too many comments here
- 56:38
- Too many comments. Okay
- 56:40
- so I'm now into the
- 56:41
- I think, penultimate one, or maybe last Couple
- 56:45
- let's see. Yeah, this is still a two-way door thing
- 56:49
- I think here. And this is now we're
- 56:50
- getting up to about 2,500 lines
- 56:53
- Now I think you've got the idea of
- 56:54
- what I'm reviewing for here
- 56:56
- which is I am trying to give my opinion
- 56:59
- give my feel just by looking at the
- 57:01
- code
- 57:01
- And I think reviews are a good surface for this
- 57:04
- but I'm not going to consider any of
- 57:05
- these blocking. In fact
- 57:06
- out of all of these PRs
- 57:07
- the only one where I would consider
- 57:09
- reviewing mandatory is this one
- 57:11
- This is an additive migration
- 57:13
- for the thumbnail diagrams concept
- 57:15
- And what I want to see is inside
- 57:17
- schema.ts here
- 57:19
- This adds the thumbnail diagram into the thumbnails
- 57:23
- here
- 57:24
- so many freaking comments here
- 57:26
- guys. Comments are also an opportunity
- 57:28
- for the agent to do
- 57:30
- like two specific texts or
- 57:33
- texts that's specific to its session
- 57:35
- In other words
- 57:35
- this
- 57:36
- session is or this text is basically just
- 57:38
- like a note for itself in a future session
- 57:41
- which is
- 57:41
- really frustrating because then
- 57:43
- that is just going to live Forever that comment
- 57:45
- and it's just totally useless
- 57:46
- Overall, this looks really solid
- 57:48
- So actually
- 57:49
- I'm happy with the one-way stuff as part
- 57:51
- of this big PR. So what I'm going
- 57:52
- to do is go back to my agent
- 57:54
- session. I'm going to compact
- 57:56
- and then I'm going to say
- 57:57
- could you merge all of those PRs in order
- 58:00
- please? Ignore all of the review comments
- 58:02
- for now. We are going to tackle those in a minute
- 58:04
- actually, once everything is merged
- 58:06
- Could you run the merge in a
- 58:07
- background sub-agent so that we can continue chatting?
- 58:09
- Okay, let's fire this off
- 58:10
- So now that we've decided that the merge is happening
- 58:14
- like, I'm going to assume that in your version of this
- 58:17
- you would have actually cloned it locally or actually
- 58:19
- you know
- 58:20
- checked that it was all working on the
- 58:21
- final branch
- 58:22
- I'm fine with checking it in a minute
- 58:24
- But
- 58:25
- what I'm going to do now is I'm
- 58:26
- going to use this session to
- 58:28
- fix or make a plan to fix
- 58:31
- all of the things I found during review
- 58:33
- And for that
- 58:34
- I'm going to use the retro skill
- 58:35
- I'm going to say
- 58:36
- could you take a look at all of
- 58:38
- the agent sessions that ran as part of this
- 58:40
- and could you also pull in my review
- 58:42
- comments?
- 58:43
- And the retro skill essentially runs A retrospective
- 58:45
- on the process that we just saw
- 58:48
- Even though we've run compact
- 58:49
- it still has access to all of the
- 58:51
- sessions that ran before because they're just saved
- 58:53
- locally on disk
- 58:54
- And it's going to use the GitHub CLI
- 58:56
- to pull in all of my comments
- 58:57
- so it's going to have access to that too
- 58:59
- Now
- 58:59
- I will usually run retro
- 59:01
- after a big chunk of work like this
- 59:03
- or I'll also just run it regularly
- 59:05
- just checking up on my agent sessions
- 59:08
- If you have some kind of software factory
- 59:09
- setup where you can automate this
- 59:11
- then I would save your agent sessions somewhere in the cloud
- 59:14
- and then
- 59:15
- retro can take a look at those and
- 59:16
- see what the agents struggled with
- 59:19
- So it's going off and sending agents through
- 59:20
- the 30-odd session transcripts and your 12 review comments
- 59:24
- and it's going to come back with a
- 59:24
- ranked list. So we're essentially throwing a bunch
- 59:26
- of tokens at this and looking for improvements
- 59:29
- or potential improvements
- 59:31
- to the environment our agents are operating in
- 59:32
- Now
- 59:33
- we didn't pay attention at all
- 59:34
- to the implementers while they were running
- 59:36
- I didn't look at the sub-agent traces
- 59:38
- haven't looked at that at all
- 59:39
- This is our opportunity to apply
- 59:42
- some tokens to it and actually
- 59:44
- Look at those and get some improvements from it
- 59:46
- Okay, and here we go
- 59:47
- We have the retro back
- 59:49
- So all 12 of my review comments
- 59:51
- have one root cause
- 59:53
- Nothing in the CVMs written standards cover your
- 59:55
- points. The existing code in the docs model
- 59:57
- do the
- 59:58
- opposite. So existing stuff in coding standards.md
- 60:01
- in docs front end states show the patterns you rejected
- 60:04
- and main already has thousands of instances
- 60:07
- Thousands of instances. Blimey. Okay
- 60:09
- so I am trying to
- 60:11
- turn an oil tanker around apparently
- 60:12
- And also the prototype branch was copied
- 60:16
- Okay
- 60:16
- so its first recommendation is to turn your
- 60:18
- comments into ratcheted checks
- 60:20
- The first one is as outside shared utilities
- 60:22
- Yes, lovely. Sounds good
- 60:25
- There, that is 2,000 instances it will find
- 60:28
- Okay, brutal. Intersections into interface extends
- 60:31
- Oh, excellent. Has no rule for this
- 60:32
- but a small parser. So it's basically saying
- 60:35
- create a custom lint rule for this rule
- 60:39
- Excellent. Next
- 60:40
- I said we should kill use effect
- 60:42
- Lovely job. We turn it off
- 60:44
- Specific directory, so hooks
- 60:46
- and there are currently around 230 user effects
- 60:50
- in 140 files. Next
- 60:52
- we have use search params and magic strings
- 60:54
- so this would point at a new typed search params helper
- 60:57
- lovely. We ban raw fetch
- 60:59
- and instead we have a typed API client
- 61:01
- module. And then it's saying too many comments
- 61:04
- so here no lint rule fits
- 61:06
- Yeah, I don't know what to do about comments
- 61:08
- I mean, like Laurent Tan, potato, says they just ban comments
- 61:11
- no comments allowed
- 61:13
- And I kind of
- 61:14
- I kind of want to go that direction
- 61:16
- but I wonder whether we do that in
- 61:17
- a phased approach where we ban comments in
- 61:19
- certain folders. This is interesting
- 61:21
- So 18% of lines of comments in the
- 61:23
- new features diagrams against a 12% repo average
- 61:26
- 12% of code in my repo is comments
- 61:29
- and 20% of the new feature is comments
- 61:33
- It's crazy. Okay, I really like number one
- 61:35
- All of those, I think
- 61:35
- should be raised as issues on the repo
- 61:38
- Can we also
- 61:39
- start trialing the no comments rule in certain folders?
- 61:43
- And actually
- 61:44
- I want you to give me your opinion
- 61:46
- on
- 61:46
- what we would lose inside features diagrams
- 61:50
- if we just deleted all the comments immediately
- 61:52
- Is there any information contained within the comments
- 61:55
- that cannot be divined from the code itself?
- 61:57
- Okay, number two
- 61:58
- it says fix the docs that teach the rejected patterns
- 62:00
- Obviously, yes. Number three
- 62:02
- prototype code was copied and not re-implemented
- 62:05
- Yeah, this is interesting. So
- 62:07
- it's actually recommending a change to my skills
- 62:10
- about the prototype skill and the
- 62:12
- implement spec implementer brief. For number three
- 62:14
- could you raise an issue
- 62:16
- on the Matt Pocock skills repo?
- 62:17
- By the way, this is often how my skills get improved
- 62:19
- is
- 62:20
- I raise issues about my own skills based
- 62:22
- on what I find in these retros
- 62:24
- and they get subtly improved
- 62:25
- The reviewer can't cover a big diff with one agent
- 62:29
- Yep, so this is the code review skill
- 62:31
- kind of
- 62:31
- not being able to perform At its limits
- 62:34
- Can you raise an issue for number four
- 62:36
- please? And I think there might be an existing issue
- 62:38
- so maybe you can just
- 62:39
- add this as a kind of extra
- 62:42
- you know
- 62:42
- reason for me to look at it
- 62:44
- There's no argument to say that you should do
- 62:45
- instead of like doing automated review on the spec
- 62:49
- you should do it on every single ticket
- 62:51
- But I found that that is just
- 62:52
- you know
- 62:53
- extraordinarily slow. It's an absolutely hugely slow approach
- 62:56
- But it looks like the code review skill
- 62:58
- is kind of like
- 62:59
- you know, taken under a 400 word limit
- 63:01
- which doesn't make sense to me
- 63:03
- So I think maybe the code review skill
- 63:05
- might need to change
- 63:06
- might need to
- 63:07
- divide more chunks of it into sub-agents so
- 63:10
- that it can more easily do it
- 63:12
- Okay
- 63:13
- it's
- 63:13
- asking some stuff about the implement spec skill
- 63:16
- With number six, I really like that idea
- 63:19
- It's basically saying that in implement spec
- 63:21
- it
- 63:23
- Creates a merger agent too early
- 63:25
- it should just try to
- 63:26
- like, merge it indirectly. For number seven
- 63:28
- could you raise an issue about this and
- 63:30
- see if we can hill climb towards a
- 63:32
- like a faster suite? In number eight
- 63:34
- there's a skill I have called verify CVM
- 63:37
- which basically kicks in
- 63:38
- every time I want to verify some
- 63:41
- code or an implementation on the CVM
- 63:43
- It runs it inside a browser
- 63:46
- does all sorts of stuff
- 63:47
- Okay, could you add eight as an issue?
- 63:49
- And all of number nine
- 63:50
- all the smaller items should be added as
- 63:52
- separate issues. That all looks great
- 63:54
- So honestly, this retro was extremely
- 63:57
- extremely good. We've
- 63:58
- I think we might get like 20 issues
- 64:01
- out of this
- 64:01
- Things that can go
- 64:03
- like, that we probably don't have to grill on
- 64:05
- things that we can just pass off to
- 64:06
- agents and get PRs back from
- 64:08
- that will over time slowly improve the repo
- 64:11
- While this has been running
- 64:12
- our agent has been merging
- 64:13
- It looks like it's just sort of waiting
- 64:15
- for all of the CI checks to complete
- 64:17
- and then it just merges each one in turn
- 64:19
- All right, while that's going on
- 64:20
- we have our comment audit
- 64:22
- Has come back
- 64:24
- So it's saying of the about 2,000 lines of comments
- 64:27
- about 940 can go now
- 64:29
- and about 1,000 explain a reason
- 64:31
- you can't see in the code
- 64:33
- It says seven comments are already stale
- 64:35
- There you go
- 64:36
- and there are no ESLint disable or TS
- 64:38
- expect error directives to worry about
- 64:40
- Okay
- 64:41
- let me
- 64:41
- lay into the agent for a second
- 64:43
- The thing that concerns me about comments is
- 64:45
- that they allow workarounds to proliferate
- 64:49
- and
- 64:49
- the comments kind of act as a
- 64:51
- fig leaf
- 64:52
- that the agent is able to put there and say
- 64:55
- look, this is fine because we've got an explanatory comment
- 64:58
- The, you know
- 65:00
- idea behind lots of folks like Lauren Tan
- 65:03
- is that
- 65:04
- the comments are actually a
- 65:05
- safety valve that you are using too often
- 65:08
- that you are basically saying
- 65:09
- okay, because I've got this comment
- 65:13
- I'm allowed to do the workaround
- 65:14
- Whereas if you were banned from doing the comment
- 65:16
- you might actually not take the workaround
- 65:18
- All this to say
- 65:19
- what I would like you to do
- 65:20
- is to tell me which of these could
- 65:22
- be moved into ADRs
- 65:23
- which of these could actually just become non-workarounds
- 65:27
- in other words, use something you don't need a comment for
- 65:30
- and which of them could
- 65:32
- just disappear because they're not
- 65:34
- actually needed or you can actually infer it
- 65:36
- from the code
- 65:37
- I want you to be more aggressive here
- 65:38
- Let's see what it says here
- 65:40
- I don't know, let's see
- 65:40
- Okay, guess what? It agrees with me
- 65:42
- The comment next to a workaround
- 65:43
- Off amongst a place where the code should
- 65:44
- have been designed differently
- 65:45
- not explained. Haha
- 65:47
- It's guessing that the TL drawer and command
- 65:49
- K quirks mostly become adapter wrappers
- 65:52
- The race constraints mostly become test names
- 65:55
- Yeah
- 65:55
- that's a legitimate place to have some kind
- 65:57
- of like
- 65:58
- free text is just like the name of
- 66:00
- a test. And while that has happened
- 66:02
- our PR
- 66:03
- the
- 66:03
- mega PR stack has now been merged to
- 66:06
- production. So I've got this loaded up
- 66:08
- We should be able to capture my face
- 66:09
- here. And there we go
- 66:10
- We have my face
- 66:12
- and we can see that I can
- 66:13
- add text to the thumbnail like this
- 66:16
- So I can add a big box here
- 66:17
- that my hand is kind of pointing to
- 66:20
- I can go
- 66:20
- all the way down
- 66:22
- and I can see that is how it's
- 66:23
- going to look. There's like a button here
- 66:25
- saying
- 66:25
- a dark panel behind the drawing so I
- 66:28
- can take that out or put that in
- 66:29
- if I want to
- 66:30
- And I mean it's pretty good guys
- 66:32
- Like that is not bad
- 66:33
- If I exit out to here
- 66:35
- then we can see my thumbnail is ready
- 66:36
- to go. I can select it when
- 66:38
- I then upload it to YouTube
- 66:40
- I can click back through and edit it
- 66:42
- and back
- 66:43
- We are should be back to this screen
- 66:45
- here. Nice. So you can see exactly how
- 66:47
- much fidelity it has to the prototype that
- 66:49
- we created. It seems to be working okay
- 66:52
- I probably do have some like follow-up feedback here
- 66:54
- but overall, yeah, I can insert icons
- 66:57
- Let's just add this one
- 66:58
- for instance. Bam, like that
- 67:00
- Put that on my finger
- 67:01
- Lovely. And there we go
- 67:02
- Just like that, I've built a feature
- 67:04
- something that I will enjoy
- 67:06
- using for the rest of the time that
- 67:08
- I use this application. And inside our agents view here
- 67:11
- we can see that
- 67:12
- yes, okay, when I tell it to be more aggressive
- 67:14
- it says 22 can simply be deleted
- 67:17
- There are 18 workarounds that need changing
- 67:20
- Look at that. I mean
- 67:21
- that is wild. I feel like there might
- 67:23
- be a skill here
- 67:24
- you know
- 67:24
- like
- 67:25
- Just looking at all of your comments that
- 67:26
- are in the repo and just using those
- 67:29
- as smells and just say
- 67:30
- okay, sniff, sniff, sniff, take a look at that comment
- 67:33
- see what workaround it's describing
- 67:35
- Can we just get rid of the workaround
- 67:36
- get rid of the comment?
- 67:37
- And it does say that there are three ADRs
- 67:40
- Yeah, there you go. So
- 67:41
- CVM custom shapes are a persistence contract
- 67:43
- face cam centering is tuned by eye
- 67:45
- diagram PNGs render in the running app
- 67:47
- All three of which I agree
- 67:49
- make good ADRs. And it's giving me a
- 67:50
- suggested PR
- 67:52
- order here. So let's prompt it one more
- 67:53
- time just to tie this up
- 67:55
- I want you to take all of
- 67:57
- these findings and put them in
- 67:59
- one more issue
- 68:00
- that will be the kind of umbrella issue
- 68:01
- for this idea. And that
- 68:03
- as they say, is that
- 68:05
- Now what I would do next is I
- 68:06
- would go
- 68:07
- into the thing that I just created
- 68:09
- I would
- 68:10
- try to find
- 68:11
- anything that I could improve and QA it
- 68:14
- And then those would go into issues
- 68:15
- which I would then just slam through with an agent
- 68:18
- probably not even grill them
- 68:19
- I would then go back through the issues
- 68:21
- get my agent to take a look at all of them
- 68:23
- and turn those into code base improvements
- 68:25
- And all of those
- 68:25
- I would expect the diff to be really
- 68:27
- really small
- 68:28
- and so I could just one-shot them
- 68:29
- But that is it
- 68:30
- folks. Hopefully you enjoyed this kind of deeper
- 68:32
- dive of me actually walking through how I
- 68:34
- do things. And hopefully you get a good sense for
- 68:37
- like
- 68:38
- where I am most valuable in this process
- 68:41
- I'm most valuable early on
- 68:42
- in the alignment phase where we talk through everything
- 68:45
- we get alignment, I understand exactly what's being built
- 68:47
- I create a prototype
- 68:49
- we figure that all out together
- 68:50
- And then I am really useful at the
- 68:52
- end
- 68:52
- where my technical expertise can come in and
- 68:54
- I can impose my coding standards
- 68:57
- And I can say
- 68:57
- no, that looks bad, let's do this instead
- 68:59
- In other words
- 69:00
- grilling and retro
- 69:01
- are really the two main points at which
- 69:03
- I interact with the system
- 69:05
- And then, of course, you know, QA and, you know, putting in my taste
- 69:09
- making sure that everything feels right and is
- 69:11
- working how I want it to
- 69:12
- But thank you so much for watching
- 69:14
- Hopefully you enjoyed this
- 69:15
- wow
- 69:15
- nearly
- 69:16
- hour and ten minute video so far and
- 69:18
- that you learned something from it
- 69:19
- I love doing these deep dives
- 69:20
- I don't get enough time to do them
- 69:22
- So, you know, if you like this
- 69:24
- then do share. With your friends
- 69:25
- and that will incentivize me to make more
- 69:27
- of them. Thanks so much for watching
- 69:28
- See you very soon
Watch these videos on your TV
XTTV brings video content from X to your living room. Download now and start watching on the big screen.