Skip to content
Artwork for Forward Deployed
Forward Deployed · September 2 · 1 hr 6 min

The State of Computer Use Agents | Anthropic, Browser Use & KERNEL

Bot traffic on the internet just passed human traffic, two years ahead of forecast. Most of it is agents clicking through websites built for people. So I got three of the people building those agents in a room: Lucas Gonzalez Pagliere, who works on computer use at Anthropic, the team that shipped the first computer use model back in 2024. Reagan Hsu, founding engineer at Browser Use, whose open source library is sitting at around 100K GitHub stars. And Eric Feng, founding customer engineer at KERNEL, which runs the browser infrastructure underneath a lot of this - he was also first GTM at Sentry. We covered where the models genuinely are today vs where the benchmarks say they are, what it costs to run an agent long enough to finish real work, and the arms race between agents and the anti-bot systems trying to keep them out. Then the harder question underneath all of it: whether the web reorganizes itself around agents, or hardens against them. They disagreed on plenty of it. If you want to know what agents can actually pull off on a real website today - and what still stops them cold - this one's worth your time! Chapters 00:00 Welcome and Guests 00:49 Companies and Stacks 01:14 Everyday Agent Use Cases 03:38 Defining Computer Use Agents 05:15 How Computer Use Works 08:41 Screenshots vs DOM Hybrid 11:03 Benchmarks and OSWorld 13:51 OSWorld 2 Difficulty Jump 15:43 Training Models and Cost 18:57 Speed Infrastructure and Stealth 21:55 Anti Bot and KYC Future 27:24 Reverse Engineering vs UI Automation 29:43 Computer Use vs Browser Use 31:05 Scaling Laws and Harnesses 32:49 Playwright Selenium Still Matter 33:17 Playwright Still Dominates 33:27 Why Run 1000 Agents 34:39 Long Running Agent Challenges 35:45 Memory and Compaction 38:10 State Changes Mid Task 39:36 OS and Browser Fingerprints 40:42 DOM Efficiency and WebMCP 41:36 Recsys and Agent Personas 43:57 Agent Friendly Websites 45:41 Human Speed vs Agent Power 48:32 Context Window Tradeoffs 50:34 Harnesses for Temporal State 51:59 Speed Optimizations and Tabs 54:23 Human Collaboration Limits 58:09 Raw Capability vs Better APIs 01:00:47 Training Methods and Bottlenecks 01:01:41 End State Interfaces 01:05:09 Next 12 Months Predictions 01:06:17 Closing Thanks

0:00-1:06:40

transcript

No transcript — this publisher did not publish one.

show notes

Bot traffic on the internet just passed human traffic, two years ahead of forecast. Most of it is agents clicking through websites built for people.


So I got three of the people building those agents in a room: Lucas Gonzalez Pagliere, who works on computer use at Anthropic, the team that shipped the first computer use model back in 2024. Reagan Hsu, founding engineer at Browser Use, whose open source library is sitting at around 100K GitHub stars. And Eric Feng, founding customer engineer at KERNEL, which runs the browser infrastructure underneath a lot of this - he was also first GTM at Sentry.

We covered where the models genuinely are today vs where the benchmarks say they are, what it costs to run an agent long enough to finish real work, and the arms race between agents and the anti-bot systems trying to keep them out. Then the harder question underneath all of it: whether the web reorganizes itself around agents, or hardens against them. They disagreed on plenty of it.

If you want to know what agents can actually pull off on a real website today - and what still stops them cold - this one's worth your time!

Chapters
00:00 Welcome and Guests

00:49 Companies and Stacks

01:14 Everyday Agent Use Cases

03:38 Defining Computer Use Agents

05:15 How Computer Use Works

08:41 Screenshots vs DOM Hybrid

11:03 Benchmarks and OSWorld

13:51 OSWorld 2 Difficulty Jump

15:43 Training Models and Cost

18:57 Speed Infrastructure and Stealth

21:55 Anti Bot and KYC Future

27:24 Reverse Engineering vs UI Automation

29:43 Computer Use vs Browser Use

31:05 Scaling Laws and Harnesses

32:49 Playwright Selenium Still Matter

33:17 Playwright Still Dominates

33:27 Why Run 1000 Agents

34:39 Long Running Agent Challenges

35:45 Memory and Compaction

38:10 State Changes Mid Task

39:36 OS and Browser Fingerprints

40:42 DOM Efficiency and WebMCP

41:36 Recsys and Agent Personas

43:57 Agent Friendly Websites

45:41 Human Speed vs Agent Power

48:32 Context Window Tradeoffs

50:34 Harnesses for Temporal State

51:59 Speed Optimizations and Tabs

54:23 Human Collaboration Limits

58:09 Raw Capability vs Better APIs

01:00:47 Training Methods and Bottlenecks

01:01:41 End State Interfaces

01:05:09 Next 12 Months Predictions

01:06:17 Closing Thanks