My AI Agent Bought Me A Car
Sam Herring’s first real test of Hermes Agent was a shopping tool that matched clothes to photos of ancient pottery. A few months later, the same agent was checking government auction sites every day until it found him a retired three-wheel parking ticket car, which he now drives around San Francisco. In this episode, recorded at AGNTCon + MCPCon North America in San Jose, Sam, who works on the RL post-training team at Nous Research, goes from those everyday wins to the question everyone in AI is asking: can agents improve themselves? He explains how Hermes grew from an internal productivity tool into one of the most talked-about open-source agents, and why letting it write its own skills was the moment it clicked for users. He also describes an early version that could design its own RL environment, train on Thinking Machines’ Tinker, and hot-swap the new weights into a model running locally. Sam then breaks down Trace to Environment, an idea he credits to Xiaomi’s MiMo-V2.6-Pro work, and the loop Nous is building around it: find the exact point where an agent fails, cluster those failures by type, synthesize environments that recreate them, and train on them again and again. The second half moves from training to inference. We get into whether reasoning models overthink, why leaning on pretraining knowledge can be a model’s Achilles heel, and why choosing a reasoning effort level may be a temporary quirk. Sam also makes his case for activation steering as an underexplored way to get more out of small models, explains where steering meets interpretability, and describes what true continual learning could look like. Demetrios Brinkmann: https://www.linkedin.com/in/dpbrinkmSam Herring: https://www.linkedin.com/in/sam-herring-67225819a Timestamps:[0:00] Cold open[1:21] Meet Sam from Nous Research[2:39] How Hermes Agent started[3:38] From pottery to a parking ticket car[7:40] Benchmarking life admin tasks[8:58] Agents that write their own skills[10:10] RL at Nous a year ago[12:24] Hot swapping self-trained weights[15:00] Xiaomi and Trace to Environment[17:22] Turning agent failures into training[20:27] Where the RSI conversation is heading[21:34] Token efficiency and overthinking[26:30] Steering models at inference time[32:35] Toward true continual learning[34:54] Wrap up