I've never run an AI model on my own machine, and I assumed the cloud was the best way to get good agentic coding. Then I talked to Deepu K Sasidharan, a Principal Developer Advocate at Auth0 and long-time open source maintainer, who now does most of his coding with a model running on his laptop. He's also the creator of LlamaStash, a Rust tool for running local models.
From skeptic to local-first. Initially, Deepu tried every new model version and wasn't so impressed, until one found bugs in code he had written by hand. That was the moment he stopped resisting and started experimenting.
Local models vs. frontier models. By his own benchmark, a local model on a 128GB laptop lands in the same range as a top cloud model on a complex task, with similar speed apart from a roughly twenty-second wait on the first big prompt.
Why he built his own tool. Ollama felt slow and LM Studio didn't give him enough control, so an initial bash script turned into LlamaStash, a Rust CLI and terminal UI that can run any engine and model combination.
The engineer as architect and manager. He doesn't write most of the code anymore. He plans, sets requirements, and manages a few agents, and he says he doesn't miss the old way.
Things that stuck with me
I asked Deepu to explain llama.cpp like I'd never heard of it. It's the engine that loads the model file into your GPU or RAM and does the computation, and tools like Ollama are wrappers on top of it. His suggestion was to try LlamaStash's init command, or LM Studio if you prefer a GUI.
The part I keep coming back to is how he sets up his harness:
- One setup, two harnesses. He uses a different harness for the cloud model and for the local model, and they share the same rules file, plugins, and memory location through symlinks.
- Cloud for planning, local for building. An orchestrator skill has the cloud model write the plan and do the final review, while local agents implement it in parallel. A feature that would have cost around $20 on cloud alone now costs about $1, since the local part is just electricity.
- Reviews from many angles. Three review rounds can look at a change from twelve angles, including security, correctness, and an adversarial one, which is more than one person skimming a diff would do. If problems remain after three rounds, it stops and tells him, and he says it's usually his own unclear requirement.
- Small context on purpose. A huge context window doesn't need to be filled, and the more you fill it, the slower and less accurate the model gets.
Final thoughts
Deepu's setup isn't about picking local over cloud. It's about deciding what runs where, and staying in the seat of the person who sets the requirements and checks the work. Local models make that choice yours, including what happens to your code, your prompts, and your secrets.
Would you give a local model the task of doing most of your coding? Let me know in the comments!
Listen to the full episode
Available on YouTube, Apple Podcasts, and Spotify.
Thanks for reading!