I bought a Mac Studio in February, before the craze happened. A month later I added an ASUS GX10, which is basically the equivalent of NVIDIA’s DGX Spark. Two machines sitting in my office in Austin, quietly running AI models that never touch the internet.

Here’s the funny part: I didn’t need them. I run everything on frontier models in the cloud. Claude, GPT, Gemini… that’s my daily stack and it works great. So why did I spend over 200 hours tuning models on hardware I technically didn’t need?

Because I truly believe the future of AI is local. And I wanted to understand it before everyone else woke up to it.

Why local, when the cloud is so good?

Think about your iPhone for a second. It has your messages, your photos, your health data, your location history. Would you want all of that streamed to a cloud company, all the time, forever? Of course not. At some point we want our privacy back.

Now apply that same logic to a business. There are companies right now that simply cannot go to the cloud. Their IP is the business. Their client data is regulated. KYC documents, financial records, medical files, trade secrets. For them, “just use ChatGPT” was never an option. Local AI is.

When Alex Karp at Palantir started talking about this, I noticed something shift. Entrepreneurs who had never once asked me about local models suddenly wanted to know what their AI strategy should look like if the data can never leave the building. Microsoft’s CEO wrote something recently that stuck with me too: as AI gets better and stronger, the only moat left is your own internal IP. If that’s true (and I think it is), guarding that IP becomes the whole game. That’s exactly where local comes in.

What 200 hours of tuning taught me

Let me be honest about what running local models is actually like, because most articles on this are written by people who spun up a model once, generated a haiku, and called it a day.

The models I’ve spent the most time with are the Qwen family. I run a 27B dense model on the Mac Studio and a 35B mixture-of-experts model on the GX10. If those terms mean nothing to you: a dense model uses all of its brain for every single question, while an MoE model routes each question to a small group of specialists inside itself. The MoE is faster per token. The dense model is steadier on long, complex work. I didn’t learn that from a benchmark chart. I learned it by running the same jobs on both, over and over, and watching where each one fell apart.

And they do fall apart. That’s the thing nobody tells you. A local model out of the box is like a brilliant intern with no manager: capable, fast, and fully willing to repeat the same mistake 400 times in a row without noticing. Getting reliable work out of one is less about the model and more about the harness you build around it. My favorite harness right now is pi, a bare-bones terminal agent that I’ve written custom extensions for. I wrote a whole post about that journey, the model vs. the harness, if you want the technical war story.

Here’s what surprised me most: this is the most I’ve ever learned about AI, period. Not just local AI. When you use frontier models, all the rough edges have been sanded off for you. You never see how the sausage is made. When you run a model on your own hardware, you see everything: how it fails, how context windows actually behave, why one model style beats another on a given task. It made me sharper with the cloud models too.

The economics are stupid

Once the hardware is paid for, a local model costs you electricity. That’s it.

I have automations running at home right now where the marginal cost per task is effectively $0. High-volume, repetitive work (triage, extraction, classification) that would rack up a real API bill runs all day on the GX10 for pennies. My routing rule is simple: cheap local models handle the high-volume grunt work, a frontier model does the hard builds, and a different frontier model verifies the work so the checker doesn’t share blind spots with the builder.

Do the math on that for a business processing thousands of documents a month. The cloud bill never ends. The Mac Studio was paid for in February.

This stopped being a hobby

For months this was my nights-and-weekends obsession. I’d tell entrepreneur friends about it and get polite nods.

Then a financial services company here in Austin came to me with a problem: they need AI, and their data cannot go to the cloud. Not “prefers not to.” Cannot. So right now I’m deploying local AI inside that company. Models running on their hardware, on their network, touching their sensitive data without a single byte leaving the building.

That’s when I knew the 200 hours weren’t a hobby. They were preparation for a wave that’s just starting to form. More companies are waking up to this every month. The hardware is capable, the open models are good enough, and the pressure around data privacy keeps building.

Should you go local?

Probably not yet, honestly. If your data can live in the cloud, the frontier models are better, easier, and worth every penny. I still run my own work on them.

But ask yourself 3 questions. Does your business have IP that can never be shared? Do you hold client data (KYC, health records, financials) that regulators or clients expect you to guard? Would a data leak be an extinction event for the trust you’ve built?

If you answered yes to any of those, local AI needs to be on your radar this year. Not deployed tomorrow. On your radar. Start by figuring out which of your workflows touch data that can never leave, because that list is your local AI roadmap. Everything else can stay in the cloud.

The machines are cheap, the models are good enough now, and your IP is the moat. Guard it.