AI & Automation ·
Part 5 — Local vs. frontier: when the AI shouldn't live in someone else's cloud
So far, we've looked at developing our skills using Claude, ChatGPT, maybe even Grok, or Gemini. But we haven't really questioned whether we should be using a frontier model provided by one of the frontier lab providers, or if there is an alternative. We’ve assumed that you need and want the best and most capable model out there. Which up till about 6 months ago the answer was probably yes you do. But in the last six months we’ve seen a dramatic improvement in capabilities in both the frontier labs and open source.. now the frontier labs models are all more than capable to the point where you do not need to use their premier models for all tasks. Additionally open source models have caught up and are maybe as little as six months behind the frontier labs. This changes everything.
Everything I've described so far quietly assumed one thing: that the AI lives in someone else's data center, and you reach it over the internet. For most people that's just what AI is. We hear about the data center buildout and the shortage of Nvidia chips and the assumption is that that’s the only option. As of today this is a choice and not the only option. So let's look at making that choice in a deliberate fashion.
Call it the cloud fallacy — the unspoken belief that serious AI can only run on a giant provider's servers, and your only option is to send your data there and pay per use. That was true two years ago. It is not true now. And the gap between what people assume and what's actually possible is, again, enormous.
Let me start with why a business would even want to run a model locally — on its own hardware, inside its own walls — before I get to the fun stuff.
Data privacy and sovereignty. This is the big one for anyone in enterprise. If the model runs on a machine you control, your proprietary data and your customers' data never leave the building. Nothing is shared, nothing is sent anywhere, nothing is sitting in a third party's logs. For a huge number of data-protection and contractual requirements, "it physically never left our infrastructure" is the cleanest answer you can give. No DPA negotiation, no "where is this processed," no awkward questions in the security review.
Dedicated models with proprietary knowledge. When you run your own, you can load it up with your own material and tune it to your domain — a model that knows your products, your configuration, your documentation — without ever exposing any of that to an outside provider. It becomes your asset, not a feature you rent.
Resilience — the denial-of-service problem. When your process depends on someone else's API, you've quietly taken on their outages, their rate limits, their price rises, and their right to deprecate the exact model you built around. I've had cloud models change under me overnight. A local model doesn't get rate-limited, doesn't go down because a provider three time zones away is having a bad day, and doesn't suddenly cost triple. You own your uptime.
And here's what's changed that makes all of this real rather than theoretical: in just the last six to eight months, local open models have become dramatically more capable, dramatically smaller, and dramatically cheaper — often completely free. Things that genuinely needed a data center last year now run comfortably on a Mac Studio, or a gaming PC with a couple of Nvidia cards in it. The cost of entry keeps falling, and the quality keeps climbing. The line of "what you can do on your own hardware" is moving every month.
Let me show you two of my own builds that sit at different points on this spectrum.
Fully local: an offline companion. This is the one closest to my heart. It's an offline companion I built for a family member — a little edge device, about the size of a paperback, that sits in their home. They can talk to it; it helps them remember things, and it's someone to have a conversation with. Under the hood it's a small edge computer running a local language model, with speech in and out, a vision pipeline that's aware of the room, and its own memory of past conversations. The critical design constraint was this: it had to work with no internet connection and no maintenance. It can't depend on a cloud that might be down, a subscription that might lapse, or me logging in to fix something. It just has to sit there and work. And because everything runs on the device, every conversation they have with it stays on the device. For something this personal, that wasn't a nice-to-have — it was the whole point.
This was a great learning project, more than a product that will eventually be built. It might be clunky in its speech compared to the ChatGPT app on your phone, but it was a great learning experience. It helped me understand how, when you deal with AIs in a system, you actually deal with multiple AIs that have very specific functions. Whether it's speech-to-text embedding with one model, RAG encoding with another model, text-to-speech with a third model, conversational guard rails with yet another model and an immediate conversation filler model to avoid long pauses. All of this in addition to the thinking engine behind that. All of those things combine so that to the outside user, it is one AI, but it taught me that an AI system actually uses multiple AIs, with some code to string them together to work in a certain way. That is a massive learning that takes you from just being a passive user to a deeper understanding as to how the systems work and it's not one size or one model fits all.
Hybrid: a video engine. Most real systems aren't purely local or purely cloud — they're a sensible mix, using each where it wins. My AI video engine is a good example. For the genuinely hard creative work — generating the slides and imagery — it calls out to frontier models, because that's where they're worth paying for. But it's deliberately frugal everywhere else. I didn't want to pay a hosted service to keep a clone of my voice, so I used an open-source voice-cloning model that did it for free, running locally. And the final video gets rendered on my own MacBook rather than a paid cloud render farm. Cloud for the parts that need it, local for cost and control. That's the pattern most businesses will actually land on.
So, back to the business case for a moment, because it's the thing I'd want a client to hear: if you have proprietary or customer data, a local model means none of it is ever shared — it simply sits on a machine you own, which clears a remarkable number of data-safety hurdles before you've even started. And on cost, the math increasingly favors owning the hardware for steady, high-volume work. A Mac Studio or a modest GPU box is a one-time spend, not a meter that runs every time someone asks a question.
The honest decision rule is simple: run it local when the data can't leave, when you need guaranteed uptime, or when you're doing high volume and want predictable cost. Use a frontier model in the cloud when you need the absolute best reasoning or to scale on demand. And reach for hybrid — like the video engine — for almost everything in between.
Speaking of that engine, let me show you rather than tell you. I produced a short explainer video with it, which you can watch on the LinkedIn version of this article. It's still a work in progress. But the takeaway here is it's a mixture of local and frontier models.
A couple of things to know as you watch. The voice you're hearing isn't actually me — anyone who knows me might be confused. It's a clone, generated for free on my own machine. And the slides weren't designed by hand; they were generated off the back of a chat-based storyboarding process, where I talk through what I want and the system builds the deck. The whole thing means I can knock out an explainer presentation in a fraction of the usual time. It was originally built to produce educational and marketing content for the BluCanto website — but honestly, with some polish, it's good enough that it could become a product in its own right.
Which is the perfect place to end this one. Because everything in this series — the simple wins, working in the folder, the DevShop, the local-versus-cloud thinking — all of it converged on one real, live product. In Part 6, the finale, I'll tell you the story of www.blucanto.com: how it started as a research exercise in ranking business ideas, how I picked the one I believed in, and how I took it all the way to a shipped, paying-customer SaaS.
If you take one thing from this post: "it has to be in the cloud" is a habit, not a requirement. For a lot of serious work — especially anything involving data you can't afford to share — the right answer is increasingly a model that runs on hardware you own.
— Jerome
Originally published on LinkedIn, July 4, 2026.