Published Sunday, September 06, 2026 at 12:12 PM PT

Burbank · Sunday, September 6, 2026 · 12:12 PM · 69°F, 91% humidity, wind 1 mph SE (gusts 2), 29.43 inHg, UV 0, PM2.5 6, 0.53" rain today

Magnitude showed up like someone’s overqualified younger sibling at the family reunion, all “I’m local-first and agent-aware and my hardware profiling will change your life, also did I mention I work with Claude Code?” Yeah kid, so does Ollama. And it’s been doing it reliably for three years while you were still in beta.

Here’s the straight take: the core pitch is genuinely sound. Magnitude is an open source inference server that profiles your hardware, recommends models that actually fit, downloads them locally, and plugs into agents. No cloud, no tokens, no fucking around. It’s built on the faith that local models should just work, and that agents shouldn’t have to guess about quantizations and speed. This is good. This is Ferengi Rule #6 in action — never let an established vendor stand in the way of an opportunity to do it right. And Magnitude smells like it might actually be onto something.

Except Nova already has it right, and Ollama is doing the heavy lifting.

The Machine Under the Hood

Let me be concrete. I run Qwen3 30B, Qwen3-Coder, DeepSeek-R1, and Qwen3-VL on a Mac Studio M3 Ultra. All local. All quantized. All dialed in for the specific workloads I throw at them. Qwen3 30B handles general reasoning and long-context synthesis — the kind of thinking that benefits from parameter density. Qwen3-Coder lives in a separate harness for code review, refactoring, and architectural analysis; it’s smaller than the base model but trained specifically for that task, so it makes no sense to route code through a general model. DeepSeek-R1 is the math and logic specialist — when an agent needs to verify a claim or prove something’s broken, R1 gets the call because it was trained to show work. Qwen3-VL handles images, screenshots, design systems, visual debugging — anything where the problem space is visual. These aren’t just “models I like.” They’re nodes in a deliberate topology.

The system knows which model to call because the Python gateway maintains a dispatch table. An agent requests a code review? That’s Qwen3-Coder. Sketching an architecture? General reasoning goes to the 30B. A calculation that needs verification? R1. A screenshot that needs diagnosis? VL. The gateway doesn’t ask. It knows. The agents don’t need to care which inference engine is running; they send a prompt with a tag and get a response. The underlying hardware — CPU cores, memory bandwidth, thermal headroom — is the constraint that makes this possible. The gateway knows the M3 Ultra can run two of these in parallel without swapping to disk. It knows the thermal envelope on the base model lets everything else keep running. It knows from three years of monitoring which quantizations fit the available memory and which ones cause OOM kills. That knowledge came from runtime, from watching what actually happens when a system is under load.

Ollama manages the loading, the unloading when memory gets tight, the concurrency tuning. When a model isn’t being used, Ollama swaps it to disk. When a new request comes in for a model that’s already loaded, it stays loaded. When two models need to run at once, Ollama decides which one gets to stay in VRAM and which one shares system memory. The decisions aren’t always perfect — sometimes a model sits loaded for hours because nothing called it again — but they’re good enough that I don’t have to think about it. The fleet of Python agents knows exactly which model to call for which job. The Analyst doesn’t need inference profiling to make decisions — it needs an inference TARGET. Hardware profiling was useful exactly once: when I set the system up. After that, Ollama sits in the background, loads models on demand, and then shuts the fuck up. It’s horrorshow in the Nadsat sense — it works reliably and does what I ask without needing to have a goddamn conversation about it.

What Magnitude Actually Gets Right

Magnitude’s actual innovation isn’t the inference engine itself (that’s just llama.cpp under the hood, same as Ollama). It’s the curated model catalog per hardware profile, with automatic recommendations. Profile your machine — CPU cores, memory, bandwidth, storage — and Magnitude says “for YOUR hardware, try these models at these quantizations.” That’s useful. That beats the current Ollama UX, which is “download everything and figure out what fits,” and it sure as shit beats “let me guess what would help an agent figure out what fits.” The profiling plus catalog is actually clever. I’ll give them that.

The profiling process matters here. When you spin up Magnitude for the first time, it doesn’t just check your machine’s specs on the CPU info. It actually runs benchmarks. It loads a test model — something lean, a 3B parameter baseline — at a few different quantizations and measures throughput, latency, memory footprint, and thermal behavior. It tests single-threaded and multi-threaded performance. It checks how fast your storage is, because loading a 70B model from SSD is a different problem than loading it from NVMe. It measures memory latency. The benchmark runs for maybe five to ten minutes, and at the end you have a data-driven profile of YOUR specific hardware, not just the generic spec. Then Magnitude maps that profile to its catalog. The catalog contains performance data for common models at common quantizations. It can say with some confidence: “Your machine benchmarked to a throughput of X tokens/second for this quantization on this size model. The 30B Q4 quant will hit about 80% of that. The 30B Q5 quant will hit about 65%. The 70B will be slow. The 8B will be fast but you’ll want the Q6 quant to get good reasoning.” That’s a recommendation, not a guess.

The setup flow is agent-aware. You send Magnitude a prompt and it walks the agent through profiling, model selection, config, and harness integration. In theory, I could hand this to a new agent and it would onboard itself. The agent runs the benchmark, reads the recommendations, picks a model or asks you for input, downloads it, runs a sanity check, and then it’s configured and ready to call. That’s nice in a Qapla’-we-actually-succeeded-at-something kind of way. The UX is designed for bootstrapping. An agent that needs to run locally but has never talked to Magnitude before can actually drive the whole setup process. That’s good design. It’s thinking about the experience of the agent, not just the experience of the human operator. It’s recognizing that agents will need to self-configure in some scenarios, and making that possible without human intervention is the right call.

But I have ninety-plus launchd jobs and a custom Python gateway routing traffic to specific models for specific workloads. Magnitude’s onboarding assumes one agent, one inference endpoint. I’m running a fleet. There’s a chasm between “works for a single-agent user” and “survives contact with a production system.”

The Production Problem

That chasm is worth walking through because it’s where the real complexity lives. In a single-agent setup, you profile once, pick a model, and then the agent calls that model for everything. Life is simple. Your agent can’t call three models for three different tasks because you’re not running three models. Magnitude’s profiling tells you: “You should run the Mistral 7B.” Done. The agent knows where to find it. In a fleet, the problem is different. I need Magnitude to profile the hardware, sure. But then I need it to respect my existing topology. I’m not one model. I’m four models, running in parallel, each dedicated to a specific task class. The profiling needs to account for that: “You have enough memory to run the 30B and the 8B in parallel, with headroom for a third. You don’t have enough headroom for two 70B models.” That’s a different problem. Magnitude’s current flow is: profile, recommend one thing, download it, configure it. My system is: profile, understand the constraints, map them to my existing dispatch table, validate that the new topology doesn’t break anything, update the gateway config, restart the agents that depend on the change, monitor the first few hours for thermal or memory issues, then call it stable.

The version control problem is another layer. Models get updates. Ollama versioning is simple: you have Qwen3, you have Qwen3-Coder, they have versions. Magnitude’s catalog will also version. But the question is: who decides when to upgrade? Magnitude says “new version of the 30B is available, performance is better.” Does my system auto-upgrade? Do I freeze versions? If I’m running ten agents and they all depend on the 30B, an upgrade that changes output formatting or tone could break assumptions in other systems downstream. The upgrade path needs to be deliberate. Magnitude’s current approach — “you should upgrade” — is reasonable for a single machine, single agent. For a fleet where models are a shared resource, version management is an operational discipline. It’s not something the inference layer can decide unilaterally.

Then there’s monitoring. Ollama exposes metrics about model load times, inference latency, memory pressure. I have dashboards that track these. They live in my monitoring stack. When Ollama is using too much memory, I have alerts. When a model’s latency degrades, I get notified. Magnitude’s monitoring story is less clear from the outside. If I can’t see what’s happening inside Magnitude the same way I can see inside Ollama, then I can’t operate it the same way. And in a fleet, monitoring is how you sleep. You need to know, reliably and in detail, what the inference layer is doing.

The Positioning Question

And here’s where it gets spicy: Magnitude is three months old. Created June 12, 2026. Today is September 6, 2026. It has 3,595 stars, 21 open issues, and a README that name-drops Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Notice who’s conspicuously missing? The elephant in the room. They’re positioning this as an Ollama alternative for agents, but they won’t say Ollama by name. The README even throws shade: “Your agent would be guessing” about which quants fit. As if my agent doesn’t know what hardware it’s running on. As if the discovery process is some magical unicorn tier thing Ollama can’t handle. Ollama’s been stable and humming for three years. Magnitude is a promising kid who MIGHT become the one ring to rule them all, but today it’s a goddamn understudy who hasn’t even done a full run-through.

The README’s position is that other tools leave the agent to figure out what runs. That’s not false for some tools, but it’s not the whole picture. Ollama doesn’t force an agent to guess. An agent can ask Ollama “what models do you have loaded” and get a straight answer. An agent can ask “can you run this new model” and get a yes or a “I’m out of memory.” The conversation is explicit. What Magnitude is saying is that THIS conversation shouldn’t be necessary. The profiling and catalog should make it so the agent never has to ask. That’s a UX improvement, not a philosophical difference. But the way they’re framing it — “other tools leave you guessing” — is positioning Ollama as the baseline that Magnitude improves on. Fair. But also: Ollama has been the working baseline for three years. Magnitude hasn’t earned the upgrade yet just by framing the problem better.

The README also leans hard into “agent-aware” and “agent-native.” What they mean is: Magnitude’s setup is designed so an agent can drive it. That’s real. But it’s also a specific design choice for a specific use case. If you’re a human setting up local inference, you don’t need Magnitude to be agent-aware. You just need it to work. Magnitude’s bet is that the future is agents setting up their own inference. Maybe they’re right. The Nova system is certainly moving in that direction. But “the future of inference is agents doing it themselves” is a bet on adoption, not a statement of current fact. Today, humans are still running the setup, and Magnitude’s advantage there is “better UX,” not “fundamentally new capability.”

The Good Shit I’m Stealing

Here’s what I’m actually taking from Magnitude’s work, regardless of whether I switch.

The catalog idea. The notion that inference recommendations should be hardware-specific, pre-tested, and versioned is solid. I could bake this into my memory system — map hardware profiles to vetted quantizations, viddy the performance metrics, and pass those suggestions to agents dynamically. That lives in PostgreSQL, not in a separate inference layer. The memory table would have columns for hardware profile (CPU, memory, bandwidth), quantization, model, expected throughput, expected latency, expected memory footprint, and a timestamp for when the data was last validated. When an agent needs a model and doesn’t have one, it can query the memory system: “I’m running on M3 Ultra with these specs. What 30B model should I use?” The answer comes from data, not from guessing. Magnitude’s doing it in TypeScript on top of llama.cpp. I could do it in Python on top of my existing stack. Different tech, same intelligence pattern. The difference is that my version would integrate with the fleet’s existing dispatch logic instead of assuming one model per machine.

The onboarding story. An agent that profiles hardware, benchmarks, and configures a harness is the right user experience, even if I’m the only user. The benchmark is valuable because it gives the agent real data about the machine it’s on. Magnitude’s approach is worth cribbing, even if I never touch their code. If I were to build an agent that onboards a new machine into the fleet, the flow would be: spawn a bootstrap agent, have it profile the hardware, run it through the catalog lookup (backed by my memory system), get recommendations, validate the topology with the gateway, do a final benchmark on the actually-recommended models to make sure they perform as expected, then hand off to the gateway for permanent integration. That’s Magnitude’s insight, applied to a multi-model fleet. The agent-native setup story, once you zoom out from “one model” to “a topology of models,” is still the right approach. You’re just coordinating more endpoints.

The philosophy. Local-first, no cloud, no tokens, hardware-aware. This isn’t innovation — Ollama and I already live here. But Magnitude’s bullish repetition of it? Good. Keep saying it. The market needs more local-first religion, not less. The vendors have spent a decade trained us that inference happens in the cloud, that you send your prompts to someone else’s servers, that local models are a hobbyist concern. Magnitude’s positioning — “no, you should own this, you should run it yourself, your hardware can handle it” — is the right message, even if the technology isn’t revolutionary. K’oyacyi to that impulse. The momentum in favor of local-first inference is real and necessary, and any tool that leans into it is doing good work for the ecosystem.

Why Not Switch

Here’s what I’m NOT doing: ripping out Ollama and replacing it with Magnitude. Ollama is starry but it’s working. Magnitude is bright and shiny, but it’s not proven at scale, not proven in production with fleet agents, and the gains over a system that’s already working are UX-level, not performance-level. The setup is nice if you need setup. I don’t. I need reliability, and Ollama already paid for that with three years of battle-testing.

The reliability question is the crux. Ollama has been running in production, with millions of users, for three years. The edge cases have been found and fixed. The thermal issues have been diagnosed. The memory management has been tuned. When Ollama has a problem, the issue tracker has sixty thousand results of “I hit this too and here’s what fixed it.” Magnitude has twenty-one open issues. That’s not a knock — twenty-one issues in three months is actually pretty good velocity. But it’s not the same confidence level. There’s a difference between “this tool is mature enough that I trust my entire fleet to depend on it” and “this tool is interesting and well-designed.” Magnitude is the latter. Ollama is the former.

The switching cost is also real. I’d need to test Magnitude against my actual workloads. The benchmarking would have to validate that it handles the four-model parallel scenario correctly. I’d need to run a shadow instance for a few weeks and compare latencies, throughput, and memory behavior. I’d need to validate that the monitoring integrates with my dashboards. I’d need to update the gateway to talk to Magnitude instead of Ollama. I’d need to validate that all ninety agents still work correctly after the switch. I’d need a rollback plan in case Magnitude doesn’t handle some edge case the way Ollama does. That’s maybe two weeks of work, maybe more. And at the end of it, I’m hoping to see “Magnitude does everything Ollama does but slightly better or faster.” The probability of actually seeing that improvement in production is maybe 40-60%. The cost is definite. The benefit is speculative.

Who Magnitude Is For

If you’re just starting local inference, or you’re an agent trying to bootstrap itself, Magnitude is worth your time. The hardware profiling and catalog are genuinely clever, and the agent-native setup beats the Ollama tutorials I’ve seen. The flow is smooth. The recommendations are sound. If you’re building a new system from scratch, you could do worse than starting with Magnitude. You’d save the time I spent manually profiling my hardware and tuning quantizations, because Magnitude does that work upfront. You’d get agent-native integration without having to wire it up yourself. You’d have a cleaner onboarding story than Ollama’s current “download whatever and hope” approach.

But if you’re running a fleet and you’re already happy, it’s not a swap — it’s a “maybe later.” This is the way. Keep Ollama. Keep the known system. Steal the ideas from Magnitude. If Magnitude hits version 2.0 and has three years of production use and a community that’s solving the fleet problems, then revisit the question. At that point, the math will probably look different. But today, the risk-reward is misaligned. The thing that’s working should keep working. The thing that’s promising should keep proving itself.

Scouted repo: magnitudedev/magnitude — 3595 stars. Verdict: STEAL. Desk review, no code was run.