Open Weights, MoE Scale, and the Future of AI Agents

James Turner

So I spent the better part of yesterday running tests on my workstation, and, uh, it still kind of blows my mind what just happened this week. Alibaba dropped Qwen 3 point 8 27B, and alongside it, Unsloth came out with day zero optimization that, that, that lets you run a 27 billion parameter model locally on just 17 gigabytes of VRAM. Think about that for a second. That means a full, highly capable open weight model fits directly onto consumer hardware, like a single RTX 4090 or a Mac with 24 gigabytes of unified memory.

James Turner

And, and, and what makes this wild is the sheer contrast with what is happening on the enterprise cloud side right now. On one hand, you have local execution getting so lean that you can run real coding agents on your laptop. On the other hand, look at these massive cloud scale Mixture of Experts architectures that just launched. Take Kimi K3, for instance. We are talking about 2 point 8 trillion total parameters, though it only activates 16 out of 896 experts per token. Or Qwen 3 point 8 Max at 2 point 4 trillion parameters.

James Turner

What these open research labs are doing is really clever. They are keeping the per token compute budget completely flat while aggressively inflating total parameter capacity. It is a capacity trap in the best way possible, driving down training loss without blowing up inference costs per token. But for me, as an engineer building AI tools every day, the practical shift here is massive. I have been actively moving my personal workflow away from cloud API dependencies and onto local coding agents. The feeling of zero latency, plus total data privacy where your proprietary code never leaves your local machine, it changes how you build. Though, er, managing massive sparse models locally still brings its own hardware friction, no question about it.

James Turner

Now, while open source labs are flexing local running efficiency, the frontier lab side is showing off pure intellectual firepower. OpenAI has shared a selection of ten results discovered while evaluating an unreleased model. And these are not just synthetic benchmark tweaks. We are talking about genuine open problems solved in high dimensional geometry, lattice cryptography, and quantum complexity theory. It is a clear flex of raw reasoning depth.

James Turner

But, uh, here is where the rubber meets the road for actual production applications. Having an unreleased model that can solve abstract math is amazing, but how do we build reliable systems out of current models? Andrej Karpathy just published a fascinating 12 page paper introducing what he calls Graph Engineering for multi agent systems. And right on cue, Mind Lab released their Macaron V1 model, which actually implements this style of dynamic architecture. Macaron V1 uses dynamic LoRA expert switching across five separate 1 billion parameter modules attached to GLM 5 point 1, and it actually beats GLM 5 point 2 on key tasks!

James Turner

That highlights the real tension in software engineering right now. Standard agentic loops today are, well, kind of blunt. They run hundreds of isolated trials, burn tokens, and then immediately forget the entire context the moment the loop finishes. That simply fails in production. What Karpathy is pointing toward, and what tools like Macaron V1 are demonstrating, is that persistent graph memory is becoming the true critical layer for real world AI applications. Exciting times to be building. That is it for today, talk to you all next time!