Oct 14, 2026·17 min read
Local LLMs Are Catching Up Fast, If Your Hardware Can Keep Up

Local LLMs Are Catching Up Fast, If Your Hardware Can Keep Up

local-llm
ollama
benchmarks
homelab
coding-agents

I run my agents on my own hardware wherever I can. That raises a fair question: am I wasting my time? Cloud models are bigger, faster and cheap. So I stopped going on impressions and measured it.

I tested two local models on the two machines I own, ran one of them on both machines, and compared everything with one cloud model, on the same 18 Python coding problems. Each problem has hidden bugs and blanks to fill in, and is scored by hidden tests. Here is the short version.

  • On correctness, a 3B-active model matched a 320B cloud model. Ornith 1.5 35B-A3B on my K8 Plus and GLM-5.3 Flash on Ollama Cloud both solved 10 of 10 basic problems and 7 of 8 hard ones on the first try. A second Ornith run, on my MacBook, scored 10 of 10 and 6 of 8.
  • The gap is speed, and speed is hardware. The cloud model finished seven hard problems in 11 minutes. The same small model needed 55 on my mini PC and 26 on my MacBook.
  • This is not a fair fight, and that’s the point. GLM-5.3 Flash has about 9 times the total parameters and 6 times the active parameters of Ornith. Even at 1-bit it needs about 100 GB of memory, which doesn’t fit my biggest machine.

The hardware

MacBook ProGMKtec K8 Plus (mini PC)Ollama Cloud
ChipApple M4 ProAMD Ryzen 7 8845HS (8 cores, 16 threads, Zen 4)Not disclosed (“datacenter-grade hardware”)
GPUIntegrated M4 Pro GPURadeon 780M iGPU, 16 GB carved out of system RAMNot disclosed
Memory48 GB unified96 GB DDR5-5600 (Crucial, 2 × 48 GB SO-DIMM), dual channel; about 78 GB visible to the OSn/a
Memory bandwidth273 GB/s89.6 GB/s theoretical (2 channels × 5,600 MT/s × 8 bytes)n/a
SoftwaremacOS, Ollama (MLX build for Qwen)Fedora, Ollama 0.40.3 with the Vulkan backendOllama API, same client
Models in the coding testsQwen3.8 27B, Ornith 1.5 35B-A3BOrnith 1.5 35B-A3BGLM-5.3 Flash
Measured speedabout 31 tokens/s (Qwen), about 61 (Ornith)about 26 tokens/sabout 165 tokens/s

Ollama Cloud runs models on Ollama’s own infrastructure through the same ollama client and API. Ollama doesn’t say which GPUs it uses. It states that its cloud does not retain your data. Its pricing page lists a pay-as-you-go plan with no subscription, and paid plans at $20 and $100 a month. The billing details have changed over time, so check the current terms before you rely on them.

The models

Qwen3.8 27BOrnith 1.5 35B-A3BGLM-5.3 Flash
Made byAlibaba (Qwen team)Ornith AIZ.ai
ReleasedAugust 2026 (open weights)2026August 26, 2026
ArchitectureDenseMixture of expertsMixture of experts (8 of 288 experts per token)
Total parameters27.8Babout 35B320B
Active per token27.8B (all of them)about 3B18B
Context (vendor)up to about 1M262K (about 1M with YaRN)1M
LicenseApache 2.0MITMIT
Vendor coding numbersSWE-bench Pro 61.7SWE-bench Pro 59.6, SWE-bench Verified 79Terminal-Bench 2.1 84.3, DeepSWE 63.4
Memory to runabout 16 to 19 GB at 4-bit21 GB download in Ollamaabout 100 GB at 1-bit, about 200 GB at 4-bit, about 306 GiB at FP8
How I ran itqwen3.8:27b-mlx on the Macornith-1.5:35b on the K8 Plus and the Mac (identical file)glm-5.3-flash on Ollama Cloud

The vendor numbers come from different benchmarks run in different ways, so don’t compare them across columns. They’re here to show what each model claims.

GLM-5.3 Flash is a serious model. Z.ai reports it close to frontier models on agentic coding, and its API price is $0.15 per million input tokens and $0.50 per million output tokens. Even its smallest 1-bit version needs about 100 GB of memory.

How I tested

I built a test suite instead of chatting with the models, because chatting misleads you. Early on, one model claimed its 350-word story was “(298 words)”. Another explained algebra beautifully and quietly set up a word problem that has no whole-number answer. Fluent output is easy to trust and hard to check.

The suite has two tiers.

  • Basic, 10 problems: small, everyday functions, each 10 to 20 lines.
  • Hard, 8 problems: longer files with several bugs that interact, where fixing one can break another.

Each problem gives the model a spec and a starter file with unmarked bugs and TODO blanks. The model is only told that bugs exist, not where. It returns the whole fixed file, and hidden pytest tests score it. The tests are split into “core” (normal behaviour) and “edge” (the traps), so a function that rejects everything can’t look half right.

Here’s what a starter looks like. This is the basic chunk problem, with my annotations added. The model sees it without the comments:

def chunk(items, size):
    """Split items into consecutive lists of length `size`."""
    # TODO: validate size
    ...

    result = []
    for start in range(0, len(items) - size, size):   # bug: drops the last piece
        result.append(items[start:start + size])      # bug: a tuple gives tuples, not lists
    return result

The problems

The traps aren’t trick questions. They’re the mistakes that make it into real code: off-by-one bounds, Python behaviour people misremember, and rules that only bite on unusual input.

Basic tier

ProblemWhat it doesThe trap
chunkSplit a list into fixed-size piecesThe last, shorter piece gets dropped. True and 2.0 must be rejected as sizes.
merge_intervalsMerge overlapping rangesUnsorted input, ranges that only touch ((1, 2) and (2, 3)), and a range inside another
LRUCacheA cache that evicts the least recently used keyBoth reading and updating a key count as a use. Stored values can be None.
slugifyTurn a title into a URL slugAccents and characters like ffi and Ⅻ need Unicode normalization
parse_csv_lineSplit one CSV line, without the csv moduleQuoted commas, "" escapes and a trailing empty field
bisect_leftBinary search for an insertion pointFinding the first of several duplicates, and off-by-one bounds
top_wordsCount the most common wordsTies sorted alphabetically, and apostrophes in words like don't
retryA decorator that retries a failing functionThe exact number of attempts, retrying only listed errors, and keeping the function’s name
flattenFlatten nested listsStrings must not be split into letters, and 10,000 levels of nesting breaks recursion
split_billSplit a bill into near-equal centsNo floating point: it must be exact for 10**20 cents

Hard tier

ProblemWhat it doesThe trap
TokenBucketA rate limiter that refills over timeIt must start full, never exceed capacity, and survive the clock going backwards
SemVer compareOrder version numbers by the SemVer 2.0 rules1.0.0-alpha ranks below 1.0.0. Numeric and text labels compare differently. "1.0.0\n" and fullwidth digits like 1.0.0 must be rejected.
wrapWord wrap that keeps paragraphs apartBlank lines that contain spaces, Windows line endings, and words longer than the line
Legacy refactorRewrite messy pricing code cleanlyIt must match the old code exactly, rounding quirks included, on 3,000 random orders
topo_sortPut dependencies in orderAlphabetical tie-breaks, reporting a real cycle, and a 5,000-node cycle that breaks recursion
glob_matchMatch file patterns like src/**.py* must stop at / but ** must not, plus character classes, escapes, and a pattern that makes naive backtracking run practically forever
Undo/redo EditorA text buffer with historyUndoing a delete needs the deleted text, plus a history limit and the redo order
JSON Pointer resolveLook up a value by path (RFC 6901)~01 decodes to ~1, not /. -1 and 01 aren’t valid list indexes, even though Python’s int() accepts both.

The Unicode cases show up on purpose. In Python, \d in a regex and str.isdigit() both accept digits like ١ (Arabic-Indic) and 1 (fullwidth), and int("10") quietly returns 10. A spec that says “ASCII digits only” needs code that actually enforces it.

I checked the suite itself before trusting it.

  1. Every reference solution passes and every starter fails.
  2. I put each of 71 planted bugs back into a correct solution, one at a time, and confirmed the tests catch it.
  3. After the runs, I compared every passing hard-tier solution against my reference solutions on more than 300,000 random inputs. Passing the tests and being correct are not the same thing.

All models ran with thinking on and one attempt per problem unless I say otherwise. The local models had a 32K-token context.

Results

Problems solved on the first try

Basic, first tryHard, first tryHard, after one repair round
Qwen3.8 27B (Mac)9/104/87/8
Ornith 1.5 35B-A3B (K8 Plus)10/107/87/8
Ornith 1.5 35B-A3B (Mac)10/106/8not run
GLM-5.3 Flash (Ollama Cloud)10/107/8not needed

Every hard-tier solution that passed on the first try, from every run, agreed with my reference on the random inputs. When these models passed, they were right.

The hard tier, problem by problem

ProblemQwen3.8 27B (Mac)Ornith (K8 Plus)Ornith (Mac)GLM-5.3 Flash
Token bucket✅ 67 s✅ 119 s✅ 53 s✅ 59 s
SemVer❌ 153 s✅ 762 s✅ 387 s✅ 127 s
Word wrap✅ 205 s✅ 452 s✅ 283 s✅ 69 s
Legacy refactor❌ 86 s✅ 490 s❌ 110 s✅ 34 s
Topological sort✅ 270 s✅ 634 s✅ 288 s✅ 165 s
Glob matching❌ 881 s⛔ ran out of tokens⛔ ran out of tokens⛔ no reply within an hour
Undo/redo editor✅ 101 s✅ 415 s✅ 259 s✅ 133 s
JSON Pointer❌ 67 s✅ 406 s✅ 165 s✅ 92 s

Total time to finish each tier

Basic: timeBasic: tokensHard (7, no glob): timeHard (7, no glob): tokens
Qwen3.8 27B (Mac)9.5 min17,66815.8 min30,428
Ornith 1.5 35B-A3B (K8 Plus)16.2 min25,51254.6 min83,594
Ornith 1.5 35B-A3B (Mac)7.6 min28,49325.8 min93,527
GLM-5.3 Flash2.2 min27,37011.3 min112,198

At Z.ai’s API list price, GLM’s output tokens for both tiers come to about 7 cents with glob left out, plus input charges.

Same model, two machines

To separate the hardware from the model, I ran Ornith on both machines. Ollama pulled the identical file on each (the file hashes match), and the suite was unchanged.

Ornith 1.5 35B-A3BK8 PlusMacBook Pro M4 Pro
Memory bandwidth89.6 GB/s273 GB/s
Generation speedabout 26 tokens/sabout 61 tokens/s
Basic tier10/10 in 16.2 min10/10 in 7.6 min
Hard tier (7, no glob)7/7 in 54.6 min6/7 in 25.8 min
Globran out of tokens after 22 minran out of tokens after 9 min

About 3 times the memory bandwidth gave about 2.3 times the speed, with the same weights.

The one score that changed wasn’t the hardware’s doing. On the Mac, Ornith kept a line from the starter that reads item["price"] before checking the quantity, the same bug Qwen made. On the K8 Plus it had rewritten that loop. Same weights, same prompt, different run. Thinking length varies too: Ornith spent 7,323 tokens on the CSV parser in one run and 11,778 in the other.

Glob shows the limit of faster hardware. The budget is measured in tokens, not minutes, so the faster machine just ran out sooner.

This also gives a fair head-to-head on one machine. On the Mac, Ornith finished the basic tier faster than Qwen (7.6 vs 9.5 minutes) with a better score (10/10 vs 9/10). On the hard tier, Qwen finished sooner (15.8 vs 25.8 minutes), but Ornith solved 6 problems to Qwen’s 4.

What I learned

1. Small active parameters went a long way

Ornith activates about 3B parameters per token. GLM-5.3 Flash activates 18B, out of 320B. On these 18 problems they were neck and neck: GLM scored 7/8 on the hard tier, and Ornith scored 7/8 and 6/8 in two runs. Both solved SemVer and JSON Pointer every time, two problems Qwen missed on its first try.

That fits what the model cards show. Ornith 1.5 reports SWE-bench Verified 79 and SWE-bench Pro 59.6, against 73.4 and 49.5 for Qwen3.6-35B-A3B, a model of the same size from about five months earlier. Those are vendor numbers, but the direction matches what I measured.

What my tests don’t measure is breadth of knowledge. A 35B model holds less of the world than a 320B one, and self-contained coding problems hide that. If your work needs that knowledge, the bigger model still matters.

2. The gap that’s left is speed, and that’s hardware

Generation speed by model and hardware

GLM didn’t win by thinking less. It generated the most tokens on the hard tier. It just generated them faster: about 6 times faster than Ornith on the mini PC, and about 2.7 times faster than the same model on my MacBook.

Generation speed on local hardware is mostly memory bandwidth. Every generated token reads every active weight from memory. That’s why dense Qwen3.8 27B runs at only about 6 tokens/s on the mini PC (about 90 GB/s). It’s also why a mixture-of-experts model fits the mini PC so well: Ornith reads only about 3B parameters per token, so it runs at about 26 tokens/s on the slower memory, and the 96 GB of RAM holds the full model easily. Move the same file to the M4 Pro and it runs at about 61.

3. Hardware setup mattered more than I expected

Out of the box, Ollama on the K8 Plus ran entirely on the CPU. The fix was two environment variables, because Ollama skips integrated GPUs unless you opt in:

[Service]
Environment="OLLAMA_VULKAN=1"
Environment="OLLAMA_IGPU_ENABLE=1"
Environment="OLLAMA_CONTEXT_LENGTH=32768"

With the Radeon 780M enabled through Vulkan, generation went from 19.6 to 26.7 tokens/s on Qwen3.6 35B-A3B (+36%), and from 17.2 to 24.3 on Gemma 4 26B (+41%). The third line matters just as much: Ollama loaded models with a 4,096-token context by default, which silently cuts off long agent prompts.

On the Mac, switching from the default Ollama build of Qwen3.8 27B to its MLX build took it from single digits to about 31 tokens/s.

4. Thinking helps, but don’t trust what it says

On an earlier parser test, Qwen3.8 27B scored 45/47 with thinking off and 47/47 with it on, three runs out of three. The thinking is where it noticed that Python’s \d and isdigit() accept non-ASCII digits like ١.

But a thinking trace isn’t proof. Qwen3.6 35B wrote a 7.5-minute trace that “traced” its code step by step and ended “All constraints met ✅”. Its code couldn’t parse "1h". It had traced the code it meant to write, not the code it wrote. Only running the tests catches that.

5. The ways they failed were consistent

  • Confident wrong knowledge. Qwen reasoned that assigning to an existing key in an OrderedDict moves it to the end. It doesn’t. More thinking can’t fix a fact the model believes. Ornith never relied on the fact at all and wrote an explicit move_to_end.
  • The same blind spot, repeatedly. Two of Qwen’s four hard-tier misses were the same trap: \d, isdigit() or $ quietly accepting Unicode digits or a trailing newline. It had caught that exact trap on my earlier parser test, then missed it here.
  • A repair fixes the symptom. Given only the failing test name, Qwen fixed SemVer, the refactor and JSON Pointer. On glob, it patched the exact case the test named and left the cause in place, which random-input testing caught.
  • No answer at all. Ornith spent all its tokens on glob four times, on both machines, and never wrote code. A wrong answer plus tests plus one retry is recoverable. A run that ends with no code isn’t.

6. I made mistakes too

The suite caught the models, and checking the suite caught me. Three specs required rejecting booleans without saying so. One test missed a Unicode case. The glob spec never said invalid patterns must be rejected before matching, and Ornith spent its whole budget debating exactly that question. My runner also once took a model’s example block instead of its solution, which turned a correct answer into “code did not run”. All of these are fixed. If you build your own evaluations, test the tests.

Caveats

  • One or two runs per model. Ornith scored 7/8 on the hard tier in one run and 6/8 in the next, with identical weights. Qwen failed JSON Pointer in one run and passed it in the next. Any score here could move by a problem or two.
  • Glob counts as unsolved for everyone. Its spec was ambiguous until the end, so it’s out of the time and token totals until I rerun all three models on the clarified version.
  • Mixed hardware. Qwen and Ornith meet head-to-head on the Mac, but the cloud comparison crosses hardware. Time compares setups, not models; tokens are the fairer measure of effort.
  • Narrow tests. Eighteen single-file Python problems. No large codebases, long contexts, tool use or general knowledge.
  • Settings I can’t see. GLM ran through Ollama Cloud with thinking on. I don’t know exactly which reasoning effort that maps to.

So, am I wasting my time?

No, but the answer is more specific than “local is good now.”

Local models really are catching up. A 21 GB model on a mini PC solved the same problems as a 320B model in a datacenter. On a laptop with faster memory, the same file finished the hard problems in 26 minutes against the cloud’s 11. That would have sounded unlikely a short while ago.

But you pay in time, and only the right hardware makes it bearable. On a mixture-of-experts model with the GPU working, the mini PC is usable for jobs nobody is waiting on. A dense model on the same machine is too slow for agent work.

Here’s how I’m setting things up now:

  • Interactive work and agent loops: a hosted model like GLM-5.3 Flash. It’s faster and costs cents.
  • Private code, offline work and overnight jobs: Ornith on the K8 Plus. It’s right about as often, and it’s the machine that stays on.
  • Local work at my desk: the same Ornith file on the MacBook, at more than twice the speed.
  • Everything: tests as the referee, a thinking budget per step, and a fallback to the cloud model when the local one fails its tests or runs out of budget.

Reproduce it

The suite runs any Ollama or OpenAI-compatible model:

pip3 install pytest
python3 run_suite.py --self-check --tier hard
python3 run_suite.py --model ornith-1.5:35b --think on --tier hard
python3 run_suite.py --model glm-5.3-flash --think on --tier hard --repair 1

Results go to results/summary.md and results/summary_hard.md, with every reply, thinking trace and code file saved.

Sources