Figure outyour AI setupwith more clarity

A focused planner for people trying to understand what their hardware can realistically do, what kind of setup makes sense, and when local AI is actually worth the effort.

Start planning Learn more

Planner

Start with your situation.

Try a preset

1. What are you doing?

2. Platform

3. GPU memory (VRAM)

4. System RAM

5. GPU family

6. Main use case

7. What matters most

8. While the model runs, what else?

9. Typical context window

Recommendation

Your result

Comfortable

Best fit

Comfortable for midsize local models

Strong for daily local use, coding, and experimentation.

Best-fit setup

MID TIER

01Qwen3-14BGOOD · 1.4×Sticky 14B workhorse; 128K context; Apache 2.0; broad runner support.Needs ≈ 11.5 GB at 16K context
02Qwen 3.5 9BPERFECT · 2.3×262K context; strong on LiveCodeBench, IFEval, MMLU-Pro for its size.Needs ≈ 7.0 GB at 16K context
03gpt-oss-20bTOO BIG · 0.9×MXFP4-native Apache 2.0; fits 16GB cleanly; reasoning + tool use at this tier.Needs ≈ 18.5 GB at 16K context

Runner: Ollama or LM StudioLM Studio for UI, Ollama for CLI + API. Jan is a good privacy-first alternative.
Quantization: Stick to Q4_K_M for most picks.
Expected speed: 30–50 tok/s on 7–8B. Long prompts pull it lower.

Next steps · Windows

Install Ollama
winget install Ollama.Ollama
Pull the model
ollama pull qwen3:14b
Run it
ollama run qwen3:14b

Local vs cloud at this tier

Local works for chat, lightweight coding, and short-document Q&A. For hard reasoning, long context (>64K), or production image-gen at quality, cloud wins on quality per dollar. Think of local as the privacy-preserving fallback and the zero-marginal-cost workhorse for small tasks, not the daily driver for heavy output.

Workflow notes

Wire into your editor — Continue.dev, Cline, or Aider.
Keep a small fast model for autocomplete, a bigger one for review.
Cloud fallback for the heaviest reasoning tasks.
Take the faster path per task — local or cloud.

Watchouts

Very large models still push this setup — quantize aggressively

Feel the quality

How the answer changes as you move up a tier.

Same prompt, each tier's leading pick. These are written to show what each tier characteristically produces — the gap is real and grounded in how these models behave, but the exact wording is ours, not a captured run.

Prompt

Write a Python function that deduplicates a list while preserving insertion order.

Low tier·Qwen 3.5 4B

def dedupe(lst):
    return list(set(lst))
# order lost :(

Mid tier·Qwen3-14B

def dedupe(lst):
    seen = set()
    return [x for x in lst
            if not (x in seen
                    or seen.add(x))]

High tier·Qwen3-Coder-30B-A3B

from typing import Iterable, TypeVar

T = TypeVar("T")

def dedupe(items: Iterable[T]) -> list[T]:
    """Remove duplicates, preserve order. O(n)."""
    seen: set[T] = set()
    out: list[T] = []
    for x in items:
        if x not in seen:
            seen.add(x)
            out.append(x)
    return out

Top tier·Qwen3-Coder-30B-A3B — room for 128K context

# Same model as the tier below. What changed is what it can see:
# at 32 GB+ your codebase fits in context, so it matches house style
# instead of guessing — here, reusing the project's existing helper.
from typing import Iterable, TypeVar

from app.utils.iteration import first_seen  # already in your repo

T = TypeVar("T")

def dedupe(items: Iterable[T]) -> list[T]:
    """Remove duplicates, preserve order. O(n).

    Delegates to first_seen() to stay consistent with utils.iteration,
    which the rest of the codebase already uses for this.
    """
    return list(first_seen(items))

Illustrative, not measured — picks re-checked against the planner July 2026. For real output from a real model on your own hardware, run the in-browser demo in Labs.

Is it worth it?

Local vs cloud over 36 months.

Your monthly usage

~5M tokens / mo · Daily coding + chat

Compare against

Flat $20/mo; ~50–100 messages per 5h.

Hardware cost ($)

Estimated from your planner inputs.

Electricity ($/kWh)

US average ≈ $0.13/kWh. Adjust for your utility.

Cumulative cost, 36 months

Cloud
Local

Headline

Pays back in 76 months (beyond 3 years).

3-year total: $1,504 local vs $720 cloud. Cloud saves $784 over 36 months at this usage.

Pricing as of September 2026. Assumes ~250 W active inference at ~400 tok/s. Your real numbers will vary.

About

The AI Bench is a practical publication for local AI decisions. No hype, no newsletter, no directory bloat — just fewer, sharper tools that help you decide what to run, what to buy, and when local AI is actually worth it.

Runs locally via Ollama, LM Studio, or ComfyUI for image models.

Popular paths

Start with the decisions that matter.

Open-weight model picksSorted by fit, license, runner friction, and use case.Can your hardware run it?Start from Qwen, Gemma, FLUX, voice, or vision models.Measured tok/s tableReal runner rows across RTX, Mac, AMD, and DGX Spark.Ollama vs LM Studio vs JanRunner tradeoffs for Mac, Windows, and Linux.AMD ROCm reality checkWhere Radeon works, where Vulkan wins, and where it hurts.Mac Studio M3 UltraQuiet 96 GB Mac workstation with 70B Q4 headroom.

Latest snapshot · September 2026

Gemma 4, GLM-5.1, and the MoE moment.

Qwen3.8-27B is the current dense top pick for 24 GB. Gemma 4 31B took #3 on Arena. Z-Image-Turbo runs on 6 GB VRAM. The big story is mixture-of-experts: 30B-A3B MoE models now hit 3B-class inference speeds on a 4090 at 30B-class quality.

Read the changes feedQuarterly snapshots + fast takes on major drops. RSS. No newsletter.

Hardware we'd actually buy

Top tier · 32 GB · verified September 2026

NVIDIA RTX 5090

GB GDDR7: 32
GB/S BANDWIDTH: 1,792
W TDP: 575
FROM (STREET, SEP 22 2026): ~$6,610

Buy it if you want one card that runs every modern MoE at Q4 without thinking about quant or context.

Smart money · 48 GB · verified September 2026

Dual RTX 3090 (used)

GB GDDR6X: 2×24
GB/S BANDWIDTH: 936
W COMBINED TDP: 700
ALL-IN (USED): ~$1,800

Buy it if you value VRAM per dollar over every other metric, have a case that handles two 350 W cards, and don't mind spending an evening testing used GPUs on arrival.

Team red · 24 GB · verified September 2026

AMD Radeon RX 7900 XTX

GB GDDR6: 24
GB/S BANDWIDTH: 960
W TDP: 355
NEW (IN STOCK, SEP 15): ~$1,300

Buy it if you already run Linux, want 24 GB without paying NVIDIA's supply-crunch tax, and treat driver tinkering as acceptable friction rather than pain.

All-rounder · Mac · 64 GB unified · verified September 2026

M5 Max MacBook Pro 64 GB

GB UNIFIED: 64
GB/S BANDWIDTH: 614
W SUSTAINED: ~40
EST. CUSTOM BUILD: ~$5,199

Buy it if you want local AI that moves with you, runs silently, and works on battery — and if the rest of your stack is already Apple.

Budget · 16 GB · verified September 2026

RTX 5060 Ti 16 GB

GB GDDR7: 16
GB/S BANDWIDTH: 448
W TDP: 180
NEW (REFURB FROM $700): ~$780

Buy it if you want new hardware with a warranty, a quiet 180 W card that fits any modern PSU, and you're OK with MoE 30B-A3B as your ceiling for 2026.

No affiliate links. These are the current editorial picks, with every price and verification date sourced from the corresponding hardware detail page.