How to run AI on your own computer in 2026 — we walk you through it

What changed since spring

In March, a widely read note said you could run a capable model on a MacBook Air. That is still true. What changed by August is which models are honest on which machines — and how little you should have to become a systems person to get there.

Gemma 4, Qwen 3.6, and the smaller Llama 4 and Phi-4 family are the names worth knowing this month. The tools that matter for most people are still LM Studio on the desktop and mlx-lm if you will teach a Mac a voice. Ollama remains a good fit if you already think in a terminal. We will not pretend they are the same job.

Why run it on your computer

Privacy. The prompt never goes to a vendor. That is the only posture we will stand behind for student work, patient notes, client files, or unpublished research.

Cost after the first day. You pay once for the machine you already own. There is no meter on every letter.

Offline. After the first download, the network can come unplugged.

You can teach it. A small adapter on your letters, or a folder of files it can cite — on disk.

What local still does poorly: long, multi-step research against the live web, and any task that needs a model larger than your memory. We will say so. The point of accompaniment is not to sell you a mountain you cannot climb.

What your computer can honestly hold

Memory decides. Not the marketing name of the chip.

8 GB. Models around 1–3 billion parameters. Phi-4 Mini, Gemma 4 E4B, Llama 3.2 3B. Fine for drafts and explanations. Not for a 70B anything.

16 GB. Gemma 4 12B, or a 7–8B model at full quality. This is the honest start for most laptops.

32 GB. Gemma 4 26B, or a mixture-of-experts such as Qwen 3.6 35B-A3B at 4-bit. Local teaching (LoRA) becomes practical.

64 GB and up. Larger Qwen and Llama 4 checkpoints. Still not “the whole internet.” Still yours.

Apple Silicon (M1–M4) is the kindest path for a laptop: unified memory, MLX, LM Studio. Windows or Linux with NVIDIA is the kindest path for training at speed (Unsloth). CPU-only is for small models only. We will not put a 30-billion-parameter model on 16 GB.

The tools — and who they are for

LM Studio. A window. You search a model, download it, chat. It speaks GGUF and, on a Mac, MLX. This is where we start with almost everyone.

Ollama. A command and a local address (localhost:11434). Best if you already write scripts.

llama.cpp. The engine under a lot of the above. You do not need to meet it on day one.

mlx-lm (Mac) and Unsloth (NVIDIA). These teach the model a voice. They come after the house is closed — not before.

You do not have to choose in the abstract. In the consult we ask what the machine is, what the work is, and what must never leave the room. Then we pick one stack and stay with it.

Models we will actually install this month

Everyday writing, 16 GB: Gemma 4 12B, 4-bit. Clear, calm, small enough to leave room for a file search.

Everyday writing, 32 GB: Qwen 3.6 35B-A3B (mixture of experts) or Gemma 4 26B. One is enough.

Coding: a Qwen 3 Coder checkpoint that fits the memory. Do not send the repository to a hosted model if you would not email it.

Reasoning practice: a DeepSeek-R1 distill that fits. Useful. Not a substitute for your judgment.

The resident guide (the small model that stays on after we leave): Llama 3.2 1B or Phi-4 Mini. It starts training, checks health, and refuses a cloud link.

Sizing rule we use: about 1 GB of memory per 1 billion parameters at 4-bit, plus 3–4 GB so the machine can still breathe.

The walk — not a command dump

1. Ask. Who the work is for, how private it is, which computer, how much memory. Eight questions. You can do this alone in our consult, or with us on a call.

2. Plan. A written architecture you can read. Model, runtime, what stays offline. You approve before anything is installed.

3. Install. FileVault or BitLocker on. A folder that is yours (we use ~/Sherpa/camps/…). LM Studio. One model, hashed. Chat on this computer only — not on the open network.

4. Teach, if you want. A folder of files it can cite, or a small training run on examples you choose. Student work and charts do not go in that folder.

5. Sign off. A written record of what we did. Not a certificate. Your name on it.

If you want the commands for a Mac you already trust, they are: turn on FileVault; install LM Studio; download the model we named in the plan; set the server to 127.0.0.1; then, if you will train, install mlx-lm and run a LoRA against a JSONL file you created. We will not leave you at that sentence. The runbook is in the Camp. So is a person, if you paid for the walk.

Local or a hosted model

Use your computer when the material is confidential or regulated, when you need it with the network unplugged, or when you are tired of paying per word.

Use a hosted model when the material is already public, when you need the largest model in the world for one hard question, or when you are learning the craft on throwaway text.

If you are not sure, take the sealed door. You can always open a public tool later. You cannot walk a record back out of a vendor.

What we will not do

We will not hand you a list of flags and wish you luck.

We will not call an architecture a “HIPAA certification.”

We will not recommend a model your memory cannot hold.

We will not connect a keep that might hold records to ChatGPT, Drive, or Dropbox.

We walk with you. Then we leave. The measure is whether you can continue on the machine you already own.