Local models that actually run on 8GB

Two are usable, one surprised me.

8GB is what most people actually have. 8GB unified memory on an M1 MacBook Air. 8GB VRAM on an RTX 4060. Not enough for the models everyone talks about. Just barely enough for something useful if you pick the right one. The problem is that "fits in 8GB" is misleading. The model weights need to fit, plus the KV cache for your conversation context, plus whatever your OS and other apps are using. On 8GB, you realistically have 4-5GB available for the model. That rules out anything above 7-8B parameters at Q4 quantization. I tested everything that actually fits on this hardware for agent work specifically. Not chat demos. Not single-turn benchmarks. Real agent tasks: tool calling through MCP servers, multi-turn conversations, email classification, and morning briefing generation through Ollama. Most models at this size are frustrating. Two are genuinely usable. One surprised me. Usable #1**: Phi-4 Mini (3.8B). The safe pick.** bash

ollama pull phi4-mini

3.8 billion parameters. About 2.5GB at Q4_K_M. Fits on 8GB with enormous headroom. Your OS, your browser, and a few other apps all run comfortably alongside it. No memory pressure. No swapping. Speed: 15-28 tokens per second depending on hardware. On an M1 MacBook Air, roughly 15-20 tok/s. On an RTX 4060 (8GB), closer to 28 tok/s. Fast enough for interactive conversation. Not instant, but not painfully slow. Microsoft trained this model with a focus on reasoning and math. It scores 80.4% on MATH, which beats Llama 3.3 8B (68.0%) and even Qwen 2.5 14B (75.6%) despite being a fraction of the size. For analytical tasks (explaining concepts, solving problems step-by-step, debugging logic), Phi-4 Mini punches way above its weight class. For agent work: basic tool calling works. Single-tool calls (search the web, read a file, query a database) succeed reliably. Classification tasks run fine. Morning briefing generation is decent if not spectacular. The limitation: 16K context window. This is tight for agent work. Three MCP servers with their tool schemas eat 3,000-5,000 tokens of context. Your conversation history eats another 3,000-5,000. By turn 10, you're pushing the ceiling. Sessions need to be short. /new after every 5-7 exchanges. For short, focused agent interactions on truly constrained hardware, Phi-4 Mini works. Just don't ask it to hold a long conversation or chain 4 tools together. Usable #2**: Gemma 4 E4B (4.5B). The multimodal pick.** bash

ollama pull gemma4

4.5 billion parameters. About 6GB for the model. Tight on 8GB but it fits if you're not running heavy background apps alongside it. What makes Gemma 4 E4B worth the squeeze: 128K context window and multimodal support. Send your agent a photo of a receipt and ask "what was the total?" Send a screenshot of an error message and ask "what's wrong?" No other model at this size does images, video, and audio. That 128K context window is 8x larger than Phi-4 Mini's 16K. For agent work, this is the difference between sessions that last 5 exchanges and sessions that last 25+. Tool schemas fit comfortably. Conversation history doesn't get truncated. You can actually have a proper back-and-forth without hitting the ceiling. Speed: 40-60 tok/s. The fastest model in this comparison. Replies feel close to instant on short answers. Native function calling is built into the training. Google designed Gemma 4 with tool use in mind. Single-tool calls are reliable. The model formats JSON arguments correctly and doesn't hallucinate tool names. The limitation: reasoning depth. At 4.5B parameters, complex analytical tasks are noticeably weaker than Phi-4 Mini despite Phi-4 being smaller. Gemma 4 E4B is broader (multimodal, long context) but shallower on any single task. It handles "classify this email" perfectly. It struggles with "analyze these three emails and tell me which client is most likely to churn and why." Apache 2.0 license. Commercially friendly. The surprise: Qwen3.5 9B. This shouldn't work this well at 8GB. bash

ollama pull qwen3.5:9b

9 billion parameters. At Q4_K_M, the model weights are about 5GB. On an 8GB GPU, it fits entirely in VRAM with room for a reasonable KV cache. No spilling to system RAM. No layer splitting across CPU and GPU. I almost skipped this model because 9B on 8GB seemed like it would be too tight. I was wrong. LocalLLM.in benchmarked it at 55-58 tokens per second on an RTX 3070 (8GB). Fully GPU-loaded. Flat speed across all context sizes up to 16K. That's faster than Phi-4 Mini on the same hardware despite being over twice the parameter count. The quality jump from 3.8-4.5B to 9B is dramatic. On multi-turn agent conversations, Qwen3.5 9B maintains coherence across 15+ exchanges where the smaller models start drifting by turn 7. Tool calling is significantly more reliable. Chained tool calls (file search, then web search, then write results) succeeded at roughly 75% on Qwen3.5 9B vs maybe 50% on the smaller models. It has a /think reasoning mode that activates deeper chain-of-thought when needed. On complex queries it pauses, reasons through the problem, then responds. On simple queries it skips the reasoning and responds immediately. The automatic mode switching is surprisingly good at knowing when to think harder. 32K+ context window. Enough for multiple MCP servers, conversation history, and extended sessions without truncation. The catch: it's tight. On 8GB VRAM, the model fits but there's minimal headroom. If your context grows past 16K tokens, KV cache pressure starts pushing against the limit. On some hardware configurations, Ollama automatically spills overflow to system RAM, which tanks speed. On others, it crashes. The practical rule: keep num_ctx at 8192-12288 on 8GB hardware. Don't try to use the full 32K context. Set it in a custom modelfile: bash

cat > qwen-agent.modelfile << 'EOF'
FROM qwen3.5:9b
PARAMETER num_ctx 8192
PARAMETER temperature 0.3
EOF
ollama create qwen-agent -f qwen-agent.modelfile

Temperature at 0.3 for agent work. Lower = more reliable tool calls. Higher = more creative but flakier function call formatting. On 8GB system RAM (not VRAM) without a dedicated GPU: don't try 9B. CPU-only inference on a 9B model gives you 3-6 tok/s. Painfully slow. Stick with Phi-4 Mini or Gemma 4 E4B on CPU-only 8GB machines. What I tested and cut: Llama 3.2 3B. 3 billion parameters. Fits easily. 10-12 tok/s on CPU. But the quality is noticeably below Phi-4 Mini on every task despite being nearly the same size. Classification accuracy was lower. Tool calling was flakier. Output felt generic. The Llama name carries weight but at 3B, Phi-4 Mini is simply better. DeepSeek R1 8B (distilled). Fits on 8GB VRAM at Q4. Reasoning is impressive with visible chain-of-thought. But tool calling is unreliable. The model wants to reason ABOUT tools instead of calling them. Asks itself "should I use the filesystem tool here?" and then writes a paragraph about why, instead of actually making the call. Great for analysis and math. Bad for agents. Mistral 7B Instruct v0.3. Fits well. Fast (20% faster than Llama 3.1 8B). Good for business communication and summaries. But function calling was inconsistent. The training didn't emphasize structured tool use the way Qwen and Gemma did. Fine for chat. Not for agent work. Qwen 2.5 7B. Previous generation. Fits on 8GB. Good tool calling (the Qwen family reputation). But Qwen3.5 9B is better on every metric AND fits on the same hardware. No reason to run the older version unless you specifically need its coding specialization.