Frank
02883d3ed2
Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
91 lines
3.6 KiB
Markdown
91 lines
3.6 KiB
Markdown
---
|
|
type: concept
|
|
title: Open Models for Web Use
|
|
description: Self-hostable, open-weight models that are strong at tool use, API calling, and internet-assisted tasks.
|
|
tags:
|
|
- open-models
|
|
- tool-use
|
|
- function-calling
|
|
- llm
|
|
- unverified
|
|
verified: 2026-08-28
|
|
timestamp: 2026-08-11
|
|
resource: https://qwenlm.github.io/blog/qwen3/
|
|
---
|
|
|
|
# Open Models for Web Use
|
|
|
|
The model is the brain behind the agent. For web research the model must call tools, follow instructions, and sometimes understand a page image. This page ranks the best self-hostable open-weight options.
|
|
|
|
> **Your setup:** Pi already serves local models through llama.cpp. See [Llama.cpp Server](../tools/llama-cpp/index.md) and [Pi Models Configuration](../pi/models.md). You can run any of the picks below on those hosts.
|
|
|
|
## The Front-Runner: Qwen3 (Alibaba, Apache 2.0)
|
|
|
|
Qwen3 is the current state of the art for open function calling and tool use. It has a thinking / non-thinking hybrid mode and strong tool-calling. Release date: April 2025.
|
|
|
|
| Model | Type | Active params | Context | Q4 VRAM |
|
|
|-------|------|---------------|---------|---------|
|
|
| Qwen3-8B | Dense | 8B | 128K | ~5 GB |
|
|
| Qwen3-14B | Dense | 14B | 128K | ~9 GB |
|
|
| Qwen3-32B | Dense | 32B | 128K | ~18 GB |
|
|
| Qwen3-30B-A3B | MoE | 3B | 128K | ~15 GB |
|
|
|
|
The 8B and 14B models run on one consumer GPU. The 30B-A3B MoE runs on a single A100-class card or equivalent.
|
|
|
|
## Best Vision Model: Qwen2.5-VL-7B
|
|
|
|
Many web tasks need a model that can read a page image, not just text. Qwen2.5-VL-7B is the best open vision model for this. It reads text, charts, icons, and layout in screenshots. It supports computer-use and phone-use agents. Apache 2.0 license. The 7B model fits on a single GPU with 8 GB or more.
|
|
|
|
Llama 4 Scout is the larger multimodal option. Verify the claims before you trust them:
|
|
|
|
- "10 million token context" — true, but **Scout only**, not Maverick.
|
|
- "Fits on a single H100" — only true with INT4 quantization (about 48 GB).
|
|
- "Beats GPT-4o" — Meta's own release benchmarks, not independent runs.
|
|
|
|
## Choose by Hardware
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Consumer GPU 16-24GB] --> B[Qwen3-8B / 14B]
|
|
A --> C[Qwen2.5-VL-7B]
|
|
D[Single server GPU 80GB] --> E[Qwen3-30B-A3B]
|
|
D --> F[Llama 4 Scout INT4]
|
|
G[Multi-GPU cluster] --> H[Qwen3-235B-A22B]
|
|
```
|
|
|
|
## Serving Options
|
|
|
|
- **llama.cpp** — CPU/GPU inference with GGUF quantization. Best for one consumer GPU. This is what Pi uses.
|
|
- **Ollama** — Easy local serving. Does not run in the Pi setup today.
|
|
- **vLLM** — High-throughput serving. Best for server and multi-GPU MoE models.
|
|
- **SGLang** — Strong for MoE serving.
|
|
|
|
```bash
|
|
# On the Pi llama.cpp host
|
|
# Serve a downloaded GGUF with the OpenAI-compatible API
|
|
llama-server -m qwen3-8b-q4_k_m.gguf -c 131072 --port 7770
|
|
```
|
|
|
|
## The Trade-Off
|
|
|
|
| Need | Pick |
|
|
|------|------|
|
|
| Best tool calling, one consumer GPU | Qwen3-8B or 14B |
|
|
| Visually read a web page | Qwen2.5-VL-7B |
|
|
| Server-scale reliable agent | Qwen3-235B-A22B |
|
|
| Cheapest acceptable agent | Qwen3-8B on a mid GPU |
|
|
|
|
## Citations
|
|
|
|
- Qwen3 technical report: <https://qwenlm.github.io/blog/qwen3/>
|
|
- Qwen3 repository: <https://github.com/QwenLM/Qwen3>
|
|
- Qwen2.5-VL-7B model card: <https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct>
|
|
- Meta AI Llama 4 announcement: <https://ai.meta.com/blog/llama-4-multimodal-intelligence/>
|
|
- Ollama library: <https://ollama.com/library>
|
|
- Browser Use supported models: <https://docs.browser-use.com/open-source/supported-models>
|
|
|
|
## See Also
|
|
|
|
- [Self-Hosted Web Agents](self-hosted-web-agents.md) — The tools these models drive.
|
|
- [Model Context Protocol (MCP)](mcp.md) — Connect tools to the model.
|