Frank
a9b3c495da
- open-models.md: Qwen2.5-VL 'best open vision model' superseded by Qwen3-VL (Apache 2.0 dense sizes; repo QwenLM/Qwen3-VL, arXiv 2511.21631, verified via web search); decision table + mermaid + citations updated - chezmoi-evaluation.md: fleet facts were fabricated — servers run Ubuntu 26.04 LTS (probed via ssh, both boxes), not 'Arch/FreeToken' (distro does not exist); dev workstation is AMD Ryzen 5 PRO 5650U, not Intel Ultra 7; frank-default-* endpoints retired 2026-10-18 - agentic-web-research/index.md: pi-era topology blurb replaced with current fleet endpoints - self-hosted-web-agents.md: sample model frank-default-small -> frank-archon (endpoint/IP already correct: dankserver) - mcp.md reference-server table verified against upstream modelcontextprotocol/servers README (exact match, no change) - 'peer-reviewed BrowseComp' claim from scout ledger: does not exist in any file — rejected, not applied Refs #29
98 lines
3.9 KiB
Markdown
98 lines
3.9 KiB
Markdown
---
|
|
type: concept
|
|
title: Open Models for Web Use
|
|
description: Self-hostable, open-weight models that are strong at tool use, API calling, and internet-assisted tasks.
|
|
tags:
|
|
- open-models
|
|
- tool-use
|
|
- function-calling
|
|
- llm
|
|
- unverified
|
|
verified: 2026-08-28
|
|
timestamp: 2026-08-11
|
|
resource: https://qwenlm.github.io/blog/qwen3/
|
|
---
|
|
|
|
# Open Models for Web Use
|
|
|
|
The model is the brain behind the agent. For web research the model must call tools, follow instructions, and sometimes understand a page image. This page ranks the best self-hostable open-weight options.
|
|
|
|
> **Your setup:** Pi already serves local models through llama.cpp. See [Llama.cpp Server](../tools/llama-cpp/index.md) and [Pi Models Configuration](../pi/models.md). You can run any of the picks below on those hosts.
|
|
|
|
## The Front-Runner: Qwen3 (Alibaba, Apache 2.0)
|
|
|
|
Qwen3 is the current state of the art for open function calling and tool use. It has a thinking / non-thinking hybrid mode and strong tool-calling. Release date: April 2025.
|
|
|
|
| Model | Type | Active params | Context | Q4 VRAM |
|
|
|-------|------|---------------|---------|---------|
|
|
| Qwen3-8B | Dense | 8B | 128K | ~5 GB |
|
|
| Qwen3-14B | Dense | 14B | 128K | ~9 GB |
|
|
| Qwen3-32B | Dense | 32B | 128K | ~18 GB |
|
|
| Qwen3-30B-A3B | MoE | 3B | 128K | ~15 GB |
|
|
|
|
The 8B and 14B models run on one consumer GPU. The 30B-A3B MoE runs on a single A100-class card or equivalent.
|
|
|
|
## Best Vision Model: Qwen3-VL (Apache 2.0)
|
|
|
|
Many web tasks need a model that can read a page image, not just text.
|
|
**Qwen3-VL** supersedes Qwen2.5-VL as the strongest open-weights vision family
|
|
(4B/8B dense through 30B-A3B/235B-A22B MoE): text, chart, icon, and layout
|
|
reading in screenshots, plus computer-use agent support. Small dense sizes
|
|
ship Apache 2.0; a quantized 4B/8B runs on a single consumer GPU. On this
|
|
fleet, vision duty is carried by the mmproj vision towers on the endpoints
|
|
(dragoon, zealot) — see [System hosts](/system/index.md).
|
|
|
|
Llama 4 Scout is the larger multimodal option. Verify the claims before you trust them:
|
|
|
|
- "10 million token context" — true, but **Scout only**, not Maverick.
|
|
- "Fits on a single H100" — only true with INT4 quantization (about 48 GB).
|
|
- "Beats GPT-4o" — Meta's own release benchmarks, not independent runs.
|
|
|
|
## Choose by Hardware
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Consumer GPU 16-24GB] --> B[Qwen3-8B / 14B]
|
|
A --> C[Qwen3-VL-4B / 8B]
|
|
D[Single server GPU 80GB] --> E[Qwen3-30B-A3B]
|
|
D --> F[Llama 4 Scout INT4]
|
|
G[Multi-GPU cluster] --> H[Qwen3-235B-A22B]
|
|
```
|
|
|
|
## Serving Options
|
|
|
|
- **llama.cpp** — CPU/GPU inference with GGUF quantization. Best for one consumer GPU. This is what Pi uses.
|
|
- **Ollama** — Easy local serving. Does not run in the Pi setup today.
|
|
- **vLLM** — High-throughput serving. Best for server and multi-GPU MoE models.
|
|
- **SGLang** — Strong for MoE serving.
|
|
|
|
```bash
|
|
# On the Pi llama.cpp host
|
|
# Serve a downloaded GGUF with the OpenAI-compatible API
|
|
llama-server -m qwen3-8b-q4_k_m.gguf -c 131072 --port 7770
|
|
```
|
|
|
|
## The Trade-Off
|
|
|
|
| Need | Pick |
|
|
|------|------|
|
|
| Best tool calling, one consumer GPU | Qwen3-8B or 14B |
|
|
| Visually read a web page | Qwen3-VL-4B / 8B |
|
|
| Server-scale reliable agent | Qwen3-235B-A22B |
|
|
| Cheapest acceptable agent | Qwen3-8B on a mid GPU |
|
|
|
|
## Citations
|
|
|
|
- Qwen3 technical report: <https://qwenlm.github.io/blog/qwen3/>
|
|
- Qwen3 repository: <https://github.com/QwenLM/Qwen3>
|
|
- Qwen3-VL repository: <https://github.com/QwenLM/Qwen3-VL> (tech report: <https://arxiv.org/abs/2511.21631>)
|
|
- Qwen2.5-VL-7B model card (predecessor): <https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct>
|
|
- Meta AI Llama 4 announcement: <https://ai.meta.com/blog/llama-4-multimodal-intelligence/>
|
|
- Ollama library: <https://ollama.com/library>
|
|
- Browser Use supported models: <https://docs.browser-use.com/open-source/supported-models>
|
|
|
|
## See Also
|
|
|
|
- [Self-Hosted Web Agents](self-hosted-web-agents.md) — The tools these models drive.
|
|
- [Model Context Protocol (MCP)](mcp.md) — Connect tools to the model.
|