Frank
a9b3c495da
- open-models.md: Qwen2.5-VL 'best open vision model' superseded by Qwen3-VL (Apache 2.0 dense sizes; repo QwenLM/Qwen3-VL, arXiv 2511.21631, verified via web search); decision table + mermaid + citations updated - chezmoi-evaluation.md: fleet facts were fabricated — servers run Ubuntu 26.04 LTS (probed via ssh, both boxes), not 'Arch/FreeToken' (distro does not exist); dev workstation is AMD Ryzen 5 PRO 5650U, not Intel Ultra 7; frank-default-* endpoints retired 2026-10-18 - agentic-web-research/index.md: pi-era topology blurb replaced with current fleet endpoints - self-hosted-web-agents.md: sample model frank-default-small -> frank-archon (endpoint/IP already correct: dankserver) - mcp.md reference-server table verified against upstream modelcontextprotocol/servers README (exact match, no change) - 'peer-reviewed BrowseComp' claim from scout ledger: does not exist in any file — rejected, not applied Refs #29
107 lines
3.6 KiB
Markdown
107 lines
3.6 KiB
Markdown
---
|
|
type: concept
|
|
title: Self-Hosted Web Agents
|
|
description: Tools and architectures that let an AI agent drive a real browser, self-hosted.
|
|
tags:
|
|
- agentic
|
|
- browser
|
|
- scraping
|
|
- web-agent
|
|
- unverified
|
|
verified: 2026-08-28
|
|
---
|
|
|
|
# Self-Hosted Web Agents
|
|
|
|
A web agent is an AI that controls a browser. It sees the page, chooses an action, and repeats until it completes the task. You can run the whole loop on your own hardware.
|
|
|
|
This page covers the leading open tools and the architecture the agent uses.
|
|
|
|
## The Main Tools
|
|
|
|
### Browser Use
|
|
|
|
Browser Use is a Python library that uses Playwright under the hood. It is MIT-licensed. It supports many model providers, including local OpenAI-compatible endpoints such as llama.cpp.
|
|
|
|
Key ideas:
|
|
|
|
- It snapshots the DOM and converts it to Markdown for the model.
|
|
- Ten watchdogs handle CAPTCHAs, crashes, popups, downloads, and security.
|
|
- Real browser profiles carry saved logins.
|
|
|
|
**On the "87.4 percent" number:** that result refers to Browser Use's managed Cloud agent (the Hybrid type that blends structured browser tools with screenshots). Peer-reviewed results for vision-only (CUA) models are much lower, about 44.5 percent on the same task type. Treat any single accuracy number as one benchmark, not a universal score.
|
|
|
|
```python
|
|
# Local model via llama.cpp (OpenAI-compatible endpoint)
|
|
from browser_use import Agent
|
|
from browser_use.llm.openai import ChatOpenAI
|
|
|
|
agent = Agent(
|
|
task="Search for the latest AI news",
|
|
llm=ChatOpenAI(
|
|
base_url="http://192.168.0.235:7770/v1",
|
|
model="frank-archon",
|
|
api_key="EMPTY",
|
|
),
|
|
)
|
|
await agent.run()
|
|
```
|
|
|
|
### Skyvern
|
|
|
|
Skyvern is an open-source browser agent (AGPL-3.0). It uses a vision model to see the page and Playwright to act. It extends Playwright with `page.act()`, `page.extract()`, and `page.validate()`. It ships a Docker Compose setup. Use it when you need a vision-first agent.
|
|
|
|
### Crawl4AI
|
|
|
|
Crawl4AI is an async crawler that returns LLM-friendly Markdown. It is built for retrieval pipelines, not for clicking through interactive pages. Use it to fetch and clean many pages quickly.
|
|
|
|
Key ideas:
|
|
|
|
- It produces clean Markdown and a noise-filtered "fit Markdown."
|
|
- It supports deep crawling with breadth-first, depth-first, and best-first strategies.
|
|
- It can run in Docker with a REST API and MCP endpoints.
|
|
|
|
```bash
|
|
pip install -U crawl4ai
|
|
crawl4ai-setup
|
|
crwl https://example.com --deep-crawl bfs --max-pages 10
|
|
```
|
|
|
|
## The Loop
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Task] --> B[Model chooses action]
|
|
B --> C[Browser or Crawler acts]
|
|
C --> D[Extract text / Markdown]
|
|
D --> B
|
|
D --> E[Task done?]
|
|
E -->|No| B
|
|
E -->|Yes| F[Return answer + citations]
|
|
```
|
|
|
|
## Which Tool to Use
|
|
|
|
| Situation | Tool |
|
|
|-----------|------|
|
|
| Need to click, log in, or fill forms | Browser Use |
|
|
| Need vision-based browsing | Skyvern |
|
|
| Need to fetch and clean many static pages | Crawl4AI |
|
|
| Research pipeline feeds an agent | Crawl4AI first, Browser Use for interactive steps |
|
|
|
|
A hybrid works best. Use Crawl4AI for mass extraction and Browser Use for the interactive steps.
|
|
|
|
## Citations
|
|
|
|
- Browser Use repository: <https://github.com/browser-use/browser-use>
|
|
- Browser Use documentation: <https://docs.browser-use.com>
|
|
- Skyvern repository: <https://github.com/Skyvern-AI/skyvern>
|
|
- Crawl4AI repository: <https://github.com/unclecode/crawl4ai>
|
|
- Crawl4AI documentation: <https://docs.crawl4ai.com>
|
|
|
|
## See Also
|
|
|
|
- [Agentic Search Best Practices](agentic-search-best-practices.md) — The loop around the tools.
|
|
- [Open Models for Web Use](open-models.md) — Models that drive these tools.
|
|
- [Research Effectiveness](research-effectiveness.md) — Orchestration around the loop.
|