Frank
986081bf1d
Changelog entries were essay-length incident writeups duplicating the concept pages they created. New entry law recorded at top of log.md: one dated entry = what changed + durable lesson + concept-page link; full narratives recoverable via git log --follow. Every date, headline, incident, and durable lesson preserved; narrative prose that already lives on concept pages (caddy handle-ordering, gluetun/limiter field notes, quadlet gotchas, faillock law, credential-store trap) replaced by pointers. Historical pi-era entries tagged '(pi era)'. Also: self-hosted-web-agents.md '44.5% peer-reviewed CUA' figure could not be re-attributed to any published benchmark via web search today — dropped with an explicit note; the valid core claim (87.4% = managed Cloud, not the self-hostable library) kept. Refs #29
114 lines
3.8 KiB
Markdown
114 lines
3.8 KiB
Markdown
---
|
|
type: concept
|
|
title: Self-Hosted Web Agents
|
|
description: Tools and architectures that let an AI agent drive a real browser, self-hosted.
|
|
tags:
|
|
- agentic
|
|
- browser
|
|
- scraping
|
|
- web-agent
|
|
- unverified
|
|
verified: 2026-08-28
|
|
---
|
|
|
|
# Self-Hosted Web Agents
|
|
|
|
A web agent is an AI that controls a browser. It sees the page, chooses an action, and repeats until it completes the task. You can run the whole loop on your own hardware.
|
|
|
|
This page covers the leading open tools and the architecture the agent uses.
|
|
|
|
## The Main Tools
|
|
|
|
### Browser Use
|
|
|
|
Browser Use is a Python library that uses Playwright under the hood. It is MIT-licensed. It supports many model providers, including local OpenAI-compatible endpoints such as llama.cpp.
|
|
|
|
Key ideas:
|
|
|
|
- It snapshots the DOM and converts it to Markdown for the model.
|
|
- Ten watchdogs handle CAPTCHAs, crashes, popups, downloads, and security.
|
|
- Real browser profiles carry saved logins.
|
|
|
|
**On the "87.4 percent" number:** that result refers to Browser Use's managed
|
|
Cloud agent (the Hybrid type that blends structured browser tools with
|
|
screenshots), not the open-source library you self-host. Academic
|
|
computer-use results on comparable web tasks run far lower, and an earlier
|
|
"44.5 percent peer-reviewed CUA" figure carried in this page could not be
|
|
re-attributed to a specific published benchmark (checked 2026-10-19) and was
|
|
dropped. Treat any single accuracy number as one benchmark, not a universal
|
|
score.
|
|
|
|
```python
|
|
# Local model via any OpenAI-compatible endpoint
|
|
from browser_use import Agent
|
|
from browser_use.llm.openai import ChatOpenAI
|
|
|
|
agent = Agent(
|
|
task="Search for the latest AI news",
|
|
llm=ChatOpenAI(
|
|
base_url="http://192.168.0.235:7770/v1",
|
|
model="frank-archon",
|
|
api_key="EMPTY",
|
|
),
|
|
)
|
|
await agent.run()
|
|
```
|
|
|
|
### Skyvern
|
|
|
|
Skyvern is an open-source browser agent (AGPL-3.0). It uses a vision model to see the page and Playwright to act. It extends Playwright with `page.act()`, `page.extract()`, and `page.validate()`. It ships a Docker Compose setup. Use it when you need a vision-first agent.
|
|
|
|
### Crawl4AI
|
|
|
|
Crawl4AI is an async crawler that returns LLM-friendly Markdown. It is built for retrieval pipelines, not for clicking through interactive pages. Use it to fetch and clean many pages quickly.
|
|
|
|
Key ideas:
|
|
|
|
- It produces clean Markdown and a noise-filtered "fit Markdown."
|
|
- It supports deep crawling with breadth-first, depth-first, and best-first strategies.
|
|
- It can run in Docker with a REST API and MCP endpoints.
|
|
|
|
```bash
|
|
pip install -U crawl4ai
|
|
crawl4ai-setup
|
|
crwl https://example.com --deep-crawl bfs --max-pages 10
|
|
```
|
|
|
|
## The Loop
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Task] --> B[Model chooses action]
|
|
B --> C[Browser or Crawler acts]
|
|
C --> D[Extract text / Markdown]
|
|
D --> B
|
|
D --> E[Task done?]
|
|
E -->|No| B
|
|
E -->|Yes| F[Return answer + citations]
|
|
```
|
|
|
|
## Which Tool to Use
|
|
|
|
| Situation | Tool |
|
|
|-----------|------|
|
|
| Need to click, log in, or fill forms | Browser Use |
|
|
| Need vision-based browsing | Skyvern |
|
|
| Need to fetch and clean many static pages | Crawl4AI |
|
|
| Research pipeline feeds an agent | Crawl4AI first, Browser Use for interactive steps |
|
|
|
|
A hybrid works best. Use Crawl4AI for mass extraction and Browser Use for the interactive steps.
|
|
|
|
## Citations
|
|
|
|
- Browser Use repository: <https://github.com/browser-use/browser-use>
|
|
- Browser Use documentation: <https://docs.browser-use.com>
|
|
- Skyvern repository: <https://github.com/Skyvern-AI/skyvern>
|
|
- Crawl4AI repository: <https://github.com/unclecode/crawl4ai>
|
|
- Crawl4AI documentation: <https://docs.crawl4ai.com>
|
|
|
|
## See Also
|
|
|
|
- [Agentic Search Best Practices](agentic-search-best-practices.md) — The loop around the tools.
|
|
- [Open Models for Web Use](open-models.md) — Models that drive these tools.
|
|
- [Research Effectiveness](research-effectiveness.md) — Orchestration around the loop.
|