Frank
02883d3ed2
Copy of the legacy ~/.pi/agent/docs tree rebuilt as the command-center knowledge base: concept files promoted to path identity, index listings stripped of frontmatter, all wikilinks/relative links resolved, every checkable claim verified against the live fleet (Dankserver = Ryzen 7 7700X + RTX 3090; Strix Halo is Dankclaw), unverifiable claims tagged. kb-lint exits 0.
107 lines
3.6 KiB
Markdown
107 lines
3.6 KiB
Markdown
---
|
|
type: concept
|
|
title: Self-Hosted Web Agents
|
|
description: Tools and architectures that let an AI agent drive a real browser, self-hosted.
|
|
tags:
|
|
- agentic
|
|
- browser
|
|
- scraping
|
|
- web-agent
|
|
- unverified
|
|
verified: 2026-08-28
|
|
---
|
|
|
|
# Self-Hosted Web Agents
|
|
|
|
A web agent is an AI that controls a browser. It sees the page, chooses an action, and repeats until it completes the task. You can run the whole loop on your own hardware.
|
|
|
|
This page covers the leading open tools and the architecture the agent uses.
|
|
|
|
## The Main Tools
|
|
|
|
### Browser Use
|
|
|
|
Browser Use is a Python library that uses Playwright under the hood. It is MIT-licensed. It supports many model providers, including local OpenAI-compatible endpoints such as llama.cpp.
|
|
|
|
Key ideas:
|
|
|
|
- It snapshots the DOM and converts it to Markdown for the model.
|
|
- Ten watchdogs handle CAPTCHAs, crashes, popups, downloads, and security.
|
|
- Real browser profiles carry saved logins.
|
|
|
|
**On the "87.4 percent" number:** that result refers to Browser Use's managed Cloud agent (the Hybrid type that blends structured browser tools with screenshots). Peer-reviewed results for vision-only (CUA) models are much lower, about 44.5 percent on the same task type. Treat any single accuracy number as one benchmark, not a universal score.
|
|
|
|
```python
|
|
# Local model via llama.cpp (OpenAI-compatible endpoint)
|
|
from browser_use import Agent
|
|
from browser_use.llm.openai import ChatOpenAI
|
|
|
|
agent = Agent(
|
|
task="Search for the latest AI news",
|
|
llm=ChatOpenAI(
|
|
base_url="http://192.168.0.235:7770/v1",
|
|
model="frank-default-small",
|
|
api_key="EMPTY",
|
|
),
|
|
)
|
|
await agent.run()
|
|
```
|
|
|
|
### Skyvern
|
|
|
|
Skyvern is an open-source browser agent (AGPL-3.0). It uses a vision model to see the page and Playwright to act. It extends Playwright with `page.act()`, `page.extract()`, and `page.validate()`. It ships a Docker Compose setup. Use it when you need a vision-first agent.
|
|
|
|
### Crawl4AI
|
|
|
|
Crawl4AI is an async crawler that returns LLM-friendly Markdown. It is built for retrieval pipelines, not for clicking through interactive pages. Use it to fetch and clean many pages quickly.
|
|
|
|
Key ideas:
|
|
|
|
- It produces clean Markdown and a noise-filtered "fit Markdown."
|
|
- It supports deep crawling with breadth-first, depth-first, and best-first strategies.
|
|
- It can run in Docker with a REST API and MCP endpoints.
|
|
|
|
```bash
|
|
pip install -U crawl4ai
|
|
crawl4ai-setup
|
|
crwl https://example.com --deep-crawl bfs --max-pages 10
|
|
```
|
|
|
|
## The Loop
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[Task] --> B[Model chooses action]
|
|
B --> C[Browser or Crawler acts]
|
|
C --> D[Extract text / Markdown]
|
|
D --> B
|
|
D --> E[Task done?]
|
|
E -->|No| B
|
|
E -->|Yes| F[Return answer + citations]
|
|
```
|
|
|
|
## Which Tool to Use
|
|
|
|
| Situation | Tool |
|
|
|-----------|------|
|
|
| Need to click, log in, or fill forms | Browser Use |
|
|
| Need vision-based browsing | Skyvern |
|
|
| Need to fetch and clean many static pages | Crawl4AI |
|
|
| Research pipeline feeds an agent | Crawl4AI first, Browser Use for interactive steps |
|
|
|
|
A hybrid works best. Use Crawl4AI for mass extraction and Browser Use for the interactive steps.
|
|
|
|
## Citations
|
|
|
|
- Browser Use repository: <https://github.com/browser-use/browser-use>
|
|
- Browser Use documentation: <https://docs.browser-use.com>
|
|
- Skyvern repository: <https://github.com/Skyvern-AI/skyvern>
|
|
- Crawl4AI repository: <https://github.com/unclecode/crawl4ai>
|
|
- Crawl4AI documentation: <https://docs.crawl4ai.com>
|
|
|
|
## See Also
|
|
|
|
- [Agentic Search Best Practices](agentic-search-best-practices.md) — The loop around the tools.
|
|
- [Open Models for Web Use](open-models.md) — Models that drive these tools.
|
|
- [Research Effectiveness](research-effectiveness.md) — Orchestration around the loop.
|