Frank
87721ab0fe
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
219 lines
15 KiB
Markdown
219 lines
15 KiB
Markdown
---
|
||
type: research
|
||
description: "The 15 most popular self-hosted AI applications of June–September 2026, ranked by GitHub stars, with homelab fit and adoption best practices."
|
||
tags:
|
||
- research
|
||
- self-hosted
|
||
- ai-apps
|
||
- landscape
|
||
timestamp: 2026-09-22
|
||
---
|
||
|
||
# Most Popular Self-Hosted AI Apps (June–September 2026)
|
||
|
||
Research brief on the most popular and trending self-hostable (Docker/Podman/
|
||
Quadlet) open-source AI applications over the last 3 months. Star counts and
|
||
licenses pulled directly from the GitHub API and each repo's LICENSE file on
|
||
**2026-09-22** (exact values). "Trending" is evidenced by **release activity
|
||
dated June–September 2026** (GitHub releases API) plus press/community
|
||
coverage — GitHub does not expose star-growth rate, so cumulative stars are the
|
||
popularity proxy and release cadence is the trend signal. The full rendered
|
||
report lives at `popular-self-hosted-ai-apps.html` beside this page.
|
||
|
||
## Summary
|
||
|
||
The self-hosted AI stack in mid-2026 consolidated into a few very large, very
|
||
active projects, and the center of gravity shifted hard toward **agents** over
|
||
the last quarter:
|
||
|
||
1. **Agents are the category of 2026.** OpenClaw (390,235 stars, MIT) is the
|
||
single most-starred self-hosted AI project and the fastest-growing repo on
|
||
GitHub; it shipped a major **2.0 rewrite on 1 September 2026** (933
|
||
contributors, 16,000+ PRs). Nous Research's **Hermes Agent** (247,907 stars,
|
||
MIT) is a close second and releases weekly. Both are model-agnostic
|
||
orchestration layers that run on your own hardware and can point at local
|
||
llama.cpp/vLLM/Ollama endpoints.
|
||
2. **The inference + UI layer is mature and huge.** Ollama (181,444), llama.cpp
|
||
(129,149), and vLLM (92,405) are the runtimes; **Open WebUI** (152,784) is
|
||
the dominant chat UI; **ComfyUI** (134,407, GPL-3.0) is the dominant
|
||
image/video-gen GUI. All release frequently in the window.
|
||
3. **RAG/agent-platforms and the audio stack round out the field.** Dify
|
||
(156,835), RAGFlow (91,138), AnythingLLM (66,318) for RAG/workflows;
|
||
whisper.cpp (53,852) + faster-whisper (25,512) for STT; Chatterbox (26,515),
|
||
Piper (11,287), Kokoro-82M (8,933) for TTS; Unsloth (76,560) for
|
||
fine-tuning.
|
||
|
||
**Strongest fits for this fleet:** Open WebUI (chat UI over the existing
|
||
llama.cpp/vLLM endpoints), ComfyUI (image gen, already integrated with Open
|
||
WebUI), Ollama (OpenAI-compatible runtime), Dify and RAGFlow (RAG/agent
|
||
platforms), and whisper.cpp / Piper / Kokoro for a local voice stack. OpenClaw
|
||
and Hermes are the highest-buzz new arrivals, well-suited to a homelab server,
|
||
with the caveat that they are tool/shell-access agents and need the strongest
|
||
isolation.
|
||
|
||
## Ranked List (Top 15, all with June–Sept 2026 release activity)
|
||
|
||
| # | App | Category | Stars (2026-09-22) | License | Latest release in window |
|
||
|---|-----|----------|--------------------|---------|--------------------------|
|
||
| 1 | [OpenClaw](https://github.com/openclaw/openclaw) | Agents | 390,235 | MIT | v2026.9.5 (09-19); 2.0 (09-01) |
|
||
| 2 | [Hermes Agent](https://github.com/NousResearch/hermes-agent) | Agents | 247,907 | MIT | v2026.9.21 (09-21) |
|
||
| 3 | [Ollama](https://github.com/ollama/ollama) | LLM runtime | 181,444 | MIT | active (pushed 09-21) |
|
||
| 4 | [Dify](https://github.com/langgenius/dify) | Agent platform / RAG | 156,835 | Modified Apache-2.0 | 1.17.1 (09-10) |
|
||
| 5 | [Open WebUI](https://github.com/open-webui/open-webui) | Chat UI | 152,784 | Open WebUI License (BSD-3-based) | v0.11.4 (09-21) |
|
||
| 6 | [ComfyUI](https://github.com/Comfy-Org/ComfyUI) | Image / video gen | 134,407 | GPL-3.0 | v0.37.0 (09-21) |
|
||
| 7 | [llama.cpp](https://github.com/ggml-org/llama.cpp) | LLM runtime (inference) | 129,149 | MIT | active (pushed 09-22) |
|
||
| 8 | [vLLM](https://github.com/vllm-project/vllm) | LLM inference engine | 92,405 | Apache-2.0 | active (pushed 09-22) |
|
||
| 9 | [RAGFlow](https://github.com/infiniflow/ragflow) | RAG engine | 91,138 | Apache-2.0 | v0.27.2 (09-10) |
|
||
| 10 | [LobeChat](https://github.com/lobehub/lobehub) | Chat UI / agent ops | 82,748 | LobeHub Community License | active (pushed 09-22) |
|
||
| 11 | [Unsloth](https://github.com/unslothai/unsloth) | Fine-tuning | 76,560 | Apache-2.0 | active (pushed 09-22) |
|
||
| 12 | [AnythingLLM](https://github.com/Mintplex-Labs/anything-llm) | RAG / local agent | 66,318 | MIT | active (pushed 09-21) |
|
||
| 13 | [whisper.cpp](https://github.com/ggml-org/whisper.cpp) | Speech-to-text | 53,852 | MIT | active (pushed 09-22) |
|
||
| 14 | [LibreChat](https://github.com/danny-avila/LibreChat) | Chat UI | 44,608 | MIT | active (pushed 09-22) |
|
||
| 15 | [Tabby](https://github.com/TabbyML/tabby) | AI coding assistant | 33,889 | Apache-2.0 (ee/ separate) | last push 2026-06-30 |
|
||
|
||
**Audio honorable mentions:** [Chatterbox](https://github.com/resemble-ai/chatterbox)
|
||
(26,515, MIT, TTS), [faster-whisper](https://github.com/SYSTRAN/faster-whisper)
|
||
(25,512, MIT, STT), [Piper TTS](https://github.com/rhasspy/piper) (11,287, MIT,
|
||
TTS), [Kokoro-82M](https://github.com/hexgrad/kokoro) (8,933, Apache-2.0, TTS),
|
||
[MisoTTS](https://github.com/MisoLabsAI/MisoTTS) (3,239, Modified MIT, TTS).
|
||
|
||
## Per-App Notes
|
||
|
||
- **OpenClaw** — self-hosted personal AI agent / orchestration layer (not a
|
||
model). Connects to cloud APIs, subscriptions, API providers, and **local
|
||
models**; drives files, browser, terminal/shell, and messaging channels. 2.0
|
||
rebuilt the browser app as the primary interface, added shared cloud sessions,
|
||
simplified onboarding. Node 24 (22.14+) + Docker; minimal footprint for
|
||
API-based LLMs (2 vCPU / 2–4 GB RAM). **Caveat:** community reaction to 2.0
|
||
was mixed — migration problems, broken gateways, lost automations, model-auth
|
||
issues (InfoQ). Tool/shell-access agent → strongest isolation.
|
||
- **Hermes Agent** — Nous Research's personal agent, centered on persistent
|
||
memory and cross-platform messaging. Model-agnostic; Docker/Podman; config in
|
||
`~/.hermes/config.yaml`. Releases weekly. Tool-access agent → isolate.
|
||
- **Ollama** — the de-facto standard local LLM runtime; OpenAI-compatible API
|
||
(default `:11434`) that nearly every app in this list consumes. The default
|
||
provider for JobSync, Resume Matcher, AnythingLLM, and more.
|
||
- **Dify** — full LLMOps/agent platform: prompt IDE, RAG pipeline,
|
||
agent/workflow builder, model management. `docker compose` full stack.
|
||
Modified Apache-2.0: commercial use allowed, but **multi-tenant operation
|
||
requires a commercial license** (one tenant = one workspace) — fine for a
|
||
single-workspace homelab.
|
||
- **Open WebUI** — the dominant ChatGPT-style chat UI: multi-user auth, model
|
||
switching, tools, pipelines, RAG, image gen (ComfyUI as engine), agents.
|
||
Single container; connect to Ollama / llama.cpp / vLLM via base-URL. Already
|
||
this fleet's chat frontend.
|
||
- **ComfyUI** — node-based image *and* video generation (SD/SDXL/Flux/LLaDA/
|
||
Qwen-Image). The most popular local image-gen frontend in 2026. Repo moved to
|
||
the `Comfy-Org` org. Already running on this fleet (Qwen-Image 2.1 GGUF) and
|
||
wired as the Open WebUI image engine.
|
||
- **llama.cpp** — the reference GGUF inference engine; `llama-server` exposes an
|
||
OpenAI-compatible endpoint. Backbone of this fleet's frank-secondary/frank-large.
|
||
- **vLLM** — high-throughput PagedAttention serving with an OpenAI-compatible
|
||
API; the multi-GPU option (frank-medium runs vLLM tensor-parallel across two
|
||
R9700s).
|
||
- **RAGFlow** — RAG engine with deep document understanding (PDFs, spreadsheets,
|
||
scans) and source citations. `docker compose` full stack; runs fully against
|
||
local models (Ollama).
|
||
- **LobeChat** — polished chat UI grown into an agent-operations layer (plugins,
|
||
agents, scheduled tasks). LobeHub Community License: commercial use allowed,
|
||
but **distributing a derivative requires a commercial license**.
|
||
- **Unsloth** — the leading open-source fine-tuning library (LoRA/QLoRA/SFT) +
|
||
GGUF quantization + a local run/train UI. GPU required; a batch job, not a
|
||
long-running service.
|
||
- **AnythingLLM** — local-first RAG + agent workspace: document ingestion, chat
|
||
over your data, agent mode, multi-user. Ollama first-class.
|
||
- **whisper.cpp** — OpenAI Whisper in C/C++; lightweight fully-local STT on CPU /
|
||
Metal / CUDA; `whisper-server` for an HTTP API.
|
||
- **LibreChat** — multi-provider, multi-user chat UI with agents, MCP, actions.
|
||
`docker compose` (app + MongoDB + Redis).
|
||
- **Tabby** — self-hosted AI coding assistant (Copilot-style) against your own
|
||
models. Apache-2.0 core; `ee/` directory under a separate enterprise license.
|
||
Last push 2026-06-30 (quieter since).
|
||
|
||
## Homelab Fit (Podman/Quadlet + Caddy + local LLM endpoints)
|
||
|
||
| App | Fit | Notes for this fleet |
|
||
|-----|-----|----------------------|
|
||
| **Open WebUI** | **Excellent** | Single container; point at existing llama.cpp/vLLM/Ollama via base-URL. Already the fleet's chat frontend. |
|
||
| **ComfyUI** | **Excellent** | GPU container (already running for Qwen-Image 2.1 GGUF). Serves the Open WebUI image-gen path. |
|
||
| **Ollama** | **Excellent** | Easiest OpenAI-compatible runtime for the other apps; expose on LAN / `host.containers.internal`. |
|
||
| **Dify** | **Good** | compose → Quadlet. Multi-tenant restriction moot for single-workspace. Heavier stack (DB + Redis + vector store). |
|
||
| **RAGFlow** | **Good** | compose → Quadlet. Runs fully against local models. Budget disk for the document index. |
|
||
| **AnythingLLM** | **Good** | Single container, Ollama-first, low footprint. Good local RAG entry point. |
|
||
| **LibreChat** | **Good** | compose (app + Mongo + Redis) → Quadlet. Multi-user, MCP/agents. |
|
||
| **LobeChat** | **Good** | Single Postgres-backed container. Watch the Community License if redistributing a derivative. |
|
||
| **whisper.cpp / faster-whisper** | **Good** | Lightweight STT; `whisper-server` behind Caddy for a LAN transcription API. CPU-fine. |
|
||
| **Piper / Kokoro / Chatterbox** | **Good** | Local TTS; pair with whisper.cpp for a fully-local voice loop. Chatterbox wants a GPU; Piper/Kokoro run on CPU. |
|
||
| **Tabby** | **Good** | compose → Quadlet; self-hosted coding assistant against local models. |
|
||
| **Unsloth** | **OK (one-off)** | Fine-tuning is a batch job, not a service. Run transiently on a GPU server, then serve the GGUF via llama.cpp. |
|
||
| **OpenClaw / Hermes** | **Good, strong isolation** | Node.js / container agent pointing at local LLMs. Highest buzz, but tool/shell access — non-root, restricted network, vetted skills, behind the guard chain. |
|
||
|
||
**GPU co-tenancy note:** ComfyUI, vLLM, Unsloth, and Chatterbox all want VRAM.
|
||
Size new containers against the existing LLM endpoints' held VRAM to avoid
|
||
OOMing a production endpoint. The macOS mini is a natural home for the
|
||
low-footprint, CPU-friendly apps (Open WebUI, AnythingLLM, whisper.cpp,
|
||
Piper/Kokoro, LibreChat, LobeChat) via Metal-backed llama.cpp/Ollama.
|
||
|
||
## Best Practices: Adopting a New App Into a Guard-Chained Homelab
|
||
|
||
1. **Route everything through the existing guard chain.** Put the app behind
|
||
Caddy under the same Anubis+Authentik chain the fleet runs. Do not expose a
|
||
new port directly; use the published-port cutover gate (confirm the port is
|
||
free before starting, or pasta's bind fails and the failure hides behind
|
||
`Restart=on-failure`).
|
||
2. **Enable app-native auth and add SSO on top.** Turn on the app's built-in
|
||
multi-user auth *and* gate it with Authentik SSO at the proxy.
|
||
3. **Prefer local endpoints for the AI; keep PII on-prem.** Point RAG/chat/agent
|
||
apps at local llama.cpp/vLLM/Ollama (base-URL override) so prompts and
|
||
documents never leave the LAN. Reach the host runtime via
|
||
`host.containers.internal`, not `localhost`.
|
||
4. **Use Quadlet for boot persistence.** Convert every compose/`docker run` to
|
||
Quadlet units. Remember the anonymous-volume trap — use named volumes for
|
||
all app data.
|
||
5. **Pin image tags and track release notes.** The window's clearest lesson:
|
||
**OpenClaw 2.0 (2026-09-01) caused migration problems, broken gateways, and
|
||
lost automations for some users.** Pin tags, read release notes, stage
|
||
upgrades (test on the mini / a spare server first).
|
||
6. **Isolate agents aggressively.** OpenClaw and Hermes have tool/shell/browser
|
||
access. Non-root, restricted network, vetted skills/plugins only, full
|
||
logging of tool execution.
|
||
7. **Treat ingested content as untrusted (prompt injection).** Keep a human
|
||
review step before any action; scope agent tool access to the minimum.
|
||
8. **Protect the secrets the app stores.** API keys in env vars (or a vault),
|
||
never committed. Set the encryption key *before* adding keys, never change
|
||
it after.
|
||
9. **Budget VRAM and co-tenancy.** Measure a new GPU container's VRAM against
|
||
the endpoints already resident on the card.
|
||
10. **Back up the data volumes.** Snapshot the app's data volume; agents store
|
||
memory/state that is expensive to rebuild.
|
||
11. **Mind the license.** ComfyUI is GPL-3.0; Dify and LobeChat are
|
||
modified-Apache with multi-tenant / derivative restrictions; Tabby has an
|
||
`ee/` directory. Fine for a single-workspace, non-redistributed homelab, but
|
||
note them before derivative work.
|
||
|
||
# Citations
|
||
|
||
- OpenClaw: https://github.com/openclaw/openclaw (MIT, 390,235 stars)
|
||
- Hermes Agent: https://github.com/NousResearch/hermes-agent (MIT, 247,907 stars)
|
||
- Ollama: https://github.com/ollama/ollama (MIT, 181,444 stars)
|
||
- Dify: https://github.com/langgenius/dify (Modified Apache-2.0, 156,835 stars)
|
||
- Open WebUI: https://github.com/open-webui/open-webui (Open WebUI License, 152,784 stars)
|
||
- ComfyUI: https://github.com/Comfy-Org/ComfyUI (GPL-3.0, 134,407 stars)
|
||
- llama.cpp: https://github.com/ggml-org/llama.cpp (MIT, 129,149 stars)
|
||
- vLLM: https://github.com/vllm-project/vllm (Apache-2.0, 92,405 stars)
|
||
- RAGFlow: https://github.com/infiniflow/ragflow (Apache-2.0, 91,138 stars)
|
||
- LobeChat: https://github.com/lobehub/lobehub (LobeHub Community License, 82,748 stars)
|
||
- Unsloth: https://github.com/unslothai/unsloth (Apache-2.0, 76,560 stars)
|
||
- AnythingLLM: https://github.com/Mintplex-Labs/anything-llm (MIT, 66,318 stars)
|
||
- whisper.cpp: https://github.com/ggml-org/whisper.cpp (MIT, 53,852 stars)
|
||
- LibreChat: https://github.com/danny-avila/LibreChat (MIT, 44,608 stars)
|
||
- Tabby: https://github.com/TabbyML/tabby (Apache-2.0 + ee/, 33,889 stars)
|
||
- Chatterbox: https://github.com/resemble-ai/chatterbox (MIT, 26,515 stars)
|
||
- faster-whisper: https://github.com/SYSTRAN/faster-whisper (MIT, 25,512 stars)
|
||
- Piper TTS: https://github.com/rhasspy/piper (MIT, 11,287 stars)
|
||
- Kokoro-82M: https://github.com/hexgrad/kokoro (Apache-2.0, 8,933 stars)
|
||
- MisoTTS: https://github.com/MisoLabsAI/MisoTTS (Modified MIT, 3,239 stars)
|
||
- OpenClaw 2.0 release coverage: https://www.infoq.com/news/2026/09/openclaw-2-release/
|
||
- OpenClaw self-hosted architecture + security: https://hostkey.com/blog/208-what-is-openclaw-complete-self-hosted-ai-agent-guide/
|
||
- Release cadence: GitHub releases API, 2026-09-22
|