docs: rename docs/ -> serverAppConfigs-docs/ per system-wide PROJECTNAME-docs convention

All in-repo path references (AGENTS.md, README, old scripts, report artifacts, test code paths) rewritten in the same commit; focused-test map extended for .gitignore, old/scripts/*, dankclaw/comfyui/*, and report/* per this repo's own convention ('add one before committing'). Obsidian wikilinks are title-based and unaffected.
2026-08-28 02:20:06 -07:00
parent 6d4a96a423
commit 5829a0bfe8
140 changed files with 1820 additions and 86 deletions
@@ -0,0 +1,296 @@
+# RomM Scraping & Asset Management
+
+This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver.
+
+**Reference:** https://docs.romm.app
+
+## 1. Metadata Providers
+
+RomM v5.1.0+ supports 11+ metadata providers. The `scan.priority.metadata` list determines which provider wins for descriptive fields. The `scan.priority.artwork` list controls cover art sourcing separately.
+
+### Screenscraper (ScreenScraper.fr)
+
+- **Type:** Free, community-driven (French provider)
+- **Credentials:** `SCREENSCRAPER_USER` + `SCREENSCRAPER_PASSWORD` (account user/pass)
+- **Registration:** https://www.screenscraper.fr/membreinscription.php
+- **Rate limits:** No documented hard cap. Community-claimed ~5 req/sec burst.
+- **Capabilities:** Titles, descriptions, cover art, screenshots, manuals, 3D boxes, CD/cartridge covers, marquee logos, fanart, bezels, gameplay videos.
+- **Region support:** Excellent. Respects `scan.priority.region` and `scan.priority.region_mode`.
+- **Best for:** Richest media coverage (all art types), multi-platform, free. Use as secondary to IGDB for metadata, or swap priority order if Screenscraper art quality is preferred.
+
+### IGDB (Internet Game Database / Twitch)
+
+- **Type:** Freemium (free tier with limits; paid for higher volume)
+- **Credentials:** `IGDB_CLIENT_ID` + `IGDB_CLIENT_SECRET` (Twitch OAuth2)
+- **Registration:** Twitch Developer Portal. Create app with name `romm-<random hash>`, OAuth redirect `localhost`, category `Application Integration`, client type `Confidential`.
+- **Rate limits:** ~10 req/sec for free tier. Cached responses reduce effective calls.
+- **Capabilities:** Titles, descriptions, cover art, screenshots, related games, release dates, genres, platforms.
+- **Limitations:** No 3D boxes or physical media art. No manuals. No marquee/fanart.
+- **Default priority:** Highest (`igdb` listed first in both metadata and artwork priority).
+
+### MobyGames
+
+- **Type:** Paid (hobbyist API subscription; non-commercial use only)
+- **Credentials:** `MOBYGAMES_API_KEY`
+- **Rate limits:** Hobbyist tier ~0.2 req/sec (one request every 5 seconds).
+- **RomM recommendation:** "We recommend using ScreenScraper instead, as it is free to use."
+- **Best for:** Supplemental metadata depth for older PC/console titles.
+
+### RetroAchievements (RAHasher)
+
+- **Type:** Free, hash-based achievement tracking
+- **Credentials:** `RETROACHIEVEMENTS_API_KEY`
+- **How it works:** RomM bundles RAHasher (v1.8.3+). ROMs are hashed during scans, matched against RA online database. Achievement progress synced per-user via their RA username.
+- **Known issues:** Folder-based disc ROMs (`.cue` + `.bin`, `.gdi`) may fail hashing. Non-zip archives (`.7z`, `.rar`) had incorrect hash computation. Ensure `filesystem.skip_hash_calculation` is `false`.
+- **Important:** Achievements cannot be earned via RomM's built-in EmulatorJS player. Must use an RA-capable standalone emulator.
+
+### Additional Built-in Providers
+
+| Provider | Slug | Enabled Via | Type |
+|----------|------|-------------|------|
+| Hasheous | `hasheous` | `HASHEOUS_API_ENABLED=true` | Free, hash-based, proxies IGDB data |
+| Playmatch | `playmatch` | `PLAYMATCH_API_ENABLED=true` | Multi-provider hash matching |
+| LaunchBox | `launchbox` | `LAUNCHBOX_API_ENABLED=true` + XML | Local XML database |
+| Flashpoint | `flashpoint` | `FLASHPOINT_API_ENABLED=true` | 180,000+ browser games |
+| HowLongToBeat | `hltb` | `HLTB_API_ENABLED=true` | Game completion times |
+| SteamGridDB | `steamgriddb` | `STEAMGRIDDB_API_KEY` | Custom cover art |
+| ES-DE gamelist | `gamelist` | Place `gamelist.xml` in platform folder | Local XML |
+
+## 2. Recommended Provider Combinations
+
+| Combo | Providers | Best For | Cost |
+|-------|-----------|----------|------|
+| Quick Starter | Hasheous only | No API keys needed | Free |
+| French Connection | Screenscraper + RetroAchievements | Rich media, free | Free |
+| Chef's Choice (recommended) | Hasheous + IGDB + SteamGridDB + RetroAchievements | Best overall quality | Free (IGDB free tier) |
+| Twitch Fanboy | IGDB + Playmatch | Single-provider solution | Free |
+
+**For Dankserver,** the recommended setup:
+- `IGDB` (primary metadata) + `SCREENSCRAPER` (artwork backup + rich media) + `RETROACHIEVEMENTS` (achievements) + `HASHEOUS_API_ENABLED=true` (fast hash-based fallback)
+
+## 3. ROM File Naming Conventions
+
+### No-Intro Naming Scheme (Recommended)
+
+The de facto standard for cartridge and handheld ROMs. Only `Title` and `Region` are required.
+
+```
+Super Metroid (Japan, USA) (En,Ja)
+Sonic the Hedgehog (USA)
+Metal Gear Solid (USA) (Disc 1)
+[BIOS] PS1 System BIOS (USA)
+Tetris [!].gba
+```
+
+**Field meanings:**
+- `(Region)`: `USA`, `EUR`, `JPN`, `UE` (USA+Europe), `JUE` (Japan+USA+Europe)
+- `(Languages)`: `En`, `Ja`, `De`, `Fr`, `Es`
+- `[Status]`: `[!]` = verified good dump, `[b]` = bad dump, `[c]` = completed
+- `(Devstatus)`: `(Proto)`, `(Demo)`, `(Release Candidate)`
+- `[BIOS]` prefix: Identifies BIOS files
+
+### RomM's Region/Language/Revision Parsing
+
+RomM parses inline tags from filenames:
+
+```
+tetris (En) [1.0001] [HACK] [!].gba
+```
+
+- Region/language: ISO-like codes or full names
+- Revision: `rev v1` or `rev-1`
+- Provider ID tags: `(igdb-xxxx)`, `(ssfr-xxxx)`, `(moby-xxxx)`, `(ra-xxxx)` — forces specific provider match
+
+### Best Practices for Dankserver
+
+1. Use No-Intro naming as the standard.
+2. Use **Igir** to rename and organize ROMs into RomM's required folder structure:
+   ```
+   igir copy zip test clean \
+     --dat "No-Intro*.zip" \
+     --input "ROMs" \
+     --output "/opt/romm/roms/{romm}"
+   ```
+3. Keep `[BIOS]` prefix for BIOS files in the `bios/` directory.
+4. One ROM file per game folder where possible.
+5. Use archive formats (`.zip`, `.7z`) — RomM supports them natively.
+
+## 4. Cover Art & Media Assets
+
+### Media Types
+
+| Media Type | Description | Best Provider |
+|-----------|-------------|---------------|
+| `box2d` | Standard 2D front cover art | IGDB, Screenscraper |
+| `box3d` | 3D box art | Screenscraper |
+| `miximage` | Composite (box + screenshot + logo) | Screenscraper |
+| `screenshot` | In-game screenshot | IGDB, Screenscraper |
+| `marquee` | Transparent logo/header | Screenscraper |
+| `bezel` | EmulatorJS-compatible bezel | Screenscraper |
+| `manual` | PDF instruction manual | Screenscraper |
+| `video` | Gameplay video/trailer | Screenscraper |
+
+### Custom Artwork Directories
+
+RomM supports manual artwork placement via subdirectories within each game folder:
+
+```
+/opt/romm/library/roms/{platform}/{game}/
+    game.rom
+    covers/
+    screenshots/
+    3dboxes/
+    manuals/
+    fanart/
+    videos/
+    marquees/
+    bezels/
+```
+
+For ES-DE gamelist imports, media maps as:
+
+| gamelist.xml Tag | Folder Name |
+|-----------------|-------------|
+| `<cover>` | `covers/` |
+| `<backcover>` | `backcovers/` |
+| `<box3d>` | `3dboxes/` |
+| `<marquee>` | `marquees/` |
+| `<screenshot>` | `screenshots/` |
+| `<fanart>` | `fanart/` |
+| `<manual>` | `manuals/` |
+
+### MAME Artwork
+
+MAME uses cabinet pics, flyers, marquees, control panels, backdrops, overlays. Stored in `{platform}/mame_artwork/{game}/` subdirectories.
+
+Sources: Mr. Do's Arcade (https://www.mameworld.info/mrdo/), ProgettoSnaps (http://www.progettosnaps.net/artworks/).
+
+## 5. Emulator Integration
+
+### EmulatorJS (Built-in)
+
+Web-based player using RetroArch cores compiled to WebAssembly. Runs in-browser with no local emulator needed. Configured via `emulatorjs.*` settings in `config.yml`. Supports netplay via STUN/TURN servers.
+
+### Emulator Streaming
+
+Launch native emulator containers (PCSX2, Dolphin, Xemu) and stream picture/sound/input back to the browser. Configured via `streaming.*` in `config.yml`.
+
+### BIOS File Requirements
+
+BIOS files placed in `/opt/romm/library/bios/{platform}/`. Platform subdirectories required. File names should match emulator expectations:
+
+| Platform | BIOS File |
+|----------|-----------|
+| PlayStation | `scph1001.bin`, `scph5501.bin`, `scph5502.bin` |
+| GB/GBC | `gb_bios.bin`, `gbc_bios.bin` |
+| SNES | `sfc_bios.bin` |
+
+BIOS matching done by hash against platform requirements.
+
+### Folder Structure Options
+
+**Structure A (recommended):**
+```
+/opt/romm/library/
+    roms/
+        └── {platform}/
+            └── {game}.rom
+    bios/
+        └── {platform}/
+            └── {bios_file}.bin
+```
+
+**Structure B (fallback):**
+```
+/opt/romm/library/
+    └── {platform}/
+        ├── roms/
+        │   └── {game}.rom
+        └── bios/
+            └── {bios_file}.bin
+```
+
+Dankserver mounts `/opt/romm/library:/romm/library` — RomM auto-detects structure.
+
+## 6. Scraping Workflow
+
+### Recommended Pipeline
+
+1. Organize ROMs → No-Intro naming + RomM folder structure.
+2. Create `config.yml` → Set provider credentials, priority lists, region prefs.
+3. First scan → Scan ALL → verify everything is matched.
+4. Review unmatched → Manually match or add provider tags to filenames.
+5. Fetch artwork → Re-scan with `scan.media` extended to desired types.
+6. Sync RetroAchievements → Set user RA username → trigger sync.
+7. Export gamelist → Enable `scan.gamelist.export` for ES-DE/Batocera.
+8. Maintain → Enable scheduled rescans for new DAT updates.
+
+### Scan Types
+
+| Scan Type | What It Does |
+|-----------|-------------|
+| ALL | Full scan — hash, match metadata, fetch artwork |
+| UNMATCHED | Only matches unmatched ROMs (faster) |
+| METADATA | Updates metadata only |
+| ARTWORK | Re-fetches artwork only |
+| RETROACHIEVEMENTS | Syncs achievement data |
+
+### Region Handling
+
+```yaml
+scan:
+  priority:
+    region:
+      - "us"
+      - "eu"
+      - "jp"
+    region_mode: prefer_rom_tags   # or prefer_config
+```
+
+- `prefer_rom_tags` (default): ROM filename region tags take precedence.
+- `prefer_config`: Config regions come first.
+- **NTSC-U** (USA): 60 Hz, typically fastest, most translations.
+- **PAL** (Europe): 50 Hz, sometimes slower, may have bonus content.
+- **NTSC-J** (Japan): 60 Hz, Japanese only unless region-patched.
+
+## 7. Collection Management Tools
+
+| Tool | Primary Role | Platform |
+|------|-------------|----------|
+| RomVault 3.7.6 | DAT-file-based verification and reorganization | Windows |
+| clrMAMEPro | ROM set validation, deduplication, 1G1R | Windows |
+| DATBios | BIOS file organization using DAT files | Windows |
+| Batocera/Lakka integration | ES-DE gamelist import from RomM | Linux/ARM |
+
+## 8. Environment Variables (Dankserver)
+
+```bash
+# Auth
+ROMM_USERNAME=admin
+ROMM_PASSWORD=<strong password>
+ROMM_AUTH_SECRET_KEY=$(openssl rand -hex 32)
+
+# Metadata Providers
+IGDB_CLIENT_ID=<your-client-id>
+IGDB_CLIENT_SECRET=<your-client-secret>
+SCREENSCRAPER_USER=<screenscraper-username>
+SCREENSCRAPER_PASSWORD=<screenscraper-password>
+MOBYGAMES_API_KEY=<mobygames-key>
+RETROACHIEVEMENTS_API_KEY=<retroachievements-api-key>
+STEAMGRIDDB_API_KEY=<steamgriddb-key>
+
+# Feature Flags
+HASHEOUS_API_ENABLED=true
+PLAYMATCH_API_ENABLED=true
+
+# Scanning
+ENABLE_RESCAN_ON_FILESYSTEM_CHANGE=true
+ENABLE_SCHEDULED_RESCAN=true
+REFRESH_RETROACHIEVEMENTS_CACHE_DAYS=7
+```
+
+## See Also
+
+- [[RomM]] — main service documentation
+- [[Dankserver Stack]]
+- [[ARR-Stack]] — Readarr for ebook auto-downloads alongside ROM management