live: heartbeat sync 2026-10-18T16:47:59-07:00 (Refs #11)

agent/memories/sharpshooter/frank-model-bench-1f78lrpnd0uoh/product.md

@@ -0,0 +1,20 @@
+## Design and Testing Phases
+- Testing workflow supports pausing and resuming.
+- Test frank-archon after design implementation.
+- Adversarial reviews are conducted via the frank-archon subagent with extreme thoroughness.
+
+## Dashboard Access
+- Dashboard results accessible at http://127.0.0.1:8787
+
+## Benchmarking Workflow
+- When harness indicates failure or configuration changes (e.g., KV quantization), discard current benchmarking results and restart the entire benchmarking run.
+
+## Judge Selection
+- Use frank-zealot as the default judge for SciCode benchmark evaluation.
+- Judge must be performant enough for AA-LCR and HLE tasks.
+
+## Reference Comparison
+- Evaluate model performance against alternatives as part of the evaluation scope.
+- Reference comparison target is qwen-flash-next.
+- In fair fight comparison, archon underperforms relative to reference.
+- Score comparison to other models is included in the report.