930 B
930 B
Design and Testing Phases
- Testing workflow supports pausing and resuming.
- Test frank-archon after design implementation.
- Adversarial reviews are conducted via the frank-archon subagent with extreme thoroughness.
Dashboard Access
- Dashboard results accessible at http://127.0.0.1:8787
Benchmarking Workflow
- When harness indicates failure or configuration changes (e.g., KV quantization), discard current benchmarking results and restart the entire benchmarking run.
Judge Selection
- Use frank-zealot as the default judge for SciCode benchmark evaluation.
- Judge must be performant enough for AA-LCR and HLE tasks.
Reference Comparison
- Evaluate model performance against alternatives as part of the evaluation scope.
- Reference comparison target is qwen-flash-next.
- In fair fight comparison, archon underperforms relative to reference.
- Score comparison to other models is included in the report.