Outlier26 of 26
August.Here.Now/systemsbench
SystemsBench: an open benchmark for systems thinking in AI. A page, with sponsor seats.
A seat is a named place on this site, beside August's work. How Renting Works


What It Is Now
SystemsBench, the open-source benchmark for systems intelligence in large language models. It asks whether a model can map a system, find the leverage and know when it is wrong. Version 0.17 is a research preview on GitHub.
The Name
A page on August's corner of here.now. The benchmark itself belongs to Outlier.Systems.
What We Are Moving On
- Version 0.17 is open source on GitHub, with five scorers, a blind jury and a 354-item register.
Help We Need
- Labs to run it on their own models and publish the result.I Can Help
- Reviewers to audit the scorers.I Can Help
- Sponsors for the next version.I Can Help
Where It Could Go
Paths to start from. Or bring your own idea.
- 1
- 2
Lab Runs
A lab runs SystemsBench on its own models, and the results are published either way.
Ask About This Path - 3
A Domain Of Its Own
SystemsBench moves to its own domain, with a sponsor named at its start.
Ask About This Path
Open Questions
- Which system should the next set of items cover?Answer This
Ways To Rent It
Seat $111 a month
A sponsor seat. It funds the next version, and sponsors are credited in the release.
3 of 3 seats open.
Seats only. The whole domain is not offered.
Price By Term
| Term | USD | BTC | DASH |
|---|---|---|---|
| 1 month | $111 | … | … |
| 3 months 5% off | $316 | … | … |
| 6 months 11% off | $593 | … | … |
| 12 months 22% off | $1,039 | … | … |
DASH totals include 5% off, after the term discount. BTC and DASH follow the live rate. Your invoice fixes the amount.
Every Rent Includes
Early Code
Early access to the code being developed for your domain.
Updates First
Early access to updates as they ship.
Decided Together
You and August decide how your seat or domain is used.
August, Directly
You talk with August himself.
Access Over Time
August shares more access the longer you stay. What opens, and when, goes into your agreement.