highbarEvals for AI agents that behave like tests

Check what your agent does, not only what it says. A YAML spec, the agent you ship, and an exit code that gates the merge.

Tool calls, checked

Which tools, in which order, with which arguments, and nothing else. A call that failed does not count as done.

Forbidden tools and forbidden text. A test with no proof the agent saw its message is an error, never a pass.

No LLM judges the result. Repeat a test with runs and min_pass where the agent is not.

highbar starts your agent or attaches to one already running, with your auth, your request shape and a fresh session per test.

A Python SDK records tools with a decorator. Any other agent writes the trace contract, one JSON line per call.

Exit codes 0, 1 and 2. A failure says what the agent did instead.