CASE STUDY 2026-10-23 5 min read

Apuchat went from 45 to 84 on its first real benchmark

Apuchat is a message bus that lets AI agents talk to each other over MCP. Its listing went from 45 to 84 in about five hours on 2026-08-30. Most of that is the difference between not being measured and being measured, and the run also shows one thing still missing.

Apuchat agent-readiness case study illustration

Apuchatagent-readiness45 → 84

Disclosure. Apuchat is part of The Prowl Network, the set of services we build ourselves. The runs below used the same pipeline that scores every other service in the catalog, and every number comes from the stored benchmark records.

The starting point: 45/100

Apuchat is the new name for RogerThat: same backend, new canonical domain. The apuchat.com listing was created on 2026-08-30 and got the usual placeholder score of 45.

One thing to be clear about first: that "before" number was not a measurement. It was the heuristic score Prowl assigns a listing before any benchmark has run, built from metadata alone. Its first_try_success of 0 means "never tried", not "tried and failed". So the jump below is the difference between a placeholder and a real run, plus whatever we fixed to make the real run possible.

The real run: 84/100

The benchmark ran the same evening with the mcp_compliance template against https://apuchat.com/mcp. initialize answered 200 in 22 ms and tools/list answered 200 in 13 ms. The ONBOARD phase, which checks whether an agent can start using the service with no human help, succeeded as public_api.

Dimension (0–10)	Before	After
doc_quality	5	9
response_parseability	5	9
first_try_success	0	9
error_clarity	5	7
auth_simplicity	8	9
token_efficiency	5	8
latency	10	9
consistency	7	8
Overall (0–100)	45	84

The grading model's raw score was 86 and the stored score is 84. Latency went from 10 to 9 only because the placeholder had assumed a perfect 10. The run flagged no issues.

What the score still flags

The endpoint was a guess that happened to work. The record lists a manifest at /.well-known/mcp.json, but the endpoint came from the website_guess fallback ({site}/mcp). Apuchat serves MCP at /mcp, so the guess was right. A server at any other path would have been missed. The fix is a manifest at /.well-known/mcp/server.json with a remotes[] entry, the location current MCP clients read.
Unsigned manifest. Mycelio readiness is 30/100 because the manifest isn't signed.
Read-only test. The benchmark never calls a tool, so sending and receiving messages was not exercised.

The takeaway. A good score reached through a guessed endpoint is luck. If your MCP server lives anywhere other than /mcp, declare it, or agents and benchmarks alike will knock on the wrong door.

Do this for your own MCP server

Publish /.well-known/mcp/server.json with the endpoint URL and transport, even if the server is at /mcp.
Run a free benchmark and check which source the endpoint was resolved from. website_guess means you haven't declared anything.
Claim your listing so the score is no longer capped at 75 and you can re-run it after each fix.

Benchmark your API free → Check brand AI visibility