CASE STUDY 2026-10-23 6 min read

Prowl went from 43 to 96 on its own benchmark

Prowl scores other people's APIs on whether an agent can use them. For months our own listing showed 43, and before that 17. Both numbers were our own fault, and neither described our MCP server. Here is what was wrong and what the score said once we fixed it.

Prowl agent-readiness case study illustration

Prowlagent-readiness43 → 96

Disclosure. Prowl is part of The Prowl Network, the set of services we build ourselves. The runs below used the same pipeline that scores every other service in the catalog, and every number comes from the stored benchmark records.

The starting point: 43/100, and a 17 before that

From 2026-06-16 to 2026-10-23 the prowl.world listing held a 43. Before that, for about a month, it showed 17/100, with the finding "server domain blocked, preventing all MCP interactions".

One thing to be clear about first: that "before" number was not a measurement. It was the heuristic score Prowl assigns a listing before any benchmark has run, built from metadata alone. Its first_try_success of 0 means "never tried", not "tried and failed". So the jump below is the difference between a placeholder and a real run, plus whatever we fixed to make the real run possible.

The 17 was worse than a placeholder: it was a measurement of the wrong thing. prowl.world is on our own sandbox blocklist, so the benchmark is not allowed to call it. The benchmark ran anyway, every request came back as our own refusal, and the grading model scored that refusal as the server being broken. The MCP server at prowl.world/mcp was answering initialize correctly the whole time.

What we changed

Fix	Why it mattered
Added `remotes[]` to `/.well-known/mcp/server.json`	Our manifest only declared an `npx` launcher. It never said where the HTTP endpoint was, which is the exact mistake we flag for other vendors.
A benchmark Prowl refuses to run is no longer scored	If every call was blocked by our own sandbox, the run aborts before grading instead of publishing a number.
The read-only MCP benchmark may reach prowl.world	It only sends `initialize` and `tools/list`, which have no side effects, so the self-block can let it through.

The re-run: 96/100

We re-benchmarked on 2026-10-23. The endpoint resolved from the declared manifest (https://prowl.world/mcp), and auto-detection picked mcp_compliance. initialize answered 200 in 34 ms and tools/list answered 200 in 38 ms.

Dimension (0–10)	Before	After
doc_quality	5	10
response_parseability	5	10
first_try_success	0	10
error_clarity	5	9
auth_simplicity	5	10
token_efficiency	5	10
latency	9	10
consistency	7	9
Overall (0–100)	43	96

The grading model's raw score was 92 and the stored score is 96, after the weighted dimension average.

What the score still flags

Error clarity was not really tested. The run got no error responses, so a 9 there is an inference, not an observation. The grader said so.
The tool schema preview was truncated. Our tools/list is long enough that the grader saw only part of it.
ONBOARD did not run. The global onboarding limit (10 attempts a day) was already used up, so this run says nothing about self-registration.
The Mycelio manifest check errored. Our tool listing is larger than the 256 KB the check accepts. That is a real finding: a long tool list costs every agent tokens.

The takeaway. A low score can come from the scorer, not the service. Before you believe a bad MCP grade, ours included, look at the raw HTTP status of each call. Twice in this listing's history the number described our own plumbing.

Do this for your own MCP server

Make sure server.json declares a remotes[] entry with the URL and transport, not just a local launcher.
Keep tools/list small. Every agent that connects pays for it in tokens.
Run a free benchmark and read the per-call status codes, not only the overall number.

Benchmark your API free → Check brand AI visibility