78
/100
prowl
Benchmarked Apr 05, 2026

Orquesta Verified

Prompt to Production platform. Install an agent on any machine, invite your team, and develop software using AI prompts. Real-time collaboration, autonomous execution, full audit trail.

developer-toolsai-agentsdevops api_benchmark
Benchmark Your API

Score Breakdown

Latency 10/10
Auth Simplicity 10/10
Parseability 10/10
Consistency 8/10
Token Efficiency 8/10
Documentation 7/10
Error Clarity 7/10
First-Try Success 6/10

Benchmark Analysis Log

Full LLM thinking from the 4-phase benchmark pipeline.

Analyze
```json
{
  "service_type": "rest_api",
  "base_url": "https://orquesta.live",
  "auth_method": "none",
  "auth_config": {
    "header": null,
    "prefix": null
  },
  "endpoints": [
    {
      "path": "/api/public/feed",
      "method": "GET",
      "purpose": "List all public projects from the community feed with prompts, votes, and agent status",
      "params": {},
      "response_format": "json",
      "is_primary": false
    },
    {
      "path": "/api/public/stats",
      "method": "GET",
      "purpose": "Get aggregate platform statistics — total prompts executed, users, and teams",
      "params": {},
      "response_format": "json",
      "is_primary": false
    },
    {
      "path": "/api/oss/publish",
      "method": "GET",
      "purpose": "List all self-hosted OSS instances published to the Orquesta community directory",
      "params": {},
      "response_format": "json",
      "is_primary": false
    },
    {
      "path": "/api/public/vote",
      "method": "POST",
      "purpose": "Upvote or downvote a public project anonymously using a browser UUID",
      "params": {
        "vote": {
          "type": "integer",
          "required": true
        },
        "projectId": {
          "type": "string",
          "required": true
        },
        "voterUuid": {
          "type": "string",
          "required": true
        }
      },
      "response_format": "json",
      "is_primary": false
    },
    {
      "path": "/api/public/propose",
      "method": "POST",
      "purpose": "Propose a prompt for a public project. Rate-limited to 3 proposals per UUID per day.",
      "params": {
        "content": {
          "type": "string",
          "required": true
        },
        "projectId": {
          "type": "string",
          "required": true
        },
        "proposerUuid": {
          "type": "string",
          "required": true
        }
      },
      "response_format": "json",
      "is_primary": false
    },
    {
      "path": "/api/oss/publish",
      "method": "POST",
      "purpose": "OSS instances publish themselves to the community feed.",
      "params": {
        "name": {
          "type": "string",
          "required": true
        },
        "description": {
          "type": "string",
          "required": true
        },
        "instanceUrl": {
          "type": "string",
          "required": true
        },
        "promptCount": {
          "type": "integer",
          "required": true
        }
      },
      "response_format": "json",
      "is_primary": false
    },
    {
      "path": "/api/contact",
      "method": "POST",
      "purpose": "Contact form for enterprise inquiries.",
      "params": {
        "name": {
          "type": "string",
          "required": true
        },
        "email": {
          "type": "string",
          "required": true
        },
        "company": {
          "type": "string",
          "required": true
        },
        "message": {
          "type": "string",
          "required": true
        }
      },
      "response_format": "json",
      "is_primary": false
    }
  ],
  "pricing_model": {
    "type": "unknown",
    "details": {},
    "free_tier": null,
    "paid_tiers": []
  },
  "rate_limits": {
    "rpm": 60,
    "tpm": null,
    "daily": null,
    "concurrent": null
  },
  "capabilities": [
    "public_projects_feed",
    "aggregate_statistics",
    "self_hosted_oss_listing",
    "voting_system",
    "prompt_proposals",
    "oss_instance_publish",
    "contact_inquiries"
  ],
  "agent_readiness": {
    "supports_x402": false,
    "supports_streaming": false,
    "has_sandbox": false,
    "sdks": [],
    "agent_auth_methods": []
  }
}
```
Plan
```json
{
  "tests": [
    {
      "name": "connectivity_and_auth",
      "endpoint": "/api/public/feed",
      "method": "GET",
      "headers": {
        "Authorization": "Bearer REPLACE_WITH_ACTUAL_TOKEN"
      },
      "payload": {},
      "expected_status": 200,
      "expected_behavior": "Returns list of public projects with prompts, votes, and agent status",
      "metrics": ["latency", "status_code"],
      "validation": {
        "field": "data",
        "type": "array",
        "min_length": 1
      }
    },
    {
      "name": "list_oss_instances",
      "endpoint": "/api/oss/publish",
      "method": "GET",
      "headers": {
        "Authorization": "Bearer REPLACE_WITH_ACTUAL_TOKEN"
      },
      "payload": {},
      "expected_status": 200,
      "expected_behavior": "Returns list of self-hosted OSS instances",
      "metrics": ["latency", "status_code"],
      "validation": {
        "field": "data",
        "type": "array",
        "min_length": 1
      }
    },
    {
      "name": "post_vote",
      "endpoint": "/api/public/vote",
      "method": "POST",
      "headers": {
        "Authorization": "Bearer REPLACE_WITH_ACTUAL_TOKEN",
        "Content-Type": "application/json"
      },
      "payload": {
        "vote": 1,
        "projectId": "12345",
        "voterUuid": "uuid-1234"
      },
      "expected_status": 200,
      "expected_behavior": "Acknowledges the upvote or downvote on a project",
      "metrics": ["latency", "status_code"],
      "validation": {
        "field": "confirmation",
        "type": "string",
        "min_length": 1
      }
    },
    {
      "name": "propose_prompt",
      "endpoint": "/api/public/propose",
      "method": "POST",
      "headers": {
        "Authorization": "Bearer REPLACE_WITH_ACTUAL_TOKEN",
        "Content-Type": "application/json"
      },
      "payload": {
        "content": "A new prompt",
        "projectId": "12345",
        "proposerUuid": "uuid-5678"
      },
      "expected_status": 200,
      "expected_behavior": "Proposal submission accepted",
      "metrics": ["latency", "status_code"],
      "validation": {
        "field": "proposalId",
        "type": "string",
        "min_length": 1
      }
    },
    {
      "name": "contact_via_form",
      "endpoint": "/api/contact",
      "method": "POST",
      "headers": {
        "Authorization": "Bearer REPLACE_WITH_ACTUAL_TOKEN",
        "Content-Type": "application/json"
      },
      "payload": {
        "name": "John Doe",
        "email": "john.doe@example.com",
        "company": "Sample Company",
        "message": "Inquiry about API usage"
      },
      "expected_status": 200,
      "expected_behavior": "Inquiry submission acknowledged",
      "metrics": ["latency", "status_code"],
      "validation": {
        "field": "submissionStatus",
        "type": "string",
        "min_length": 1
      }
    },
    {
      "name": "test_bad_input_handling",
      "endpoint": "/api/public/vote",
      "method": "POST",
      "headers": {
        "Authorization": "Bearer REPLACE_WITH_ACTUAL_TOKEN",
        "Content-Type": "application/json"
      },
      "payload": {
        "vote": "invalid_vote",
        "projectId": "12345"
      },
      "expected_status": 400,
      "expected_behavior": "Returns error due to invalid vote type",
      "metrics": ["latency", "status_code"],
      "validation": {
        "field": "error",
        "type": "string",
        "min_length": 1
      }
    }
  ],
  "pricing_probes": [],
  "stress_profile": {
    "concurrent_requests": 5,
    "duration_seconds": 10,
    "ramp_up": true
  }
}
```
Execute

3/6 tests passed

TestEndpointStatusLatency
connectivity_and_authGET /api/public/feed20041ms
list_oss_instancesGET /api/oss/publish20029ms
post_votePOST /api/public/vote40430ms
propose_promptPOST /api/public/propose40429ms
contact_via_formPOST /api/contact200280ms
test_bad_input_handlingPOST /api/public/vote40027ms
Interpret
```json
{
  "overall": 73,
  "dimensions": {
    "token_efficiency": 8.0,
    "first_try_success": 6.0,
    "response_parseability": 10.0,
    "error_clarity": 7.0,
    "doc_quality": 7.0,
    "auth_simplicity": 10.0,
    "latency": 10.0,
    "consistency": 8.0
  },
  "pricing_normalized": {
    "cost_per_1k_requests": null,
    "cost_per_1m_input_tokens": null,
    "cost_per_1m_output_tokens": null,
    "free_tier_requests": null,
    "estimated_monthly_cost_light": null,
    "estimated_monthly_cost_heavy": null
  },
  "issues": [
    {
      "severity": "medium",
      "detail": "'Project not found' error returned for known test cases, likely indicating missing or incorrect project setup for tests.",
      "endpoint": "/api/public/vote"
    },
    {
      "severity": "medium",
      "detail": "Error descriptions could provide more detailed guidance, e.g. explaining why a project might not be found.",
      "endpoint": "/api/public/vote"
    }
  ],
  "recommendations": [
    "Ensure test environment includes necessary data for all test cases to pass successfully.",
    "Improve error messages to provide more detailed guidance on how to resolve issues, especially around known common errors like 'Project not found'.",
    "Consider providing usage-based pricing model details for more comprehensive cost evaluation."
  ]
}
```

Agent Readiness

x402 Payments
Not supported
Streaming
No
Sandbox
None
Agent Auth
Unknown
SDKs
None listed
MCP Support
No

Vendor Profile

Orquesta is the Figma for Prompters. Install an AI coding agent on any machine, invite your team, and develop software using prompts. Anyone can prompt, everyone sees the results. 4 execution modes, real-time streaming, full audit trail. 110 users, 21 paying customers.

Features

4 AI execution modes: Auto, SSH, Agent (Claude CLI), Batuta (autonomous loop)Agent Grid: multi-project control room with drag, resize, interactive sessionsReal-time WebSocket log streaming with color-coded outputMulti-channel input: Dashboard, Telegram, Embed SDK, CLIFull audit trail: prompts, execution logs, git commits, token usage, costsTeam collaboration with role-based permissions and quality gatesCLAUDE.md sync for coding standards enforcementPerformance dashboard with execution times, success rates, cost trendsCloud VM provisioning from the dashboardEmbeddable SDK for third-party apps

Want the full interactive view?

See operational metrics, LLM evaluations, agent readiness, and more.

Open in Dashboard