The Prompting Company

Benchmark report

Onboarding verified

3m 53s elapsed

The saved attempt met the documented success criterion.

Experience score

93/ 100

Agent-Ready
  • SInstall
  • AAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth A, Execute S, Docs S, Current SSInstallAAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The first API key value supplied was a URL, not a key, costing a 401 round trip plus character-composition diagnosis before a corrected key was requested.

Where the time went

0:49 on the owner · 3:04 agent time
  • Starting0:01
  • Research0:29
  • Attempt1:29
  • Verify1:06
3m 53sTotal elapsed
1:39Time to first success
0:49Waited on the owner

Judge reason

Saved evidence proves a real live Exa call, not prose or a mock. exa-task/search.sh ran POST https://api.exa.ai/search with Authorization: Bearer and body {"query":"best blog posts about vector databases","contents":{"highlights":true}}; evidence_raw_output.txt (identical to raw_output.txt) ends in HTTP_STATUS:200 and parses as Exa JSON with requestId ed50a7d5dce019250fe6f27ca19094e5, searchTime 1012.1, costDollars 0.007 neural. Independent re-parse: 10 results, all 10 with non-empty title, https:// URL, and a non-empty highlights entry (e.g. exa.ai/blog/building-web-scale-vector-db). Transcript index 31-32 shows the curl invocation and the 200. One prior attempt returned verbatim 401 {"error":"Invalid API key","tag":"INVALID_API_KEY"} because the first supplied value was a 19-char URL-shaped string; a corrected key was requested and succeeded.

Attempt record

  1. 1:45Asked the owner for Exa API key (from dashboard)provided
  2. —First success verified

Attempt artifacts saved.