Method
How we judge an SDK
Eight criteria, four levels, and a check for each, so you can verify any verdict SDK Wars gives instead of taking it on trust.
Scoring
Each side gets one level per criterion. Levels turn into points:
| Level | Points | Meaning |
|---|---|---|
| Strong | 3 | Built in, complete, documented. |
| Solid | 2 | Built in for the main cases, with gaps you can name. |
| Partial | 1 | Possible, but you build or configure a real part of it. |
| Weak | 0 | Missing: every caller does it by hand. |
| Not verified | Not scored | No source confirms it. The criterion is left out of both totals and listed under the verdict. |
Criteria you mark ×2 count double. If the two totals are within 8% of the maximum possible score, the verdict is Too close to call rather than a win on a technicality.
Language support
Which languages you can ship, and how complete each one is.
Why it matters
Your users write code in the language they already use. An SDK that exists only in TypeScript leaves your Go and Java users with raw HTTP, and an SDK that exists in nine languages but is thin in six is not nine SDKs.
What each level means
Many mainstream languages, each documented as production-ready.
The main languages your users need, with production-ready status for most.
A few languages, or many with uneven completeness.
One language, or none without extra work.
How to check it
- Find the tool's own list of supported languages and the maturity it gives each one (GA, beta, alpha).
- Open the generated or published package for the language you care about and read its README and changelog.
- Check that the package is published to the registry your users install from (npm, PyPI, Maven Central, NuGet, Go modules).
Typing
Request and response types that match the API, caught at compile time.
Why it matters
Types are the documentation developers read most: they show up in the editor. Accurate types catch a wrong field name before it reaches production; loose types (any, dict, map[string]interface{}) push those bugs to runtime.
What each level means
Typed models for every request and response, including unions, enums and nullable fields, validated at runtime where the language allows.
Typed models for most of the API; some edge cases fall back to loose types.
Some typing, but large parts of the API are untyped or typed by hand and can drift.
No types: strings, maps and untyped JSON.
How to check it
- Pick an endpoint that returns a union (oneOf) or an enum and look at the type the SDK gives it.
- Change a field name in a call and see whether the compiler or type checker complains.
- Look for runtime validation (for example Zod in TypeScript or Pydantic in Python) in the package's dependencies.
Auth handling
API keys, OAuth flows and token refresh, without hand-written headers.
Why it matters
Auth is where integrations break first and where security mistakes cost the most. A client that knows the API's security schemes sets the right header, refreshes tokens before they expire and keeps secrets out of logs.
What each level means
Every security scheme the API uses is built in, including OAuth client credentials and token refresh.
Common schemes (API key, bearer, basic) are built in; some OAuth flows need your own code.
You pass a header or token yourself; refresh and rotation are yours to build.
Nothing: every caller builds auth by hand.
How to check it
- List the security schemes in your API description (securitySchemes in OpenAPI) and find each one in the SDK's docs.
- Look for OAuth client credentials and token refresh specifically, since they need more than a static header.
- Read how the SDK takes credentials: constructor, environment variable or per-call option.
Retries
Backoff on 429s and 5xx responses, configurable per call.
Why it matters
Every network call fails sometimes. Without retries, each caller writes their own loop, usually without backoff or jitter, and a brief outage turns into a thundering herd.
What each level means
Retries with exponential backoff on by default, honouring Retry-After, configurable per call.
Retries with backoff available in the main languages; off by default or uneven across languages.
Possible through the HTTP client underneath, but you configure it yourself.
No retry support: callers write their own loop.
How to check it
- Search the SDK's docs and source for retry, backoff and Retry-After.
- Point the SDK at a mock server that returns 429 and count the requests it sends.
- Check whether non-idempotent calls (POST) are retried, and whether you can turn that off.
Pagination
Iterate every page without writing cursor loops.
Why it matters
List endpoints are where callers quietly lose data: they read the first page and stop. An SDK that pages for you (an iterator or async generator) makes the correct thing the easy thing.
What each level means
Cursor, offset and URL-based paging handled for you in every language, as an iterator.
Paging helpers in the main languages; some styles or languages need manual loops.
Page tokens are exposed, but you write the loop.
Single-page calls only.
How to check it
- Call a list endpoint and see whether the result can be iterated past the first page.
- Check which paging styles are supported: cursor, offset, page number, Link header or next URL.
- OpenAPI has no standard pagination keyword, so check how the tool learns your API's paging (an extension, configuration or nothing).
Error handling
Typed errors that carry status, body and request id.
Why it matters
When a call fails, the caller needs to know whether to retry, fix their input or page someone. Typed errors with the status code, the parsed error body and a request id make that a branch in code instead of string matching.
What each level means
Each documented error response is its own typed error with status, body and headers.
One error type with status and parsed body; documented errors are mostly typed.
A generic exception with a status code and raw body.
Errors surface as raw HTTP responses or untyped exceptions.
How to check it
- Trigger a 400 and a 404 and look at what the SDK throws or returns.
- Check that the error carries the response body and any request id header your API sends.
Maintenance
What it costs to keep every SDK in step with the API.
Why it matters
An SDK is a promise to keep up. Every new endpoint, field and deprecation has to reach every language, with a version bump and a changelog. The approach that is cheapest to start is often the most expensive to keep.
What each level means
API changes reach every SDK from one source of truth, with versioning, changelogs and breaking-change checks built in.
Regenerated or updated from one source, with some manual steps or custom code to carry forward.
Each change is manual in some places, or customisations make updates risky.
Every change is hand work in every language.
How to check it
- Ask how a new endpoint reaches each SDK: which steps are automated and which are manual.
- Read the release history of an existing SDK built this way: how long do API changes take to land?
- Check how custom code survives a regeneration or a rewrite.
MCP readiness
Whether AI agents can call the API as tools, safely.
Why it matters
AI agents increasingly call APIs directly, not only code a person wrote line by line. The Model Context Protocol (MCP) is how many agents discover and call tools. An approach that produces good tool definitions, scoped credentials and an audit trail is ready for that caller.
What each level means
Produces an MCP server, or is one, with tool definitions from the API and a defined way to authenticate the caller.
Can produce an MCP server from the same source, at preview or beta maturity.
The pieces exist (a typed client, an API description) but the MCP server is yours to build.
Nothing that helps an agent call the API.
How to check it
- Look for an MCP server target or output in the tool's docs, and its maturity.
- Read the tool definitions it produces: are names and descriptions clear enough for a model to choose the right one?
- Check how the MCP server authenticates the caller and whether each tool call can be logged and attributed.
Put the criteria to work
Pick two sides and the criteria that decide it for you.