AI Product Ops · 100-app research pipeline

Which of 100 apps could become an agent toolkit today — and what stops the rest.

An agent researched every app for auth, self-serve access, API surface and official MCP, labelled each GREEN / YELLOW / RED against one rubric, then a second adversarial pass and a live web sample checked the answers. The headline is the pattern, not the table.

The headline

The patterns

Clusters across all 100 — where the easy wins are vs. what needs outreach.

Primary auth

Access model

Most common blocker (non-GREEN apps)

Official vendor MCP

Self-serve rate by category

Green = credentials with no gate. The gradient from dev-tools to AI-native is the story.

The 100-app matrix

Filter and sort. Source-tagged: agent live research loop · verified corrected by the web-verification loop.

#	App	Primary auth	Access	API	Breadth	MCP	Verdict	Main blocker	Evidence

The agent

One app = one unit of work, same schema + rubric for all 100. Composio search tools + Claude.

Where a human was needed

Rubric design. The line between SELF_SERVE_FREE, SELF_SERVE_TRIAL and APPROVAL_REQUIRED is a judgement call the agent can't invent — a human wrote and tuned it, and every agent shares it.
Disambiguation. The agent flagged that developer.copper.co (crypto custody) is a different company from Copper CRM; a human confirmed and steered it to the right domain.
The rate-limit call. The ~100-way agent fleet kept tripping a rolling account session limit, so a human ran it in three resumable waves rather than forcing it into one — the pipeline only re-researches apps missing a record. When docs were ambiguous or a fetch failed, the human decided whether to trust the agent, correct it, or mark it UNVERIFIABLE — instead of silently smoothing over the gaps.

The proof — runnable

agent/run_research.py — the core loopComposio + Claude

from composio import Composio
from composio_anthropic import AnthropicProvider
from anthropic import Anthropic

composio = Composio(provider=AnthropicProvider())
claude   = Anthropic()

# Composio's hosted search toolkit, as Claude tools (no per-app auth)
tools = composio.tools.get(user_id="research", toolkits=["COMPOSIO_SEARCH"])

# 1) agent searches official docs until it stops calling tools
while resp.stop_reason == "tool_use":
    resp = claude.messages.create(model="claude-opus-4-8", tools=tools, messages=msgs)
    msgs += composio.provider.handle_tool_calls(user_id="research", response=resp)

# 2) structured outputs force one schema-valid record per app
record = claude.messages.create(model="claude-opus-4-8", messages=msgs,
    output_config={"format": {"type": "json_schema", "schema": APP_SCHEMA}})

run the pipelineidempotent · resumable

# 1. install + keys (Composio free tier)
pip install -r requirements.txt
export COMPOSIO_API_KEY=... ANTHROPIC_API_KEY=...

# 2. research (only fills apps missing a record)
python agent/run_research.py           # -> data/pass1/

# 3. adversarial verify a sample
python agent/verify.py --ids 82 96     # -> data/verified/

# 4. cluster + rebuild this page
python agent/analyze.py --json
python agent/build_site.py             # -> site/index.html

★ source repo + README ↗ live page

Verification — how we know it's trustworthy

Three loops, ending in a live human+web spot check with hits and misses shown honestly.

App	Field checked	First-pass value	Verdict	What the docs said

→