How to Give AI Agents Social Media Data in 2026: 6 Approaches

An agent that researches creators, tracks a brand or summarises what people are saying needs fresh social data, and its training data is months old. This guide compares the six ways to give an AI agent social media data in 2026: browsing and scraping, official platform APIs, a data API called as a tool, an MCP server, an Agent Skill and llms.txt. It recommends one for each situation, with code.

What does an agent need from social media data?

Before comparing approaches, it helps to be clear about what makes data useful to an agent rather than to a person:

• Structured output. Numbers as numbers, dates as dates, the same fields on every platform. An agent that has to parse HTML wastes tokens and makes mistakes.
• Small payloads. Every byte lands in the context window. A profile should be a profile, not a page of markup.
• A known price per call. Agents loop. If a call's cost is unknown, so is the bill.
• Stable error codes. An agent should know the difference between “that handle doesn’t exist” and “wait 30 seconds”.
• Public data only, with no logins. Handing an agent your personal social accounts is a security and terms-of-service problem.

Keep these five in mind; each approach below meets a different subset.

Can't the agent just browse and scrape the site itself?

It can. Browser tools let an agent open a page, read it and click around, and some agent frameworks ship a headless browser as a built-in tool. For a one-off look at a single page, that is fine.

As a data source, it has real problems:

• Tokens. A rendered social page is large. The agent reads navigation, ads and markup to find a follower count.
• Walls. Many social sites show login prompts or bot challenges to automated browsers, especially from datacenter IPs. Completing those is not something an agent should do.
• Fragility. Layouts change, and the agent's approach that worked yesterday fails silently today.
• No structure. The output is text the agent interprets, so the same question can give different numbers on different runs.

Use browsing for pages no API covers, not as your main pipe.

What about the official platform APIs?

Official APIs are the right choice when you work with your own accounts: posting, reading your own analytics, replying to your own comments. They are stable and sanctioned.

For reading other accounts' public data, they rarely fit. Most scope access to accounts that authorised your app, some require an application and review, and research programmes are usually limited to approved academics. Each platform also has its own auth, rate limits and response shape, so an agent that covers five platforms needs five integrations. See how to scrape TikTok data for what that looks like on one platform.

How do you call a data API as a tool?

A hosted data API does the scraping, proxies and parsing for you, and returns clean JSON. You describe one or two of its endpoints to the model as tools, and your code runs the call when the model asks. This is plain function calling, and it works with any model that supports tools.

Here is a minimal, complete loop with the Anthropic TypeScript SDK and fetch. The tool reads any public profile from crawlfeed:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const tools: Anthropic.Tool[] = [
  {
    name: 'get_social_profile',
    description:
      'Get a public social media profile (bio, follower and post counts, verification). Costs 1 credit on these platforms.',
    input_schema: {
      type: 'object',
      properties: {
        platform: { type: 'string', enum: ['tiktok', 'instagram', 'youtube', 'x'] },
        handle: { type: 'string', description: 'Username without the @' },
      },
      required: ['platform', 'handle'],
    },
  },
];

async function getSocialProfile(input: { platform: string; handle: string }) {
  const url = `https://api.crawlfeed.dev/v1/${input.platform}/profile?handle=${encodeURIComponent(input.handle)}`;
  const res = await fetch(url, {
    headers: { Authorization: `Bearer ${process.env.CRAWLFEED_KEY}` },
  });
  const body = await res.json();
  // Errors carry a stable code (not_found, rate_limited, insufficient_credits, ...).
  if (!res.ok) return { error: body.code, detail: body.detail, retry_after: body.retry_after };
  return { profile: body.data, credits_used: body.meta.credits_used };
}

const messages: Anthropic.MessageParam[] = [
  { role: 'user', content: 'Compare the follower counts of @nasa on Instagram and X.' },
];

let response = await client.messages.create({
  model: 'claude-sonnet-5',
  max_tokens: 1024,
  tools,
  messages,
});

while (response.stop_reason === 'tool_use') {
  messages.push({ role: 'assistant', content: response.content });
  const results: Anthropic.ToolResultBlockParam[] = [];
  for (const block of response.content) {
    if (block.type !== 'tool_use') continue;
    const output = await getSocialProfile(block.input as { platform: string; handle: string });
    results.push({ type: 'tool_result', tool_use_id: block.id, content: JSON.stringify(output) });
  }
  messages.push({ role: 'user', content: results });
  response = await client.messages.create({
    model: 'claude-sonnet-5',
    max_tokens: 1024,
    tools,
    messages,
  });
}

console.log(response.content);

The same pattern works for posts, comments or search: one tool per endpoint you want the agent to have. The request is the same for every platform, and the API returns the same Profile shape whichever one you ask for:

curl -G https://api.crawlfeed.dev/v1/youtube/profile \
  -H "Authorization: Bearer $CRAWLFEED_KEY" \
  --data-urlencode "handle=MrBeast"

A profile costs 1 credit on YouTube and 1 credit on Instagram. Because you write the tool, you decide exactly what the agent may call and how much of each response it sees.

When is an MCP server the better fit?

If the agent lives in an MCP client (Claude Code, Claude Desktop, Cursor, VS Code, Windsurf or Codex), you do not need to write the tool at all. An MCP server publishes its tools, and the client hands them to the model.

The crawlfeed MCP server at https://mcp.crawlfeed.dev/mcp exposes 9 tools: get_profile, get_posts, get_post, get_comments, search, enrich_arabic, get_balance, list_endpoints and run_request, which reaches every other operation in the catalogue. For a client that sends headers, the whole setup is one config block:

{
  "mcpServers": {
    "crawlfeed": {
      "type": "http",
      "url": "https://mcp.crawlfeed.dev/mcp",
      "headers": {
        "Authorization": "Bearer sk_your_key_here"
      }
    }
  }
}

Tools cost the same credits as the API, return the unified data plus a compact meta, and keep the same error codes. The step-by-step setup for every client is in our social media MCP server guide.

MCP also suits agents you build, if your framework has an MCP client: you point it at the URL instead of hand-writing tool definitions.

What does an Agent Skill add?

An Agent Skill is a folder of instructions (a SKILL.md plus references) that an agent loads when a task matches. It does not fetch anything by itself. It teaches the agent how to use an API well: which route to call, how paging works, what each error code means, and habits like “answer from one call when one call is enough”.

The crawlfeed skill covers auth, the core routes, the response envelope, asynchronous cross-platform search, Arabic enrichment and every error code. Install it with:

npx skills add ilyasabdul19/crawlfeed-skills

A skill is most useful for coding agents that call the REST API from scripts they write. If the crawlfeed MCP server is also connected, the skill tells the agent to prefer the MCP tools.

Where does llms.txt fit?

llms.txt is a plain-text index of a site's docs, written for language models. crawlfeed serves one at crawlfeed.dev/llms.txt, with links to every docs page, the MCP endpoint and the skill install command. llms-full.txt adds every platform's endpoints in a single file.

Point an agent at it when it needs to write code against the API: “Read crawlfeed.dev/llms.txt, then build a script that tracks these five creators.” It contains no live data, so it complements the other approaches rather than replacing them.

Which approach should you choose?

Here is how the six compare against what an agent needs:

By situation:

• You want Claude or Cursor to answer questions about creators today: connect the MCP server.
• You are building a product agent (a support bot, a research pipeline, a CRM enrichment job): call the data API as a tool, with only the endpoints it needs.
• A coding agent is writing an integration for you: install the Agent Skill, and point it at llms.txt.
• You manage your own brand accounts: use the official APIs for those, and a data API for everyone else.
• You need one page no API serves: let the agent browse it, once.

These combine well. A typical setup is MCP for day-to-day questions plus the skill for the code the agent writes.

How do you keep an agent's costs under control?

Agents repeat themselves, so build in limits:

• Price before you loop. On crawlfeed, list_endpoints (MCP) and the docs list each call's price. Both are free to read.
• Stop paging early. Every page is billed. Tell the agent how many results you need.
• Use the meta. credits_used and credits_remaining come back on every response, so the agent can report spend as it goes.
• Handle errors by code. On rate_limited, wait retry_after seconds; on insufficient_credits, stop and tell the user. Failed calls are refunded. The full list is in the errors reference.

Wrapping up

For social data, give your agent a data API rather than a browser: MCP if it runs in an assistant, function calling if you build it, with the Agent Skill and llms.txt to help it use the API well. crawlfeed covers 26 platforms and 201 endpoints in one shape, and new accounts get 100 free credits. Platform guides: TikTok and YouTube. Everything else is in the docs.