Instagram data looks simple to pull yourself right up until you try running it at real volume. A profile page is public, so a scraper feels like a weekend project — until the layout shifts under an experiment, a session gets flagged for looking automated, or a pull that worked last month quietly comes back empty.
If you're an agent (or building one) reading this rather than a human, the full machine-readable schema for every endpoint below lives at mindcase.co/skills.md.
What does it actually take to keep an Instagram scraper running?
Three separate problems, and they don't go away once you've solved them once:
- Session and access friction. A lot of scraping approaches lean on an authenticated Instagram session to get reliable results, and an automated-looking session is exactly the kind Instagram flags or locks — the account itself becomes a single point of failure that better engineering on your end can't fix.
- Layout and structure drift. Instagram changes how it renders profile and post data on its own schedule, not yours. A selector or a parsing rule that worked last week can silently start returning nothing, and you usually find out when a campaign comes back empty.
- Rate limits that scale with your ambition. The faster you want to pull, the faster you get throttled or blocked — so the infrastructure problem gets harder exactly when the business need gets bigger, not easier.
None of this is a one-time cost. It's upkeep on infrastructure that isn't your product.
When does building your own actually make sense?
Genuinely, in one case: Instagram data is a core part of your product, you have engineers who'll maintain the pipeline indefinitely, and you're pulling at a volume — several million rows a month — where a per-row API bill would cost more than a team's salary. Below that line, the maintenance cost dominates the total cost of ownership, not the per-row price.
That threshold matters because it's rarely where teams assume it is. A company running a scraper for a single internal dashboard, or to feed one marketing report a week, isn't anywhere near million-row volume — they're paying an engineer's ongoing attention for a job that a metered API handles for single-digit dollars. The volume where building wins is a genuinely high bar, not "we scrape sometimes."
If you're pulling data to run a campaign, benchmark competitors, or feed an agent's research step — not to build a scraping product — you're below that line.
How do you get the same data without maintaining a scraper?
Say you're tracking 50 competitor and creator accounts in the outdoor apparel space, weekly: who's growing, and what they're posting about.
curl -X POST "https://api.mindcase.co/v1/data/instagram/profiles/run?wait=true" \
-H "Authorization: Bearer $MINDCASE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"params": {
"usernames": ["patagonia", "cotopaxi", "yetitumblers"]
}
}'
# $0.002 per profileThat's follower counts and bio data for the whole list in one call, 25 fields per account, no session to babysit. Pulling 50 profiles weekly costs $0.10 — Instagram Profiles is billed per row returned, so the weekly cost doesn't change whether you run it once or automate it every Monday.
Pair it with Instagram Posts for what each account has actually been posting:
curl -X POST "https://api.mindcase.co/v1/data/instagram/posts/run?wait=true" \
-H "Authorization: Bearer $MINDCASE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"params": {
"handles": "patagonia",
"maxResults": 10
}
}'
# $0.002 per postTen recent posts per account, 35 fields each including captions, engagement, and hashtags — run across all 50 accounts, that's $10 a week for the full competitive read. The bill is the same shape whether you run it by hand once or wire it into a scheduled job; nothing about the pricing changes when you automate it.
Put a number on the alternative: $10.10 a week is roughly $525 a year for this specific pull. A DIY scraper that breaks even once a quarter — a layout tweak, a session that needs re-authenticating, a rate limit that needs tuning — costs more than that in a single afternoon of an engineer's time fixing it, and it breaks on Instagram's schedule, not yours. The API bill is predictable before you've run it; the maintenance bill isn't.
What can't you get this way?
Two cases, and both are true of any vendor, not just Mindcase.
Private or connections-gated content. Anything behind a login — a private account, a close-friends story — needs an authenticated session to see at all. That's outside what a public-data API returns, by design.
An account that never engages. If the accounts you're tracking rarely post or interact, there's genuinely little to pull. No vendor manufactures activity that isn't there.
Which endpoint should you use for which job?
| Endpoint | Input | Price | Best for |
|---|---|---|---|
| Instagram Profiles | Username, or keyword search | $0.002 / profile | Follower counts, bio data, account tracking |
| Instagram Posts | Handle, post URL, hashtag, or keyword | $0.002 / post | Content and engagement analysis |
| Instagram Followers | Username | $0.00075 / profile | Sampling audience quality at scale |
FAQ
The first version usually is. The cost shows up later, in the engineering time spent fixing it every time Instagram changes how it renders a page or tightens its rate limits — and that's recurring, not one-time.
No. Instagram Profiles and Instagram Posts both run against public data with no login and no Instagram account required.
When Instagram data is a core part of your own product, you have engineers who will maintain the pipeline indefinitely, and you're pulling several million rows a month. Below that volume, maintenance cost outweighs the per-row API price.
Pay per row returned. $0.002 per profile, $0.002 per post, $0.00075 per follower row. The same rate whether you run it once or on a recurring schedule.