Changelog

When a company URL pointed at a slug LinkedIn itself answers with HTTP 404 (pages move, or a duplicate registration was removed), the endpoint reported a retryable 502 "unavailable. Retry shortly." — but no retry can fix a page that does not exist. LinkedIn's own not-found answer is now surfaced as a definitive 404.

• Dead company slugs now return 404 "Company not found on LinkedIn" with a hint to check the URL, instead of a retryable 502
• The 404 verdict comes only from LinkedIn's own answer for the page — network trouble on our side keeps the retryable wording
• Failed lookups remain unbilled (0 credits), as before

TikTok's public profile page exposes the account's region field for almost nobody, so profile-region often had to fall back to inference — and for accounts with no usable public cues that honestly answered region null with regionSource "inferred" and confidence "low". The endpoint now also reads the creator country TikTok itself attaches to the account's own video feed before resorting to inference.

• Accounts that have videos now get their region from TikTok's own account-country record (regionSource "tiktok") even when the profile page omits it
• Inference remains the fallback for zero-video accounts, unchanged contract: regionSource "inferred" with a confidence grade, null when it genuinely cannot tell
• Responses stay 2 credits; the extra lookup runs only when the profile page carries no region

Passing a facebook.com/share/… link to /v1/facebook/comments or /v1/facebook/comment-replies skipped the share-resolution step that /v1/facebook/details already had, so the extractor read the share interstitial instead of the post — and could answer 200 with zero comments or replies, billed, even when the target was unreadable. Share links are now resolved to the canonical post URL first, and a share we cannot resolve returns the same clear 400 as details, unbilled.

• share/… URLs on both endpoints resolve through the same ladder as facebook/details and the response echoes the canonical post URL
• Unresolvable share links now return 400 UNRESOLVED_SHARE_LINK at 0 credits instead of a billed empty 200
• comment-replies answers "0 replies" only when the parent comment is visible and Facebook itself reports zero replies; an unreadable page falls through to the extended fetcher instead of asserting an empty thread

facebook.com/share/p/… links whose target is a post in a public Facebook group were answered with UNRESOLVED_SHARE_LINK even though the share resolved cleanly — the resolver found the canonical group-permalink URL but our URL classifier did not recognise the /groups/<group>/posts/<id> form and discarded it. Group permalinks are now first-class post URLs.

• share/p/ links to public group posts resolve and serve full details (text, engagement, media) — the two links failing this morning now both return data
• Direct /groups/<group>/posts/<id> and /groups/<group>/permalink/<id> URLs are accepted by facebook/details without needing a share link at all
• share/r and share/v links were unaffected and keep working as before; unresolvable shares still return the same clear 400

Top Ads captures run through country-matched fetch pools, and an unhealthy pool for one country could fail the whole request even though the leaderboard itself was reachable — a burst of 503s this morning came from exactly that. A failed capture now retries once through a known-good pool before giving up (the country you query is controlled by the request itself, so results are unchanged).

• One unhealthy regional pool no longer turns into a customer 503 — verified live: a UK request that failed on the first capture was served on the rescue attempt
• Both attempts share one overall time budget, so worst-case latency stays at today's ceiling (~95s) and typical rescues land in 35-50s
• Failures remain free and retryable; the last-good fallback page (up to 6h) still serves when both attempts miss

Some LinkedIn company pages show no posts to logged-out visitors — often a stub or duplicate registration while the organization actually posts from another page. Those requests used to burn up to 110 seconds across serial fallbacks and then claim 'Retry shortly' for a state a retry cannot fix. The rescue search now runs in parallel with the extended fetcher and respects its time budget, and confirmed-empty pages get an accurate error.

• When the company page carries no public posts, the rescue search starts immediately in parallel instead of last in line — successful rescues and failures both return tens of seconds sooner
• The rescue search's engine sweep now honours one overall time budget (it could previously run double and push requests into the 110s wall), and it no longer spends time fetching other authors' posts that merely mention the company
• New honest error when the page is confirmed to show no public posts: 'LinkedIn is not showing this company's posts to logged-out visitors right now… verify the company URL' — no more 'Retry shortly' when retrying cannot help
• Transient failures keep the retryable wording, and all failures still cost 0 credits

Deep pages served by the extended fetcher were billed the same flat 2 credits as a 20-post native page, which under-priced 200-row answers. Extended serves now scale with what you actually receive; the native path and failures are unchanged.

• Extended-path pricing: 1 credit per 20 returned posts, minimum 2 — a full 200-post page is 10 credits
• Billing follows the posts you received, not the depth we fetched internally — a 20-post page from a deep window still costs 2 credits
• Native serves stay a flat 2 credits, cache hits stay 0, and failures (including timeouts) stay 0 credits

Paging a profile seconds after page 1 could start a second identical extended fetch while the first was still finishing, and profiles with fewer videos than the requested depth were re-fetched on every deep page because the reuse check measured dataset length instead of what the original fetch had asked for.

• A resume that arrives while an equally-deep fetch for the same profile is still in flight now joins that fetch and serves its result — a duplicate is never started while one is running
• Reuse now judges coverage by the original fetch's requested depth: a completed fetch that asked for at least as many rows is a complete answer even when the profile has fewer videos than requested
• Same data, same freshness window (15 minutes), lower cost — nothing about response shape, cursors, or billing changed

Instagram serves some existing profiles as a logged-out “Restricted profile” wall (age- or region-gated) that withholds every data field. All lookup sources came back empty and the endpoint answered “Instagram profile lookup failed. Retry shortly.” — but no retry can pass a login wall.

• The rendered profile page is now checked for Instagram's restricted-profile wall; when it is the reason nothing parsed, the response is 403 profile_restricted with retryable: false and a message explaining the platform limitation
• Real data still wins: the verdict is only returned when every source came back empty — an account that any fetcher can read keeps its normal 200
• Genuinely transient lookup failures keep the retryable 502, confirmed-gone accounts keep the 404, and error responses stay unbilled

Spark/Non-Spark is not on TikTok's public Top Ads data — only the extended fetcher carries it, and that fetcher is currently disabled. Every adFormat request was deterministically unservable, yet it returned a retryable “temporarily unavailable” 503 in ~460ms; one caller retried the same request in bursts for 20 minutes with zero chance of success.

• adFormat requests now return 503 filter_unavailable with retryable: false and a message that says exactly what to do: omit adFormat to get the native leaderboard now
• Still 0 credits, and the gate fires before any fetch or cache machinery — no stale page can dress an unservable filter as served data
• When the extended fetcher is re-enabled, adFormat serves again with no client change; parameter docs updated to state the contract

When the native fetch was rejected at the door and the extended fetcher was parked, the endpoint gave up in ~300ms with a retryable 503 — even when a minutes-old page for the very same parameters was sitting in cache (26 Sep: three 503s while a 27-minute-old page went unused).

• A 5xx miss is now rescued by the same parameters' last-good page from the past 6 hours, labelled stale: true + cached: true with the original fetchedAt, at 0 credits — the leaderboard is a 7/30/180-day aggregate, so a recent page is still the leaderboard
• With no last-good page the honest retryable 503 at 0 credits is unchanged, and validation errors are never masked by stale data
• Same contract as facebook profile-posts / group-posts last-good rescue

A URL that earned the two-source dead-content verdict (or Facebook's own 404) is deterministic: retrying can never succeed. Each repeat still re-ran two native renders plus an extended-fetcher read — 25-32 seconds per call for the same answer (one wrong-slug URL was called three times in three minutes).

• The definitive verdict is remembered service-wide for 6 hours: a repeat of the same URL answers the same non-retryable 404 in under a second, at 0 credits, with no scrape and no extended-fetcher run
• Only definitive verdicts are remembered — a transient retryable 503 always gets a fresh attempt, and a URL that starts working again is never blocked by a stale verdict for long
• Verified there is no honest rescue for a wrong page slug: Facebook resolves a post URL only under its owner's slug — bare post-ID forms answer 404 on every public surface

TikTok's TikTok One migration removed the song chart from the anonymous Creative Center — the music page lost its songs tab and the chart API answers a no-permission error to every anonymous session. Hourly rediscovery probes kept confirming the removal, so instead of answering persistent 503s forever the endpoint is shelved with a definitive 410 Gone.

• Every call now answers 410 Gone (code ENDPOINT_REMOVED, retryable false) at 0 credits — before any scrape or billing, so it cannot drag TikTok platform status down
• The docs page carries a 'Removed 26 Sep 2026' badge with the full reason; the free trending-songs tool explains the removal instead of suggesting a retry
• All chart machinery is kept intact: if TikTok restores the chart, the endpoint will be re-enabled with the same path and parameters
• Hashtag, video, and creator trend endpoints are unaffected; song-details and music-posts still work for a specific sound

A post URL whose page slug does not match the post's owner (or a deleted/restricted post) renders Facebook's 'content isn't available' page — which was classified as a transient miss and answered 503 'retry shortly' forever. Retrying such a URL can never succeed.

• When both native renders serve Facebook's dead-content page and the extended fetcher cannot read the post either, the answer is a non-retryable 404 explaining that the post was deleted, restricted, or the URL's page slug does not match the post author
• summarize now has the same rescue ladder as details: a transient native miss gets one extended-fetcher read before any error verdict
• A login wall never counts as the dead-content verdict, and a single ambiguous render keeps the honest retryable 503

The public timeline embed misses for two very different reasons: a transient outage (retry helps) and an account that has simply never tweeted (retry can never help). Both collapsed into the same 502 'Retry shortly' — an account created in 2016 with zero tweets sent callers into a retry loop that could never succeed.

• On an embed miss the account's public profile is checked (~1s); an explicit zero tweet count returns 200 with tweets: [], truncatedReason no-tweets and an explanatory note — at 0 credits
• Accounts that do have tweets keep the honest retryable 502 when the embed is withheld or the fetch flakes

An existing-but-private Threads account has zero visible posts on its hydrated page, which collapsed into the generic 404 suggesting the handle might be wrong. The page itself carries the account's private flag — that verdict is now read and reported.

• A private account returns 200 with posts: [], isPrivate: true, the author card (username, display name, avatar) and an explanatory note — at 0 credits
• The Instagram bio-link handle remap is skipped for private accounts: the handle is right, so a lookalike swap would be wrong
• Dead handles and public zero-post profiles keep their existing 404 verdict

Group feeds render through a single headless fetch whose yield is inherently variable — the same group polled repeatedly flip-flopped between 200s and 502/503s (26 Sep: three groups, a dozen calls, roughly every third one failing). Failures were honest but useless to a caller who got a full page minutes earlier.

• Successful pages are remembered for 6 hours per group+sort; when a fresh fetch fails or times out, the last good page is served as a stale 200 (stale:true, source cache) at 0 credits — the same contract profile-posts and marketplace-search already have
• A first-ever fetch that fails still returns the honest retryable 5xx — stale pages are only ever real pages we served before

Lookups for handles that do not exist on Truth Social (its own API answers NOT_FOUND 'Record not found') returned a bare 'Truth Social resource not found' — easy to read as an API fault rather than a wrong handle.

• A missing account is now a typed ACCOUNT_NOT_FOUND naming the handle and pointing at the profile URL to verify — handles are exact, and guessed ones (e.g. a politician's 'obvious' username) are the usual cause
• A missing or deleted post is a typed POST_NOT_FOUND with the post id
• Status code and billing are unchanged: 404, retryable:false, 0 credits

TikTok removed the public Creative Center song chart in its TikTok One migration (re-verified 26 Sep: the music page lands on a hub with no songs tab and the chart API answers 40101 anonymously), so the endpoint's honest 503 stands. But the hourly recovery probe formed that verdict from the anonymous session-check call — which answers the same deny code even when chart APIs are public — so a restored chart could never have been detected.

• The probe now inspects the full page traffic: chart rows win immediately (recovery auto-resumes 200s), a deny answered by the chart API itself is a real login gate, a loaded page with no chart call at all is the removed surface, and nothing loading is a transport miss that still tries the extended fallback
• Customer-facing behavior is unchanged during the outage: 503 CREATIVE_CENTER_UNAVAILABLE, retryable:false, 0 credits, one ~20s rediscovery probe per hour with instant replays in between

The earlier newest-first fix covered the extended fetcher path only. The native search leg trusted the search engine's date ordering — but that hint only reaches one of the two engines and concurrent post hydration shuffles the order anyway, so a sort=date page could come back as Sep, Feb, Mar, Jun.

• Both fetch paths now sort the batch by real publish time before slicing: the post's activity-id snowflake first, with the row's publishedAt timestamp as fallback
• Batch cache key bumped so pre-fix unsorted batches cannot serve inconsistent page 2s

The no_shop verdict for ttSeller:false accounts trusted a single empty shelf fetch. Affiliate creators are ttSeller:false by design yet still list showcase products — a transient empty run could brand them no_shop and bill a credit for a wrong answer.

• The profile's own commerce bit now gates the verdict: when TikTok flags the account as a commerce user and the shelf comes back empty, the response is an honest 502 SHOWCASE_UNAVAILABLE with 0 credits instead of a guessed no_shop
• Genuinely non-commerce accounts (both flags false) keep the 200 no_shop verdict at the 1-credit looked-and-empty rate — verified live against a non-seller profile from three independent signals

On the extended path, publishedAt was passed through from the upstream fetcher's own locale-rendered date string — a 25 Sep page showed posts published at 21:45 on a response fetched at 20:06 UTC (+2h skew, no timezone marker). The same pages also arrived as shuffled ascending runs instead of the recency order sort=date promises.

• publishedAt now derives from the row's epoch timestamp rendered in UTC (the post's activity-id snowflake confirms it); the vendor's local string is only a fallback when no timestamp exists
• sort=date pages are now sorted newest-first before slicing — the cursor re-slices the same sorted batch, so page 2 stays consistent
• sort=relevance ordering is untouched — that ranking belongs to the search engine
• timings now include serpMs and hydrateMs on the extended path too: 70 of 74 seconds used to be invisible when the native leg burned its budget before the fallback

Salt Lake City was missing from the Marketplace city directory, so its searches were snapped to the nearest known metro — Las Vegas, 360+ miles away — and the response still echoed the requested location. Results looked scoped to Salt Lake City but were another city's listings (or an empty page when the Vegas search found nothing).

• Salt Lake City now resolves to its own verified Marketplace hub — searches return Utah-metro listings (verified live: Midvale, Ogden, Provo, Draper results)
• Bare city names like 'salt lake city' or 'chicago' now hit the directory directly when the name is unambiguous — no more geocode detour (resolve dropped from ~700ms to ~1ms)
• A real US city with no scopeable hub within 90 miles now returns an honest 422 UNSUPPORTED_LOCATION naming the nearest supported metro, instead of silently answering with another city's listings
• Genuinely empty searches are unchanged: a Facebook-confirmed zero-hit still returns an honest empty page

Three sort=date searches on 25 Sep fell through to the extended fetcher and failed after 60-80 seconds: the public sort value was passed to it raw, but the fetcher's vocabulary is relevance/date_posted, so every fallback died with an invalid-input error. That raw error — naming the vendor and its payload — then surfaced verbatim as a 500.

• The public sort=date value is now mapped to the extended fetcher's date_posted before the call — sort=date searches succeed on both paths (verified live)
• An upstream fetcher failure now returns the honest 502 UPSTREAM_UNAVAILABLE envelope with a generic retry message — the raw vendor payload stays in server logs only
• The request log now records these as 502 UPSTREAM_UNAVAILABLE too; it used to say 500 while the customer actually received a 502, so the dashboard disagreed with the wire
• Failed calls were already unbilled (0 credits) — that behaviour is unchanged

Every first-page request used to start a paid extended fetch at t0 sized at three times the requested page — even when the free native leg was about to answer, and even when an identical fetch had finished seconds earlier. Repeat lookups of the same profile now reuse a recent result set at no extra latency, and the extended fetch only starts if the native leg hasn't answered within its head start.

• A succeeded extended fetch for the same handle from the last 15 minutes serves page 1 directly when its result set covers the requested window — timings gains reuseAgeMs so a reused answer is visible
• Page 1 now fetches limit+1 rows (one leftover row is all hasMore needs) instead of a flat 3x over-fetch; cursor pages keep the deeper window the createTime cutoff requires
• The extended leg waits out a 5s native head start before starting a fetch — a healthy native page lands in 2-5s and no extended fetch is started at all
• Verified live: two back-to-back calls for the same handle produced one extended fetch (second served from reuse), sized 21 rows for limit=20

In one degraded window (25 Sep 11:21-11:36 UTC) 14 of 32 requests 502'd at ~20s: TikTok's mobile API was answering in 30+ seconds (over its 16s leg budget) while the Apify actor's slow tail overran its own 16s cutoff — and the route gave up with ~100 seconds of its 125s wall unused. The same handles served 200s minutes later once the actor finished; the 16s cutoff never kills the run server-side.

• On a full miss the route now rejoins the still-running actor run (up to 45s) or reads one that finished right after the cutoff, instead of paying for a duplicate
• The rescue runs only on the miss path — successful races are exactly as fast as before
• timings gains rescueMs and apifyStatus: 'rescued' so a saved request is visible; a genuine full outage still returns the honest 502

share/r/ links resolve through Facebook's own canonical, which is the videos-slug form (/<page>/videos/<slug>/<id>/). That form renders a watch-style page that the native ladder reads as 'no node' and the Apify rescue as a bare Video node with an id-only owner — so three live requests on one share link served pages with no author name, no caption and no shares, while the /reel/<id>/ form of the same video carried them all.

• A share/r/ canonical in the videos-slug form is rewritten to /reel/<id>/ (the reel id is the slug URL's trailing segment) before fetching
• The rewrite also applies when reading the 30-day resolve cache, so entries poisoned before this fix heal on their next hit — no cache surgery
• Non-reel share links keep whatever canonical they resolved to

When the native render misses and the Apify rescue serves a reel, two fields were being lost in shaping. The scraper stamps the numeric page id as pageName on vanity-less pages — and that id outranked the real name sitting in video_owner.name, so displayName came back as '61584242948204' instead of the page's actual name. And shares arrive only as the compact token ('41K'), which the integer parser silently dropped to null.

• A digits-only pageName no longer masks the real author name (verified against the live actor payload: displayName now 'Whisprs "', not the page id)
• Compact share counts parse like the native ladder's (41K → 41000) with an honest sharesIsApproximate: true; exact integers stay exact and unflagged
• Views stay null on the actor path — the payload genuinely carries no view count, and an honest null beats a guess

25 Sep 09:18 UTC: one more 88.4s 503. The concurrent plain safety net shipped on 24 Sep fired a single shot at t0 — when that one shot flaked (plain misses are per-session random, like the render ones) a stalled grown render left ~70 seconds with no rescue in flight. Its twin case at 24 Sep 16:47 UTC, where the shot landed, came back 200 in 88.5s.

• The safety net now keeps a fresh plain attempt in the air until the request window is spent (capped, so an instant-error upstream cannot run up Decodo spend)
• Small-limit (clicks=0) requests get the same net beside their headless retry pair — previously they had no rescue at all after the primary plain shot missed
• Happy path unchanged: a landed grown render still cancels the net immediately; verified live at 33-49s with zero extra attempts

country=India had never returned a page (six straight 502 fetch_empty at ~78s over 23-24 Sep). Two root causes: the country-level IN proxy pool cannot finish a headless render of instagram.com/reels at all (systematic read timeout at the full wall, 9/9 across prod and probes), and the old serial listing walk let two of those stalls eat the whole fetch window so the Explore page never even got a try. Indian CITY egress pools render the same India-localized page in 15-17 seconds.

• Listing renders now go out as parallel /reels + /explore/reels pairs (two rounds) — one stalled arm can no longer starve the other
• India rides a city-pool geo ladder: Mumbai egress first, the country pool as the second-round retry; hydration reuses whichever egress actually served the listing
• Verified live: India 7 reels in 23s (was a guaranteed ~78s 502); the same pair design also covers the Turkey-style empty-first-render misses

Instagram's login/WAF gate on explore/tags walled off every native tier (four 502 login_wall rows on 24 Sep), and a wall also gates the Decodo rung by design — so the whole ladder died with the session pool. The managed Instagram rung now answers the same hashtag sections feed (top for posts, clips for Reels) when the native listing misses outright. Both hashtag-search and reels-search share the rescue.

• Native listing miss → managed hashtag grid: raw media sections mapped by the exact same code path as a session page
• Media rows arrive complete (author, caption, counts), so the page ships without hopping dead sessions for enrichment
• Verified live with the four failing tags — #trendalert, #reeltrend, #newtrend, #trendingreels: 20 posts each in 14-22s (was 502 login_wall)

LinkedIn lazy-loads SERP creatives: the real media.licdn URL sits in a data-delayed-url attribute while src holds a placeholder ghost. The extractor read only src, so most cards shipped with an empty media array even though the images were on the page (one audited SERP carried 48 delayed URLs against 16 hydrated src). Separately, any card with media was labelled 'Single Image Ad' — including video posters and document pages, whose CDN paths name the creative kind.

• SERP extraction reads data-delayed-url before src — live media fill went from near-zero to 20/20 (microsoft/US) and 22/24 (Forex/DE)
• adFormat derives from the creative URL shape: videocover → Video Ad, ads-document-cover → Document Ad, multiple images → Carousel Ad; the same inference backs /linkedin/ad-details when the page carries no About-the-ad label (LinkedIn's own label still wins)
• advertiser/cta/paidForBy stay null on search rows by design — the SERP card does not carry them; hydrate via /linkedin/ad-details as the platformNote says

After the plain-first ladder shipped, five more 503s appeared with one shape: all on limits > 25, all at exactly ~88s. The grown browser render had stalled to its full 84s wall, leaving 11s — too little for the retry pair (needs 30s) and too little for the serial plain rescue (a plain fetch takes 3-12s, it got 8). The plain fetch now starts at t0 beside the browser render, so its first ~25 cards are in hand by ~15s no matter what the render does.

• Grown requests (limit > 25) start a cheap no-JS fetch in parallel with the browser render; it is discarded when the render lands
• When every browser render misses, the pre-fetched plain page ships the first ~25 cards with an honest hasMore=true instead of a 503
• Verified live against the exact failing shape: Forex/DE limit=48 with all three browser renders down — 24 ads in 83.5s (was a guaranteed 88s 503)

The LinkedIn Ad Library search page turns out to be fully server-rendered: a plain proxy fetch returns the first ~25 ads in 3-12 seconds, while the headless browser render we used for every request took 25-80s and carried both remaining failure modes (degraded-pool session misses and the stall that eats the retry window — two more 503s on 24 Sep even after the parallel retry). The browser is now only used to grow the page past 25 cards.

• limit ≤ 25 (the default): plain fetch first, ~3-10s typical; the parallel headless pair only runs if it misses — three independent chances, no serial-stall gap
• limit > 25: unchanged grown ladder, plus a new last-resort plain fetch that ships the first ~25 cards as an honest partial page (hasMore=true) instead of a 503
• A partial rescue that can't reach a requested offset window stays a retryable miss — never a fake isLastPage
• Verified live: 20 ads in 6.6s, 12 in 9.1s, zero-hit in 2.3s, grown 40-ad page in 30.8s

When our Instagram session cookies die (roughly weekly), the timeline feed 401s on every proxy tier, the tagged feed hits a login wall and the highlights tray rate-limits — a public profile answered 502 on the next feed page, a 90s timeout 503 on tagged posts, and a definitive-sounding 422 on highlights that actually had six live albums. All three surfaces now fall back to the same managed Instagram pool that already rescues channel-reels and comments, returning identical raw payloads through the existing mappers.

• Timeline feed pages (channel-posts pagination), the tagged feed and the highlights tray each gained the managed rung; public cursors pass through unchanged
• The tagged-posts native tier walk is time-boxed (dead sticky sessions burned ~2m45s of redirect loops — past the route deadline before any fallback could run)
• Tagged-posts profile resolve skips the session pool (same reasoning as highlights: dead-pool redirect loops burned ~72s of the 90s deadline)
• Author-stats enrichment is budgeted — when the pool is dead you get lean posts instead of a 503
• Verified live on the reported profile: next feed page 12 posts, 15 tagged posts, all 6 highlight albums with titles

A photo permalink 400'd as an invalid URL: the classifier knew permalink.php and story.php but not photo.php?fbid=, /photo/?fbid= or /page/photos/set/id — and the details parser only looked for Story nodes while photo pages embed a Photo node instead. Both layers now understand photos: the same single render serves caption, author, reactions, comments and the full-size image, and the feedbackId works with /v1/facebook/comments.

• All three photo permalink forms classify as posts and pass the details gate
• The Photo node's container story supplies the caption, author and canonical post URL — no second render needed
• A photo id that doesn't match any node on the page stays an honest retryable miss, never another photo's data
• Bonus: share links that land on photo URLs now resolve instead of failing as unresolved

During a degraded proxy-pool window (24 Sep 10:55–11:01 UTC) LinkedIn search answered three ~80s 503s interleaved with 200s of the very same query — the misses are per-session random, not systemic, yet the ladder retried with a single session and one more unlucky draw ended the request. The retry round now renders through two parallel sessions and serves whichever paints the library, cutting the joint-miss rate quadratically. The extra render is only spent when the first attempt already missed.

• Verified live: a first-attempt miss on a heavy advertiser recovered via the parallel pair — 20 ads in 87.5s where the single-session retry had answered nothing
• When both arms paint, the one with more ads wins; failures remain 0-credit and retryable
• Residual: a first attempt that stalls to its full 80s wall still can't fit a retry inside the 95s deadline — that rare case stays an honest retryable 503

When the first-page render missed, the pagination rescue could capture TikTok's own definitive answer — code 0, zero ads — and still throw it away, because zero rescued rows never beat an empty pool. Five zero-hit queries burned ~82 seconds each and answered 503 upstream_unavailable in one afternoon, while identical queries minutes later answered an honest empty 200 in 15 seconds. A code-0 empty answer from either rescue arm is now always a 200. The rescue round also runs both capture modes in parallel (they fail independently), so a 503 now requires three independent capture misses instead of two sequential ones.

• Verified live on all five failing queries: four zero-hits answered empty 200 in 13–19s, the fifth served 24 rows — none returned 503
• Rescue page-1 rows ship with honest hasMore instead of being dropped when pagination misses
• The extra parallel render costs only on the miss path; the direct JSON API stays fenced off (TikTok's per-request x-ccl-str signature answers 421 to every replay — verified across datacenter, residential, and direct egress)

Some videos sign audio-only stream URLs that answer 403 on every egress, while the combined audio+video stream from the very same player response downloads fine. The extractor treated the 403 as a dead end, burned its whole 20-second budget walking fallback clients that couldn't help, and answered 502 audio_extract_failed — the yt-dlp rescue never got a turn. The native rung now falls back to the muxed stream (video is stripped during the speech re-encode), the extract budget grew to 45s sized against measured ASR times under the 110s route deadline, and the fallback rungs must leave the yt-dlp rescue a reserved slice instead of starving it.

• Verified live on the failing video: audio-only 403 → muxed stream 200 in 2s, full 77s of audio transcribed end-to-end in 22s
• Interrupted yt-dlp downloads (.part files) can no longer be served as if they were the full audio
• Failures remain 0-credit; the 502 now only fires when every rung — including yt-dlp — actually ran and missed

After the tab-first ladder reorder, both light hops (first-paint home, then the /posts tab) ran under the same 22s deadline. Production showed renders of heavy pages like nasa spiking past 22s on both hops in the same request (listing phase 44s), landing a 503 that the failure marker then replayed fast. The home hop now gets 20s, the /posts tab hop gets its own 28s window sized to what's left of the budget, and old failure markers written before the wider window are versioned away.

• Home first-paint 20s + /posts tab 28s fits the 55s listing budget with margin for the details fetch
• Failure-marker cache key bumped so pre-fix fast-503s can't outlive the fix
• Deadline arithmetic is unit-tested against the observed 12–19s (calm) and 22s+ (spiky) render times

TikTok's One Creative Suite migration removed the public Creative Center song chart: the music page now redirects to a trends hub with only hashtag/video/creator tabs, the chart API answers 40101 no-permission to anonymous callers, and the extended actor scraping the same surface returns nothing (all verified 24 Sep). The endpoint already refused to substitute For You sounds; it now names the removal explicitly and remembers the outage for an hour instead of five minutes, so at most one 20–30s rediscovery probe runs per hour while auto-recovery stays in place should TikTok restore the chart.

• 503 CREATIVE_CENTER_UNAVAILABLE message now explains the chart was removed in the TikTok One migration (retryable: false)
• Persistent negative cache extended 5min → 1h — prod had burned eight 20–34s scrapes across 22–24 Sep rediscovering the same removal
• Last-good chart pages still serve as stale 200s when cached; all failures remain 0-credit

A stalled proxy render can run ~25s past its granted budget (read drain + wall cap). The retry attempt granted up to 55s whenever 30s remained on the clock, so a shell-page miss followed by a stalled retry walked one request to 113s against the 95s deadline before answering 503. Every attempt's grant now subtracts the stall overshoot from the time remaining, so the ladder mathematically cannot exceed its deadline.

• Root cause from prod 24 Sep: a 251-byte proxy shell at ~33s, then a full 55s retry grant whose stall burned 80s
• Attempts too small to fit a render plus its overshoot are skipped instead of started
• Same two-attempt retry contract — one blip still never 503s a search

Asking for more than 12 TikTok ads ran two 35-second first-page render attempts; when the proxy pool was slow, each stall overran its grant and together they ate the whole budget, then the route declared the library down (503) even though the pagination capture path was healthy in the same minute. The ladder now spends at most one tight ~15s probe on the first page and, when that misses, hands the remaining budget to the pagination capture as a rescue instead of giving up.

• Verified live: a first-page miss now recovers via pagination — 24 rows in 44s where the old ladder 503'd after 72s
• Zero-hit answers from TikTok's own API are still an honest empty 200, never a rescue
• Failures remain 0-credit and retryable

The episode archive interleaves RestrictedContent stubs (geo-blocked episodes) among real rows — Joe Rogan's page 1 opens with one. The stubs are unusable and get dropped, which shorted the page (limit=3 → 2 rows), and the offset cursor advanced by rows returned rather than archive entries consumed, so the next page re-fetched — and could re-serve — rows around the stub. Pages now backfill from the next archive slice to fill the requested limit, and cursors advance by entries actually consumed.

• Dropped RestrictedContent stubs are backfilled — limit=3 returns 3 episodes again
• nextCursor advances by archive entries consumed: no duplicates, no skipped episodes across pages
• Verified live on JRE: pages 1–2 are consecutive (#2557…#2552) with zero overlap
• Response-cache version bumped so short cached pages cannot serve

Both endpoints rode a single scrape-pool attempt with a tight timeout. The pool answers reddit JSON in 2–5s normally, but an individual request occasionally hangs well past any sane cap — that lone coin-flip turned into 503s on /search (the 6s attempt died) and 502s on /post-comments (the 15s comments.json fetch ReadTimed-out on a live thread). Both paths now retry once with a fresh request, which reliably lands.

• search: two scrape attempts budgeted inside a raised deadline (12s → 25s); direct-fetch rungs unchanged
• post-comments: comments.json fetch retries once (15s + 12s) after the authoritative existence probe
• Verified locally 3/3 on both paths against the previously failing thread and query

A single degraded page render that resolved no reel ids, combined with an actor miss, was treated as proof that the profile has no public reels — a 200 with noPublicReels: true and zero rows. Absence-of-evidence is not evidence: the affirmative 200 now requires either the actor's explicit empty answer or a two-render native confirmation. Anything less stays a retryable 5xx at 0 credits.

• noPublicReels: true requires affirmative evidence (actor-confirmed empty or double native confirm)
• Degraded single-render zero-id results no longer masquerade as an empty profile

When the native listing missed, a web-profile-info timeline fallback served whatever videos sat in the profile's ~12-post timeline edge — sometimes just 2 — and canceled the richer scrape/actor race that was still running. A limit=25 ask got 2 rows and a posts-style cursor. The thin timeline result is now held as a last-resort partial instead of served eagerly: the full ladder keeps running, and the thin set ships (flagged partial) only if every richer rung misses.

• Timeline fallback serves immediately only when it can fill min(limit, 12) rows
• Thinner sets are kept as a partial-of-last-resort while the scrape/actor race finishes
• Served thin sets are flagged partial: true with a wpi_thin stage marker
• Response-cache version bumped so cached 2-row pages cannot serve

The listing ladder tried a scrolled home-page render as its second hop. On heavy pages that render mostly dies in a ~40s ReadTimeout and eats the whole 55s budget, so the third hop — a light /posts tab render that actually paints post permalinks in ~19s — never ran. nasa returned 503 after ~53s on every attempt, and the failure marker then served fast 503s for the cache window. The /posts tab hop now runs second, and the light-hop deadline was raised from 16s to 22s to cover measured 12–19s first paints.

• Hop order is now: first-paint home → light /posts tab → scrolled home (last, only if time remains)
• Light-hop deadline 16s → 22s — real first paints of heavy pages were being cut off mid-render
• Verified live: nasa now answers 200 with hydrated post data in ~20s instead of 503 at 53s
• Response-cache and failure-marker versions bumped so pre-fix 503 markers cannot fast-serve

Comments beyond Instagram's 2-row logged-out preview depended on our own session-cookie pool, whose cookies die roughly weekly — when the pool was down, every limit>2 request answered 503 SESSION_UNAVAILABLE. A managed logged-in rung (HikerAPI /v2/media/comments) now pages the same raw comment shape with no cookies of ours: ~15 rows per hop with a real resume cursor. The session pool still runs first when alive; the 503 remains only for the rare case where every rung misses.

• Full comment threads page again during session-pool outages — verified live: 15 rows + working second page on an 820-comment post
• New cursor form {mediaPk}:h:{pageId} so managed pagination can never be mistaken for a session resume token
• Existing session cursors are honored by the pool only — the managed rung won't restart page 1 under a foreign cursor (no duplicates)
• timings.hikerMs exposes the new rung; failures stay 0-credit and retryable

Facebook's modern profile URLs come in a name+id pseudo-slug form (facebook.com/Amber-Daniels-6159…) alongside the plain numeric form (facebook.com/6159…). The listing scraper treated the pseudo-slug as a vanity slug: it fetched /{pseudo-slug}/reels — a degraded page — and skipped the numeric /videos permalink mining entirely. The same profile that listed 21 reel ids via its numeric URL returned a single reel via its pseudo-slug URL. Both URL forms (and /people/Name/id/ URLs, whose display-name segment had the same problem) now resolve to the numeric profile id before listing.

• Name+id pseudo-slugs (…-61590755182701) and /people/Name/id/ URLs resolve to the numeric id for /reels and /videos listings
• Numeric permalink mining now runs for these profiles, so the full recent grid is found
• Applies to both /v1/facebook/profile-reels and /v1/facebook/profile-posts; response-cache versions bumped so stale 1-item rows cannot serve
• Vanity slugs (nasa, blink-182) are untouched — only a trailing 10+ digit id run triggers the rewrite

Two deeper causes behind the remaining upstream_unavailable bursts. First, the HTTP client's read timeout never bounded total call duration: when the scrape provider kept the connection warm while its headless browser ground on, a 40s-grant pagination attempt ran 87 seconds and ate the entire request budget — every rung behind it starved. Every scrape call now carries a hard wall-clock cap, so a time grant means what it says. Second, the ladder opened with the flaky pagination rung even though most polled queries answer empty or with a single page. The ladder is now page-1-first: the reliable capture rung answers zero-hits in ~12s, pagination only runs when page 1 proves there are more ads, and a pagination miss ships the page-1 pool as an honest partial (truncated: true, hasMore: true) instead of a 503.

• Hard asyncio wall cap on every Decodo scrape call — a trickling connection can no longer run 2x past its grant
• limit>12 searches fetch page 1 first: zero-hit queries answer in ~12s instead of burning 40–90s of pagination
• Pagination failure degrades to the page-1 pool with truncated/hasMore flags — no more 503 when TikTok answered
• Fast-path (limit≤12) grants bounded to 35s so a degraded window cannot eat the retry's budget

With limit above 12 the search ladder ran two flaky Decodo paginate attempts back-to-back (~40s + ~26s of a 87s budget) before falling back to the fetch_resource rung that powers the fast path — leaving it an ~18s window that rarely landed. During degraded Decodo windows (21–23 Sep) identical queries flipped between 200s at 56–70s and 503 upstream_unavailable at ~90s, purely on the coin-flip of the second paginate attempt. The second paginate attempt now yields whenever it cannot leave the fallback a real (≥35s) window, so the rung that actually answers gets the budget instead.

• Second xhr-paginate attempt skipped when it would starve the fetch_resource fallback below ~35s
• Fallback still returns an honest partial pool (has_more/truncated flags) when pagination fails
• 503 upstream_unavailable remains retryable with 0 credits charged — but should now be rare instead of hourly

profileImage on profile-region is a signed tiktokcdn URL whose x-expires stamp lands ~2–3 days after fetch. Every other TikTok profile/video surface ships mediaUrlsExpireAt next to signed links; this endpoint shipped the dying URL with no warning, so a client caching the avatar past the stamp got a dead image with no way to know when it broke. The route now parses the x-expires stamp into mediaUrlsExpireAt, and the response-cache version is bumped so pre-fix rows without the key cannot serve.

• mediaUrlsExpireAt parsed from profileImage's x-expires (null when the URL carries no stamp)
• Cache version v9 → v10 — cached rows from before the fix are recomputed, not served without the key
• regionSource/regionConfidence semantics unchanged: tiktok = authoritative (confidence null), inferred = estimate with its grade

LinkedIn's experience location line mixes geography with the workplace arrangement — "Gurugram, Haryana, India · On-site", or just "On-site" when the member set no city. The profile endpoint passed that line through as location, so a live profile shipped an experience row whose "location" was On-site. Arrangement labels are now split out: experience rows carry location (geography only, absent when the member set none) and workplaceType (On-site / Remote / Hybrid, only when LinkedIn shows one).

• experience[].workplaceType: "On-site" | "Remote" | "Hybrid" — split from the location line, absent when LinkedIn shows none
• experience[].location keeps geography only; a bare arrangement label no longer masquerades as a place
• Combined lines ("City, State, Country · Hybrid") map to both fields

TikTok trims the seller's stray whitespace on each SKU's property pair but ships saleProperties with it intact — a live US phone-case PDP returned "iPhone 11 Pro Max " / "iPhone X " / "iPhone XR " / "iPhone XS " in the variant axis while every SKU said the trimmed name, so a client joining SKUs to the axis by value name silently lost 4 of 32 models. Both sides now trim names and values; whitespace-only axis values are dropped.

• skus[].saleProps propName/propValue and saleProperties[].name/values[].name are whitespace-trimmed to the same form
• Verified live on the reporting PDP: 32 axis values ↔ 32 distinct SKU values, zero unjoinable names
• IDs, prices, stock and everything else pass through unchanged

The 13:36–13:49 UTC wall burst walked 15 shortcodes through a fully walled ladder (GraphQL, embed and Decodo all gated) at 41–54s each. The Apify actor's not_found verdict shipped one false 404 for a live, public, verified account's carousel (hbro__), and four genuinely deleted posts spent the full deadline to die as retryable 502 fetch_exhausted — inviting pointless retries at dead URLs. HikerAPI's logged-in media/by/code answered all 15 correctly in ~2s, so it is now a rung in the ladder: media in hand is a 200, Hiker's own 404 is a definitive death certificate (404 not_found, no Apify wait), and a Hiker miss changes nothing.

• New rung after embed: Hiker v2 media/by/code (raw api/v1 envelope — v1's flattened shape made carousels look hollow)
• Hiker media → 200 via the same map_feed_post contract; false-404 case CjaD8JErmKH now returns the post in ~16s
• Hiker 404 → non-retryable 404 not_found, Apify skipped (skipped_hiker_not_found); dead posts stop 502ing as retryable
• stages.hikerStatus / ms.hiker in error traces; Decodo still runs first so private/deleted keep their precise labels

tiktok/popular-songs 503'd CREATIVE_CENTER_UNAVAILABLE again today. Two separate causes: TikTok's Creative Center login gate (up since Sep 4) still blocks the song chart on every path — native page render, direct rank_list API (40101), and the extended actor all hit the same wall, so the honest 503 stands. But the probe also caught a real regression: Decodo removed the legacy {type: wait, timeout: N} browser action, so the shared Creative Center xhr capture (hashtag / video / creator charts) failed validation with a 400 before ever reaching TikTok. The capture now sends the current wait_time_s schema. Failure streaks for the song chart also survive sparse probes: a 42h gap used to reset consecutiveFailures to 1 and re-open “Retry shortly” for a chart dead for 18 days.

• Creative Center xhr captures send {type: wait, wait_time_s: N} — the legacy timeout key 400'd at Decodo validation since ~mid-September
• popular-songs failure streak TTL 24h → 7 days: sparse daily probes keep consecutiveFailures and persistent=true honest; any 200 still clears the streak instantly
• song chart login gate re-verified 2026-09-22: anonymous page renders no chart (user/info answers 40000), direct rank_list answers 40101, extended actor returns 0 rows

search-companies → company-ads for 28 Day Lymphatic Reset Plan returned 20 near-identical inactive vagus-nerve variants (first row hollow, no hasMore) while Meta's live page was showing newer active creatives. company-ads always requested active_status=all, sliced the first HTML batch to limit, and omitted the search envelope — so 20 historical collation siblings looked like the whole library. Default is now ACTIVE (same as /facebook/search); status=ALL keeps the archive. nextCursor pages the fetched HTML batch; hasMore is true only when a cursor is present.

• status=ACTIVE (default) / INACTIVE / ALL — Meta URL + local filter, so leaked inactive collation siblings are dropped
• hasMore / nextCursor / retrievableCount / searchResultsCount match /facebook/search
• libraryUrl from search-companies (active_status=all) is rewritten to the requested filter

A nightly Turkey scrape 502'd fetch_empty after six successful native runs. The first /reels render came back empty and a shared 20s budget never tried again or reached /explore/reels; yesterday's grid had also aged out of the 6h last-good window. Each listing page now has its own timeout with one /reels retry, last-good lasts 26h (same as channel-reels), and the 502 body is flat (code/country/message) instead of error.error.

• Empty first /reels render retries once, then /explore/reels, instead of immediately 502ing
• Last-good snapshot is kept 26h so a once-a-night miss serves yesterday's grid as stale
• 502 envelope is {code, country, message, retryable} — no nested error.error

Nightly polls of @vogue and @theverge 502'd UPSTREAM_UNAVAILABLE / fetch_exhausted for a week. Those handles are no longer the magazine accounts: @vogue is a private stub (Vogue Covers, 0 followers, 1 post) and @theverge is a public profile with 0 posts. A live profile whose Reels tab is empty or locked is a successful read, not a server failure — channel-reels now returns 200 with an empty list, noPublicReels: true, and isPrivate when the account is locked. Deleted handles still 404 PROFILE_NOT_FOUND.

• Private profiles → 200, reels: [], noPublicReels: true, isPrivate: true
• Public profiles with zero posts or an actor no_items row → 200 empty, not a retryable 502
• Deleted / renamed handles stay 404 PROFILE_NOT_FOUND (retryable: false)

Facebook's logged-out pages sometimes serve a degraded snapshot where the Reels grid or post payload never hydrates — profile-reels answered false NO_PUBLIC_REELS for pages with public reels, and details returned FACEBOOK_UNAVAILABLE for live watch URLs. Both endpoints now fall back to a managed scraping pool when the direct read misses: profile-reels lists the same page in seconds instead of burning ~22s re-confirming an empty render, and details rescues post, reel, and watch URLs after its two direct attempts.

• profile-reels: false NO_PUBLIC_REELS on reel-having pages is rescued by the fallback pool
• details: watch/post/reel URLs that fail both direct renders get a second chance instead of a 503
• profile-reels: a page with no publicly visible Reels now answers 200 with an empty list, noPublicReels: true, and a note — not a 5xx (it is not a server failure)
• The empty verdict is double-confirmed (direct render + fallback pool) before being claimed
• Rescued responses may report rounded view counts (viewsIsApproximate: true)

A live sweep of all Pinterest endpoints with valid and deliberately wrong inputs found two honesty gaps: a nonexistent pin id answered a billed 200 with an empty card, and pin URLs passed to profile/board endpoints were scraped for ~20 seconds as username 'pin' before failing. Valid pin-details, user-pins, user-boards, board and search paths (including cursor paging) already filled correctly.

• pinterest/pin-details: a pin id Pinterest does not have is now 404 Pin not found (was a billed 200 carrying only platform/id/url and a synthesized imageOriginal)
• pinterest/pin-details: a profile URL is 400 WRONG_RESOURCE_TYPE suggesting /v1/pinterest/user-pins (was a generic BAD_REQUEST)
• pinterest/user-pins + /board: a /pin/ URL is rejected instantly with 400 WRONG_RESOURCE_TYPE suggesting /v1/pinterest/pin-details (was ~20s of scraping username 'pin' ending in 404)

A live sweep of the Amazon seller storefront endpoint with valid and deliberately wrong inputs confirmed products, seller cards, cursor paging and metadata-only calls fill correctly, and dead sellers, influencer /shop/ URLs, product URLs and garbage are definitive 400/404s. One gap: an unknown marketplace code silently scraped amazon.com while the response stayed stamped with the junk code.

• amazon-shop/page: marketplace codes outside the supported set (US, UK, DE, FR, IT, ES, CA, JP, IN, AU, MX, BR) are now 400 INVALID_MARKETPLACE instead of silently answering US data stamped marketplace=ZZ

A live sweep of all Truth Social endpoints with valid and deliberately wrong inputs confirmed profile, post and cursor-paged user-posts fill correctly on the native path, dead handles and dead post ids are definitive 404s, and cross-platform URLs are rejected with WRONG_RESOURCE_TYPE. One gap: a Truth Social profile URL passed to /post answered a generic BAD_REQUEST.

• truth-social/post: a profile URL (truthsocial.com/@user) is now 400 WRONG_RESOURCE_TYPE with suggestedEndpoint /v1/truth-social/profile instead of a generic BAD_REQUEST
• URL classifier now recognises Truth Social posts and profiles, so detected{} blocks and endpoint suggestions include truthsocial.com URLs on every platform's endpoints

A live sweep of all Rumble endpoints with valid and deliberately wrong inputs found two honesty gaps: a channel that does not exist answered 200 with an empty video list, and passing a channel URL to a video endpoint (or vice versa) returned a generic BAD_REQUEST. Valid video-details, transcript, comments, channel-videos and search paths already filled correctly.

• rumble/channel-videos: a channel Rumble 404s is now 404 Channel not found (was a billed 200 with videos: [])
• rumble/comments: a video page Rumble 404s is now 404 Video not found instead of an empty 200
• rumble video endpoints: a Rumble channel URL is 400 WRONG_RESOURCE_TYPE suggesting /v1/rumble/channel-videos (and a video URL on channel-videos suggests /v1/rumble/video-details) instead of a generic BAD_REQUEST

A 27-call live sweep of the LinkedIn surface found four issues. post-details/post-transcript accepted /in/ and /company/ URLs, spent up to 70s scraping, then answered an all-null 200 built from the Apify actor's 'No post found or wrong input' stub — wrong resource types are now WRONG_RESOURCE_TYPE 400s before any scrape, and hollow actor stubs are 404s. Ad Library ad-details answered 503 retryable for ids that are not ads (LinkedIn serves a hydrated 'Ad Details' page without an ad body) — that page is now a definitive 404 that skips the Apify fallback. company-posts 502'd whenever native guest embeds came back empty: the 85s native budget left the Apify actor (needs ~60-80s) at most 17s, so every native miss was a guaranteed 502 — budgets are rebalanced (native 35s; an empty batch hands the actor the whole remaining window).

• linkedin/post-details + post-transcript: /in/ and /company/ URLs are WRONG_RESOURCE_TYPE 400s with suggestedEndpoint (were all-null 200s after a 70s scrape)
• linkedin/post-details + post-transcript: deleted/nonexistent posts are 404s (Apify 'No post found' stubs no longer normalize into null payloads)
• ad-library/linkedin/ad-details: ids that are not ads are a definitive 404 (hydrated detail page without an 'About the ad' body) instead of a 50s 503
• linkedin/company-posts: native budget 85s -> 35s and the Apify stage gets the full remaining window when nothing was collected — native misses no longer guarantee a 502

A 37-call live sweep of the YouTube surface found two honesty bugs. /comments answered 502 retryable on videos whose uploader turned comments off — it now returns 200 with commentsDisabled: true and an empty list. /shorts/* accepted any ≤3-minute long-form video because details were fetched through a pre-canonicalized /shorts/ URL, which stamped isShort on everything; the gate now asks YouTube directly (youtube.com/shorts/<id> 303-redirects to /watch for long-form) with no added latency, so 61–180s genuine Shorts shared as watch links are accepted and long-form pretenders are 422 before billing. All other valid and invalid-input paths (channel, videos, streams, playlists, community, search, hashtag, transcript, sponsors) already answered correctly.

• youtube/comments: uploader-disabled comments are a definitive 200 {commentsDisabled: true, comments: []} — no more 502 retryable loop
• youtube/comments: commentsDisabled: false is now always keyed on normal pages
• youtube/shorts/*: the Shorts gate is decided by YouTube's own /shorts/<id> routing, probed concurrently with details (no extra latency)
• youtube/shorts/*: 61–180s genuine Shorts pasted as watch?v= links are now accepted; ≤3-min long-form videos no longer masquerade as Shorts

A live sweep of Instagram social endpoints found two honesty bugs on /highlights: a YouTube URL 400'd as a missing url/userId instead of WRONG_RESOURCE_TYPE, and url=nope walked Decodo for 72s then answered 503 retryable. Valid channel, posts, details, comments preview, transcript, embed, hashtag and profile-search paths already answered correctly.

• instagram/highlights: a cross-platform URL is WRONG_RESOURCE_TYPE before handle extract (youtube.com/@nasa used to 400 as 'Provide url or userId')
• instagram/highlights: a handle Instagram positively does not have is 404 without the Decodo tray walk (url=nope was a 72s 503 retryable)

A live sweep of TikTok social endpoints with valid and deliberately wrong inputs found two honesty bugs: live?url=nope answered 200 isLive:false as if the account existed, and search-suggestions country=XX echoed the junk code on a real suggestion list. Valid channel, posts, video, comments, transcript, search and followers paths already filled correctly.

• tiktok/live + live-info: a handle TikTok positively does not have is 404 Creator not found (was a billed 200 isLive:false with an empty creator)
• tiktok/search-suggestions, popular-creators, popular-hashtags, trending-feed: non-ISO country codes are 400 INVALID_COUNTRY instead of a global result stamped country=XX
• tiktok/song-details: a profile URL is WRONG_RESOURCE_TYPE (expected music) instead of a generic BAD_REQUEST

A live sweep of every Facebook Ad Library endpoint with valid and deliberately wrong inputs found two honesty bugs: junk ad ids and page URLs answered 503 retryable in under a second, and a garbage company-ads url silently became a keyword search that returned 20 unrelated ads. Valid search, company-ads, ad-details and transcript paths already filled to the requested limit.

• ad-library/facebook/ad-details + ad-transcript: garbage ('nope'), short ids ('1') and facebook.com page URLs are a definitive 400 before any fetch (was 503 upstream_unavailable / retry shortly)
• ad-library/facebook/company-ads: inputs that are not a pageId, libraryUrl with view_all_page_id, or a facebook.com page URL are 400 BAD_REQUEST (url='not a page' used to return 20 keyword-search ads)
• ISO country, q length, status/media/date/limit validation already 400/422 as documented; cross-platform URLs stay WRONG_RESOURCE_TYPE

Logged-out Instagram still cannot page a 500-comment thread or resolve highlight albums from profile-HTML ids. Two caller-facing lies around that limit are fixed: asking for the 2-comment preview no longer 503s, and an ids-only highlight shelf is no longer a retryable 503.

• instagram/comments: limit<=2 returns the logged-out Polaroid as 200 with truncatedReason (was 503 SESSION_UNAVAILABLE after a session walk when Instagram embedded 1 of 509)
• instagram/comments: limit>2 of a larger thread is still 503 SESSION_UNAVAILABLE at 0 credits — that window still needs a live session
• instagram/highlights: HTML-only ids that do not resolve on highlights-details are 422 ids_unresolvable, retryable:false (was 503 retry-in-30s). Retrying does not make those ids work

A live sweep of every Reddit and Twitter endpoint with valid listings plus deliberately wrong inputs. Feeds, search, pagination and most validation were already honest. Three caller-facing bugs are fixed: common share URLs were 400, and a junk community id answered 200 with an empty card after a 21s Apify walk.

• reddit/post-details, post-comments, post-transcript: redd.it/ID and reddit.com/comments/ID are accepted (were 400 Invalid Reddit URL)
• twitter/tweet-details and transcript: x.com/i/web/status/ID and x.com/i/status/ID share forms are accepted (were 400)
• twitter/community: ids under 10 digits are 400 before scrape; a nameless Apify stub is 404 Community not found (was 200 {platform,id,url} in 21s)
• twitter/community-tweets: a tweet/status URL is WRONG_RESOURCE_TYPE pointing at tweet-details

A live sweep of Threads and Spotify with valid listings plus deliberately wrong inputs. Spotify was already honest (wrong-kind URLs 400, dead artist 404, search fills to the requested limit). Threads post-details 404'd a URL we had just listed — Meta no longer puts the target post inside thread_items — and garbage handles were scraped then 404'd instead of rejected.

• threads/post-details: permalink HTML still embeds the post; the parser now walks out from "code" past nested caption objects instead of requiring thread_items (those arrays are suggested posts)
• threads/profile and user-posts: handles with spaces or other illegal characters are 400 INVALID_HANDLE before any scrape (was 404 after a 4s fetch)
• Spotify artist/track/album/podcast/search: no fill or validation bugs on this sweep

A live sweep of TikTok Shop, TikTok/LinkedIn/Google Ad Library found two places where we already had the ads and then threw them away, plus a shop catalog URL that silently returned zero products. Literal keyword hits still lead; the rest of the already-fetched page fills behind them up to limit.

• ad-library/tiktok/search: literal hits first, already-fetched SERP ads fill up to limit (rankedFill + matchedFrom [])
• ad-library/linkedin/search-ads: keyword no longer drops scraped cards — they fill the limit behind literal hits
• tiktok-shop/shop-products: slug-only /shop/store/{name} is a definitive 400 (needs numeric shop id)

A live sweep of the TikTok Shop and TikTok/LinkedIn/Google ad library endpoints with deliberately wrong inputs found several cases where a definitive answer was disguised as a retryable 503. Limit fill was also audited: every listing endpoint returns exactly the requested count when upstream has the ads.

• ad-library/tiktok/ad-details: an id TikTok's library positively reports as nonexistent is now 404 AD_NOT_FOUND (was a 42s fallback walk ending in 503 retryable)
• ad-library/google/ad-details: a creative absent under the advertiser is 404 AD_NOT_FOUND when upstream answered (503 stays reserved for transport failures)
• ad-library/google company-ads + advertiser-search: unknown brands return the documented resolution: not-in-library 200 at 0 credits (Google's empty-suggestions reply was misread as an outage → 503)
• ad-library tiktok/linkedin ad-details: garbage ids like 'notanumber' are a definitive 400 before any fetch
• ISO 3166 country validation extended to linkedin/search-ads (each code in countries), google company-ads/ad-details/advertiser-search and tiktok/top-ads — non-ISO codes are 400 INVALID_COUNTRY instead of silently empty/global results
• ad-library/google/company-ads: limits above 40 now page Google's RPC until the limit is met (limit=50 used to return 40 with hasMore:true)
• ad-library/linkedin/search-ads: limits above one SERP page (~25 cards) pull further pages automatically within the time budget, and the returned paginationToken always follows the last fetched page

A live sweep of all 20 Facebook endpoints with deliberately wrong inputs surfaced four behaviour bugs, all fixed. Caller errors that used to look transient (retryable 502s, silently 'working' filters) are now definitive 400s with actionable messages, and profile-reels no longer declares NO_PUBLIC_REELS off a single degraded render.

• facebook/group-posts with a profile/page URL: definitive 400 pointing to profile-posts (was 502 retryable:true)
• facebook/marketplace-search with an unresolvable place name: 400 UNKNOWN_LOCATION before any scrape (was 200 with listings from an unrelated metro)
• ad-library/facebook search, company-ads, search-companies: non-ISO country codes are 400 INVALID_COUNTRY (Meta used to silently search globally while echoing the junk code)
• facebook/profile-reels: a second confirmation render precedes the NO_PUBLIC_REELS verdict — degraded snapshots no longer masquerade as 'no public Reels'
• facebook/comments: a page/profile URL returning the newest post's comments is now documented

A keyword with one whole-word hit used to return just that one ad, while a keyword with zero hits returned TikTok's full keyword-ranked page — the better your match, the fewer ads you got. Literal hits now come first and the rest of the keyword-ranked leaderboard page fills in behind them up to limit. Per-ad matchedFrom separates the two ([] = ranked fill) and the envelope gains rankedFill next to literalMatches.

• Literal whole-word hits ordered first; keyword-ranked fill follows up to limit
• matchedFrom is always stamped when q is set — [] marks ranked fill
• New envelope key rankedFill (null when q is omitted)

TikTok's Commercial Content Library exists to satisfy the EU Digital Services Act — it only covers the EU/EEA, the UK and Switzerland (32 countries). Searches with any other country (SA, US, AE, …) used to walk the full native fetch for ~37s and answer 503 'temporarily unavailable, retry shortly', sending callers into a retry loop for data TikTok will never have. /tiktok/search and /tiktok/ad-details now reject unsupported codes instantly with 400 UNSUPPORTED_REGION, retryable: false and the supported-country list in the response. Ad Library error bodies were also flattened: code/retryAfterSeconds now sit directly under error instead of a nested error.error.

• Unsupported country → instant 400 UNSUPPORTED_REGION with supportedCountries[] (never billed)
• retryable: false — this is a TikTok coverage limit, not an outage
• Error envelope flattened: error.code, not error.error.code

Every miss used to be the same 'unavailable — retry shortly' 502, even when retrying could never help. The endpoint now separates the two cases: NO_PUBLIC_REELS (Facebook rendered the profile but shows no Reels to a logged-out viewer — login-walled profile or simply no Reels; retryable: false) and FACEBOOK_RENDER_FAILED (the render itself failed this time — transient, retryable: true with retryAfterSeconds). Stop-or-retry decisions no longer require guessing.

• NO_PUBLIC_REELS — profile rendered, Reels not publicly visible, retrying won't help
• FACEBOOK_RENDER_FAILED — transient fetch miss, retry shortly
• both carry a machine-readable code and retryable flag

A single slow or walled proxy render used to surface straight to you as a 503 FACEBOOK_UNAVAILABLE — sometimes after a 140-second wait — even though the same URL often succeeds seconds later. The fetch now runs as two shorter attempts: a transient miss (timeout, wall render, missing story data) gets one fresh render before we answer, and the worst-case wait dropped from ~140s to ~120s. A Facebook 404 is still answered immediately without a retry.

• transient render misses retry once automatically before a 503
• worst-case latency capped lower (two 40s attempts, not one 120s wait)
• 404 stays definitive — no retry, no extra wait

Numeric profiles (facebook.com/61593…) were timing out at 90s (503) or failing at ~85s (502) — the /reels and /videos tabs were rendered one after another, and per-reel hydration had no overall deadline, so a slow batch was cancelled wholesale at the route ceiling. The two listing tabs now race in parallel, hydration keeps every reel that finished inside the budget instead of discarding them, and reel permalinks on numeric profiles (which have no vanity slug) are now mined too.

• /reels and /videos listing renders race in parallel — failures answer in ~45s instead of ~85s
• per-reel hydration keeps finished rows at the deadline instead of a 503 with nothing
• numeric-profile /videos permalinks (no vanity slug) now yield reel ids

facebook.com/share/r/… (mobile Share button) was a 400 UNRESOLVED_SHARE_LINK asking you to open the link in a browser — logged-out Facebook stopped answering the redirect we relied on. Share links (/share/, /share/p|r|v/) now resolve via a headless render that reads Facebook's own og:url, cached 30 days per share token. The /share/r/<id> form also no longer mis-parses its id as 'r'.

• share/p|r|v links resolve to the canonical post/Reel automatically
• share→canonical mapping cached 30 days — repeat pastes are instant
• slug-form /videos/<seo-slug>/<id>/ URLs classify as video, not page

Keyword search can now reorder the fetched Meta Top page. relevant (default) keeps query ranking; post_dated sorts by publishedAt newest-first; engagement sums likes+replies+reposts+quotes+views. Unknown values are 400. Sorts the returned window — not a second Threads index.

• orderBy on /v1/threads/search (default relevant)
• Envelope echoes the canonical orderBy name

An extended / Apify 200 for a URL-only channel-reels call returned 15 complete reels but a hollow top-level user{} (id, displayName, userId all null) even though every reel author carried the numeric id and display name. Handle→ID resolve had missed, so the envelope kept a username stub and never copied the matching author. user.id / userId / displayName now lift from authors whose username matches the queried handle — never from a collab first row. followers / verified / avatar stay null when Instagram omitted them. hasMore stays null on degraded (unknown), not a false end.

• user.id / userId / displayName filled from matching reel authors on the extended path
• native and extended share one envelope key set (user{}, userId, partial, degraded)
• collab first rows (herbalife on cristiano) still do not become user{}

A same-day burst of /instagram/details 502s (one user, six shortcodes, /p/ then /reel/ 150ms later) was not a global Instagram outage — 108 other details calls were native 200s. Logged-out GraphQL could sit until the 15s wrapper cancelled the session hop, views backfill could throw away a mapped post, /p/ fail-cache replayed /reel/ without fetching it, and APIFY_BLOCK_ENDPOINTS still disabled the actor on details. Logged-out is now 8s, session always gets 6s, a mapped post is kept, Decodo tries the other permalink, and details ignores the Apify kill switch the same way listing already did.

• logged-out GraphQL hard-capped at 8s so the cookie hop still runs
• /p/ and /reel/ both fetched before the 60s fail-cache locks the shortcode
• APIFY_BLOCK_ENDPOINTS cannot disable /instagram/details

region=BR search returned www.tiktok.com/shop/pdp/{id} with integer centavos and no currency. Pasting that URL into product-details defaulted to US and 404ed a live BR listing. Search now emits shop.tiktok.com/{cc}/pdp/{id}, hydrates that storefront, fills currency from the market, and converts integer minor units (1282 → 12.82 BRL). A full extended page sets hasMore=true instead of pretending the catalog is exhausted.

• BR/MX/… search URLs are shop.tiktok.com/{cc}/pdp/{id}
• SERP hydrate fetches the market PDP, not the US www path
• missing currency filled from region; integer tiles become major units
• extended window hasMore=true when the page is full

When native Creative Center missed, the parked Apify fallback still started with apifyProxyGroups:[SHADER]. This account does not have datacenter proxies, so Apify returned 400 invalid-input and we surfaced that as a 503 after ~56s — then retried the same broken payload. The actor now uses RESIDENTIAL (same group as the other TikTok actors). A proxy-group denial retries once without a named group; other invalid-input 400s are not retried.

• Top Ads Apify input defaults to RESIDENTIAL, not SHADER
• SHADER / proxy-group 400 retries once without apifyProxyGroups
• other invalid-input 400s fail immediately (no second hop)

q=healthy drink scanned 15 Creative Center rows and returned ads=[] because none of the titles had those whole words. TikTok had already ranked that page for the keyword — the same list the website shows. Those rows now come back as matchBasis=ranked with literalMatches=0 (no matchedFrom). Whole-word hits still filter as before. orderBy=like echoes likes, matching the documented enum.

• no whole-word hit → ranked page (matchBasis=ranked), not ads=[]
• Spark author nickname is in the literal haystack (same as brandName)
• orderBy echoes likes / impressions, not CC's like / impression

www.tiktok.com/shop/pdp/{id} is the canonical PDP URL — we already parse it. A live BR listing (shop.tiktok.com/br/pdp/{id}, priced in R$) still 404ed because the default region is US and the path has no market. The endpoint now reads /{cc}/pdp/ from shop.tiktok.com and, after a US-default miss, probes other Shop storefronts in parallel before declaring the product gone.

• shop.tiktok.com/br/pdp/{id} sets region=BR from the path
• US-default miss probes BR/MX/GB/ID/TH/VN before 404
• resolved market is echoed as data.region

Healthy Marketplace searches return in ~28s. A hung Decodo first-paint sat through 55s + 20s drain and came back as 504 UPSTREAM_TIMEOUT at exactly ~75s; an empty login shell was 502 UPSTREAM_UNAVAILABLE with no second chance. Search now hedges like marketplace-item: a fresh session starts at 32s and the first HTML that carries listing cards wins. A fast empty/transport miss retries once. resourceUrl is the Facebook search URL instead of null.

• hedged second Decodo session at 32s — first usable listings win
• empty login-shell / transport miss retries once instead of 502
• resourceUrl is the Marketplace search URL, not null

Country-localized /reels listing found shortcodes (Turkey, and the same class of JP/KR misses) but Polarıs hydrate ran from generic US residential and returned zero posts. In-flight GraphQL calls had no per-item cap, so 24 codes walked past the 110s router deadline. Listing HTML media is mapped first; remaining codes hydrate through a country-pinned residential hop with an 8s cap; a last Decodo permalink pass uses the same geo. Failures still 502 with no cached lie — they just stop hanging.

• listing HTML media hydrates the grid without a Polarıs round-trip
• Polarıs residential egress is pinned to the requested country (TR, JP, …)
• each Polarıs item is wait_for-capped at 8s; Decodo geo permalink fills a Polarıs miss

The endpoint treated user.ttSeller=false as “no shop” and returned products:[] without fetching the showcase (showcaseMs=0). Affiliate creators are ttSeller:false by definition, so every affiliate lookup was a billed empty array. The shelf is always fetched now. no_shop means we looked and found nothing; has_products includes affiliate creators. Looked-and-empty stays 1 credit; products bill per result.

• affiliate (ttSeller:false) showcases are fetched instead of short-circuited
• sellerStatus no_shop is a looked-and-empty result, not a skipped fetch
• docs + FAQ no longer say ttSeller:false skips Apify

GET /v1/twitter/search is shelved. Around 2026-08-01, X stopped serving search results to guest sessions: every query now returns an empty timeline envelope from X itself (last successful scrape 2026-07-31), so there is no honest data to return. The endpoint answers 410 Gone at 0 credits instead of a failing scrape, and comes back when a replacement source ships. All other Twitter/X endpoints — profile, user-tweets, tweet-details, transcript, community, community-tweets — are unaffected.

• GET /v1/twitter/search → 410 Gone (0 credits) until a replacement source ships
• Removed from docs, playground, and the MCP / CLI / n8n / Zapier / Make / Apify tool catalogs
• Profile, user-tweets, tweet-details, transcript, and community endpoints unchanged

advertiser-search no longer returns a single Italian NIKE SRL for q=nike&country=US. Brand queries are expanded (Nike, Inc. + siblings), ranked with country preference (US prefers Inc. over SRL/BV), and return multiple AR entities so you can pick. company-ads adds resolvedAdvertiser {id,name,url}, sort=last_shown|first_shown, and stable null keys for text/headline/cta/landingUrl/spend. Chain: search -> advertisers[0].id -> company-ads?advertiser=AR...

• advertiser-search: expand + rank multi-result (Nike, Inc. first for US)
• company-ads: resolvedAdvertiser + sort + stable null schema
• Docs/examples use AR167 Nike, Inc. for the creatives chain

Facebook Ad Library search collapses the same page id to one advertiser name (no Facebook vs Facebook App split), adds platforms filter + nextCursor paging through the HTML batch, and documents status/date/media filters for live campaigns. TikTok Ad Library search relevance-filters so query tokens must appear in advertiser/copy (empty beats spam), keeps FB-parity null keys for headline/cta/landingUrl/spend/advertiser.id, ISO firstShown dates, and bills flat 2 credits native (Apify capped at 5 - never the old ~70 trap). DSA still withholds Meta-style spend; use Creative Center Top Ads for CTR/brand ranking.

• FB: same advertiser id -> one canonical name/url/logo per response
• FB: platforms + cursor/nextCursor; status/date/media already wired
• TT: relevance filter + stable null schema + ISO dates
• TT pricing: 2 native / 5 Apify cap (documented vs old ~70)

Post analytics no longer ships views-only YouTube rows when watch-page enrich is thin - incomplete native responses fall through to Apify so likes, comments, publishedAt, and @handle populate on the vitrin example. author.username never echoes the display name. metrics.engagementRateBasis is always interactions/views on post/compare (do not compare to popular-creators). Compare adds status, failedCount, and failed[] so partial batches are auditable; publishedAt normalizes to UTC Z; docs never fall back to example.com lorem.

• YouTube thin native -> Apify when likes/comments/handle/date missing
• username = @handle only; displayName stays separate
• engagementRateBasis=interactions/views on every post metrics object
• compare: status + failed[] + failedCount; real docs example fallback

LinkedIn /profile no longer short-circuits on a thin native shell — when HTML omits experience/education it enriches from Apify (detail, then full-sections) so B2B core fields actually arrive. Additive sections: experience[], education[], similarProfiles[], projects, publications, articles, activity, recommendations, certifications, languages. Guest-masked asterisk copy is returned as description:null with restricted:true instead of star spam. connections remaining null is documented as a LinkedIn logged-out platform limit.

• Native->Apify enrich when experience/education missing
• similarProfiles[] + richer section pass-through
• Masked ******* -> null + restricted:true
• connections null = LinkedIn guest limit (not a bug)

Account trust signals that make Creator Verification / Partnership Qualification real. TikTok channel-details and popular-creators now expose createTime / createTimeUnix (account age), bioLink{link,risk}, ttSeller, commerce/organization flags, and contact{}. Twitter/X profile keeps blue / legacy / identity verification as separate bits (isBlueVerified != isLegacyVerified != isIdentityVerified), plus fastFollowers/normalFollowers, tipjarSettings -> contact{emails,paymentHandles,links}, and expanded bioUrls. New cacheMaxAge=1d|3d|7d|14d|30d on key profile endpoints; JSON envelope already includes cached + cachedAt on hits.

• TikTok: createTime + bioLink.risk + ttSeller on channel-details / popular-creators
• Twitter: isLegacyVerified + tipjar contact{} + fastFollowers
• cacheMaxAge 1d-30d on TikTok/Twitter/Instagram profile endpoints
• Envelope cached + cachedAt (not header-only)

trending-feed stays the rich For You feed by default, and now accepts SC-style filters — orderBy (hot|like|comment|repost), period (7|30|120), countryCode, and page — to read TikTok Creative Center popular videos (same chart as videos/popular) with pagination.totalCount (~500) while still hydrating engagement/author when possible. Instagram basic-profile / channel-details / profile-search now surface categoryName, fbid, relatedProfiles[] (edge_related_profiles), businessAddress, and likeAndViewCountsDisabled from the existing web_profile_info payload — no new scrape, niche discovery + IG↔FB join without a creator-search endpoint.

• trending-feed: orderBy / period / countryCode / page + pagination.totalCount
• Instagram profiles: relatedProfiles[] + likeAndViewCountsDisabled
• categoryName + fbid + businessAddress parity across profile endpoints
• Post mappers: likeAndViewCountsDisabled (0 ≠ hidden likes)

YouTube search already returned typed results and filters (type, sortBy, uploadDate, duration, region). Each hit now also exposes viewCountText (e.g. 750K) plus viewCountInt (750000) so compact UI labels stay honest. Response adds lives[] / shelves[], and continuationToken as an alias of nextCursor. Docs note: duration filters apply to long-form videos, not Shorts.

• viewCountText + viewCountInt on search hits
• lives[] / shelves[] partitioned arrays
• continuationToken alias for nextCursor

engagement.views is video_view_count (reach-style) when Instagram exposes it; engagement.plays is total play count including replays — the gap can be ~2x. viewsInstagram remains Instagram-only plays (excludes Facebook cross-post); viewsFacebook is the FB share. location{} now includes slug, hasPublicPage, addressJson / address{}. New GET /v1/instagram/highlights and /highlights-details for persistent Story Highlight albums (not live Stories). isAd / isAffiliate / isPaidPartnership and previewComments.authorId are filled on feed and search paths when Instagram exposes them.

• views vs plays vs viewsInstagram / viewsFacebook documented and emitted
• location{id,name,slug,address…} on post/reel mappers
• GET /highlights + /highlights-details (flat 1 credit each)
• isAd / isAffiliate / isPaidPartnership on enriched search/feed rows

popular-hashtags and popular-creators now read TikTok Creative Center charts (ads.tiktok.com/business/creativecenter) instead of sample co-occurrence / invented formulas — so videoCount is a real population total (not "17 from a 20-video sample") and engagementRate is TikTok's official interact rate when exposed. New GET /v1/tiktok/popular-songs adds popular|surging sounds with rankDiff, trend[] time series, and commercialMusic / ifCml for brand-safe audio. Flat 2 credits on the Creative Center path. Optional query= on popular-hashtags still runs the legacy related-tag co-occurrence enrich.

• popular-hashtags: Creative Center chart + rankDiff + trend[] (flat 2 credits)
• popular-creators: Creative Center first, then FYP / Apify fallthrough
• New popular-songs: rankType, commercialMusic, trend[], rankDiff
• country / period (7|30|120) / page / newOnBoard filters

GET /v1/account/balance (and sibling usage/history/daily-usage/most-used-routes/limits) now use camelCase keys — monthlyQuota, subscriptionCredits, topupCredits, totalCredits, subscriptionRenewsAt, creditsUsed, createdAt, etc. — matching the rest of the Captapi surface. Same data; naming only.

• balance: monthlyQuota, subscriptionCredits, topupCredits, totalCredits, subscriptionRenewsAt
• usage/history/routes: creditsUsed, cacheHit, statusCode, createdAt, …

GET /v1/truth-social/profile drops from 5 → 1 credit and adds bot, isPrivate (locked), group, discoverable, location, avatarStatic/headerStatic, emojis[], plus acceptingMessages/chatsOnboarded/tvAccount when present. Bio stays HTML-stripped. Docs warn that as of late 2025 Truth Social typically only exposes public profiles for prominent accounts without login — auth-gated accounts return a clear 404. Native lookup retries via Decodo when datacenter IPs hit Cloudflare.

• 1 credit (was 5)
• bot, isPrivate, group, location, avatarStatic/headerStatic, emojis
• Docs + 404 copy for auth-gated / non-prominent accounts

GET /v1/amazon-shop/page now returns seller name, scrapedAt, stable product shapes (price/currency/priceFormatted always present), isPrime/isBestSeller/isSponsored, canonical /dp/{ASIN} URLs, display priceFormatted ($1,234.56), and cursor pagination (nextCursor/hasMore). Removes rawFirstItem leakage. Docs clarify scope: seller storefronts (/sp?seller=), not influencer /shop/<handle> vitrines. Influencer shop URLs return 400 with a clear message. List price aligned to 1 credit per ~16-product page.

• scrapedAt + isPrime/isBestSeller/isSponsored on products
• seller.name, canonical /dp URLs, stable price fields, cursor pagination
• Removed rawFirstItem; scope note vs influencer /shop/

GET /v1/linktree/page drops from 4 → 1 credit. Returns email when the creator publishes a mailto social, plus verticals and linkPlatforms. GROUP folders now nest child links under links[] (parentId on nested rows); typed links (SPOTIFY_*, SOUNDCLOUD_*, …) kept. socialAccounts includes soundcloud; socials remains the icon list. Browser-like fetch headers avoid Linktree 403s on datacenter UAs.

• 1 credit (was 4)
• email + verticals + linkPlatforms
• GROUP.links nesting + parentId; soundcloud in socialAccounts

GitHub user drops from 3 credits to 1 (matching ScrapeCreators) and adds email (when public), nodeId, apiUrl, hireable, and siteAdmin. type is now user or organization. Docs note that this wraps GitHub's free public REST API — use Captapi for one-key workflows; call api.github.com directly for GitHub-only jobs.

• Price: 3 → 1 credit
• Additive email, nodeId, apiUrl, hireable, siteAdmin
• Honesty note: free GitHub API alternative for GitHub-only workloads

Compare analytics docs no longer show placeholder example.com rows. The live example returns count/resolved/results[] with the same metrics object as post analytics (views, likes, comments, engagementRate, …). Adds cache=true (per-URL cache shared with /post; hits free) and honest billing copy — 1 credit per resolved URL, no bulk discount.

• Live snapshot example with two real YouTube URLs and full metrics{}
• Additive cache param (shared with /v1/analytics/post)
• Docs/FAQ: same shape as post analytics; no bulk credit discount

Pinterest pin-details keeps board{}, author{}, image, and saves, and adds the fields needed for commerce and creator intel: title/description/seoAltText, link/destinationUrl, ISO createdAt, originAuthor vs author (pinner), repinCount/shareCount/reactionCount, and images{} including originals.

• Additive title, description, seoAltText, link/destinationUrl, domain
• Additive createdAt/publishedAt (ISO-8601 from pin page)
• Additive originAuthor + images{236x,564x,originals} + repin/share/reaction counts

Cross-platform post analytics now uses the same enriched YouTube path as video-details, so likes, comments, publishedAt, and engagementRate populate instead of staying null when views alone were present. author.username prefers the channel handle; shares/saves remain null on YouTube (not publicly exposed).

• YouTube analytics reuses enriched video-details (likes/comments/publishedAt)
• author.username ← channelHandle when available; displayName stays channel name
• Docs/FAQ: same metrics shape; platform-missing fields stay null

Google Ads Transparency company-ads keeps the 2-credit media[] response, and adds cursor pagination (nextCursor/hasMore), adsCountEstimate, region alias, and start_date/end_date overlap filters. Docs note public commercial creatives only — login-gated and political ads are out of scope.

• Additive hasMore / nextCursor / adsCountEstimate
• New params: cursor, region, start_date, end_date (topic=all only)
• Honesty note: public ATC creatives only; shapes can vary

Facebook details keeps every existing field (videoUrl, author.username, engagement), and adds stable author.id, feedbackId, captionsUrl, videoSdUrl/videoHdUrl, video dimensions, nested video{}, and music when Facebook exposes them. Docs note that some Reel view counts on the post page can lag the public Reels grid badge.

• Additive author.id + feedbackId
• Additive captionsUrl, videoSdUrl/videoHdUrl, videoWidth/videoHeight, video{}, music{}
• Docs warning for Reel view-count badge mismatch vs profile Reels grid

YouTube channel-details keeps every existing numeric/stats field, and adds email (from About/description when publicly exposed) and tags[] from channel SEO keywords. CAPTCHA-gated business emails stay null.

• Additive email when present in channel About/description or mailto links
• Additive tags[] from channelMetadataRenderer keywords (list, not a comma string)
• Existing subscriberCount/videoCount/viewCount numbers, handle, verified, links unchanged

TikTok Shop product-details keeps price as a float + currency, and now returns seller id/url, originalPrice/discount, description, skus[], and optional region for the Apify path. Related affiliate videos remain best-effort when upstream provides them.

• seller.id / seller.url restored (were stripped in details mode)
• originalPrice, discount, description, skus[], images[], region param
• Native path still bills 2 credits; Apify fallback remains 14

Reddit post-comments publishedAt is now ISO 8601 (was a stringified unix float like 1785330725.0, which broke Date parsers). Existing flat comments + depth/parentId stay; response adds post, score/downs, authorFullname, and hasMore.

• publishedAt is ISO 8601 UTC
• Additive score, downs, authorFullname, distinguished, controversiality
• post object + hasMore/nextCursor (cursor paging still deferred)

LinkedIn profile no longer copies og:description into about, and no longer invents connections/currentCompany from that SEO trailer (the Bill Gates 8 connections bug). about prefers JSON-LD Person description and is omitted when only SEO meta is available; connections matching the privacy/SEO placeholder are null.

• about from JSON-LD only - SEO meta never returned as about
• connections matching SEO N connections on LinkedIn chrome -> null
• Additive experience[] / education[] when the Apify fallback returns them

Facebook Ad Library search keeps every existing ad field your integrations already parse (including media[] and string spend/impressions), and adds the filters and structured fields needed for competitor intel. Default status is ACTIVE so results match what advertisers are running now.

• New filters: status, media_type, ad_type, search_type, sort_by, start_date, end_date (max limit 200 documented)
• Additive ad fields: isActive, publisherPlatforms, cards[], images[], videos[], spendRange/impressionsRange, pageLikeCount, disclaimer/byline, fetchedAt
• searchResultsCount / hasMore / nextCursor on the response (cursor paging deferred); docs note spend/impressions are usually political/issue-only

YouTube comments responses now include authorChannelId and publishedTime (ISO when InnerTube provides it) alongside the existing author string and publishedTimeText. hasCreatorHeart no longer false-positives on inactive heart tooltips. Billing stays a flat 2 credits per call.

• authorChannelId for stable commenter joins
• publishedTime ISO when available; publishedTimeText unchanged
• hasCreatorHeart only true when the creator actually hearted the comment

Instagram channel-details keeps every existing field your integrations already parse, and adds the IDs and account flags needed for joins, outreach, and private-account detection. externalUrl and profileImage are unchanged; bioLinks and profileImageHd are additive.

• New id and fbid for stable Instagram user identity
• isPrivate, isBusinessAccount, isProfessionalAccount, categoryName
• bioLinks[], profileImageHd, businessAddress when present, plus fetchedAt

YouTube and TikTok transcript responses now include source in the body so RAG and analytics can weight caption quality vs Whisper. Existing transcript, transcriptSegments, wordCount, segments, and language fields are unchanged. YouTube also adds isAutoGenerated, isTranslated, and availableLanguages when captions are used.

• source: "captions" | "whisper" (TikTok) | "fallback" (YouTube secondary path)
• YouTube: isAutoGenerated, isTranslated, availableLanguages[]
• No rename of segment fields; float seconds and ISO language codes unchanged

TikTok channel-details keeps every existing field your integrations already parse, and adds stable IDs, account age, commerce/seller flags, and bio-link risk for vetting and joins. category, private, and exact follower counts are unchanged.

• New id and secUid for stable TikTok identity and chaining
• createTime, uniqueIdModifyTime, nickNameModifyTime for account-age and takeover signals
• isCommerceUser, isSeller, isOrganization, isAdVirtual, bioLinkRisk, friendCount/diggCount, avatar sizes, privacy settings, fetchedAt

The in-dashboard free Tools section is gone so the product stays clearly API-first. Public free tools at /tools are unchanged. Bookmarked /dashboard/tools links redirect to the API Playground.

• Removed Tools from the dashboard sidebar
• /dashboard/tools -> /dashboard/playground
• Marketing free tools at /tools remain available

Successful API responses now include cached, creditsUsed, requestId, fetchedAt, and cachedAt in the JSON body (in addition to the existing x-captapi-* headers). Use fetchedAt for time-series snapshots and requestId when contacting support.

• Body fields: cached, creditsUsed, requestId, fetchedAt, cachedAt
• New response header: x-captapi-request-id
• cachedAt is set when the response was served from cache; otherwise null

YouTube and TikTok video-details now return the fields teams need for joins, media pipelines, and filtering — without dumping a raw platform blob. Existing keys are unchanged; new fields are additive.

• YouTube: channelHandle, contentType/isShort, liveStatus, availableCaptions[], thumbnails[], descriptionLinks[], language and access flags, fetchedAt
• TikTok: authorId/secUid, musicId/musicAuthor/isOriginalSound, mediaType, videoUrl/downloadUrl/downloadUrlNoWatermark, mentions[], status flags, isAd/isCommerce, region/width/height, mediaUrlsExpireAt, fetchedAt
• TikTok engagement.isApproximate marks rounded legacy counters vs exact statsV2 counts

Native-only list and search endpoints now bill a flat 2 credits per call instead of scaling per result. This covers YouTube comments, comment replies, channel videos, search, and channel playlists; TikTok top search and trending feed; Instagram hashtag search; Twitter/X user tweets and search; and Reddit subreddit posts, comments, transcript, search, and subreddit search. Apify-backed endpoints are unchanged.

• YouTube comments, replies, channel videos, search, and playlists -> flat 2 credits
• TikTok top search and trending feed -> flat 2 credits
• Instagram hashtag search -> flat 2 credits
• Twitter/X user tweets and search -> flat 2 credits
• Reddit list, comments, transcript, and search endpoints -> flat 2 credits

The Instagram Music Posts API has been removed - it was a duplicate of the Instagram Reels By Audio ID API (same scraper, same data) at a higher price. Use Reels By Audio ID instead; it accepts both audio IDs and full audio page URLs.

• GET /v1/instagram/music-posts no longer exists (returns 404)
• Migrate to GET /v1/instagram/reels-by-audio-id - pass your audio page URL or the numeric audio ID as audio_id
• Old docs links redirect to the Reels By Audio ID page automatically

Every data endpoint now accepts an optional cache=false query parameter to bypass the cache and always fetch fresh data. Previously only transcript, summarizer, and Spotify endpoints supported it.

• Add cache=false to any request when you need live numbers instead of the cached copy (fresh calls are billed as usual; the result still refreshes the cache)
• Documented on all 179 API pages and available in the playground, MCP server, SDKs, CLI, and n8n node
• Account endpoints (balance, usage) are always live and unaffected

A Report a bug button on every API docs page and in the dashboard lets you flag a wrong response, an error, or something slow in a couple of clicks.

• Modal form with an endpoint picker (prefilled on API pages), a description field, and an optional email for logged-out users
• Reports are linked to your account automatically when you're signed in — no need to type your details
• Sits next to "Try it" on each API page and in the dashboard sidebar

Endpoints with a 24-hour cache (transcripts, AI summaries, and Spotify catalog details) now accept an optional cache query parameter. It defaults to true, so existing behaviour is unchanged.

• Pass cache=false to skip the cache and always fetch fresh data; the fresh result still refreshes the shared cache
• Bypassed calls bill normal credits, cached calls remain free
• Available on YouTube/Shorts, TikTok, Instagram, Facebook and Twitter/X transcript & summarize endpoints, plus Spotify artist/track/album/podcast
• Documented on every endpoint page and exposed in the MCP, CLI and n8n catalogs

Instagram post endpoints accept every permalink shape Instagram generates.

• Plural /reels/CODE/ links (as copied from the Reels feed) now work
• Profile-scoped links like instagram.com/username/reel/CODE/ now work

Pin the speech language and control the summary output language on YouTube, TikTok, and Instagram.

• transcript?language=tr pins Whisper's language - recommended for short clips where auto-detection is unreliable
• summarize?language=tr also writes the summary, key points, and topics in that language (default stays English)
• Documented on the endpoint pages and synced to the MCP, CLI, n8n, and Playground catalogs

Whisper-based transcription no longer invents text for videos that contain only music or silence.

• Known hallucination phrases, subtitle credits, and music annotations are filtered out
• Low-confidence segments are dropped; music-only clips now return 422 No speech instead of made-up text
• Wrong-language output (e.g. Turkish speech labeled as Russian) fixed via validated retries with a pinned language

Transcript and summarize endpoints for TikTok and Instagram now resolve media directly from the platform and transcribe speech with our own pipeline; third-party actors remain only as a fallback.

• Instagram Reel media resolves via Instagram's GraphQL in ~1-2 s; TikTok media downloads natively over our proxies
• Typical TikTok transcript latency dropped from ~35-80 s to ~15-20 s
• Transcripts contain only spoken words - post captions and descriptions are never returned as speech

Point a monitor at any list-returning endpoint and get only new items POSTed to your webhook — no polling loops on your side.

• POST /v1/monitors: watch subreddit posts, channel videos, ad-library searches and more on your schedule (15 min to 24 h)
• Deliveries are HMAC-SHA256 signed (X-Captapi-Signature over timestamp.body) so you can verify authenticity
• POST /v1/monitors/{id}/test sends a signed test delivery
• Runs bill credits exactly like direct API calls; cached results stay free

Typed clients generated from the same catalog that powers the API, MCP server, and CLI.

• npm install @captapi/sdk — zero dependencies, works on Node 18+, Deno, Bun, and edge runtimes
• pip install captapi — sync (Captapi) and async (AsyncCaptapi) clients on httpx
• A typed, namespaced method for every endpoint; errors always throw with status + code
• x-api-key header now accepted as an alias for Authorization: Bearer

Live API health at /status, computed from real production traffic — not a manually flipped switch.

• GET /v1/status (no auth) returns overall and per-platform success rates and response times over the last 24 h
• Human view at captapi.com/status refreshes every 2 minutes

Run up to 20 endpoint calls concurrently in a single HTTP request.

• Per-item status and per-item billing — one failed item never fails the batch
• Cached items are free, exactly like single calls
• Ideal for enriching lists of profiles or videos in one round-trip

Follower, view, and like counts now accumulate into a time series automatically whenever tracked profile or post endpoints are fetched fresh.

• Chart growth without building your own snapshot pipeline
• Covers profile and details endpoints across YouTube, TikTok, Instagram, X, Reddit, Bluesky, Twitch and more
• Query by endpoint + URL with a configurable window (up to 365 days)

A dedicated page listing every official integration with setup guides, plus a hover dropdown in the navbar.

• MCP Server (hosted + local), TypeScript & Python SDKs, CLI, n8n, Make.com, Apify Actor, and the REST API in one place
• Each card links to its setup guide and npm/PyPI package
• Machine-readable manifests highlighted for AI agents (/mcp.json, /llms.txt, OpenAPI 3)

Closed field gaps against competitors with dedicated upstream sources.

• Reddit: comment upvotes + threading (residential routing, optional OAuth app support)
• Twitch: clip/profile metadata and streamer schedules via a dedicated actor
• TikTok Shop: product reviews via a dedicated actor
• Also enriched: link-in-bio socials/email, Rumble embeds/streams/comments, Pinterest pin details, SoundCloud artist fields, Bluesky embeds, Truth Social website, GitHub repo/license/topics, X profile counts

The hosted MCP server and every published package now expose the full endpoint catalog.

• @captapi/mcp 0.4.0, @captapi/cli 0.3.0, n8n-nodes-captapi 0.3.0, Apify Actor 0.3 published
• OpenAPI 3 spec linked from llms.txt manifests for AI agent discovery

Every platform got a dedicated landing page with endpoint lists, FAQ, and structured data; the navbar now opens the full API catalog.

Every endpoint page now shows a real captured response, refreshed in batches across the whole catalog.

• Batches covered Twitter/LinkedIn/Reddit, TikTok Shop/Threads/Snapchat/ad libraries, GitHub/Facebook Marketplace/Events/Pinterest/Spotify, and Rumble/Twitch/Bluesky/SoundCloud/Kwai
• Dozens of mapper fixes along the way: timestamped transcript segments, YouTube comment author metadata, duration parsing, Instagram actor replacement and more

Major catalog expansion with reliability fixes across the board.

• New platforms and endpoints across the catalog, audited live end-to-end
• Endpoint reliability, agent URL validation, and integration discovery improvements
• Comprehensive live audit reports added to the repo

Expanded TikTok, Instagram, YouTube, and Facebook coverage to close competitor gaps.

• 19 new endpoints: TikTok hashtag/top/user search, song details, trending; IG hashtag/profile search, story highlights, embed; YT comment replies, channel playlists, community posts; FB profile posts/reels, group posts, comment replies
• Per-result pricing normalized to guarantee margins; dashboard analytics tab with daily usage charts

One API for structured public social-media data: transcripts, AI summaries, comments, profiles, search, and downloads.

• REST API with Bearer auth, credit-based billing, and cached-result discounts
• Dashboard with API keys, playground, usage analytics, and billing
• SEO/AEO foundation: llms.txt, structured data, programmatic docs