lexiforest

Today I learned about "curl_cffi", a Python binding for curl that can make it pretend it's a request from Chrome (or Firefox, or Safari) lexiforest/curl_cffi

How to Find TikTok Creators' Business Emails (Bio + Linktree), No Login Needed Brands looking for TikTok creators to work with usually end up copying emails from bios by hand, one profile at a time. Most creators who want brand deals do publish a business email: either in the bio itself, or on the Linktree-style page their bio links to. This tutorial shows how to collect those contacts at scale without logging in: a no-code route, a few lines of Python that turn this week's trending US TikToks into a creator lead list, and a do-it-yourself script that reads a profile and its bio link directly. Disclosure: I built the Apify actor used in options 1 and 2 (TikTok Transcript Scraper, which now has a creator-contacts option). Option 3 doesn't use it. How many creators publish an email? From my test runs on Apify's cloud on 27 Oct 2026: - 14 of 25 creators behind this week's trending US videos had an email address in their bio. - In a second run, 7 of 8 creator profiles produced an email once the bio-link page was checked too: - 6 had an email in the bio (3 of them repeated it on their link page), - 1 had no email in the bio, only on the bio-link page (a msha.ke page), - 1 link page gave a second, different address next to the one in the bio, - 1 link page (beacons.ai) refused the request with HTTP 403. - Most addresses were management or talent-agency inboxes rather than personal ones, which is what you want for brand outreach. - Brands are different: none of the six brand accounts I checked (Scrub Daddy, Duolingo, Gymshark, Ryanair, Liquid Death, Chipotle) published an email in their bio or on their link page. This works for creators, not for companies. What you get per creator One row per creator (recordType: "creator"). Real values from the run; handle and email redacted: | Field | Example | | --- | --- | | username, url, nickname, verified | <handle>, https://www.tiktok.com/@<handle> | | followers, following, likes, videoCount | 133097, 234, 2400000, 133 | | isSeller, businessCategory, isOrganization | false, null, false (a TikTok Shop seller in the same run: true, Shopping & Retail) | | bio, bioLink | the bio text, https://linktr.ee/<handle> | | emails | ["<name>@<agency>.com", "<handle>@gmail.com"] | | emailSources | {"bio": ["<name>@<agency>.com"], "bioLink": ["<handle>@gmail.com"]} (where each address was found) | | bioLinkNote | HTTP 403 when a link page refused the request, or redirected to <site> | | creatorSource | input, or video_author for creators found through videos | Every video row also carries the author's authorFollowers, authorLikes, authorBio and authorEmails, and you can add the transcript of what they said. Option 1: No code (Apify Console) 1. Open the actor page and click Try for free. Apify's free plan includes monthly platform credit; the actor itself is pay-per-event (current price on the Store page), and unavailable profiles aren't charged. 2. Paste profile URLs or @handles into Creator profiles (contacts). 3. Keep Scan bio-link pages for emails on. 4. Click Start and open the Creator contacts tab of the results: followers, seller flag, category, emails and bio link, ready to export as CSV or Excel for your CRM. To find creators instead of bringing your own list, turn on Trending US videos and Add a creator row for each video's author: you get this week's trending videos with transcripts plus a contact row for each creator behind them. Option 2: Python pip install apify-client, then take your API token from Apify Console → Settings → Integrations. from apify_client import ApifyClient client = ApifyClient("<YOUR_APIFY_TOKEN>") run = client.actor("axlymxp/tiktok-transcript-scraper").call(run_input={ "trendingVideos": True, # this week's trending US videos... "trendingCategories": ["Beauty & Care"], "creatorsFromVideos": True, # ...plus one contact row per creator "creatorProfiles": ["@creator_one", "https://www.tiktok.com/@creator_two"], "scanBioLinks": True, "maxVideos": 20, }) rows = list(client.dataset(run["defaultDatasetId"]).iterate_items()) creators = [r for r in rows if r["recordType"] == "creator" and r["emails"]] for c in sorted(creators, key=lambda r: r["followers"] or 0, reverse=True): print(f'{c["followers"]:>10,} @{c["username"]:<25} {", ".join(c["emails"])} ({", ".join(c["emailSources"])})') Video rows (recordType: "video") from the same run carry the transcript, so you can check what a creator actually says before you write to them, and mention it in your first line. Option 3: DIY, profile + bio link with no login A TikTok profile page embeds its data as JSON (__UNIVERSAL_DATA_FOR_REHYDRATION__ → webapp.user-detail), readable without signing or cookies. With curl_cffi (pip install curl_cffi): import json, re from curl_cffi import requests EMAIL = re.compile(r"[A-Za-z0-9._%+-]+@[A-Za-z0-9-]+(?:\.[A-Za-z0-9-]+)*\.[A-Za-z]{2,}") s = requests.Session(impersonate="chrome124") handle = "scrubdaddy" html = s.get(f"https://www.tiktok.com/@{handle}", headers={"Accept-Language": "en-US,en;q=0.9"}).text m = re.search(r'<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__"[^>]*>(.*?)</script>', html, re.S) info = json.loads(m.group(1))["__DEFAULT_SCOPE__"]["webapp.user-detail"]["userInfo"] user, stats = info["user"], info["stats"] print(user["uniqueId"], stats["followerCount"], "seller:", user.get("ttSeller"), "category:", (user.get("commerceUserInfo") or {}).get("category")) print("bio emails:", EMAIL.findall(user.get("signature") or "")) link = (user.get("bioLink") or {}).get("link") if link: r = s.get(link if link.startswith("http") else "https://" + link) print("bio link:", link, "->", r.url, "emails:", sorted(set(EMAIL.findall(r.text)))) For this brand account it prints scrubdaddy 4400000 seller: True category: Others, then empty email lists for the bio and its likeshop.me link page. Swap in a creator's handle and the emails show up. Things I ran into: - The bio is called signature and the link is bioLink.link. TikTok Shop sellers have ttSeller: true and a commerceUserInfo.category such as Shopping & Retail, which is useful for filtering. - Check where the link page ends up. One creator's link had died and redirected to a creator-marketing company's homepage, which lists that company's own email. Only trust emails from a page that stayed on the host the creator linked. - Bios often contain links without https:// (Duolingo.com). Add the scheme before requesting it, or your redirect check will fail. - Not every link page lets you in. Linktree, msha.ke and hoo.be pages loaded fine; beacons.ai answered 403. - Unknown or banned users don't return 404: the page loads and webapp.user-detail.statusCode is non-zero (10221 for a handle that doesn't exist). - Filter false positives. An email regex also matches image names like logo@2x.png, so drop matches that end in a file extension. Use the contacts responsibly These addresses are published by the creators for business enquiries, but they're still personal data under GDPR/CCPA. Write to people about something relevant to them, keep it to one or two messages, and honour opt-outs. Don't resell lists or add them to newsletters without consent. FAQ Do I need a TikTok account? No. Only public profiles are read, logged out. Can it search creators by keyword or niche? Not directly: TikTok's search needs signed requests. Use the trending videos and their categories, your own list of handles, or the authors of videos you already collected. Questions or a profile that doesn't work? Open an issue on the actor page and I'll take a look.

TikTok Creative Center Trends Changed: How to Get This Week's Trending TikToks (With Transcripts) If you scrape TikTok Creative Center's trending lists, you may have noticed your script quietly stopped returning data. The old creative_radar_api/v1/popular_trend/... endpoints for trending hashtags, videos, sounds and creators now answer {"code": 0, "data": null, "msg": "deprecated"}, even with a valid signature, and the Creative Center itself has moved to a new app. This tutorial shows what's still reachable without logging in, a short Python script that gets this week's trending US videos from the new endpoint, and a no-code route that also transcribes those videos and pulls the creator's contact email from their bio. Disclosure: I built the Apify actor used in options 1 and 2 (TikTok Transcript Scraper). Option 3 doesn't use it. What changed in October 2026 I checked this on 27 Oct 2026: - The old URLs (/business/creativecenter/inspiration/popular/...) now redirect to a new app at ads.tiktok.com/creative/creativeCenter/trends. - The old trend endpoints return "deprecated". Only the Top Ads endpoints on the old app still work. - Logged out, the new Trends page shows the top 3 hashtags per country, and a Video tab backed by a new, unsigned endpoint: ads.us.tiktok.com/CreativeOne/Report/CreativeCenterGetTopContentsList. - That video endpoint returns 4 videos per call, US only. Asking for page 2 or a bigger limit returns the same 4, and Japan, Vietnam, Thailand, Indonesia and the UK came back empty. "Go to TikTok One to view more" is the login wall. Combining the sort orders, the five content categories and the organic-only filter gives 81 distinct trending US videos for the week ending 25 Oct. Earlier weeks also work, at least 4 back. What the trending videos look like I transcribed 25 of them on Apify's cloud: - 22 had TikTok's own captions, 1 was transcribed with Whisper AI, and 2 had no speech (music only). - 23 of 25 were marked as ads. Sorting by views mostly surfaces sponsored posts: the top Technology & Finance video had 16.4M weekly views, of which only 840K were organic. - Sorting by engagement rate gives a different picture: small videos (2K-12K weekly views) with engagement rates above 50 %. - 14 of 25 creator bios contained an email address, mostly management or talent-agency inboxes. That's the contact a brand needs when it wants to work with a creator who is trending right now. What you get per video Real values from the run (the Technology & Finance #1 video for the week ending 25 Oct): | Field | Example | | --- | --- | | url, authorUsername | https://www.tiktok.com/@owennurminen/video/7676518404357999903 | | trendWeekEnding, trendRank, trendCategories | 2026-10-25T00:00:00Z, 1, ["Technology & Finance"] | | weeklyViews, lifetimeViews, organicViews | 16434109, 97959655, 840503 | | engagementRate | 0.0011 | | creatorFollowers, creatorEmails | 137350, [] (filled for 14 of 25 videos) | | transcriptSource, language | tiktok_captions, en | | hook | If you're a young guy with motion. One of my friends just put me onto something | | transcript | [{"start": 0.1, "end": 1.38, "text": "If you're a young guy with motion."}, …] | | playCount, likeCount, isAd | 100000000, 104300, true (from the video page) | Option 1: No code (Apify Console) 1. Open the actor page and click Try for free. Apify's free plan includes monthly platform credit; the actor itself is pay-per-event (current price on the Store page), and videos without speech aren't charged. 2. Turn on Trending US videos. Optionally set Weeks of trending videos (1-5) and Trending categories (Beauty & Care, Fashion, Food & Beverage, Lifestyle & Leisure, Technology & Finance). 3. Leave TikTok videos empty, or add your own links to transcribe them in the same run. 4. Click Start. The Trending US videos tab of the results shows rank, views, engagement, followers, creator emails and the transcript, ready to export as CSV, JSON or Excel. Option 2: Python pip install apify-client, then take your API token from Apify Console → Settings → Integrations. from apify_client import ApifyClient client = ApifyClient("<YOUR_APIFY_TOKEN>") run = client.actor("axlymxp/tiktok-transcript-scraper").call(run_input={ "trendingVideos": True, "trendingWeeks": 1, # 1-5 weekly rankings "trendingCategories": ["Fashion", "Beauty & Care"], "outputFormat": "both", # segments + one joined string }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item["trendRank"], item["weeklyViews"], item["url"]) print(" emails:", item["creatorEmails"]) print(" hook:", item["hook"]) Find creators to pitch rows = list(client.dataset(run["defaultDatasetId"]).iterate_items()) leads = [r for r in rows if r.get("creatorEmails")] for r in sorted(leads, key=lambda r: r["creatorFollowers"] or 0, reverse=True): print(f'{r["creatorFollowers"]:>10,} @{r["authorUsername"]} {", ".join(r["creatorEmails"])}') Option 3: DIY, the new endpoint with no signing The Video tab's endpoint needs no user-sign, no cookies and no login, just browser-like headers. With curl_cffi (pip install curl_cffi): from curl_cffi import requests BASE = "https://ads.us.tiktok.com/CreativeOne/Report" HEADERS = {"Origin": "https://ads.tiktok.com", "Referer": "https://ads.tiktok.com/", "agw-js-conv": "str"} s = requests.Session(impersonate="chrome124") week_end = s.get(f"{BASE}/GetTopContentsOverview", headers=HEADERS).json()["lastWeeklyEndTimestamp"] params = { "countryCode": "US", "contentLabelIDs": "", # "" = all categories, "11003" = Fashion, ... "orderByMetric": 1, # 1 = views, 2 = engagement rate "organicOnly": "false", "periodDimension": 3, # 3 = weekly "periodEndTimestamp": week_end, "page": 1, "limit": 20, } data = s.get(f"{BASE}/CreativeCenterGetTopContentsList", params=params, headers=HEADERS).json() for item in data["entityInfos"]: info, author = item["itemInfo"], item["itemAuthorInfo"] url = f'https://www.tiktok.com/@{author["handlerName"]}/video/{info["itemID"]}' print(item["itemMetrics"]["videoViews"], url) print(data["pagination"]) It prints four videos and {'hasMore': True, 'limit': 4, 'page': 1, 'pageCount': 25, 'totalCount': 100}. Things I ran into: - totalCount: 100 is a tease. Logged out, page and limit are ignored. To get more distinct videos, vary orderByMetric (1-3; 4 and 5 return nothing), contentLabelIDs (11002 Beauty & Care, 11003 Fashion, 11010 Food & Beverage, 11013 Lifestyle & Leisure, 11015 Technology & Finance) and organicOnly. That's 36 calls for about 80 videos. - Each item carries itemAuthorInfo.bio, so a regex for email addresses gives you creator contacts without visiting each profile. - Pace your calls. After a quick burst of around 40 requests, my connection to the host started failing at the TLS handshake for a while. One request every 2 seconds was fine. - The videoURL in the response expires, so to transcribe a video, open its TikTok page and use the caption track there, as in my earlier TikTok transcript tutorial. Scheduling and integrations Schedule a weekly run on Monday (the weekly ranking ends on Sunday 00:00 UTC) to collect each week's trending videos, transcripts and creator contacts into Google Sheets, Slack or a webhook. FAQ Why only the US? Logged out, the endpoint returns videos for the US only; other regions came back empty in my tests. Hashtags are available for 27 countries, but only the top 3 each. Are the creator emails personal data? They come from public profile bios and are mostly business contacts, but they're still personal data under GDPR/CCPA. Use them for relevant business outreach and respect opt-outs. Is it legal? These are public pages read without logging in. How you use the data is on you: respect TikTok's terms and privacy law. Questions or a run that doesn't work? Open an issue on the actor page and I'll take a look.

How to Get Facebook Video and Reel Transcripts (Even When There Are No Captions) Facebook plays captions on many videos and Reels, but if you don't own the video there's no button to export them, and there's no public API for transcripts of other pages' videos. Most online tools want you to download the file first, and several of the URL-based ones only return captions that already exist or stop at clips of about two minutes. This tutorial covers three routes: a no-code run, a few lines of Python for bulk jobs, and a do-it-yourself script that reads Facebook's own caption file and audio track with no login and no API key. I'll also show what happens with videos that have no captions, which in my test was half of them. Disclosure: I built the Apify actor used in options 1 and 2 (Facebook Video & Reel Transcript Scraper). Option 3 doesn't use it. Captions vs. no captions: what I found On 27 Oct 2026 I ran a test on Apify's cloud with 7 links I picked (NASA, Gordon Ramsay, MrBeast Gaming, Tasty, BBC News, plus one deleted video) and the 4 latest videos from the CNN and Gordon Ramsay pages. Two of the Gordon Ramsay page videos were already in my link list, so that's 13 different videos: - 6 had a Facebook caption track, which came back as is. One of them was a 12-minute MrBeast Gaming video (2,613 words). - 6 had no caption track and were transcribed with Whisper AI, including all four CNN videos. - 1 (the deleted video) came back as a not_available row. The whole run took 61 seconds. A captions-only tool would have returned nothing for half of the live videos. What you get per video One row per video. Real values from that run: | Field | Example | | --- | --- | | url, videoId, isReel | https://www.facebook.com/reel/1863947697952668/ | | ownerName, ownerId | NASA - National Aeronautics and Space Administration | | title, description | NASA Moon Base: Rovers and Lunar Terrain Vehicles, plus the post text | | createdAt | 2026-10-20T15:04:42Z | | viewCount, reactionCount, commentCount | 54203, 2840, 142 | | durationSeconds | 45.82 | | transcriptSource | facebook_captions (or ai_whisper when the video had none) | | language, isAutoGenerated, isTranslated | en, false, false | | hook | Think about hundreds of square miles where we have different assets. | | transcript | [{"start": 0.934, "end": 5.205, "text": "Think about hundreds of square miles where we have different assets."}, …] | | srt, vtt | ready-to-use subtitle text (optional) | | sourcePage | cnn when the video came from a page input | Facebook doesn't show view counts for every link style: in my runs /reel/… links often had none, while watch and page-video links had them. Option 1: No code (Apify Console) 1. Open the actor page and click Try for free. Apify's free plan includes monthly platform credit; the actor itself is pay-per-event (current price on the Store page), and videos without speech or removed videos aren't charged. 2. Paste links into Facebook videos and Reels: Reel links (facebook.com/reel/…), watch links (facebook.com/watch/?v=…), page video links or bare numeric video IDs. 3. Or put a page name such as cnn into Pages (latest videos) to transcribe the videos on that page's Videos tab. 4. Pick the Transcript source: - auto: Facebook's captions when present, Whisper AI otherwise, - captions: Facebook's captions only (cheapest; no AI), - ai: always transcribe the audio, e.g. to translate everything to English. 5. Click Start and export the results as CSV, JSON or Excel. Turn on Add SRT and VTT subtitles if you need subtitle files. Option 2: Python pip install apify-client, then take your API token from Apify Console → Settings → Integrations. from apify_client import ApifyClient client = ApifyClient("<YOUR_APIFY_TOKEN>") run = client.actor("axlymxp/facebook-video-transcript-scraper").call(run_input={ "videoUrls": [ "https://www.facebook.com/watch/?v=1863947697952668", "https://www.facebook.com/reel/1083087217889684", ], "pageUrls": ["cnn"], # latest videos from a page's Videos tab "maxVideosPerPage": 4, "transcriptSource": "auto", "outputFormat": "both", # segments + one joined string }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): if item["error"]: # no_speech, not_available, too_long, ... print("skip", item["videoId"], item["error"]) continue print(item["ownerName"], item["viewCount"], item["transcriptSource"]) print(" hook:", item["hook"]) print(" text:", item["transcriptText"][:200]) outputFormat accepts segments ([{start, end, text}]), text (one string) or both (segments in transcript, the string in transcriptText). Track what a competitor page talks about Schedule the same input daily with only pageUrls, then search the new rows: keywords = ["election", "tariff", "AI"] for item in client.dataset(run["defaultDatasetId"]).iterate_items(): text = (item.get("transcriptText") or "").lower() hits = [k for k in keywords if k.lower() in text] if hits: print(item["createdAt"], item["url"], hits) Example output One real row from a CNN video without captions (Whisper; trimmed to three segments): { "videoId": "1806375253903444", "url": "https://www.facebook.com/reel/1806375253903444/", "sourcePage": "cnn", "description": "On Sunday, during stormy weather, a TUI Cruises passenger ship clashed with two container ships while departing Germany on its way to the Canary Islands.", "ownerName": "CNN", "createdAt": "2026-10-27T00:25:01Z", "durationSeconds": 51.67, "viewCount": 12591, "reactionCount": 150, "availableCaptionLanguages": [], "transcriptSource": "ai_whisper", "language": "en", "hook": "This is the moment a cruise liner bound for the Canary Islands collides with two container ships", "transcript": [ {"start": 0, "end": 6, "text": "This is the moment a cruise liner bound for the Canary Islands collides with two container ships"}, {"start": 6, "end": 13.28, "text": "while departing Bremerhaven Germany. During Stormy Weather a T.Y. cruise's ship was caught by a gust of wind"}, {"start": 13.28, "end": 19.68, "text": "and local police said that it caused it to collide with two more container ships. The impact"} ], "wordCount": 130, "audioSeconds": 51.69, "error": null } Note "T.Y. cruise's" in the second segment: the post text says "TUI Cruises". Whisper (base model) handles clear speech well but can mishear brand names, so check names against the description when they matter. Option 3: DIY, Facebook's own captions with no login A public Facebook video page embeds its data as JSON inside <script type="application/json"> tags. The video's caption tracks and an audio-only stream are in there, readable without cookies or login. You need curl_cffi (pip install curl_cffi) so the request looks like a normal browser: import json, re from curl_cffi import requests video_id = "1863947697952668" s = requests.Session(impersonate="chrome124") html = s.get(f"https://www.facebook.com/watch/?v={video_id}", headers={"Accept-Language": "en-US,en;q=0.9", "Sec-Fetch-Mode": "navigate"}).text def walk(o): if isinstance(o, dict): yield o for v in o.values(): yield from walk(v) elif isinstance(o, list): for v in o: yield from walk(v) dicts = [d for m in re.finditer(r'<script type="application/json"[^>]*>(.*?)</script>', html, re.S) for d in walk(json.loads(m.group(1)))] # caption tracks sit on the video node; the original track is "Auto-generated" or uploaded (null) tracks = next((d["video_available_captions_locales"] for d in dicts if d.get("id") == video_id and d.get("video_available_captions_locales")), []) for t in tracks: print(t["locale"], t["localized_creation_method"]) if tracks: print(s.get(tracks[0]["captions_url"]).text[:200]) # audio-only track for speech-to-text when there are no captions audio = next((r["base_url"] for d in dicts if str(d.get("video_id")) == video_id for r in d.get("representations") or [] if r.get("mime_type", "").startswith("audio")), None) print("audio:", audio[:80] if audio else None) For this NASA video it prints en_US None (a caption file the page uploaded), the first SRT cues, and an audio URL. Things I ran into while building this: - The video is split across many JSON fragments. Duration, captions, owner and the post text live in different objects that share the video's id, and the audio stream sits in a separate all_video_dash_prefetch_representations block keyed by video_id. Collect everything first, then merge. - Don't trust the first counter you find. The page also embeds comments and "related videos", each with its own reaction_count. My first version gave three different MrBeast videos the same reaction count. Read counters only from the post's own feedback object and skip the comment and related-video branches. - Many caption tracks are machine translations. One Gordon Ramsay Reel had 10 tracks. localized_creation_method is written in the track's own language: Auto-generated marks the original speech-recognition track, null an uploaded file, and strings like Traducido automáticamente a translation. Pick the original unless you want a translation. - The audio-only track is small. It's an AAC stream of about 70 kbit/s that a plain GET downloads in full, so you don't need the video file for speech-to-text. Transcribe it with faster-whisper. - A deleted video doesn't return 404. A dead Reel loads a normal page whose embedded query returns "video": null, and a dead watch link shows the logged-out "Discover" feed instead. Check for that before reporting a block. - Page listings stop at the first screen. A page's Videos tab embeds 9 to 41 videos logged out; loading more needs a login. For a page's full history, collect the video links another way. Scheduling and integrations Run the actor from a schedule (for example, daily on a list of pages), Make, Zapier or the API, and send the dataset to Google Sheets, Slack or a webhook. An interrupted run resumes where it stopped when started again with the same input. FAQ Do I need a Facebook account? No. Only public videos are read, logged out. Group videos, private videos and Stories aren't supported. Long videos? Caption tracks work at any length (the 12-minute MrBeast Gaming video above came from captions). AI transcription skips videos longer than Skip AI transcription above (minutes), 30 by default, such as multi-hour live-stream replays; they come back as free too_long rows. Other languages? Set your preferred caption order with Preferred caption languages; isTranslated: true marks Facebook's machine translation. Whisper detects ~100 spoken languages and can translate them to English. Is it legal? These are public videos read without logging in. How you use the data is on you: respect Meta's terms and privacy law (GDPR/CCPA). Questions or a video that won't work? Open an issue on the actor page and I'll take a look.

How to Get TikTok Video Transcripts (Even When There Are No Captions) TikTok shows auto-captions on many videos, but there's no button to export them and no official transcript API. Most "TikTok transcript API" guides end with signing up for a third-party key, and many of those tools only return text for videos that already have captions. This tutorial covers three routes: a no-code run, a few lines of Python for bulk jobs, and a do-it-yourself script that pulls TikTok's own caption file with no key at all. I'll also show what happens with videos that have no captions, which in my test was more than half of them. Disclosure: I built the Apify actor used in options 1 and 2 (TikTok Transcript Scraper). Option 3 doesn't use it. Captions vs. no captions: what I found I ran 24 random public TikToks (from #dance, #photomode and #asmr) through the actor on 26 Oct 2026: - 10 had TikTok's own auto-generated caption track, which came back as is. - 14 had no caption track. Of those, 3 contained speech and were transcribed with Whisper AI; the other 11 were music-only dances or silent ASMR, so there was nothing to transcribe. A captions-only tool would have returned nothing for those 3 spoken videos. That's why the actor falls back to speech-to-text. What you get per video One row per video. Real values from a run on 26 Oct 2026: | Field | Example | | --- | --- | | url, videoId | https://www.tiktok.com/@renreports/video/7307726351291960622 | | authorUsername | renreports | | createdAt | 2023-12-01T20:30:54Z | | playCount, likeCount, commentCount, shareCount, collectCount | 66900, 2703, 54, 228, 2927 | | durationSeconds | 139 | | transcriptSource | tiktok_captions (or ai_whisper when the video had none) | | language, isAutoGenerated, isTranslated | en, true, false | | hook | I work in influencer marketing and I look at media kids all day long. | | transcript | [{"start": 0.08, "end": 1.28, "text": "I work in influencer marketing"}, …] | | wordCount | 512 | | hashtags, musicTitle, isOriginalSound | the post's hashtags and sound | Note "media kids" in the hook: that's TikTok's own auto-caption (it means "media kits"). Captions are returned exactly as TikTok generated them. Option 1: No code (Apify Console) 1. Open the actor page and click Try for free. Apify's free plan includes monthly platform credit; the actor itself is pay-per-event (current price on the Store page), and videos without speech aren't charged. 2. Paste links into TikTok videos: full URLs (tiktok.com/@user/video/…), mobile links or bare numeric video IDs. 3. Pick the Transcript source: - auto: TikTok's captions when present, Whisper AI otherwise, - captions: TikTok's captions only (cheapest; no AI), - ai: always transcribe the audio, e.g. to translate everything to English. 4. Click Start and export the results as CSV, JSON or Excel. Option 2: Python pip install apify-client, then take your API token from Apify Console → Settings → Integrations. from apify_client import ApifyClient client = ApifyClient("<YOUR_APIFY_TOKEN>") run = client.actor("axlymxp/tiktok-transcript-scraper").call(run_input={ "videoUrls": [ "https://www.tiktok.com/@renreports/video/7307726351291960622", "7477698081920748830", # a bare video ID works too ], "transcriptSource": "auto", "outputFormat": "both", # segments + one joined string }) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): if item["error"]: # no_speech, not_available, ... print("skip", item["videoId"], item["error"]) continue print(item["authorUsername"], item["playCount"], item["transcriptSource"]) print(" hook:", item["hook"]) print(" text:", item["transcriptText"][:200]) outputFormat accepts segments ([{start, end, text}]), text (one string) or both (segments in transcript, the string in transcriptText). Rank hooks by plays rows = [i for i in client.dataset(run["defaultDatasetId"]).iterate_items() if i["hasSpeech"]] for i in sorted(rows, key=lambda i: i["playCount"] or 0, reverse=True): print(f'{i["playCount"]:>10,} {i["hook"]}') Example output One real row (trimmed to three segments; description and coverUrl removed): { "videoId": "7307726351291960622", "url": "https://www.tiktok.com/@renreports/video/7307726351291960622", "authorUsername": "renreports", "createdAt": "2023-12-01T20:30:54Z", "durationSeconds": 139, "playCount": 66900, "likeCount": 2703, "shareCount": 228, "hashtags": ["influencermediakit", "influencermarketingtips", "contentcreatortips", "ugccreatortips2024", "socialmediamarketingtips"], "transcriptSource": "tiktok_captions", "language": "en", "isAutoGenerated": true, "isTranslated": false, "hook": "I work in influencer marketing and I look at media kids all day long.", "transcript": [ {"start": 0.08, "end": 1.28, "text": "I work in influencer marketing"}, {"start": 1.28, "end": 3.36, "text": "and I look at media kids all day long."}, {"start": 3.36, "end": 5.08, "text": "And here is one mistake that I see"} ], "wordCount": 512, "error": null } Option 3: DIY, TikTok's own captions with no API key A public TikTok video page embeds its data as JSON in a <script id="__UNIVERSAL_DATA_FOR_REHYDRATION__"> tag, including links to the caption files. No signing, cookies or login are needed to read it. You need curl_cffi (pip install curl_cffi) so the request looks like a normal browser: import json, re from curl_cffi import requests url = "https://www.tiktok.com/@renreports/video/7307726351291960622" s = requests.Session(impersonate="chrome124") html = s.get(url, headers={"Accept-Language": "en-US,en;q=0.9"}).text m = re.search(r'<script id="__UNIVERSAL_DATA_FOR_REHYDRATION__"[^>]*>(.*?)</script>', html, re.S) if not m: raise SystemExit("No page data: TikTok served a challenge page to this IP") item = json.loads(m.group(1))["__DEFAULT_SCOPE__"]["webapp.video-detail"]["itemInfo"]["itemStruct"] tracks = (item["video"].get("claInfo") or {}).get("captionInfos") or [] for t in tracks: print(t["languageCode"], "auto" if t["isAutoGen"] else "manual", "original" if t["isOriginalCaption"] else "translation") if tracks: vtt = s.get(tracks[0]["url"], headers={"Referer": "https://www.tiktok.com/"}).text print(vtt[:300]) # WebVTT: "00:00:00.080 --> 00:00:01.280 / I work in influencer marketing" ... Things I ran into while building this: - Some IPs get a challenge page instead of the data. From Apify's cloud servers (datacenter and residential) the page came back fine. From one AWS server in Singapore, TikTok returned a "SlardarWAF" challenge every time. If m is None, try a different network. - Use the same session for the page and the files. The video file URL (video.playAddr) returned HTTP 403 when downloaded without the cookies from the page request. Keep one session for the page, the caption file and the media. - "Original sound" isn't always this video's audio. music.playUrl is a handy audio-only MP3 when music.original is true, but in one case an 8-second video pointed to a 46-second sound from another upload. Compare music.duration with video.duration before using it. - No captions? Run Whisper on the audio. Download the audio (music.playUrl when it matches, otherwise video.playAddr) and transcribe it with faster-whisper. Expect small differences on mumbled clips: for one video TikTok's captions said "Bagels to make him dance" and Whisper heard "Beggles to make you dance". Scheduling and integrations Run the actor from a schedule, Make, Zapier or the API whenever you collect new TikTok links, and send the dataset to Google Sheets, Slack or a webhook. An interrupted run resumes where it stopped when started again with the same input. FAQ Do I need a TikTok account? No. Only public videos are read, logged out. What about videos with music only? They come back with error: no_speech and aren't charged. Sung lyrics, on the other hand, are transcribed like speech. Other languages? TikTok often has caption tracks in several languages; set your preferred order with Preferred caption languages. isTranslated: true marks a track that is TikTok's translation rather than the original language. Whisper detects ~100 spoken languages and can translate to English. Can it take a username or hashtag? Not yet: it takes video links or IDs. Listing a creator's videos needs TikTok's signed web API, which a plain HTTP client can't produce. Is it legal? These are public videos read without logging in. How you use the data is on you: respect TikTok's terms and privacy law (GDPR/CCPA). Questions or a video that won't work? Open an issue on the actor page and I'll take a look.

«Качалка видео для Windows и Mac: почему TikTok не пускал собранную программу, а QuickTime принимал видео за музыку» Качалка видео для Windows и Mac: почему TikTok не пускал собранную программу, а QuickTime принимал видео за музыку Вставляешь ссылку на видео с YouTube, TikTok, VK или Instagram, выбираешь 1080 p, 720 p или 480 p и жмёшь «Скачать». Через несколько секунд на Рабочем столе лежит mp4, который открывается в QuickTime, на айфоне и в любом плеере Windows. Пока идёт загрузка, по пиксельному офису бегает человечек и складывает в папку листки. Это Качалка — бесплатная программа с открытым кодом. Никаких аккаунтов, рекламы и водяных знаков. Ниже — из чего она собрана и два бага, на которые я потратил больше всего времени: оба проявлялись только в собранной программе и ни разу при запуске из кода. Давно хотел делать бесплатно для людей, время пришло. Из чего собрано yt‑dlp делает всю тяжёлую работу: достаёт видео с сотен сайтов. FFmpeg склеивает видео со звуком. Он едет внутри программы через пакет imageio-ffmpeg, так что ставить ничего не нужно. CustomTkinter рисует окно: тёмная тема, фиолетовые кнопки. PyInstaller собирает Kachalka.exe для Windows и kachalka.app для Mac. GitHub Actions собирает обе версии по тегу. У меня нет компьютера с Windows, так что exe я ни разу не собирал сам. «Код писал вместе с Claude Code; идея, решения и проверка — мои». Почему H.264, а не «лучшее качество» Первое желание — качать максимум. Но YouTube отдаёт высокие разрешения в VP9 и AV1, а QuickTime и айфон такие файлы часто не открывают. Пользователь скачал видео, кликнул — а оно не играет. Поэтому Качалка всегда просит H.264 и звук AAC: "format": "bv*+ba/b", "format_sort": ["vcodec:h264", f"res:{height}", "acodec:m4a"], "merge_output_format": "mp4", sourcecraft.dev Тонкость с res: yt‑dlp считает разрешение по меньшей стороне. Поэтому вертикальный TikTok 1080×1920 честно считается 1080 p и не отсекается фильтром «не больше 1080». Если бы я фильтровал по height, вертикальные видео скачивались бы в плохом качестве. Цена решения — нет 4K: в H.264 его практически никто не отдаёт. Перекодировать 4K на компьютере пользователя — минуты ожидания, для простой качалки это лишнее. Баг 1: TikTok пускает Python, но не пускает собранную программу Из кода всё качалось. Собранная программа на той же ссылке падала: ERROR: [TikTok] 7698495498537143604: Unexpected response from webpage request sourcecraft.dev Я последовательно исключал причины: Сеть и VPN. VPN работает на весь компьютер, системного прокси нет. И скрипт, и программа ходят в сеть одинаково. Окружение. Программа, запущенная из терминала, а не из Finder, падала так же. Версия yt‑dlp. В логе сборки GitHub та же версия, что у меня, 2026.08.19. Состав библиотек. Почти совпадал. Остался Python. Локально у меня одна сборка Python, на GitHub — другая, с другими настройками TLS. По всей видимости, TikTok смотрит на «почерк» TLS‑соединения и одной сборке отдаёт нормальную страницу, а другой — заглушку. yt‑dlp, кстати, сам об этом предупреждал, а я не обратил внимания: WARNING: [TikTok] The extractor is attempting impersonation, but no impersonate target is available. sourcecraft.dev Решение — пакет curl_cffi: он умеет подключаться с отпечатком настоящего Chrome или Safari. Достаточно поставить yt‑dlp с нужным дополнением и не забыть положить пакет в сборку: requirements.txt yt-dlp[default,curl-cffi] build.py "--collect-all=curl_cffi", sourcecraft.dev Главный урок: проверять надо собранную программу, а не код. Локально ошибка не воспроизводилась ни разу. Я собрал пробную версию в отдельной ветке, скачал её из GitHub Actions и только после этого выпустил исправление. Баг 2: видео из TikTok открывается как музыка TikTok начал качаться, но в Finder вместо превью у файла была нота, а QuickTime видел в нём только звук. При этом ffprobe показывал нормальный H.264 1080×1080, а ffmpeg декодировал кадры без единой ошибки. Странным было одно поле — масштаб времени видеодорожки: stream|codec_type=video|time_base=1/5995319 sourcecraft.dev Почти шесть миллионов делений в секунду. Я спросил у видеодвижка macOS напрямую, что он видит в файле: asset = AVFoundation.AVURLAsset.URLAssetWithURL_options_(url, None) len(asset.tracksWithMediaType_(AVFoundation.AVMediaTypeVideo)) исходный файл — 0 видеодорожек после переупаковки — 1 sourcecraft.dev Ноль. Для macOS видео в этом файле просто не существовало. Лечится переупаковкой без перекодирования: качество то же, работы на пару секунд. ffmpeg -i in.mp4 -map 0 -c copy -video_track_timescale 90000 -movflags +faststart out.mp4 sourcecraft.dev Теперь Качалка делает это после каждой загрузки. На Windows ffmpeg запускается с флагом CREATE_NO_WINDOW, иначе на секунду мелькает чёрное окно консоли. Плавный прогресс, когда файлов два YouTube отдаёт видео и звук отдельно, и прогресс yt‑dlp идёт дважды от 0 до 100%. Пользователь видит, как полоска доходит до конца и прыгает в ноль. Поэтому перед загрузкой Качалка смотрит размеры обеих частей и считает общий прогресс с весами. Полоска не прыгает к цели, а плавно догоняет её — каждые 16 мс проходит 12% оставшегося пути. Последние 3% оставлены на склейку. Человечек в офисе Гифка нарисована кодом на Pillow, без готовых ассетов. Холст 160×80 «пикселей» увеличивается в три раза без сглаживания. Офис — это кусок шириной 96 пикселей, который едет на 2 пикселя за кадр: за 48 кадров он проезжает ровно свою ширину, поэтому петля бесшовная. 48 кадров по 40 мс дают 25 кадров в секунду. Ещё одна мелочь для Retina: CustomTkinter сам масштабирует картинки и размывает пиксель‑арт. Поэтому кадры заранее увеличиваются в два раза «по пикселям», и масштабировать окну уже нечего. Сборка для двух систем без Windows Одна матрица GitHub Actions: windows-latest собирает однофайловый exe, macos-latest — .app в zip. По тегу v* оба файла уходят в релиз, а к релизу прикладывается подпись происхождения. Любой может проверить, что файл собран из этого репозитория: gh attestation verify Kachalka-Windows.exe --owner kozliq sourcecraft.dev Две вещи, которые пришлось объяснять в README: Программа без платной подписи. Windows показывает «Система Windows защитила ваш компьютер», macOS — «не удаётся проверить разработчика». Это обходится за один шаг, но предупредить нужно. Блокировки. Многие обходят их VPN‑расширением или DNS в настройках браузера. Программа настроек браузера не видит, и YouTube для неё недоступен. Поэтому сырые ошибки yt‑dlp переводятся на человеческий язык: вместо getaddrinfo failed Качалка пишет «Нет связи с сайтом — похоже, он заблокирован у провайдера» и советует включить VPN для всего компьютера. Что дальше В планах значок в строке меню Mac: нажал, вставил ссылку, скачал — без большого окна. Код и готовые файлы для Windows и Mac: kozliq/kachalka Если что‑то не качается — заведите issue со ссылкой, разберусь.

Most sites that block curl only check its name --- title: "Most sites that block curl only check its name" published: true description: "I sent curl to the top 10,000 sites under a dozen names. Two in three that 403 curl let it in as something else, and Chrome isn't the best disguise." tags: "security, web, curl, scraping" canonical_url: "blog.peng.ly" cover_image: "blog.peng.ly" ai_assisted: true --- Here's chess.com this morning, with the same curl asking five times under different names: $ for ua in curl/8.18.0 Curl/8.18.0 python-requests/2.32.5 Mozilla/5.0 \ 'myfetcher/1.0 (+https://example.org/bot)'; do curl -sL -o /dev/null -w "%{http_code} $ua\n" -A "$ua" https://chess.com/ done 403 curl/8.18.0 200 Curl/8.18.0 403 python-requests/2.32.5 200 Mozilla/5.0 200 myfetcher/1.0 (+https://example.org/bot) A capital C gets you in. That one's chess.com being a bit lazy, and most sites aren't that easy, but the handshake is identical evey time, so the only thing it can be looking at is the name. That's not what you'll usually hear. Ask why curl gets a 403 and the answer is TLS fingerprinting: big sites look at how your client shakes hands, curl looks nothing like Chrome, so changing the User-Agent won't get you far and you want curl-impersonate or a headless browser. Even Daniel Stenberg, who started curl, calls the User-Agent check "the original and much simpler way" of spotting bots. That's true for some sites. I wanted to know how many, so I went through the Tranco top 10,000. Two in three I sent four clients at every homepage on 18 October, from my server: stock curl 8.18, the same curl with a Chrome 141 User-Agent and Chrome's Accept headers, the same curl calling itself myfetcher/1.0, and curl_cffi pretending to be Chrome right down to the TLS handshake. The fake Chrome got a proper page from 5,874 of the 10,000. Of the rest, 1,508 blocked even that, which I'd guess is mostly about my server being in a datacentre, and most of the others just didn't answer. So read everything below as "sites that'll talk to a datacentre at all". The strictest ones aren't in it. Stock curl got blocked at 703 of those 5,874, nearly all with a 403, and about two in five of those a Cloudflare challenge page. Give it Chrome's User-Agent and 451 of the 703 hand over the full page, which is 64%. The made-up myfetcher gets into 421, so 60%. stanford.edu, un.org, cnbc.com, zendesk.com and flightradar24.com are all in there. It's not a fluke of the day either. Of 453 sites that only blocked the name in runs on 17 October, 441 still did today. A sample of inner pages and page titles from those runs matched what Chrome got, so these aren't stripped-down pages being waved through. The closest count I could find is a paper on headless Chrome, where swapping HeadlessChrome for Chrome cleared 75% of the blocks that only hit headless Chrome. That's a real browser already, though, and I couldnt find anyone who'd tried it with curl. So which names get in? I took the 451 sites that let the Chrome User-Agent through and tried ten User-Agents on each, going by the status code this time: !Sites that let each User-Agent in Sites that let each User-Agent in (the data) _Share of the 451 sites that block curl but let a Chrome User-Agent in, 18 October_ python-requests, Wget and Go's default client get turned away almost as often as curl, and so does sending no User-Agent at all. 96 of the sites seem to block "curl" anywhere in the string, as myfetcher/1.0 (curl) gets turned away where the honest one doesn't. Only 55 are like chess.com and miss Curl with a capital. The 2% for plain curl is seven sites that had started letting it in by then. I didn't expect Mozilla/5.0 on its own to do so badly. It's what alot of advice tells you to use, and it only gets into a third of them. Something that reads like a polite bot gets into 85% of these 451, and Miniflux's feed reader string 88%. Feed readers seem fine generally, for what it's worth: of 153 feeds advertised on the top 2,000 homepages, 7 turned Miniflux away. So if your script gets a 403, give it a name first. Anything that isn't a tool's default will do, and a URL in it is polite: curl -A 'myfetcher/1.0 (+https://example.org/bot)' https://example.com/ Does it matter who's in front? Quite a lot. I read the headers on each block to see whose edge sent it. That tells you who's in front of the site, not who wrote the rule, since a Cloudflare customer can block curl in thier own firewall rules without Cloudflare having any say. Here's the share of curl blocks that the Chrome User-Agent and myfetcher each got past (AWS is CloudFront and friends): | In front | Blocks | Chrome | myfetcher | | ---------- | -----: | -----: | --------: | | Cloudflare | 392 | 77% | 78% | | Akamai | 130 | 23% | 28% | | AWS | 57 | 82% | 32% | | Others | 124 | 60% | 51% | Behind Cloudflare it's mostly a name check, and any other name will do. Behind Akamai it's mostly the fingerprinting people warn you about, and three in four still won't give curl a page with a Chrome User-Agent on. Behind AWS it wants something that looks like a browser: Mercado Libre's sites, realtor.com and osha.gov let Chrome's string in and turn myfetcher away. Akamai's also why the middle of the list does worse. Of the sites ranked 2,001 to 5,000 that block curl, only 46% let the Chrome string in, because Akamai's in front of 50 of those 169. Further down, from 5,001 to 10,000, it's in front of 55 of 401, and the Chrome string gets into 71%. What about from somewhere else? All of that is from one server in a datacentre. Last night I ran the 157 sites that blocked anything in the top 2,000 again through Cloudflare's WARP VPN, and through Tor. The Chrome User-Agent did a bit better on WARP (81% of curl blocks) and the same on Tor (72%) as from my server. About half the Akamai sites that blocked even the Chrome string from my server only checked the name on WARP, so the address counts as well as the name. I haven't tried it from home broadband, so that's the bit I can't tell you. Don't fake Chrome by halves There's a catch with the Chrome string. 79 sites served stock curl fine, then blocked the same curl as soon as it said it was Chrome. adobe.com, slack.com, cisco.com, intel.com, cdc.gov and opensource.org are on that list, and so is w3.org, so the W3C, who write a good chunk of the web's standards, will serve curl all day, just not curl claiming to be Chrome. 32 of the 79 are behind Akamai. My guess is the rule there is "says Chrome, doesn't shake hands like Chrome", which is exactly what curl with a Chrome User-Agent is. 61 of them still let myfetcher in. And it's partly the address again. 12 sites in the top 2,000 did this to my server. Over Tor, all seven I could reach still did, but over WARP only 3 of the 12 did. So try the honest name first, and save the Chrome string for when that fails, or for anything behind AWS.

For me, curl-impersonate [0] works great. It makes curl fingerprint identically to Chrome/Firefox. [0] lexiforest/curl-impersonate