Scraping TikTok's Mobile API

Technical guide · 24 endpoints · Measured TikTok's Android app talks to a private HTTP+JSON API that is faster than the web endpoints and returns considerably more. This is a technical guide to reaching it: how devices are registered, how requests are signed, how the regional hosts are partitioned, and how the TLS handshake is fingerprinted. A system built on it collected 3.23 billion creator profiles, 5.94 billion videos and 2.8 billion comments in three weeks. Get the full code Free dataset. I uploaded 4.5 billion of those videos to Hugging Face: captions, view, like, comment and save counts, the sound, the country and the posting time. huggingface.co/datasets/kuben-developer/tiktok-videos-4b What you can pull. Creator profiles, every video a creator has posted, followers and following lists, and TikTok's own similar-creator graph. Full video detail with the complete statistics block. Comments and comment replies, each with the commenter's account. Sounds, the videos using them, and the trending sounds chart. Hashtags and their videos, newest or most popular. Keyword search across videos, creators and sounds. Trending shelves and camera effects. 24 endpoints in all, each with a measured success rate. What this is Almost every TikTok scraper you will find drives a headless browser or hits the public web endpoints. Both are the wrong layer: slow, fragile, and missing most of the interesting fields. The Android app does not use either. It talks to a private HTTP+JSON API, the same one com.zhiliaoapp.musically hits when you scroll, and that API is fast, stable, and returns far more. Getting into it is the hard part, and it is hard in a specific way. Four completely unrelated things have to be right at once: a device credential TikTok issued, a valid request signature, the correct regional host, and a TLS handshake that looks like a phone. Get any one of them wrong and you receive the identical response: a clean HTTP 200 with an empty body. No error message. No status code. Your HTTP client reports success, your logs stay green, and your database fills with nothing. There is no signal telling you which of the four you are standing at. This article walks through all four, then documents the 24 endpoints that come out the other side. It names the primitives, shows the real pipeline and gives measured numbers rather than claims. None of the four has a feedback loop. A wrong rotation constant, a wrong byte order, a wrong host and a wrong cipher suite in the handshake all produce the same well-formed request and the same empty response, so there is no error to bisect on and no partial credit. Scope Everything below is anonymous device traffic. There is no login anywhere in this system, no account, no session cookie. That also means anything genuinely account-gated (your own DMs, private videos, who liked what) is out of reach and stays out of reach. No amount of tuning gets you there. Anatomy of a request Before anything else, here is what one of these requests actually looks like. This is a real call, with the identifying values shortened: GET /aweme/v1/aweme/post/ ?# ── what you are asking for ────────────────────────────── source=0 &user_id=6744630345964389381 &count=20 &max_cursor=1751028792000 &sort_type=0 &# ── who is asking: 38 params, order matters ───────────── ts=1788361402&ac=mobile&ac2=lte &aid=473824 # app id: TikTok Lite &iid=7680617333853718293 # install id ← from register &device_id=7680616891110524437 # device id ← from register &cdid=4a1d... # client-generated uuid &openudid=8f2c... # client-generated 16-hex &device_brand=Samsung&device_type=SM-A136U&os_version=12&os_api=30 &resolution=1080*2280&dpi=440&host_abi=arm64-v8a &region=SG&carrier_region=SG&sys_region=SG&mcc_mnc=52506 &language=ja&app_language=ja&locale=ja-SG&timezone_name=Asia%2FSingapore &version_name=32.8.2&version_code=320820&manifest_version_code=320820 &_rticket=1788361402193&channel=googleplay&app_type=normal Headers: user-agent: com.ss.android.ugc.tiktok.lite/320802 (Linux; U; Android 12; ...) x-tt-trace-id: 00-6a9f...-6a9f...-01 x-ss-req-ticket: 1788361402193 x-khronos: 1788361402 # timestamp x-ladon: XKp9... # Speck-128/256 x-argus: cQqbRZm8k1x... # the hard one x-gorgon: 0404b0d30000... # legacy digest A single creator-timeline request. Everything below the fold is device identity. Three things to notice, because each one bites later: Two thirds of the URL is device identity. Thirty-eight common parameters describe the handset, the carrier, the region and the app build. They are not decoration. The signature covers them. device_id and iid are issued by TikTok, not chosen by you. cdid and openudid you generate and submit at registration. Getting the distinction wrong is the first wall. Parameter order is fixed. The signature hashes the query string as a literal, so url.Values.Encode(), which sorts keys alphabetically, silently produces an invalid signature. In Go you have to build the query by hand. The vocabulary, since it recurs throughout: FieldWhat it isOrigin aidApplication id. 1233 is the main app (musically), 473824 is Lite, 1340 is musically_go. Different aid means a different signing key and a different endpoint set.Constant device_idThe durable device identity. 19 digits.TikTok, at register iidInstall id. Pairs with device_id.TikTok, at register cdidClient device id. A UUID you generate.You openudid16 hex characters you generate.You license_idFeeds the X-Ladon key schedule.Constant per app version_codeApp build. Gates which endpoints answer at all.You choose The first empty 200 A correctly implemented signer produces output that verifies against captured traffic, with parameters matching byte for byte. The response is still this: $ curl -sD- -o /tmp/body "https://api16-normal-c-alisg.tiktokv.com/aweme/v1/user/profile/other/?..." HTTP/1.1 200 OK content-type: application/json content-length: 0 x-tt-logid: 2026090117... server: TLB $ wc -c /tmp/body 0 /tmp/body Two hundred. Zero bytes. No status_code, because there is no body to put one in. This is TikTok's soft block, and it is the single most important thing to understand about this API. It is not a 403. It is not a 429. It is not a challenge page. It is a successful HTTP response containing nothing. Which means this code, which is what everyone writes first, is silently broken: res = requests.get(url, headers=signed) if res.ok: # True. Always true. store(res.json()) # {} stored, no exception # six hours later: 400,000 rows in the database, all empty, # nothing in the error log, dashboard green It is expensive to debug because four unrelated failures produce it: Your device was never activated (§ activation) Your signature is wrong (§ X-Argus) You are talking to the wrong regional host (§ regions) Your TLS handshake looks like a server, not a phone (§ JA3) There is nothing in the response to tell you which. You cannot bisect it by reading errors, because there are none. The only way through is to fix all four and measure each one in isolation. Where device IDs come from You cannot invent a device_id. TikTok issues it, from /service/2/device_register/ on its logging host, in exchange for a plausible handset. The request body is a JSON document (app header, device header, custom block) encrypted with TTEncrypt (TikTok's own body cipher, a simple byte-level transform with a fixed key schedule) and posted as application/octet-stream;tt-data=a. It goes out with the full signature set, so you need working signing before you can get a device, and the signing needs a device. You bootstrap with the client-generated fields and zeros where the issued ones go. The body's shape, with the parts that matter: { "magic_tag": "ss_app_log", "header": { // app identity: must agree with the aid in the query string "aid": 473824, "package": "com.ss.android.ugc.tiktok.lite", "app_version": "32.8.2", "version_code": 320820, "sdk_version": "...", "git_hash": "...", "sig_hash": "...", // hardware: every field here has to be internally consistent "device_model": "SM-A136U", "device_brand": "Samsung", "device_manufacturer": "samsung", "cpu_abi": "arm64-v8a", "os_version": "12", "os_api": 30, "resolution": "2280*1080", "density_dpi": 440, "rom": "...", "rom_version": "...", // identity you generate and are about to trade in "cdid": "<uuid4>", "openudid": "<16 hex>", "clientudid": "<uuid4>", "google_aid": "<uuid4>", // region: carrier must plausibly exist in this country "region": "SG", "sim_region": "sg", "carrier": "Singtel", "mcc_mnc": "52506", "tz_name": "Asia/Singapore", "tz_offset": 25200, "custom": { "screen_width_dp": 408, "screen_height_dp": 883, "web_ua": "Dalvik/2.1.0 (Linux; U; Android 12; SM-A136U Build/...)", "apk_last_update_time": 1788361409271 }, "apk_first_install_time": 1788360902118 }, "_gen_time": 1788361402240 } Every field there is checked against the others. A Samsung SM-A136U has a specific screen resolution, a specific DPI, a specific ABI, and shipped with a specific range of Android versions. It is sold on carriers in some countries and not others. A flagship handset on a network that never carried it is not a real phone, and the registration is refused. Rather than generating these procedurally, I build them from a catalogue of ~250 real Android device profiles crossed with a carrier table of MCC/MNC pairs (roughly 2,000 rows, derived from public numbering-plan data). Pick a handset, pick a carrier that actually exists in the target country, fill in the coherent values. A successful registration comes back with the two ids you needed: { "device_id_str": "7680616891110524437", "install_id_str": "7680617333853718293", "new_user": 1 } Most implementations stop here. The activation call With registration working, most endpoints answered. Video listings, search, hashtags, sounds, all fine. But /aweme/v1/user/profile/other/, the full profile record, returned the empty 200 every single time, on every device I made, forever. The obvious suspect is the signature, and it is the