How to Scrape TikTok User Data with Go and Goroutines

Why Go for TikTok Scraping at Scale

If you are building a data pipeline that pulls TikTok user profiles, posts, or follower lists at scale, Go is hard to beat. Goroutines make it trivial to run hundreds of concurrent HTTP calls without the memory overhead of threads, the standard library ships with everything you need (net/http, encoding/json, context, encoding/csv), and the final artifact is a single static binary you can drop on any Linux box - no interpreter, no virtualenv, no surprises.

This tutorial walks through building a production-grade TikTok scraper in Go using TikLiveAPI. We will cover struct design for the real response shapes (which mix camelCase and snake_case), pagination with cursors and timestamps, retry with exponential backoff, and a bounded worker pool for concurrent fetching. At the end you will have a daily follower tracker that writes to CSV - a real piece of code you can run as a cron job.

Prerequisites

• Go 1.21 or newer (we use log/slog and the modern errors package)
• A TikLiveAPI account - sign up and grab 100 free credits to verify your email
• Familiarity with structs, interfaces, and goroutines

The only external dependency we use is golang.org/x/sync/errgroup and golang.org/x/sync/semaphore for the worker pool. Everything else is standard library.

go mod init github.com/you/tiktok-scraper
go get golang.org/x/sync/errgroup golang.org/x/sync/semaphore

Step 1: Get Your API Key

After verifying your email, copy the API key from your profile dashboard. Do not hardcode it - export it as an environment variable:

export TIKLIVE_API_KEY="your_key_here"

All requests authenticate via the X-Api-Key header. The base URL is https://api.tikliveapi.com and every successful call costs one credit. See pricing for credit packages.

Step 2: Define Structs That Match Real Response Shapes

This is the part most tutorials get wrong. TikLiveAPI responses mix conventions: top-level keys are camelCase (hasMore) but nested video items are snake_case (play_count, digg_count). The /userinfo-by-username/ endpoint returns camelCase user and stats objects (uniqueId, followerCount), while /user-posts/ returns a videos array with snake_case fields. Get the struct tags wrong and you get zero values.

Create tiktok/types.go:

package tiktok

// UserInfoResponse is the shape returned by /userinfo-by-username/ and /userinfo-by-id/
type UserInfoResponse struct {
    User  User  `json:"user"`
    Stats Stats `json:"stats"`
}

// User uses camelCase per the API truth map
type User struct {
    ID             string `json:"id"`
    UniqueID       string `json:"uniqueId"`
    Nickname       string `json:"nickname"`
    AvatarThumb    string `json:"avatarThumb"`
    AvatarMedium   string `json:"avatarMedium"`
    AvatarLarger   string `json:"avatarLarger"`
    SecUID         string `json:"secUid"`
    Signature      string `json:"signature"`
    Verified       bool   `json:"verified"`
    PrivateAccount bool   `json:"privateAccount"`
    OpenFavorite   bool   `json:"openFavorite"`
    CommentSetting int    `json:"commentSetting"`
}

// Stats also uses camelCase
type Stats struct {
    FollowingCount int64 `json:"followingCount"`
    FollowerCount  int64 `json:"followerCount"`
    HeartCount     int64 `json:"heartCount"`
    VideoCount     int64 `json:"videoCount"`
    DiggCount      int64 `json:"diggCount"`
    Heart          int64 `json:"heart"`
}

// UserIDResponse is the dead-simple shape from /userid/
type UserIDResponse struct {
    ID string `json:"id"`
}

// PostsResponse is what /user-posts/ returns
// Note: hasMore is camelCase, but Videos items are snake_case
type PostsResponse struct {
    Videos  []Video `json:"videos"`
    Cursor  string  `json:"cursor"`
    HasMore bool    `json:"hasMore"`
}

// Video uses snake_case fields per the truth map
type Video struct {
    AwemeID       string `json:"aweme_id"`
    VideoID       string `json:"video_id"`
    Region        string `json:"region"`
    Title         string `json:"title"`
    Cover         string `json:"cover"`
    OriginCover   string `json:"origin_cover"`
    Duration      int    `json:"duration"`
    Play          string `json:"play"`     // no-watermark URL
    Wmplay        string `json:"wmplay"`   // watermarked
    PlayCount     int64  `json:"play_count"`
    DiggCount     int64  `json:"digg_count"`
    CommentCount  int64  `json:"comment_count"`
    ShareCount    int64  `json:"share_count"`
    DownloadCount int64  `json:"download_count"`
    CollectCount  int64  `json:"collect_count"`
    CreateTime    int64  `json:"create_time"`
    IsTop         int    `json:"is_top"`
}

Note the cursor type. The API returns cursor values as strings in some envelopes and numbers in others - using string with explicit conversion when sending it back as a query param keeps things simple. If you hit a decode error, switch to json.RawMessage and inspect.

Step 3: Fetch User Info

Build a thin client that holds the API key and an *http.Client with a sane timeout. The http.DefaultClient has no timeout, which is a footgun in production.

package tiktok

import (
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
    "net/url"
    "time"
)

const BaseURL = "https://api.tikliveapi.com"

type Client struct {
    APIKey string
    HTTP   *http.Client
}

func New(apiKey string) *Client {
    return &Client{
        APIKey: apiKey,
        HTTP: &http.Client{
            Timeout: 30 * time.Second,
        },
    }
}

func (c *Client) get(ctx context.Context, path string, params url.Values, out any) error {
    u := BaseURL + path
    if len(params) > 0 {
        u += "?" + params.Encode()
    }

    req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
    if err != nil {
        return fmt.Errorf("build request: %w", err)
    }
    req.Header.Set("X-Api-Key", c.APIKey)
    req.Header.Set("Accept", "application/json")

    resp, err := c.HTTP.Do(req)
    if err != nil {
        return fmt.Errorf("http do: %w", err)
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        return fmt.Errorf("read body: %w", err)
    }

    if resp.StatusCode != http.StatusOK {
        return fmt.Errorf("status %d: %s", resp.StatusCode, string(body))
    }

    if err := json.Unmarshal(body, out); err != nil {
        return fmt.Errorf("decode %s: %w", path, err)
    }
    return nil
}

func (c *Client) UserInfo(ctx context.Context, username string) (*UserInfoResponse, error) {
    var out UserInfoResponse
    err := c.get(ctx, "/userinfo-by-username/", url.Values{"username": {username}}, &out)
    if err != nil {
        return nil, err
    }
    return &out, nil
}

Usage:

client := tiktok.New(os.Getenv("TIKLIVE_API_KEY"))
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()

info, err := client.UserInfo(ctx, "tiktok")
if err != nil {
    log.Fatal(err)
}
fmt.Printf("%s has %d followers\n", info.User.UniqueID, info.Stats.FollowerCount)

Step 4: Resolve the Numeric User ID

Most post-listing endpoints want the numeric userid, not the @handle. The /userid/ endpoint returns a flat {"id": "..."}:

func (c *Client) UserID(ctx context.Context, username string) (string, error) {
    var out UserIDResponse
    err := c.get(ctx, "/userid/", url.Values{"username": {username}}, &out)
    if err != nil {
        return "", err
    }
    return out.ID, nil
}

Step 5: Get User Posts

The /user-posts/ endpoint takes userid, an optional count (default 10, max 35), and an optional cursor. The response wraps videos with a camelCase hasMore flag and a cursor to feed back in.

func (c *Client) UserPosts(ctx context.Context, userID, cursor string, count int) (*PostsResponse, error) {
    params := url.Values{"userid": {userID}}
    if cursor != "" {
        params.Set("cursor", cursor)
    }
    if count > 0 {
        params.Set("count", fmt.Sprintf("%d", count))
    }

    var out PostsResponse
    if err := c.get(ctx, "/user-posts/", params, &out); err != nil {
        return nil, err
    }
    return &out, nil
}

Step 6: Pagination with a Channel

Idiomatic Go pagination uses a goroutine that pushes results onto a channel. Callers can range over it and break early. The producer closes the channel when hasMore goes false:

// StreamUserPosts yields all posts page by page until hasMore is false.
func (c *Client) StreamUserPosts(ctx context.Context, userID string) (<-chan Video, <-chan error) {
    videos := make(chan Video, 64)
    errCh := make(chan error, 1)

    go func() {
        defer close(videos)
        defer close(errCh)

        cursor := ""
        for {
            page, err := c.UserPosts(ctx, userID, cursor, 35)
            if err != nil {
                errCh <- err
                return
            }
            for _, v := range page.Videos {
                select {
                case videos <- v:
                case <-ctx.Done():
                    errCh <- ctx.Err()
                    return
                }
            }
            if !page.HasMore {
                return
            }
            cursor = page.Cursor
        }
    }()

    return videos, errCh
}

One gotcha: /user-followers/ and /user-following/ do NOT use a cursor param. They paginate via a time timestamp parameter, and /user-following/'s top-level key is followings (plural with trailing s), not following. Adjust your structs accordingly. See the users documentation for full schemas.

Step 7: Timeouts and Exponential Backoff

Networks fail. The API might hit a transient 502 or a rate cap. Wrap get with retry logic that respects context.Context and uses exponential backoff with jitter - no external retry library needed:

import (
    "errors"
    "math/rand"
    "time"
)

func (c *Client) getWithRetry(ctx context.Context, path string, params url.Values, out any) error {
    const maxAttempts = 4
    baseDelay := 500 * time.Millisecond

    var lastErr error
    for attempt := 0; attempt < maxAttempts; attempt++ {
        if attempt > 0 {
            jitter := time.Duration(rand.Int63n(int64(baseDelay)))
            backoff := baseDelay*time.Duration(1<

package main

import (
    "context"
    "log"

    "golang.org/x/sync/errgroup"
    "golang.org/x/sync/semaphore"
)

func FetchAll(ctx context.Context, client *tiktok.Client, usernames []string) (map[string]*tiktok.UserInfoResponse, error) {
    const concurrency = 20
    sem := semaphore.NewWeighted(concurrency)
    g, gctx := errgroup.WithContext(ctx)

    results := make(map[string]*tiktok.UserInfoResponse, len(usernames))
    var mu sync.Mutex

    for _, u := range usernames {
        u := u
        if err := sem.Acquire(gctx, 1); err != nil {
            break
        }
        g.Go(func() error {
            defer sem.Release(1)
            info, err := client.UserInfo(gctx, u)
            if err != nil {
                log.Printf("fetch %s: %v", u, err)
                return nil // keep going on individual failures
            }
            mu.Lock()
            results[u] = info
            mu.Unlock()
            return nil
        })
    }
    if err := g.Wait(); err != nil {
        return results, err
    }
    return results, nil
}

package main

import (
    "context"
    "encoding/csv"
    "fmt"
    "os"
    "strconv"
    "time"

    "github.com/you/tiktok-scraper/tiktok"
)

func main() {
    apiKey := os.Getenv("TIKLIVE_API_KEY")
    if apiKey == "" {
        fmt.Fprintln(os.Stderr, "TIKLIVE_API_KEY not set")
        os.Exit(1)
    }

    usernames := []string{"tiktok", "khaby.lame", "charlidamelio", "mrbeast", "bellapoarch"}

    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
    defer cancel()

    client := tiktok.New(apiKey)
    results, err := FetchAll(ctx, client, usernames)
    if err != nil {
        fmt.Fprintf(os.Stderr, "fetch: %v\n", err)
        os.Exit(1)
    }

    f, err := os.OpenFile("followers.csv", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
    if err != nil {
        fmt.Fprintf(os.Stderr, "open csv: %v\n", err)
        os.Exit(1)
    }
    defer f.Close()

    // Write header if file is new
    info, _ := f.Stat()
    w := csv.NewWriter(f)
    defer w.Flush()
    if info.Size() == 0 {
        w.Write([]string{"date", "username", "follower_count", "video_count", "heart_count"})
    }

    today := time.Now().UTC().Format("2006-01-02")
    for username, r := range results {
        w.Write([]string{
            today,
            username,
            strconv.FormatInt(r.Stats.FollowerCount, 10),
            strconv.FormatInt(r.Stats.VideoCount, 10),
            strconv.FormatInt(r.Stats.HeartCount, 10),
        })
    }
}