YouTube offers a free official API (Data API v3), but it comes with hard quota limits. Every developer gets 10,000 units per day. A single search costs 100 units, so you're capped at 100 searches daily. Read operations like channel lookups and video details cost 1 unit each, which is more reasonable, but any product combining search with bulk data collection burns through the quota fast.
You can request a quota increase, but it triggers a full compliance audit: legal details, demo credentials, design docs, and detailed usage math. Approval can take days or months depending on how Google prioritizes your case. They can also revoke quota during periodic re-audits. And use cases involving data aggregation, competitive intelligence, or resale tend to get denied outright since they don't fit what Google considers beneficial to the YouTube ecosystem.
Influencer marketing platforms are the primary customer ($25B+ industry). Companies like CreatorIQ, Grin, Upfluence, and Traackr need bulk access to channel stats, engagement data, and content catalogs across thousands of creators daily. They can't rely on official quotas for that volume without a lengthy approval process and ongoing audit risk.
Other segments: social analytics tools (Brandwatch, Sprout Social), AI companies pulling transcripts for training data, and ad agencies verifying creator metrics before sponsorship deals.
Google property. YouTube runs on Google infrastructure. The proxy rotation, session management, and anti-bot experience built across 15+ Google products applies directly.
Vertical depth. 8 endpoints covering channels, videos, comments, transcripts, and trending content. Same approach as the Amazon API (products, reviews, offers, sellers, deals) or Local Business API (30+ fields per entity).
Recurring usage. Influencer platforms continuously monitor thousands of channels, which drives high call volume and sticky revenue. Similar pattern to how e-commerce tools poll the Amazon API for price changes.
Google Ads Intelligence. Scraping paid ads from Google SERPs plus the Ads Transparency Center. Low engineering lift since OpenWeb Ninja already parses these pages.
TikTok Data API. No official public data API. But it requires entirely new scraping infrastructure with no shared foundation from the existing Google stack. Higher risk, higher cost.
ChatGPT Search API. Tracking how brands appear in AI-generated answers (Generative Engine Optimization). No real competitors, but the market barely exists yet. Willingness to pay is unproven and the data is non-deterministic. + Requires entirely new scraping infrastructure
npm install
npm run setup
npm run dev
API runs on http://localhost:3000, frontend on http://localhost:5173. Search any YouTube channel by handle (e.g. @MrBeast, @GothamChess) and browse the profile details.
You can also hit the API directly:
curl http://localhost:3000/youtube/channel?handle=@MrBeast
8 endpoints covering the core YouTube data surface: channel profiles, video metadata, comments, transcripts, search, and trending. Two detailed specs below for /youtube/channel and /youtube/search.
| Method | Path | Description |
|---|---|---|
| GET | /youtube/channel | Channel metadata by handle or channel_id |
| GET | /youtube/channel/videos | Paginated video list for a channel |
| GET | /youtube/search | Search videos, channels, or playlists |
| GET | /youtube/video | Video details by video_id |
| GET | /youtube/video/comments | Paginated comments for a video |
| GET | /youtube/video/transcript | Transcript/captions for a video |
| GET | /youtube/trending | Trending videos by country and category |
| GET | /youtube/shorts | Shorts for a channel or search query |
| Parameter | Type | Required | Description |
|---|---|---|---|
| handle | string | One of handle or channel_id | e.g. @MrBeast |
| channel_id | string | One of handle or channel_id | e.g. UCX6OQ3DkcsbYNE6H8uQQuVA |
| language | string | No | Default: en |
| region | string | No | Default: US |
{
"status": "OK",
"request_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"data": {
"channel_id": "UCX6OQ3DkcsbYNE6H8uQQuVA",
"handle": "@MrBeast",
"name": "MrBeast",
"description": "SUBSCRIBE FOR A COOKIE!",
"subscribers_count": 467000000,
"total_views": 111229443913,
"video_count": 948,
"joined_date": "2012-02-19",
"country": "United States",
"verified": true,
"avatar_url": "https://yt3.googleusercontent.com/...",
"banner_url": "https://yt3.googleusercontent.com/...",
"links": [{ "title": "Beast Games", "url": "https://linktr.ee/beastgames" }],
"keywords": ["mrbeast6000", "beast", "mrbeast"]
}
}| Parameter | Type | Required | Description |
|---|---|---|---|
| query | string | Yes | Search query |
| type | string | No | video, channel, or playlist. Default: video |
| sort_by | string | No | relevance, date, view_count, rating. Default: relevance |
| limit | integer | No | Max 50. Default: 20 |
| page_token | string | No | Continuation token from previous response |
| upload_date | string | No | hour, today, week, month, year |
| duration | string | No | short (under 4 min), medium (4-20), long (over 20) |
{
"status": "OK",
"request_id": "f8e7d6c5-b4a3-2190-fedc-ba0987654321",
"parameters": { "query": "web scraping tutorial", "type": "video", "limit": 2 },
"data": {
"results": [
{
"type": "video",
"video_id": "XVv6mJpFOb0",
"title": "Web Scraping for Beginners",
"channel_name": "Tech With Tim",
"channel_id": "UC4JX40jDee_tINbkjycV4Sg",
"view_count": 2400000,
"publish_date": "2024-03-15",
"duration": "PT22M14S",
"thumbnail_url": "https://i.ytimg.com/vi/XVv6mJpFOb0/hqdefault.jpg"
}
],
"next_page_token": "EpMDEgtN...",
"total_results": 15400
}
}Scaling. YouTube server-renders a JSON blob (ytInitialData) into every public page. Most endpoints can be served with a single HTTP GET, no headless browser. The cost driver is proxy bandwidth, not compute. Channel metadata changes slowly, so even a 5-15 minute cache TTL cuts costs significantly.
Blocking. YouTube uses standard Google anti-bot: TLS fingerprinting, behavioral analysis, rate limiting per IP. One YouTube-specific detail: aggressive scraping of a single channel can trigger soft limits that return truncated data rather than hard blocks. Residential proxy rotation and small random delays between paginated requests handle this.
Limitations. Responses are real-time snapshots, no historical data. Subscriber counts come in abbreviated form ("467M") so the parsed integer is an approximation. Private and unlisted content is excluded. Age-restricted videos need authenticated sessions, better handled as a separate tier. Country and external links require a secondary continuation request to YouTube's browse API, adding roughly 200ms per call.
Implemented the /youtube/channel endpoint. Channel is the core entity that every other endpoint references, so it made sense as the starting point. NestJS v11, TypeScript, no external HTTP libraries.
cd api && npm install && npm run start
curl http://localhost:3000/youtube/channel?handle=@MrBeast
Fetches the real YouTube page, extracts the ytInitialData JSON from the HTML source, and parses it into a structured response. Falls back to demo data if YouTube is unreachable.
The project includes a stealth module that models how requests would work in production. Three services handle the pipeline:
- FingerprintService generates browser fingerprints from a seeded RNG: user agent, viewport, locale, GPU renderer, hardware specs. Deterministic per seed so the same channel always gets the same identity.
- ProxyService creates proxy sessions with geo-targeted credentials. In production these would route through residential proxies; here they return mocked connection strings.
- SessionService ties both together. Each outbound request goes through a session that carries its own fingerprint and proxy config, mirroring how a real scraping pipeline isolates identity per request.
The services are functional but simplified. Proxy routing is mocked, fingerprint pools are small, and there is no session persistence. The architecture is there so swapping in real infrastructure is a config change, not a rewrite.
Fields like total_views, joined_date, and country require a secondary continuation request to YouTube's browse API. The current implementation returns empty values for those rather than making the extra round-trip.
What I would build next: session pooling with automatic rotation so identities cycle across requests, IP ban detection with re-queue logic when a proxy gets flagged, remaining 7 endpoints, response caching with per-endpoint TTLs, rate limiting middleware, retry with exponential backoff, the browse API continuation for complete channel data, and integration tests.