TGScope 如何以約 90 KB 首屏資料載入 300 萬個 Telegram 公開頻道地圖
How we open a map of 3 million Telegram channels with a 90 KB download
TGScope 建成一張涵蓋 300 萬個 Telegram 公開頻道的語義地圖,首屏載入約 90 KB 資料,其他內容會隨縮放逐步載入。地圖以四叉樹切分資料,每個 tile 最多容納 4,096 個頻道,頻道名稱則由伺服器按懸停、點擊或深度縮放時查詢。每個頻道點位以 8 bytes 編碼,並由 WebGL 2 每個 tile 一次繪製。
At TGScope we built a semantic map of 3 million public Telegram channels. Channels that talk about similar things sit next to each other, so the whole ecosystem is visible at once: from big regions like crypto or news down to tiny niches.
Try it: tgscope.io/map
We had one hard requirement: the map has to be usable within a second, on any device, phones included. That rules out the obvious approach of shipping every point to the browser. This post is about what we did instead.
The first-frame budget
To draw the first frame the browser fetches three files:
File Size
manifest.json (tile index, categories, languages) 1.3 KB
labels.json (area names for three zoom levels) 59 KB
root tile (the largest channels) 32 KB
About 90 KB in total. Everything else streams in as you zoom.
Quadtree tiles, like a slippy map
The points are cut into a quadtree, the same idea OpenStreetMap uses for its tiles. Each tile holds at most 4,096 channels. The root tile holds the biggest channels, which is enough to show the overall shape of the map. Smaller channels live in deeper tiles, and the client only requests tiles that intersect the viewport and sit no more than two levels below the current zoom. New tiles fade in, so nothing pops.
A tile is a flat binary buffer. No JSON, no names, no descriptions:
Bytes Field
0–1 x inside the tile, uint16
2–3 y inside the tile, uint16
4 category
5 language
6 subscriber tier (7 tiers, 1K to 3M+)
7 padding
A full tile is 32 KB, and decoding it is just reading typed arrays:
const u16 = new Uint16Array(buf);
const u8 = new Uint8Array(buf);
for (let i = 0; i < n; i++) {
x[i] = (tx + u16[i * 4] / 65535) / k;
y[i] = (ty + u16[i * 4 + 1] / 65535) / k;
cat[i] = u8[i * 8 + 4];
lang[i] = u8[i * 8 + 5];
tier[i] = u8[i * 8 + 6];
}
Names stay on the server
A dot only knows its position, category, language and size. Which channel is behind it is the server's business: the client asks when you hover or click, and at deep zoom it requests names for the visible area in one batch.
This keeps tiles tiny, and as a side effect the catalog can't be downloaded as one file. To collect it you would have to crawl the whole map at maximum zoom.
WebGL 2: one draw call per tile
Each tile is uploaded to the GPU once and drawn with a single gl.drawArrays(gl.POINTS, ...). Colors, sizes and the language filter are computed in the vertex shader:
bool on = u_lang < 0 || lang == u_lang;
float a = (on ? 0.88 : 0.07) * u_alpha;
gl_PointSize = u_sizes[tier] * u_dpr * (on ? 1.0 : 0.8);
Switching the language filter doesn't touch the data at all. We change one uniform, and every dot on the screen repaints in the same frame.
Warm start
Channel pages on TGScope show a small preview of the map. While you look at it, the map code, the manifest and the labels are already downloading in the background. By the time you click "open map", almost everything is in the cache.
What's on the map
3 million public channels, 24 categories, 24 languages
three levels of area labels: 25 regions, 250 topics, 2,500 sub-topics
search by meaning: describe what you want instead of guessing a channel name
a language filter, a channel card on click, and a shareable link to any spot on the map
The layout itself comes from a multilingual embedding model and UMAP, but that's a topic for another post.
Open the map and tell me in the comments what you'd change in the tile format. I'm curious whether anyone has tried something smaller than 8 bytes per point.
來源:DEV Community · dev.to


