我如何在 128MB RAM 下構建零代理 Shopify 價格及補貨監察工具,並解決 50 倍計費瓶頸
How I Built a Zero-Proxy Shopify Price & Restock Monitor in 128MB RAM (and Fixed a 50x Billing Bottleneck)
作者以 Node.js 構建 Shopify 價格及補貨監察工具,在 128 MB RAM 下監察每個目標最多 2,500 件產品,無需 Chromium 或代理伺服器。工具以 node:http2 傳輸及每批 50 項的 Pay-Per-Event 計費,將計費延遲降低約 50 倍,並避免超出預算時輸出未付款資料。
Monitoring Shopify competitor stores sounds deceptively simple: hit /products.json, compare prices, done.
In practice, naive monitors fail in two expensive ways:
- They burn gigabytes of RAM running headless Chromium (and residential proxies) on public JSON endpoints that don't even need JS rendering.
- When a store launches 400 variants at once, firing 400 sequential billing or webhook calls inside a tight loop blows past execution timeouts and API rate limits.
Here is how I built a zero-proxy, 128 MB RAM Shopify delta monitor in Node.js — including the two architectural fixes (node:http2 transport and 50-item batched Pay-Per-Event billing) that turned a sluggish scraper into a sub-second monitoring primitive.
1. Bypassing HTTP/1.1 Throttling with Native node:http2 (Zero Dependencies)
Many Shopify storefronts sit behind Cloudflare. If your Node.js worker fires repeated HTTP/1.1 bursts using default TLS/socket settings, you quickly hit 429 Too Many Requests — pushing most developers straight toward expensive residential proxies.
Instead of spinning up Playwright or paying proxy per-GB fees, Node.js already ships a battle-tested built-in node:http2 module plus native node:zlib Brotli/Gzip decompression:
import http2 from 'node:http2';
import zlib from 'node:zlib';
export function decompressBody(buffer, contentEncoding) {
if (!buffer || buffer.length === 0) return buffer;
const enc = String(contentEncoding ?? '').toLowerCase().trim();
if (!enc || enc === 'identity') return buffer;
if (enc.includes('br')) return zlib.brotliDecompressSync(buffer);
if (enc.includes('gzip')) return zlib.gunzipSync(buffer);
if (enc.includes('deflate')) return zlib.inflateSync(buffer);
return buffer;
}
Pair http2.connect(urlObj.origin) with:
- Clean
:method: 'GET',accept-encoding: gzip, deflate, brheaders - Strict
AbortController/ timer cleanup so stalled sockets never leak memory in a 128 MB container - A polite
1500 msdelay between paginated/products.json?limit=250&page=Ncalls plus exponential backoff honoringRetry-After(capped at 20s)
The result? No Chromium binary (~400 MB saved), no proxy middleman ($0 proxy spend), and clean pagination up to 2,500 products per target in 128 MB RAM.
2. Stateful Delta Tracking Without an External Database
Running an external Postgres or Redis cluster just to know whether a sneaker dropped from $110 to $88 creates unnecessary infra overhead.
Instead, persisting compact normalized variant snapshots in a named Apify Key-Value Store (shopify-delta-monitor-state) turns each run into a stateful diff engine:
{
"recordType": "delta",
"eventType": "price_drop",
"store": "https://www.allbirds.com",
"title": "Men's Tree Runners",
"variantTitle": "Kaikoura White / 10",
"sku": "TR3MWHT100",
"oldPrice": 98,
"newPrice": 79,
"priceDiff": -19,
"priceChangePercent": -19.39,
"available": true,
"mapViolation": false,
"productUrl": "https://www.allbirds.com/products/mens-tree-runners",
"detectedAt": "2026-11-21T02:00:00.000Z"
}
We track 8 distinct variant-level delta events:
-
new_product&removed_product -
price_drop&price_increase(configurableminPriceChangePercentnoise floor) -
discount_started&discount_ended(usingcompare_at_price) -
out_of_stock&restocked - Plus optional
mapThresholdsper SKU or product handle (mapViolation: trueif price drops below MAP).
Crucially, the first baseline run stores the full catalog snapshot and emits 0 billable deltas (emitBaselineSummary: true logs a single free baseline_initialized marker). Subsequent runs with zero changes cost $0.
3. Fixing the 50x Pay-Per-Event (PPE) Billing Bottleneck
Early on, calling Actor.charge() individually for every detected delta meant 500 price drops during Black Friday triggered 500 sequential HTTP billing calls (~15–25 seconds of pure network overhead and budget overshoot risk).
Switching to a 50-item batch controller (chargeBatchSize: 50) solved both speed and exact budget enforcement in one sweep:
for (let i = 0; i < items.length; i += 50) {
const chunk = items.slice(i, i + 50);
const chargeRes = await Actor.charge({ eventName: 'delta-event', count: chunk.length });
if (chargeRes?.eventChargeLimitReached && chargeRes?.chargedCount === 0) {
break; // Stop immediately without leaking unbilled records
}
const allowed = typeof chargeRes?.chargedCount === 'number'
? Math.min(chunk.length, chargeRes.chargedCount)
: chunk.length;
const paidChunk = chunk.slice(0, allowed);
if (paidChunk.length > 0) {
await Actor.pushData(paidChunk);
}
if (chargeRes?.eventChargeLimitReached || allowed < chunk.length) {
break;
}
}
That cut billing latency by ~50x while guaranteeing zero unbilled dataset leaks when a user's maxTotalChargeUsd budget stops mid-batch.
Plug Into n8n, Claude/Cursor MCP, or Node.js
If you want to run this immediately without deploying infrastructure:
-
Apify Store Actor (
$2.00 / 1,000real deltas,$0baseline/unchanged): Shopify Competitor Price, Stock & New Product Monitor -
Ready-to-import n8n Slack Alert Workflow + MCP Config + Node/Python Examples: GitHub —
apify-b2b-ecom-automation-kit
Direct import path for n8n: n8n-workflows/01-shopify-price-stock-alert-slack.json — drop in your Apify token, pick your competitor storeUrls, and get Slack alerts only when prices or inventory actually move.
來源:DEV Community · dev.to