如何用 JavaScript 和 Python 從 HTTP 標頭、HTML 和 cookies 偵測網站技術棧
Detect a website's tech stack from headers, HTML and cookies (JS + Python)
Fikri Marhan 提供 JavaScript 和 Python 範例,示範如何從網站的 HTTP 標頭、cookies、generator 標籤、script URL 和 HTML 路徑辨識技術棧。
I wrote this article with AI assistance. I ran every snippet against the live sites shown on 29 Oct 2026, and all output is copied from those runs. Websites change often, so your output will differ.
Browser extensions such as Wappalyzer tell you what a website is built with. They work by looking for clues that the site gives away in every response. You can read most of those clues yourself with a few lines of code.
A site's tech stack is the set of products it runs on. That includes its content management system (CMS), such as WordPress, its JavaScript framework, its analytics tags and its hosting provider.
Who this is for: developers who want to check the tech stack of one site or a list of sites. For example, you might research competitors, qualify sales leads, or audit your own domains for outdated software.
What you need: Node 18.14.1 or later (Node 20 or later recommended) for the JavaScript script, or Python 3.9 or later for the Python script. Neither script needs any packages.
This post covers:
- The five places a site reveals its stack
- A rule-based detector in JavaScript
- A clue printer in Python, for writing your own rules
- Why detection is a guess, not a fact
1. Know the five places a site reveals its stack
One GET request to a homepage returns five kinds of clues:
| Clue | Example | What it often reveals |
|---|---|---|
| Response headers |
server: Vercel, x-generator: Drupal 11
|
Web server, host, CDN, sometimes the CMS |
| Cookies |
_shopify_essential, datadome
|
Ecommerce platform, bot protection, analytics |
The generator meta tag |
<meta name="generator" content="WordPress ..."> |
CMS and often its version |
| Script URLs |
https://cdn.shopify.com/..., .../jquery.min.js
|
Platforms, libraries and third-party tags |
| Paths in the HTML |
/wp-content/, /_next/static/, data-wf-page=
|
CMS and framework file layouts |
A CDN (content delivery network) is a service that caches a site's files on servers near its visitors. Cloudflare is one example.
Each clue is a fingerprint: a pattern that one product leaves behind. A detector is a list of fingerprints and a loop that checks each one.
2. Detect the stack with a rule list (JavaScript)
The script below fetches each site once. It collects the headers, cookie names, the generator tag and the script URLs. Then it checks 24 rules and prints the first piece of evidence for each technology it finds.
Save it as detect.mjs:
// detect.mjs - guess a website's tech stack from one HTTP request.
// Node 18.14.1+ (built-in fetch and getSetCookie). No dependencies. Usage: node detect.mjs example.com [more.com ...]
const RULES = [
// [technology, where to look, pattern]
['WordPress', 'html', /\/wp-content\/|\/wp-includes\//i],
['WordPress', 'generator', /^WordPress/i],
['Drupal', 'header:x-generator', /Drupal/i],
['Drupal', 'generator', /^Drupal/i],
['Shopify', 'script', /cdn\.shopify\.com/i],
['Shopify', 'header:x-shopid', /./],
['Wix', 'header:x-wix-request-id', /./],
['Squarespace', 'html', /static1\.squarespace\.com/i],
['Webflow', 'html', /data-wf-page=/i],
['Next.js', 'html', /\/_next\/static\//],
['Next.js', 'header:x-powered-by', /Next\.js/i],
['Nuxt', 'html', /\/_nuxt\/|window\.__NUXT__/],
['Gatsby', 'generator', /^Gatsby/i],
['Google Tag Manager', 'html', /googletagmanager\.com\/gtm\.js/],
['Google Analytics', 'html', /googletagmanager\.com\/gtag\/js|google-analytics\.com\/analytics\.js/],
['HubSpot', 'script', /js\.hs-scripts\.com|js\.hsforms\.net/i],
['jQuery', 'script', /jquery[.-]?(\d[\d.]*)?(\.min)?\.js/i],
['Cloudflare', 'header:server', /^cloudflare$/i],
['Cloudflare', 'cookie', /^__cf_bm$/],
['Vercel', 'header:server', /^Vercel$/i],
['Netlify', 'header:server', /^Netlify$/i],
['Nginx', 'header:server', /^nginx/i],
['Apache', 'header:server', /^Apache/i],
['PHP', 'header:x-powered-by', /PHP/i],
];
async function detect(site) {
const url = site.startsWith('http') ? site : `https://${site}`;
const res = await fetch(url, {
headers: { 'User-Agent': 'stack-check/1.0 (+https://dev.to/neorhapsody)' },
redirect: 'follow',
signal: AbortSignal.timeout(15000),
});
const html = (await res.text()).slice(0, 2_000_000);
const signals = {
html,
script: [...html.matchAll(/<script[^>]+src=["']([^"']+)/gi)].map((m) => m[1]),
generator: html.match(/<meta[^>]+name=["']generator["'][^>]+content=["']([^"']+)/i)?.[1] ?? '',
cookie: res.headers.getSetCookie().map((c) => c.split('=')[0].trim()),
};
const found = new Map(); // technology -> first piece of evidence
for (const [tech, where, re] of RULES) {
if (found.has(tech)) continue;
let hit = null;
if (where.startsWith('header:')) {
const v = res.headers.get(where.slice(7));
if (v && re.test(v)) hit = `${where.slice(7)}: ${v.slice(0, 40)}`;
} else if (where === 'html') {
const m = html.match(re);
if (m) hit = `html contains "${m[0].slice(0, 40)}"`;
} else if (where === 'generator') {
if (re.test(signals.generator)) hit = `meta generator "${signals.generator}"`;
} else {
const v = signals[where].find((s) => re.test(s));
if (v) hit = `${where} ${v.slice(0, 60)}`;
}
if (hit) found.set(tech, hit);
}
return { site, status: res.status, finalUrl: res.url, found };
}
for (const site of process.argv.slice(2)) {
try {
const r = await detect(site);
console.log(`\n${r.site} -> HTTP ${r.status} ${r.finalUrl}`);
if (!r.found.size) console.log(' (nothing matched)');
for (const [tech, why] of r.found) console.log(` ${tech.padEnd(19)} ${why}`);
} catch (e) {
console.log(`\n${site} -> failed: ${e.cause?.code ?? e.name}`);
}
}
Run it with one or more domains:
node detect.mjs wordpress.org nextjs.org allbirds.com drupal.org discord.com www.etsy.com
On 29 Oct 2026, I got this output:
wordpress.org -> HTTP 200 https://wordpress.org/
WordPress html contains "/wp-content/"
Google Tag Manager html contains "googletagmanager.com/gtm.js"
Nginx server: nginx
nextjs.org -> HTTP 200 https://nextjs.org/
Next.js html contains "/_next/static/"
Vercel server: Vercel
allbirds.com -> HTTP 200 https://www.allbirds.com/
Shopify script https://cdn.shopify.com/s/files/1/1104/4168/files/jsEncrypt.
Google Tag Manager html contains "googletagmanager.com/gtm.js"
Cloudflare server: cloudflare
drupal.org -> HTTP 200 https://new.drupal.org/
Drupal x-generator: Drupal 11 (https://www.drupal.org)
jQuery script /core/assets/vendor/jquery/jquery.min.js?v=4.0.0
Apache server: Apache
discord.com -> HTTP 200 https://discord.com/
Webflow html contains "data-wf-page="
Google Tag Manager html contains "googletagmanager.com/gtm.js"
Cloudflare server: cloudflare
www.etsy.com -> HTTP 403 https://www.etsy.com/
(nothing matched)
Some points to note:
-
Each rule needs only one clue. The Shopify rule matched a script on
cdn.shopify.com. The Drupal rule matched thex-generatorheader. - The script prints its evidence. When a result looks wrong, you can see which clue caused it.
- Etsy returned HTTP 403 (Forbidden). Its bot protection blocked the request, so there was no page to check. Section 4 covers this case.
To add a technology, add a row to RULES. Use a clue that only that product leaves. For example, /wp-content/ is a strong clue for WordPress, but the word wordpress alone isn't, because any page can mention it.
3. Print the raw clues to write your own rules (Python)
You need to see a site's clues before you can write a rule for it. The script below prints them without judging them: selected headers, cookie names, the generator tag, and the hosts that serve the page's script files. It counts only scripts loaded from a URL (a src attribute), not code written inline in the page.
Save it as signals.py:
# signals.py - print the raw tech-stack clues one page gives away.
# Python 3.9+, standard library only. Usage: python3 signals.py example.com
import re
import sys
import urllib.error
import urllib.request
from urllib.parse import urljoin, urlparse
HEADERS = ["server", "x-powered-by", "x-generator", "via", "x-vercel-id", "x-shopid"]
def fetch(url):
req = urllib.request.Request(url, headers={"User-Agent": "stack-check/1.0 (+https://dev.to/neorhapsody)"})
try:
with urllib.request.urlopen(req, timeout=15) as res:
return res.status, res.geturl(), res.headers, res.read(2_000_000).decode("utf-8", "replace")
except urllib.error.HTTPError as err: # 403, 404 ... still carry headers
return err.code, url, err.headers, err.read(2_000_000).decode("utf-8", "replace")
def signals(site):
url = site if site.startswith("http") else "https://" + site
status, final_url, headers, html = fetch(url)
print(f"{site} -> HTTP {status} {final_url}")
for name in HEADERS:
if headers.get(name):
print(f" header {name}: {headers[name][:60]}")
cookies = sorted({c.split("=", 1)[0].strip() for c in headers.get_all("set-cookie") or []})
if cookies:
print(" cookies", ", ".join(cookies))
gen = re.search(r'<meta[^>]+name=["\']generator["\'][^>]+content=["\']([^"\']+)', html, re.I)
if gen:
print(" meta generator:", gen.group(1))
srcs = re.findall(r'<script[^>]+src=["\']([^"\']+)', html, re.I)
hosts = sorted({urlparse(urljoin(final_url, s)).hostname or "" for s in srcs})
print(f" scripts {len(srcs)} external, from: {', '.join(hosts) or '-'}")
for site in sys.argv[1:]:
try:
signals(site)
except Exception as err: # DNS failure, timeout, TLS error
print(f"{site} -> failed: {err}")
Run it:
python3 signals.py www.etsy.com wordpress.org allbirds.com nextjs.org
On 29 Oct 2026, I got this output:
www.etsy.com -> HTTP 403 https://www.etsy.com
header server: DataDome
header via: 1.1 varnish
cookies datadome, exp_ebid
scripts 1 external, from: ct.captcha-delivery.com
wordpress.org -> HTTP 200 https://wordpress.org
header server: nginx
meta generator: WordPress 7.2-beta3-64914
scripts 5 external, from: stats.wp.com, wordpress.org
allbirds.com -> HTTP 200 https://www.allbirds.com/
header server: cloudflare
cookies _shopify_analytics, _shopify_essential, _shopify_marketing, cart_currency, localization
scripts 25 external, from: cdn.shopify.com, khmms.allbirds.com, shop.app, www.allbirds.com
nextjs.org -> HTTP 200 https://nextjs.org
header server: Vercel
header x-vercel-id: iad1::f4gb2-1793239586400-1333ca7073d6
scripts 29 external, from: nextjs.org
This output gives you new rules to add to detect.mjs:
-
The
generatortag gives a version. wordpress.org reportedWordPress 7.2-beta3-64914. The JavaScript detector matched/wp-content/first, so it didn't show the version. -
Cookie names confirm a platform. Allbirds set three cookies that start with
_shopify_. -
A blocked page still has clues. Etsy's 403 response named its bot protection, DataDome, in the
serverheader and in a cookie. Your own run may get HTTP 200 from Etsy instead. Section 4 explains why.
4. Treat every result as a guess
These scripts read one page, once, without running its JavaScript. That design is fast and cheap, but it has limits.
Sites can block you, and not always the same way. A few minutes before the run above, the same Python script got HTTP 200 from Etsy, with seven cookies and five scripts. The next run got HTTP 403. Bot protection decides per request, so a single failed request doesn't mean the site always blocks you. Don't try to get around a block. Record it and move on.
No match doesn't mean the product isn't there. Many tags load from JavaScript after the page starts, so their script URLs never appear in the HTML. My Google Tag Manager rule looks for the literal text googletagmanager.com/gtm.js, and it found nothing on drupal.org. That page adds the tag from JavaScript, and its HTML only contains the tag's ID (GTM-...) in a settings block. A larger fingerprint set, which I describe next, did detect Google Tag Manager there.
Fingerprints age. I also ran the same pages through the open-source matching engine of my Tech Stack Detector (the hosted tool described at the end of this post). It checks about 7,600 fingerprints from the community webappanalyzer project. It found 13 technologies on wordpress.org, 11 on discord.com and 10 on allbirds.com, against 3 each from my 24 rules. But on nextjs.org it reported React and Vercel, not Next.js. That set's Next.js fingerprint looks for an x-powered-by header, a NEXT_LOCALE cookie, script code that contains .next.react, or JavaScript variables that exist only after the page runs. nextjs.org's response showed none of them. My one-line rule for /_next/static/ caught it. After that test, I added Next.js rules to the Tech Stack Detector, including one for the /_next/static/ path. It now reports Next.js on nextjs.org. More fingerprints find more products, but no list is complete.
Clues can be false. Any site can send a misleading header or include an unused script. Versions are often hidden or out of date. Treat a detection as "this page shows signs of X", not "this site runs X".
5. Be polite when you check many sites
- Send a descriptive User-Agent (the request header that names your client) with a way to contact you. Both scripts do.
- Fetch one page per site. The homepage usually carries the most clues.
- Set a timeout. Both scripts give up after 15 seconds.
- Limit parallel requests, especially to sites on the same host. Both scripts check one site at a time.
- Respect blocks. A 403 or 429 (Too Many Requests) response is an answer, not a puzzle to solve.
My hosted version (disclosure: this is my own project)
I built and sell a hosted version of this idea: the Bulk Tech Stack Detector on Apify, a platform for running scraping and automation tools. It follows the approach in this post, one request per homepage with no browser, but it uses the full webappanalyzer fingerprint set. That set has about 7,600 fingerprints and also checks page elements and visible text. The detector also has the extra Next.js rules from section 4. It returns versions where the page shows them, and it adds implied technologies, such as PHP and MySQL for WordPress.
You give it a list of domains, and it returns one row per site with categories such as CMS, Analytics and CDN. It costs $2 per 1,000 sites, plus Apify's per-run start fee (a fraction of a cent), and sites it can't reach are free. The code is open source under GPL-3.0-or-later. It has the same limits as the scripts above: it doesn't run JavaScript, and it doesn't bypass bot protection.
The scripts in this post work on their own. You don't need the hosted version to use them.
Summary
A website reveals its stack in five places: headers, cookies, the generator tag, script URLs and paths in its HTML. A list of rules and one request per site is enough to detect the common products. Print the raw clues before you write a rule, show the evidence for each match, and treat every result as a guess.
Which clue do you find most reliable? If you know a fingerprint that survives framework upgrades, please share it in the comments.
來源:DEV Community · dev.to