跳到正文
DEV Community· Fikri Marhan·· 2 小時前AI 評分55

如何用 JavaScript 和 Python 從 HTTP 標頭、HTML 和 cookies 偵測網站技術棧

Detect a website's tech stack from headers, HTML and cookies (JS + Python)

AI 導讀

Fikri Marhan 提供 JavaScript 和 Python 範例,示範如何從網站的 HTTP 標頭、cookies、generator 標籤、script URL 和 HTML 路徑辨識技術棧。

正文

I wrote this article with AI assistance. I ran every snippet against the live sites shown on 29 Oct 2026, and all output is copied from those runs. Websites change often, so your output will differ.

Browser extensions such as Wappalyzer tell you what a website is built with. They work by looking for clues that the site gives away in every response. You can read most of those clues yourself with a few lines of code.

A site's tech stack is the set of products it runs on. That includes its content management system (CMS), such as WordPress, its JavaScript framework, its analytics tags and its hosting provider.

Who this is for: developers who want to check the tech stack of one site or a list of sites. For example, you might research competitors, qualify sales leads, or audit your own domains for outdated software.

What you need: Node 18.14.1 or later (Node 20 or later recommended) for the JavaScript script, or Python 3.9 or later for the Python script. Neither script needs any packages.

This post covers:

  1. The five places a site reveals its stack
  2. A rule-based detector in JavaScript
  3. A clue printer in Python, for writing your own rules
  4. Why detection is a guess, not a fact

1. Know the five places a site reveals its stack

One GET request to a homepage returns five kinds of clues:

Clue Example What it often reveals
Response headers server: Vercel, x-generator: Drupal 11 Web server, host, CDN, sometimes the CMS
Cookies _shopify_essential, datadome Ecommerce platform, bot protection, analytics
The generator meta tag <meta name="generator" content="WordPress ..."> CMS and often its version
Script URLs https://cdn.shopify.com/..., .../jquery.min.js Platforms, libraries and third-party tags
Paths in the HTML /wp-content/, /_next/static/, data-wf-page= CMS and framework file layouts

A CDN (content delivery network) is a service that caches a site's files on servers near its visitors. Cloudflare is one example.

Each clue is a fingerprint: a pattern that one product leaves behind. A detector is a list of fingerprints and a loop that checks each one.

2. Detect the stack with a rule list (JavaScript)

The script below fetches each site once. It collects the headers, cookie names, the generator tag and the script URLs. Then it checks 24 rules and prints the first piece of evidence for each technology it finds.

Save it as detect.mjs:

// detect.mjs - guess a website's tech stack from one HTTP request.
// Node 18.14.1+ (built-in fetch and getSetCookie). No dependencies. Usage: node detect.mjs example.com [more.com ...]
const RULES = [
  // [technology, where to look, pattern]
  ['WordPress', 'html', /\/wp-content\/|\/wp-includes\//i],
  ['WordPress', 'generator', /^WordPress/i],
  ['Drupal', 'header:x-generator', /Drupal/i],
  ['Drupal', 'generator', /^Drupal/i],
  ['Shopify', 'script', /cdn\.shopify\.com/i],
  ['Shopify', 'header:x-shopid', /./],
  ['Wix', 'header:x-wix-request-id', /./],
  ['Squarespace', 'html', /static1\.squarespace\.com/i],
  ['Webflow', 'html', /data-wf-page=/i],
  ['Next.js', 'html', /\/_next\/static\//],
  ['Next.js', 'header:x-powered-by', /Next\.js/i],
  ['Nuxt', 'html', /\/_nuxt\/|window\.__NUXT__/],
  ['Gatsby', 'generator', /^Gatsby/i],
  ['Google Tag Manager', 'html', /googletagmanager\.com\/gtm\.js/],
  ['Google Analytics', 'html', /googletagmanager\.com\/gtag\/js|google-analytics\.com\/analytics\.js/],
  ['HubSpot', 'script', /js\.hs-scripts\.com|js\.hsforms\.net/i],
  ['jQuery', 'script', /jquery[.-]?(\d[\d.]*)?(\.min)?\.js/i],
  ['Cloudflare', 'header:server', /^cloudflare$/i],
  ['Cloudflare', 'cookie', /^__cf_bm$/],
  ['Vercel', 'header:server', /^Vercel$/i],
  ['Netlify', 'header:server', /^Netlify$/i],
  ['Nginx', 'header:server', /^nginx/i],
  ['Apache', 'header:server', /^Apache/i],
  ['PHP', 'header:x-powered-by', /PHP/i],
];

async function detect(site) {
  const url = site.startsWith('http') ? site : `https://${site}`;
  const res = await fetch(url, {
    headers: { 'User-Agent': 'stack-check/1.0 (+https://dev.to/neorhapsody)' },
    redirect: 'follow',
    signal: AbortSignal.timeout(15000),
  });
  const html = (await res.text()).slice(0, 2_000_000);
  const signals = {
    html,
    script: [...html.matchAll(/<script[^>]+src=["']([^"']+)/gi)].map((m) => m[1]),
    generator: html.match(/<meta[^>]+name=["']generator["'][^>]+content=["']([^"']+)/i)?.[1] ?? '',
    cookie: res.headers.getSetCookie().map((c) => c.split('=')[0].trim()),
  };
  const found = new Map(); // technology -> first piece of evidence
  for (const [tech, where, re] of RULES) {
    if (found.has(tech)) continue;
    let hit = null;
    if (where.startsWith('header:')) {
      const v = res.headers.get(where.slice(7));
      if (v && re.test(v)) hit = `${where.slice(7)}: ${v.slice(0, 40)}`;
    } else if (where === 'html') {
      const m = html.match(re);
      if (m) hit = `html contains "${m[0].slice(0, 40)}"`;
    } else if (where === 'generator') {
      if (re.test(signals.generator)) hit = `meta generator "${signals.generator}"`;
    } else {
      const v = signals[where].find((s) => re.test(s));
      if (v) hit = `${where} ${v.slice(0, 60)}`;
    }
    if (hit) found.set(tech, hit);
  }
  return { site, status: res.status, finalUrl: res.url, found };
}

for (const site of process.argv.slice(2)) {
  try {
    const r = await detect(site);
    console.log(`\n${r.site} -> HTTP ${r.status} ${r.finalUrl}`);
    if (!r.found.size) console.log('  (nothing matched)');
    for (const [tech, why] of r.found) console.log(`  ${tech.padEnd(19)} ${why}`);
  } catch (e) {
    console.log(`\n${site} -> failed: ${e.cause?.code ?? e.name}`);
  }
}

Run it with one or more domains:

node detect.mjs wordpress.org nextjs.org allbirds.com drupal.org discord.com www.etsy.com

On 29 Oct 2026, I got this output:

wordpress.org -> HTTP 200 https://wordpress.org/
  WordPress           html contains "/wp-content/"
  Google Tag Manager  html contains "googletagmanager.com/gtm.js"
  Nginx               server: nginx

nextjs.org -> HTTP 200 https://nextjs.org/
  Next.js             html contains "/_next/static/"
  Vercel              server: Vercel

allbirds.com -> HTTP 200 https://www.allbirds.com/
  Shopify             script https://cdn.shopify.com/s/files/1/1104/4168/files/jsEncrypt.
  Google Tag Manager  html contains "googletagmanager.com/gtm.js"
  Cloudflare          server: cloudflare

drupal.org -> HTTP 200 https://new.drupal.org/
  Drupal              x-generator: Drupal 11 (https://www.drupal.org)
  jQuery              script /core/assets/vendor/jquery/jquery.min.js?v=4.0.0
  Apache              server: Apache

discord.com -> HTTP 200 https://discord.com/
  Webflow             html contains "data-wf-page="
  Google Tag Manager  html contains "googletagmanager.com/gtm.js"
  Cloudflare          server: cloudflare

www.etsy.com -> HTTP 403 https://www.etsy.com/
  (nothing matched)

Some points to note:

  • Each rule needs only one clue. The Shopify rule matched a script on cdn.shopify.com. The Drupal rule matched the x-generator header.
  • The script prints its evidence. When a result looks wrong, you can see which clue caused it.
  • Etsy returned HTTP 403 (Forbidden). Its bot protection blocked the request, so there was no page to check. Section 4 covers this case.

To add a technology, add a row to RULES. Use a clue that only that product leaves. For example, /wp-content/ is a strong clue for WordPress, but the word wordpress alone isn't, because any page can mention it.

3. Print the raw clues to write your own rules (Python)

You need to see a site's clues before you can write a rule for it. The script below prints them without judging them: selected headers, cookie names, the generator tag, and the hosts that serve the page's script files. It counts only scripts loaded from a URL (a src attribute), not code written inline in the page.

Save it as signals.py:

# signals.py - print the raw tech-stack clues one page gives away.
# Python 3.9+, standard library only. Usage: python3 signals.py example.com
import re
import sys
import urllib.error
import urllib.request
from urllib.parse import urljoin, urlparse

HEADERS = ["server", "x-powered-by", "x-generator", "via", "x-vercel-id", "x-shopid"]


def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": "stack-check/1.0 (+https://dev.to/neorhapsody)"})
    try:
        with urllib.request.urlopen(req, timeout=15) as res:
            return res.status, res.geturl(), res.headers, res.read(2_000_000).decode("utf-8", "replace")
    except urllib.error.HTTPError as err:  # 403, 404 ... still carry headers
        return err.code, url, err.headers, err.read(2_000_000).decode("utf-8", "replace")


def signals(site):
    url = site if site.startswith("http") else "https://" + site
    status, final_url, headers, html = fetch(url)
    print(f"{site} -> HTTP {status} {final_url}")
    for name in HEADERS:
        if headers.get(name):
            print(f"  header  {name}: {headers[name][:60]}")
    cookies = sorted({c.split("=", 1)[0].strip() for c in headers.get_all("set-cookie") or []})
    if cookies:
        print("  cookies", ", ".join(cookies))
    gen = re.search(r'<meta[^>]+name=["\']generator["\'][^>]+content=["\']([^"\']+)', html, re.I)
    if gen:
        print("  meta    generator:", gen.group(1))
    srcs = re.findall(r'<script[^>]+src=["\']([^"\']+)', html, re.I)
    hosts = sorted({urlparse(urljoin(final_url, s)).hostname or "" for s in srcs})
    print(f"  scripts {len(srcs)} external, from: {', '.join(hosts) or '-'}")


for site in sys.argv[1:]:
    try:
        signals(site)
    except Exception as err:  # DNS failure, timeout, TLS error
        print(f"{site} -> failed: {err}")

Run it:

python3 signals.py www.etsy.com wordpress.org allbirds.com nextjs.org

On 29 Oct 2026, I got this output:

www.etsy.com -> HTTP 403 https://www.etsy.com
  header  server: DataDome
  header  via: 1.1 varnish
  cookies datadome, exp_ebid
  scripts 1 external, from: ct.captcha-delivery.com
wordpress.org -> HTTP 200 https://wordpress.org
  header  server: nginx
  meta    generator: WordPress 7.2-beta3-64914
  scripts 5 external, from: stats.wp.com, wordpress.org
allbirds.com -> HTTP 200 https://www.allbirds.com/
  header  server: cloudflare
  cookies _shopify_analytics, _shopify_essential, _shopify_marketing, cart_currency, localization
  scripts 25 external, from: cdn.shopify.com, khmms.allbirds.com, shop.app, www.allbirds.com
nextjs.org -> HTTP 200 https://nextjs.org
  header  server: Vercel
  header  x-vercel-id: iad1::f4gb2-1793239586400-1333ca7073d6
  scripts 29 external, from: nextjs.org

This output gives you new rules to add to detect.mjs:

  • The generator tag gives a version. wordpress.org reported WordPress 7.2-beta3-64914. The JavaScript detector matched /wp-content/ first, so it didn't show the version.
  • Cookie names confirm a platform. Allbirds set three cookies that start with _shopify_.
  • A blocked page still has clues. Etsy's 403 response named its bot protection, DataDome, in the server header and in a cookie. Your own run may get HTTP 200 from Etsy instead. Section 4 explains why.

4. Treat every result as a guess

These scripts read one page, once, without running its JavaScript. That design is fast and cheap, but it has limits.

Sites can block you, and not always the same way. A few minutes before the run above, the same Python script got HTTP 200 from Etsy, with seven cookies and five scripts. The next run got HTTP 403. Bot protection decides per request, so a single failed request doesn't mean the site always blocks you. Don't try to get around a block. Record it and move on.

No match doesn't mean the product isn't there. Many tags load from JavaScript after the page starts, so their script URLs never appear in the HTML. My Google Tag Manager rule looks for the literal text googletagmanager.com/gtm.js, and it found nothing on drupal.org. That page adds the tag from JavaScript, and its HTML only contains the tag's ID (GTM-...) in a settings block. A larger fingerprint set, which I describe next, did detect Google Tag Manager there.

Fingerprints age. I also ran the same pages through the open-source matching engine of my Tech Stack Detector (the hosted tool described at the end of this post). It checks about 7,600 fingerprints from the community webappanalyzer project. It found 13 technologies on wordpress.org, 11 on discord.com and 10 on allbirds.com, against 3 each from my 24 rules. But on nextjs.org it reported React and Vercel, not Next.js. That set's Next.js fingerprint looks for an x-powered-by header, a NEXT_LOCALE cookie, script code that contains .next.react, or JavaScript variables that exist only after the page runs. nextjs.org's response showed none of them. My one-line rule for /_next/static/ caught it. After that test, I added Next.js rules to the Tech Stack Detector, including one for the /_next/static/ path. It now reports Next.js on nextjs.org. More fingerprints find more products, but no list is complete.

Clues can be false. Any site can send a misleading header or include an unused script. Versions are often hidden or out of date. Treat a detection as "this page shows signs of X", not "this site runs X".

5. Be polite when you check many sites

  • Send a descriptive User-Agent (the request header that names your client) with a way to contact you. Both scripts do.
  • Fetch one page per site. The homepage usually carries the most clues.
  • Set a timeout. Both scripts give up after 15 seconds.
  • Limit parallel requests, especially to sites on the same host. Both scripts check one site at a time.
  • Respect blocks. A 403 or 429 (Too Many Requests) response is an answer, not a puzzle to solve.

My hosted version (disclosure: this is my own project)

I built and sell a hosted version of this idea: the Bulk Tech Stack Detector on Apify, a platform for running scraping and automation tools. It follows the approach in this post, one request per homepage with no browser, but it uses the full webappanalyzer fingerprint set. That set has about 7,600 fingerprints and also checks page elements and visible text. The detector also has the extra Next.js rules from section 4. It returns versions where the page shows them, and it adds implied technologies, such as PHP and MySQL for WordPress.

You give it a list of domains, and it returns one row per site with categories such as CMS, Analytics and CDN. It costs $2 per 1,000 sites, plus Apify's per-run start fee (a fraction of a cent), and sites it can't reach are free. The code is open source under GPL-3.0-or-later. It has the same limits as the scripts above: it doesn't run JavaScript, and it doesn't bypass bot protection.

The scripts in this post work on their own. You don't need the hosted version to use them.

Summary

A website reveals its stack in five places: headers, cookies, the generator tag, script URLs and paths in its HTML. A list of rules and one request per site is enough to detect the common products. Print the raw clues before you write a rule, show the evidence for each match, and treat every result as a guess.

Which clue do you find most reliable? If you know a fingerprint that survives framework upgrades, please share it in the comments.

來源:DEV Community · dev.to