Playwright 配合 Python 批量擷取網站截圖：完整頁面、手機版、PDF 及隱藏 Cookie 橫幅（2026）

Taking one screenshot with Playwright is a two-liner. Taking a few hundred that actually look right is where it gets fiddly: cookie banners cover half the page, lazy-loaded images come out blank, news homepages produce 60,000-pixel PNGs, and some sites never reach "network idle".

Here's the script I ended up with, plus what happened when I ran it on six sites on October 21, 2026 (example.com, python.org, crawlee.dev, MDN, BBC and the Guardian).

Setup

pip install playwright
playwright install chromium   # or pass --chrome /usr/bin/google-chrome to reuse an installed Chrome

The script

"""Bulk website screenshots with Playwright: viewport, full-page, mobile and PDF, with cookie banners hidden.
Usage: python3 screenshots.py urls.txt out/ [--mobile] [--full] [--pdf] [--workers 4]
"""
import argparse, asyncio, csv, pathlib, re, time
from playwright.async_api import async_playwright

HIDE_CSS = """
#onetrust-banner-sdk, #onetrust-consent-sdk, #CybotCookiebotDialog, .cc-window, .cookie-banner,
#cookie-banner, #cookie-notice, .fc-consent-root, #truste-consent-track, .qc-cmp2-container,
[id^="sp_message_container"], .sp-message-open #sp_message_iframe, #didomi-host, #usercentrics-root,
[id*="cookie" i][class*="banner" i], [class*="cookie-consent" i], [aria-label*="cookie" i][role="dialog"]
{ display: none !important; }
html, body, html.sp-message-open { overflow: auto !important; position: static !important; }
"""

def slug(url):
    return re.sub(r'[^a-zA-Z0-9]+', '-', url.split('://', 1)[-1]).strip('-')[:80]

async def shoot(browser, url, out, a):
    ctx_args = dict(viewport={'width': 1280, 'height': 800}, device_scale_factor=1)
    if a.mobile:
        ctx_args = dict(viewport={'width': 390, 'height': 844}, device_scale_factor=3, is_mobile=True, has_touch=True,
                        user_agent='Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 '
                                   '(KHTML, like Gecko) Version/17.0 Mobile/15E148 Safari/604.1')
    ctx = await browser.new_context(**ctx_args)
    page = await ctx.new_page()
    t0 = time.time()
    try:
        resp = await page.goto(url, wait_until='domcontentloaded', timeout=30000)
        try:
            await page.wait_for_load_state('networkidle', timeout=8000)
        except Exception:
            pass  # sites with analytics beacons never go idle
        await page.add_style_tag(content=HIDE_CSS)
        # trigger lazy-loaded images before a full-page capture
        if a.full:
            await page.evaluate("""async () => { for (let y = 0; y < document.body.scrollHeight; y += 600) {
                window.scrollTo(0, y); await new Promise(r => setTimeout(r, 100)); } window.scrollTo(0, 0); }""")
            await page.wait_for_timeout(500)
        path = out / (slug(url) + ('.pdf' if a.pdf else '.png'))
        if a.pdf:
            await page.emulate_media(media='screen')
            await page.pdf(path=str(path), format='A4', print_background=True)
        else:
            clip = None
            if a.full:  # cap very long pages (news homepages can run 60,000+ px)
                h = await page.evaluate('Math.max(document.body.scrollHeight, document.documentElement.scrollHeight)')
                if h > a.max_height:
                    clip = dict(x=0, y=0, width=page.viewport_size['width'], height=a.max_height)
            await page.screenshot(path=str(path), full_page=a.full, clip=clip)
        return dict(url=url, status=resp.status if resp else '', file=path.name,
                    bytes=path.stat().st_size, seconds=round(time.time() - t0, 1), error='')
    except Exception as e:
        return dict(url=url, status='', file='', bytes=0, seconds=round(time.time() - t0, 1), error=str(e)[:120])
    finally:
        await ctx.close()

async def main():
    p = argparse.ArgumentParser()
    p.add_argument('urls'); p.add_argument('out')
    p.add_argument('--mobile', action='store_true'); p.add_argument('--full', action='store_true')
    p.add_argument('--pdf', action='store_true'); p.add_argument('--max-height', type=int, default=15000); p.add_argument('--workers', type=int, default=4)
    p.add_argument('--chrome', default=None, help='path to an installed Chrome (else Playwright Chromium)')
    a = p.parse_args()
    urls = [u.strip() for u in open(a.urls) if u.strip() and not u.startswith('#')]
    out = pathlib.Path(a.out); out.mkdir(parents=True, exist_ok=True)
    sem = asyncio.Semaphore(a.workers)
    async with async_playwright() as pw:
        browser = await pw.chromium.launch(executable_path=a.chrome)
        async def one(u):
            async with sem:
                return await shoot(browser, u, out, a)
        t0 = time.time()
        rows = await asyncio.gather(*(one(u) for u in urls))
        await browser.close()
    with open(out / 'report.csv', 'w', newline='') as f:
        w = csv.DictWriter(f, fieldnames=list(rows[0])); w.writeheader(); w.writerows(rows)
    ok = sum(1 for r in rows if not r['error'])
    print(f"{ok}/{len(rows)} captured in {time.time() - t0:.1f}s -> {out}/report.csv")
    for r in rows: print(r)

if __name__ == '__main__':
    asyncio.run(main())

Run it with a text file of URLs, one per line:

python3 screenshots.py urls.txt shots/                  # 1280x800 desktop viewport
python3 screenshots.py urls.txt shots/ --mobile --full  # iPhone viewport, whole page
python3 screenshots.py urls.txt shots/ --pdf            # A4 PDF with backgrounds

Every run also writes report.csv with the HTTP status, file size, seconds per page and any error, so failures don't disappear silently.

What I learned testing it

1. Generic cookie selectors miss the big consent platforms. My first version hid the usual suspects (OneTrust, Cookiebot, .cookie-banner), and the BBC came out clean. The Guardian's screenshot was 80% consent wall. It uses Sourcepoint, which injects #sp_message_container_* with an iframe and also locks scrolling with an sp-message-open class on <html>. Hiding the container and resetting overflow/position on html fixed it. If a site still shows a banner, open DevTools, find the outermost fixed-position element and add its selector to HIDE_CSS. Hiding with CSS is safer than clicking "Accept": you don't consent on anyone's behalf, and it doesn't break when the button text is translated.

2. Don't rely on networkidle alone. News sites keep analytics and ad beacons firing forever, so wait_until="networkidle" can hang until the timeout. The script loads on domcontentloaded, then gives networkidle up to 8 seconds and moves on regardless.

3. Scroll before a full-page capture. Lazy-loaded images only load when they enter the viewport. A quick scroll from top to bottom in 600 px steps before the screenshot fills them in.

4. Cap the height. The BBC homepage on an iPhone viewport at 3x scale produced a 1170 x 60,114 px PNG, 22 MB, and Pillow refused to open it without a "decompression bomb" warning. With --max-height 15000 (CSS pixels), it's 1170 x 45,000 and 16 MB. Measure with Math.max(document.body.scrollHeight, document.documentElement.scrollHeight): on the Guardian, the <html> height alone was too small and the cap didn't kick in until I checked both.

5. PDFs of busy pages are huge. The BBC homepage as an A4 PDF was 40.7 MB and took 34 seconds; the Guardian was 0.5 MB in 2 seconds. Use emulate_media("screen") so you get the page as people see it instead of its print stylesheet.

Timings (4 parallel contexts, one shared browser):

Reusing one browser and opening a fresh context per URL is the main speed trick: launching Chrome costs more than most screenshots.

Scaling it up

This works well for dozens or a few hundred URLs from your laptop. Past that you start dealing with memory (each full-page capture of a long page holds the whole bitmap), retries, IP blocks and somewhere to store thousands of files.

If you'd rather not run the browsers yourself, we packaged the same approach (full-page, mobile, PDF, cookie-banner hiding) as the Website Screenshot API on Apify. Disclosure: it's ours. It costs $1.50 per 1,000 viewport screenshots and $5 per 1,000 full-page captures or PDFs, and Apify's free $5 monthly credit covers about 3,300 viewport shots. You can call it over REST or as an MCP tool from an AI agent.

Either way, the five lessons above are the part that saves the most time.