跳到正文
Simon Willison·· 11 小時前AI 評分34

webframe:作為網頁抓取工具及沙盒的評估

webframe: evaluated as a scraper and a sandbox

AI 導讀

評估發現,webframe 是一款快速、輕量的瀏覽器 JavaScript 執行工具,適合用於網頁抓取,效能與 shot-scraper 相若。它仍有多項不足,包括 CSP 失敗、視窗大小為 0×0、載入停滯、缺少 HTTP 中繼資料,以及瀏覽器控制功能有限。

正文

Research webframe: evaluated as a scraper and a sandbox — An evaluation of webframe found it to be a fast, lightweight browser-JavaScript runner suitable for scraping, with performance comparable to shot-scraper, but with important rough edges including CSP failures, a 0×0 viewport, hanging loads, missing HTTP metadata, and limited browser controls.

webframe is a neat little project from Jeff Lindsay which lets you "run a web page like a program" - you point it at a URL or file and it executes it in a hidden headless web view and outputs console.log() to standard output and any exceptions to standard error.

It's written in Go on top of the glaze library, which provides access to a system browser ("WKWebView on macOS, WebKitGTK on Linux, WebView2 on Windows") so it doesn't need to bundle its own browser engine.

It reminds me of my own shot-scraper javascript tool, albeit in a much smaller package that doesn't depend on Playwright.

I had Claude Opus 5.5 put it through its paces and prepare a research report for me. Here's the prompt I used:

Explore https://github.com/tractordev/webframe try running it and put it though its paces, build a few exploratory prototypes, then evaluate it as a potential sandboxing tool and as a tool for scraping web pages using the same patterns encouraged by the "shot-scraper JavaScript" command (see simonw/shot-scraper on GitHub, docs directory)

來源:Simon Willison · simonwillison.net