跳到正文
DEV Community· Vignesh·· 45 分鐘前AI 評分60

PHP 和 curl 如何安全抓取陌生人提供的網址

Fetching a URL a stranger gave you: a safe pattern for PHP and curl

AI 導讀

Vignesh 分享一套以 PHP 和 curl 抓取訪客提供網址的做法,用來降低伺服器端請求偽造(SSRF)風險。五項規則包括只允許 http/https 的 80 或 443 連接埠、確認 DNS 回應全部為公用 IPv4 並固定連線位址,以及自行逐跳檢查重新導向;同時限制總時限和回應大小,並停用代理伺服器。

正文

Some tools take a web address from a visitor and fetch it from your server. SEO audits do this. So do link previews, webhook testers and "import from URL" buttons.

That hands the visitor control over where your server connects. If nothing limits it, they can aim your server at places only it can reach. That includes a service on localhost, machines on your private network, and the cloud metadata address, where cloud providers can expose credentials. This attack is called server-side request forgery, or SSRF. The OWASP SSRF prevention cheat sheet is the usual reference.

We run a free website audit tool, so our server fetches whatever address a stranger types in. This is the pattern we use: five rules and about 90 lines of PHP.

Why checking the hostname is not enough

The common first fix is to check that the hostname is public, then call curl with redirects switched on. It fails in two ways.

Redirects. A public site can answer with a 302 and a Location header that points to http://169.254.169.254/. curl follows it, and your check never saw that address. OWASP's advice is to turn off redirect following in your HTTP client. Until 9 November 2026, our audit tool made exactly this mistake. A public site could have sent the audit to an internal address, and the report would have shown that page's title and heading.

DNS. You look up the name and check the address. Then curl looks up the name again by itself. A DNS server run by an attacker can answer with a public address the first time and 127.0.0.1 the second time. This is called DNS rebinding. OWASP's advice is to check the addresses a name resolves to, then make sure the client connects only to those addresses.

Five rules

  1. Allow plain web addresses only. Use http or https, port 80 or 443, and no user:password@ part. SSRF is not limited to HTTP: OWASP lists schemes such as file://, gopher:// and dict://, and a redirect can name any of them.
  2. Resolve the name yourself, and require every answer to be public. If one answer is private, refuse the request. Do not pick the good one. Keep one address per host for the whole fetch.
  3. Connect to the address you checked. The CURLOPT_CONNECT_TO option tells curl which IP to connect to instead of looking the name up. According to curl's documentation, it only decides where the network connection goes. The host name used for TLS (the server name and the certificate check) stays the same, so HTTPS still verifies the real site.
  4. Follow redirects yourself. Switch CURLOPT_FOLLOWLOCATION off, read the Location header, and run rules 1 to 3 again on every hop. Stop after five hops.
  5. Limit time, size and proxies. Set one deadline for the whole chain. Stop reading after a byte cap. Turn the proxy off, because a proxy would look the name up for you.

The sketch

This is a trimmed, generic version, not our production code. The comments match the rule numbers above.

<?php
const MAX_HOPS  = 5;
const MAX_BYTES = 2_000_000;

/** True only for a public IPv4 address. */
function is_public_ipv4(string $ip): bool
{
    $flags = FILTER_FLAG_IPV4 | FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE;
    if (!filter_var($ip, FILTER_VALIDATE_IP, $flags)) {
        return false;
    }
    // Ranges those two flags leave open (from the IANA special-purpose registry).
    $more = ['100.64.0.0/10', '192.0.0.0/24', '192.0.2.0/24', '198.18.0.0/15',
             '198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4'];
    foreach ($more as $cidr) {
        [$net, $bits] = explode('/', $cidr);
        $mask = (-1 << (32 - (int) $bits)) & 0xFFFFFFFF;
        if ((ip2long($ip) & $mask) === (ip2long($net) & $mask)) {
            return false;
        }
    }
    return true;
}

function safe_get(string $url, int $timeout = 10, ?callable $resolve = null): array
{
    $resolve ??= 'gethostbynamel';
    $deadline = microtime(true) + $timeout;
    $pins = [];                                        // host => the address we checked

    for ($hop = 0; $hop <= MAX_HOPS; $hop++) {
        $p      = parse_url($url) ?: [];
        $scheme = strtolower($p['scheme'] ?? '');
        $host   = strtolower(rtrim($p['host'] ?? '', '.'));
        $port   = (int) ($p['port'] ?? ($scheme === 'https' ? 443 : 80));

        // 1. Plain web addresses only: usual ports, no credentials.
        if (!in_array($scheme, ['http', 'https'], true) || !in_array($port, [80, 443], true)
            || isset($p['user']) || isset($p['pass']) || $host === '') {
            return ['status' => 0, 'error' => 'address not allowed'];
        }

        // 2. Resolve the name ourselves. Every answer must be public.
        if (!isset($pins[$host])) {
            $ips = $resolve($host) ?: [];
            if (!$ips) {
                return ['status' => 0, 'error' => 'could not resolve host'];
            }
            foreach ($ips as $ip) {
                if (!is_public_ipv4($ip)) {
                    return ['status' => 0, 'error' => 'host not allowed'];
                }
            }
            $pins[$host] = $ips[0];
        }

        // 3. Connect to the address we checked. curl never looks the name up again.
        $location = '';
        $body     = '';
        $ch = curl_init($url);
        curl_setopt_array($ch, [
            CURLOPT_CONNECT_TO     => ['::' . $pins[$host] . ':' . $port],
            CURLOPT_FOLLOWLOCATION => false,           // 4. We follow redirects ourselves.
            CURLOPT_PROXY          => '',              //    A proxy would resolve names itself.
            CURLOPT_IPRESOLVE      => CURL_IPRESOLVE_V4,
            CURLOPT_TIMEOUT_MS     => max(1, (int) (($deadline - microtime(true)) * 1000)),
            CURLOPT_HEADERFUNCTION => function ($c, string $h) use (&$location): int {
                if (stripos($h, 'location:') === 0) {
                    $location = trim(substr($h, 9));
                }
                return strlen($h);
            },
            CURLOPT_WRITEFUNCTION  => function ($c, string $chunk) use (&$body): int {
                $body .= $chunk;
                return strlen($body) > MAX_BYTES ? 0 : strlen($chunk);   // 5. Stop reading at the cap.
            },
        ]);
        curl_exec($ch);
        $status = (int) curl_getinfo($ch, CURLINFO_RESPONSE_CODE);

        if ($status >= 300 && $status < 400 && $location !== '') {
            $url = absolute_url($location, $url);      // then steps 1 to 3 run again on the new address
            continue;
        }
        return ['status' => $status, 'url' => $url, 'body' => $body];
    }
    return ['status' => 0, 'error' => 'too many redirects'];
}

The first two empty fields in the CURLOPT_CONNECT_TO string mean "any request". The last two are the address and port to connect to. absolute_url() turns a relative Location header into a full address. It is a dozen lines and not shown here.

One gotcha from our own tests. We first pinned the address with CURLOPT_RESOLVE. In the curl build our tests run on, that pin was skipped for a hostname written with a trailing dot (example.com.), and curl did its own lookup. With CURLOPT_CONNECT_TO the same test passes, and we keep it as a test case.

What counts as public

PHP's FILTER_FLAG_NO_PRIV_RANGE and FILTER_FLAG_NO_RES_RANGE deny 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16, plus 0.0.0.0/8, 127.0.0.0/8, 169.254.0.0/16 and 240.0.0.0/4, according to the PHP manual. They leave some ranges open that are not meant for the public internet. Two examples are the shared address space 100.64.0.0/10 and the benchmarking range 198.18.0.0/15, which IANA's special-purpose registry marks as not globally reachable. The sketch adds those, the documentation ranges and multicast. From PHP 8.2 you can use FILTER_FLAG_GLOBAL_RANGE instead, which allows only global addresses.

What this does not cover

IPv6. The sketch looks up IPv4 addresses only and tells curl to use IPv4. OWASP says to check both A and AAAA records. If your fetcher has to reach IPv6 sites, add the same checks for IPv6 ranges first.

A block-list is not a wall. OWASP calls deny-lists bypass-prone and prefers allow-lists. A tool that has to fetch any public site cannot use an allow-list. For that case OWASP recommends a block-list, while admitting it is not an impenetrable wall. So treat these rules as one layer.

The network. OWASP also recommends limiting what the application can reach, with firewall rules and network segregation. On AWS it recommends moving to IMDSv2 and turning IMDSv1 off. If your fetcher runs on a network that cannot reach your internal services, a mistake in the code does much less harm.

Test it like an attacker

Our test network sends every private address to a stand-in internal site whose pages say INTERNAL SECRET. If that text ever appears in a report, the guard has failed.

The tests cover twelve hostile redirects. They point to:

  • a private range, loopback and localhost
  • the cloud metadata address
  • a name that points to a private address, and a name with a mix of public and private answers
  • shared address space and IPv6
  • a non-standard port and an address with a username
  • ftp:// and mailto: addresses

They also cover a host whose DNS answer changes between lookups, a redirect loop, a byte cap and a time limit. All 62 checks in the suite pass, and none of the 42 connections made during the tests went to a private address.

Let your function take the resolver as an argument, as the sketch does. Then your tests can fake DNS and never touch the network.

You can see the pattern at work in our free SEO audit tool.


I'm B. Vignesh, Founder & CEO of VTurnU, a B2B digital marketing and web development company. I write about the technical details behind websites that search engines and AI assistants can read. This post was written with the help of AI tools. The address checks in the sketch were run on PHP 8.5 against 13 blocked inputs and a table of 20 sample addresses. The production version of this code is covered by an automated suite of 62 checks.

來源:DEV Community · dev.to