跳到正文
DEV Community· Paper & Linen Studio·· 1 小時前AI 評分57

如何用 Python 建立 RSS 和 Atom 關鍵字新聞監察器，並只接收新項目

Build a keyword news monitor from RSS and Atom feeds in Python (and only get new items)

AI 導讀

文章示範用 Python 標準函式庫讀取 RSS 和 Atom，並以 RSS Feed Reader Actor 建立可按關鍵字篩選、只輸出新項目的新聞監察流程。約 50 行的讀取器涵蓋兩種格式，並説明日期格式、穩定 ID 去重及 Atom 連結解析等易錯點。文中亦提供 Python 和 Node.js 範例，以及定時執行和避免重複輸出的設定。

正文

RSS isn't dead. Most news sites, blogs, GitHub releases, podcasts and job boards still publish a feed, and a feed is the cheapest, most polite way to follow a source: one small XML request instead of scraping pages.

This post covers:

  1. A dependency-free RSS + Atom reader in Python (about 50 lines).
  2. The traps you hit when you turn it into a monitor: dates, duplicates, missing feed links, and a classic ElementTree gotcha.
  3. A hosted feed reader with keyword filters and an "only new items" memory, called from Python and Node.js.

Disclosure: I built the RSS Feed Reader Actor used in part 3. Parts 1 and 2 don't need it.

1. A minimal reader (Python standard library)

RSS 2.0 and Atom look different: <item> vs <entry>, <pubDate> vs <published>, a text <link> vs <link href="…">, and Atom puts everything in an XML namespace. This handles both:

import json
import sys
import urllib.request
import xml.etree.ElementTree as ET
from email.utils import parsedate_to_datetime
from datetime import datetime

ATOM = "{http://www.w3.org/2005/Atom}"
UA = {"User-Agent": "Mozilla/5.0 (feed-reader script)"}

def to_iso(raw):
    """RSS uses RFC 822 dates, Atom uses ISO 8601. Return ISO 8601 or None."""
    if not raw:
        return None
    raw = raw.strip()
    try:
        return parsedate_to_datetime(raw).isoformat()
    except (TypeError, ValueError):
        pass
    try:
        return datetime.fromisoformat(raw.replace("Z", "+00:00")).isoformat()
    except ValueError:
        return None

def read_feed(url):
    req = urllib.request.Request(url, headers=UA)
    with urllib.request.urlopen(req, timeout=20) as r:
        root = ET.fromstring(r.read())
    items = []
    if root.tag == ATOM + "feed":                       # Atom
        for e in root.iter(ATOM + "entry"):
            link = e.find(ATOM + "link[@rel='alternate']")
            if link is None:                             # don't use `or`: empty Elements are falsy
                link = e.find(ATOM + "link")
            items.append({
                "title": (e.findtext(ATOM + "title") or "").strip(),
                "link": link.get("href") if link is not None else None,
                "id": e.findtext(ATOM + "id"),
                "published": to_iso(e.findtext(ATOM + "published") or e.findtext(ATOM + "updated")),
            })
    else:                                                # RSS 2.0
        for e in root.iter("item"):
            items.append({
                "title": (e.findtext("title") or "").strip(),
                "link": e.findtext("link"),
                "id": e.findtext("guid") or e.findtext("link"),
                "published": to_iso(e.findtext("pubDate")),
            })
    return items

if __name__ == "__main__":
    url = sys.argv[1] if len(sys.argv) > 1 else "https://github.blog/feed/"
    for item in read_feed(url)[:5]:
        print(json.dumps(item, ensure_ascii=False))

Output (03.11.2026):

$ python rss_stdlib.py
{"title": "Game Off 2026: The theme is SHIFT", "link": "https://github.blog/open-source/gaming/game-off-2026-the-theme-is-shift/", "id": "https://github.blog/?p=99918", "published": "2026-11-01T21:36:00+00:00"}
...
$ python rss_stdlib.py https://www.heise.de/rss/heise-atom.xml
{"title": "KI treibt Rechenzentren – doch Deutschland hinkt global hinterher", "link": "https://www.heise.de/news/KI-treibt-Rechenzentren-doch-Deutschland-hinkt-global-hinterher-11509550.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag", "id": "http://heise.de/-11509550", "published": "2026-11-03T14:26:00+01:00"}

2. Where feed monitors go wrong

  • Two date formats. RSS uses RFC 822 (Sun, 01 Nov 2026 21:36:00 +0000), Atom uses ISO 8601 (2026-11-03T14:26:00+01:00), and plenty of feeds get either one slightly wrong. Normalise everything to ISO 8601 in UTC before you compare or sort, and decide what to do with items that have no date at all.
  • The ElementTree truthiness trap. e.find("a") or e.find("b") looks harmless, but an Element with no children is falsy, so an Atom <link href="…"/> gets skipped and you silently fall back to the wrong link. Always compare with is None (newer Python versions warn about this).
  • Deduplication needs a stable ID. Use guid / Atom id first and fall back to the link. Titles get edited, and links pick up tracking parameters (?wt_mc=rss…). Keep the IDs you've already delivered somewhere persistent, or your "monitor" re-sends everything on every run.
  • Not every site links its feed. Auto-discovery reads <link rel="alternate" type="application/rss+xml"> from the homepage. Many sites have it, but some big ones don't. For example, heise.de's homepage doesn't advertise its feed, so you need the direct URL (/rss/heise-atom.xml). Guessing common paths (/feed, /rss.xml, /atom.xml) catches many WordPress and static sites.
  • Be polite. Send If-None-Match / If-Modified-Since with the ETag / Last-Modified from the previous response. Unchanged feeds then answer 304 with no body, which saves bandwidth on both sides.
  • More formats. RSS 1.0 (RDF), JSON Feed and podcast extensions (itunes:duration, enclosures) all show up in the wild.

3. Hosted option: RSS Feed Reader on Apify

The RSS Feed Reader reads RSS 2.0, RSS 1.0, Atom and JSON Feed into one normalised dataset. It auto-discovers feeds from website URLs, filters by date and keyword (whole words, or re: for regular expressions), and its onlyNew mode remembers delivered items and ETags between runs. That makes it a good fit for a schedule. It's plain HTTP with no browser, so runs are tiny.

Set APIFY_TOKEN from a free Apify account (Console → Settings → API & Integrations).

Python (pip install apify-client) 

"""Keyword monitor over several RSS/Atom feeds: prints only items you haven't seen yet.

Actor: https://apify.com/paperlinenstudio/rss-feed-reader
Usage: APIFY_TOKEN=... python rss_monitor.py
Run it on a schedule (cron, Apify schedule): thanks to onlyNew, each run returns new matches only.
"""
import os

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

run = client.actor("paperlinenstudio/rss-feed-reader").call(
    run_input={
        "feedUrls": [
            {"url": "https://github.blog/"},                # a website: its feed is auto-discovered
            {"url": "https://www.heise.de/rss/heise-atom.xml"},  # a feed URL
        ],
        "includeKeywords": ["KI", "AI", "Copilot", "security"],  # whole words, case-insensitive
        "publishedWithinDays": 30,
        "maxItemsPerFeed": 5,
        "onlyNew": True,                 # remember delivered items between runs
        "stateKey": "tech-monitor-v2",      # separate memory per monitor
        "maxTextLength": 0,              # titles and links are enough here
    }
)
items = list(client.dataset(run.default_dataset_id).iterate_items())
print(f"{len(items)} new matching item(s)")
for i in items:
    print(f"{i['publishedAt'][:16]}  [{i['feedTitle']}]  {i['title']}\n    {i['link']}")

First run (03.11.2026, shortened):

10 new matching item(s)
2026-11-01T21:36  [The GitHub Blog]  Game Off 2026: The theme is SHIFT
    https://github.blog/open-source/gaming/game-off-2026-the-theme-is-shift/
2026-10-28T16:09  [The GitHub Blog]  GitHub Universe 2026: Evolving every layer of the stack
    https://github.blog/news-insights/company-news/github-universe-2026-evolving-every-layer-of-the-stack/
...
2026-11-03T13:26  [heise online News]  KI treibt Rechenzentren – doch Deutschland hinkt global hinterher
    https://www.heise.de/news/KI-treibt-Rechenzentren-doch-Deutschland-hinkt-global-hinterher-11509550.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag
2026-11-03T10:08  [heise online News]  Kommentar: KI ist ein Fiebertraum
    https://www.heise.de/meinung/Kommentar-KI-ist-ein-Fiebertraum-11508391.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag

Keywords match the title, the text and the categories, so an item can match on a word that isn't in its title.

Second run, a minute later:

4 new matching item(s)
2026-10-19T18:47  [The GitHub Blog]  How to continue a GitHub Copilot app session in VS Code
...

Third run:

0 new matching item(s)

Why did the second run still find items? maxItemsPerFeed: 5 caps how many items are saved per feed per run. The first run delivered the 5 newest matches per feed. The second run picked up the older matches still inside the 30-day window. After that, only genuinely new posts come through. If you don't want the backlog, set publishedWithinDays: 1 for the first run.

Node.js (npm i apify-client) 

// Keyword monitor over several RSS/Atom feeds: prints only items you haven't seen yet.
// Usage: APIFY_TOKEN=... node rss_monitor.mjs
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('paperlinenstudio/rss-feed-reader').call({
    feedUrls: [
        { url: 'https://github.blog/' }, // a website: its feed is auto-discovered
        { url: 'https://www.heise.de/rss/heise-atom.xml' }, // a feed URL
    ],
    includeKeywords: ['KI', 'AI', 'Copilot', 'security'],
    publishedWithinDays: 30,
    maxItemsPerFeed: 5,
    onlyNew: true, // remember delivered items between runs
    stateKey: 'tech-monitor-node',
    maxTextLength: 0,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(`${items.length} new matching item(s)`);
for (const i of items) console.log(`${i.publishedAt.slice(0, 16)}  [${i.feedTitle}]  ${i.title}\n    ${i.link}`);

A single record (real, from the run above):

{
  "title": "KI treibt Rechenzentren – doch Deutschland hinkt global hinterher",
  "link": "https://www.heise.de/news/KI-treibt-Rechenzentren-doch-Deutschland-hinkt-global-hinterher-11509550.html?wt_mc=rss.red.ho.ho.atom.beitrag.beitrag",
  "guid": "http://heise.de/-11509550",
  "publishedAt": "2026-11-03T13:26:00+00:00",
  "updatedAt": "2026-11-03T13:26:00+00:00",
  "publishedRaw": "2026-11-03T14:26:00+01:00",
  "author": "Moritz Förster",
  "categories": [],
  "summary": null,
  "contentText": null,
  "imageUrl": "https://www.heise.de/scale/geometry/450/q80//imgs/18/5/1/9/3/7/0/1/shutterstock_2827255519-199c6b4bc64f6c05.jpeg",
  "enclosures": [],
  "feedTitle": "heise online News",
  "feedUrl": "https://www.heise.de/rss/heise-atom.xml",
  "feedLink": "https://www.heise.de/",
  "feedType": "atom",
  "itemId": "9254648e73d946379bb1",
  "scrapedAt": "2026-11-03T13:43:30+00:00"
}

Run it on a schedule

The simplest option is an Apify schedule (for example hourly) on a saved task with the input above. Each run's dataset then only contains new matches, and you can attach a webhook or the Slack/email integrations to forward them. If you'd rather keep it on your own machine, a cron line works too. Keep the token in a file that only you can read, not in the crontab:

# ~/.apify_env contains: export APIFY_TOKEN=your_token_here   (chmod 600)
0 * * * *  . "$HOME/.apify_env" && cd "$HOME/monitor" && python3 rss_monitor.py >> monitor.log 2>&1

Inputs you'll probably change 

{
  "feedUrls": [{ "url": "https://your-favourite-blog.example" }],
  "includeKeywords": ["re:\\bkubernetes|k8s\\b", "postgres"],
  "excludeKeywords": ["sponsored", "webinar"],
  "publishedWithinDays": 3,
  "onlyNew": true,
  "stateKey": "infra-news",
  "maxTextLength": 500
}

Use a different stateKey per monitor so their "seen" lists don't interfere.

Wrap-up

  • Reading one or two feeds in a script: the ~50-line reader in part 1 is enough.
  • Keyword monitoring across many feeds with deduplication, scheduling and CSV/JSON export: RSS Feed Reader.

Feedback and feature requests are welcome, either in the comments or on the Actor's Issues tab.

來源：DEV Community · dev.to

#教學/實踐#搜尋#部署/工程