跳到正文
DEV Community· novosai0day-a11y·· 10 天前AI 評分60

Google News Scraper with Original Article URLs 解析 Google News RSS 並返回原始文章網址

A Google News alert with the real article URLs (not news.google.com redirects)

AI 導讀

Google News Scraper with Original Article URLs 可將 Google News RSS 的跳轉連結解析為出版商原始網址,並按查詢、語言及國家返回標題、出版商、日期和摘要。

正文

novosai0day-a11y

Google News RSS is the easiest way to monitor a brand, a competitor or a topic in any country — until you notice that every link is a news.google.com/rss/articles/CBMi… redirect. You can't dedupe by publisher URL, you can't fetch the article, and your Slack alerts look like spam.

Google News Scraper with Original Article URLs (disclosure: I built it) resolves those links to the publisher's real URL and returns headline, publisher, date and snippet for any query, language and country.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("fujiwaranosai/google-news-scraper").call(run_input={
    "queries": ['"Anthropic" OR "OpenAI"', "Tesla -recall"],
    "timeRange": "1d",
    "language": "en",
    "country": "us",
    "onlyNew": True,
})
for a in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(a["published_at"], a["publisher"], a["title"], a["article_url"])

Ideas

  • PR monitoring: one query per brand/executive, schedule hourly with timeRange: "1h" and onlyNew.
  • Market research: track tickers or sectors per country edition (es/mx, pt/br, de/de).
  • AI digests: pass article_url to your content crawler and summarize the day's news with an LLM.

Google News operators work too: "exact phrase", OR, -exclude, site:reuters.com. The first 100 articles of every account are free.

More ready-to-run examples: https://github.com/novosai0day-a11y/public-data-apis

來源:DEV Community · dev.to