Google News Scraper with Original Article URLs 解析 Google News RSS 並返回原始文章網址
A Google News alert with the real article URLs (not news.google.com redirects)
Google News Scraper with Original Article URLs 可將 Google News RSS 的跳轉連結解析為出版商原始網址,並按查詢、語言及國家返回標題、出版商、日期和摘要。
Google News RSS is the easiest way to monitor a brand, a competitor or a topic in any country — until you notice that every link is a news.google.com/rss/articles/CBMi… redirect. You can't dedupe by publisher URL, you can't fetch the article, and your Slack alerts look like spam.
Google News Scraper with Original Article URLs (disclosure: I built it) resolves those links to the publisher's real URL and returns headline, publisher, date and snippet for any query, language and country.
from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("fujiwaranosai/google-news-scraper").call(run_input={
"queries": ['"Anthropic" OR "OpenAI"', "Tesla -recall"],
"timeRange": "1d",
"language": "en",
"country": "us",
"onlyNew": True,
})
for a in client.dataset(run["defaultDatasetId"]).iterate_items():
print(a["published_at"], a["publisher"], a["title"], a["article_url"])
Ideas
-
PR monitoring: one query per brand/executive, schedule hourly with
timeRange: "1h"andonlyNew. -
Market research: track tickers or sectors per country edition (
es/mx,pt/br,de/de). -
AI digests: pass
article_urlto your content crawler and summarize the day's news with an LLM.
Google News operators work too: "exact phrase", OR, -exclude, site:reuters.com. The first 100 articles of every account are free.
More ready-to-run examples: https://github.com/novosai0day-a11y/public-data-apis
來源:DEV Community · dev.to