跳到正文
DEV Community· frankxie·· 2 小時前AI 評分48

如何用 Python 追蹤 Greenhouse、Lever、Ashby 和 Workday 招聘網站上的數據工程師職位

Which Companies Are Hiring Data Engineers? Track Job Openings on Greenhouse, Lever, Ashby and Workday Career Sites with Python

AI 導讀

教程示範如何用 Apify Actor 從 Greenhouse、Lever、Ashby、Workday 等招聘系統抓取職位,並將資料整理成統一格式,再以 Python 追蹤各公司招聘情況。Actor 支援多種職位、地點及日期篩選,亦可匯出 JSON、CSV 或 Excel;文中一次測試用約 64 秒從六家公司取得 28 個職位。

正文

Job boards are a few days late and full of reposts. If you want to know which companies are hiring for a role right now, the most reliable place to look is each company's own career site. Recruiters use this to find teams that are growing, sales teams use it because a company that hires is a company that buys, and job seekers use it to catch new roles before they spread.

The catch is that career sites run on different systems. Greenhouse, Lever and Ashby each publish a public job board API, but each one has its own URL scheme and its own field names. Workday career sites page through jobs in small batches and never return more than 2,000 results for one search, so a big employer needs several searches. Tracking 30 companies means writing and maintaining several clients, then mapping them all to one format.

This tutorial uses an Apify Actor that reads all of these systems and returns the same columns for every job.

Disclosure: I built and maintain this Actor. It is a paid tool ($1.00 per 1,000 results). I've tried to describe what it can't do as plainly as what it can.

What you can do with the data

  • See who is hiring for a role. Count open "data engineer" jobs per company across a list you care about.
  • Spot new and closed roles. Run the same list every day and compare jobId values.
  • Salary research. When a company publishes pay, it lands in salaryMin and salaryMax.

Step 1: Set up a run in Apify Console

Open Career Site Jobs Scraper and click Try for free. The input form has these fields:

Field What it does
Career pages (careerPages) One company per line: a job board URL like https://jobs.lever.co/spotify, a short id like greenhouse:stripe, or the company's own careers page.
Max jobs per company (maxJobsPerCompany) Stops after this many matching jobs per company. Newest first where the board shows dates.
Max results (maxItems) Total limit for the run. Default 1000, so raise it for a long company list.
Job title contains (titleKeywords) Keep jobs whose title contains one of these, e.g. data engineer. Not case sensitive.
Job title does not contain (excludeTitleKeywords) Skip jobs whose title contains one of these, e.g. manager.
Location contains (locations) Keep jobs whose location or country contains one of these, e.g. London, Germany.
Remote jobs only (remoteOnly) Keep only jobs marked as remote. Hybrid jobs are not included.
Posted within (days) (postedWithinDays) Keep only jobs posted in the last N days. Jobs without a date are skipped.
Department or team contains (departments) e.g. Engineering, Sales.
Include job description (includeDescription) On by default. Turning it off makes Workday, SmartRecruiters and BambooHR runs much faster.
Max description length (descriptionMaxLength) Cuts descriptions to keep the dataset small.

Supported systems are Greenhouse, Lever, Ashby, Workday, SmartRecruiters, Workable, Recruitee, Personio, Teamtailor and BambooHR. The Actor recognises the system from the URL, and no login or proxy is needed. To find the URL, click through from the company's careers page to its list of jobs and copy the address bar. For Workday, keep the part after the domain, as in https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite.

Step 2: Run it and export

Click Start. When the run finishes, open the Output tab and export as JSON, CSV or Excel.

For this article I checked six companies on four systems for data and machine learning engineering roles:

{
  "careerPages": [
    "https://job-boards.greenhouse.io/airbnb",
    "https://boards.greenhouse.io/stripe",
    "https://jobs.lever.co/spotify",
    "https://jobs.ashbyhq.com/openai",
    "https://jobs.ashbyhq.com/notion",
    "https://nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite"
  ],
  "titleKeywords": ["data engineer", "analytics engineer", "machine learning engineer"],
  "maxJobsPerCompany": 5,
  "maxItems": 30,
  "includeDescription": false
}

It took about 64 seconds and saved 28 jobs from all six companies (Notion had 3 matches, the others hit the cap of 5). Here are two of them, with some fields removed (values unchanged):

[
  {
    "company": "OpenAI",
    "companySlug": "openai",
    "ats": "Ashby",
    "jobId": "2796d32a-9f7c-4008-a2c5-50dd53b0f2fe",
    "title": "Data Engineer, Core Experimentation",
    "department": "Applied AI",
    "team": "Applied AI Engineering",
    "location": "Seattle",
    "country": "United States",
    "countryCode": "US",
    "workplaceType": "hybrid",
    "isRemote": false,
    "employmentType": "Full-time",
    "salaryMin": 347000,
    "salaryMax": 385000,
    "salaryCurrency": "USD",
    "salaryPeriod": "year",
    "salaryText": "$347K – $385K • Offers Equity",
    "postedAt": "2026-10-06T20:36:36Z",
    "jobUrl": "https://jobs.ashbyhq.com/openai/2796d32a-9f7c-4008-a2c5-50dd53b0f2fe",
    "careerPageUrl": "https://jobs.ashbyhq.com/openai",
    "scrapedAt": "2026-11-02T13:26:50Z"
  },
  {
    "company": "Spotify",
    "companySlug": "spotify",
    "ats": "Lever",
    "jobId": "05b8d6ee-0773-4255-b8b9-ae73d74a5ee5",
    "title": "Senior Machine Learning Engineer, Personalization",
    "department": "Engineering",
    "team": "Personalization",
    "location": "New York, NY",
    "country": "United States",
    "countryCode": "US",
    "workplaceType": "remote",
    "isRemote": true,
    "employmentType": "Full-time",
    "salaryMin": null,
    "salaryMax": null,
    "postedAt": "2026-10-14T16:28:30Z",
    "jobUrl": "https://jobs.lever.co/spotify/05b8d6ee-0773-4255-b8b9-ae73d74a5ee5",
    "careerPageUrl": "https://jobs.lever.co/spotify",
    "scrapedAt": "2026-11-02T13:26:49Z"
  }
]

A few things stood out in this small run:

  • Every word of a phrase must appear in the title, in any order. Words match from their start and common endings count, so engineer also finds "Engineering". That is why analytics engineer also matched "Software Engineer, Analytics Infrastructure" at Airbnb, and machine learning engineer matched "Senior Machine Learning Engineering Manager" at Spotify. Use Job title does not contain for words like manager.
  • Workday rows are thin without descriptions. 4 of the 5 NVIDIA jobs had no location and 2 had no postedAt. Workday needs one extra request per job for those fields, so turn descriptions on when you need them.
  • Departments are written the way each company writes them. Stripe's interns were listed under "5112 General University".

Step 3: See who is hiring with Python

Install the official client and pandas:

pip install apify-client pandas

Run the Actor for your company list. Leave out maxJobsPerCompany so you get every matching job:

from pathlib import Path

from apify_client import ApifyClient
import pandas as pd

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("pipiagent/career-site-jobs-scraper").call(run_input={
    "careerPages": [
        "https://job-boards.greenhouse.io/airbnb",
        "https://boards.greenhouse.io/stripe",
        "https://jobs.lever.co/spotify",
        "https://jobs.ashbyhq.com/openai",
        "https://jobs.ashbyhq.com/notion",
        # ... the rest of your list
    ],
    "titleKeywords": ["data engineer", "analytics engineer"],
    "excludeTitleKeywords": ["manager"],
    "includeDescription": False,
    "maxItems": 5000,
})
items = list(client.dataset(run.default_dataset_id).iterate_items())
df = pd.DataFrame(items)

Now one row per company: how many matching roles, the newest posting, how many are remote, where they are, and the median published pay. Pay is only averaged over yearly USD ranges, so currencies don't get mixed:

df["postedAt"] = pd.to_datetime(df["postedAt"], utc=True)
usd_year = df["salaryCurrency"].eq("USD") & df["salaryPeriod"].eq("year")
df["salaryMid"] = ((df["salaryMin"] + df["salaryMax"]) / 2).where(usd_year)

summary = (
    df.groupby("company")
    .agg(
        open_roles=("jobId", "count"),
        newest_post=("postedAt", "max"),
        remote_roles=("isRemote", lambda s: s.eq(True).sum()),
        countries=("countryCode", lambda s: ", ".join(sorted(s.dropna().unique()))),
        median_salary=("salaryMid", "median"),
    )
    .sort_values("open_roles", ascending=False)
)
print(summary)

Run on my 28-job sample, it prints this. The counts stop at 5 only because my test run capped each company at 5:

         open_roles               newest_post  remote_roles   countries  median_salary
company
Airbnb            5 2026-10-16 16:32:46+00:00             5      BR, US       226000.0
Nvidia            5 2026-10-31 00:00:00+00:00             0          CN            NaN
Spotify           5 2026-10-23 16:53:57+00:00             1  GB, SE, US            NaN
OpenAI            5 2026-11-02 01:05:15+00:00             0          US       292000.0
Stripe            5 2026-10-30 16:19:28+00:00             0      CA, US            NaN
Notion            3 2026-10-28 18:42:47+00:00             0          US            NaN

To track changes, save each run and compare it with the previous one. jobId is only unique within one job board, so the key combines the system, the company and the id:

df["key"] = df["ats"] + ":" + df["companySlug"] + ":" + df["jobId"]

snapshot = Path("jobs_previous.csv")
if snapshot.exists():
    before = pd.read_csv(snapshot)
    new_jobs = df[~df["key"].isin(before["key"])]
    closed = before[~before["key"].isin(df["key"])]
    print(f"{len(new_jobs)} new, {len(closed)} closed since last run")
    print(new_jobs[["company", "title", "location", "jobUrl"]].to_string(index=False))
df.to_csv(snapshot, index=False)

Schedule this daily and the "new" list is your alert. Keep the filters the same between runs, or a changed filter will look like opened and closed jobs.

What it costs

You pay per saved job: $1.00 per 1,000 jobs, with no start fee and no monthly rental. Filters run before jobs are saved, so jobs that don't match are free.

  • My 28-job test: about $0.03.
  • A filtered daily run that saves 100 jobs: $0.10 per run, about $3 for a 30-day month.
  • All 2,683 NVIDIA jobs (the largest board in the README's tests): $2.68.

You can also set a maximum charge per run in the run options, and the Actor stops when it reaches that amount.

Limitations

  • You choose the companies. The Actor reads the job boards you give it. It does not search every company on the internet.
  • Big boards take minutes. Per the README, all 2,683 NVIDIA jobs took about 3 minutes without descriptions and about 11 minutes with them. Small boards finish in seconds. Don't run many copies against the same Workday site at once; it slows down.
  • Company names come from the URL on Lever, BambooHR and Workday ("Nvidia" from nvidia.wd5.myworkdayjobs.com).
  • Country and remote flag are sometimes worked out from text, such as "Remote - Germany", which is right in most cases but not always. Where the board sets the flag, you get the company's value: one OpenAI job located "US - Remote" came back as hybrid.
  • Salary only when published. Greenhouse ranges don't say per year or per hour, so ranges below 500 are marked hourly.
  • Not supported: iCIMS, Taleo, SuccessFactors, Jobvite and custom-built career sites.

Related tools

If all your companies use one system, there are single-platform versions with the same output: Greenhouse Jobs Scraper, Lever Jobs Scraper, Ashby Jobs Scraper and Workday Jobs Scraper. For remote jobs from job boards instead of employers, Remote Jobs Scraper merges Remote OK, We Work Remotely, Jobicy and Working Nomads.

If you want to try it with your own company list, the Career Site Jobs Scraper store page has the input form and more examples. If a company's job board isn't recognised, open an issue there with the URL and I'll look into it.

來源:DEV Community · dev.to