kestrel actors docs

kestrel actors › Guides

How to scrape Indeed company reviews across every country

Indeed does not serve one global review list. The US site shows the US-visible subset, the UK site shows the UK one, and they barely overlap — so most Indeed review datasets are a country sample presented as a company total. There is a single parameter that fixes it, and a paging bug that will bill you forever if you miss it.

The short version

  1. Find the company slug. It is the part after /cmp/ in the reviews URL: indeed.com/cmp/Google/reviews gives the slug Google.
  2. Ask for the worldwide view. Reading the US domain with Indeed's own worldwide filter returned 6,257 of a declared 6,258 reviews for one company in a single walk, against 4,044 on the default US view.
  3. Bound the walk three ways. Stop on the declared found-review count, on the page number Indeed reports, and on review ids already seen — because a page past the end returns page one again with HTTP 200.
  4. Read the sub-ratings, not the average. Five sub-ratings per review — work-life balance, pay and benefits, security and advancement, management, culture — are what makes the dataset actionable. A zero in any of them means "not rated", not zero stars.

The country split is the whole story

Indeed serves each country domain its own subset of a company's reviews, in that country's language, and the subsets barely overlap. On the day this was verified, one large employer showed:

If you scrape the US domain and call the result "reviews of company X", you have two thirds of them and none of the German ones. If you scrape all 44 country domains you make 44 walks and pay for the overlap.

There is a third option that most people miss: the US domain accepts Indeed's own worldwide filter, and under it a single walk returned 6,257 of the 6,258 reviews. One domain, one walk, essentially everything. That is what the countries: ["all"] value does here — it is not a loop over 44 domains, it is one parameter on one domain.

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("kestrel/indeed-company-reviews").call(run_input={
    "companies": ["Google"],
    "countries": ["all"],          # Indeed's worldwide filter: 6,257 of 6,258 in one walk
    "maxReviewsPerCompany": 0,
    "sort": "newest",
})
rows = list(client.dataset(run["defaultDatasetId"]).iterate_items())
company = next(r for r in rows if r["type"] == "company")
print(company["review_count"], "worldwide;", company["found_review_count"], "on this view")

Use individual country domains when you want the language: a French review is on the French domain, written in French, and that is where an employer-brand analysis for a French site should read.

The paging trap that never terminates

Indeed answers an offset past the last page with HTTP 200 and page one's twenty reviews, reporting currentPage: 1. Never a 404, never an empty list, never an error.

Write the obvious loop — "increment the offset until the page comes back empty" — and it runs forever, re-delivering the same twenty reviews, and if your tool bills per row it bills for every repeat. The genuine last page for the 4,044-review view was at offset 4,040 and returned four items; offsets of 4,060, 6,000 and 20,000 all returned page one.

Three bounds together make the walk safe:

  1. Stop when the offset reaches the declared found-review count for the view.
  2. Stop when the page Indeed reports is not the page you asked for.
  3. Stop when you see a review id you already have.

Any one of them alone can be defeated by a filter or a count that lags; all three together cannot.

What a review row carries

Twenty reviews per page arrive fully server-rendered — there is no XHR, no GraphQL, no cookie and no CSRF token involved in reading them. Each carries:

One detail decides whether your averages are right: a sub-rating of 0 means "not rated", not zero stars. Three of twenty reviews on one company page had them. Treated as zeros they drag a management average from 3.4 to 2.9 and you will read a leadership crisis that is not there. Convert them to null.

The company row you get for free

Before any review is billed, the company row carries Indeed's own rating, the worldwide review count, the 1-5 star histogram, the five category averages, Indeed's Work Happiness score and grade, the topics it extracts with a rating and count each, the pros and cons it summarises, plus industry, size, headquarters, website, founded year, revenue band and CEO.

For a competitor scan that is often the whole answer: five companies, five free rows, one comparison of histograms rather than headline scores. A 4.0 built from mostly fives and a few ones is a very different workplace from a flat 4.0, and only the histogram shows it.

Filters that pay for themselves

run = client.actor("kestrel/indeed-company-reviews").call(run_input={
    "companies": ["Starbucks"],
    "countries": ["all"],
    "maxRating": 2,                 # complaints only
    "requireText": True,
    "jobTitleContains": "barista",  # the population you actually recruit
    "sinceDate": "90 days",
    "sort": "rating_asc",
})

Sorting lowest-first with a rating ceiling means the walk ends at the first page entirely above the cut, so the requests you save are real and not just the rows. At $0.002 per delivered review, a quarterly pull of every negative barista review across five competitors is a few dollars.

The honest limitations

Run it

Indeed Company Reviews Scraper — $0.002 per delivered review, the cheapest employee-review row here, with company rows, status rows, unknown companies and filtered reviews free. Related: which review sites can be scraped without a browser for the Glassdoor evidence in full, and building a voice-of-customer dataset if employee reviews are one input among several.

FAQ

How many reviews does Indeed hold for a company, really?

Two numbers, and they are supposed to differ. The company row's review_count is what Indeed holds worldwide; found_review_count is what the domain you read actually lists. For one large employer that was 6,258 worldwide against 4,044 on the US domain, 215 on the UK domain, 186 on the Canadian one and 13 on the German one.

Is this a Glassdoor alternative?

In practice it is the only one. Glassdoor review pages returned 403 on 23 of 23 attempts across eleven TLS profiles, both proxy pools and a direct connection; headed real Chrome with the automation flag disabled sat on the Cloudflare interactive challenge for 45 seconds and never cleared it, from a residential proxy and from a clean home IP. Page 2 is an explicit 401 redirecting to a login-required screen. Indeed publishes the same shape of data — overall rating, five sub-ratings, pros, cons, job title, location, current or former, employer reply — and answers a plain HTTP request.

Why do some reviews have no pros and cons?

Because Indeed has two submission forms. Reviews written with the pros/cons form carry those fields; the others carry only the body. Both are complete reviews — the fields are absent, not empty.

Does it need a residential proxy?

No. Datacenter with a Chrome TLS profile is the cheaper and slightly more reliable pool: six of six sessions on a dedicated run, against four of six on residential. Roughly one fresh session in three meets a Cloudflare check; rotating the session and retrying clears it, and once a session is served it keeps being served — one session read ten consecutive pages and three other companies without a failure.

Run Indeed Company Reviews Scraper on Apify →

More guides