Testing whether a job postings indicator adds to official statistics

Testing whether a job postings indicator adds information beyond official labour market statistics is a contest between two forecasts of one published figure: a model that sees only the official series' own past, and the same model with the indicator added. This guide gives you the design, the official vintage tables it depends on, how central bank economists have run it and a worked example. It makes no claim of its own for any dataset, and nothing in it is investment advice.

How do you test whether a job postings indicator adds information beyond official labour market statistics?

Fit two models to one official target. The benchmark uses only the target's own earlier values. The second is the same model with the postings indicator added. Forecast a held-out period one date at a time, using only figures published by each date, then compare the two sets of errors and state the difference with its interval.

The NIST/SEMATECH e-Handbook of Statistical Methods describes an autoregressive model as a linear regression of the current value of a series against one or more prior values. An indicator that only repeats what those prior values say can track the official series on a chart and add nothing here. Fix six things before you look at a result.

The handbook defines the error as the true value minus the estimate and the MSE as the mean of the squared errors, and calls the estimator with the smallest MSE the best. Its confidence limits for the mean are the mean plus or minus a percentile of the t distribution times the standard deviation over the square root of the sample size. Apply them to the per-period difference between the two models' squared errors: limits that contain zero leave the hypothesis of no difference standing. Most standard tests depend on randomness, so check that the autocorrelations of the differences are near zero.

The official series to test against, and where first-release values are kept

An official figure for one period has several values: a first release, its revisions and a benchmarked figure. The test has to know which one each forecast date could see. The table follows each publisher's own pages.

The BLS vintage tables follow the published value of each reference month from its first preliminary release to the current one. In the Philadelphia Fed's Real-Time Data Set for Macroeconomists each column is the whole history as available on its vintage date, and ALFRED returns each release of a series that was available on a specific date.

If you target the benchmarked figure, add the first release to the benchmark as a competitor. The target has sampling error of its own: the BLS puts the confidence interval for the monthly change in total nonfarm employment, at the 90 per cent level of confidence, on the order of plus or minus 122,000.

A methodology for nowcasting employment with alternative data and job postings

Choose the employment series and the release you want to predict, then build the postings indicator on the same reference period. Estimate a benchmark on the series' own published history and a second model that adds the indicator. Re-estimate both at every forecast date on the vintage then available, and score them on held-out periods.

1. Line up the reference period. Payroll employment refers to the pay period that includes the 12th, job openings to the last business day of the month. Read a stock of open postings on that day, or a flow of new postings over the weeks that end there.
2. Set the forecast date by rule, such as the day before the official release, and admit only indicator rows that existed by then.
3. Build the real-time table. For each forecast date, store the official history exactly as it stood on that date, from the vintage tables.
4. Settle both models on the estimation sample. The indicator enters as one added variable, in a form chosen before the hold-out begins.
5. Roll forward. Estimate on an initial sample, forecast the next period, add that period and repeat until the data end. Report every specification you tried.

Nowcasting job vacancies: a methodology for incremental predictive power over official statistics

Take the official vacancy estimate as the target and its publication lag as the gap the nowcast fills. The benchmark carries the last published vacancy figures forward. The test model adds postings observed since the last published period. Incremental predictive power is the fall in out-of-sample error against that benchmark, reported with its confidence limits.

In a box of its Economic Bulletin, Issue 2/2025, European Central Bank economists Claudia Foroni and Christofer Schroeder note that the flash estimate of the euro area job vacancy rate typically comes around one and a half months after its quarter ends, and the first release around three months after. They forecast the rate with MIDAS regressions, which predict a low-frequency variable from high-frequency data, on an indicator built from corporate earnings calls, and report that forecasts built on online job postings came out broadly in line, especially in the short term.

A three-month average, the headline of the UK release, shares two months with the one before it, so forecast the single-month figure or model the overlap.

How Federal Reserve economists approach nowcasting the labour market with alternative data on job postings and employment

Federal Reserve economists have published two designs. In a FEDS Note of 20 September 2019, Board staff built an employment series from a payroll processor's microdata and pooled it with the official payroll survey. A June 2025 Chicago Fed Letter describes an unemployment nowcast blending official job flows with private indicators, including a job openings rate from online postings.

The FEDS Note, by Tomaz Cajner and four co-authors giving their own views, estimated employment changes independently of the BLS payroll series. A Kalman filter placed roughly equal weight on the two, the same exercise on the official series alone gave a confidence interval around underlying employment gains about 20 per cent wider, and the pooled estimate held information about future readings of payroll employment.

Chicago Fed Letter 506, by Scott Brave, Ben Henken and Ezra Karger, whose opinions are their own, describes a weekly tracking estimate of the BLS unemployment rate. The authors backtested it by iteratively predicting out-of-sample values from January 2018 to April 2025 and report a mean absolute error of 0.20 percentage points, against 0.24 for the consensus forecast and 0.31 for a random walk. They add that their predictions used revised data and current seasonal factors, while the consensus used unrevised data: a disclosure of vintage worth copying. They call it an open question how helpful these alternative data are more generally for predicting unemployment.

Does an academic paper show incremental predictive information in a job postings indicator for nowcasting official employment statistics?

The published tests we read are central bank studies. A Reserve Bank of Australia Bulletin article of 24 April 2025 found that models holding only a job advertisements series forecast Australia's pre-pandemic unemployment rate with lower errors than two benchmark models, significantly so on average over the first three quarters of the forecast period. This page quotes official sources only.

That article, by Joyce Tan, is the closest match to this design among the studies we read. Its benchmarks are an autoregressive model of the percentage change in the unemployment rate with one lag, and a white noise model whose expected change is zero. Each indicator model holds up to two lagged values of one indicator. Forecasts come from a model estimated on an initial subset of the data, with a quarter added each time, and are evaluated over the period before the pandemic by root mean squared error and a Diebold-Mariano test. A summary of three indicators did as well, significantly so on average over the first six quarters.

Nowcasting technology sector employment with alternative data: job postings, payroll employment and the research

A technology sector can be defined in various ways, so define the target first. A Bureau of Labor Statistics article of May 2018 identified high-tech industries by their share of STEM jobs, software publishers and computer systems design among them. Searching the statistics offices and central banks cited here, we found no study that nowcasts technology employment from job postings.

The BLS article, by Brian Roberts and Michael Wolf, notes that the definition chosen can affect an analysis. It counts a service-providing industry as high-tech when STEM workers held more than 14.7 per cent of its jobs in 2016, at least two and a half times the average for all service-providing industries. Your target is then payroll employment summed over the industries you choose, each checked for a BLS vintage file.

Industry employment counts everyone at a software publisher, so use the postings of companies in the chosen industries, not software roles at every employer.

Worked example: an illustrative hold-out test for Acme Robotics

Acme Robotics is an invented maker of warehouse robots, and this example is illustrative: every figure is invented, including those that stand in for an official series. Its strategy team asks whether a postings indicator improves a forecast of the first-release payroll change in its customers' industry.

Assumptions:

• The target is the first-release monthly change in the industry's payroll employment, in thousands of jobs.
• Twelve months are held out. Each forecast was made the day before the release, by a model re-estimated on the values published by then.
• The benchmark is an autoregressive model on the target's published history, and the test model adds a postings indicator read at the reference period.
• The error is the first release minus the forecast. The last column is the benchmark's squared error less the test model's.

The working:

1. Mean squared error. The benchmark's squared errors sum to 741 and the test model's to 400: 741 / 12 = 61.75, a root of 7.86, against 400 / 12 = 33.33, a root of 5.77. The root mean squared error is 26.5 per cent lower.
2. Difference by month. The differences sum to 341, a mean of 28.42, and the test model was closer in 8 of the 12 months.
3. Spread. Their standard deviation is 52.42, so the standard error of the mean is 52.42 / √12 = 15.13.
4. Confidence limits. For 11 degrees of freedom the handbook's t table gives 2.201 for 95 per cent limits: 2.201 × 15.13 = 33.30, so the limits are 28.42 plus or minus 33.30, about -4.9 to 61.7. Zero lies inside.
5. Concentration. Months 4 and 9, the two in which employment fell, supply 144 + 105 = 249 of the 341, or 73 per cent.

Acme's team has an error a quarter lower and, on two-sided 95 per cent limits, no evidence yet that the gain is more than chance: two months carry it. Had the same mean and spread held over 24 months, the table's 2.069 for 23 degrees of freedom would give 2.069 × 52.42 / √24 = 22.14 and limits of about 6.3 to 50.6. The count of periods is a property of the test.

Running the test on weekly market series from Fokals

Market Series is the Fokals dataset for the indicator side: 28 families of weekly series across 21 industries, countries, company size bands and markets, over windows of 7, 30 and 180 days, as of each Sunday. Each row carries a count, its cohort, a rate, the previous window, growth and an index. Two properties give the indicator what vintage tables give the official side.

• The as-of date is the vintage. Rows are written once for each as-of date, after the week closes, and never rewritten, so the record is point-in-time by construction, as why we write each table once sets out.
• Coverage growth stays out. A rate is taken over a same-store cohort: companies or websites covered since before the window began and still covered at its end. For share and pay metrics the cohort is postings.

The query pairs each of your forecast dates, a table of your own, with the latest row whose as-of date lies before it. The metric, the dimension, the window and both dates are parameters.

select f.target_period,
       f.forecast_date,
       s.as_of,
       s.rate,
       s.growth,
       s.cohort
from my_forecast_dates f
join lateral (
  select m.as_of, m.rate, m.growth, m.cohort
  from market_series m
  where m.metric         = :metric
    and m.dimension_kind = :dimension_kind
    and m.dimension      = :dimension
    and m.window_days    = :window_days
    and m.as_of          < f.forecast_date
  order by m.as_of desc
  limit 1
) s on true
where f.forecast_date between :first_forecast_date and :last_forecast_date
order by f.forecast_date;

Use the rate or the growth as the indicator, because each as-of date has its own cohort. Market Series classifies companies by its own industries and by headquarters country, so map those to the official classification once and keep the map fixed. Store the day you loaded each row, as building a point-in-time dataset for back-tests describes. The data dictionary defines every field in the query.

Frequently asked questions

The queries and code on this page are examples to adapt. Test them in your own environment before you rely on them.

What this page says about the products it names was checked against their public documentation on 7 October 2026. Product and company names are trademarks of their owners. Fokals is not affiliated with them or endorsed by them.