A label version names the frozen questions, rules and model used to assign labels to records, such as a company's industry or a posting's job function. Recording it beside each label, as part of data provenance, lets you reproduce the label, compare labels made at different times and see when the method changed.
Why labels need versions
Labels produced by a classifier change when the classifier does: a new model, a different question wording or a new confidence cut-off moves records between classes. Without a version, a change in the count of senior roles could be a change in hiring or a change in the labeller, and nobody could say which. Freezing the method under a name makes a label reproducible, lets accuracy be measured per version and lets a buyer who built a product on a label decide when to move.
Working with two versions
Store the version with every label you load. Before comparing periods, count rows per version and check whether the new version keeps the meaning of the old keys. Then choose: filter to one version, analyse both and compare, or relabel history yourself. A change in a share across a version boundary deserves a second look. This query shows the postings under each version:
select label_version, count(*) as postings
from job_postings
group by label_version;Label versions in Fokals data
Every label in Fokals data is produced under a named, frozen version, and the version travels with the record. Job Postings are labelled under jobs-v2, with jobs-v1 on earlier postings. Website labels carry sites-v2, Company News carries news-v1 or sec-items-v1, sales postings sales-v1, leadership roles roles-v1 and Intent Scores intent-v2.
A posting keeps the version it was labelled under until it closes, while a website takes the current version at its next refresh. Every jobs-v1 key keeps its meaning in jobs-v2. A breaking change ships as a new version with at least 90 days' notice. The methodology lists the versions, and the data dictionary lists the labels each carries.
Related terms
A label version is named in the data dictionary and in a data manifest, and it is one part of data provenance. Frozen versions also protect a back-test from silent changes, the concern of point-in-time data.
Frequently asked questions
What happens to old records when a label version changes?
That depends on the vendor's policy, so ask. In Fokals data a posting keeps the version it was labelled under until it closes, and a website takes the current version at its next refresh. Every jobs-v1 key keeps its meaning in jobs-v2, so older rows stay comparable on the keys they share, and the keys added in jobs-v2 sit on rows labelled under it.
How is a label version different from a dataset version?
A dataset version or release identifies a snapshot of the whole data. A label version identifies the method behind one kind of label, and it moves independently of the data. A vendor can refresh data every day under one label version and change that version only when the questions, rules or model change. Store both if you need to reproduce an analysis exactly.
How is the accuracy of a label version measured?
By hand, per version. A person is shown a random labelled website or posting with the evidence the model read, and grades each label field as correct, wrong or unsure. Precision is correct divided by correct plus wrong, and an unsure verdict decides nothing. A new version starts its own count, and Fokals publishes a precision figure for each version, graded by hand on at least 100 websites or 100 postings.
The queries and code on this page are examples to adapt. Test them in your own environment before you rely on them.