All documentsMethod · Version 2.7

Methodology

What every dataset holds, how it is collected, dated and refreshed, in overview.

This overview describes the Fokals company-signals datasets: what each one holds, what it covers, the kinds of public source the data comes from, how it is collected and dated, how often it is refreshed, and how changes are managed. The section numbers are those of the full methodology, which is shared with clients and with teams evaluating the data. Field definitions are in DATA_DICTIONARY.md; how the data is sourced, and what is never collected, is in SOURCING.md.

1. Products

ProductWhat it answersDatasetsRefresh
Marketing stackWhich technologies a company runs on its own website, what the site offers (markets, languages, currencies, apps, key pages, promotions, pricing) and every change, dated; each vendor's companies won and lost by month; how ready each company is for AI agentsTechnology Stack, Technology Changes, Website Profile, Vendor Scorecards, Agent Readiness, Agent Readiness ChangesDaily to weekly per company; scorecards monthly; readiness weekly
HiringWhat a company is hiring for, posting by posting and day by day, how its sales organisation is built and paid, and the projects its postings stateJob Postings, Hiring Activity, Sales Team Metrics, Sales Pay Benchmarks, Job Posting ProjectsDaily
IntentWhat a company is likely to buy or build soon, by topic, with the evidence; the technologies it is likely to add next, with their support counts; the sequences that come before an adoptionIntent Scores, Company Signals, Company Funding, Likely Next Adoptions, Adoption SequencesWeekly (signals daily)
Announcements and scaleWhat a company announces about itself, its headcount over time, its website's traffic tier month by month, worldwide and in each country, and its health each weekCompany News, Employee Headcount, Web Traffic, Traffic Tier History, Company HealthDaily to every three days; headcount yearly; traffic monthly; health weekly
Filings and recordsWhat a company's insiders and large holders file, line by line; its financial statements as filed, so every figure can be read as it was known on any day; the events its current reports disclose; and the public records that describe itInsider Filings, Insider Transactions, Large Holder Filings, Financial Statements As Filed, Company Events From FilingsDaily; financial statements with each filing
Market seriesHow industries, countries, size bands and markets moved over 7, 30 and 180 days, on same-store cohortsMarket SeriesWeekly, as of each Sunday

Every file carries the company's identifiers, so the products join to each other.

2. Sources

All sources are public and first-party: the company publishes the information itself, or it is a public record. The kinds of source are:

  • Company websites: the homepages of each company's own domains, the tags they load, and the careers and pricing pages they link to.
  • Public DNS records of each company's domain, and the domain registries' public records of its registration and expiry.
  • Company online stores: the public product catalogue of a company's own store.
  • Company job boards that companies publish so that job sites can pick up their postings.
  • Company newsrooms and feeds: the company's own announcements, in its own words.
  • Regulatory filings: private capital raises, current reports of listed companies and the headcount they state, the reports insiders and large holders file, and the structured financial statements companies file.
  • Open reference data: company identifiers and facts published under open licences.
  • Public web traffic data: the monthly popularity tier of each website, with no visitor data of any kind.
  • Central bank reference rates, to put pay on one currency scale.

Open web measurement crawls: public crawls that test website home pages each month and record the technologies each page uses. Only the technologies detected on each company website in each month are kept.

Open data is used under its licence, and any credit a licence asks for is delivered with the data, in its licence notices (LICENCE_NOTICES.md).

3. Collection principles

Collection is by an identified crawler that names itself in every request and links to a public page explaining what it reads and how to opt out. It reads and obeys each host's robots.txt, including crawl delays, before every request; makes one request at a time to a host, with pauses between requests; reads public pages only, with no sign-in and no forms; and never changes its identity. A robots.txt refusal always stops the read, and no challenge is ever solved; how a read that a site's network protection refuses is handled is set out in SOURCING.md. Regulatory filings are read within the regulator's published fair-access rules.

4. Coverage

Coverage is the Fokals company index: listed companies worldwide, the brands they own, and checked private companies, each with one stable company ID. Listed companies headquartered in a priority market are read daily, other listed companies every three days, and other companies weekly; a weekly website that has shown no change for four readings in a row is read every two to four weeks until it changes again. The priority markets are the United States, Canada, the United Kingdom, Ireland, the European Union, Switzerland, Norway, Australia, New Zealand and Japan; no country is excluded.

4.1 Listed companies

Every listed equity issuer worldwide is in the index, with its ticker, venue and MIC, its ISIN, LEI and share-class FIGI, and its CIK where it has one. Only an issuer's own equity counts: common and foreign shares, depositary receipts, REITs and partnership shares. Each listing is tied to a company by its identifiers first and by its exact name in the same country last, and states how it was matched. A listing that stops appearing is marked delisted and kept, so history joins stay intact. An ISIN or LEI is stored only when its check digit is valid.

4.2 Languages

The datasets cover websites written in English. A site that publishes in several languages is read in the English version the site itself declares or links. A website's language is judged from the page's own words, with the language it declares as the tie-breaker, under a versioned rule; when the rule changes, every website is judged again from the words stored with its last reading. A company is delivered once one of its websites is read in English.

5. Marketing stack

5.1 Detection

Each reading combines up to seven views of a site: the homepage, the tag-manager containers it loads, a browser read for script-built pages, the response headers, the public DNS records, the site's robots.txt, and the llms.txt file at its root. A technology is detected when its known signature appears in one of these views.

The catalogue holds 6,324 technologies in 69 categories, in two lists. The fokals list holds the technologies whose signatures Fokals writes and tests: advertising, analytics, consent, commerce and content platforms, marketing automation, CRM, chat, payments, security, workplace software, domain verifications, AI access and readiness, and what a site publishes for machines and states about its security (a sitemap, an MCP server, carts and checkout for AI agents, an OpenAPI description, compliance badges). The open list adds the long tail of business tools and the software a site is built on, from an open-source catalogue of signatures. Every technology row says which list recognised it. An open technology has weekly market series once it is established across the delivered companies.

AI access and readiness. Under the category AI crawler access: the AI crawlers a site's robots.txt keeps off its homepage, one entry per crawler (GPTBot, OAI-SearchBot, ClaudeBot, Google-Extended, PerplexityBot, CCBot, Applebot-Extended, Meta-ExternalAgent, Bytespider); the content signals it states and a refusal of AI training; a text and data mining reservation (TDMRep); and a noai directive. Under Artificial intelligence: WebMCP tools offered to AI agents, an llms.txt for AI assistants, and the AI assistants and AI search tools sold as such. A file that keeps every crawler out is not counted as a stance on AI.

5.2 Site facts

Each reading also records what the site offers: the markets and languages it declares, the currencies of its prices, its own social accounts and mobile apps, the key pages it links to (pricing, demo, free trial, contact sales, careers, investors, newsroom, partners and others) and the message in its announcement bar.

5.3 Website labels

Each company website carries labels for industry (21 values), business model, offering, sales motion, price tier, target customer size, pricing model, geography, primary persona and growth stage; three 0 to 3 scores; and fifteen flags. A site's labels are renewed when what it shows changes.

5.4 Changes

The first reading of a site sets its baseline and writes no change: every change in Technology Changes was seen happen between two readings. A technology absent from a reading is recorded as removed only when a later reading confirms it, and only a reading that covered the view it was seen in can call it missing. When the catalogue grows, the first reading with a new signature or a new view sets that technology's baseline. No date in the change record is the date Fokals began to recognise something. On a website built on a hosted platform, the CDN and web server in front of it are the platform's, so a change in them is not recorded as the company's. Nor is a vendor's update that puts its mark on every website already running it. A change found afterwards not to have been the company's doing stays in the record, flagged as excluded, and is left out of every count and list of changes.

Technology Changes also records what a site's pricing page shows changing (a price raised or cut, a plan or a free plan added or removed, usage-based or per-seat pricing or an AI add-on taken up or dropped, prices given way to contact sales), a website label that moves, and a reworded title or description. The first reading of a pricing page sets its baseline.

5.5 Vendor scorecards

For each technology companies take up and drop, each month and each segment (all companies, size band, industry, growth stage and country): the companies running it at the start of the month, the companies that took it up and dropped it, the churn rate, and the rivals its new companies left and its lost companies moved to. A company counts once, whichever of its websites runs the technology. Industry and growth stage are the companies' labels as they stand when the month is written. Each month is written once, after it closes, and never rewritten; a month not watched from its first day is flagged partial. The API adds the last 30 days, counted on request.

Some hosting providers serve every website they hold through a network of their own choosing. From 16 November 2026 the scorecard of a CDN, of what a CDN adds to the pages and of a web server counts its companies in two parts: those that chose it, whose gains, losses and churn it gives, and beside them those that run it only because their hosting provider serves their website through it.

5.6 Agent readiness

A score from 0 to 100 per company for how ready it is for AI agents, in four parts of 0 to 25: find (AI search and assistants can find the company and what it offers), price (an agent can learn its prices), buy (an agent can buy from it) and act (an agent can act through it). Each part carries the dated evidence behind it, from the company's main website. Scores are read weekly; a company's row changes only when its score or a part does, and every move of the score is kept, dated, in Agent Readiness Changes. A company's first score is its baseline.

5.7 Online stores

For a company whose own website runs an online store and sells online: the size of its catalogue, the products it added in the last 30 days, its median price and currency, the share of its priced products on discount and the share out of stock, read from the store itself once a week (a store found for the first time within days), and every rise or cut of its regular prices, kept with the day it was seen. A sale is not a cut: only regular prices move the record. The store figures are those of stores read in the last 60 days. Store aggregates are delivered, not product listings.

5.8 Domain records

For each company's main website: its email provider and any security gateway in front of it, and its email security, meaning the DMARC policy its domain publishes (reject, quarantine, monitoring or none, whether reports are asked for) and its SPF rule, read with every reading of the website; and its domain's registration date, expiry date, last registry change and registrar, read from the registry's public record before each expiry and at least once a year. Nothing about the registrant, a contact or any person is read or kept, and report addresses are never stored.

5.9 Referring companies

How many other companies' websites link to each company's website, its rank among every company linked from at least one other company's website (0 to 100), and links begun less links ended over the last 90 days. Compared at every reading of a website and ranked daily. A company's own websites never count, and a company counts once however many of its websites link. A website's first reading is its baseline. Counts only: which companies link is never delivered.

5.10 Technology history

The technologies on each company website month by month, from open web measurement crawls that test website home pages each month from desktop and mobile clients: one row per company website and month tested, with the technologies detected as catalogue ids and any name the catalogue does not hold kept as detected. The crawl's series is its own and is never compared with the daily readings of section 5.1. A month the crawl did not test a website is no reading, not an empty one.

Adoptions and losses are worked out within that series: a change counts only when two tested months in a row show it, and is placed between the last tested month of the old state and the first of the new. A website's first tested month, its first month after a change in the crawl's method or a long gap in testing, and a month in which the crawl's own rules moved a technology's count across every website set a new starting point and are no change; a website entering or leaving the crawl's sample is never an adoption or a loss. A removal of a tracking tool in a month a consent tool appeared is marked low confidence rather than dropped.

6. Hiring

6.1 Postings

Every posting on a company's board is kept with its title, department, locations and country, work mode, employment type, posting date and the dates it was first and last seen. From its text are read the pay, on one yearly US-dollar scale, the experience, degree, visa sponsorship and languages required, and the software tools it names. Contact details are removed before the text is stored, and posting descriptions are not delivered.

6.2 Opening and closing

A posting already on a board when the board is first read is marked as such and is not counted as new. A posting is closed only when a later complete read confirms it is gone.

6.3 Labels

Each posting is labelled by job function (26 values), seniority (9 levels), remote policy, contract type, a 0 to 3 technical depth, ten flags (among them AI role, people manager and team build) and fifteen tool families. The label definitions are versioned, and each row names the version it was written under.

6.4 Daily hiring

Hiring Activity is written once for each closed UTC day: open, new and closed postings per company, by function, seniority, country and work mode, with AI roles, tools named and median pay.

6.5 Accuracy checks

Website labels are 97–98% accurate and job posting labels 98–99%, measured field by field on random samples graded against the evidence each label was read from. The samples are graded again every quarter, and every grade is kept.

6.6 Sales organisation

A company's sales organisation is what it advertises: on-target earnings and base pay, the base-to-variable split, quota, ramp, lead mix, deal size and sales cycle as the posting states them, with each sales posting's role, segment and five flags. Sales Team Metrics are written once per company and closed week, and Sales Pay Benchmarks give the quartiles of advertised pay by role and country.

6.7 Projects in job postings

Job Posting Projects gives, for each company, every project its postings state, once however many postings state it: a migration from one system to another, the first team of a job function, a compliance programme it is working towards, an expansion into a country or region, and an AI build. Each comes with its details in readable names and in plain words, the number of postings that state it, when they were first and last seen, and the ids of the postings. The facts are delivered, never the postings' text.

7. Intent

7.1 Signals

A signal is a dated event at one company, tied to one or more of 69 topics in nine groups. Each is delivered with the time it was observed, its kind, its topics and what was seen.

7.2 Weekly scores

For each company, topic and closed week, a score from 0 to 100 is written once, with the dated evidence behind it. A surge flag marks a week in which a company's score for a topic rises sharply against its own recent weeks. Weeks run Monday to Sunday, UTC.

7.3 Likely next adoptions

Likely Next Adoptions gives, each week, the technologies a company does not yet run that it is likely to add next, each with an estimated likelihood (0 to 1) and the changes the prediction stands on. Every prediction carries its support counts: for each change behind it, how many companies made that change and then added the technology, of how many made it. Read the counts with the likelihood. Adoption Sequences gives the sequences themselves: a change, a technology added in a later week, the companies that followed it, the share that did and how that share compares with comparable companies. Both are written once a week and never rewritten.

7.4 Lookalikes at the buying moment

Given up to 20 customers, each with the date it bought where known, and optionally the technology they bought, the API answers with the companies in the state those customers were in when they bought, this week: where each customer's buying moment was found, the changes the customers shared before buying, and the companies ranked, each with its reasons in plain words. Customers and companies already running the technology are left out. The answer is computed on request.

8. Funding, announcements and scale

8.1 Funding

Private capital raises are read from regulatory filings for the offering itself: amounts offered and sold, date of first sale, securities offered and number of investors. The people named in a filing are never read. A filing is tied to a company only when its legal name matches exactly one company in the index.

8.2 Announcements

An announcement is something the company said about itself in public: an item on its newsroom, an entry in its feed, or a current report of a listed company. Each is stored once with its title, date, a short excerpt of the company's own text and the address of the original, and labelled with its event types (13 in all) and five flags. For a leadership change, the role concerned and the direction are labelled; the person is not stored.

8.3 Headcount

Employee counts are kept over time, one row per company, date and source, as the company or the source stated them. No estimate is made between points.

8.4 Traffic tier

The traffic tier of a company's website is its monthly popularity bucket, from the top 1,000 to the top fifty million websites. It contains no visitor-level information.

8.5 Company health

Every company followed has a row each week: a distress score and a breakout score of 0 to 100, the band they put it in (distress, watch, steady, growing, breakout), the reasons that count most in plain words, and the shifts in how it positions itself (moving upmarket or downmarket, turning to AI, selling to a new buyer, prices going up or down). Steady is the common case: most companies, in most weeks, show neither strain nor fast growth, and their rows say so, so every company has a full weekly series. Each week is written once and never rewritten.

8.6 Company groups

Each company's direct and ultimate parent, from the ownership records legal entities report to the legal-entity reference data and from the subsidiary lists listed companies file with their annual reports, and how many subsidiaries it has, with the subsidiaries held as companies in the index. A parent outside the index keeps its LEI, legal name and country. A subsidiary name links to a company only where exactly one company carries it; every other name is kept as filed. Refreshed monthly and with each new annual report.

8.7 Business verification

Whether a company is registered and active, with its registry numbers. From its LEI record: the entity and registration status, legal form (ISO 20275), registration authority and registry number, jurisdiction and the date it was established. For a US securities filer: its CIK, filer category, state or country of incorporation, industry code and latest periodic report. For a UK company: the company number its own website states and, where the UK company register holds it under the company's name, its status, type and date of incorporation. Each company has one status (active, inactive, dissolved or insolvency): the UK register's first, then a filer still reporting, then the LEI record. Refreshed with the monthly LEI read, every weekly filing visit and every reading of the website; a UK number is checked on first finding and monthly after. No officer, owner or address is read or kept.

8.8 Traffic tier history

The monthly traffic tier of each company website as a series of its own, worldwide and within each country, never compared with the current tier of section 8.4. Each row carries the bucket as published and the bucket rounded up to a power of ten, so months on either side of a change in the published buckets compare like with like. A month is delivered once it is read in full; a website not ranked in a month has no row, which means under the source's threshold, not a drop. No visitor data of any kind, and the credit the licence asks for travels with every row.

8.9 Insider and large-holder filings

The reports a company's insiders file when they trade its shares are read line by line: the day, the kind of transaction (bought or sold on the market, granted, exercised, withheld for tax), the shares, the price, the holding after it, direct or indirect, and whether a trading plan set in advance applied. The statements of anyone holding more than five per cent of a listed company are read for the share of the class, the shares held and whether the holder may seek influence or holds passively; a statement filed as text before the structured form carries its filers and none of these figures, which are never estimated. A person is kept as a role and a reference that holds within one company, never a name; an organisation keeps its name. A filing belongs to the company that carries its issuer's registration number with the regulator.

8.10 Financial statements as filed

Twenty figures from the structured financial statements companies file, from revenue and net income to cash, debt and shares outstanding, each as the filing states it: the concept and unit it was filed under, the period it covers and the day it was filed. A restatement is a later row for the same period, never a change to an earlier one, so the figures filed by any day are the figures as known that day. Only figures the filer states under a standard concept are held, and no figure is worked out from others.

8.11 Company events from filings

Every current report a listed company files, with the item numbers it was filed under and the event types they make, in the same thirteen types as announcements (section 8.2). The types come from the item numbers alone, so they are kept apart from the announcements' own types and are never mixed with them; a leadership change read from its item number alone is marked as such.

9. Market series

The company-level tables are aggregated once a week into market series: how an industry, a country, a size band or a market moved over 7, 30 and 180 days, as of each Sunday. Each row carries the count, the cohort it is over, a rate, the same count over the window before, the growth between the two and an index against the series' first as-of date.

Cohorts are same-store: a rate is taken over the companies or websites covered since before the window began, so a growing company index is not mistaken for a growing market. A row over fewer than 20 companies is not written. Rows are written once per as-of date and never rewritten. A run that fails leaves its as-of to the next run, which writes every missing as-of in order; one written more than seven days after it closed is flagged reconstructed.

10. Time conventions

All times are UTC. Daily and weekly rows are written once after the period closes and never rewritten; a period written more than seven days late is flagged reconstructed. Vendor scorecards are written once after the month closes, and one written after the 9th day of the next month is flagged reconstructed. Website changes, postings, signals, announcements and moves of an agent readiness score carry the time they were observed. Every dataset is point-in-time by construction.

The history datasets (the filings and records, the traffic tier history and the private funding filings among them) are point in time as published by the source. Every row carries the source's own time (the day a filing was filed, the month of a traffic tier), when the source made the row public (public_from), the moment the row was recorded, which is never changed, and whether the row was recorded at the time (live) or read later from the source's history (backfill). History read back and the collection that goes on every day share one table, one key and one reading of the source, so a row read back differs from a row read live only in those last two. Rows are never revised: an amendment, a restatement or a revision by the source is a row of its own.

public_from is the time to backtest on: what anyone could have read by a moment is the rows public by then, whether we read them at the time or later. It is never earlier than the source shows and never later than the moment we recorded the row:

  • Securities filings: the end of the day filed, US Eastern time, since the regulator publishes a filing by the end of its filing day.
  • An open web measurement crawl: never before the crawl released a month's results, by a conservative delay after the month, since it states no release time for each crawl; a change is public once every month it rests on is.
  • Public web traffic rankings: the end of the day the source published the month's ranks, as its release notes state.

To rebuild what we held on a day, keep the rows recorded by then instead. The feeds through the API follow the recorded time, so a row read back late still reaches a subscriber.

11. Scope

The datasets describe what companies publish themselves: their websites, job boards, announcements and filings. A detection shows presence on the website; a posting records an intention to hire; advertised pay is what the employer states; an intent score comes with the dated evidence behind it; a project is what a posting states about the company; an agent readiness score describes what the company's main website shows; a health score describes what a company shows in public that week; a prediction comes with the counts of companies behind it. A filing is what the filer states, as filed; a financial figure is the one a filing states, never one worked out from others.

12. Change management

Labels, intent scores, vendor scorecards, agent readiness scores, projects in job postings, company health, likely next adoptions and lookalikes are written under named versions of their definitions. A breaking change to a label, a topic, a score or a project is released under a new version, with at least 90 days' notice. Each history dataset names the reader or rule that wrote each row; a new rule works the series out again from what is stored, under its own version, or starts a new series where the source keeps no copy for us, and an old version's rows are never compared with a new one's.

13. Versions

Every labelled or scored row names the version it was written under, and so does every row of the history datasets. The technology catalogue is at version 10.

The full methodology

Request the full methodology

The complete methodology, with every rule of collection, the label definitions and accuracy checks, the scoring, the versions and the changelog, is shared with clients and with teams evaluating the data.