Building thematic baskets from technology adoption data

A thematic basket groups companies by exposure to an idea: running a given commerce platform, accepting a given payment method, adopting a category of tool. The membership rule is the product, so it has to be written down, dated and repeatable. This guide builds baskets from technographic data, using the Technology Stack and Technology Changes datasets. It joins them to listed securities, rebalances on dated additions and removals, tests the result, and shows how to read website technology as a proxy for exposure.

Write the theme as a rule over the datasets

A company's website is refreshed daily to weekly, and Fokals detects technologies from page content, scripts, network requests, response headers and DNS records. The catalogue recognises 6,283 technologies in 68 categories. Technology Stack holds one row for each technology on a company website, with its first-seen and last-seen dates, its category and the view that detected it. The marketing stack dataset describes the three datasets that hold the data. In the rules that follow, the identifiers are those of company_technologies, which holds Technology Stack, and company_tech_events, which holds Technology Changes.

A theme maps onto the catalogue: choose the technologies, or the categories, that express it. Three kinds of rule are available, and each answers a different question.

• Stock rule. Companies that run a technology now: a row for the technology ID with missing_since empty and a recent last_seen_at. The question is who is exposed today.
• Flow rule. Companies that adopted it in the last N days: a company_tech_events row with category of technology and change of added. A commerce platform migration is recorded as an event with category of platform, carrying before and after. The question is who is moving.
• Category rule. Companies running any technology of one technology_category, such as commerce platforms or payments, whatever the vendor. The question is who is exposed to a class of tool.

Choose the evidence with seen_via, which records the view that detected the technology: page content, script and network requests, response headers or public DNS records. A tag in page content or a script loads for visitors. A technology seen in DNS records, such as an email provider or a domain verification, shows a back-office relationship, which suits a theme about workplace software and not one about the customer experience.

Stock and flow differ because of the baseline. A website's first observation sets a baseline and writes no events, so a technology present at that observation has a first_seen_at equal to the first observation, which is a record of when it was first seen. Use Technology Changes for dated adoption and Technology Stack for who runs a technology now. A basket built on first_seen_at alone mixes recent adopters with long-standing users.

From domains to securities

A listed group can have several websites, one for the parent and one for each brand. Each is a company with its own company ID, and a brand carries the identifiers of its listed parent, so group the rows on ISIN or FIGI, not on domain. Decide whether a group is a member when any of its sites runs the technology or only when a share of them does, and write the choice down. The first rule is simpler and favours groups with many brand sites. Join ISIN or FIGI to your security master as set out in mapping alternative data to a security master.

select
  t.isin,
  count(distinct t.domain)  as sites_running,
  min(t.first_seen_at)      as first_seen
from company_technologies t
where t.technology = :technology
  and t.missing_since is null
  and t.last_seen_at >= current_date - 14
  and t.isin is not null
group by t.isin;

The last_seen_at condition keeps out a technology that no recent observation has seen. The isin condition keeps listed companies and the brands of listed parents, which is the list a basket needs.

Rebalance on dated events

Technology Changes records each change between two readings, with observed_at, change (added, removed or changed), key (the technology ID) and the technology's name and category. Each event carries the time it was observed, so a rule that uses observed_at never uses an event before it existed. Four rules decide how a basket moves.

1. Effective date. Add a member on the first trading day after observed_at, not on observed_at itself, and record the date. Rows reach you daily to weekly, so the next trading day is the first day a trade could have used the event.
2. Persistence before inclusion. Tags come and go with consent banners and tests. Require the technology still to be present, with missing_since empty, a set number of days after the added event, 14 for example, before the company joins. This rule is yours. Removals are already confirmed before they are written, so a removed event is firm.
3. Exit on removal. A technology marked missing is not yet counted as removed. Keep the member until the removed event arrives, then apply the same lag as for entry.
4. A turnover budget. Count joins and leaves each month before you trade on them. A rule that flips a tenth of the basket in a month may be measuring noise as much as adoption.

Two kinds of event need care. A platform migration is recorded as a platform event, and it moves a company out of the basket for the old platform and into the basket for the new one at the same observed_at. And an event with category of technology_id records a new account ID under a technology that is already present, so it is a sign of expansion and should not trigger a join.

select
  e.isin,
  min(e.observed_at) as added_at
from company_tech_events e
where e.category = 'technology'
  and e.change = 'added'
  and e.key = :technology
  and e.observed_at >= current_date - 90
  and e.isin is not null
group by e.isin;

The query returns the companies that added the technology in the last 90 days. Apply the persistence test to each before it joins, and keep the date it first qualified so that the basket can be rebuilt as it stood on any day.

The timeline below is illustrative, for the invented company Acme Robotics and a rule with a 14-day persistence test and a one-day lag.

Weights and versions

Weights come from your own market data. Equal weight is the neutral start, and a cap on any one sector stops a theme from becoming a sector bet. Freeze each basket under a version: the rule text, the technology IDs, the lags, the weights and the date of the data you used, so that you can explain any past membership and reproduce any past rebalance. Fokals datasets are written once and never revised, and labels and scores come under named, frozen versions, so the same discipline holds all the way down.

Test the basket before you trade it

• Coverage. Compare the basket's members by country and sector with your list, so that you know which markets the basket speaks for.
• Flicker. Measure the share of added events followed by a removed event for the same company and technology within 30 days. A high share says the persistence test needs more days.
• Concentration. Count members by sector, size and country. A basket of commerce platform adopters may turn out to be a retail basket under another name, so measure its return against sector peers and not only against an index.
• Market-wide adoption. Compare the basket with the technology added and technology prevalence families in Market Series, which count adoption over same-store cohorts of websites. Members adopting faster than the market is a different claim from members adopting at the market's pace.

How to read a technology as exposure

Website technology is a proxy, and a good one when the theme is a tool a company runs. A company running a commerce platform on its website is exposed to online sales, and the degree of that exposure comes from the company's own reports and your market data, which you join on ISIN. The dated Technology Changes record gives the timing, and your own data gives the weight.

A detection shows presence on the company website. It is binary, so a tool on a website does not distinguish a pilot from a full rollout, or one brand's site from the whole group. Counting sites per group, as the first query does, adds the breadth.

Every observation is dated and the record is point-in-time by construction, so a basket can be back-tested as it would have stood on any past day, using only what had been observed by then. If your question is about the vendor's growth and not its adopters', see reading technology adoption as a signal of vendor revenue. For the dating of observations in a test, see building a point-in-time dataset for back-tests.

How Fokals delivers it

The marketing stack dataset is refreshed daily to weekly, depending on the company. It reaches you through the REST API or as bulk files, and the data dictionary defines each field used here.

Frequently asked questions

The queries and code on this page are examples to adapt. Test them in your own environment before you rely on them.