Retrieval-augmented generation over structured company signals

Retrieval-augmented generation works over company data when each retrievable item is one dated fact with a company and a source attached, and when the questions that a database answers exactly are sent to the database. This guide covers both halves for Fokals rows: how to turn rows into facts, which metadata to keep as filters, how to keep the index in step with the feeds, and a rule for choosing between SQL and vectors.

What there is to retrieve

Most fields in the Fokals datasets are typed: counts, flags, codes and dates, which a database answers exactly. The words sit in a few short fields, and that shapes the design. The table lists where they appear.

The corpus is therefore a set of short, dated statements, each tied to a company and a source. Index at the level of the row, and expect semantic search to earn its place on announcements first. The announcements and scale dataset holds Company News, and every other column named here is defined in the data dictionary.

Turning a row into a fact

Write each fact with code, from a template, one row to one fact. The same row then always gives the same text, so you can hash it, rebuild the index and get an identical result. Keep the label as the label: never paraphrase an event type with a model at index time.

Illustrative facts for Acme Robotics, built this way:

On 2026-10-02, Acme Robotics added Salesforce (CRM) to its website.
2026-09-24: Acme Robotics published "Acme Robotics opens a service centre in Rotterdam". The centre will serve customers in the Netherlands and Belgium.
For the week of 2026-09-21, Acme Robotics scored 68 on CRM from 5 signals.

Keep the date twice: in the sentence, so that the model can read it, and in metadata, so that a filter can use it. Keep each fact to one row and one date. A summary written at index time becomes a claim that nothing in the data supports.

Filters first, then similarity

A vector does not know what latest means, and two companies with similar names have similar vectors. Resolve the company to its Fokals company ID before you retrieve, apply the filters, and rank by similarity only inside what is left: the company, the date range, the event type and the label version.

-- facts is your index table; <=> is cosine distance in pgvector, so use your store's equivalent
select fact_id, text, occurred_at, url
from facts
where company_id = :company_id
  and occurred_at >= :since
  and occurred_at <= :as_of
  and 'funding_round' = any(event_types)
order by embedding <=> :query_embedding
limit 8;

The condition on as_of is what makes an as-of question safe. For a strict test of what was known on a date, filter on your own loaded_at as well. The at of an announcement is its publication date, and the observed_at of a signal can be a posting's own date, so the time your pipeline received the row is the one that says when it reached you. The glossary entry on point-in-time data explains why that distinction decides whether a test can be trusted.

Rank the filtered set two ways and merge the results. Event types, disclosure item numbers, technology names and tickers are exact strings, which a keyword match finds more reliably than an embedding. A paraphrase of an announcement is the case an embedding finds better. Take the best eight facts from the merged ranking, then order them by date, not by score, so that the model reads a sequence of events instead of a pile.

What the model should receive

Send the facts as they were written, each with its date, its company, its source and a handle to cite. Pass them unsummarised: a summary is a second model's claim, and the dated rows are the evidence. Say what came back empty. A line such as no announcements found for this company between two dates is itself a fact, and it stops the model from filling the gap from memory. Tell the model in the instruction what the corpus holds: dated announcement titles and excerpts, labelled postings and signals. The guide to company data as context for AI agents shows the result format and the checks on citations that go with this.

When SQL beats vectors

The rule: if the answer is a count, a sum, a rate, a date or a list defined by columns, write SQL. If it depends on what a company said in its own words, retrieve, with filters. Give the model a run_sql tool and a search_announcements tool and let it choose, instead of retrieving for every question.

For run_sql, supply the data dictionary entries for the datasets it may use, the conventions (times in UTC, day a closed UTC day, week_start a Monday, the first observation of a site a baseline), a read-only connection and a row limit. Several columns of Hiring Activity are JSON objects in one cell: by_function, by_seniority, by_country, by_work_mode and tech_mentions. Unnest them into views of one row per company, day and value, so that a generated query needs no JSON functions. In Postgres, for example:

create view hiring_by_function as
select h.company_id, h.day, f.key as job_function, f.value::int as open_postings
from company_hiring_daily h,
     jsonb_each_text(h.by_function::jsonb) as f;

Keeping the index in step

A daily or weekly period is final once it has closed, so its facts are append-only: add the facts of each newly closed period on every run, using the last cursor of the feed as your bookmark, and leave old facts in place. The guide to incremental sync covers the loop. Upsert event rows on their natural key: company_id, observed_at, category, key and change for Technology Changes, and company_id with url for Company News. Rows of Job Postings move as a posting is seen again or closes, so key them by posting_id.

Carry label_version as metadata. A breaking change to a label arrives as a new version announced at least 90 days ahead. Facts under the old version stay valid for their period, so re-index only what you want read under the new one, and filter on the version so that each answer reads one version.

Evaluating retrieval

Compute the answers with SQL first, then test retrieval against them. Measure recall at k for announcement questions with the filters on and with them off, the number of facts outside the requested date range in the top k, which should be zero if the filters work, and whether each claim in the generated answer traces to a retrieved fact. For numeric questions compare the routed answer with the SQL result. Log which tool the router chose: a drift towards retrieval on numeric questions shows there first.

Passing the reading notes on

Each dataset has its own reading notes in the methodology: a posting records an intention to hire, a detection shows presence on the website, a first observation is a baseline, and the datasets refresh daily to weekly. A retrieval system should pass the relevant note on with the facts it returns, so that an answer states what a row means as well as what it says.

How Fokals delivers it

The rows reach you direct, by REST API or as bulk exports in JSON, JSON Lines or CSV, with a manifest that names the label versions in each export. You load them into the search index or vector store of your choice with its own loader, and the facts you build from them stay keyed to the company ID.

Frequently asked questions

The queries and code on this page are examples to adapt. Test them in your own environment before you rely on them.