A bulk export delivers a whole dataset, or a defined slice of it, as files that the buyer downloads and loads into its own systems, instead of as many individual API calls. It suits first loads, backfills and analysis across the full set.
When files serve better than calls
| Task | Better fit |
|---|---|
| First load of history | Bulk export |
| Back-test or study across the whole list | Bulk export |
| Daily top-up of a copy you hold | A feed read from a bookmark, as in incremental sync |
| Look-up of one company inside a product | API call |
The usual pattern uses both. Build the copy from an export, note where it ended, and keep it current from there. Paging a long history through an API spends request limits and time on rows that one file download would carry.
What to check on arrival
- Completeness: every part of the export is present, and the manifest matches what you received.
- Period: the dates covered are the ones you asked for.
- Format: the encoding, the header row in CSV and the handling of nested values.
- Repeatability: key each table so that loading the same file twice changes nothing.
In Fokals data
Bulk exports give any table for any period as JSON, JSON Lines or CSV: page by page through the export endpoints of the API, or as a package of files with a manifest naming sources, period, label versions and licence. The buyer pulls the files and loads them where needed. Daily and weekly tables are written once, after the period closes, and are not revised, so a closed period needs loading once.
Each export endpoint returns up to 1,000 rows a page, and a package of files carries the same rows for the whole period. A team that works in a columnar format converts after loading. For the export endpoints, see the delivery page and the API documentation. The comparison of API and bulk files helps to choose between the two.
Related terms
- Data manifest: the file that describes what an export contains.
- Cursor pagination: how an API returns a long list in pages.
- Data warehouse: where exported files are usually loaded and modelled.
Frequently asked questions
What is the difference between a bulk export and an API?
An API answers one request at a time, with a page of records or a single look-up. A bulk export hands over many records at once as files. Use the API for look-ups and for keeping a copy current, and use a bulk export to build that copy in the first place or to analyse the whole set. Providers often offer both, and Fokals does.
How often should I run a bulk export?
Run it once for the first load, then switch to an incremental feed from the point the export ended, rather than repeating full exports. A full export on a schedule downloads rows you already hold. Repeat a full export only when you need to rebuild a table from scratch or to check your copy against the source.
Which file format should I choose for a bulk export?
Choose the one that loads into your system with least work. CSV is read everywhere but holds nested values as text. JSON Lines keeps nesting and types and streams well. A single JSON file has to be parsed whole. Fokals offers CSV, JSON and JSON Lines.