Topic
Data delivery
APIs, bulk files, formats, schedules and incremental sync.
- Keeping a warehouse in step with incremental API syncA daily sync that resumes after a failure, runs twice without harm and notices a period that arrives late. The design, table by table, with the keys and the checks.
- Delta Sharing, explained for data licensing teamsThe open protocol behind Databricks Marketplace: how providers, shares and recipients work, what changed in its name in 2026, and what a licensing team should settle when data is shared.
- Exploring bulk company data with DuckDBRead a sample file with read_csv or read_ndjson, then run seven checks on grain, dates, nulls, JSON cells, labels and baselines. The queries are written for the Fokals tables.
- Ingesting a vendor API with Fivetran or AirbyteFokals is delivered direct by REST API and bulk files. This guide maps a cursor API onto the Fivetran Connector SDK and the Airbyte Connector Builder, setting by setting.
- Loading company data into Amazon RedshiftWorked SQL for loading company data into Amazon Redshift: a table definition, COPY from S3, the SUPER type for JSON cells, and a daily incremental load that can run twice without duplicates.
- Loading company data into BigQueryA worked path from files in a bucket to partitioned BigQuery tables: LOAD DATA with an explicit schema, JSON columns for the object cells, MERGE for daily files, and checks after each load.
- Loading company data into Databricks with Auto LoaderA worked load of Fokals files into Databricks: landing in a volume, bronze with Auto Loader, typed silver tables kept free of duplicates with MERGE, and a gold join to your identifiers.
- Loading company data into Snowflake from files and an APIA worked pipeline from delivered files and API pages to history tables: stages, loading CSV by header name, JSON Lines into VARIANT, and an idempotent daily MERGE.
- Serving company data to AI agents over Model Context ProtocolA team wraps the Fokals REST API as MCP tools in a thin server of its own. This guide covers tool design, paging, keys, rate limits and untrusted text against the current specification.
- API vs bulk files for company data deliveryMost teams need both: files to build the copy, an API to keep it current and to answer look-ups. How to assign each job to a route, with the HTTP and file-format rules that decide it.
- CSV vs JSON Lines vs Parquet for bulk data deliveryThree file formats, three sets of trade-offs. What each keeps and loses when a vendor hands you a dataset as files, and a tested way to turn a CSV export into typed Parquet.
- Delta Sharing vs Snowflake Secure Data SharingOne is a protocol that any client can speak, the other a feature inside one platform. A comparison built on two questions a licence turns on: who can be a recipient, and what ends access.
- Fivetran vs Airbyte for ingesting a vendor APIA vendor's REST API reaches your warehouse through a connector you build. How the two tools differ, as each documents itself: where the code runs, what you write, where the bookmark lives and how rows are counted.
- Bulk exportA bulk export hands over a dataset as files, not as individual calls. This entry shows when to use one, what to check when it arrives and how Fokals delivers bulk files.
- Cursor paginationCursor pagination marks where a page ended so the next call can resume from there. This entry compares it with offset paging and shows how to use it safely.
- Data dictionaryA data dictionary says what every table and column of a dataset means. This entry lists what a useful one states and how to test it against a sample.
- Data freshnessData freshness is the age of a record's observation when you use it. How it differs from refresh cadence, how to measure it in your own warehouse, and the cadence of each Fokals dataset.
- Data manifestA data manifest is the file that describes one delivery of data. This entry shows what it holds, how to check a load against it and what Fokals states in its own.
- Incremental syncIncremental sync fetches only the records added since the last run. This entry shows the bookmark, the load and the failure modes, with an example on daily company data.
- JSON LinesJSON Lines holds one JSON record per line, so large exports can be streamed and appended to. This entry gives the rules, a worked example and the traps.
- Rate limitA rate limit caps the requests a client may send in a period. This entry shows how to plan a backfill against minute and daily limits and how to act after a refusal.