Documentation

Data & pipeline reference · Last updated 6 June 2026

This page documents where ForexFin's numbers come from, how often they refresh, and the data structures behind each tool. It is the operational companion to the methodology page, which covers the formulas. ForexFin does not yet expose a public API; this is reference documentation for anyone — human or machine — citing or auditing the data.

Architecture in one paragraph

Every calculator is a self-contained static HTML page whose math runs in the browser; no server round-trip is needed to compute a result. Market-data pages (currency strength, correlation, DXY, gold, oil, yields, calendar, COT, and the weekly reports) are pre-rendered server-side: a set of small Node.js workers poll the data sources on a fixed cadence, cache the results in a local SQLite database, and write fully-formed HTML — including a quotable answer block and a citation line — back to disk atomically. The crawler-visible page is therefore always complete and dated, with no dependence on client-side JavaScript.

Data sources

Refresh cadence

Data structures

The local cache holds a small number of normalized tables, all keyed for idempotent upserts:

• Daily closes — one row per (symbol, date) with the closing price. The base series for strength, correlation, volatility and the moving averages.
• Instrument quotes — latest spot per symbol with day change, previous close, and 50/200-day moving averages, plus a fetch timestamp.
• Treasury curve — one row per date holding the full 1-month-to-30-year curve.
• Economic events — upcoming high/medium-impact FX releases with time, currency, estimate and prior.
• COT positioning — per-currency net non-commercial position, weekly change, and percentage of open interest.

Internally, FX rates are normalized to a single canonical shape — units of each currency per one USD — so that the FMP path and the ECB fallback feed the math identically and the source can switch without changing results.

The spread-cost algorithm

The spread-cost figure accounts for three factors, in order:

1. Spread in pips — the bid/ask gap you input, interpreted with the correct pip size for the pair (0.01 for JPY pairs, 0.0001 otherwise).
2. Pip value in the account currency — derived from contract size, lot size, and the live quote-to-account conversion rate, exactly as in the pip-value method.
3. Trade frequency — multiplied by round trips per day/month/year to produce the itemized cost ladder.

The spread is treated as a full cost paid on entry; commissions, if any, are separate and not assumed.

Freshness & "as-of" conventions

Every market-data page carries an explicit as-of stamp. Live spot surfaces stamp the UTC minute of the last poll; end-of-day surfaces stamp the date of the close used. The weekly reports pin their as-of to the data's closing date so that a settled week renders identically forever — its numbers never change after the week closes. If a data source is temporarily unavailable, the page serves the last good cached value rather than a blank, and the as-of reflects that cached timestamp.

Machine-readable summary

A concise site summary for language models lives at /llms.txt. Formulas are documented on the methodology page; the published data archive is under /reports/.