CommodityWatch methodology and data notes

CommodityWatch / Methodology

How CommodityWatch handles sourcing, calculation, and disclosure.

This is the long-form version of the methodology notes that appear inline throughout the product. If a metric is not obvious from its card-level explanation, the fuller treatment is here.

Public and official sources Operational source metadata Explicit caveats

01

Sourcing principle #

CommodityWatch prioritises public, official, or institutionally disclosed sources. Source classifications are operational product metadata, not legal advice. In practice that means four source families.

  1. US government public-domain data. EIA, USDA, DOE, FERC, CFTC, FRED, CPC, NOAA, and NWS datasets are used where they match the product scope.
  2. Creative Commons-licensed institutional content. This includes institutions such as the IEA, European Commission, World Bank, Ember, and similar sources where usage terms are explicit.
  3. Official public releases. Exchange notices, operating reports, market bulletins, and other documents published directly by the issuing institution.
  4. Terms-sensitive disclosures. Some exchange or third-party institutional figures may be visible publicly but still require source-specific caution for independent reuse.

Deliberate exclusions matter just as much as inclusions. CommodityWatch does not mirror article bodies or reproduce paid assessment values. That leaves known coverage gaps in some specialty markets, but the trade-off is intentional: every number on the site should carry a visible source and caveat trail.

02

Data pipeline #

Ingestion runs on schedules calibrated to each source cadence. Weekly series such as the EIA Weekly Petroleum Status Report and EIA natural gas storage are refreshed around their release windows. Monthly series such as WASDE, FRED macro indicators, and related institutional releases are pulled on their own schedules. Daily series such as benchmark prices and AGSI+ storage are refreshed on a fixed cycle.

Where the underlying source publishes revisions, CommodityWatch preserves release timestamps, vintage metadata, and revision markers instead of silently overwriting the prior observation. Selected derived calculations also store deterministic hashes of their inputs so changes to the numerator, denominator, or source vintage can be traced.

Current product note

DemandWatch is served through the root app's DemandWatch API path, with route JavaScript owning presentation mapping.

03

Seasonal context #

Absolute values are often misleading in isolation. A crude inventory reading can look comfortable in one month and tight in another. CommodityWatch therefore pairs many inventory and demand series with seasonal context drawn from the same week of year over the prior five years.

Seasonal bands are expressed through percentile ranges rather than a single average. The displayed bands use the 10th, 25th, 50th, 75th, and 90th percentiles, which makes it easier to identify whether the latest print sits inside the normal range, toward an edge, or outside it entirely.

04

Days of supply #

Days of supply is one of the clearest examples of a metric that looks simple but depends on denominator discipline. CommodityWatch documents the convention used for each series instead of presenting a single generic formula.

US crude and petroleum products

CommodityWatch currently publishes computed days-of-supply for crude, gasoline, and distillate using EIA inventory and demand inputs. Other petroleum-product days-of-supply series are added only when the public store supports them.

US natural gas

Natural gas storage is currently shown as reported inventory and seasonal context. Natural gas days-of-supply is not currently published in the root app.

European natural gas

AGSI+ working gas is shown as reported storage at aggregate or country level. European gas days-of-supply is not currently published in the root app.

Grains

For grains, the standard metric is stocks-to-use ratio rather than days of supply. CommodityWatch keeps the naming distinct to avoid mixing weekly storage coverage with marketing-year balance-sheet math.

CommodityWatch does not force days-of-supply across markets where the denominator would be weak or misleading. Base metals, many softs, precious metals, and ETF-linked holdings only receive the metric when a defensible consumption basis exists.

05

The demand indicator taxonomy (T1–T7) #

Demand data is not homogeneous. Some indicators are direct measurements of physical offtake; others are proxies or contextual signals. CommodityWatch makes that distinction explicit through a seven-tier reliability taxonomy.

T1 · Direct Consumption

Measured physical offtake such as product supplied, electricity demand, or domestic use.

T2 · Throughput Proxy

Processing volumes that track underlying demand closely, such as refinery inputs or crush data.

T3 · Trade Flow

Import and export measures used where direct domestic demand coverage is weak or delayed.

T4 · End-Use Activity

Sector-level activity such as construction, auto output, or vehicle miles traveled.

T5 · Leading Indicator

Forward-looking measures such as new orders, permits, or similar pre-consumption signals.

T6 · Macro Context

Broad economic backdrop rather than commodity-specific demand measurement.

T7 · Weather-Derived

Heating, cooling, and precipitation-linked signals whose reliability varies by market and season.

06

Headline enrichment and AI disclosure #

HeadlineWatch standardises source attribution, timestamps, commodity mapping, and sentiment metadata where that metadata is available from the ingestion workflow. The enrichment layer combines deterministic taxonomy rules with model-assisted classification used during processing.

The current shipped product does not publish AI-written summaries or trading calls. PriceWatch TimesFM projections are the only shipped model-generated market-context overlay, and they are labelled as projections rather than CommodityWatch market views. If LLM-generated fields are added later, they will carry explicit disclosure rather than being presented as silent editorial judgement.

07

PriceWatch TimesFM projection pilot #

PriceWatch can display an optional TimesFM projection on the detail chart. The projection is generated only when the optional projection step runs after a PriceWatch data refresh. The actual published observations remain the solid line. The model-implied future path is shown as a dashed line only after the user requests it.

The pilot uses TimesFM 2.5 with checkpoint google/timesfm-2.5-200m-pytorch. Inputs are published PriceWatch price histories regularized to the source cadence: daily series use a business-day grid, weekly series keep weekly cadence, and monthly series keep monthly cadence. The system uses raw price levels, model input normalization, positive-output enforcement, quantile crossing fixes, and stores 80% interval bounds when quantiles are returned.

One 12M projection is stored per successful series run and sliced into the supported 1M, 3M, 6M, and 12M presets. Custom horizons are not supported in this pilot. Successful runs are preserved as audit history, but the UI exposes only the latest successful run.

Required warning

TimesFM projections are model-implied market context from published PriceWatch history. They are not investment advice, trading recommendations, or guarantees of future prices.

08

Data sources directory #

Source families currently used in the product or supported by the repo-visible ingestion workflows include the following.

Energy

EIA, DOE, FERC, AGSI+, Ember, FRED, and related official or public reference sources.

Metals

USGS, World Bank reference series, FRED, IMF, and selected exchange warehouse disclosures where source terms support public display.

Agriculture

USDA WASDE, PSD Online, NASS, FAS GAIN, CBOT, ICE, and other official agricultural releases.

Macro context

FRED, Eurostat, and other public economic releases that help frame cross-market demand conditions.

Weather and release timing

CPC, NOAA, NWS, and the publication calendars of the agencies and institutions being tracked.

Page-level disclosure

Each module still carries its own local attribution and metric notes; this page is the shared policy layer.

09

What we deliberately do not do #

No CommodityWatch price forecasts

TimesFM projections are labelled model-implied context overlays, not CommodityWatch market views.

No trade signals

Indicators are surfaced as context for interpretation, not buy-or-sell instructions.

No proprietary assessment reconstruction

Paid vendor datasets and inferred reproductions of them sit outside scope by design.

No paywall mirroring

If a source is paywalled, CommodityWatch can link to it but does not reproduce its body text.

10

Methodology changelog #

Material changes to formulas, source coverage, taxonomy rules, or page-level disclosure will be logged in the public methodology changelog.

Methodology is treated as a versioned asset, not a launch-day checkbox. If a convention changes, the change should be visible.