CommodityWatch methodology and data notes

CommodityWatch / Methodology

How CommodityWatch handles sourcing, calculation, and disclosure.

This is the long-form version of the methodology notes that appear inline throughout the product. If a metric is not obvious from its card-level explanation, the fuller treatment is here.

Public and official sources Operational source metadata Explicit caveats

01

Sourcing principle #

CommodityWatch prioritises public, official, or institutionally disclosed sources. Source classifications are operational product metadata, not legal advice. In practice that means four source families.

  1. US government public-domain data. EIA, USDA, DOE, FERC, Federal Reserve Board, CFTC, CPC, NOAA, and NWS datasets are used where they match the product scope.
  2. Creative Commons-licensed institutional content. This includes institutions such as the IEA, European Commission, World Bank, Ember, and similar sources where usage terms are explicit.
  3. Official public releases. Exchange notices, operating reports, market bulletins, and other documents published directly by the issuing institution.
  4. Terms-sensitive disclosures. Some exchange or third-party institutional figures may be visible publicly but still require source-specific caution for independent reuse.

Deliberate exclusions matter just as much as inclusions. CommodityWatch does not mirror article bodies or reproduce paid assessment values. That leaves known coverage gaps in some specialty markets, but the trade-off is intentional: every number on the site should carry a visible source and caveat trail.

DemandWatch's U.S. plastics material and resin manufacturing pilot uses the Federal Reserve Board's G.17 Monthly Supplement Table 1B and Table 2 for NAICS 325211: seasonally adjusted industrial production (2017=100) and capacity utilization (percent). These are industry activity signals, not polymer-specific production, tonnes, consumption, inventory, or spot prices. Observations are monthly, initially preliminary, and may be revised; a delayed latest month is identified explicitly. The Board disclaimer provides the public-domain reuse basis and requests source attribution.

02

Data pipeline #

Refresh jobs follow each source's release calendar and cadence. Data becomes available on CommodityWatch after a validated update is published. Observation, release, and publication timestamps show the data available in the current release; source publication does not imply an immediate site update.

Where the underlying source publishes revisions, CommodityWatch preserves release timestamps, vintage metadata, and revision markers instead of silently overwriting the prior observation. Selected derived calculations also store deterministic hashes of their inputs so changes to the numerator, denominator, or source vintage can be traced.

Current product note

Source cadence and site freshness are distinct. Check the displayed observation and publication times when comparing indicators from different releases.

03

Seasonal context #

Absolute values are often misleading in isolation. A crude inventory reading can look comfortable in one month and tight in another. Eligible InventoryWatch series use comparable weeks, months, or marketing-year periods from prior completed reporting years, subject to the available history.

InventoryWatch seasonal bands use the 10th, 25th, 50th, 75th, and 90th percentiles where supported. DemandWatch detail charts use a rolling baseline with one- and two-standard-deviation bands. These describe different comparisons; missing history is not replaced with synthetic bands.

04

Days of supply #

Days of supply is one of the clearest examples of a metric that looks simple but depends on denominator discipline. CommodityWatch documents the convention used for each series instead of presenting a single generic formula.

US crude and petroleum products

CommodityWatch currently publishes computed days-of-supply for crude, gasoline, and distillate using EIA inputs: refinery crude inputs for crude stocks, and product supplied for gasoline and distillate. Each denominator uses a four-week trailing average daily rate.

US natural gas

Natural gas storage is currently shown as reported inventory and seasonal context. Natural gas days-of-supply is not currently published in the root app.

European natural gas

AGSI+ working gas is shown as reported storage at aggregate or country level. European gas days-of-supply is not currently published in the root app.

Grains

For grains, the standard metric is stocks-to-use ratio rather than days of supply. CommodityWatch keeps the naming distinct to avoid mixing weekly storage coverage with marketing-year balance-sheet math.

CommodityWatch does not force days-of-supply across markets where the denominator would be weak or misleading. Base metals, softs, precious metals, and ETF-linked holdings are presented as stocks without a days-of-supply estimate.

05

The demand indicator taxonomy (T1–T7) #

Demand data is not homogeneous. Some indicators are direct measurements of physical offtake; others are proxies or contextual signals. CommodityWatch makes that distinction explicit through a seven-tier reliability taxonomy.

T1 · Direct Consumption

Measured physical offtake such as product supplied, electricity demand, or domestic use.

T2 · Throughput Proxy

Processing volumes that track underlying demand closely, such as refinery inputs or crush data.

T3 · Trade Flow

Import and export measures used where direct domestic demand coverage is weak or delayed.

T4 · End-Use Activity

Sector-level activity such as construction, auto output, or vehicle miles traveled.

T5 · Leading Indicator

Forward-looking measures such as new orders, permits, or similar pre-consumption signals.

T6 · Macro Context

Broad economic backdrop rather than commodity-specific demand measurement.

T7 · Weather-Derived

Heating, cooling, and precipitation-linked signals whose reliability varies by market and season.

06

Headline enrichment and AI disclosure #

HeadlineWatch standardises source attribution, timestamps, commodity mapping, and sentiment metadata where that metadata is available from the ingestion workflow. The enrichment layer combines deterministic taxonomy rules with model-assisted classification used during processing.

The CommodityWatch website does not publish AI-written summaries or trading calls. PriceWatch TimesFM projections are the only website-published model-generated market-context overlay, and they are labelled as projections rather than CommodityWatch market views.

The authenticated private connector, when enabled for an approved account, exposes six bounded read-only tools: a cross-module commodity evidence brief, a precomputed inventory-to-seasonal comparison, governed coverage discovery, bounded headline search, latest published benchmark prices and upcoming releases. The connector returns structured CommodityWatch evidence, source links, dates, units, freshness, coverage limits and explicit empty-result meaning. A user's chosen model provider may formulate prose from that evidence; that prose is not a CommodityWatch-authored market view. External titles, labels, attribution, organiser names and other source metadata are marked as untrusted data and must never be treated as instructions. The connector does not expose article bodies, raw databases, unrestricted history, private-vendor archives, trading advice or price forecasts.

07

USDA Mississippi River grain barge freight #

PriceWatch publishes the three discrete weekly transaction-rate tenors in USDA Agricultural Marketing Service Grain Transportation Report Table 9: Nearby, 1M forward, and 3M forward. Nearby is USDA's current/nearby, or spot, quote. The 1M and 3M observations are rates for freight transactions in those future months. They are not forecasts, futures contracts, executable quotes, exchange settlements, or an interpolated forward curve.

The observation date is the reported week-ending date. For the two forward tenors, the separately reported rate month is retained verbatim as the delivery/reference-month field. USDA reports that field as a month number, so CommodityWatch does not infer a year, substitute it for the observation week, or infer either field from the series key. Blank rates remain missing observations rather than zeroes. Reissued rows are upserted by series and observation date, with source retrieval time, raw artifact checksum, and record hash retained for an auditable, idempotent revision path.

USDA quotes each series as percent of 1976 tariff. The location benchmarks used for secondary USD/ton context are Twin Cities $6.19, Mid-Mississippi $5.32, Illinois River $4.64, St. Louis $3.99, Cincinnati $4.69, Lower Ohio $4.46, and Cairo-Memphis $3.14. The live API labels the Illinois River row Lower Illinois; CommodityWatch retains that source value in provenance and uses the Table 9 location label in the interface.

The conversion is (rate × benchmark dollars per ton) / 100, calculated with decimal arithmetic. For example, USDA's 271 St. Louis quote is 271 × $3.99 / 100 = $10.81 per ton after display rounding. USDA's Table 9 footnote defines one ton as 2,000 pounds. The interface retains USDA's concise “USD/ton” wording rather than relabelling the denominator.

These are freight benchmarks, not commodity prices. Coverage is limited to southbound grain freight at seven Mississippi River System locations; it does not cover ocean or container freight, vessel charter rates, northbound movements, or proprietary assessments. The source publishes weekly. PriceWatch shows both the observation week and retrieval freshness, and retains the last accepted observations with a visible stale/error state if a later refresh fails. TimesFM projections are available only for the seven Nearby histories. The 1M and 3M forward transaction-rate series are explicitly ineligible because applying a model path to already-forward freight quotes would be misleading.

USDA AMS Transportation and Marketing Program is the data provider. CommodityWatch reads the public SODA resources deqi-uken, svms-9yya, and uuhv-5etw through the AgTransport/Socrata delivery platform and validates conversions against USDA dataset 7spn-fbua where comparable. USDA's GTR dataset directory identifies Table 9 and says its series are aggregated from non-confidential and non-copyrighted sources. CommodityWatch therefore records the source as US public domain operational metadata, with USDA attribution and no implication of endorsement; this classification is not legal advice.

08

PriceWatch TimesFM projections #

PriceWatch can display an optional TimesFM projection on the detail chart. The projection is generated only when the optional projection step runs after a PriceWatch data refresh. The actual published observations remain the solid line. The model-implied future path is shown as a dashed line only after the user requests it.

The model uses TimesFM 2.5 with checkpoint google/timesfm-2.5-200m-pytorch. Inputs are published PriceWatch price histories regularized to the source cadence: daily series use a business-day grid, weekly series keep weekly cadence, and monthly series keep monthly cadence. The system uses raw price levels, model input normalization, positive-output enforcement, quantile crossing fixes, and stores 80% interval bounds when quantiles are returned.

One 12M projection is stored per successful series run and sliced into the supported 1M, 3M, 6M, and 12M presets. The interface provides these four horizons. Successful runs are preserved as audit history, but the UI exposes only the latest successful run. Series declare projection capability explicitly; the seven USDA barge freight Nearby series are eligible, while all 14 forward quotes are excluded.

Required warning

TimesFM projections are model-implied market context from published PriceWatch history. They are not investment advice, trading recommendations, or guarantees of future prices.

09

Data sources directory #

Source families currently used in the product or supported by the repo-visible ingestion workflows include the following.

Energy

EIA, DOE, FERC, AGSI+, Ember, FRED, and related official or public reference sources.

Metals

USGS, World Bank reference series, FRED, IMF, and selected exchange warehouse disclosures where source terms support public display.

Agriculture

USDA WASDE, PSD Online, NASS, FAS GAIN, CBOT, ICE, and other official agricultural releases.

Freight

USDA AMS Transportation and Marketing Program agricultural transportation data, including Grain Transportation Report Table 9.

Macro context

FRED, Eurostat, and other public economic releases that help frame cross-market demand conditions.

Weather and release timing

CPC, NOAA, NWS, and the publication calendars of the agencies and institutions being tracked.

Page-level disclosure

Each module still carries its own local attribution and metric notes; this page is the shared policy layer.

10

What we deliberately do not do #

No CommodityWatch price forecasts

TimesFM projections are labelled model-implied context overlays, not CommodityWatch market views.

No trade signals

Indicators are surfaced as context for interpretation, not buy-or-sell instructions.

No proprietary assessment reconstruction

Paid vendor datasets and inferred reproductions of them sit outside scope by design.

No paywall mirroring

If a source is paywalled, CommodityWatch can link to it but does not reproduce its body text.

11

Methodology changelog #

Material changes to formulas, source coverage, taxonomy rules, or page-level disclosure will be logged in the public methodology changelog.

Methodology is treated as a versioned asset, not a launch-day checkbox. If a convention changes, the change should be visible.