AI for Energy Trading & Demand Forecasting

Power Demand Forecasting: Drivers in Deregulated Markets

Power demand in US and EU markets has characteristics that make forecasting genuinely hard: extreme temperature sensitivity (air conditioning load surges during heat waves, electric heating spikes in cold snaps), the growing impact of distributed solar (which masks gross demand as net load), holiday and event-driven swings, and the fast-rising load from electrification (EV charging, data centers, heat pumps).

Open data/power-demand-data.csv — 15-minute interval demand data for an ISO/balancing-authority zone with temperature, humidity, day type classification, and behind-the-meter solar status.

Feature Engineering for Demand

The standard features (temperature, hour-of-day, day-of-week) explain 70% of demand variation. The market-specific features close the gap:

Standard features:
  temperature (dry bulb, wet bulb), humidity, wind_speed
  hour_of_day, day_of_week, month
  is_weekend, is_holiday
  lagged_demand (1h, 24h, 168h)

Market-specific features:
  heating_degree_days, cooling_degree_days (nonlinear weather response)
  federal_and_state_holidays (each has distinct load shape)
  behind_meter_solar_proxy (net-load masking from rooftop PV)
  ev_charging_load (managed/unmanaged charging penetration by zone)
  data_center_ramp (large interconnection additions — measurable step changes)
  major_event_flag (Super Bowl, major sporting/TV events — evening demand spike)

  Structural growth features:
    trend_component (linear or exponential growth)
    industrial_production_index (monthly, lagged 1 month)
    new_interconnection_load (cumulative, quarterly update)

Model Architecture

For ISO/RTO operators and load-serving entities, the day-ahead forecast is the most critical — it determines unit commitment, day-ahead market bids, and interchange scheduling. The difference between 3% and 5% MAPE on a 20 GW system is 400-1,000 MW of scheduling error — worth $0.5-2 million/day in real-time balancing and imbalance costs.

Electrification Load: The Net-Load Effect

Behind-the-meter solar and EV charging together reshape the net-load curve in markets like CAISO and ERCOT. This load appears, disappears, and shifts based on:

ML models trained on satellite-derived irradiance, AMI (smart meter) net-load data, and EV telematics/charging-session data predict net demand within 8-12% — a major improvement over the 20-30% error in traditional methods that rely on historical gross-load averages.

Wholesale Price Prediction: Merit Order and Supply-Demand Balance

Day-ahead market (DAM) prices in PJM, ERCOT, CAISO, MISO, Nord Pool, and EPEX determine the marginal cost of power for utilities, retail suppliers, and merchant generators. Prices range from $15-25/MWh off-peak to $1,000-9,000/MWh during scarcity events (ERCOT's offer cap) — an enormous spread that creates significant trading opportunities and hedging requirements.

Merit Order Modeling

The DAM clearing price (LMP / area price) is determined by the intersection of supply and demand bid curves. The supply curve (merit order) is roughly:

Merit order (approximate variable cost, $/MWh):
  1. Wind/solar (must-take, ~zero marginal cost but intermittent): 0
  2. Nuclear: 8-15
  3. Hydro (run-of-river): 5-15
  4. Combined-cycle gas (efficient, low gas price): 20-40
  5. Coal: 25-45
  6. Combined-cycle gas (high gas price): 40-90
  7. Combustion turbine peakers (gas/oil): 90-250
  8. Scarcity pricing (ORDC / shortage adders): up to offer cap

Demand side:
  Load-serving entities with day-ahead scheduling
  Large industrial / data center loads (price-responsive)
  Virtual bids and financial traders (DAM-RT convergence)

An ML model predicts DAM clearing price by modeling both supply and demand:

Supply-side features:
  thermal_plant_availability_mw (from ISO generator outage reports)
  hydro_reservoir_levels (regional data, weekly)
  gas_price_mmbtu (Henry Hub, regional basis, day-ahead)
  renewable_generation_forecast (from wind/solar forecasting models)
  nuclear_availability (scheduled outage calendar)
  transmission_constraint_status (binding constraints / congestion flags)

Demand-side features:
  zonal_demand_forecast (ISO-wide, from previous section)
  regional_demand_distribution (zone/hub split)
  bid_behavior_patterns (historical supply offer curves)

Model: LightGBM regression
Target: DAM clearing price ($/MWh) per time block (hourly/5-min)
MAPE: 12-18% at 24-hour horizon

The forecast accuracy is sufficient for portfolio optimization — a generating company with 5,000 MW can increase DAM revenue by 3-5% by optimizing scheduling between bilateral contracts and the exchange, guided by price forecasts.

Gas Contract Optimization: Hub-Indexed vs Term

The natural gas market operates with several pricing references: Henry Hub (US benchmark), regional basis (e.g., Waha, SoCal Citygate, Algonquin, TTF/NBP in Europe), spot LNG (JKM and TTF-linked), and long-term contracts (oil-linked or hub-indexed formulas, often with take-or-pay obligations).

Open data/gas-trading-data.json — historical gas procurement data: hub-indexed purchases, spot purchases, long-term contract deliveries, prices, and demand by sector (power, industrial, LDC/residential, LNG feedgas).

Portfolio Optimization

A large gas marketer or utility manages a portfolio of:

The optimization problem: how much spot gas/LNG to procure given long-term contract positions, demand forecasts, and price forecasts.

Decision variables:
  spot_cargo_nominations (per month, 30-day horizon)
  long_term_delivery_scheduling (within ACQ tolerance band)
  customer_allocation_by_sector (firm vs interruptible service)

Objective: minimize total gas procurement cost
Constraints:
  meet sectoral demand obligations
  respect take-or-pay in long-term contracts
  LNG terminal slot availability (Sabine Pass, Calcasieu, Gate, Zeebrugge, Grain)
  regasification/liquefaction capacity constraints
  pipeline transport capacity constraints (firm transportation contracts)

An ML model predicts JKM (Japan Korea Marker) and TTF spot prices at 30 and 60-day horizons using:

Forecast MAPE of 10-15% at 30-day horizon enables a marketer to save $10-40 million/year on spot procurement timing versus fixed procurement schedules.

Carbon Credit Valuation and Cap-and-Trade Compliance

Compliance Carbon Markets

The EU Emissions Trading System (EU ETS), the Regional Greenhouse Gas Initiative (RGGI) in the US Northeast, and the California Cap-and-Trade program set declining emission caps for covered entities. Over-compliant facilities can sell allowances; under-compliant entities must buy allowances or face penalties.

Open data/carbon-credit-data.json — compliance cycle data: emission cap/allocation, actual emissions, allowance auction/purchase, and penalty calculations for power and industrial facilities.

Cap-and-trade compliance model:
  For each covered facility:
    allowance_allocation (free allocation + auction purchases)
    actual_emissions (verified annually under MRV rules)
    output_normalization (capacity utilization adjustment)

  Decision: invest in abatement vs buy allowances
    abatement_investment_cost ($/tCO2e abated)
    allowance_market_price ($/tCO2e — EUA, RGGI, or CCA)
    penalty_rate (if short and not covered by allowances)

  ML contribution:
    predict allowance price trajectory (supply-demand + policy model)
    predict own emissions under different investment scenarios
    optimize: minimize (investment_cost + allowance_purchase_cost + penalty_risk)

Evolving Carbon Markets

Carbon markets continue to tighten — the EU ETS Market Stability Reserve, the EU CBAM (border carbon adjustment), and expanding RGGI/CCA caps reshape allowance prices. New compliance and voluntary markets continue to emerge alongside them.

ML models for carbon market participation:

For energy companies (Duke, Vistra, RWE, Iberdrola, Ørsted), the trajectory of tightening caps represents a strategic shift. Companies that can accurately predict their compliance position and allowance prices will optimize the timing of investments and allowance transactions.

Key Takeaways