The Problem

While building a RAG pipeline for energy industry analytics recently, I ran into an interesting failure mode: even very strong LLMs can struggle with multi-step causal reasoning.

They often retrieve relevant facts correctly, but miss the deeper causal structure connecting those facts — sometimes treating downstream symptoms as if they were independent root causes.

This becomes particularly noticeable when answering questions that require reasoning across multiple reports, infrastructure constraints, policy frameworks, commodity markets, and interdependent economic systems.

The issue is subtle because the answers often look correct. The model retrieves relevant facts, uses the right terminology, and produces something that reads as analytically sound.

But underneath, the reasoning chain can be incomplete.

An Example

"How are the increasing frequency of negative wholesale electricity prices and the regulatory shift towards two-sided Contracts for Difference (CfDs) in Europe altering the revenue expectations and financial flexibility of developers investing in utility-scale solar PV?"

Baseline LLM Response

The increasing frequency of negative wholesale electricity prices and the regulatory shift towards two-sided Contracts for Difference (CfDs) in Europe are significantly altering the revenue expectations and financial flexibility of developers investing in utility-scale solar PV. Here are some key implications:

  1. Negative wholesale electricity prices: With the growing share of intermittent renewable energy sources, such as solar and wind, wholesale electricity prices are becoming increasingly volatile. Negative prices occur when there is an oversupply of electricity, and generators are forced to pay to feed their excess energy into the grid. This can reduce revenue for solar PV developers if they are unable to sell electricity at profitable prices.

  2. Two-sided Contracts for Difference (CfDs): Two-sided CfDs are a type of contract that exposes both the buyer and seller to price risk. In a two-sided CfD, the developer receives a fixed strike price for its electricity, but must pay the difference when the wholesale price falls below the strike price. This means developers are more exposed to market risk and may face financial penalties when wholesale prices are low.

  3. Reduced revenue expectations: The combination of negative wholesale prices and two-sided CfDs can reduce revenue expectations for solar PV developers. With the risk of negative prices and potential financial penalties, developers may need to reassess project economics and adjust revenue projections downward.

  4. Increased financial risk: The shift toward two-sided CfDs increases financial risk for developers because they are more exposed to market volatility. This may lead to higher financing costs as lenders and investors view these projects as riskier investments.

  5. Impact on project financing: Altered revenue expectations and increased financial risk may affect developers’ ability to secure project financing. Lenders may require stronger credit enhancements, such as higher debt-service coverage ratios or more comprehensive risk-management strategies, to mitigate risks associated with two-sided CfDs.

  6. Need for innovative financing structures: To mitigate risks associated with two-sided CfDs, developers may need to explore innovative financing structures, such as revenue put options or proxy revenue swaps, to manage exposure to market risk.

  7. Focus on grid-scale energy storage: The increasing frequency of negative wholesale prices and the shift toward two-sided CfDs may accelerate development of grid-scale energy storage. By storing excess energy generated during periods of low demand, developers can reduce exposure to negative prices and increase revenue potential.

  8. Diversification of revenue streams: Developers may need to diversify revenue streams to reduce dependence on wholesale electricity prices. This could include alternative revenue sources such as grid services, frequency response, reserve power, storage, or balancing services.

That sounds reasonable. So what’s the issue?

At first glance, the baseline LLM response looks solid; it correctly identifies negative pricing dynamics, explains two-sided CfD economics reasonably well, and recognizes increased revenue volatility.

The problem is that it flattens the causal structure.

It treats negative wholesale electricity prices and two-sided CfDs as separate risks that developers need to manage independently. A stronger interpretation is that both emerge from a common underlying issue: renewable generation capacity is scaling faster than overall system flexibility.

The IEA frames this directly in Renewables 2025:

Curtailment and negative prices signal a lack of flexibility in electricity systems.

In that framing, two-sided CfDs are not a primary driver of developer risk. They are a market adaptation responding to the same underlying system constraint.

The model partially recognizes this by noting that negative prices result from oversupply — but it never connects oversupply, storage constraints, and evolving CfD structures as part of the same causal chain.

As a result, it recommends managing symptoms instead of identifying the underlying cause.

The response does mention grid-scale storage, but frames it primarily as a revenue optimization strategy.

That misses the bigger picture.

At utility scale, storage increasingly acts as a core infrastructure requirement for renewable integration, not simply an optional revenue-enhancement layer. Many of the financial risks these CfD structures are attempting to manage exist precisely because storage deployment has not scaled alongside renewable generation growth.

Most of the individual facts are correct.

But the reasoning structure is incomplete.

One relatively simple improvement

For multi-step causal analysis, I’ve found GraphRAG to be a useful addition.

Traditional vector retrieval works well for semantic similarity, but it often misses the chain of intermediate entities that actually connect a question to its answer — especially when those relationships are spread across multiple reports, or when key entities sit several reasoning steps away from the query.

GraphRAG augments semantic retrieval with structured entity relationships: rather than retrieving only semantically similar chunks, it builds a knowledge graph from the same source corpus and expands retrieval through graph traversal.

Architecture example with Energy Industry Corpus

PDF chunks (380 tokens, 60 overlap)
       │
       ├─► LLMGraphTransformer (gpt-4.1-mini)
       │     extracts typed entities + relationships
       │     using a constrained schema
       │
       ├─► Neo4j knowledge graph
       │     nodes:  EnergySource, Region, Policy,
       │             Technology, Infrastructure, Sector,
       │             Metric, TimeFrame
       │     edges:  AFFECTS, FORECASTS, CONSUMES,
       │             PRODUCES, DRIVES, REQUIRES,
       │             LOCATED_IN, ENACTED_BY, CONSTRAINS,
       │             REPLACES, COMPETES_WITH, MENTIONS
       │
       └─► Neo4j vector index
             same bge-base embeddings on Document nodes

At query time:

  1. Vector seed search: Retrieve the top four chunks by semantic similarity to the query.
  2. Graph expansion: Walk edges out from seed chunks to entities, then back to other document chunks that link to the same entities. Rank by count of shared entities and keep the top four.
  3. Combine and deduplicate: Pass up to eight chunks — vector seeds plus graph neighbors — into the same investment-committee briefing prompt used in tuned RAG.

This allows retrieval to follow relational structure rather than relying entirely on embedding similarity.

Chain-of-Verification — catching plausible but unsupported causal claims

One terminology note, because it matters for the evaluation numbers below.

Chain-of-Verification (CoV) is a correction step. It rewrites a draft answer while keeping only claims it can re-ground in source evidence.

Causal correctness is scored separately by an independent evaluation model that grades whether each causal claim is actually supported by cited evidence.

Keeping those two processes separate is important. If the same mechanism both corrected and evaluated the answer, the benchmark becomes circular.

CoV directly targets that failure mode — plausible-sounding causal claims the sources don’t actually support.

Workflow

  1. Generate the initial GraphRAG draft answer

  2. Decompose the response into atomic claims, classifying each as:

  3. Quantitative
  4. Causal

  5. Procedural

  6. For each causal claim, perform targeted retrieval around both cause and effect entities, then use a secondary reasoning model to verify whether the cited evidence actually supports the claimed causal relationship.

  7. Rewrite the final briefing, keeping only supported claims, explicitly marking plausible but inferred claims, and removing unsupported claims entirely.

The objective here is not factual verification in the traditional sense.

The objective is verifying whether the relationships themselves are actually supported by evidence.

This distinction matters more than many current RAG evaluation frameworks acknowledge.

Conclusions

In an admittedly small early test set (n=5), GraphRAG + CoV improved causal correctness from 0.875 to 1.0 on questions involving coal generation and global coal markets, where complex supply chains and multi-jurisdictional regulatory environments create significant causal complexity.

For simpler single-hop factual questions, where standard tuned RAG was already performing well, the difference was negligible.

The takeaway was fairly straightforward.

Traditional RAG systems are generally good at retrieving facts.

But when questions depend on multi-step causal reasoning — especially across multiple documents, regulatory systems, commodity markets, and interconnected economic dependencies — augmenting retrieval with explicit relational structure can materially improve answer quality.

In other words:

Semantic similarity retrieves facts.

Relational structure retrieves reasoning.