Answering the wrong question well
Forecast accuracy has been one of the most discussed topics in electricity trading in recent years: better weather models, more training data, longer historical records. Production forecasts really are becoming more accurate with these advances.
In the field, however, a different picture emerges:
Even as forecast accuracy improves, imbalance costs do not fall by the same measure.
This does not mean the models are inadequate. There is a more fundamental reason: a good forecast does not, by itself, mean a good decision. Because two similar but distinct questions are constantly discussed in the same sentence:
"What will happen?" → a forecasting problem
"What should I do?" → a decision problem
When these two questions are handled under the same heading, the solution to one is assumed to solve the other. It does not.
Forecasting and decision-making are different problems
Estimating how much a plant will produce in the coming hours is a forecasting problem. Deciding how much energy to buy or sell is a problem of optimization under uncertainty.
The fundamental difference between the two emerges less from the data used than from the objective function.
| Forecast | Decision | |
|---|---|---|
| Core question | What will happen? | What should I do? |
| Problem type | Regression | Optimization under uncertainty |
| Objective | Minimize forecast error | Minimize total cost |
| Typical inputs | Weather, physical model, historical production | Market regime, open position, price, liquidity, decision time |
Forecast models are evaluated with metrics such as MAE, RMSE or nMAE. What these metrics have in common is that they measure error symmetrically: a 10 MWh over-forecast and a 10 MWh under-forecast are deviations of similar magnitude from the metric's point of view.
In trading, however, the outcome of these two deviations is not the same. Energy left short can be procured at a higher price at delivery time or become subject to imbalance settlement. Energy bought in excess, on the other hand, creates the cost of the position left on hand.
The conclusion that follows is this:
A forecast model cannot, on its own, know which direction of error is more expensive — because that information is not part of the forecasting problem.
Same MWh, different hour, different bill
The economic value of a MWh depends not only on its quantity; the hour in which it occurs also determines that value.
In Turkey, the hourly market price moves within a wide band over the day. The midday hours when solar production peaks and the evening hours when demand rises do not show the same price behaviour. For this reason "we have a shortfall, let's close it" is not, by itself, a sufficient decision. The decision is in fact threefold:
- Should I buy?
- How much should I buy?
- At what hour should I buy?
The first question is relatively simple and is the one that creates the least cost difference. The economic outcome is formed mainly by the second and third questions — because the cost of the same quantity of energy at different hours is not the same.
Moreover, the hours are not independent of each other. An extra MWh bought today changes charging and discharging decisions if storage is available; if there is no storage, it creates a position carried into the future. The market itself also divides these decisions into different time windows:
- Day-Ahead Market (DAM): the basic stage where the next day's hourly schedule is formed; the schedule notification originates here.
- Intraday Market (IDM): allows the position to be corrected up to 60 minutes before delivery. The window exists, but liquidity and price at the end of the window may not always be at the desired level.
- Imbalance settlement: reveals the economic consequence of the unsettled difference between the notified schedule and actual production.
The decision layer is not a simple calculation that follows the forecast; it is a separate system operating within the time structure of the market.
The most critical day, the worst forecast
Forecast error does not behave the same every day. In clear and stable weather conditions, uncertainty stays narrow. On partly cloudy days, during rapid weather changes and in ramp periods when production rises and falls sharply, uncertainty widens — because what determines the error is no longer the model's general skill, but the uncertainty specific to that day itself.
Here a paradox emerges:
The days when uncertainty grows are the days when the decision is most critical economically.
Ramp periods are the very periods when imbalance and the need for urgent purchases intensify. That is, forecast performance is weakest at the moment when decision quality matters most.
For this reason, looking only at average error is not sufficient. While a model's average error improves, its infrequent but large errors may stay the same or worsen. The economic loss arises predominantly in the tail of the distribution.
The more accurate question is this:
not "How accurate is the model on average?", but
"On which days, in which weather and market regimes, and how often do large errors occur, and what is their economic equivalent?"
Why a single forecast number is not enough
Receiving a production forecast as a single number means not showing the decision-maker a significant part of the uncertainty.
A P50 forecast gives the value in the middle of the distribution. Yet the same P50 can represent two completely different situations: the middle of a narrow uncertainty distribution, or the middle of a wide one. The forecast is the same; the decision that should be taken may not be.
For this reason, the decision layer needs not only the expected value but uncertainty itself: percentiles such as P10, P50, P90.
Here there is a second distinction:
Having a forecast interval does not mean that interval is correctly calibrated.
If the model says "we will stay within this interval with 90% probability", whether this actually materialised at that rate in historical data must be tested. Calibration is measured for this purpose with standard methods: the reliability diagram and pinball/quantile loss. In other words, not only producing a forecast but measuring how far the forecast can be trusted is required.
How do we turn a forecast into an economic decision?
Once the uncertainty distribution is obtained, the decision problem is defined more clearly. There are two fundamental costs:
- S: the cost of being short — the purchase at delivery time and the related imbalance components.
- W: the cost of unnecessary purchase — the burden of excess energy remaining on hand or creating a position.
If these two costs are not equal, the economically correct decision need not be the mean of the forecast distribution either. The critical threshold is this:
θ* = W / (S + W)
Accordingly, the appropriate purchase quantity is determined so as to cover this quantile of the open position. If the cost of being short is higher than the cost of unnecessary purchase, the threshold shifts above the mean: what must be protected is not the median value but the tail of the distribution.
The core idea here is simple:
If you do not know which direction of error is more expensive, you cannot make an economically optimized decision, however good your forecast is.
And S and W are found inside no weather forecast model. These two values live in the plant's contract structure, its open position, market conditions, and the moment the decision is made.
A frequently misquoted example: DeepMind's wind study
A widespread misreading needs correcting here.
DeepMind's wind energy study published in 2019 is frequently cited as having "reduced forecast error by about 20%". The result the study actually highlights is a roughly 20% increase in the market value of wind energy.
This difference is not a minor nuance: the first statement describes a model performance, the second an economic outcome. The reason a more accurate forecast creates value is that it allows production to be offered to the market at more valuable hours.
So the same distinction appears here too: it is not the forecast itself but how the forecast is used that determines economic value.
Source note: This 20% is a figure reported in DeepMind's own corporate announcement; it is not an independent measurement. In addition, the metric heading under which the ratio was reported (market value, revenue increase, harmonisation gain) must be verified from the study's own text.
The real difficulty is not in the mathematics
Much of what has been described so far is not technically new. Forecast distributions, calibration tests and cost-sensitive decision rules are long-known methods. So why is this problem still unsolved in practice? Because the real difficulty gathers at three points, and none of them is mathematics.
1. Knowing the right numbers
What is the real open position? Which plant operates under which contract regime? What is the real cost of being short in that hour? What are the liquidity and price conditions at the moment of decision?
The answers to these questions are not found in the forecast model. All of them require a solid data infrastructure: source systems, adapters, time-series stores, consistent data definitions. This layer is often postponed because it looks boring — yet the process most often gets stuck here.
Optimization done with wrong data does not reduce error; it amplifies it.
2. Acting on a forecast with confidence
A system's declaration that "we are accurate" is not sufficient; against what and how accuracy is measured must be set out:
- What is the performance against simple reference models (naive persistence, MOS)?
- In which weather and production regimes does the error grow?
- Has seasonality been separated out from the results?
- At which hours and how often do large errors occur?
- What does the forecast error mean when converted into an economic equivalent?
nMAE = 15% is not, by itself, a meaningful economic indicator. The real question is this:
At which hours does this 15% error occur, and how much does it cost the business?
3. Defining responsibility for the outcome
This is the least discussed and probably the most decisive. The system said "buy today", the purchase was made and the price fell. Or no purchase was recommended and the price rose in the evening. Who is responsible for this difference?
For decision support systems to be usable in real life, not only technical accuracy but also governance is required: who will decide, within which limits they may act, in which cases human approval is mandatory, and how the outcome will be evaluated. A technically correct system finds no operational traction if the authority and responsibility limits of the recommendation are not defined.
A glossary correction is also needed
In the field this issue is often described as follows: "We fall below our commitment and pay a quota penalty."
Operationally, this statement describes the difficulty experienced. However, its counterpart in the legislation is different: in renewable energy support legislation there is no sanction concept in the form of a "daily quota" or "minimum production commitment". What is experienced is market operation: the schedule formed in the DAM, the corrections made in the IDM, and imbalance settlement.
This distinction is not merely terminological:
How you define the problem determines where you look for the solution.
If the problem is seen as a "quota penalty", the solution is sought in contract clauses. Yet if the economic outcome arises from market position and imbalance management, the majority of the solution lies in the decision mechanism.
Why the support regime matters
The same decision support system does not produce the same economic value at every plant. This value depends on:
- the market regime and contract structure in which the plant operates,
- the scope and duration of the support mechanism,
- the open position,
- production uncertainty.
In a support regime where the price of the energy produced is largely fixed, the hour at which the energy is sold makes a limited economic difference. At a plant selling in the free market, on the other hand, hourly price differences translate directly into economic outcomes.
For this reason it is decisive that new-generation support specifications leave plants to the free market for an initial period and protect them only with a floor price: this structure turns the decision layer from an optional improvement into a structural requirement. For plants whose support period has ended, the free-market risk increases in the same direction.
Therefore the issue is not only "a better forecast model". The real issue is placing the forecast in its economic context.
Why automation should be the last step
At this point the natural reflex is: "Then let's automate the whole process." Automation need not be the first step.
If the forecast's real performance has not been measured, the cost of being short has not been calculated, and decision responsibility has not been defined, automation only enables wrong decisions to be taken faster. A healthier order is this:
1. A human-approved advisory layer
The system produces recommendations; but it does not merely say "buy" or "do not buy". It shows the reasoning behind the recommendation, the forecast uncertainty and the expected economic impact. The final decision is made by a human.
This stage is not only a safeguard but also a measurement tool: which recommendations were accepted, which were rejected, and what the outcome was can be used over time to measure decision quality.
2. Conditional automation
Once sufficient historical data has accumulated and decision behaviour has been verified, automation can be brought in within certain limits: if the confidence level is sufficient, if the decision is within a defined risk band, if backtests provide sufficient evidence, and if authority limits are documented.
3. Full automation
Full automation becomes meaningful only when the previous stages have been sufficiently verified. Because a decision system carries responsibility not only technically but also economically and operationally.
The difference shows up not on the day you are right, but on the day you are wrong
In energy trading, an accurate forecast is no longer an advantage but a basic requirement. It is not possible to make a good decision with a bad forecast. But a good forecast alone does not guarantee a good outcome either — because the economic outcome is formed not on the day the forecast is right, but in the decision taken on the day it is wrong. And that day is, by definition, the day when uncertainty is highest.
Therefore the real question:
not "How accurate is our forecast?", but
"What do we do when our forecast turns out wrong, who makes that decision, and how is the outcome evaluated?"
When the answers to these questions become clear, a few points of improvement in the forecast model turn into measurable economic value. In a decision mechanism whose answers have not become clear, the best model cannot go beyond a detailed weather report.
What Corius does in this field
As Corius, we develop production forecasting models. But our focus on the energy side is not only producing "more accurate forecasts"; it is the layer where the forecast turns into an economic decision.
We measure the performance of your existing forecast models on your real data, convert forecast errors into economic equivalents, and analyse where the decision support layer can create value. This does not require building an extensive system.
We can start with a free preliminary analysis. The topics we can examine together on your existing data:
- forecast performance and skill level relative to reference models,
- error distribution and critical error periods,
- calibration of forecast uncertainty,
- the possible economic equivalent of the error.
Because the right starting point is not always a bigger model. Sometimes what is needed first is to ask the right question.