Project LEO Baselining Working Group Summary Report

This report aims to summarise the discussion and learnings relating to baselining that arose throughout the real-world trials carried out as part of Project LEO and TRANSITION.

These trials ran between November 2021 and February 2023. The work includes an analysis of the accuracy of the methods used, alongside variations of Historical Baselining and other separate methods, applying these to a range of DER types that took part in the real-world trials. The work also highlights some of the challenges and shortcomings of the baselining methodology and the impact these have on the wider market process.

This report aims to summarise the discussion and learnings relating to baselining that arose throughout the real-world trials carried out as part of Project LEO and TRANSITION.

These trials ran between November 2021 and February 2023. The work includes an analysis of the accuracy of the methods used, alongside variations of Historical Baselining and other separate methods, applying these to a range of DER types that took part in the real-world trials. The work also highlights some of the challenges and shortcomings of the baselining methodology and the impact these have on the wider market process.

Some of the key findings and recommendations include:

• The context specific analysis of baseline errors applied across different DERs provides
  valuable insights into method performance and suitability. The process should be available
  to market facilitator and industry actors to provide greater market transparency and could
  be included within the baselining process itself.
• The accuracy of the baselining and verification process has impacts on the wider operation
  of the market. The settlement rule put in place to discourage under-delivery is asymmetric
  with a grace window of only 5%. For the data analysed, baseline error analysis suggests that on the order of 15% of events could be underpaid because of baseline errors. The
  settlement rule should be modified to reduce this and avoid potential capacity sterilisation
  due to flexibility providers holding capacity in reserve.
• The simplest method of Meter Before Meter After (MBMA) which just uses an interpolation of the profile from the points immediately before and after the event, consistently saw the best accuracy for the methods and data analysed. It also appears to be a close match to the way small scale DER flexibility is controlled (the response being set relative to usage at the time of the event). However, it is the most prone to manipulation if the service provider has prior knowledge of the event. Further study is needed to see if the risk of manipulation materialises within local flexibility markets that overcomes the benefits of a simple and transparent method.
• The use of Same Day Adjustment (SDA) to correct for daily variations in usage resulting from external factors such as temperature typically provides a more accurate (smaller average error) historic baseline. However, the trials exposed practical limitations to the implementation of SDA if the DER undergoes any pre-conditioning, also exposing the
  method to manipulation. The use of SDA needs careful consideration if the gain in accuracy 7 is to be realised over the potential for manipulation or greater inaccuracy. It is most suitable for services with immediate real-time instruction with little to no warning.
• The ability to stack services within both local and national flexibility markets will be
  important in any DERs business case. Service stacking can impact the baseline if the stacked services occur within historical days contributing to the baseline calculation or within the SDA window. To overcome this, market facilitators need information about a DERs participation in other services; this could either be through notification by the flexibility provider as part of the market rules, or better, managed through a centralised database that all market facilitators (DSOs, ESOs, registered aggregators etc) have access to – this reduces burden on the flexibility provider.
• To baseline at scale with low transaction costs, baselining must be an automated process
  which is integrated into the wider market processes. To achieve this, data must be of sufficient quality and in the correct format. During the Project LEO trials, quality assurance
  and data cleaning was a manual (and laborious at times) process. Industry wide data standards and integrated cleaning/formatting tools are needed but must be a balance so as not to make participation too strict that discourages participation.

In general, the baselining processes were seen by market participants as complicated and not transparent. Alternative flexibility markets that do not rely so heavily on baselining for verification, such as firm capacity markets, should be explored as an alternative.

The insights presented herein are an output of the baselining working group within Project LEO with input from all project partners. Work is ongoing as part of TRANSITION with collaboration with the FUSION project