COMAND and ARK Energy collaboration – innovative forecasting of electricity prices

ARK Energy - innovative forecasting of electricity prices

ARK Energy is a technology company working within the energy sector. Its aim is to create accessible, reliable software for its clients. Solutions are based in the cloud, and the company has offices in Italy and Ireland.

The team helps operators within the energy sector increase their efficiency through a combination of business and IT knowledge. They have a dedicated software lab, which leverages the latest cloud technologies, actualising a vision of next-generation energy trading and risk management software for our clients.

Problem

ARK Energy’s current area of interest is the application of analytics to the execution of trading strategies and the management of associated risks. A large area of research for the electricity market in Ireland and all over the world is the forecasting of electricity prices. Accurate forecasts will allow both individuals and industries to plan for their energy consumption needs.

Partnership

With this in mind, ARK Energy conducted a feasibility study in partnership with the Software Research Institute in the Athlone IT. The study focused on the use of a variety of data including historical market data, electricity demand, wind power generation, climate data and other data for the building of several different types of machine learning models for the prediction of the following day’s electricity price.

Automated machine learning, time series and neural networks were investigated as appropriate modelling techniques. Code was written and saved in a repository for ARK Energy to have easy access to. The parameters and predictors used in the models were saved in ARK Energy’s database for future use. Each of the models created was evaluated against test data and the results were also uploaded to the database.

AIT benefitted from the data mining and machine learning aspects of this study. As many new projects wish to leverage machine learning for “smart” products, this exposure to the tools and techniques associated with machine learning and data mining is invaluable.

The research assistant involved benefitted through practical use of new machine learning tools and techniques including H2O and TPOT automated machine learning, R machine learning libraries, Python machine learning libraries. An included benefit for the researcher was exposure to the specifics of the electricity market in Ireland and other countries.

Solution

Ultimately, the objective of the project was to produce a model or models using machine learning techniques that were capable of accurately predicting short-term electricity prices. Although ARK Energy had carried out previous work in this area in conjunction with the University of Ulster, the work was academic in nature. Although the models created in the previous work saw some success, it was thought that predictions could be improved.

For these reasons, it was necessary that the previously created models were evaluated on the new data provided by ARK Energy. As the nature of the previously created models mostly concentrated on neural network techniques, the scope of the investigated modelling techniques was expanded to include time series, stacked ensembles, random forests, deep learning and many other techniques. Finding the optimal model was the key to making successful forecasts.

Model inputs

The models based their forecasts on:

  • Historical time series of ex-ante SMP
  • Historical actual load
  • Historical renewable actual production
  • Historical available capacity of interconnections into/out of market area
  • Predicted load
  • Predicted renewable production
  • Predicted available capacity of interconnections
  • And other inputs

Model outputs

  • Many kinds of machine learning models
  • Code provided in ARK Energy’s repository for model creation and data manipulation.
  • Details of each model created provided to ARK Energy in their database.
  • Low-level details of each of the models created.
  • A report to explain the work carried out.

Feasibility

In any machine learning based study, the optimal configuration of a model or the best modelling technique can be difficult to find. Although the relationship between the market electricity prices and 65 different variables were investigated, there is no guarantee that some variable(s) have not been excluded from consideration inadvertently in this study. This cannot be determined without careful thought and consideration.

The models produced and provided to ARK Energy in this report performed well overall but were found to under-predict at peak times. This was not just a phenomenon experienced with one modelling technique but will all modelling techniques investigated during this study.

This under-prediction at peak times suggests that yet another or several more variables must be added to the models to optimize prediction at the peak times. It is, as yet, unknown what these predictors are but they were not available at the time of study and therefore have not been included in the models.

Further Considerations

It was suggested that particular care should be given to the following aspects:

  • Addition of new data to the pool of existing data to build even more accurate models.
  • Further exploration of more complex models, incorporating the new data, to maximize predictive accuracy.
  • After evaluation of the model, create a production prototype to include with ARK Energy’s existing systems.
  • Analyze the applicability of the generated models to other electricity markets, such as Italy.
/by