Skip to main content
  • English (en)
  • Accessibility
  • Sitemap
  • About this site
  • FAQ
  • Legal notice
  • Cookies
  • Contact
  • Search
SETIS

SETIS

Strategic Energy Technologies Information System
Home
Menu

You are here

European CommissionSETISPublicationsSETIS MagazineEnergy Systems ModelingThe importance of open data and software for...

Header menu

  • Home
  • Search
  • Home
  • About SETIS
    • SET Plan community
    • A European Green Deal
  • Actions towards implementing the Integrated SET Plan
    • No 1 in Renewables
    • Smart Solutions for Consumer
    • Smart Resilience and Secure Energy System
    • Energy Efficiency in Buildings
    • Energy Efficiency in Industry
    • Batteries and e-Mobility
    • Renewable Fuels and Bioenergy
    • Carbon Capture Utilisation and Storage
    • Nuclear Safety
    • Towards an Integrated Roadmap
    • Implementation plans
  • Low Carbon Energy Technologies
    • No1 in Renewables
    • Smart Systems and Consumers
    • Efficient Energy Systems
    • Sustainable Transport
    • Carbon Capture Utilisation and Storage
    • Nuclear Safety
  • Publications
    • SET Plan Implementation Progress Reports
    • SETIS Research & Innovation data
    • SETIS Magazine
    • Relevant reports
    • Integrated SET Plan infographics
  • Calendar

Search form

Recent news

Recent News

Energy Systems Modeling

SETIS Magazine, November 2016

Subscribe

Index

Editorial
SET-Plan Update
The EU Reference Scenario 2016
A Better Life with a Healthy Planet. Pathways to Net-Zero Emissions
Marc Oliver Bettzüge talking to SETIS
Energy system modelling in the industry: the EDF R&D perspective
David Connolly talking to SETIS
METIS: the new Directorate-General for Energy short-term energy system model
Alistair Buckley talking to SETIS
The importance of open data and software for energy research and policy advice
Mark O’Malley talking to SETIS
The Nordic Energy Technology Perspectives 2016
OSeMOSYS: open source software for energy modeling
Shared experiences in integrated energy systems modeling

Publications

  • SET Plan Implementation Progress Reports
  • SETIS Research & Innovation data
  • SETIS Magazine
  • Relevant reports
  • Integrated SET Plan infographics

The importance of open data and software for energy research and policy advice

LinkedIn Twitter Facebook Google Plus 

Given the uncertainty and complexity of the energy system, quantitative models are vital tools to explore alternative scenarios and help guide public policy. Yet most models and data remain inscrutable “black boxes” – whether small econometric models or large linear optimisation models with hundreds of thousands of input variables. In contrast to closed models, “open” models imply that anyone can freely access, use, modify, and share both model code and data for any purpose (Open Knowledge Foundation, 2015).[1] In this article we argue why energy data and models urgently need to become open; discuss the key reasons why many are currently not; and propose some next steps for the energy research community.

Why models and data should be open


©istock/maxkabakov

Given the critical guidance that energy models and data provide to decision-makers, they should be made open and freely available to researchers as well as the general public, for four reasons:

  1. Improved quality of science. Transparency, peer review, reproducibility and traceability lead to higher quality science. Yet these principles are almost impossible to implement without access to models and data (DeCarolis et al., 2012; Nature, 2014).[2] Human error is inevitable under pressure to deliver, and model mistakes can have profound implications. For example, the Reinhart-Rogoff[3] spreadsheet error arguably skewed the international debate on austerity (Herndon et al., 2014)[4]. Such incidents serve as warnings against poor programming practices such as a lack of auditing, as well as closed models and data: it was only through sharing the spreadsheet that the errors were discovered.
  2. More effective collaboration across the science-policy boundary. Better and more transparent science itself ought to enable better policy outcomes. Academic peer review routinely does not check model arithmetic and data validity, and so a separate process of quality assurance is required. While mostly absent from academic practice, this is often implemented as a formal procedure in government (e.g., DECC, 2015)[5]. Unlike academics, governments often model for numbers rather than insight. The specific numbers can be of great societal importance, such as the level at which to set subsidies or the cost of specific policies. Often, the most important aspect is the quality or transparency of input data, rather than the novelty of the modelling methodology. In large datasets used in government decision-making, traceability and referencing can become major problems, as civil servants developing models and data are often not trained scientists. Openly available, collaboratively developed datasets and reference models would allow the burden of this work to be shared more widely, and across both academia and government.
  3. Increased productivity through collaborative burden sharing. Collecting data, formulating models and writing code are resource-intensive, while research funding and time are scarce resources. Society benefits if researchers avoid unnecessary duplication and learn from one another. Individual researchers gain more time to spend on pressing research questions rather than redundant work on model or dataset development. Furthermore, research only matters if it is seen and used, and open-access publishing has been shown to increase readership and citations (McCabe and Snyder, 2014)[6]. Since openly shared code or data is more likely to be known to others, it is more likely to be used and further improved. This benefits the original researcher through peer recognition and academic credit, and moves the research community as a whole forward.
  4. Profound relevance to societal debates. Reengineering the energy landscape will affect everyone, producing winners and losers. A balanced societal and political debate requires transparent arguments based on scientific justifications, but escalating concern about reproducibility in some fields is shaking public confidence in scientific research (Goodman et al., 2016)[7]. Finally, besides the practical considerations outlined above, there remains the ethical argument that research funded by public money should be available to the public in its entirety.

Why they are (mostly) not open                                                                                                  

Despite these arguments, we see four main reasons why closed models and data may remain attractive and rational in some cases:

  1. There is a range of valid ethical and security concerns, particularly with data. Researchers may have access to data containing commercial sensitivities or personal information (particularly relevant when moving towards more decentralised smart grids with their focus on individual households).
  2. Openly sharing details of models, analysis and data can create unwanted exposure. Flawed code or data can discredit research results and cause embarrassment to their authors, but only if they are visible. Some may also fear that inexperienced researchers will use an open model or open data to produce flawed analysis that reflects poorly on its original authors.
  3. It is time-consuming to write legible and reusable code, track processing steps, write documentation and respond to feature requests. Because model and dataset development are large investments, it is often rational for researchers and institutions to maintain “trade secrets” to compete for third-party research funding: a classical collective action problem where individual actors are trapped in a suboptimal non-cooperative equilibrium.
  4. Finally, there is simple institutional and personal inertia, often alongside complex and uncoordinated institutional setups.

While understandable from the perspective of individual actors, collectively these engender a sense of mistrust in complex, impenetrable models and enigmatic datasets. For example, the European Commission faced criticism for using the proprietary PRIMES model to deliver key results for its Energy Roadmap 2050 (Helm et al., 2011).[8] More significantly, the UK’s decarbonisation was arguably delayed for years by models that underestimated the scale of the challenge due to opaque and heroically optimistic cost assumptions for onshore wind (House of Lords, 2005).[9]

What needs to be done

Individual researchers and research groups must understand the practicalities of open code and data. These range from issues like considering the intended target audience and choices such as licensing and distribution channels. Pfenninger et al. (2016)[10] give guidance specifically for energy research. More importantly, the energy research community as a whole needs to move forward on several fronts:

  1. Work towards reducing parallel efforts and duplication of work. There should be better coordination between different modelling efforts. This can include the development of common code bases, common datasets, community standards to ensure interoperability, and coordinated efforts to enable third-party verification of model-based results.
  2. Increase transparency and reproducibility. Community efforts towards tested and documented code packages for specific tasks can serve an important purpose. But one-off analyses created for specific papers, or code that is written with the understanding that it will never be made public, may be poorly documented and structured, meaning its release would be of limited use.
  3. Change incentives and bring aboard different stakeholders. The energy research community and specifically the emerging open modelling and open data communities must engage with other stakeholders to ensure institutional and academic recognition for open energy models, and to start tackling the harder problems that follow. Open and transparent research is not currently incentivised: in fact, the opposite is often perceived as advantageous for scientific career advancement. Changing these incentives will require efforts not only from researchers themselves but also from their employers, from grant agencies, and other stakeholders like publishers (Nosek et al., 2015).[11]

Given the importance of rapid global coordinated action on climate mitigation and the clear benefits of shared research efforts and transparently reproducible policy analysis, the community still has much work ahead.

 

Dr Lion Hirth

Dr Lion Hirth is a post-doctoral researcher at the think tank MCC, a fellow at the PIK research institute, and director of the consulting firm Neon. His research interest lies in the economics of wind and solar power. He has been concerned with open energy data in a number of projects and maintains the open-source power market model EMMA.

 

Dr Stefan Pfenninger

Dr Stefan Pfenninger is a member of the Climate Policy Group at ETH Zurich, Switzerland. He holds degrees in environmental science, environmental policy, and environmental engineering. His research interests span the challenges and opportunities in the clean energy transition, particularly the modelling of futures with high shares of wind and solar energy.

 

Joseph DeCarolis

Joseph DeCarolis is an Associate Professor in the Department of Civil, Construction, and Environmental Engineering at North Carolina State University. His research is focused on the interdisciplinary assessment of technologies and public policies that promote long-term energy sustainability. He is particularly interested in developing robust decision-making strategies for climate mitigation by conducting analysis with technology-rich energy system optimisation models.

 

Sylvain Quoilin

Sylvain Quoilin obtained a European Doctorate in Mechanical/Energy Engineering in 2011 at the University of Liege (Belgium), where he then became lecturer. He is now a scientific officer at the Joint Research Centre of the European Commission, focusing on energy policy support and on the modelling of future European energy systems.

 

Iain Staffell

Iain Staffell is a lecturer in Sustainable Energy at Imperial College London, holding degrees in Physics, Chemical Engineering and Economics. His research centres around decarbonising electricity, ranging from the economics of battery storage and nuclear power to modelling the integration of renewables into electricity systems.

 




[1] Open Knowledge Foundation, 2015. Open Definition 2.1 - Open Definition - Defining Open in Open Data, Open Content and Open Knowledge [WWW Document]. URL http://opendefinition.org/od/2.1/en/ (accessed 3.15.16).

[2] DeCarolis, J.F., Hunter, K., Sreepathi, S., 2012. The case for repeatable analysis with energy economy optimization models. Energy Economics 34, 1845–1853. doi:10.1016/j.eneco.2012.07.004; Nature, 2014. Journals unite for reproducibility. Nature 515, 7–7. doi:10.1038/515007a

[3] Herndon, T., Ash, M., Pollin, R., 2014. Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Camb. J. Econ. 38, 257–279. doi:10.1093/cje/bet075

[4] Herndon, T., Ash, M., Pollin, R., 2014. Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff. Camb. J. Econ. 38, 257–279. doi:10.1093/cje/bet075

[5] DECC, 2015. Quality Assurance tools and guidance in DECC [WWW Document]. URL https://www.gov.uk/government/collections/quality-assurance-tools-and-gu... (accessed 6.2.16).

[6] McCabe, M.J., Snyder, C.M., 2014. Identifying the Effect of Open Access on Citations Using a Panel of Science Journals. Economic Inquiry 52, 1284–1300. doi:10.1111/ecin.12064

[7] Goodman, S.N., Fanelli, D., Ioannidis, J.P.A., 2016. What does research reproducibility mean? Science Translational Medicine 8, 341ps12–341ps12. doi:10.1126/scitranslmed.aaf5027

[8] Helm, D., Mandil, C., Vasconcelos, J., MacKay, D., Birol, F., Mogren, A., Hauge, F., Bach, B., van der Linde, C., Toczylowski, E., Pérez-Arriaga, I., Kröger, W., Luciani, G., Matthes, F., 2011. Final report of the Advisory Group on the Energy Roadmap 2050.

[9] House of Lords, 2005. The Economics of Climate Change. Vol. II. 2005, HL 12-II of 2005-06, QQ 407-408.

[10] Pfenninger, S., Schmid, E., Wiese, F., Hirth, L., Davis, C., DeCarolis, J.F., Fais, B., Krien, U., Matke, C., Momber, I., Müller, B., Pleßmann, G., Quolin, S., Reeg, M., Richstein, J.C., Schlecht, I., Shivakumar, A., Staffell, I., Tröndle, T., Wingenbach, C., 2016. Benefits, challenges and solutions for open energy modelling. Open Energy Modelling Initiative Working Paper. URL https://openmod-initiative.github.io/openmod-working-paper/ (accessed 1.7.16).

[11] Nosek, B.A., Alter, G., Banks, G.C., Borsboom, D., Bowman, S.D., Breckler, S.J., Buck, S., Chambers, C.D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D.P., Hesse, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Mabry, P., Madon, T., Malhotra, N., Mayo-Wilson, E., McNutt, M., Miguel, E., Paluck, E.L., Simonsohn, U., Soderberg, C., Spellman, B.A., Turitto, J., VandenBos, G., Vazire, S., Wagenmakers, E.J., Wilson, R., Yarkoni, T., 2015. Promoting an open research culture. Science 348, 1422–1425. doi:10.1126/science.aab2374

 

  • Archive
  • Newsroom
  • Toolbox
  • Last update: 15/12/2020
    • Accessibility
    • Sitemap
    • About this site
    • FAQ
    • Legal notice
    • Cookies
    • Contact
    • Search
    | Top