How Big Data Analytics is Disrupting the Energy Industry

  • 12 March 2018
  • Damon Lapping

The energy sector collects large amounts of data on a continuous basis. With the applications of sensors, wireless transmission, network communication, and cloud computing technologies. The amount of data being collected on both the supply and demand side of the coin are quite staggering. With the advent of the SMART Grid, this amount of data is only expected to increase. A perfect example of this is how much data 1 million SMART meter’s collect every 15 minutes over a year. 2 920 Terabytes to be exact and that’s only 1 million households or businesses! One can start expanding that outwards to include other intelligent devices. Examples of these include sensors and thermostats. Used throughout the whole process of power generation, transmission, distribution and substations. Then the volumes of data collected increases exponentially. Data is only valuable if it’s used so the challenge up to now has been to how can businesses and utilities use this “Big Data” in an efficient manner. What value can be derived from all this data? In the utility industry there are four types of big data sources in utilities. SMART meters, grid equipment, third-party data (off-grid data sets), and asset management data. Let’s look at how utilities are using big data to improve operational efficiencies, drive down costs and reduce carbon emissions. “Big Data” analytics is also playing a major role in energy management on the demand side. We’ll also look at achievements in this space too.

Big Data Analytics in the Energy Utility Industry

Through Big Data Analytics, energy utilities can optimize power generation and planning. Power generation planning, and economic load dispatch are the two most important decision-making processes in power generation. Economic load dispatch is an energy utility term. In simple terms it's matching power supply with the demand for energy from the grid over a short-period of time. At the lowest possible cost subject to transmission and distribution constraints. Matching energy supply and demand on the grid has always been tight balancing act and data analytics has a big role to play. By taking advantage of collected energy big data and advanced big data analytics techniques. The energy production efficiency can be improved, and production costs reduced.

Renewable energy is another important grid component that can benefit from Big Data analytics. In the SMART grid, wind power and solar power are two major renewable energy power generation methods. Yet, weather conditions significantly affect their outputs. Through using data analytics, renewable energy power generation forecasting will be more accurate and efficient. All based on weather data analysis. The integration of energy production, consumption data, GIS data, and the weather data. Examples, temperature, atmospheric pressure, humidity, cloud cover, wind speed, and wind direction. Can support the sites selection of renewable power generation devices. Thereby improving power output and energy efficiency. GIS data is another aspect of Big Data analytics. It includes geographical information from satellite imagery that aids with spatial planning. By analyzing topography, location to water and solar irradiation etc.

The energy utility industry is also an asset intensive industry. They often face many asset management challenges. Such as resource sharing, asset retirement monitoring, operation and maintenance management, procurement monitoring and inventory management. The efficiency of asset management and collaborative operation can be improved based on energy big data analytics.

Big Data Analytics in Energy Management on the Demand Side

There’s an old expression that every energy manager lives by, “You can’t manage, what you can’t measure”


Data analytics automates this process nowadays for energy manager’s responsible for managing energy consumption. Whether it be an office block, a factory, farm or even a retail store. Much of the advances made in sustainable energy has been on the demand side. Electrical devices are being designed with energy efficiency in mind reducing power requirements. Energy efficiency will be a major part of the solution to reduce global greenhouse emissions. Companies are looking for solutions to reduce energy costs. Plus become more sustainable in their energy usage. Enter Big Data analytics into the equation. Data coming from SMART meters, costs, production figures, asset operations, business policies and even weather data can be used once integrated. Once this data is analysed over the long term. Very powerful results can be achieved such as not-so-obvious energy leakages. Examples include chronic equipment efficiency issues, insulation problems and operational improvements. Energy consumption can even be forecasted, and energy savings predicted. There are a are lot of energy management software packages on the market. These systems have automated data feeds to SMART Meter’s and other data collection systems. Production data systems are an example, historical climate data and data from Building Automation Systems are others. The energy management software is then able to integrate all this data. Analyse the data and show patterns in energy consumption. Using business drivers or climatic data that affect energy consumption. Data analytics also assist’s in identifying projects that will save the most energy and money. The data analytic results that comes out of these energy management systems are in real time. Providing energy managers with a proactive tool of identifying energy issues as they occur. Not retrospectively such as analysing past utility bills when it's more challenging to identify issues.

Big Data and Cheaper Energy

According to Citi Bank, combining Big Data and data analytics with cheap energy solutions could signal free energy at some point. By matching energy supply and demand more accurately utilities can supply cheaper power. However, this is only the beginning. The concept of free energy starts with giving consumers the ability to store unused power and then sell it back to the grid, essentially recycling energy itself. Virtual Power stations is another technology to look forward to. The technology brings energy storage devices together and operates them in a centralised, digital location. Free energy is still far off, but one thing is for sure. Big Data analytics is driving down the cost of power generation and consumption!

About Damon Lapping

Damon is an experienced energy management consultant, passionate about sustainable energy. He provides Energy Management and Resource Efficiency consulting services to the UNIDO NCPC (National Cleaner Production Center) program being rolled out to developing countries. In the past he was involved in Africa’s largest utility, Eskom’s Demand Side Management program in the corporate sector. He also provides sustainable energy consulting services to clients globally. He currently lives in Cape Town, South Africa.