Big Data in Energy Industry

Introduction to Big Data

Big Data or Big Data Analytics is one of the concepts in digitization and is the use of advanced analytic techniques against very large, diverse data sets that include structured, semi-structured and unstructured data, from different sources, and in different sizes from terabytes to zettabytes.

Big data is a term applied to data sets whose size or type is beyond the ability of traditional relational databases to capture, manage and process the data with low latency. Big data has one or more of the following characteristics: high volume, high velocity or high variety. Artificial intelligence (AI), mobile, social and the Internet of Things (IoT) are driving data complexity through new forms and sources of data. For example, big data comes from sensors, devices, video/audio, networks, log files, transactional applications, web, and social media — much of it generated in real-time and at a very large scale.

What does Big Data constitute of?

The process begins with data acquisition followed by data analysis, curation, storage and then its usage. There are several technologies within individual value chain components.

Splunk, Hadoop, Tableau are some of the leading solution providers in Big Data space with established solutions.

Big Data in Energy, and how is it disrupting the industry?

Energy and utility organizations apply smart technology to their landscape, including sensors, cloud computing technologies, wireless, power planning, and network communication. These produce large data sets on a continuous basis which gets collected over a period of time. For example, a utility company, using smart meters and power, can collect around three petabytes of data every 15 minutes for a year for about one million households.

If we start expanding intelligent devices like sensors and thermostats, we are talking of large-volume data sets being generated across power generation to transmission to distribution and then to consumers via substations. Businesses across the utility industry are facing a lot of challenges to draw insights out of this valuable data and conduct power planning.

Source of data and the value derived

In an energy and utility company, there are various sources of data, including grid equipment, smart meters, weather data, and measurements from power systems, GIS data, storm data, and data related to asset management. These companies are using this data to bring in operational efficiencies, reduce costs, lower carbon emissions, and manage energy demand for end consumers.

• Power generation planning: Utility companies can optimize their power planning and generation using analytics. There are two key decision-making processes in power generation – power planning and dispatching of the economic load. Once we gather all the data from multiple sources, there are multiple models run on top of that data to arrive at power planning. By economic load dispatch, we mean matching energy demand with the optimal power supply from the grid over a specific time frame.
• Efficient and accurate forecasting: Data analytics helps in accurately forecasting the energy consumption which plays a pivotal role in the generation and thus, dynamic pricing. Similarly, it plays an important role in forecasting the power generation, especially for renewable energy sources which include solar as well as wind, which gets impacted due to changing weather conditions. This all gets taken care by doing predictive analysis on all the data taken from weather systems.
• Site selection: The integration of all the data, be it energy production, energy consumption, GIS, and weather data like wind direction, temperature, humidity, atmospheric pressure, cloud, and wind speed can support the sites selection where renewable power generation devices have to be installed. This improves energy efficiency as well as power output and brings in a lot of efficiencies. GIS data equally plays an important role. It includes geographical information data from satellite data or LiDAR (light detection and ranging) that helps in spatial (three dimensional) planning.
• Asset management: The industry has asset-intensive units. Companies regularly face a lot of asset management-related challenges; for example, asset operations, asset monitoring, sharing of resources, asset maintenance, asset procurement, inventory management, etc. Utility companies can achieve efficiency based on insights drawn from the analytics.
• Energy efficiency: Data coming from smart meters, asset operations, business policies, and weather data can be integrated and analyzed over a period of time which helps in designing electrical devices with energy-efficiency parameters, thus reducing power requirements. Energy efficiency plays an important role to reduce carbon emissions. This also includes various other issues like equipment efficiency issues and problems in insulation, as well as improvements in operational areas. This way, companies can forecast their energy consumption and predict energy savings.

Interesting Start-ups to watch out in Big Data

• Currant: Based in USA and focuses on energy efficiency improvements
• SparkCognition: Based in USA and focuses on asset failure detection
• VIA: Based in USA and focuses on predictive analytics
• Ambyint: Based in Canada and focuses on upstream E&P operations
• Raptor Maps: Based in USA and focuses on virtual inspections
• Avrioenergy: Based in India and focuses on energy efficiency improvements
• deMITasse Energies: Based in India and focuses on efficient power generation