Energy data: where does it come from and what is it good for?

21 Jul 2015 by Peter

464 Flares Twitter 0 Facebook 0 LinkedIn 464 Google+ 0 464 Flares ×

With so much talk about big data, you couldn’t be blamed for thinking that it’s the panacea of energy management and that the more you have, the more you can learn, and the better your buildings will perform.

When it comes to building data though, as with most things, the devil is in the detail. Your data can come in lots of shapes and sizes, from lots of different places, and these differences can have major implications for what it can be useful for, who it can be used by and, ultimately, how much value can be extracted from it.

Whilst the topic is a broad one, in this post I’ve attempted to summarise some of the most common data sources (together with their key characteristics) that you should be thinking about when it comes to managing the performance of your buildings. Use the links below if you want to skip ahead to the data you’re most interested in:


At The Building Level

We’ll start by looking at the data that is available from your main utility meters (also known as gate meters or market meters) and then go on to discuss sub metering.

Account or billing data

Summary: Account or billing data is typically available once a month, sometimes as infrequently as once a quarter, and often in the form of an invoice. It normally includes the total amount of energy (or water) that you consumed over the period, together with the cost of that resource. This info is widely available and it’s free, but may well require some manual work to get it into a useful format.

Resources coveredElectricity, water and gas
  • Utility invoices
  • Landlord invoices if your site sits within an embedded network
  • Your energy broker, if you use one
  • Paper bill or electronic PDF file
  • Some retailers will provide an Excel file of all your accounts if you’re a large customer, or ask nicely.
Data QualityVaries depending on how the meter is being read. This can be more of an issue if you’re part of an embedded network, where the meter reading process is often still manual.
ProsUseful for long term trending and reporting for property and environmental teams and, for the finance guys, good for bill validation when crossed-checked against interval data and your tariffs (see below).
ConsThis type of data is too coarse to be used to detect ongoing performance issues, such as a building running its HVAC system through a public holiday. Also, accessing and collating this type of data can be time consuming, particularly where multiple suppliers are involved.

Figure 1 Example of a performance report using monthly account data.png
Figure 1 Example of a performance report using monthly account data. Note the reduced energy cost in July associated with a tariff switch. Source: Greensense

“Day behind” interval data for electricity

Summary: For larger commercial buildings on Australia’s east coast, day behind interval data is available as a 24-hour delayed feed of either 15-minute or 30-minute interval data. Interval data is more useful as a performance tool than monthly account data as it provides a more granular breakdown of when you’re consuming energy.

Resources coveredElectricity
SourcesYour Meter Data Provider (MDP), if you’re based within the National Energy Market (NEM) and you have the correct meter type. If you’re in Western Australia then Western Power offer a similar data feed on a weekly or monthly basis. To find out who your MDP is contact your energy retailer. For more info on MDPs check out this link.
FormatsTypically provided as a csv file in NEM12 format
Data QualityHigh. The MDPs have processes in place to ensure meter data is complete and accurate.
  • Bill validation – when you apply your energy tariff to this form of data you can generate a “shadow bill” to compare to the one you got from your utility provider.
  • Ongoing performance management – interval data, and the ability to automate its ongoing collection and processing, make it a good starting point for identifying efficiency opportunities.
  • Measurement and verification of efficiency projects and building upgrades.
  • Because this approach leverages the existing metering reading process, no additional hardware or site visits are required.
ConsGiven the data is from your main meter, identifying the specific loads that are causing efficiency issues is difficult. You may need sub metering for that. Depending on the size and geographic spread of your building portfolio you may have to liaise with several MDPs.

Figure 2 Heat map showing weekly energy consumption patterns for a small office, generated using NEM12 interval data
Figure 2 Heat map showing weekly energy consumption patterns for a small office, generated using NEM12 interval data. Source: Greensense

“Day behind” interval data for water and gas

It’s unlikely that your main water and gas meter are read remotely by your retailer, meaning that there is no automated data feed to hook into in the way that you can with your electricity meter. However you can still get your hands on good quality interval data by installing a 3G data logger or similar communications hardware.

Resources coveredWater and gas
SourcesData logger attached to your main water and gas meter.
FormatsDepends on the data logger but typically csv files or a web service.
Data QualityGood, if the loggers are installed and maintained correctly. Consideration needs to be given to things like 3G network coverage.
ProsGood for leak detection and general performance monitoring.
ConsRequires the purchase, installation and ongoing maintenance of some logging hardware. In the case of gas meters you’ll also need to get permission from your gas network operator before connecting up any monitoring hardware to the meter.

Figure 3 Real world example of an alert triggered by unusually high overnight water use at a facility
Figure 3 Real world example of an alert triggered by unusually high overnight water use at a facility. Source: Greensense

Near real-time data (electricity)

Summary: Near real-time data for your main electricity meters can provide updates as frequently as every minute. Whilst real time data is not always required to support building performance management, it is invaluable for clients looking to reduce costs related to peak demand at their facilities.

Resources coveredElectricity
SourcesTypically you’ll need to install some additional logging hardware, however some MDPs are now beginning to offer a near real-time service in response to growing interest in demand response/management.
FormatsTypically csv files or a web service.
Data QualityPretty good if the loggers are installed and maintained correctly, although the nature of real-time data does make it more susceptible to transient issues like brief communications outages.
  • Critical component of demand response programs.
  • Identifies operational issues as they occur.
  • Good for educating and engaging building occupants around energy use. Nobody finds old, stale data interesting.
  • Can have higher costs both to set up and to maintain.
  • Generates LOTS of data. You need to know what you need it for.

Figure 4 Real world example of an alert configured to warn a property manager when demand approaches their Contract Maximum Demand
Figure 4 Real world example of an alert configured to warn a property manager when demand approaches their Contract Maximum Demand. Source: Greensense

Sub Meter Data

Summary: For buildings with sub metering installed then you’re going to be able to access data at a more granular level. Sub meters are pretty much ubiquitous in more modern commercial buildings where they are installed to monitor consumption through the main building systems, such as HVAC, lighting, and general power, but are far less common in older buildings. Depending on the type of sub metering and data collection system you have in place, data can be available in near real-time, or as infrequently as once a month if you’re relying on a manual meter reads.

Resources coveredElectricty, Water and gas
  • Building Management System (BMS) – many metering networks will feed data into the BMS, where it often remains, ignored and unloved. The good news is that, with a bit of work with your BMS contractor, you can normally get access to it.
  • Gateway Hardware – if you have a metering network that isn’t connected to the BMS then you’ll need a gateway device. This is a piece of hardware that is physically connected to the metering network, reads the meters on an ongoing basis and then makes that data available to other system, often in the form of a csv file export.
  • Manual meter reading. Not much to say here. If you’re unlucky enough to only have manually read meters then you can expect the data to come through to you once a month or thereabouts, probably as an Excel file.
FormatsVaries based on the data source and include Excel files through to sophisticated web services.
Data QualityCan be very variable depending on how well the sub metering network was installed, commissioned and maintained.
ProsProvides a level of insight into building performance that is simply not possible to get from your utility meter.
ConsThe installation of sub metering can be expensive and, particularly in older buildings, quick complex.Generates lots of data which can be overwhelming if you don’t have the right tools and experience to handle it.

Figure 5 Sub metering data showing the breakdown of energy use in a commercial building
Figure 5 Sub metering data showing the breakdown of energy use in a commercial building

Other data

Summary: The two biggest variables influencing energy performance for most commercial property are weather, in particular temperature, and occupancy. In simple terms, if the weather never changed and the occupancy levels within your building remained constant then you’d expect energy performance to flat line. Given that both weather and occupancy can vary significantly over time it makes sense to try and incorporate some of this data into your performance monitoring process.

Weather Data

Resources coveredTemperature and humidity are the most useful here.
  • Nearest weather station from the Bureau of Meteorology (BOM) or your national equivalent.
  • Local weather station for the building, if it has one.
FormatsVaries but common formats include manually downloaded csv files. The BOM also provide an automated data feed for the tech savvy.
ProsThis data allows you to understand how changes to weather impact the performance of your facilities. There are lots of examples where this is useful, one of the most financially significant can be managing HVAC driven peak demand events during very hot or cold weather. See this article.
ConsNormalising building performance to take weather into account can be tricky and requires experience. Like any kind of statistical analysis, it’s prone to abuse and misinterpretation. Be warned. You can read more about Degree Day analysis here.

Occupancy Data

Resources coveredOccupancy – both in terms of typical hours that your buildings are occupied, and also the occupancy levels.
  • Building schedules, although these often don’t represent the true occupancy hours for a building
  • Movement tracking hardware
  • Security systems
FormatsVaries depending on the source of the data
ProsSome of the best and most obvious efficiency opportunities are associated with empty buildings, or buildings with very low levels of occupancy – think late at night or over weekends, when a facility may have only a handful of people working but may still be consuming a lot of energy. More on that here.
ConsAccess to good quality occupancy data and be difficult. Ideally you want real information of occupancy patterns rather than just the typical operating hours, which are much easier to find, but much less useful.

The wrap

No matter where you are on the energy management journey, you’re going to need good quality data to take you to the next level. Hopefully this post has given you a few ideas on where to go looking for that data and how to use it. If you want to discuss your options in more detail, the Greensense team are always happy to help, just drop us a line at


Leave a Reply

Request a demo

Please complete the form below to organise a one-on-one demo