With many initiatives in full swing to put ratings on homes and buildings across the country, there may be a lesson to be learned from recent cases where consumers have prevailed in suing car companies for promising an MPG that is not based on real-world driving.
Before you read any farther, I want to make clear that these models are not the problem, it is actually how they are being applied. Applying any predictive model to an individual building is the problem. It is up to us to use these tools as a way to manage this risk in pools, and avoid driving that risk down to homeowners. Great examples of this can be seen in the success of solar PPA and Leasing models.
Most labels, including the National RESNET HERS and California HERSII labeling system and the DOE Home Energy Score, are in fact asset scores that, similar to an MPG, score a house based on a set of average users. These scores have potentially wide variance for any particular building, and have a tendency in many climate zones to over-predict savings.
A recently released study called Modeled vs. Actual Savings for Energy Upgrade California Retrofits, analyzes predicted savings based on the CEC’s required energy modeling software, against actual results from customer bills. The study shows that California HERSII is over-predicting by and average of 50%, with a huge amount of variance between winner and losers. Resulting in more than 78% of homeowners not achieving the savings being predicted.
Before you read any farther, I want to make clear that these models are not the problem, it is actually how they are being applied. Applying any predictive model to an individual building is the problem. It is up to us to use these tools as a way to manage this risk in pools, and avoid driving that risk down to homeowners. Great examples of this can be seen in the success of solar PPA and Leasing models.
Most labels, including the National RESNET HERS and California HERSII labeling system and the DOE Home Energy Score, are in fact asset scores that, similar to an MPG, score a house based on a set of average users. These scores have potentially wide variance for any particular building, and have a tendency in many climate zones to over-predict savings.
A recently released study called Modeled vs. Actual Savings for Energy Upgrade California Retrofits, analyzes predicted savings based on the CEC’s required energy modeling software, against actual results from customer bills. The study shows that California HERSII is over-predicting by and average of 50%, with a huge amount of variance between winner and losers. Resulting in more than 78% of homeowners not achieving the savings being predicted.
A recent LBNL report on the Home Energy Score called "Accuracy of the Home Energy Saver Energy Calculation Methodology" was announced with an email headline declaring that HES is now within 1% accuracy on average, which is a great result. Of course on closer examination it is clear that there is wide variance for any individual project - perhaps even wider than other tools in the study. Which of course goes back to the original issue that homeowners are truly not interested in the average, especially if they get the bad end of the stick.
This issue is not exclusive to the United States. The Green Deal in the UK has also suffered from estimated savings that are outpacing reality. In an April 11, 2012 article in The Telegraph, called “Green measures for homes 'save less on fuel bills than forecast”
"An official pilot study of 67 homes in Sutton, South London, found all of those who took the 25-year payback package rather than ten years - a third of the owners - faced repayments higher than the savings.
Another study of 139 council houses in Sunderland found savings on energy bills were just 12 per cent rather than the expected 19 per cent.
Luciana Berger, Labour's climate change spokesman, said the proposals were "complex, confusing and leave customers exposed to mis-selling".
"Before anyone takes out the Green Deal they have a right to know how much it will cost them and how much they will save. Relying on guesswork just isn't good enough. If people are promised savings which never arrive, they will think the Green Deal is a con."
Both the CA and UK rating systems are based on a faulty notion that relative scores are more important than accuracy. In striving to achieve a relative indicator of performance, and removing behavior (how people actually use their homes) we are left with a system that is really more about policy and theory then what matters to real people - which generally boils down to, how much does it cost, and what will I save.
The idea that we are going to put MPG stickers on every home in CA, or the US, at great expense (It will cost $5B to label home in CA alone) is a mistake. At least with a real MPG on a car it is based on actually testing the vehicle, and you are not having to test every single vehicle on the road, vs. labeling buildings where we are attempting to derive energy use from physics calculations and every house needs an expensive custom test.
Here is an article on the topic of MPG from the Huffington Post, Hyundai Fuel Economy Lawsuit Filed Alleging Misleading Ads, that I think should have us all worried:
"For carmakers, the new trendy thing is to have a vehicle in the lineup that gets 40 mpg. One huge problem is that fuel efficiency figures are not based on real-world driving. And automakers opt to advertise with the fuel economy figures that are most impressive -- for highway driving -- rather than lower city or average mileage calculations, which would make their cars look less efficient.
But the Hyundai lawsuit is the second one in recent months to challenge automakers over lofty fuel economy claims. In February, California attorney Heather Peters sued Honda in small claims court over the fuel economy claims for her 2006 Honda Civic hybrid. She said she never got anything close to the 50 mpg she was promised. A judge awarded her $9,867 in the case, which Honda is appealing."
In the end our goal is to save energy and drive consumer adoption. There is a distinct risk that all of our efforts may backfire when consumers come to understand how imperfect our ratings are, and when the financial community tries to underwrite investments based on energy savings that don’t really exist.
It is time that we start focusing on real data and actual savings, rather than more complicated regulatory schemes. All this is not to say that energy efficiency does not work, instead it should tell us that we need to move from energy efficiency expressed through code and complicated ratings or scores, and instead focus on turning savings into a resource that can be valued and traded.
We are in a unique moment in time where we can move past regulatory frameworks, to engage markets that can finance our long-term goals, which are simply too expensive to achieve driven primarily with public dollars.
This article is not to say that modeling is worthless or wrong. In point of fact, it is far better at predicting savings than just simple one size fits all deemed savings, and can be very good predictors of a large pool of buildings. However, we need to rapidly move from a model where we have policy anointed solutions based on long regulatory process and instead start measuring and valuing savings predictions based on actual results. Once we have a system that can measure real savings versus predictions, markets can step in, invest, and manage savings risk so that building owners and households don't have to.
If you look at the Solar industry as a guide, where in CA residential Energy Service Contracts (Leases and PPAs) are currently 75% of the market, building owners on completely insulated from risk through performance guarantees, private capital is flowing, and quality has become a function of industry.
A system built on real proven savings at the meter will drive innovation and investment, which is the path towards a real and sustainable solution that can achieve the promise of energy efficiency in the build environment.
Enough talk and theory... get the actuarial data and the market will follow!
"An official pilot study of 67 homes in Sutton, South London, found all of those who took the 25-year payback package rather than ten years - a third of the owners - faced repayments higher than the savings.
Another study of 139 council houses in Sunderland found savings on energy bills were just 12 per cent rather than the expected 19 per cent.
Luciana Berger, Labour's climate change spokesman, said the proposals were "complex, confusing and leave customers exposed to mis-selling".
"Before anyone takes out the Green Deal they have a right to know how much it will cost them and how much they will save. Relying on guesswork just isn't good enough. If people are promised savings which never arrive, they will think the Green Deal is a con."
Both the CA and UK rating systems are based on a faulty notion that relative scores are more important than accuracy. In striving to achieve a relative indicator of performance, and removing behavior (how people actually use their homes) we are left with a system that is really more about policy and theory then what matters to real people - which generally boils down to, how much does it cost, and what will I save.
The idea that we are going to put MPG stickers on every home in CA, or the US, at great expense (It will cost $5B to label home in CA alone) is a mistake. At least with a real MPG on a car it is based on actually testing the vehicle, and you are not having to test every single vehicle on the road, vs. labeling buildings where we are attempting to derive energy use from physics calculations and every house needs an expensive custom test.
Here is an article on the topic of MPG from the Huffington Post, Hyundai Fuel Economy Lawsuit Filed Alleging Misleading Ads, that I think should have us all worried:
"For carmakers, the new trendy thing is to have a vehicle in the lineup that gets 40 mpg. One huge problem is that fuel efficiency figures are not based on real-world driving. And automakers opt to advertise with the fuel economy figures that are most impressive -- for highway driving -- rather than lower city or average mileage calculations, which would make their cars look less efficient.
But the Hyundai lawsuit is the second one in recent months to challenge automakers over lofty fuel economy claims. In February, California attorney Heather Peters sued Honda in small claims court over the fuel economy claims for her 2006 Honda Civic hybrid. She said she never got anything close to the 50 mpg she was promised. A judge awarded her $9,867 in the case, which Honda is appealing."
In the end our goal is to save energy and drive consumer adoption. There is a distinct risk that all of our efforts may backfire when consumers come to understand how imperfect our ratings are, and when the financial community tries to underwrite investments based on energy savings that don’t really exist.
It is time that we start focusing on real data and actual savings, rather than more complicated regulatory schemes. All this is not to say that energy efficiency does not work, instead it should tell us that we need to move from energy efficiency expressed through code and complicated ratings or scores, and instead focus on turning savings into a resource that can be valued and traded.
We are in a unique moment in time where we can move past regulatory frameworks, to engage markets that can finance our long-term goals, which are simply too expensive to achieve driven primarily with public dollars.
This article is not to say that modeling is worthless or wrong. In point of fact, it is far better at predicting savings than just simple one size fits all deemed savings, and can be very good predictors of a large pool of buildings. However, we need to rapidly move from a model where we have policy anointed solutions based on long regulatory process and instead start measuring and valuing savings predictions based on actual results. Once we have a system that can measure real savings versus predictions, markets can step in, invest, and manage savings risk so that building owners and households don't have to.
If you look at the Solar industry as a guide, where in CA residential Energy Service Contracts (Leases and PPAs) are currently 75% of the market, building owners on completely insulated from risk through performance guarantees, private capital is flowing, and quality has become a function of industry.
A system built on real proven savings at the meter will drive innovation and investment, which is the path towards a real and sustainable solution that can achieve the promise of energy efficiency in the build environment.
Enough talk and theory... get the actuarial data and the market will follow!
RSS Feed