Over the last four years, California Energy Commission has rolled out its HERSII rating system, which was designed as a way to provide a score to homeowners and prospective buyers that compares the energy performance of a house to similar homes (without occupant behavior). The HERSII system is based on the same engine that drives California Title 24 Energy Code and was identified as a key part of the State’s approach to driving energy efficiency in existing buildings.
The goal of the HERSII system is to label all California homes in order to equate energy performance with building value at time of sale, and as a way for a homeowner to get third-party recommendations as to the most cost effective options to save energy.
The goal of the HERSII system is to label all California homes in order to equate energy performance with building value at time of sale, and as a way for a homeowner to get third-party recommendations as to the most cost effective options to save energy.
| HERSII has been a centerpiece of the CEC’s thinking about how to implement Assembly Bill 758 (AB758), which is legislation that gives the CEC regulatory authority over existing buildings. While not explicitly called out as the solution in the legislation, the CEC has laid out a plan of attack that uses the HERSII rating as the driver to eventually mandate improvements. While there is a desire by many to consider alternatives, the HERSII policy has yet to officially change. Before we move forward with this strategy, it is critical that we evaluate all of our options and the data from early HERSII pilots. This is particularly true in light of phase three of AB758 which includes the potential of mandating ratings and cost effective energy efficiency improvements on existing buildings. | Potential Mandatory Approaches could include: |
It was brought to my attention, that while the AB758 Action Plan did not discuss HERSII issue, there was an on the record discussion on HERSII at the AB758 Fresno Workshop where Commissioner McAllister and also Bill Pennington of the CEC, at which time both expressed a willingness to at least revisit or maybe even "reinvent" the HERSII program. I would like to commend them for being willing to go on the record, but I will also implore them that we need real resolution and clarity on this issue and that waiting for the next AB758 Report to emerge and then another proceeding is prolonging uncertainty that is affecting the market.
Download: AB758 Workshop Transcript with HERSII comments highlighted
Download: AB758 Workshop Transcript with HERSII comments highlighted
Like puppies and apple pie... or is it?
HERSII sounds great and asset ratings and third-party raters make for really nice policy white papers, but when put into practice on existing buildings, the results speak for themselves ― and don't necessarily agree with the theory. The HERSII ratings system is extremely costly, has low consumer demand, and does not result in significant conversion to energy efficiency projects.
In an attempt to develop a one-size-fits-all solution to provide an asset rating for use at time of sale and in the appraisal process and provide an actionable workscope and savings projection all in one system, we have instead created a solution that does not work well for either use.
So the big question is, will the HERSII strategy work?
The good news is that we now have results back from our initial tests of the system. However, the unfortunate reality is that the test phase has demonstrated serious shortcomings in the current theory, which should at a minimum call the HERSII system into question and hopefully lead to thoughtful evaluation of the shortcomings that emerge to avoid costly missteps for the marketplace and government programs.
Based on results from HERSII tests over the last four years, the HERSII rating system does not appear to be catching on with homeowners, nor does it seems to be driving market transformation in terms of conversion to energy savings or discernible asset value improvements (though the latter is harder to know with the current dataset). Admittedly, some of our data is a little sparse, but that is a function of how hard it is to get real numbers out of these pilots.
HERSII sounds great and asset ratings and third-party raters make for really nice policy white papers, but when put into practice on existing buildings, the results speak for themselves ― and don't necessarily agree with the theory. The HERSII ratings system is extremely costly, has low consumer demand, and does not result in significant conversion to energy efficiency projects.
In an attempt to develop a one-size-fits-all solution to provide an asset rating for use at time of sale and in the appraisal process and provide an actionable workscope and savings projection all in one system, we have instead created a solution that does not work well for either use.
So the big question is, will the HERSII strategy work?
The good news is that we now have results back from our initial tests of the system. However, the unfortunate reality is that the test phase has demonstrated serious shortcomings in the current theory, which should at a minimum call the HERSII system into question and hopefully lead to thoughtful evaluation of the shortcomings that emerge to avoid costly missteps for the marketplace and government programs.
Based on results from HERSII tests over the last four years, the HERSII rating system does not appear to be catching on with homeowners, nor does it seems to be driving market transformation in terms of conversion to energy savings or discernible asset value improvements (though the latter is harder to know with the current dataset). Admittedly, some of our data is a little sparse, but that is a function of how hard it is to get real numbers out of these pilots.
Can you guess when the incentives ran out? What can we learn from the Sonoma County HERSII Pilot?
Sonoma County ran the largest HERSII pilot in the state, paying a subsidy of $700 per HERSII rating, which resulted in 100 percent free audits and ratings to customers, and unsustainably high margins for raters.
In Sonoma, we see a clear jump in the number of HERSII ratings while this lucrative incentive was on the table. However, shortly after the massive subsidy went away, the number of ratings drop immediately back to virtually the same number of ratings as were occurring before the incentive program. It would appear that even after this very expensive attempt to seed the market, there is little to no actual consumer demand in the market for ratings and one is left to wonder what happened to all the HERSII raters who got trained in Sonoma for this brief pilot, now that there is no more work.
Clearly, California does not have the funds to pay $700 or more per rating out of public funds on every home in California, so the fact that Sonoma’s 100 percent rebate pilot seemingly did nothing to jumpstart actual consumer demand is very concerning, and means an attempt to roll this system out would force California homeowners to shell out for an expensive service they are not valuing.
The fact that the Sonoma pilot program did not result in any discernable market transformation, either in terms of sustainable businesses conducting ratings or demand for ratings by homeowners, should be of serious concern.
Sonoma County ran the largest HERSII pilot in the state, paying a subsidy of $700 per HERSII rating, which resulted in 100 percent free audits and ratings to customers, and unsustainably high margins for raters.
In Sonoma, we see a clear jump in the number of HERSII ratings while this lucrative incentive was on the table. However, shortly after the massive subsidy went away, the number of ratings drop immediately back to virtually the same number of ratings as were occurring before the incentive program. It would appear that even after this very expensive attempt to seed the market, there is little to no actual consumer demand in the market for ratings and one is left to wonder what happened to all the HERSII raters who got trained in Sonoma for this brief pilot, now that there is no more work.
Clearly, California does not have the funds to pay $700 or more per rating out of public funds on every home in California, so the fact that Sonoma’s 100 percent rebate pilot seemingly did nothing to jumpstart actual consumer demand is very concerning, and means an attempt to roll this system out would force California homeowners to shell out for an expensive service they are not valuing.
The fact that the Sonoma pilot program did not result in any discernable market transformation, either in terms of sustainable businesses conducting ratings or demand for ratings by homeowners, should be of serious concern.
Do homeowners value HERSII ratings?
When we look at the adoption curve of HERSII ratings during the recovery act period, we find that ratings are highly correlated to rating incentive programs and appear to have little organic consumer demand.
In fact, as is apparent in the chart on the right, it appears that these incentive programs had little lasting impact on demand in the marketplace.
When you couple this with the emerging facts related to the propensity of the HERSII system to overpredict potential savings by as much as a factor of 3x (see: Ex Ante Tool Review Findings Disposition for Energy Upgrade California Custom (‘Advanced’) Measure Savings, 1 Mar 2013), one realizes that not only is this system not working in terms of driving customers to take action, but it is failing to do so while massively inflating the level of savings and ROI being sold to the customer.
The propensity of the HERSII system to project significantly higher savings than what is delivered means that if HERSII in its present form was applied as a mandate, there would be many millions of California homeowners who would see only a small fraction of the savings they were told to expect.
This fundamental issue is based on the CEC’s attempt to create a system that is based on code, and then apply it to operational predictions of savings. These two uses are nearly opposite in terms of how a model is constructed, and speaks to the need for systems designed for the specific purpose being asked - one size fits all is not cutting it (learn more about this issue here).
The CEC may have the legal authority to regulate ratings in California, and AB758 may give the Commission the legal basis to attempt to implement this approach, however it is highly unlikely that they have the political capital to make it stick once we start subjecting real people to these outcomes.
Asset ratings may in fact have a place, but they are really not compatible with delivering actionable workscopes in the real world, and the one size fits all approach has resulted in a system that really fits nobody very well. HERSII is an expensive and complex system to deliver ratings that have significant accuracy issues, don't work for industry, and consumers don't seem to value.
Is HERSII worth the cost?
Each HERSII rating costs at least $500 and often more, based on the fact that conducting a rating consumes half a day in the field and then more hours back in the office inputting data into the California Energy Commission (CEC) mandated software tool. There is little room for substantial economies of scale.
When you consider that we have upward of ten million homes in California, the cost of having a HERSII rating done for all California homes will require California homeowners to invest $5 billion dollars in ratings - and that is just for ratings, not actual energy efficiency.
Do HERSII Ratings turn into energy efficiency projects?
California home performance contractors have a very hard time converting HERSII third-party audits into viable leads that result in customers who are ready to make energy efficiency upgrades.
HERSII Ratings conducted in Sonoma were cross-referenced with Energy Upgrade California™ (EUC) projects and it was found that conversion from rating to energy efficiency retrofit was under 10 percent (this analysis was conducted prior to 100 percent completion of the program, and until it is redone more accurately should be considered a clear directional indicator). This is especially surprising given that in the Sonoma program, unlike elsewhere in the state, home performance contractors were allowed to provide the rebate to customers for their diagnostic test-in as long as the project included a HERSII rating upon completion. Given that contractor close rates are typically 30 percent or greater, their participation in the Sonoma pilot would have been expected to inflate the actual conversion rates when compared to HERSII on the open market by third-party raters.
At a 10 percent conversion rate, a $700 rating incentive translates into $7,000 of public funds in rating incentive per closed retrofit in addition to Utility incentives, and other Energy Upgrade California program spending. Clearly this was not a cost effective way to drive demand for retrofitting.
This trend is borne out in PG&E figures for its entire territory, which shows that of the 582 homeowners who obtained HERSII ratings, only 59 projects have been completed energy upgrades through Energy Upgrade California, which pencils out to a similar 10% conversion rate. This of course compared to conversion rates of audits to retrofits that are typical of integrated home performance contractors of over 30%.
Why is it so hard to get homeowners to act on their ratings?
The HERSII process is convoluted, complex, and expensive. From a homeowner perspective, it goes something link this:
First, a third-party rater does a HERSII rating for a homeowner who shells out something like $500 for the audit and report. If that customer is interested in doing work, they bring the rating report to a home performance contractor to get the upgrades implemented. However, here is where it starts to gets tricky and the real world collides with theory.
Contractors are ultimately responsible for a project’s outcome, including its energy performance, and cost. When presented with a third-party HERSII rating, a contractor must determine if the report recommendations match the conditions he/she finds in the home. This means the contractor always must go back to the customer and do what amounts to another audit of the building to confirm recomendations, code compliance, and pricing, but — because the customer has already paid for a rating audit — the contractor has to conduct this work for free.
On a very large percentage of these third-party HERSII ratings, the contractor will end up not agreeing with either the estimated price coded into the rating (which drives the Return on Investment and ranking in the software), or they will disagree with the solution being recommended once someone with construction experience visits the site and creates an actionable workscope.
This puts the homeowner in a bind. On one hand, they have the third-party rater sporting a CEC logo on their report telling them one thing, and a contractor telling them another thing, and far too often the homeowner becomes uncertain as to how to proceed and who to trust, so no work gets done. The third-party model ends up with very low conversion rates, and is generally not a profitable source of customers for home performance contractors.
How do we move forward from here?
It is time that we focus our energies in California on moving to a market that relies on actual performance, not complex regulatory structures translated into software and a numerical scales. The web of regulation we have created has many unintended consequences, and with the advent of smart meters and big data, we have the opportunity to move beyond regulatory proxies and instead harness private markets and sources of capital. We should embrace this change as a great success of the California system, not a failure. It is time for State policy to advance and lead the nation towards a smarter more efficient model to deliver the deep energy efficiency required if we are going to hit our climate, energy, and economic goals.
In light of the results from HERSII to date, the California Energy Commission strongly considers adopting simpler approaches to providing homeowners with ratings, that are substantially lower in cost to homeowners and do not pretend to be of investment grade accuracy. One such solution is the national DOE Home Energy Score. A very simple model can be input through an API from a variety of software tools, and is simple enough to be part of a home inspection, resulting in a 1 to 10 score for the homeowner.
These simpler systems work as a way for homeowners to gauge a home's performance but will not be confused with real energy audits capable of delivering solutions to homeowner that contractors can actually build and that will deliver reliable results. We should separate upfront simple asset ratings from the more complex needs of a diverse contractor marketplace who can deliver the level of quality auditing necessary to develop an actionable workscope and a real price. Instead of applying an expensive and inaccurate comprehensive HERSII rating on every house in the State, we instead would only do an investment grade audit when there is pathway to getting a project built.
We need to do more than tweak the knobs on the current model. We need to be open to new ideas and substantial change in our approach. Solutions are out there, but we need to be willing to admit that many current ideas are not working, and minor changes will not be sufficient to reverse the trend. It is time for California to step back into the lead and help the country move towards a sustainable market for energy efficiency.
This process should start today by admitting that the HERSII system needs to be reevaluated based on evidence and feedback from the market. It does not matter how much we have invested, we need to look at empirical results and change course. We cannot afford to keep marching forward with the same basic theories, while hoping for different results.
When we look at the adoption curve of HERSII ratings during the recovery act period, we find that ratings are highly correlated to rating incentive programs and appear to have little organic consumer demand.
In fact, as is apparent in the chart on the right, it appears that these incentive programs had little lasting impact on demand in the marketplace.
When you couple this with the emerging facts related to the propensity of the HERSII system to overpredict potential savings by as much as a factor of 3x (see: Ex Ante Tool Review Findings Disposition for Energy Upgrade California Custom (‘Advanced’) Measure Savings, 1 Mar 2013), one realizes that not only is this system not working in terms of driving customers to take action, but it is failing to do so while massively inflating the level of savings and ROI being sold to the customer.
The propensity of the HERSII system to project significantly higher savings than what is delivered means that if HERSII in its present form was applied as a mandate, there would be many millions of California homeowners who would see only a small fraction of the savings they were told to expect.
This fundamental issue is based on the CEC’s attempt to create a system that is based on code, and then apply it to operational predictions of savings. These two uses are nearly opposite in terms of how a model is constructed, and speaks to the need for systems designed for the specific purpose being asked - one size fits all is not cutting it (learn more about this issue here).
The CEC may have the legal authority to regulate ratings in California, and AB758 may give the Commission the legal basis to attempt to implement this approach, however it is highly unlikely that they have the political capital to make it stick once we start subjecting real people to these outcomes.
Asset ratings may in fact have a place, but they are really not compatible with delivering actionable workscopes in the real world, and the one size fits all approach has resulted in a system that really fits nobody very well. HERSII is an expensive and complex system to deliver ratings that have significant accuracy issues, don't work for industry, and consumers don't seem to value.
Is HERSII worth the cost?
Each HERSII rating costs at least $500 and often more, based on the fact that conducting a rating consumes half a day in the field and then more hours back in the office inputting data into the California Energy Commission (CEC) mandated software tool. There is little room for substantial economies of scale.
When you consider that we have upward of ten million homes in California, the cost of having a HERSII rating done for all California homes will require California homeowners to invest $5 billion dollars in ratings - and that is just for ratings, not actual energy efficiency.
Do HERSII Ratings turn into energy efficiency projects?
California home performance contractors have a very hard time converting HERSII third-party audits into viable leads that result in customers who are ready to make energy efficiency upgrades.
HERSII Ratings conducted in Sonoma were cross-referenced with Energy Upgrade California™ (EUC) projects and it was found that conversion from rating to energy efficiency retrofit was under 10 percent (this analysis was conducted prior to 100 percent completion of the program, and until it is redone more accurately should be considered a clear directional indicator). This is especially surprising given that in the Sonoma program, unlike elsewhere in the state, home performance contractors were allowed to provide the rebate to customers for their diagnostic test-in as long as the project included a HERSII rating upon completion. Given that contractor close rates are typically 30 percent or greater, their participation in the Sonoma pilot would have been expected to inflate the actual conversion rates when compared to HERSII on the open market by third-party raters.
At a 10 percent conversion rate, a $700 rating incentive translates into $7,000 of public funds in rating incentive per closed retrofit in addition to Utility incentives, and other Energy Upgrade California program spending. Clearly this was not a cost effective way to drive demand for retrofitting.
This trend is borne out in PG&E figures for its entire territory, which shows that of the 582 homeowners who obtained HERSII ratings, only 59 projects have been completed energy upgrades through Energy Upgrade California, which pencils out to a similar 10% conversion rate. This of course compared to conversion rates of audits to retrofits that are typical of integrated home performance contractors of over 30%.
Why is it so hard to get homeowners to act on their ratings?
The HERSII process is convoluted, complex, and expensive. From a homeowner perspective, it goes something link this:
First, a third-party rater does a HERSII rating for a homeowner who shells out something like $500 for the audit and report. If that customer is interested in doing work, they bring the rating report to a home performance contractor to get the upgrades implemented. However, here is where it starts to gets tricky and the real world collides with theory.
Contractors are ultimately responsible for a project’s outcome, including its energy performance, and cost. When presented with a third-party HERSII rating, a contractor must determine if the report recommendations match the conditions he/she finds in the home. This means the contractor always must go back to the customer and do what amounts to another audit of the building to confirm recomendations, code compliance, and pricing, but — because the customer has already paid for a rating audit — the contractor has to conduct this work for free.
On a very large percentage of these third-party HERSII ratings, the contractor will end up not agreeing with either the estimated price coded into the rating (which drives the Return on Investment and ranking in the software), or they will disagree with the solution being recommended once someone with construction experience visits the site and creates an actionable workscope.
This puts the homeowner in a bind. On one hand, they have the third-party rater sporting a CEC logo on their report telling them one thing, and a contractor telling them another thing, and far too often the homeowner becomes uncertain as to how to proceed and who to trust, so no work gets done. The third-party model ends up with very low conversion rates, and is generally not a profitable source of customers for home performance contractors.
How do we move forward from here?
It is time that we focus our energies in California on moving to a market that relies on actual performance, not complex regulatory structures translated into software and a numerical scales. The web of regulation we have created has many unintended consequences, and with the advent of smart meters and big data, we have the opportunity to move beyond regulatory proxies and instead harness private markets and sources of capital. We should embrace this change as a great success of the California system, not a failure. It is time for State policy to advance and lead the nation towards a smarter more efficient model to deliver the deep energy efficiency required if we are going to hit our climate, energy, and economic goals.
In light of the results from HERSII to date, the California Energy Commission strongly considers adopting simpler approaches to providing homeowners with ratings, that are substantially lower in cost to homeowners and do not pretend to be of investment grade accuracy. One such solution is the national DOE Home Energy Score. A very simple model can be input through an API from a variety of software tools, and is simple enough to be part of a home inspection, resulting in a 1 to 10 score for the homeowner.
These simpler systems work as a way for homeowners to gauge a home's performance but will not be confused with real energy audits capable of delivering solutions to homeowner that contractors can actually build and that will deliver reliable results. We should separate upfront simple asset ratings from the more complex needs of a diverse contractor marketplace who can deliver the level of quality auditing necessary to develop an actionable workscope and a real price. Instead of applying an expensive and inaccurate comprehensive HERSII rating on every house in the State, we instead would only do an investment grade audit when there is pathway to getting a project built.
We need to do more than tweak the knobs on the current model. We need to be open to new ideas and substantial change in our approach. Solutions are out there, but we need to be willing to admit that many current ideas are not working, and minor changes will not be sufficient to reverse the trend. It is time for California to step back into the lead and help the country move towards a sustainable market for energy efficiency.
This process should start today by admitting that the HERSII system needs to be reevaluated based on evidence and feedback from the market. It does not matter how much we have invested, we need to look at empirical results and change course. We cannot afford to keep marching forward with the same basic theories, while hoping for different results.
RSS Feed