As energy efficiency programs increasingly apply behavioral approaches to maximize savings, the use of two-way communication technologies has emerged as a valuable tool. Connected technologies, e.g., smart thermostats, web portals, and smart phone apps, can open up new opportunities to achieve behavior-based energy savings and can even assist in the evaluation of behavioral efforts. This paper explores the ways in which energy efficiency programs are leveraging new technologies and behavioral approaches to change electricity use behavior in the residential and small commercial sectors.
Three pilots serve as examples of what this work looks like in practice and how it is evaluated: Focus on Energy’s iCanConserve pilot, Pacific Gas and Electric’s Home and Business Area Network Pilot, and the Sacramento Municipal Utility District’s In-Home Display Pilot. These pilots have facilitated two-way interaction via different technologies including smart phone apps, web portals that provide detailed electricity use information, and near real-time feedback provided through displays in customers’ homes or businesses. These pilots provide actionable information to energy users in a way informed by social science research in order to encourage customer engagement and reduce electricity consumption.
This paper focuses on the information provided via these technologies, the behavioral insights leveraged to maximize the impact of this information, and the early lessons learned about appropriate evaluation approaches and related results.
Smart Devices and energy efficiency
The energy efficiency industry has increasingly recognized that many of the behavioral techniques used successfully by other disciplines such as psychology and public health have great potential to shift energy use practices and achieve new energy savings as well. There has also been growing interest in efficiency programs that incorporate connected technologies such as smart thermostats, web portals, and smart phone apps. These technologies provide new ways to engage customers and increase energy savings. More recently, programs have begun to include both connected technologies and behavioral approaches, leveraging the combined power of technology and social science insights with the goal of amplifying energy savings.
An ongoing challenge for behavior-based programs has been the ability to demonstrate quantitatively that they result in measurable and significant energy savings. With the goal of verifying savings in mind, a number of program administrators have begun the process of testing out innovative pilot types that utilize new technologies in combination with longstanding social science knowledge. Although still in the early stages, these programs may eventually help us achieve—and more rigorously demonstrate—quantifiable energy efficiency gains from behavior. In this paper, three pilots serve as examples of what this work looks like in practice: Focus on Energy’s iCanConserve pilot, Pacific Gas and Electric’s Home and Business Area
Network Pilot, and the Sacramento Municipal Utility District’s In-Home Display Pilot. For each pilot, we outline the key program information, including the technologies used, behavioral insights incorporated, evaluation approaches employed, and both quantitative and qualitative results achieved. A high level overview of each pilot is provided in Table 2, including details such as the evaluation design, geographic location, and energy savings for each pilot. Later on in the paper, we distill some early themes, with a focus on challenges, successes, and potential areas for future research.
Purpose, Scope, and Methods
The purpose of this paper is to shed light on how energy efficiency program administrators have begun to incorporate connected technologies and behavioral approaches in their programs. We accomplished this through a secondary research effort.
The methods used reflect the intended goal of illustrating some of the different approaches underway. There were two main criteria for inclusion in this study. First, the organization implementing the pilot or program had to be a member of the Consortium for Energy Efficiency (CEE) in order to facilitate access to the evaluation report as well as supplemental pilot information. Secondly, the pilot or program had to have at least a preliminary impact evaluation completed and made available to CEE by March 16, 2015.
CEE staff identified relevant pilots and programs from the Behavior Program Summaries that CEE uses to collect information about members’ behavior-based efficiency programs as well as through informal inquiries from CEE staff to program administrators. In order to gather relevant evaluation reports, CEE staff reached out to the program contacts listed for all relevant pilots and programs via e-mail to find out whether evaluations of those efforts had been completed, and followed up with any contacts who did not initially respond. As a result of these efforts, CEE staff received impact evaluation reports for pilots or programs underway at the following organizations: Focus on Energy, the Sacramento Municipal Utility District (SMUD), and Pacific Gas and Electric (PG&E). (Although nine relevant pilots were identified, these were the only three that were far enough along to have completed evaluation reports.)
The programs described here include only CEE member programs, though CEE members direct nearly $6.4 billion of the $8 billion in energy efficiency and demand response program expenditures in the U.S. and Canada (CEE 2015). The results discussed here are intended to be qualitative in nature and do not aim to represent behavior programs with connected technologies—or CEE members’ behavior programs with connected technologies—on the whole. Thus, other organizations implementing these types of programs might not experience similar results to those reported here. Rather, the intent is to inform other program administrators of some key program findings from these specific program examples and provide tangible details on the various approaches used in these pilots.
Program Approaches
Focus on Energy’s iCanConserve
Pilot Overview. The iCanConserve pilot by Focus on Energy and Wisconsin Public Service had a number of goals, among them to achieve cost-effective energy usage reductions, better understand both customers’ information needs related to energy use as well as “the impact
of programs that are offered in conjunction with advanced feedback mechanisms,” and to test the effectiveness of social science approaches designed to alter behavior (KEMA 2013). While this was primarily a residential pilot, it did include some commercial customers, though too few on which to base any statistically significant evaluation conclusions (KEMA 2013). The pilot focused on reducing both electricity and natural gas usage.
This pilot took place from 2010-2013 in three communities in Wisconsin: Brillion, Plover, and Allouez. The program components included energy efficiency program offerings, nonstandard rates, “Tools and Technology” options, and a community reward to recognize high area-wide participation (KEMA 2013). The Tools and Technology component of the pilot used a number of different technologies, including smart thermostats, in-home displays, and home energy management systems (KEMA 2013). Customers were automatically enrolled in the TOU rates in Plover, but these rates were opt-in in both Brillion and Allouez (KEMA 2013). Participants had the option to receive notification of peak events via a variety of technologies— including text, email, and phone.
Behavioral Insights. Focus on Energy used a whole host of behavioral insights based on social science throughout these pilots. For instance, interpersonal communication took place during the in-home audit portion of the program, and the evaluators for this pilot reported hearing positive feedback on in-person interaction in many different aspects of the pilot (KEMA 2013). Discounting the future—that people tend to place a lower value on future costs or rewards than current ones—was another behavioral insight that was quite successfully incorporated into these pilots (CEE 2010); Focus on Energy effectively generated repeated surges in demand for participation in the pilot by releasing time-limited offers that helped overcome this bias (KEMA 2013). Moreover, community rewards helped leverage the behavior insights of competition and social norms, tying individuals’ actions to the success or failure of the broader group and encouraging participants to perform energy efficient behaviors with the aim of winning a communal reward (CEE 2010). Community rewards bestowed upon participating communities (into which participants could provide input) included a $25,000 grant towards exterior lighting at a local sports park in Plover and LED path lighting in Allouez.
Finally, pilot participants in Plover were automatically set up to receive time of use (TOU) rates—while they had the opportunity to opt-out if they preferred, this approach helped avoid the status quo bias of opt-in programs, where people are far more likely to stick with the default option rather than opt into a program (CEE 2010). As a result, 57 percent of the Plover customers who were automatically set up on TOU rates did not opt-out of these rates, while a mere 3 percent of customers across the other two communities, most of whom were asked to opt- in, chose to participate (KEMA 2013).
Evaluation Design and Results. The evaluation for this pilot was conducted using a quasi-experimental design, which incorporated comparison groups for all three communities but did not include random assignment. The evaluation of the pilot aimed to determine any impacts on participants’ attitudes and behaviors related to energy efficiency, assess savings achieved (including savings for different rate options), and to distill key takeaways that could inform future program designs (KEMA 2013). The data source used for the evaluation of these pilots was primarily billing data, supplemented with customer interviews. Interestingly, this pilot was unique because energy savings were also measured at the community level in the aggregate (KEMA 2013).
Overall, the pilot resulted in similar savings across all three communities, with the average savings hovering right around three percent in each (KEMA 2013). The combination of time of use rates with the Tools and Technology piece appeared to create an interactive effect on achieved savings. While customers participating in time of use rates only saw a two percent reduction in their energy usage and customers only using Tools and Technology reduced their electricity use by four percent, customers participating in both saved nine percent (KEMA 2013); however, this finding was not statistically significant due to the small sample size. Additionally, there wasn’t any evidence that pilot participants did, in fact, shift their energy use to off-peak times in response to the TOU rates (KEMA 2013).
The qualitative aspects of the evaluation are equally noteworthy. Based on survey results and interviews that served as the primary data source, the evaluation found that knowledge changed more than attitudes, and that the impact was stronger among residential participants than commercial (KEMA 2013). That said, customer interviews throughout the pilots revealed that these efforts were successful at the intended goal of raising customers’ awareness of the pilots and their related energy options (KEMA 2013).
Key Takeaways. One main takeaway from the evaluation was that the speed of the pilot rollout made it all but impossible to apply lessons learned in one community to the next (KEMA 2013). In the future, a more gradual program rollout might facilitate the process of applying key takeaways from one community to another, while still keeping in mind that potential demographic or geographic factors might not make the lessons learned entirely transferable.
Participants in these communities frequently reported that saving money was a key factor motivating their participation (KEMA 2013). Future iterations of this pilot could perhaps focus more heavily on messaging around saving money to take advantage of this finding. By leveraging the behavioral insight of loss aversion—the idea that people hate to lose something they already have more than they like to gain something they don’t (CEE 2010)—this messaging might resonate best with potential participants.
One consideration that this pilot demonstrated was the importance of aligning the intended behaviors with the technical capabilities of the technologies. For instance, customers had the option to receive notification of peak events via a variety of technologies, including text, email, and phone. Yet many who received notifications of upcoming peak events could not take any action because they were away from home without the ability to control electricity-using devices remotely (smart thermostats were just one of the technologies that were available, and many customers were using other technologies that didn’t include remote capabilities) (KEMA 2013). This is an indication of the potential value of the connected technologies included in these pilots. Perhaps creating a closer link between TOU notifications and mobile connected technologies that allow customers who are away from home to still act in advance of peak events could further enhance savings.
PG&E’s Home and Business Area Network Pilot
Pilot Overview. The purpose of Pacific Gas and Electric’s (PG&E) Home and Business Area Network (HAN) Phase 3 pilot was “to help participating residential as well as small and medium business (SMB) customers monitor their electricity usage and costs in real-time, better understand the monthly cost of their electric consumption, and reduce their peak demand and/or conserve electricity usage” (Churchwell et al. 2014). This demand response pilot, which ran from August to November 2014, provided customers with real-time and cumulative information on their electricity usage and related costs as well as advanced notice that a SmartDay event was on the horizon. The pilot aimed to reduce overall electricity consumption in addition to load impacts and to determine whether customers perceived the HAN devices to be a useful tool in managing their electricity use.
The pilot included 1,685 customers, who were recruited from existing participants in PG&E’s TOU and SmartRate efforts. Participating customers were assigned to receive either a Bidgely gateway or an Aztech in-home display (IHD), though some participants early in the pilot were able to select their preferred device (Conley 2015). The Bidgely device makes the relevant electricity use and cost data available via the customer’s phone, tablet, or computer via an app or web portal. In contrast, the Aztech device displays customers’ electricity usage and related cost info directly on its display. While initial satisfaction with the Bidgely gateway was higher, participants reported using the Aztech IHD far more frequently. Although most Bidgely users indicated they looked at the information provided by the device about once per week, nearly half of Aztech users reported looking at it more than once daily. Consequently, Aztech users reported far more electricity reducing actions (Churchwell et al. 2014).
Table 1 below illustrates the different treatment groups into which participants were divided, with the “Group” designation indicating the customers’ electric rate.
For example, there were 277 customers in the E-6 Group, and their average monthly savings were statistically significant at 46 kWh, or 7.7 percent; their on-peak hourly demand savings of 0.01 kWh was not statistically significant (Churchwell et al. 2014). A SmartDay event was a day of high anticipated use during which customers were asked to reduce their use. The E-6 refers to a PGE Residential Time-of-Use Service and EV refers to a Residential Time-of-Use Service for Plug-In Electric Vehicle Customers (see www.pge.com/tariffs/ERS.SHTML#ERS for electric tariff schedules).
Table 1. Treatment Groups for PG&E HAN Pilot
Behavioral Insights. This pilot incorporated a number of behavioral insights. For example, by providing customers with information on their projected monthly electricity bill (Churchwell et al. 2014), this pilot leverages the fact that humans tend to discount the future. By informing customers of their current electricity usage, this pilot also taps into loss aversion, which is the fact that people are generally more concerned with losing something, such as money, that they already have, as opposed to any potential gains (CEE 2010). The ongoing presence of the Aztech IHD also served as a continual reminder about energy usage (Churchwell et al. 2014); unsurprisingly, research indicates that such prompts can help convert behavioral intention into action (CEE 2010).
Evaluation Design and Results. In order to determine the degree to which the pilot met its intended objectives, the evaluation used insights gathered from data collected from smart meters, participant surveys, and focus groups (Churchwell et al. 2014). The evaluation design for this pilot was a quasi-experimental design. Propensity score matching was used to determine the electricity savings achieved by the different treatment and control groups; in this technique, the comparison group is compiled to account for underlying factors that could impact both people’s likelihood to participate in the treatment group and also their tendency to save electricity as a result of participation. Propensity score matching builds a comparison group of people who are similar to the treatment group in characteristics that could impact the pilot’s measured outcome. A differences-in-differences technique was used in the impact evaluation of the pilot. In this approach, the difference in the treatment group’s average electricity use at the beginning and end of the pilot is compared to the differences in average electricity use of the comparison group during that time.
Only one of the participant groups, the E-6 TOU group identified in Table 1, experienced statistically significant overall electricity savings—a savings of 7.7 percent (with a 90 percent confidence interval of 3.8 percent to 11.5 percent)—compared to the comparison group. The results were less promising on the load management side, where a fixed-effects regression model was used to determine any additional benefit on electricity use reductions from the Bidgely or Aztech devices during event days. While there was an average of three percent load reduction across all event days, these results were not statistically significant at the 90 percent confidence level (Churchwell et al. 2014), when comparing the electricity savings from the comparison group. Overall, statistically significant changes to households’ electricity use behaviors—all based on self-report data—included installing power strips to reduce vampire load, washing dishes and laundry in cold water as well as doing fewer loads of both, and turning off lights when not in use.
Key Takeaways. One lesson learned from this effort is that pilot participants indicated a preference for additional customer support from PG&E (Churchwell et al. 2014). Future improvements to the written customer information would likely be beneficial, although the PG&E support provided via phone and email during this most recent pilot phase was a marked improvement over an earlier pilot. Survey results also suggest that additional customer education about the electricity data provided via the HAN could be helpful to participants. For instance, the accuracy of the electricity consumption information provided improved over the course of a given billing cycle, which is unlikely to be intuitive to participants.
Additionally, future iterations of this pilot would benefit from larger participant groups, if only for evaluation purposes. As noted previously, this pilot resulted in promising peak savings, but the sample size was too small for these changes to reach statistical significance. Larger sample sizes might help detect any differences between the treatment and control groups in the future.
A number of caveats are necessary in interpreting these results. All participants in this opt-in pilot volunteered and were among PG&E’s more engaged customers to begin with; as a result, the electricity savings this select group achieved through the pilot may not be representative of the potential success the broader service territory might experience. Additionally, this pilot was in the field for a short period of time. Given that customer interactions with IHDs likely evolve over time—as could electricity savings—the results observed here might differ were the pilot to be implemented for a longer period of time (Churchwell et al. 2014). Finally, self-report data can be inherently prone to bias, and so it is difficult to know the extent to which the self-reported behavior changes reflect actual actions. Due to the social desirability bias, participants will tend to over report the behaviors they know to be targeted by the program.
SMUD’s In-Home Display Check Out Pilot
Pilot Overview. The Sacramento Municipal Utility District (SMUD) offered an In-Home Display (IHD) Check Out Pilot for residential customers from 2012-2013 (Herter and Okuneva 2014). Customers had the opportunity to borrow an IHD from SMUD for two months (customers were responsible for the installation themselves), during which time the IHD interfaced with customers’ electricity meters to provide information on their electricity use and related costs approximately every 15 to 30 seconds (Herter and Okuneva 2014). A total of 1,155 customers in the Sacramento area participated in this research pilot period and were sent an IHD by SMUD (Herter and Okuneva 2014).
Behavioral Insights. This pilot uses similar behavior insights to other efforts that use comparable technology. When customers are provided with information on their current usage, the pilot is leveraging the capacity of feedback to change behavior, along with the fact that feedback is most effective when given soon after the target behavior occurs (CEE 2010). Providing timely cost information also taps into loss aversion. By providing customers with explicit information on the running total of their monthly electricity costs, the device helps make these future expenses more tangible, helping to minimize participants’ tendency to discount the future.
Evaluation Design and Results. The evaluation design for this pilot was quasi- experimental. Due to the nature of the pilot, random assignment was not possible, but a comparison group was created from participants who received their IHDs after the treatment group (delayed recruitment). Given that the comparison group responded to the same offer as the treatment group, just at a later time, the comparison group is likely to be similar to the treatment group in terms of its propensity to save from the IHD (Herter and Okuneva 2014).
The evaluation aimed to determine the electricity savings, peak savings, and customer bill impacts attributable to the program (Herter and Okuneva 2014). In order to properly account for weather in the analyses, customers were assigned to the closest of SMUD’s ten weather stations. The electricity and peak demand results were determined using three-level mixed effects regression models and a difference-in-differences approach (Herter and Okuneva 2014).
Customers’ meter data served as the primary data source for the evaluation.
In the first year after IHD installation and after the first two months, the electricity savings from this pilot averaged approximately 2.6 percent, though there were periods in which savings reached as high as three to four percent (Herter and Okuneva 2014). Peak period impacts were also statistically significant at 3.4 percent, as were customer bill savings at 3.4 percent (Herter and Okuneva 2014). Interestingly, on average, the pilot did not lead to statistically significant electricity savings during the two-month period in which customers’ had the IHDs installed, but statistically significant savings occurred for nearly all the months thereafter (Herter and Okuneva 2014).
Key Takeaways. Perhaps the most compelling takeaway from this pilot is that the timing of electricity savings for these programs can differ significantly from expected. As previously described, there were no statistically significant savings observed during the first two months, but electricity savings were statistically significant for nearly every other month measured (Herter and Okuneva 2014). This may reflect the fact that it took time for customers to learn to use the IHDs and take action in response to the information provided; customers may also have needed the additional time to determine which efficiency measures might help reduce their use and then to install these measures or upgraded appliances (Herter and Okuneva 2014). Regardless of the cause, this trend has potentially interesting implications for persistence, and warrants closer examination.
It may be worth taking a closer look at the usability of IHD devices to determine what role, if any, that may have played in the delayed savings. Only one IHD product was used in this pilot, and usability was not measured or otherwise determined, so it is unclear whether usability might have contributed to—or, instead, detracted from—the achieved savings. Given that a number of different products currently exist on the market, the evaluators suggested usability testing for several options before selecting a specific brand to use in future pilots (Herter and Okuneva 2014).
One challenge of this pilot was the difficulty in obtaining certainty whether an IHD was actually installed. Given that this wasn’t a direct install program, the evaluators assumed that any IHD that had been sent to a customer but hadn’t yet been returned to SMUD was “installed,” but this is probably a generous assumption. Lacking certainty, it is difficult to determine the precise impact of these devices on energy savings. Future iterations of this pilot might benefit from seeking additional information from participants—perhaps surveys or phone interviews that specifically ask about device installation and frequency of use.
Successes, Challenges, and Related Caveats
While this area of program design continues to emerge, the three case studies outlined here provide initial insights into the benefits and challenges of these programs that future programs can learn from. Practices around combining different behavioral approaches, the usability of connected devices, and the benefit of including social science-based behavioral insights emerged as particularly noteworthy.
One promising takeaway from this work is initial evidence that combining connected technologies with other interventions designed to motivate behavior change may be more effective than the sum of its parts. These interactive effects were evident in the Focus on Energy pilot, in which customers participating in time of use rates reduced their electricity usage two
percent and customers only using Tools and Technology reduced their electricity use by four percent, whereas customers participating in both saved nine percent (KEMA 2013). Although the sample size was too small to achieve statistical significance, this potential for an interactive effect from combining behavioral approaches and engaging technologies warrants a closer look.
Another lesson learned from these pilots is the role of usability, satisfaction with, and engagement with the different technologies and, in particular, how it relates (or doesn’t) to electricity savings. Counterintuitively, in the PG&E HAN pilot, customers gave a lower preference rating to the IHD that they ended up using more frequently during the study period. In the SMUD IHD pilot, the evaluators noted that usability was key, and recommended usability testing for different IHD options before selecting one for use in future pilots (Herter and Okuneva 2014). There was somewhat less clarity about customer use of the technology in the case of the SMUD IHD pilot, and seeking additional information from participants with regard to their engagement with these technologies would likely only improve future pilots.
One notable success from these pilots was the benefits accrued from using social science- based behavioral insights. For instance, Focus on Energy set up its TOU rate in one community as opt-out rather than an opt-in, thereby dramatically increasing the number of customers who participated in TOU rates (KEMA 2013). This finding is supported by the default bias, also known as the status quo bias, which has long demonstrated that people will tend not to change from the default option (CEE 2010). While setting programs up as opt-out can entail logistical challenges, these pilots provide additional evidence that program participation numbers benefit substantially. A similar success that was also grounded in behavioral insights was that PG&E leveraged social norms by framing information relative to other customers’ use—for example, illustrating that a participant’s refrigerator was using more electricity than the average household’s (Churchwell et al. 2014).
One challenge implicit in a discussion of these types of programs is that applying behavioral approaches and connected technologies to energy efficiency programs is still a nascent area. As a result, there are relatively few programs for which evaluations have been completed. The pilots and programs described here may not be representative of the efforts underway more broadly. While these examples shed light on the variety of approaches currently being explored, it is important to exercise caution in drawing any broader conclusions. In addition to the variations in approach, differences in geography and target audience likely impacted the results for each pilot as well. Thus, what worked well for one of these variations in one context may not work as well in another area with a demographically distinct target audience. The commonalities and lessons learned across these efforts are best understood within the context of these caveats.
Potential Areas for Future Research
Beyond providing initial illustrations of the opportunities offered by programs combining connected technologies and behavioral insights, these three pilots also shed light on areas of program development that would benefit from further research. These include issues of technology selection, the attitude-behavior gap, persistence, and cost-effectiveness.
In the PG&E HAN pilot, customers indicated higher levels of satisfaction with one of the two technologies (the Bidgely), while reporting more frequent use of the electricity information provided via the other (the Aztech IHD). This disparity warrants further attention. There has been ongoing research into usability as it relates to resulting electricity savings from different devices, and this is one example of how further study might be helpful. Why did customers look less frequently at a device that was more appealing? Was it technical factors, such as the fact that the IHD was inherently always on, or design features, or something else that led to this disparity? Additionally, several of these pilots measured electricity savings quantitatively and also used survey data to assess any changes in knowledge and attitudes. Yet research has generally demonstrated inconsistency between individuals’ reported attitudes and behavior—known as the “Attitude-Knowledge-Behavior Gap” (CEE 2010). Future research taking a more granular look at how individuals’ self-reported changes in knowledge, attitudes, and even behavioral intentions align, or do not align, with objectively observed changes in electricity consumption would lend great value to future program development. A more up-close look at this issue might help shed light on which specific messages or framing delivered through devices such as in-home displays and smart phone apps are most effective at changing knowledge, attitudes, and behaviors.
Further research is also needed to understand the extent to which the observed electricity savings might persist over time, both while these technologies are still in place and also after customers are no longer using them. These pilots were conducted for relatively short periods of time. Persistence is a huge consideration with many types of behavior-based efficiency programs, and those employing connected technologies are no different.
Finally, none of these three evaluation studies examined the cost-effectiveness of these pilots. Connected technologies add to existing program costs, though the benefit in terms of electricity and related cost savings may outweigh these additional upfront expenses. Yet the only way to determine this is by performing cost-effectiveness calculations for these types of programs. These analyses would be particularly useful if they were conducted for several different studies using a variety of technologies, given that cost-effectiveness may vary depending on the specific technology used.
Concluding Thoughts
Programs that incorporate both connected technologies and behavioral insights represent a relatively new program area and are generally still in the early stages. Currently, a number of different utilities—including Duke Energy, Baltimore Gas and Electric, and San Diego Gas and Electric—have pilots and programs underway that similarly integrate both connected technologies and behavior insights into efficiency programs. Evaluations of these ongoing efforts will continue to advance our collective knowledge of how to maximize savings through new technologies and behavioral approaches.
In the meantime, the three pilots presented here have demonstrated the potential of these efforts, and provide important insight into the value of incorporating multiple program approaches, the benefit of opt-out approaches over opt-in, and the necessity of larger sample sizes to more effectively detect smaller program impacts. Similarly, these pilots have together helped illustrate the potential importance of factors such as device usability and how programs can leverage behavioral insights in newly effective ways on a community-wide scale. This early work also raises key questions about the cost-effectiveness and persistence of connected behavior programs, and helps reveal areas that require further research. But perhaps most importantly, these pilots serve as a crucial starting point for future work.
Table 2. Pilots and Programs Incorporated Connected Technology and Behavioral Approaches
Reference List
Churchwell, C., Sullivan, M., Thompson, D., Oh, J. 2014. HAN Phase 3 Impact and Process Evaluation Report. Nexant.
Conley, K. (Pacific Gas and Electric.) 2015. Personal communication. May 7.
Consortium for Energy Efficiency. State of the Efficiency Program Industry: Budgets, Expenditures, and Impacts 2014. http://www.cee1.org/annual-industry-reports, posted May 2015. © 2015 Consortium for Energy Efficiency. All rights reserved.
Consortium for Energy Efficiency. 2010. Behavior Insights and Tools. Boston, MA.
KEMA, iCanConserve Final Report. 2013. Prepared for Wisconsin Public Service.
Herter, K., and Y. Okuneva. 2014. SMUD’s IHD Checkout Pilot – Load Impact Evaluation. Prepared for the Sacramento Municipal Utility District, Sacramento, CA.