Sign In | Sign Up


WP7 - Evaluation


To evaluate the PEBBLE system regarding energetic, GHG emission, economic and comfort performance.


The objective of WP7 is the evaluation of the PEBBLE system and its impact on the EPB’s performance in terms of technical, operational, and economic aspects. Hereby, the evaluation will be based on both quantitative performance indices like primary energy savings and demand reductions from external sources and qualitative features like user comfort and satisfaction. In order to evaluate the PEBBLE performance, an extensive data acquisition system has to be installed at all three buildings, which complements the data acquisition system of the BO&C control system. Quali-tative features are assessed both on data gathered and their evaluation as well as user information. The latter can be determined through the electronic questionnaires available on the building’s intranet and the evaluation of the user interaction with the BO&C control system. The principal evaluation will be carried out by the WP7 leader in collabora-tion with the system developers and the site leaders. The resources allocated to WP7 are sufficient to ensure a seri-ous, comprehensive, and conclusive evaluation. Special emphasis will be given in the development of the evaluation plan as well as during the realization of the evaluation procedure in order for:

  • The evaluation of the “no-control” case and the PEBBLE system – which will be alternatively applied during the demonstration phase on a weekly basis – produces comparable results (i.e. the two systems’ perform-ance is evaluated at comparable weather and occupancy conditions).
  • The user-acceptance and comfort levels are defined in accordance to European standards.

The findings of WP7 will provide a major input for the dissemination and exploitation activities of the project. WP7 will be structured into 3 tasks as follows.

Evaluation Planning

The first step in this task is the preparation of the plan for the demonstration and evaluation phase. The Evaluation Plan will prescribe the general evaluation approach, the assessment objectives, the indicators to be used, the meth-ods and timing of measurement and the measurement conditions, and statistical issues such as sample sizes. The core of the evaluation plan is an ‘Indicator Table’ for each demonstration building, which contains for each indicator

  • The name and number of the indicator (e.g. EP, GCEI, TC1, TC2, PPP,GHG emissions, Life-cycle costs, etc);
  • The maximum and minimum allowable values of the indicator;
  • The European standards related to the particular indicator
  • The data required to measure the indicator;
  • The data sources (e.g. which sensors and devices to be used, the particular time-intervals to be used within the evaluation, etc) and
  • The data analysis (formulas, statistics tests).

The Evaluation Plan will be prepared in cooperation with the local system demonstration managers. For all three demonstration buildings, it has to be ensured that the same conditions are reached for the evaluation in all sites, how-ever, the differences in the building design, energy generation and building use have to be addressed. Establishing this Evaluation Plan is an iterative process, where the WP7 leader will stipulate what would be desirable and discuss, and agree with the site leaders point by point what is feasible, what is not possible and what alternative solutions are available. Therefore, the plan will be issued first as a draft version as a basis for further discussion within the project and with the project officer, and later as a consolidated final plan. In a second step, the Evaluation Plan will be com-plimented by the definition of questionnaires gathering information about user comfort and satisfaction as well as the acceptance of the different user interfaces. Once the Evaluation Plan is established and agreed, the WP7 leader will prepare the Outline Evaluation Report. This obviously includes the definition of the overall structure of the report, the breakdown into chapters and sections, and an agreed assignment which sections will eventually be filled by the partners in the sites and which by the WP7 lead-er. But, crucially, it will also contain mock-ups of all tables and figures that are to be produced by the sites to list and visualize the evaluation results. Experience in other projects has shown that this

  • facilitates the communication between the WP7 leader and the sites,
  • reduces the risk of misunderstandings about the rules for data analysis,
  • greatly improves the readability of the overall report by an outsider due to the homogeneity of the presentation.

Finally, depending on the Evaluation Plan, additional sensors and monitoring devices have to be installed at each of the three systems

Evaluation Per Demonstration Building

The first part of the evaluation will focus on the impact assessment based on the performance of the BO&C systems during the demonstration phase (and, mainly, in comparison with the no-control case performance). Indicators to be used in the evaluation relate to NEP, NEB, user-comfort, and cost-efficiency of the installed sensor, control and user-interfacing devices. From NEP and NEB it is then possible to make calculations that provide an estimate of the effect of the impact in emissions (CO, CO2, NOx, HC, SO2 and particulates). The second part of the evaluation is the anal-ysis of the users’ acceptance. Users in this case are both the building operators and the end-users (building occu-pants). To gain the views of the both types of users, the following sources will be used:

  • inputs from user interfaces as they are logged in the data base created during demonstration,
  • detailed interviews and questionnaires will be conducted with both types of users.

The prime focus in the interviews and questionnaires will be the effect of the EPB operation in their everyday life and energy savings, but attention will also be paid to the users’ personal judgment of the effect of the automated EPB sys-tem operation as well as their views on the user-interfaces inputs. The final part of the site evaluation is a socio-economic analysis. The elements that are used for the calculation of benefits are:

  • energy savings;
  • reduction of energy demand from external (non-renewable) sources;
  • Savings in GHG emissions; and
  • Life-cycle assessments.

All of these will be calculated on the basis of the German guidelines for socio-economic evaluation (EWS). On the cost side, there are normally the costs for the system purchase and installation, costs of the system operation as well as the costs for system maintenance. However, in the context of the PEBBLE project the expected savings in energy costs are expected to be significant as compared to the three first items; therefore the only cost to be held against the expected benefits is the initial implementation of the PEBBLE systems.

Conclusions and Transferability of Results (This task will be led by ARM)

  • In the last step of the evaluation, the transferability of the PEBBLE system will be investigated. This covers a number of aspects, including in particular:
  • the type and complexity of the building system,
  • the building use,
  • the type and variety of energy-influencing components involved
  • the weather and human-influencing conditions.
  • The coupling of the PEBBLE system with an energy audit decision support methodology where other energy efficiency measures are investigated [Diakaki2008].

From the evaluation results, recommendations and guidelines will be generated in order to accelerate the implemen-tation of EPBs. The focus of these recommendations will be on the potential of the new BO&C system on energy use. Therefore, the recommendations will be valid for new installations as well as retrofitting applications, where existing building can be converted into PEBBLE buildings.

© 2010-18 PEBBLE Consortium Partners. All rights reserved.  W3C Valid W3C Valid
Summary Impact Technical Approach Work Packages Project Info
News Events
Scientific Papers Conference Publications Dissemination Materials Deliverables Others