Performance indicators for the evaluation of business plans

The Twenty-fifth Meeting of the Executive Committee decided to request the Secretariat, in collaboration with the Implementing Agencies:

(a) to make proposals on the appropriate indicators for the evaluation of business plan performance;

(c) to take into account the importance of the indicators relating to ODP reduction, speed of delivery and distribution of projects among countries when making proposals;

(d) to consider the need to revise the cost of project preparation and cost-effectiveness performance indicators;

(e) to apply any new or weighted indicators to evaluation of 1998 business plans.

The Twenty-sixth Meeting of the Executive Committee decided to maintain two sets of indicators for investment projects, as follows:

(a) the first to track various aspects of the work of agencies, including the previous indicators and a new indicator covering the net emission/reduction of ODP resulting from implementation delays/early completion (expressed in ODP tonnes);

(b) the second set to be weighted indicators, to guide the allocation of agencies’ shares. The indicators would assess performance in relation to what had been planned and would be weighted on a pro rata basis as shown in the following table:

Investment project performance indicators	Weighting
Actual ODS phased out from completed projects (ODP tonnes) – the amount of ODS eliminated from completed projects within the plan period	40
Disbursement (US$) – the amount of funds to be disbursed (excluding unliquidated obligations) within the plan period per tables 1 and 4 of the business plans	30
Satisfactory project completion reports received (percentage) – the number of project completion reports submitted expressed as a percentage of the number of project completion reports due for submission.	20
Distribution of projects among countries in business plans – the number of countries for which activities are planned for approval in the year of the business plan per table 5 of the business plans	10
Total	100

For non-investment projects, the Executive Committee decided to maintain two sets of indicators, as follows:

(a) the first set to track the results associated with non-investment projects for all Implementing Agencies. The first set would consist of four indicators which would assess performance in relation to what had been planned and would be weighted on a pro rata basis as shown in the following table:

Non-investment project performance indicators	Weighting
Number of non-investment projects completed – the number of training, technical assistance and country programme preparation projects completed within the plan period.	50
Disbursement (expressed in US$ million) – the amount of funds disbursed within the planned period for training, technical assistance, country programme preparation, and institutional strengthening projects.	30
Speed of first disbursement (expressed in number of months) – the average number of months between the date of project approval and the date of first disbursement for training, technical assistance, country programme preparation, and institutional strengthening projects by year of approval.	10
Speed of project completion (expressed in number of months) – the average number of months between the date of project approval and the date of project completion for training, technical assistance, and country programme preparation projects by year of approval.	10
Total	100

(i) appropriate and timely policies initiated by countries either as a result of networking, training, information exchange, country programme development and/or institutional strengthening ;

(ii) reduction in ODS consumption over and above that effected by investment projects.

The Twenty-sixth Meeting of the Executive Committee requested UNEP, in view of its specific mandate, to continue the monitoring of its activities according to the set of nine indicators set out in its business plan.

(b) to request all Implementing Agencies, when finalizing their 2002 business plans, to apply the Secretariat’s advice contained in document UNEP/OzL.Pro/ExCom/35/16 relating to the identification of targets for the non-weighted, non-investment project indicators: (1) appropriate and timely policies initiated by countries either as a result of networking, training, information exchange, country programme development and/or institutional strengthening ; and (2) the reduction in ODS consumption over and above that effected by investment projects;

(c) to consider revising those indicators unique to UNEP’s specific mandate (Decision 26/6) on the basis of the outputs identified by UNEP in Annex 1 to its draft 2002 business plan (UNEP/OzL.Pro/ExCom/35/8 and Add.1) for the Compliance Assistance Programme (CAP) and to request UNEP to work with the Secretariat in revising Annex I;

(d) to request all Implementing Agencies to include a target for the performance indicator “timely submission of progress reports” in their final 2002 business plans;

(e) also to request all Implementing Agencies to include a new non-weighted, investment project performance indicator for project completion pursuant to Decision 28/2 by setting a target for the number of investment projects to be completed in the year of the business plan;

(f) further to request Implementing Agencies to consider revising their targets for their 2002 business plan cost-effectiveness performance indicators in the light of the fact that, historically, cost-effectiveness values achieved by projects had been lower than those targeted by the agencies.

The Thirty-sixth Meeting of the Executive Committee decided to request the Sub-Committee on Monitoring, Evaluation and Finance to consider the possibility of a new performance indicator for the timely financial completion of projects at its 17th Meeting.

(a) to request the Secretariat in co-operation with the Implementing Agencies and Article 5 countries to prepare a set of performance indicators for the compliance period in the light of the three-year phase-out plan and taking into consideration the discussion during the meeting on performance indicators;

(b) to invite Parties which had views on the topic to submit them to the Secretariat within four weeks of the end of the 38th Meeting.

(a) to take note of the document on performance indicators presented in UNEP/OzL.Pro/ExCom/40/21;

(b) to request the Implementing Agencies and to invite Parties to submit proposals for performance indicators to the Secretariat eight weeks prior to the 41st Meeting of the Executive Committee;

(c) also to request the Secretariat to prepare a paper based on the concepts outlined in UNEP/OzL.Pro/ExCom/40/21, comments provided by members of the Executive Committee during the 40th Meeting, and the input of the Implementing Agencies; and

(d) further to request the Implementing Agencies to explore the feasibility of including the rapidity of financial completion of projects as an indicator.

(a) to take note of document UNEP/OzL.Pro/ExCom/41/80 and the definitions of performance indicators presented therein;

(b) to approve the following performance indicators for the evaluation of the performance of the Implementing Agencies, starting with the year 2004, with the weightings indicated in the following table and subject to review from time to time:

Category	Item	Weighting
Approval	Number of annual programmes of multi-year agreements approved vs. those planned, may not be applicable to UNEP	20
Approval	Number of individual projects/activities (investment projects, RMPs, halon banks, TAS) approved vs. those planned	20
Implementation	Milestone activities completed (e.g. policy measures, regulatory assistance)/ODS levels achieved for approved multi-year annual tranches vs. those planned, may not be applicable to UNEP	20
Implementation*	ODP phased-out for individual projects vs. those planned per progress reports, may not be applicable to UNEP	5
Implementation*	Project completion (pursuant to Decision 28/2 for investment projects) and as defined for non-investment projects vs. those planned in progress reports	5
Implementation	Percentage of policy/regulatory assistance completed vs. that planned	10
Administrative	Speed of financial completion vs. that required per progress report completion dates	10
Administrative*	Timely submission of project completion reports vs. those agreed	5
Administrative*	Timely submission of progress reports and responses unless otherwise agreed	5

(c) to request the Secretariat to continue to monitor the following indicators of performance on the basis of trend analysis in future evaluations of the performance of Implementing Agencies: value of projects approved, ODP to be phased out, cost of project preparation, cost-effectiveness, distribution among countries, funds disbursed, speed of first disbursement, speed of completion, and net emission due to delays;

(d) also to request Implementing Agencies to propose qualitative performance indicators as part of their 2004 business plans to be submitted to the 42nd Meeting and to continue to consider the possibility of performance indicators for National Ozone Units; and

(e) to adopt the following performance indicators for UNEP’s Compliance Assistance Programme (CAP) to replace the former performance indicators unique to UNEP’s specific mandate:

Indicator	Data	Assessment
Usefulness of the region network/thematic meetings	Average of all ratings by network meeting participants	Average versus maximum rating
Assistance to countries for data reporting for the purpose of establishing baselines	Number of countries with missing data that received specific CAP assistance for data reporting in order to determine compliance baselines. UNEP should give a clear indication of the assistance provided per Decision 40/11 (b)(i)	Proportion of countries receiving assistance that subsequently provided data pursuant to Article 7
Assistance with Article 7 data reporting	Number of countries reporting data in compliance with Article 7 of the Protocol pursuant to Decision 40/11 (b)(ii)	Proportion of countries accepting assistance that achieved compliance
Countries in actual or potential non-compliance as per MOP decisions	Number of countries offered assistance under the CAP and number of countries that accepted assistance (after consultations with other implementing and bilateral agencies) and received CAP special assistance outside of network meetings listing the countries and type of assistance	Proportion of countries accepting assistance that achieved compliance
Countries at risk of becoming in non-compliance as per Article 7 data trends	Number of countries offered assistance under the CAP including those countries not yet officially declared in non-compliance but where data indicate as much and number of countries that accepted the offered assistance (after consultations with other implementing and bilateral agencies) and received CAP special assistance outside of network meetings, listing the countries and type of assistance	Number of countries provided special assistance outside of network meetings as a proportion of the number of countries that appeared to be in non-compliance
Information clearing-house	Number of newsletter subscriptions Number of discrete visitors (IP addresses) to the web site Number of PDF documents downloaded from web site Number of hard-copy publications disseminated Number of query responses answered; average time for response	Comparison with previous years’ data