Global Fund Board paves the way for the transition to a new independent evaluation function that replaces the TERG
At its 46th meeting held virtually on 8-10 November 2021, the Global Fund Board has approved the creation of the Independent Evaluation Panel (IEP), which is one of the structures in the new independent evaluation function. This new evaluation function will replace the Technical Evaluation Reference Group (TERG), as the Global Fund has approved TERG’s dissolution following a transition period ending 31 December 2022. To facilitate this, the Board has instructed the Strategy Committee to develop and oversee transitional arrangements between the IEP and the TERG.
The Board has also approved revisions to core governance documents required to implement the new independent evaluation model. The revised documents include the Global Fund Bylaws and the Charter of the Strategy Committee (SC). It also approved Terms of Reference (TORs) for the new Independent Evaluation Panel (IEP). These revisions were necessary as the new model requires changes to the structure of how independent evaluation is managed and how oversight of the quality and independence of the evaluation function is assured.
In this decision point, the Board requested the Secretariat to update any policies, relevant governance documents, charters, TORs, agreements, or prior decisions needed to align with the above documents. It further requested the SC, in fulfilling its oversight role over the independent function, to consider the effectiveness of the guidance and procedures developed to safeguard both the independence and the learning components of the evaluation function.
This article provides a summary of the paper submitted to the Board and the decision point ratified by the Board (the full Board Decision is available at the bottom of the article). It also highlights amendments to the Board paper and feedback from the constituencies.
New independent evaluation function approved after extensive discussions
During the 15th SC held in March, an independent consultant proposed three models for a new independent evaluation function. Model 1 embedded the evaluation function in the Secretariat with limited independent oversight, Model 2 embedded the evaluation function in the Secretariat with significant independent oversight, and Model 3 proposed a structurally independent evaluation function independent from the Secretariat. Since then, the SC, the M&E Working Group, TERG, and Secretariat have extensively discussed the different models and assessed them against their potential to address current weaknesses. The SC finally approved Model 2 at its 17th meeting held in October 2021.
The new independent evaluation function will cover evaluations such as strategic evaluations (such as the Strategic Reviews), thematic and programmatic evaluations, and country program evaluations on a case-by-case basis. It will draw on learning, best practice, and innovations in evaluation methodology, evaluation management, and the use of evaluations to promote learning within the Global Fund.
Roles and responsibilities spread out across different bodies
The new model for the independent evaluation function is a shared responsibility between the Global Fund Governance bodies and the Secretariat. Independent evaluation is delivered by two structures: 1) an Evaluation Unit, which is situated in the Office of the Executive Director (OED) headed by the Chief Evaluation and Learning Officer; and 2) an Independent Evaluation Panel (the IEP), an advisory group, independent from the Secretariat and accountable to the Board through the SC.
Figure 1: Structure of the new independent evaluation model
Evaluation Unit to perform day-to-day evaluation activities
The Evaluation Unit will execute all Global Fund independent evaluations according to a multi-year evaluation calendar approved by the Board through the SC and regularly engage with stakeholders across the Secretariat, with the SC and IEP, and with global and country-level actors. It will also offer administrative support to the IEP to fulfill its effective functioning.
The unit lead, CELO, will be accountable for the effective operation of the Evaluation Unit under the oversight of the OED and the IEP. The SC, IEP, and OED will take part in the recruitment of the CELO. However, as the IEP has yet to be established, the SC representative on the TERG and the current TERG chair will be responsible for the first CELO recruitment. The Secretariat has already prepared the job description for the CELO, which the SC reviewed during the October meeting. The SC will review any future changes to the job description.
IEP to replace the TERG with revised terms of reference
The IEP will be composed of renowned technical experts in the field of evaluation. It will provide quality and independent assurance to the Board over the Global Fund’s independent evaluation activities. However, unlike its predecessor TERG, the IEP will not be involved in the day-to-day management of evaluations, as a way of preserving its independence over the evaluation process. The IEP will become involved upon completion of an evaluation, where it will provide an assessment of quality and independent conduct of the evaluation and an implication analysis on the findings and recommendations including key areas of policy, process, and implementation that require specific attention of the SC or the Board or both. The commentary will be published alongside the evaluation.
In its advisory role, the IEP will recommend improvements to evaluation methodologies, procedures, quality-assurance mechanisms, and safeguards for independence; and recommend innovative ways to strengthen conduct and use of evaluations, including improvements to knowledge-sharing and dissemination. In an amendment to the Board paper, the IEP will provide an annual report to the Board highlighting key learnings and observations from the evaluations. The IEP can also use the annual report to escalate any concerns that may arise on the performance of the Evaluation Unit or on its independence.
The IEP TOR is among the documents approved by the Board.
Revised SC function requires changes to governance documents
The SC is responsible for appointing the IEP members and its Chair, with the support of the Secretariat. It will also recommend the Board’s approval of the multi-year evaluation calendar, following its development by the CELO and endorsement by IEP. The SC will approve the annual work plan for evaluations and recommend the budget associated with the work plan as part of the regular operating expenses (OPEX) process. The SC will have a representative on the IEP as an ex-officio member.
The revised SC Charter and the Global Fund Bylaws reflect the SC’s new responsibilities.
Executive Director to ensure the Evaluation Unit is well-resourced
The OED will be accountable for safeguarding the integrity and independence of the evaluation function within the Secretariat. It will ensure that the Secretariat’s Evaluation Unit is sufficiently resourced and mandated to fulfill its role. The OED will ensure the development and implementation of the management responses derived from evaluation findings and recommendations.
The new model addresses evaluation weaknesses
The new independent evaluation model seeks to address the weaknesses and challenges within the current independent evaluation function, particularly related to fragmentation, relevance, and learning, as identified by the independent assessment of TERG in 2019 and a review of M&E functions in 2020. It was the preferred option among other models because:
- By placing the Evaluation Unit within the OED, the new model will ensure learning is driven from the top. The current independent evaluation function has failed to foster an organizational culture of learning. The model will also enhance the use of evaluation findings by decision-makers, including, for example, communication tools tailored to specific audiences and their specific needs.
- The new model will also address the fragmentation issues and mitigate duplication. The current evaluation function lacks clarity on which evaluations should be led by the Secretariat and the TERG. In this new model, the Evaluation Unit will serve as a one-stop for all independent evaluations.
- The new model will improve the accountability of evaluation findings. An embedded Evaluation Unit in the Secretariat will enable a stronger link and insight to the operations and day-to-day business of the Global Fund to facilitate greater relevance and timing of evaluation findings.
- Despite not being structurally independent, the model includes a series of safeguards to protect different aspects of independence. The Evaluation Unit will be independent of the operational, policy-making, management, and decision-making functions in the Secretariat.
The new model faces potential risks to its independence
The Board paper outlined several risks to functional and behavioral independence and the safeguards put in place to mitigate these risks at the various evaluation stages. The Global Fund has introduced several safeguards for independence. Organizational independence will be enhanced by the dual reporting line of the CELO (to the OED and the IEP), the oversight role of the IEP, and a planned review of the Evaluation Function by the Office of the Inspector General (OIG) and the IEP by not later than 31 December 2023.
The various bodies will promote functional independence at the different evaluation stages. Examples include Board approval of the multi-year evaluation calendar via the SC at the planning phase or recruiting independent consultants or firms to conduct evaluations at the commissioning phase. At the implementation phase, the IEP will review the TORs, inception reports, and evaluation reports with a focus on independent conduct of the evaluation. The IEP will provide position papers on evaluations alongside an annual report to the Board, during the findings and response phase. At the last phase, dissemination and learning, the IEP will monitor the publication of the evaluation reports.
CELO recruitment and IEP establishment are the next key steps
After this Board approval, the Global Fund will proceed with the recruitment of the CELO and, with the current TERG leadership, initiate transition planning from the TERG to the new IEP. Once the CELO is recruited and the IEP established, the Secretariat will develop standard operating procedures (SOPs) to clarify the role of each stakeholder for different evaluation-related procedures. The SC will revise its TERG Document Procedures (GF/SC05/07) before the completion of the first evaluation report overseen by the IEP, in collaboration with the Secretariat, to correspond with the new evaluation model.
The Board paper underscores the need for timely CELO recruitment and establishment of the IEP to ensure operational aspects, including SOPs are ready going into 2023 with a multi-year evaluation calendar for the next Strategy period. TERG will also need to finalize all its reviews and evaluations by end-2022.
Feedback from the constituencies
The constituencies expressed their immense support for the new independent evaluation function. However, the constituencies expressed concern on the issue of safeguarding the independence of the new evaluation function. They underscored the need to protect independence during the operationalization of the model, particularly during the development of the SOPs. They called for appropriate oversight by the Board and its Committees during this stage. In addition, the constituencies welcomed the planned 2023 OIG review, which will assess the adequacy and effectiveness of the Evaluation Function and the IEP.
The constituencies also highlighted their concerns around promoting learning from the evaluation findings. They sought clarification on how the evaluation recommendations will be developed and how binding they will be. The constituencies also pushed for a more defined process for the Board to track learning and the implementation of recommendations either as agreed management actions, such as those applied by the OIG or a similar mechanism.
The constituencies also pushed for more involvement and engagement of the in-country partners, including Country Coordinating Mechanisms (CCMs) and communities. They called for a more strategic and selective approach in determining future evaluation priorities. The various bodies need to promote the contribution of in-country stakeholders in identifying evaluation priorities. Finally, they also called for the Secretariat to promptly make the evaluation reports public to improve knowledge sharing, learning, and dissemination of good practices.
Decision Point GF/B46/DP06 Independent Evaluation Function
The Board notes the recommendation of the Strategy Committee and of the Ethics and Governance Committee, as set forth in GF/B46/05 rev1, with respect to the new independent evaluation model which shall replace the Technical Evaluation Reference Group ("TERG") and decides to approve the creation of the Independent Evaluation Panel, as described in GF/B46/05 rev1.
Budgetary implications (included in, or additional to, OPEX budget).
The Board Document GF/B46/05 rev1, Independent Evaluation Function and the Decision Point GF/B46/DP06, should be available shortly at https://www.theglobalfund.org/en/board/meetings/46