Afghanistan: Evaluation Expert, Youth and Adult Literacy

Job Title: Evaluation Expert, Youth and Adult Literacy

Job Level: Senior level consultant, P-5

Contract Name: Final Evaluation of the Programme for the Enhancement of Literacy in Afghanistan (ELA3)

Duty Station: Kabul, Afghanistan

Organization: UNESCO/Kabul Office

Contract Duration : 8-10 weeks

Announcing Date : 14 June 2018

Closing Date: 30^thJune 2018 (midnight, Kabul time)

Proposed starting date Mid-July 2018

Background

United Nation Educational, Scientific and Cultural Organization (UNESCO) strives to promote education in Afghanistan as a fundamental human right while aiming to provide access to quality, relevant and inclusive education for all. Across Afghanistan, UNESCO Kabul is currently providing technical and financial support to the Ministry of Education’s Literacy Department (DMoEL) to implement Basic General Literacy (BGL) and Skills Based Literacy (SBL) programmes, and to strengthen the quality of adult literacy and non-formal education more broadly. The Programme for Enhancement of Literacy in Afghanistan (ELA) is a national programme that aims to improve the proficiency level of literacy and numeracy skills of the youth and adult population, particularly for women and girls in Afghanistan. The principal goal of the ELA Programme is to increase awareness about the importance of literacy among national-level stakeholders and to address the challenges of literacy in a sustainable and proactive manner, noting that the current budget allocated to literacy under the MoE is roughly only 2% of the national education budget.

In addition to literacy provision, ELA provides technical support to the DMoEL in capacity building and institutional reform, programme development and implementation, resource mobilization, national social mobilization, advocacy and outreach, curriculum development, teacher training, monitoring and evaluation, management information system (MIS), and progress reporting. It also puts high emphasis on supporting the Literacy Department in establishing a decentralized modality of implementation that connects it with provinces, districts and communities in all phases of planning, implementation, monitoring and reporting. UNESCO seeks to further strengthen the institutional and organizational capacity of the DMoEL so that its relevant departments are able to gain the capacity to sustain this important decentralized modality of literacy provision following ELA termination.

The first phase of the ELA Programme (2008 – 2010), funded by the Government of Japan, provided literacy courses to 255,438 youth and adults in 50 districts in 9 provinces[1] of Afghanistan. With additional funding from the Government of Japan in 2010-2013 the outreach of the ELA Programme (Phase II) was expanded to 99 districts of 18 provinces[2], targeting a total of 325,662 youth and adults, of whom 60 percent were women.

Building on the lessons learned from the previous phases, ELA Programme Phase III started in November 2013 and ended in March 2018. The total funding for ELA phases I, II and III was USD15 million, USD19 million and USD34 million respectively. Notably, funding for the ELA3 was further increased by support from the Government of Sweden (December 2014) and Finland (July 2015), resulting in the expansion of the geographical coverage to 30 provinces. In June 2016, the Government of Finland committed additional support to ELA3 that allowed for expansion to the remaining four Afghan provinces and effectively increasing ELA3’s total funding portfolio to approximately $32 million over four years. It realized a total coverage of 155 districts across all the 34 provinces[3], and reached out to 634,000 youth and adults. So far, ELA Programme remains the largest national literacy programme in Afghanistan.

Substantively, ELA3 also prepared a Youth and Basic Adult Education Curriculum Framework and instructional materials for Level One (equivalent to grade 1 -3 of formal education); and it began plans to develop equivalent Non-Formal Education Curriculum Framework and instructional material for Levels 2 and 3 (equivalent to grade 4 - 9). Through ELA3, UNESCO has continued to support the DMoEL to develop Skills Based Literacy (SBL) workbooks to complement basic literacy courses for those learners who may opt to further continue their education in eight different skill areas^[4].

Objective of the ELA3 Final Evaluation

The overall purpose of the final evaluation is to assess the extent to which ELA3, as the latest phase of the ELA programme, has or has not been successful in meeting its objectives. In this, the relevance, effectiveness, efficiency, impact, sustainability, and gender perspective of ELA-3 will be measured. The organizational capacity of ELA3 managers in DMoEL and UNESCO will be taken into account with respect to (but not limited to) programme design, planning, implementation and monitoring -- including data management, progress reporting, provision of feedback and the timeliness and appropriateness of management decisions.

The evaluation will assess these and other factors influencing ELA3 operations and results at both national and subnational levels (provincial, district, community). This will include the perspectives of ELA3 policy-makers, programme developers or monitors, managers and, especially, beneficiaries (learners and facilitators) with respect to (i) what they expected from the ELA programme; (ii) their satisfaction with ELA in terms of programme design, implementation and management of risks/challenges; and (iii) their assessment of “what difference” the ELA3 programme has made (e.g. what has changed for them as policy-maker, programme developer or monitor, facilitator, community member - and female or male).

Issues related to strengths, weaknesses, opportunities and threats (SWOT) from the perspective of the DMoEL and UNESCO in managing the ELA3 delivery mechanisms, and of the three donors, should be clearly identified and addressed. This should include an assessment of the nature, quality and long-term implications of the partnership between DMoEL and UNESCO with respect to providing technical assistance support, to the design/management of the Implementing Partnership Agreement (IPA) and, more broadly, the levels and types of capacity that were required by – and provided to - the Department to independently design and implement a national literacy programme of the scope of ELA.

Scope of Evaluation

The time period that will be covered by this evaluation is from the initiation of ELA 3, in November 2014, to its’ conclusion in September 2018. The three DP/donor project proposals of ELA3 were consolidated in 2016 with the agreement of Japan, Sweden and Finland. This included their respective targets and expected results. The final evaluation of ELA3 shall, therefore, assess the ELA3 as a consolidated programme with respect to the following outcomes:

• OUTCOME 1 (Improved Quality and Relevance): Promoting diversification of literacy curriculum, teaching-learning materials, learning contents based on various needs and prior learning of youth, adults and other socially vulnerable groups;

• OUTCOME 2 (Learner Assessment)**:** Learner assessment system developed for quality learning under the programme;

• OUTCOME 3 (Literacy Provision/ Improved Equitable Access): Opportunities for continuing education and lifelong learning expanded among youth and adults over fifteen years of age, especially for vulnerable groups and women. [Or] Improved equitable access to literacy among youth and adults over 15 years of age, especially women and other vulnerable groups;

• OUTCOME 4 (Capacity Development/ Strengthening Self-reliance): Enhance the sustainability of the programme through strengthening national capacity in cross-ministerial, national/ sub-national level coordination and management for literacy education;

• OUTCOME 5 (Monitoring and Evaluation): Monitoring and evaluation capacity improved in the Literacy Department through a well-functioning Non Formal Education Management System (NFEMIS); M&E used for programme performance measurement with reliable and regular data collection;

• OUTCOME 6 (Networking and Coordination): Importance of adult literacy and non-formal education is mainstreamed within the general education framework through coordinated and harmonized efforts and advocacy; and

• OUTCOME 7 (Advocacy and Policy Inputs): Increased demand for literacy learning opportunities.

Main Responsibilities of the international consultant

Under the overall authority of the Director of UNESCO Kabul, the general direction of the Chief of Education, and the direct supervision of the Monitoring and Evaluation Specialist for ELA, the consultant will conduct the final evaluation of ELA3, providing a detailed evidence-based analysis and assessment of its major achievements, constraints and lessons learned; and, based explicitly on this evidence, recommend directions for future programming as applicable to both UNESCO and the DMoEL.

Specifically, the international consultant will perform following tasks:

1. Lead the evaluation process, including the evaluation questions, the evaluation design, data collection tools, data collection activity, collation and analysis of data, results, report writing including recommendations
2. Work collaboratively with national staff members in data collection and analysis;
3. Conduct meetings with key stakeholders to design the evaluation;
4. Conduct fieldwork (KII, FGD, observation) for primary data collection from provider and beneficiary stakeholders;
5. Conduct a knowledge sharing workshop with key literacy stakeholders to confirm, elaborate and/or correct preliminary findings;
6. Prepare a draft report on evaluation and incorporate comments and feedback from the stakeholders in final report; and
7. Submit final report of final evaluation of ELA3.

Methodology

The evaluation approach and method must ensure the most reliable and valid answers to evaluation questions, which will be designed and developed by the international consultant in consultation with LD and UNESCO team members within the limits of resources; an evaluation matrix will be prepared as part of the Inception process. The methodology will include:

• Desk Study: The Evaluation Team will examine all relevant ELA3 documents. The desk review should explore other literacy initiatives globally and within the region, including of UNESCO, as a basis for future advice and relevant lessons learned and best practices.

à In May 2018, an evaluability assessment of ELA was conducted. Through intensive engagement of key stakeholders, it recommended key areas to be included in a final ELA3 evaluation. As such, the EA report will serve as one of the guiding documents for this final evaluation.

• Development and finalization of methodology: The international consultant will design and finalize the tools for collection of data. This will be done in close consultation and discussion with the ELA teams including LD and UNESCO.

• Field Visits: Where possible (and dependent on security clearance), the international consultant, alongside LD and UNESCO team members will collect primary data from DMoEL offices at national and subnational levels in a few representative provinces.

• Interviewing key stakeholders: At national level, the international consultant will pay particular attention to interviewing the key stakeholders, i.e. senior management, heads of departments and key focal points in the DMoEL, senior management and other key focal points in the UNESCO and Department ELA teams, key managerial and advisory personnel. At subnational level, the international consultant and/or LD and UNESCO team members should also interview the Provincial Governor, Provincial Education Officer, Provincial Literacy Manager, Provincial ELA Coordinator, District Education Officer, District ELA Coordinator, Shura Members, Community Development Committee (CDC), INGOs implementing literacy programme in Afghanistan, and interested civil society organizations, women’s groups, and so on. The international consultant should hold interviews with key persons in Ministry of Education (MoE) at central level; as well as development partners, especially ELA donors, UN agencies and relevant members of the national and international “literacy community”. A final list of KII/FGD respondents will be developed/approved as part of the Inception process.

• Review and finalization of report: The draft of the final evaluation report will be shared with all stakeholders for feedback/ comments and inputs incorporated as applicable in the final report.

Time and Deliverables

The final evaluation should be completed between mid-July and mid-September 2018 with a final report submitted by end of October 2018.

Expected deliverables as follows-

1. An Inception report outlining the evaluation questions (and reasons for the selection of the evaluation questions), a detailed outline of the evaluation process, the data collection tools and a detailed work plan for the final evaluation;
2. A draft report of the narrative evaluation of ELA3 that includes achievements, lessons learned, challenges and constraints, and recommendations on the scope for new directions of youth and adult literacy programme in Afghanistan as directly derived from the evidence of the ELA3 evaluation;
3. A knowledge sharing workshop on evaluation findings and recommendations in presence of all key stakeholders; and
4. A final narrative report of the evaluation incorporating all the comments and feedback from stakeholders.

Qualifications and Experience

The international consultant will have:

• at least 10 years of professional experience in the evaluation of education policy development and implementation, with particular focus on Adult Literacy and Non-Formal Education. Relevant experience in education and related areas, program/project development and management in a UN system agency or organization is an asset.
• a university degree preferably PhD in education, psychology, sociology or related studies in social science field is required. Applicants for the position of national consultant should have at least Master’s degree in similar fields.
• experience in organizational management, structures and systems, operations, capacity development, reporting, and monitoring;
• a minimum of two years of work experience in conflict-affected areas; experience in Afghanistan is highly desirable;

In addition, it is required that the international consultant will be able and willing to travel to the provincial areas covered by the programme (where security clearance is given).

Competencies (for all applicant positions)

• Core: Integrity, Commitment, Building and maintaining partnerships, Teamwork, Communication, Creativity and Innovation, Results Based Management and Results Based Budgeting;
• Skills: Demonstrated ability to effectively work within a complex multicultural and conflict-affected environment, proven interpersonal and communication skills, strong technical assessment and analytical skills, and ability to organize and coordinate multiple tasks, and deliver the result within specified timelines.

Language requirements:

• Applicants must be fluent in both written and oral English; capacity in Dari and/or Pashto is desirable.

Evaluation Ethics

The evaluation will be conducted in accordance with the principles outlined in the Ethical Guidelines for Evaluation (UNEG 2008) and the evaluation team must take measures to ensure compliance with the evaluator code of conduct e.g. measures to safeguard the rights and confidentiality of their sources, provisions to collect and report data, particularly permissions needed to interview or obtain information about children and young people, provisions to store and maintain security of collected information; and protocols to ensure anonymity and confidentiality.

[4] These skill areas are; Bee Keeping, poultry farming, fish farming, tailoring, Basic health and first aid, starting your own business and vegetable gardening, were identified through a rapid needs assessment survey for the development of this instructional materials.

The interested candidates should send the following documents to ki.consultant@unesco.org before June 30 2018 (by midnight, Kabul time), with subject KAB/EDU/2018/07.

1. (Up-to-date UNESCO curriculum vitae (https://en.unesco.org/careers/media/3705)

2. Cover letter explaining qualifications and experience relevant for the position

3. A technical proposal which includes an indication of the approach which would be adopted to carry out the assignment (including methodology and theoretical framework)

Please note that applications received after June 30 2018 (midnight Kabul time) will not be considered. Only shortlisted candidates whose application responds to the above criteria will be contacted for interview. UNESCO regrets its inability to reply individually or attend to telephone queries on the advertised post.