《简洁大方ppt模板.ppt》由会员分享，可在线阅读，更多相关《简洁大方ppt模板.ppt（33页珍藏版）》请在三一办公上搜索。
1、,2011年11月,Monitoring and Evaluating ODA,This presentation will cover Why monitor?Why evaluate?Monitoring in the World Bank,other multi-lateral development banks,bilateral donorsEvaluation in the World Bank,other multi-laterals,bilateral donors,What is monitoring?,Definition of monitoring:a continuou
2、s process that uses systematic collection of data about progress in an on-going development project,for use by managers and other stake-holders,Why monitor?,To improve implementation of on-going projects;must provide practical informationTo assess whether development objectives likely to be metTo id
3、entify systematic issues and risks across projects,by country,by sector,and overallTo provide input for ex post evaluation,What is Evaluation?,Definition of(ex post)evaluation:assessment of a completed project,program,or policy,its design,implementation,and results.,Why evaluate and who cares?,Why?T
4、wo reasons:To learn from experienceTo provide accountabilityWho uses(or should use)evaluations:Boards of Directors(MDBs)and Ministers and Directors General for oversight;ODA managers for policies and strategies;ODA staff for program and project designs,ways of doing business;Client countries,externa
5、l partners,and the public for policies,aligning programs,research,M&E in context,World Bank commits/disburses$20-23 billion/year;Approves 280 loans and credits/year;active loan portfolio:1,400 in 120 countriesBank also provides grants and other non-lending services(diagnostic work)Bank has total adm
6、inistrative budget of$2 billionMost other MDBs commit$1 6 billion/year Bilateral ODA:$200 million(Greece,Luxembourg,New Zealand,Portugal)-$25 billion/year(USA)Bilateral agencies have administrative budgets of$11 million(Luxembourg)to$700 million(Japan),Monitoring in the World Bank,Every on-going len
7、ding operation is monitored by visits from Bank staff about twice a year;Implementation status and results(ISR)report issued twice each year:Progress on implementationProgress in meeting development objectivesProblem projects identifiedPortfolio at risk reports separate methodology updated monthlyOt
8、her quality control activities include quality at entry reviews;quality of supervision,Evaluation:the topics,Self evaluationIndependent evaluation:why it mattersTools:projects;sectors;countries;etc.Coverage:how many evaluationsResources/budgetMain challenge for evaluationDisclosureOther Evaluation I
9、ssues,Self-Evaluation in the Bank,Self evaluation in the Bank:Every project has an implementation completion report,done by staff of responsible unit(about 280/year)Every new country assistance strategy now has completion report covering previous country assistance strategy,done by staff of responsi
10、ble unit(about 20/year),Self-evaluation in other donors,Multi-lateral institutions:almost all MDBs have requirement for self-evaluation,although in some MDBs(Asian DB,EBRD,EIB),the evaluation department is involvedBilateral donors:some require for projects over given value,others dont;SDC(Swiss):ext
11、ernal evaluators do self-evaluation;GTZ(German)has innovative eVAL(electronic)system.,Independent evaluation,What is independence?The extent to which the evaluation unit:Reports directly to a Board or other governing unit;Has staff,budget,and functions independent from management;not involved with m
12、anagement of ODABehaves independently by issuing objective,impartial,and balanced reports,Independent evaluation,In the World Bank,independent evaluation,since 1973,has looked like this:,Independent evaluation,In bilateral donors,independent evaluation has evolved to look like this:,Independent eval
13、uation,Why independence matters:Allows for objective,unbiased evaluationsEvaluation findings are not tailored to managements wishesIndependence lends credibility to findingsIndependence difficult to achieve and maintain,Evaluation Tools:World Bank,Project evaluationsImplementation Completion Report(
14、ICR)reviewsProject performance assessment reviews(PPARs)Country evaluationsCAS Completion Report(CASCR)ReviewsCountry assistance evaluations(CAEs)Sector,thematic,global evaluationsProcess evaluationsCorporate evaluations(ARDE,AROE),Evaluation Tools:interactions,Synergies among different evaluation t
15、ools can provide valuable input,Evaluation Coverage:World Bank,Evaluation coverage,In most MDBs,full range of evaluations:project,country,sector,thematic evaluationsIn most bilaterals,few project evaluations;some country level;most focus on themes or programsMost bilaterals produces 3-10 evaluations
16、/yearDFID produces 20-25 reports/yr;Potential weakness of country and thematic evaluations without underpinnings of project evaluations to use as input,Resources for evaluation:MDBs,Resources vary,but as percent of total administrative budget,fairly similar:,Resources for evaluation:bilaterals,Bilat
17、eral donor agencies are more likely to hire external consultants for virtually all evaluations(UK,Norway,Denmark,Switzerland,Australia)Based on numbers from a few bilaterals(DFID,NORAD,SDC),evaluation budget is 2%of total administrative budget;figures of 0.5 2%have been cited by others.Smaller donor
18、s(Greece,New Zealand)are still setting up their evaluation units,Main challenge for evaluation,How to make evaluations usefulMost managers,staff focused on future,not on pastEvaluation must/should be:TimelyOperationally relevant:avoid generalizations,platitudesEasily accessible:avoid long reportsCon
19、structive:good practice should be rewardedWidely disseminated and disclosed,Disclosure of evaluation reports,World Bank:public disclosure of all:PPARsCAEsSector,thematic,global evaluations Process evaluationsCorporate evaluations EBRD,ADB:selective public disclosure:evaluations involve private secto
20、rBilaterals:majority disclose,but varies:UK,Norway full disclosure;Germany selective disclosure,Disclosure,For World Bank:reports and information on evaluation tools,approaches,on website:http:/worldbank.org and scroll down to right-had side of home page,or:For other donors:check their websites,Othe
21、r issues in evaluation,Evaluation approachUse of performance indicatorsSources of informationRatingsProcessesPartnershipsOther challenges,Evaluation Approach,Within context of results-based management,Bank uses objectives-based approach,focus on outcomes and impact,Evaluation Approach,Bank assesses:
22、Outcome(relevance,efficacy,efficiency)Use of performance indicators for outputs,outcomes,impactRisk to DevelopmentBank and Borrower Performance,Sources of information for evaluation,Baseline data in project appraisal documents(sometimes)Monitoring reports should have information on performance indic
23、atorsData bases for project area;sector;country at beginning of project,end of projectOther country-level indeces(IMF financial statistics,Doing Business surveys;World Bank Indicators of corruption surveys;UNDP human development indicators)SurveysInterviews,focus groups,Use of Ratings,World Bank use
24、s ratings on project outcomes and outcomes of country assistance.No ratings on sector,thematic,other evaluations Other MDBs use ratings for project outcomes;not for country assistanceBilaterals:some use ratings,only for projects;some do not use any ratings,Evaluation Approach:ratings,Project Ratings
25、 in World Bank:Outcomes:6 point scale(HS-HU)Risks to Development:4 point scale(H-N)Bank and Borrower performance:4 point scale(HS to HU)Pros and cons of ratingPro:provides data and accountabilityCon:takes focus away from lesson learning and richness of evaluation,Processes,Independence does not mean
26、 isolationMDBs and bilateral donors incorporate comments from operational managers and staff and from stakeholders in client countries into evaluation reports:Banks initiating papers for country,thematic,global evaluations sent to management for comment and to Board of Directors for information;IMFs
27、 initiating paper for all evaluations posted on IMF website and public comment invited;wide consultations throughout process.,Partnerships in evaluation,Country and sector evaluations sometimes involve partnerships with other MDBs and/or bilateral donors(joint or parallel evaluations)Opportunistic,n
28、ot systematic,partnershipsPros and cons of evaluation partnershipsPros:wider coverage,other viewpointsCons:more costly and more time required;may not agree on outcomes;may affect timeliness of final report,Other challenges,Data difficult to obtain,or different sources have conflicting dataOften difficult or impossible to have a control group(especially in sector and country evaluations)Beneficiary surveys can be costly,low response rateProblem of attribution(cause-effect)difficult or impossible to establish,
