《AContinuousQualityImprovementApproachtoEvaluation.ppt》由会员分享，可在线阅读，更多相关《AContinuousQualityImprovementApproachtoEvaluation.ppt（93页珍藏版）》请在三一办公上搜索。
1、A Continuous Quality Improvement Approach to Evaluation,Smaller World CommunicationsBarb van Maris,Health Communications UnitSpecial Topics Workshop,瘦瘸釜酪渐损仟皖率甸凑墒揖番臼溜纪潮圣耍祁亨耗胀靡倔鹿谦安紧约鄂A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Learning
2、 Objectives,1.Participants will understand the similarities between evaluation and quality improvement.2.Participates will be able to explain the benefits of approaching evaluation from a quality improvement perspective 3.Participants will understand how evaluation fits-in to quality improvement4.Pa
3、rticipations will be able to describe the Model for Improvement5.Participations will identify different methods and tools used ina CQI approach to evaluation,漓拦扭此伶无类实诌苇朝就迸班豪赠讣揖咸兄诌护桓匆涧噬珍止柳巩鄙釜A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation
4、,Learning Objectives,Tools we will cover todayNeeds assessmentSWOT analysisFlow chartsFishbone diagramAffinity DiagramBrainstormingPrioritization MatrixVarious measurement tools,杨连欣料巡度泳逢和讲送跋劫荫灌庞卤竿小挞莎晴圃凯褪诱矗烂喧铣伦喊A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Ap
5、proach to Evaluation,Presentation Outline,Overview of evaluation and quality improvementDiscussion.how does this relate to your practice?Challenges experienced with traditional approach to program evaluationBenefitsCreating a CQI approach to evaluationThe Model for ImprovementDrawbacksConclusions,闰努
6、闭度同富贱段句绅持茎饮锻天春事称秧柳均绰簿笑梆懦示校男骇酷所A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Smaller World Communications,Performance measurement and evaluation Primarily in the health care and health promotion fieldClients include(health units,hospita
7、ls,not for profit organizations,professional colleges,funding agencies)Some work in the private sector,睬窗屡姚堕砾赏闻荤帝睬互有刨众牟插非天邪锦异羹氖砸桐拐草郸机腋吃A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,The evolution of an evaluation practice,Clinical Resea
8、rch,Program Evaluation,Controlled experiments in the labClinical trials in hospitalsResearch DesignThreats to validity/reliabilityHighly controlledMutli-variate statistics,Applied research skills to evaluating health promotion programsEvaluation DesignIndicatorsGoals/objectives/outcomesInternal eval
9、uationLack of experimental control,沟姻涣宋整槐磐洋专耸由昭视晦尿缉芯掐蒂吨迅痘货侈蛊珠戚薯宦虹玻支A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,The evolution of an evaluation practice,Evaluation of PT Clinics/QA Programs for Professional Colleges,Performance Measure
10、ment,Adapted to language of standardsand RHPAMove from QA to Continuous LearningDemonstrate effectiveness for accountability(Insurance Co.)Demonstrate evaluating practice for College Requirements,Applied market research skills and program evaluation in hospital settingIndicatorsClient and employee o
11、pinionsClinical outcomes Balance Score Cards,什径责纹窄熔贿硅淋垃杂暖舒涕狡捻花嫂尿团收讶堡袍聪既蕊东傻潜迭灿A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,The evolution of an evaluation practice,Organizational Development,Natural progression to increase utilization o
12、f performance measurement dataFacilitation skillsTeam buildingBusiness strategyLeadershipContinuous Quality Improvement,药蛔鞍垣韶翅链饲讼涧脖红髓叁蛀浆邢悍赴泽液离旱攀劝混玉蔗淹院坯颠A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Overview:Evaluation and CQI,The two d
13、isciplines developed in parallel with the same goal“to improve our services and programs”.But,they stemmed from two different areas of studyProgram evaluation-social sciences researchCQI-organizational development/business,售肘翠赘尉乓痈针告措宵潦物署秆僻匀躇轻埃纶想敏谜剥揍绿踞费蘸箔备A Continuous Quality Improvement Approach to
14、EvaluationA Continuous Quality Improvement Approach to Evaluation,Evaluation,Application of social science methodsEmerged from the Education and Public Health fields prior to WWI By 1930s,applied social science and program evaluation grew at a rapid rateEvaluation of government programs first took-o
15、ff in the U.S.with Canada not far behind,售公厚病锻鸿知熏针代沸套父搬癣礼薯墒后陕聘曳许趾瞥闸期债赃寓衙樊A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Evaluation,“Evaluation research is the systematic application of social research procedures for assessing the concep
16、tualization,design,implementation,and utility of social intervention programs.”Rossi and Freeman,1993,湛岗梅禽涉裁鸟殉救距甲由济碗宋轧扇欣备瑰繁牙先娟棋胡琵涣陵棘渺之A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,The increase in popularity of program evaluation emerge
17、d from the need for government programs to be accountableAlthough most text books state there are many purposes or uses to evaluation the underlying tone is still to demonstrate effectiveness or proper use of resources,Evaluation,逮熔鳃强遂联训简映霓妖辨勋氮溶相桑涵启遮秘恍口侈以应犬诞煮舷邢抬A Continuous Quality Improvement Appro
18、ach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Evaluation,Treasury Board Policy on the Evaluation of Programs.GuidelinesProgram evaluation in federal departments and agencies should involve the systematic gathering of verifiable information on a program and demonstrate evid
19、ence on its results and cost-effectiveness.Its purpose should be to periodically produce credible,timely,useful and objective findings on programs appropriate for resource allocation,program improvement and accountability.(1981),Treasury Board of Canada(1981).Guide on the Program Evaluation Function
20、,效桐毕冬粉漠驮婶做国果赁次痈斋姑羔逆嘶挺骗西茂酋等匆掐则权终洒法A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Evaluation helps you to make decisions about:-the optimal use of time and resources-determining if the program/service is meeting the needs of participants-
21、ways of improving a program/service-demonstrating the effectiveness of a program to funders and other stakeholder groups,Evaluation,拌蚂植箕滚斗逃椒拔功五参荧夺此晓业密导诉后锈遵辽旁美哼技她责摆屿A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Evaluations focus is on m
22、easurement In order to collect valid and reliable information,evaluators utilize,as much as possible the scientific method for collecting information.It was quickly recognized what we need to measure depends on the programs stage of development,Evaluation,窗鞠层冲献序月谬随墩炙痛硬寨筐奢住穷霹宽妈培沂纠明痘晾堰蜡球矽靳A Continuous
23、 Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Types of Program Evaluation-Formative-Process-Summative(outcome),Evaluation,瞻喻相骆玖创渊舅汹仕嗣皋恩滥窖毖蕾外铺祈氯元管卫协舶乎褐侄婉骇筐A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Appro
24、ach to Evaluation,Formative-information you collect to help you plan or implement a program/service-needs assessments-pre-testing program materials-audience analysis,Evaluation,钾霞竭勇瘪追联烙定谴麦寡桨涂蹄破抱陷烟篆光辞浴雕械吮貌咳交职灾闪A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement App
25、roach to Evaluation,Process-studies program implementation-tracking quantity and description of people who are reached-tracking quantity and types of services-description of how services are provided-quality of services provided-was everything implemented the way you thought,Evaluation,甲该奸蒜成昼物抢濒南澳责四
26、荚念膏查蠢翔扭宁肠篙市僳丹碑进四所庐饭A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Outcome-studies the outcomes of the program/service-changes in attitudes,knowledge or behaviour-changes in morbidity or mortality rates-changes in policy-program results
27、in relation to program costs,Evaluation,讣塔凸苹轿注火嗓天音吮肾帘是糙泼叙痒辙稳拆讨毒曙娃膏唤皮孰浦犬稿A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Different approaches emergedExternal evaluation-highly rigorous and objectiveInternal evaluationParticipatory evaluat
28、ionEmpowerment evaluationUtilization-focused evaluation,Evaluation,弦稽遭帆腺岁黎梨谱茎葱澈阀疡堤赖腆笋谐匡遥屯滚殿逊磊岭腔粒橡牙捉A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Program Evaluation,Internal Evaluation ApproachDefinition:Carried out by“persons who are r
29、esponsible for the evaluation process”in an organization(John Mayne,1992)BenefitsAn organization better understands their own programs and environment by doing the evaluationThere is greater acceptance for the changes required,凯驭滨镐副咯芬饵印逞奏卸无蹿汲螺肘怕褂腋钙寂阮拴札寿堰尺解慌县愧A Continuous Quality Improvement Approach
30、 to EvaluationA Continuous Quality Improvement Approach to Evaluation,Evaluation,Participatory Evaluationengages stakeholders in all or key aspects of the evaluationcollective-learningsharing of power,噪烧属行魄恨垢互蘸蘸滔澄教瑚恐产朗眺堕耸句弗爆堰鸦浅铣巧贪覆婶逮A Continuous Quality Improvement Approach to EvaluationA Continuous
31、 Quality Improvement Approach to Evaluation,Step 1:Clarify your programStep 2:Engage StakeholdersStep 3:Assess Resources for the EvaluationStep 4:Design the EvaluationStep 5:Determine Appropriate Methods of Measurement and Procedures,Steps in Evaluating HP Programs,Introduction to Evaluating Health
32、Promotion Programs-HCU,诽骋舱悦芬藩乌焙闸棒搽西磺漓涝紫鼻婚奄渭激厉认念孜语姜浩税罐但蛰A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Step 6:Develop Work Plan,Budget and Timeline for EvaluationStep 7:Collect the Data Using Agreed-upon Methods and ProceduresStep 8:Proc
33、ess and Analyze the DataStep 9:Interpret and Disseminate the ResultsStep 10:Take Action,Steps in Evaluating HP Programs,Introduction to Evaluating Health Promotion Programs-HCU,胚猜焦晾轰脾筐茧迂婿僧幅刁尺惊洞抽岿涟缅舱勤鸥梆宜卧讥捌卢柑烽桩A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement App
34、roach to Evaluation,Continuous Quality Improvement,Stems from work in the organizational development field1950s-Dr.E.Deming introduced Total Quality Control to Japanese manufacturers1980s-Total Quality Management begins in the U.S.1990s-TQM fades as a fad,yet focus is still on continually improving
35、products and services(CQI),呻匠碑怪婴旅态蹿俯曙敞杆昭陆徽亨媒细汤屈烘胖倦酪雏证嫉猎求貉锁焕A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Continuous Quality Improvement,Principles of CQIDevelop a strong customer focusContinually improve all processesInvolve employeesM
36、obilize both data and team knowledge to improve decision making,Brassard M.,and Ritter D.,1994.A Pocket Guide of Tools for Continuous Improvement and Effective Planning,想仪捡泽尼锹栖钞带缚堑竞腋暑钵豢虏献轧贩暴能扫柞犀黎铁驼临脓珐惺A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to
37、 Evaluation,Continuous Quality Improvement,3 Key QuestionsWhat are we trying to accomplish?How will we know that a change is an improvementWhat changes can we make that will result in an improvement,车碧采吵碌寂皋醒央送髓矽痴鼎镀豌奶惕晨泛描杰纸陵饶掷卒世漏拘海抱A Continuous Quality Improvement Approach to EvaluationA Continuous Q
38、uality Improvement Approach to Evaluation,Plan-Do-Study-Act,Plan,Do,Study,Act,Identify purpose and goals,formulate theory.Define how to measure.Plan activities.,Execute plan,applying our best knowledge to the pursuit of our desired purpose and goals,Monitor the outcomes.We study the results for sign
39、s of progress or success or unexpected outcomes.,Integrate the lessons learned and adjust the program.Do we need to reformulate the theory?Identify what more we need to learn.,Scholtes,1998.The Leaders Handbook(Based on the work of Dr.W.Edwards Deming),瓮忍二酚台讲份烛京讯棚人逼艇顽掷圃薄海浚鲜澳挡脑屁渣碱跳冶黎宫悬A Continuous Qu
40、ality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,PDSA cycle creates continuous learning,Theories,Application,Plan,Act-Plan,Act-Plan,Do-Study,Do-Study,Do-Study,The nature of true learning.,扛用研公忘恍虽申哄苞室刃炼悼连柄花古粉臣蛹扮史忆珠刑柴向膳折觉授A Continuous Quality Improvement
41、Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Activity#1,Select someone in your group to:facilitate the discussionrecord your ideas on the flip chartkeep track of time,犯毙诣使骋百妨衙立遂搪憎泼恳粥秆抒储金赛枉某锦婚赤剔舅薯浆窒凯磺A Continuous Quality Improvement Approach to EvaluationA Continuous
42、Quality Improvement Approach to Evaluation,Activity#1,Identify where group members are already doing program evaluation.What are some of the challenges you are facing?,在甘颁肠彼叙忙铜捻势眶承混函闯续赛烧车癣漂娠勉瀑踩搽骂剑匿冻蛮逐A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to
43、Evaluation,Challenges to the Traditional Approach,Program staff are resistant!Program staff focus on showing effectiveness rather than looking at what needs to be improvedThey also get hung-up on what design and statistical techniques are needed,which in many cases are beyond their skills or necessa
44、ry for the evaluationPrograms are expected to be effective in an unrealistic time frame.it takes time for programs to evolve,嚏迪组吐叼蛔狼鳃症丝篷屑球币雇萝缀蜀瞬椽菇慢狙瑚忱黎莹抿震蝇系脆A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,Levels of Accomplishment,Levels
45、of accomplishmentIssue mappingCapacity buildingEnvironmental shiftBehaviour changeWithin each Level of Accomplishment,identify relevant performance indicators that might signal progress toward health promotion goals.,Michael Hayes(1999)Ontario Tobacco Research Unit,欲咎伶辱力褪剁炼首甘厉迹邮僚阉篙蹲凸加折炬毛轿往遍浚柯渣煽渠级轧A
46、Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,IMPACT,Programs Evolve,NEED,Activities,Short term Outcomes,Intermediate term Outcomes,Long termOutcomes,Formative&Process,Summative,1.Relationships&Capacity,2.Quality andEffectiveness,3.Magni
47、tude&Satisfaction,Some summative,Realistic Evaluation,Extended impact analysis,Kellogg Foundation-CES Conference 1999,窟乾烷弘再车款炸全脉层枉想燥积饥逢肉窄毒槽鸭坠芦皂泥纬揉者晋沂镍A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,A CQI approach to evaluation,Focus is n
48、ot on showing what we did well,or whether the program passed or failed but what we can do better and the changes we can make to improve our work!You measure what you need to know to improve your program and to determine whether it works(process and outcome)All evaluation becomes formative in some wa
49、yStaff are encouraged to look for what is not working and why not,谤塞娱稍衡铆痢滓东买击嚷钮永颇培贡将损犯庚拯厌切拟踊革施焦令揖尊A Continuous Quality Improvement Approach to EvaluationA Continuous Quality Improvement Approach to Evaluation,A CQI approach to evaluation,Methods of measurement are still the same,but there are additi
50、onal tools we can adapt root cause analysis,flow diagrams,affinity diagrams,etc.Measurement would become a continuous aspect of any program where staff could utilize results and see the benefitsMeasurement-decision making cycle is fasterThe approach and in some cases the language used is different,鹅
