《How to design and evaluate research in education.doc》由会员分享，可在线阅读，更多相关《How to design and evaluate research in education.doc（27页珍藏版）》请在三一办公上搜索。
1、How to Design and Evaluate Research in EducationByJack R. Fraenkel and Norman E. WallenChapter 1The Nature of Researchn Ways of knowing Sensory experience (incomplete/undependable) Agreement with others (common knowledge wrong) Experts opinion (they can be mistaken) Logic/reasoning things out (can b
2、e based on false premises)n Why research is of value Scientific research (using scientific method) is more trustworthy than expert/colleague opinion, intuition, etc.Chapter 1 - continuedThe Nature of Researchn Scientific Method (testing ideas in the public arena) Put guesses (hypotheses) to tests an
3、d see how they hold up All aspects of investigations are public and described in detail so anyone who questions results can repeat study for themselves Replication is a key component of scientific methodChapter 1 - continuedThe Nature of Researchn Scientific Method (requires freedom of thought and p
4、ublic procedures that can be replicated)n Identify the problem or questionn Clarify the problemn Determine information needed and how to obtain itn Organize the information obtainedn Interpret the resultsn All conclusions are tentative and subject to change as new evidence is uncovered (dont PROVE t
5、hings)Chapter 1 - continuedThe Nature of Researchn Types of Research Experimental (most conclusive of methods) Researcher tries different treatments (independent variable) to see their effects (dependent variable) In simple experiments compare 2 methods and try to control all extraneous variables th
6、at might affect outcome Need control over assignment to treatment and control groups (to make sure they are equivalent) Sometimes use single subject research (intensive study of single individual or group over time)Chapter 1 - continuedThe Nature of Research(Types of Research continued) Correlationa
7、l Research Looks at existing relationships between 2 or more variables to make better predictions Causal Comparative Research Intended to establish cause and effect but cannot assign subjects to trtmt/control Limited interpretations (could be common cause for both cause and effectstress causes smoki
8、ng and cancer) Used for identifying possible causes; similar to correlationChapter 1 - continuedThe Nature of Research(Types of Research continued) Survey Research Determine/describe characteristics of a group Descriptive survey in writing or by interview Provides lots of information from large samp
9、les Three main problems: clarity of questions, honesty of respondents, return rates Ethnographic research (qualitative) In depth research to answer WHY questions Some is historical (biography, phenomenology, case study, grounded theory)Chapter 1 - continuedThe Nature of Research(Types of Research co
10、ntinued) Historical Research Study past, often using existing documents, to reconstruct what happened Establishing truth of documents is essential Action Research (differs from above types) Not concerned with generalizations to other settings Focus on information to change conditions in a particular
11、 situation (may use all the above methods)n Each of these methods is valuable for a different purposeChapter 1 - continuedThe Nature of Researchn General Research Types Descriptive (describe state of affairs using surveys, ethnography, etc.) Associational (goes beyond description to see how things a
12、re related so can better understand phenomena using correl/causal-comparative Intervention (try intervening to see effects using experiments)Chapter 1 - continuedThe Nature of ResearchQuantitative v. Qualitative Quantitative (numbers) Facts/feelings separate World is single reality Researcher remove
13、d Established research design Experiment prototype Generalization emphasizedChapter 1 - continuedThe Nature of Researchn Meta-Analysis Locate all the studies on a topic and synthesize results using statistical techniques (average the results)n Critical Analysis of Research (some say all research is
14、flawed) Question of reality (are only individual perceptions of it) Question of communication (words are subjective) Question of values (no objectivity only social constructs) Question of unstated assumptions (researchers dont clarify assumptions that guide them) Question of societal consequences (r
15、esearch serves political purposes that are conservative or oppressive; preserve status quo)Chapter 1 - continuedThe Nature of Research Overview of the Research Process (Fig. 1.4)n Introduction chaptern Problem statement that includes some background info and justification for studyn Exploratory ques
16、tion or hypothesis (relationship among variables clearly defined); goes last in Ch.n Definitions (in operational terms)n Review of related literature (other studies of the topic read and summarized to shed light on what is already known)Chapter 1 - continuedThe Nature of Research Overview of the Res
17、earch Process (Fig. 1.4)n Methods chaptern Subjects (sample, population, method to select sample)n Instruments (tests/measures described in detail and with rationale for their use)n Procedures (what, when, where, how, and with whom); Give schedule/dates, describe materials used, design of study, and
18、 possible biases/threats to validity4. Data analysis (how data will be analyzed to answer research questions or test hypothesis)Chapter 2The Research Problemn Statement of the Problem (identify a problem/area of concern to investigate) Must be feasible, clear, significant, ethicaln Research Question
19、s (serve as focus of investigation, see p. 28 list) Some info must be collected that answers them (must be researchable) Cannot research “should” questions See diagram, p. 29Chapter 2 - ContinuedThe Research Problemn RQ should be feasible (can be investigated with available resources)n RQ should be
20、clear (specifically define terms usedoperational needed, but give both) Constitutive definitions (dictionary meaning) Operational definitions (specific actions/steps to measure term; IQ=time to solve puzzle, where 20 sec. is high; 20-40 is med.; 40+ is low)n RQ should be significant (worth investiga
21、ting; how does it contribute to field and who can use info)n RQs often investigate relationships (two characteristics/qualities tied together)Chapter 3 Variables and Hypothesesn Important to study relationships Sometimes just want to describe (use RQ) Usually want to look for patterns/connections Hy
22、pothesis predicts the existence of a relationshipn Variables (anything that can vary in measure; opposite of constant)n Variables must be clearly definedn Often investigate relationship between variablesChapter 3 - ContinuedVariables and Hypothesesn Variable Classifications (Fig. 3.4, p. 42) Quantit
23、ative (variables measured as a matter of degree, using real numbers; i.e. age, number kids) Categorical (no variationeither in a category or not; i.e. gender, hair color) Independent: the cause (aka the manipulated, treatment or experimental variable) Dependent: the effect (aka outcome variable) Ext
24、raneous: uncontrolled IVs (see Fig. 3.2, p. 46) All extraneous variables must be accounted for in an experimentChapter 3 - ContinuedVariables and Hypothesesn Hypotheses predictions about possible outcome of a study; sometimes several hypotheses from one RQ (Fig 3.3) RQ: Will athletes have a higher G
25、PA that nonathletes? H: Athletes will have higher GPAs that nonathletesn Advantages to stating a hypothesis as well as RQ Clarifies/focuses research to make prediction based on previous research/theory Multiple supporting tests to confirm hypothesis strengthens itn Disadvantages Can lead to bias in
26、methods (conscious or un) to try to support hypothesis Sometimes miss other important info due to focus on hypothesis (peer review/replication is a check on this)Chapter 3 - ContinuedVariables and Hypothesesn Some hypothesis more important than othersn Directional v. nondirectional Directional says
27、which group will score higher/do better Nondirectional just indicates there will be a difference, but not who will score higher/do better Directional more risky, so be careful/tentative in using directional onesChapter 4 Ethics and Researchn Examples of unethical practices Requiring participation fr
28、om powerless (students) Using minors without parental permission Deleting data that dont agree w/ hypothesis Invading privacy of subjects Physically or psychologically harming subjectsn APA statement of ethical principles in research Each student must sign one and have it signed by workplace supervi
29、sorChapter 4 - Continued Ethics and Researchn Protecting participants from harm requires informed consent Subjects must know the purpose of the study, possible benefits/harm; participation is voluntary and they can w/draw without penalty any time (Fig. 4.3, p. 59)n Researchers should ask: Could subj
30、ects be harmed? Is there another way to get the info? Is the info valuable enough to justify study?n Researchers must ensure confidentiality of data (limit access; no names if possible; tell subjects confidential or anonymous)n Deceiving subjects is sometimes necessary (Milgram study), ask if result
31、s justify ethical lapse When deception used subjects they should be okay with it after (and they can refuse use of their data)Chapter 4 - Continued Ethics and Researchn Research with children Parental consent required (signed permission from parents APA Ethics in Research Form addresses this alson R
32、egulation of Research (National Research Act of 1974) If federal funding received must have an IRB to check: risks to subjects, informed consent guidelines met, debriefing plans for subjects HHS made changes in 1981 so that educational research is exempt under certain conditionsVideo 1Chapter 5 Revi
33、ew of the Literaturen Value of the Literature Review Glean ideas from others interested in topic See results of related studies (must be able to evaluated those objectively)n Types of sources General References indexes (of primary sources and abstracts (ERIC, Psych Abstracts) Primary Sources publica
34、tions where researchers report their results (peer reviewed/refereed journals) Secondary Sources publications where authors describe works of others (encyclopedias, tradebooks, textbooks)Chapter 5 - Continued Review of the Literaturen Steps in the Literature Review (manual or electronic) See example
35、s p. 74n Define problem precisely as possiblen Review some secondary sources*n Review some general reference works*n Formulate search terms (keywords/descriptors)n Search general references for primary sourcesn Obtain and read primary sources (make notes/summarize) *May be based on existing knowledg
36、e or previous readingChapter 5 - Continued Review of the Literaturen Making notes Include problem/purpose; hypotheses/RQ; procedures w/ subjects/methods; findings/conclusions; citation!n Searching strategiesuse Boolean operators (AND, OR, NOT) Searching wwwbe careful of reliabilityn Writing up the L
37、iterature Review Introduction - describes problem and justification for study; Body discuss related studies together (#2, p.88) Summary ties literature together/give conclusions arising from literature Reference listn Dont replace a review of primary sources with meta-analysis (a combined review of
38、all available research on a topic w/ results averaged)End Part 1Chapter 6 Samplingn Sample any group on which info is obtainedn Population group that researcher is trying to represent Population must be defined first; more closely defined, easier to do, but less generalizable Study a subset of the p
39、opulation because it is cheaper, faster, easier, and if done right, get same results as a census (study of whole pop) Accessible population the group you are able to realistically generalize tomay differ from target populationChapter 6 - Continued Sampling(Random v. Nonrandom Sampling)n Random every
40、 population element has an equal and independent chance to participate Uses names in a hat or table or random numbers Elimination of bias in selecting the sample is most important (meaning the researcher does not influence who gets selected) Ensuring sufficient sample size is second most importantn
41、Nonrandom/purposive - troubles with representativeness/generalizing Chapter 6 - Continued Sampling(Random Sampling Methods)n Simple random sampling Names in a hat or table of random numbers-p.99 Larger samples more likely to represent pop. Any difference between population and sample is random and s
42、mall (called random sampling error)n Stratified random sampling Ensures small subgroups (strata) are represented Normally proportional to their part of pop. Break pop into strata, then randomly select w/in strata Multistage sampling (see p. 94)Chapter 6 - Continued Sampling(Random Sampling Methods,
43、cont.)n Cluster random sampling Select groups as sample units rather than individuals REQUIRES a large number of groups/clusters Multistage sampling (see p. 94)n Systematic (Nth) sampling Considered random is list if randomly ordered or nonrandom if systematic w/ random starting point Divide pop siz
44、e by sample size to get N (ps/ss=N)Chapter 6 - Continued Sampling(Non-Random Sampling Methods)n Systematic can be nonrandom if list is orderedn Convenience sampling Using group that is handy/available (or volunteers) Avoid, if possible, since tend not to be representative due to homogeneity of group
45、s Report large number of demographic factors to see likeliness of representativenessn Purposive sampling Using personal judgment to select sample that should be representative (i.e., this faculty seems to represent all teachers) OR selecting those who are known to have needed info (interested in tal
46、king only to those in power) Snowball is a type (used with hard to identify groups such as addicts)Chapter 6 - Continued Samplingn Sample size affects accuracy of representation Larger sample means less chance of error Minimum is 30; upper limit is 1,000 (see table)n External validity how well sampl
47、e generalizes to the population Representative sample is required (not the same thing as variety in a sample) High participation rate is needed Multiple replications enhance generalization when nonrandom sampling is used Ecological generalization (gen to other settings/conditions, such as using a method tested in math for English class)Video 17Chapter 7 Instrumentation(Measurement)n Data information researchers obtain about subjects Demographic data are characteristi
