《An Introduction to English Language Testing.docx》由会员分享，可在线阅读，更多相关《An Introduction to English Language Testing.docx（24页珍藏版）》请在三一办公上搜索。
1、An Introduction to English Language TestingAn Introduction to English Language Testing Definition of terms: measurement, test, evaluation Measurement: the process of quantifying the characteristics of persons according to explicit procedures and rules. Test: a procedure designed to elicit certain be
2、havior from which one can make inferences about certain characteristics of an individual. Evaluation: the systematic gathering of information for the purpose of making decisions. Approaches to language testing The essay-translation approach: the subjective judgment of the teacher is considered to be
3、 of paramount importance. Tests usually consist of essay writing, translation, and grammatical analysis, have a heavy literary and cultural bias. The structuralist approach: characterized by the view that language learning is chiefly concerned with the systematic acquisition of a set of habits, iden
4、tify and measure the learners mastery of the separate elements and skills of the target language. It is considered essential to test one thing at a time. The integrative approach: involve the testing of language in context and is thus concerned with meaning and total communicative effect of discours
5、e. Designed to assess the learners ability to use two or more skills simultaneously, and concerned with a global view of proficiency. The communicative approach: concerned with how language is used in communication. Success is judged in terms of the effectiveness of the communication which takes pla
6、ce rather than formal linguistic accuracy. Based on precise and detailed specifications of needs of the learners. Difference between approach and method Approach: theoretical positions and beliefs about the nature of language, the nature of language learning, and the applicability of both to testing
7、. Method: the way in which language or knowledge of language is elicited from a test taker. Test methods A framework of test method facets Test environment Familiarity of the place and equipment Personnel Time of testing Physical conditions Test rubric Test organization Time allocation Instruction F
8、acets of the input Format Nature of language Facets of the expected response Format Nature of language Restrictions on response Relationship between input and response Reciprocal Nonreciprocal Adaptive Characteristics of individuals Personal characteristics Age Sex Nationality Resident status Native
9、 language Level and type of general education Type and amount of preparation The topical knowledge that test takers bring to the language testing situation Their affective schemata Their language ability Communicative language ability A theoretical framework of communicative language ability Languag
10、e knowledge Organizational knowledge Grammatical knowledge Textual knowledge Pragmatic knowledge Functional knowledge Sociolinguistic knowledge Strategic competence Goal setting Assessment Planning Execution Psychophysiological mechanisms Uses of language tests Uses of language tests in educational
11、programs The information regarding educational outcomes is essential to effective formal education, to make decisions To improve learning and teaching through appropriate changes in the program, based on feedback To measure educational outcomes Research uses of language tests Research on language pr
12、oficiency Research on the nature of language processing Research on the nature of language acquisition Research on the nature of language attrition Investigation of effects of different instructional settings and techniques on language acquisition Classifying types of language tests according to int
13、ended use Selection: whether or not the students should enter the program Placement: placing students into appropriate groups Diagnosis: diagnosing students areas of strength and weakness in order to determine appropriate types and levels of teaching and learning activities Progress and grading: pro
14、viding continuous feedback to both the teacher and the learner for making decisions regarding appropriate modifications in the instructional procedures and learning activities. Classifying types of language tests according to content Proficiency tests: measuring general ability or skill Aptitude tes
15、ts: measuring capability or potential related to language acquisition as well as the use of language Achievement tests: measuring the extent of learning of the material presented in a particular course, textbook, or program of instruction Classifying types of language tests according to format Direc
16、t tests: measuring ability directly in an authentic context and format Indirect tests: fostering inference about one kind of behavior or performance through measurement of another related kind performance. Classifying types of language tests according to complexity of response Discrete-point tests:
17、employing items measuring performance over a unitary set of linguistic structures or features Integrative tests: measuring knowledge of a variety of language features, modes, or skills simultaneously Classifying types of language tests according to scoring Objective tests: scored with reference to a
18、 scoring key and not requiring expert judgment in the scoring process Subjective tests: depending on impression and opinion at the time of scoring Classifying types of language tests according to norm of reference Norm-referenced tests: evaluating ability against a standard of mean or normative perf
19、ormance of a group, implying standardization through prior administration to a large sample of examinees Criterion-referenced tests: assessing achievement or performance against a cut-off score that is determined as a reflection of mastery or attainment of specified objectives. Classifying types of
20、language tests according to time limit Speed tests: limiting time allowed for completion so that the majority of examinees would not be expected to finish it, containing so easy items that, given enough time, most persons would respond correctly. Power tests: allowing sufficient time for nearly all
21、examinees to complete it, but containing material of sufficient difficulty that it is not expected that a majority of examinees will get every item correct. Test usefulness Reliability Validity Authenticity Interactiveness Impact Practicablity Reliability The consistency of the scores obtainable fro
22、m a test. Test-retest method: calculated by the means of product-moment correlation of two sets of scores for the same person. Parallel forms method: two tests are administered to the same sample of persons and the results are correlated using product-moment correlation. Split half reliability: divi
23、ding a test into two nearly equal parts, correlating the scores together for the two parts, and adjusting the coefficient using the Spearman-Brown Prophecy Formula. Inter-rater reliability: correlation between different raters ratings of the same objects or performances, adjusted by the Spearman-Bro
24、wn Prophecy Formula. Validity The extent to which a test measures the ability or knowledge that it is purported to measure. Face validity: a subjective impression, usually on the part of examinees, of the extent to which the test and its format fulfills the intended purpose of measurement. Content v
25、alidity: a non-empirical expert judgment of the extent to which the content of a test is comprehensive and representative of the content domain purported to be measured by the test. Concurrent validity: the magnitude of the correlation between scores for a given test and some recognized criterion me
26、asure. Construct validity: the extent to which we can interpret a given test score as an indicator of the ability(ies), or construct(s), we want to measure. Response validity: the extent to which examinee responses to a test or questionnaire can be said to reflect the intended purpose in measurement
27、. Predictive validity: an indication of how well a test predicts intended performance. Relationship between reliability and validity Reliability: how much of the variance in test scores is reliable variance; examining variance in test scores themselves; agreement between similar measures of the same
28、 trait. Validity: what abilities contribute to this reliable variance; examining the relationship between test performance and factors outside the test itself; agreement between different measures of the same trait. A test cannot be valid unless it is reliable; it is quite possible for a test to be
29、reliable but invalid. Maximizing reliability may lead to reducing validity. Authenticity the degree of correspondence between the characteristics of a given language test task to the features of a TLU task. Real-life approach The appearance or perception of the test and how this may affect test perf
30、ormance and test use. The accuracy with which test performance predicts future non-test performance. Interactional/ability approach The interaction between the language user, the context, and the discourse the extent to which test performance reflects language abilities, or construct validity . Inte
31、ractiveness the extent and type of involvement of the test takers individual characteristics in accomplishing a test task. language ability language knowledge strategic competence topical knowledge affective schemata Impact the positive or negative feedback of a test on teaching and learning. washba
32、ck: the effect of a test on instruction. Practicability the relationship between the resources that will be required in the design, development, and use of the test and the resources that will be available for these activities. Reasons for test planning Providing the best means for assuring that the
33、 test will be useful for intended purposes Increasing accountability: the ability to say what was done and what was right. Increasing the amount of satisfaction we experience. Stages of test development Statement of the problem Writing specifications for the test Writing the test Pretesting Validati
34、on of the test Statement of the problem What kind of test is it to be? What is its precise purpose? What abilities are to be tested? How detailed must the results be? How accurate must the results be? How important is washback? What constraints are set by unavailability of expertise, facilities, tim
35、e? Writing specifications for the test Test specifications - the blueprint to be followed by test and item writers, and essential in the establishment of tests construct validity. Content Operations Type of text Addressees Topics Format and timing Criterial levels of performance Writing the test Sam
36、pling Item writing and moderation Writing and moderation of scoring key Pretesting Purposes Assessing the usefulness of the test Making the inferences or decisions for which the test intended Administering tests and collecting feedback Analyzing test scores Archiving Collecting feedback for assessin
37、g usefulness Kinds of feedback Methods of obtaining feedback Kinds of feedback Feedback about test takers language ability Feedback about the testing procedure itself Methods of obtaining feedback Questionnaires Multiple-choice questionnaires Rating scales Open-ended questions Think-aloud protocols
38、Observation and description Interviews Item types Objective-type items Multiple choice Dichotomous items Matching Information transfer Ordering tasks Editing Gap filling Cloze C-test Dictation Short-answer questions Subjectively marked tests Compositions and essays Summaries Oral interviews Informat
39、ion gap activities General problems of items What an item is actually testing? Each item should be independent of others Instructions for all items must be clear Multiple choice items The correct answer must be genuinely correct There is only one correct answer Each wrong alternative should be attra
40、ctive to at least some of the students Multiple choice items should be presented in context The correct alternative should not look so different from the distractors that it stands out from the rest Each option should fit equally well into the stem Item should not be independent from the reading or
41、listening passage Dichotomous items 50% possibility of getting any item right by chance It is necessary to have a large number of such items in order to discount the effect of chance Including a third category “not given” or “does not say” Matching To give more alternatives than the matching task re
42、quires Each item in the first column only matches one item in the second Information transfer The task can be complicated in the transfer but linguistically easy May be culturally or cognitively biased Ordering tasks Not easy to provide words or phrases which only makes sense in one order Marked who
43、lly right or wholly wrong The effort in constructing and in answering the item may not be considered Editing There is one mistake per line Students should be told how many errors there are Gap filling It is important to reduce the number of alternative answers to the minimum and to ensure that there
44、 are no other possible answers which are not listed in the answer key Candidates may not think of an answer not because they have poor language but because the word does not spring to mind A banked gap-filling task may be used It is important to tell students whether each gap is to be filled by one
45、or by more than one word Cloze Words are deleted mechanically The choice of the first deletion can have an effect on the validity of the test There may be many possible answers for any one gap Few of the items may test the aspects of language with which the tester is concerned C-test Instructions ar
46、e too complicated The number of missing letters should be shown in each gap Enough clues should be provided Dictation It is important to be presented in the same way to all the students It is not clear whether a word is misspelt or just wrong in the process of marking It is both time-consuming and b
47、oring to mark There may be many possible answers if students are required to write down the main points Short-answer questions Candidates must know what is expected of them There are many ways of saying the same thing Compositions and essays Instructions must be clear The students are required to ha
48、ve a wide general knowledge Give students some information before writing Summaries It may be impossible to know whether the test taker is poor in comprehension or in writing Marking is complex To provide a bank of possible words and phrases Oral interviews Only a limited vocabulary is used, not stretching the students ability to use complex structures Needs to be carefully structured to cover the aspects of language to be tested Each student is tested in a sim
