Topic: The Application of Language Testing Methodology
Details:
Dissertation - A deeper look at and discussion of the concepts of Validity and Reliability used to assess language tests is required in this chapter.
A short discussion of Washback in Language Testing would be beneficial as well. Some definitive notes are provided below to assist in the drafting of the chapter.
1). Validity in testing relates to the test actually testing what it should be testing. Therefore, for example, if it is a writing skills test, does it test the learner's writing skill? If it does, then the test is valid, and if it doesn't, then it is not.
There are different types of Validity to be considered in more detail, and these are:
- Face validity
- Content validity
- Criterion-related validity. Within this type of validity there are 2 distinct types:
• Predictive validity
• Concurrent validity
- Construct validity.
2). Reliability relates to tests producing consistent results, (also referred to as stable results), when it is conducted at different times, within accepted criteria for test administration and marking. So if the same range of scores is recorded in repeated tests, then it is reliable. Conversely, if the scores fluctuate a great deal, then the test is said to be unreliable. Discrete point testing of such things as grammar or vocabulary lends itself to this type of standard/traditional notion of reliability.
The Reliability of tests can be quantified statistically via the reliability coefficient. This figure can be measured via the following methods:
• the Test-retest Method
• the Split Half Method
• the Internal Consistency Method. This last method is useful in calculating reliability in larger tests.
There is another useful statistic worth mentioning, that can be used in the area of Reliability, called the Standard Error of Measurement (SEM). It uses the reliability coefficient in its calculation. It basically serves to tell us by how many points a student's test score would be likely to vary if we gave the student the same test again.
Furthermore, with the emergence of performance tests within the concept of communicative competence, other types of reliability have been identified, which are focused on the consistent performance of the test's raters (those persons marking & scoring the test) . Again they can be statistically measured and quantified. These rater reliabilities are:
- Inter-rater Reliability, which is used to measure rater agreement in the scoring of the same piece of writing or speech by 2 different raters
- Intra-rater Reliability, that measures the extent to which an individual rater agrees with him- or her- self over a period of time when scoring the same piece of writing or speech.
3). Wasback is defined the effect that a test can have on the actual classroom teaching and learning. It basically means that classroom teaching and learning activities can become adapted to help prepare for the test, and not necessarily follow the course's overall content and learning objectives so closely. It can also become more evident in the run up to the taking of certain tests and the teaching process can become test-driven, rather than curriculum focused.
Various factors can mediate Washback, including:
- the nature of the course / curriculum, (e.g. if the course and test is a locally produced one in a particular institution, there may be more Washback evident than with a nationally recognised test / curriculum)
- the training background of the teachers involved
- the culture / traditions of the institution in question
- the quality of the support and resources available for the teachers / learners engaged in the learning process
- learners' motivation
Washback can be negative if it means aspects of the course's content and general learning objectives are neglected because the teachers feel the need to only focus on the content which they believe will be present in the test. So for instance, if it is thought a lot grammar and vocabulary discrete point testing questions (such as cloze or multiple-choice) are present in the test, but little productive skill assessment with speaking and writing, then the latter 2 skills will be overlooked, even if they are actually an integral aspect of the curriculum. If a test is high stake, i.e. success or failure will have a very great affect on the learner's future prospects, there will be considerable pressure on the teacher to engage in ‘teaching to the test'.
Conversely, it may be considered positive because it can focus learning on what the course / curriculum is trying to achieve in general language learning terms. For example, if the test involves assessment of communicative language competence, such as a speaking skills, this will encourage the teacher to provide broader, more realistic language use practice in class, and so promote more valid language learning.
Within the discussion of Wasback it is worth touching on the concept of Content Alignment. It concerns the close correlation of what is taught in the course / curriculum and the content of the test. Of course, test designers have traditionally aligned their tests to a certain degree with the curriculum. However, Content Alignment is more comprehensive in that it aims to directly match what is taught with what is tested.
It is considered by some as being beneficial because it uses the test to control the curriculum. This in turn, will help improve teaching by removing the ambiguities described above, when teachers are not quite sure what or what not to focus their teaching on in respect of the test. It also makes the teachers / institution more directly accountable for the learning outcomes / test results.
Critics of Content Alignment feel it doesn't promote authentic language learning. They believe a test should be aligned to standards but not directly to the curriculum. Therefore, the precise course of study may vary from institution to institution, but it should still prepare the learner to be generally equipped with the language knowledge and skills to successfully take the test.
Academic Level: Master
Subject Area: Linguistic
Paper Style: Harvard
No# of Sources Required: 16
No# of Pages: 10 pages (2,500 words)