IR System Evaluation
An attempt has been made to explain to you the concept of an IR System and its important components as well as functions. The discussion has particularly emphasised the working of a computer-based IR System.
In this section, let us try to understand some important aspects relating to the evaluation' of an IR System.
In recent years the evaluation of IR Systems and techniques for indexing, sorting, searching and retrieving information have become important. This growth in interest is due to the growing number of retrieval systems being used and the emphasis on the evaluation methods themselves. For example, the introduction of a large standardised test database and a forum for yearly analysis via the creation of the annual Text Retrieval Evaluation Conference (TREC) has provided a methodology for evaluating the performance of algorithms and systems. There can be many reasons to evaluate the effectiveness of an IR System. Some of these are:
- To aid the selection of a system to procure;
- To monitor and evaluate system effectiveness;
- To evaluate the query generation process for improvements;
- To provide inputs to cost-benefit analysis of an IR system; and
- To determine effects of the changes made in the existing information system