INFOS11 Introduction to Information Retrieval Research (5 ECTS cr)

Literature for the set book exam

Book exam is optional for the lecture and seminar of the course INFOS11.

Goal

The module gives an overall view of the central orientations of information retrieval (IR) research and helps students to critically assess published research results.

Requirements

All take elements 1 and 2 and select two groups from element 3. All articles listed in elements 2 and 3 are available in full-text via all work stations in the network of the University of Tampere.

Elements

1) The book

Ingwersen, P. & Järvelin, K. The Turn: Integration of Information Seeking and Retrieval in Context. Springer/Kluwer 2005. Chapters 1-5 pp. 1-258. See also: Definitions pp. 381-392.

Reading hints

• The book describes different subfields and paradigms in information retrieval and seeking (IS&R) research. The thread of the text is to study the role of the human actor in different research traditions and theoretical models.  
• The theoretical basis of the book - the cognitive viewpoint - is introduced in Chapter 2.  
• Chapter 3 discusses research on information seeking and gives the reader an essential background and context for comprehending research on information retrieval. Especially one should read carefully section 3.1 to get a basic understanding of information seeking models and  how information seeking  can be studied in the context of work tasks. Section 3.2 reviews various methods used in IS research. 
• Chapter 4 introduces system-oriented research in IR starting from the laboratory model of IR (Fig. 4.1). The whole chapter should be read carefully; especially: Sections 4.1 Models; 4.3 Documents, Requests and Relevance; 4.4 Indexing, classification and indexing; 4.6 Interaction and Query Modification; 4.7 NLP; 4.9 Research methods; 4.10 Test collections and Performance measures and 4.11 Limitations and open problems. 
• In Chapter 5, the most important are the sections discussing research on interactive IR (5.3-5.6), on the notion of relevance (5.7) and on research methods (5.8).

Try to figure out how different research paradigms a) model information retrieval phenomena b) define and operationalize basic concepts and c) select research methods.   

2) Compulsory articles

1. Voorhees, Ellen & Harman, Donna (2000). Overview of the Sixth Text REtrieval Conference (TREC-6). Information Processing & Management 36(1) 1-204. ja
2. Borlund, Pia (2003). The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information Research, 8(3), paper no. 152 .

Reading hints

• Voorhees ja Harman describe the international laboratory-oriented research forum TREC and how research is conducted in its framework.  It is essential to figure out the goals and history of TREC, ad hoc task, components of the test collection, the principles of relevance assessments and  evaluation procedure. Issues releated to the routing task only can be skipped. Special tracks do not require detailed reading but the idea of each should be understood.
  
•  Borlund introduces the user-oriented evaluation model for interactive information retrieval. The criticism against the Cranfield paradigm adopted by the TREC is in the background of the model. Borlund criticizes the laboratory model of the lack of realism and of neglecting the user. You should focus on what aspects of the laboratory model Borlund regards as problematic and what experimental components of it she designs and implements differently.  Performance measures (RR measure, RHL indicator, etc.) have a minor priority. Consentrate on the overall ideas of and the differences between measures proposed.  
• You might benefit from reading parallelly and comparing the articles with the text in "The Turn", especially chapter 1.2-1.4 and 4.9-4.11.

3) Optional articles (select two groups)

Reading hints

Articles in each group are related to a selected theme but discuss different viewpoints and approaches within the theme. You should pay attention on the differences between articles. 

Group I: "Classical evaluations in operational IR systems"

1. Blair D.C. & Maron M.E. (1985). An evaluation of retrieval effectiveness for a full-text document retrieval system. Communications of the ACM, 28(3)289-299.
2. Salton G (1986). Another look at automatic text-retrieval systems. Communications of the ACM, 29(7):648-656. (kritiikkiä edelliseen).
3. McKinin EJ, Sievert ME, Johnson ED and Mitchell JA (1991). The Medline Full-Text Project. Journal of the American Society for Information Science, 42(4):297-307.

Group II: ”Graded relevance scales in the evaluation of IR systems”

1. Sormunen, E. (2002). Liberal Relevance Criteria of TREC - Counting on Negligible Documents? In: Beaulieu, M. et al. (Eds): Proceedings of the Twenty-Fifth Annual ACM SIGIR Conference on Research and Development in Information Retrieval. August 11-15, 2002, Tampere, Finland. Special Issue of SIGIR Forum 36():324-330.
2. Kekäläinen, J. & Järvelin, K. (2002). Using Graded Relevance Assessments in IR Evaluation. Journal of the American Society for Information Science and Technology 53(13): 1120-1129. 
3. Tang, R., Shaw, WM & Vevea, J.L. (1999). Towards the identification of the optimal number of relevance categories. Journal of the American Society for Information Science 50(3):254-264.

Group III: ”Information retrieval and learning”

1. Halttunen, K. (2003). Scaffolding performance in IR instruction : exploring learning experiences and performance in two learning environments. Journal of Information Science 29(2003):5, 375-390.
2. Halttunen, K. & Järvelin, K. (2005). Assessing learning outcomes in two information retrieval learning environments. Information Processing & Management 41(2005):4, 949-972.
3. Pharo, N. & Järvelin, K. (2006). "Irrational" searchers and IR-rational researchers. Journal of the American Society for Information Science and Technology 57(2):222-232.

Group IV: "Research on image retrieval"

1. Markkula M. and Sormunen E. (2000) End-User searching challenges indexing practices in the digital photograph archive. Information Retrieval 1(4): 259-285.
2. Sormunen E., Markkula M. and Järvelin K. (1999). The Perceived Similarity of Photos - A Test-Collection Based Evaluation Framework for the Content-Based Image Retrieval Algorithms. In: Draper S. et al., eds. Mira 99: Evaluating interactive information retrieval. Electronic Workshops in Computing.
3. Jörgensen, C & Jörgensen, P. (2005). Image querying by image professionals. Journal of the American Society for Information Science and Technology 56(12): 1346 - 1359.