Search Solutions 2024

Retrieval Augmented Generation

Tutor(s):

• Alessandro Benedetti, Email: a.benedetti@sease.io
  

  Apache Lucene/Solr committer and PMC member and Director and R&D Software Engineer at Sease Ltd 

Delivery format:

In-person

Target audience:

• Technical Managers
• Data scientists
• Software Engineers
• Developers
• Machine Learning passionate

Learning outcomes:

• You will have a complete overview of Retrieval Augmented Generation from its strengths to its differences with a fine-tuning approach
• You will learn about different RAG architectures and optimisation techniques with an additional focus on prompt engineering
• You will have the opportunity to see a real RAG implementation with snippets of code
• You will learn RAG best practices, applications, and challenges

Tutorial schedule and description:

• 9:30-9:50 (1) Introduction to Retrieval Augmented Generation (20 mins)
• 9:50-10:30 (2) RAG Shortcomings (40 mins)
• 10:30-11:15 (3-4) RAG Solutions and Optimisation (45 mins)
• 11:15-11:30 Coffee Break (15 mins)
• 11:30-12:15 (5) Prompt Engineering Techniques and Libraries (45 mins)
• 12:15-12:45 (6) Code Example Hands-on (30 mins)
• 12:45-13:15 (7) Evaluation and wrap-up (30 mins)

Tutorial logistics/materials:

Slides and code snippets will be provided. Bring your own laptop.

Technology Assisted Review of Document Collections

Delivery format:

In-person

Tutor(s):

• Mark Stevenson, School of Computer Science, University of Sheffield, Email: mark.stevenson@sheffield.ac.uk
• Aaron Fletcher, School of Computer Science, University of Sheffield, Email: ahafletcher1@sheffield.ac.uk

Introduction:

The problem of reviewing a collection of documents for relevance arises in a range of areas, including searching for research literature, legal disclosure, social media content moderation and test collection development. Technology Assisted Review (TAR) provides tools to support this task, particularly for large collections and when high recall is required. This tutorial will outline the core techniques used within TAR and review the current open research questions within the field. The main topics will include the definition of the TAR problem, reviewing TAR application areas, screening prioritisation, determining when to stop examining documents and future directions.

Target audience:

Anyone interested in the problem of identifying a large proportion of the relevant documents within large collections. For example, legal professionals required to disclose information as part of the discovery process; researchers wishing to conduct reviews of scientific literature; social media content moderators.

Learning outcomes:

Appreciation of the challenges involved in reviewing large collections of documents for relevance and the role of Technology Assisted Review in their mitigation.

Knowledge of the main techniques currently used within Technology Assisted Review, such as using Active Learning to rank documents based on relevance likelihood and methods for determining when to stop examining documents.

Knowledge of current open research questions related to Technology Assisted Review.

Tutorial schedule and description:

• 14:30-14:45 Introduction (15 mins)
• 14:45-15:45 Screening Prioritisation (60 mins)
• 15:45-16:00 Coffee Break (15 mins)
• 16:00-17:00 Stopping Rules (60 mins)
• 17:00-17:15 Future Directions (15 mins)

Tutorial logistics/materials:

Slides and code snippets will be provided. Bring your own laptop.

09:30 - 09:50 Registration and coffee
09:50 - Introduction
10:00 - Tanja Svarre (University of Aalborg) “People search in the enterprise”
10:20 - Eugene Morozov (ISKO) “Search and Browse Use Cases in Data Governance”
10:40 - Panel questions and answers
10:55 - Break

11:10 - Dyaa Albakour (SIGNAL AI) “Mapping the landscape of narratives in Global Media”
11:30 - Alessandro Benedetti (Sease) “What I don’t like about RAG: can we do better?”
11:50 - Taketomo Isazawa (Microsoft Research) “Beyond RAG: Integrating Knowledge with LLMs”
12:10 - Panel questions and answers
12:25 - Lunch

13:30 - Maria-Inti Metzendorf (Cochrane Evidence Synthesis Unit Düsseldorf) “Searching for Living Systematic Reviews – overview and case report”
13:50 - James Thomas (UCL) “The responsible use of AI in evidence synthesis: collaborative development of guidance and recommendations”
14:10 - Panel questions and answers
14:25 - Break

14:40 - Gabriella Kazai (Amazon) “What Matters in a Measure? A Perspective from Large-Scale Search Evaluation”
15:00 - Charlie Hull (OpenSource Connections) “Measure and Tune your Search with User Behaviour Insights”
15:20 - Daniel Tunkelang (Consultant) & Aritra Mandal (eBay) “Modeling Queries as Bags of Documents”
15:40 - Panel Questions and answers
15:55 - Break
16:30 - Panel Session
Panel topic: “Has generative AI removed the need for standalone search? A discussion”
Panel chair: Michael Upshall (The Search Network)
17:15 - BCS Search Industry Awards
17:25 - Closing words
17:30 - Evening Reception