Deep Learning with Humans-In-The-Loop: Active Learning for NLP

With Lukas Rauch
Kassel, July 4th 2024

The abundance of text data and the advent of powerful deep learning models has led to rapid advancements in Natural Language Processing (NLP). However, adapting models to specialized tasks still requires a nuanced understanding of the data.

This adaption often requires human-annotated data that are considered to be time-consuming and labor-intensive. Active Learning addresses this by strategically involving humans in the learning loop, selecting instances for annotation that are most likely to maximize performance gains.

This approach not only optimizes human effort but also enhances the model’s adaptability to specific tasks with fewer annotated instances, making the training process more efficient and effective.

In this full day workshop in Kassel, you will get introduced to the concepts of Human-In-The-Loop Learning and (Deep) Active Learning. In a hands-on practical session, you will design a (Deep) Active Learning Cycle using Python.

Program

Part 1: Introduction to Human-in-the-loop Learning and Deep Active Learning

09:00 – 09:30 Introduction to Human-in-the-loop Learning and NLP 
  • Motivation & Basic Principles
  • Workshop Goals
09:30 – 10:30 Foundations of Active Learning
  • Getting Started with Active Learning
  • Code Tutorial: Actively Querying Instances in AL
  • What is Deep Active Learning?
10:45 – 12:00 Deep Active Learning in NLP
  • Challenges & Motivation
  • Deep Active Learning Cycle
    • Model & Data Selection
    • Cycle Initialization
    • Query Strategies
      • Uncertainty Sampling
      • Diversity Sampling
      • Hybrid Sampling
    • Budget Determination
    • Model Training
  • Working with Humans as Annotators
    • Practical Deep Active Learning
    • Principles of Annotations
    • Uncertain Annotators

Part 2: Practical Session: Designing a Deep Active Learning Cycle

13:00 – 14:30 Setting Up An Deep Active Learning Cycle
  • Tools & Libraries,
  • Common Pitfalls,
  • Adjusting the Deep Active Learning Settings & Modules
14:30 – 15:45 Implementing An Deep Active Learning Cycle
  • Implementing Query Strategies
  • Annotating Data
  • Running a Cycle

15:45 – 16:00 Wrap-up

Requirements

  • Basic Python Skills
  • Basic experience in Machine Learning/Deep Learning
  • Bringing your own laptop to the workshop

About the Instructor

Lukas Rauch is a researcher in the AI for Computationally Intelligent Systems (AI4CIS) team at the University of Kassel, Germany. After completing his master’s degree, he began his PhD in Natural Language Processing, with a focus on Transformer models and Deep Active Learning. Currently, he is working in the field of Avian Bioacoustics to apply these research methods to practical and challenging problems.