Multi-modal NLP Übung Erweiterungsmodul Computerlinguistik Sommersemester 2024
Graduate course, Ludwig-Maximilians-Universität München, 13 Fakultät für Sprach- und Literaturwissenschaften, Department II, Centrum für Informations- und Sprachverarbeitung, 2024
Serve as: Teacher
Teacher: Dr. Özge Alaçam, Beiduo Chen
Course Description
As a complementery to the theory course, in this practice session, we will step by step learn (i) different modalities and their characteristics, (ii) how to represent them, and (iii) bring them together to complete various downstream tasks (image captioning, multimodal retrieval, multimodal VQA, situated language understanding, and many more).
Course Schedule:
Week 1 (16.04.24) getting set-up ready, small exercise (Pytorch)
Week 2 (23.04.24): Multimodal Representations and Classicial Ensemble Models
Week 3 (30.04.24): Multimodal Representation Learning
Week 4 (07.05.24): Project Ideas & How to implement MLMs
Week 5 (14.05.24): Project Work (Coding/Paper writing)
Week 6 (Holiday) : No Lecture
Week 7 (28.05.24): Image Captioning
Week 8 (04.06.24): Hateful Meme Classification
Week 9 (11.06.24): Project Work (Coding/Paper writing)
Week 10 (18.06.24): Visual Question Answering
Week 11 (25.06.24): Project Work (Mid-Evaluation - mini presentations)
Week 12 (02.07.24): Project Work (Mid-Evaluation - mini presentations)
Week 13 (09.07.24): Project Work (Coding/Paper writing)
Week 14 (16.07.24) : Reproducible Code (Finalizing and publishing the git repo)
Final Evaluation:
- Final Project Submissions
- Project Presentation
- Technical Paper
- Reproducible Code