Till startsida
University of Gothenburg
To content Read more about how we use cookies on gu.se

CLASP/CLT seminar: Desmond Elliott "Compositional Generalization in Image Captioning"

Research profile seminar

CLASP/CLT seminar: Desmond Elliott "Compositional Generalization in Image Captioning"

Image captioning models are usually evaluated on their ability to describe a held-out set of images, not on their ability to generalize to unseen concepts. We study the problem of compositional generalization, which measures how well a model composes unseen combinations of concepts when describing images. State-of-the-art image captioning models show poor generalization performance on this task. We propose a multi-task model to address the poor performance, that combines caption generation and image--sentence ranking, and uses a decoding mechanism that re-ranks the captions according their similarity to the image. This model is substantially better at generalizing to unseen combinations of concepts compared to state-of-the-art captioning models.

Lecturer: Desmond Elliott

Date: 2/20/2020

Time: 10:00 AM - 12:00 PM

Categories: Linguistics

Organizer: CLASP/CLT

Location: Department of Philosophy, Linguistics and Theory of Science
C562, Renströmsgatan 6

Contact person: Simon Dobnik


To the calendar

Page Manager: Stergios Chatzikyriakidis|Last update: 5/23/2016

The University of Gothenburg uses cookies to provide you with the best possible user experience. By continuing on this website, you approve of our use of cookies.  What are cookies?