Till startsida
University of Gothenburg
Sitemap
To content Read more about how we use cookies on gu.se

CLASP/CLT seminar: Desmond Elliott "Compositional Generalization in Image Captioning"

Research profile seminar

CLASP/CLT seminar: Desmond Elliott "Compositional Generalization in Image Captioning"

Image captioning models are usually evaluated on their ability to describe a held-out set of images, not on their ability to generalize to unseen concepts. We study the problem of compositional generalization, which measures how well a model composes unseen combinations of concepts when describing images. State-of-the-art image captioning models show poor generalization performance on this task. We propose a multi-task model to address the poor performance, that combines caption generation and image--sentence ranking, and uses a decoding mechanism that re-ranks the captions according their similarity to the image. This model is substantially better at generalizing to unseen combinations of concepts compared to state-of-the-art captioning models.

Lecturer: Desmond Elliott

Date: 2/20/2020

Time: 10:00 AM - 12:00 PM

Categories: Linguistics

Organizer: CLASP/CLT

Location: Department of Philosophy, Linguistics and Theory of Science
C562, Renströmsgatan 6

Contact person: Simon Dobnik

Page Manager: Stergios Chatzikyriakidis|Last update: 5/23/2016
Share:

The University of Gothenburg uses cookies to provide you with the best possible user experience. By continuing on this website, you approve of our use of cookies.  What are cookies?