Till startsida
University of Gothenburg
To content Read more about how we use cookies on gu.se

Marco Baroni "Systematic compositionality in recurrent neural networks (and, if time allows, humans)"

Research profile seminar

Marco Baroni "Systematic compositionality in recurrent neural networks (and, if time allows, humans)"

Systematic compositionality in recurrent neural networks (and, if time allows, humans)

Joint work with: Brenden Lake, João Loula, Adam Liska, Germán Kruszewski, Tal Linzen

Recurrent neural networks (RNNs) are remarkably general learning systems that, given appropriate training examples, can handle complex sequential processing tasks, such as those frequently encountered in language and reasoning. However, RNNs are remarkably sample-heavy, typically requiring hundreds of thousands of examples to master tasks that humans can solve after just a few exposures. The first set of experiments I will present shows that modern RNNs, just like their nineties ancestors, have problems with systematic compositionality, that is, the ability to extract general rules from the training data, and combine them to process new examples. As systematic compositionality allows very fast generalization to unseen cases, lack of compositional learning might be one root of RNNs' training data thirst. I will next present a study where RNNs must solve an apparently simple task where correct generalization relies on function composition. The results suggest that a large random search in RNN space finds a small portion of models that converged to a (limited) compositional solution. Finally, if time allows, I will present ongoing work in which we study the compositional abilities of human subjects, trying to uncover the priors that subtend their generalization skills.

Lecturer: Dr Marco Baroni

Date: 10/24/2018

Time: 1:15 PM - 3:00 PM

Categories: Linguistics

Location: Faculty of Arts, Lundgrensgatan 1B

Contact person: stergios chatzikyriakidis


To the calendar

Page Manager: Stergios Chatzikyriakidis|Last update: 5/23/2016

The University of Gothenburg uses cookies to provide you with the best possible user experience. By continuing on this website, you approve of our use of cookies.  What are cookies?