Till startsida
Göteborgs universitet
Till innehåll Läs mer om hur kakor används på gu.se

Marco Baroni "Systematic compositionality in recurrent neural networks (and, if time allows, humans)"


Marco Baroni "Systematic compositionality in recurrent neural networks (and, if time allows, humans)"

Systematic compositionality in recurrent neural networks (and, if time allows, humans)

Joint work with: Brenden Lake, João Loula, Adam Liska, Germán Kruszewski, Tal Linzen

Recurrent neural networks (RNNs) are remarkably general learning systems that, given appropriate training examples, can handle complex sequential processing tasks, such as those frequently encountered in language and reasoning. However, RNNs are remarkably sample-heavy, typically requiring hundreds of thousands of examples to master tasks that humans can solve after just a few exposures. The first set of experiments I will present shows that modern RNNs, just like their nineties ancestors, have problems with systematic compositionality, that is, the ability to extract general rules from the training data, and combine them to process new examples. As systematic compositionality allows very fast generalization to unseen cases, lack of compositional learning might be one root of RNNs' training data thirst. I will next present a study where RNNs must solve an apparently simple task where correct generalization relies on function composition. The results suggest that a large random search in RNN space finds a small portion of models that converged to a (limited) compositional solution. Finally, if time allows, I will present ongoing work in which we study the compositional abilities of human subjects, trying to uncover the priors that subtend their generalization skills.

Föreläsare: Dr Marco Baroni

Datum: 2018-10-24

Tid: 13:15 - 15:00

Kategorier: Lingvistik

Plats: Humanisten, Lundgrensgatan 1B

Kontaktperson: stergios chatzikyriakidis


Till kalendern

Sidansvarig: Stergios Chatzikyriakidis|Sidan uppdaterades: 2016-05-04

På Göteborgs universitet använder vi kakor (cookies) för att webbplatsen ska fungera på ett bra sätt för dig. Genom att surfa vidare godkänner du att vi använder kakor.  Vad är kakor?