Lifelong learning with non-i.i.d tasks Conference Paper

Author(s): Pentina, Anastasia; Lampert, Christoph
Title: Lifelong learning with non-i.i.d tasks
Title Series: Advances in Neural Information Processing Systems
Affiliation IST Austria
Abstract: In this work we aim at extending the theoretical foundations of lifelong learning. Previous work analyzing this scenario is based on the assumption that learning tasks are sampled i.i.d. from a task environment or limited to strongly constrained data distributions. Instead, we study two scenarios when lifelong learning is possible, even though the observed tasks do not form an i.i.d. sample: first, when they are sampled from the same environment, but possibly with dependencies, and second, when the task environment is allowed to change over time in a consistent way. In the first case we prove a PAC-Bayesian theorem that can be seen as a direct generalization of the analogous previous result for the i.i.d. case. For the second scenario we propose to learn an inductive bias in form of a transfer procedure. We present a generalization bound and show on a toy example how it can be used to identify a beneficial transfer algorithm.
Keywords: Generalization bound; Learning tasks; Information science; Theoretical foundations; Bayesian Theorem; Data distribution; Inductive bias; Life long learning; Task environment
Conference Title: NIPS: Neural Information Processing Systems
Volume: 2015
Conference Dates: December 7 - 12, 2015
Conference Location: Montreal, Canada
ISBN: 10495258
Publisher: Neural Information Processing Systems  
Date Published: 2015-01-01
Start Page: 1540
End Page: 1548
Sponsor: This work was in parts funded by the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement no 308036.
Open access: no
IST Austria Authors
  1. Christoph Lampert
    87 Lampert
Related IST Austria Work