Weakly-Supervised Segmentation and Unsupervised Modeling of Natural Images Dissertation Thesis


Author(s): Kolesnikov, Alexander
Advisor(s): Lampert, Christoph
Committee Member(s): Kolomogorov, Vladimir; Wojtan, Chris
Title: Weakly-Supervised Segmentation and Unsupervised Modeling of Natural Images
Affiliation IST Austria
Abstract: Modern computer vision systems heavily rely on statistical machine learning models, which typically require large amounts of labeled data to be learned reliably. Moreover, very recently computer vision research widely adopted techniques for representation learning, which further increase the demand for labeled data. However, for many important practical problems there is relatively small amount of labeled data available, so it is problematic to leverage full potential of the representation learning methods. One way to overcome this obstacle is to invest substantial resources into producing large labelled datasets. Unfortunately, this can be prohibitively expensive in practice. In this thesis we focus on the alternative way of tackling the aforementioned issue. We concentrate on methods, which make use of weakly-labeled or even unlabeled data. Specifically, the first half of the thesis is dedicated to the semantic image segmentation task. We develop a technique, which achieves competitive segmentation performance and only requires annotations in a form of global image-level labels instead of dense segmentation masks. Subsequently, we present a new methodology, which further improves segmentation performance by leveraging tiny additional feedback from a human annotator. By using our methods practitioners can greatly reduce the amount of data annotation effort, which is required to learn modern image segmentation models. In the second half of the thesis we focus on methods for learning from unlabeled visual data. We study a family of autoregressive models for modeling structure of natural images and discuss potential applications of these models. Moreover, we conduct in-depth study of one of these applications, where we develop the state-of-the-art model for the probabilistic image colorization task.
Publication Title: IST Dissertation
Degree Granting Institution: IST Austria  
Degree: PhD
Degree Date: 2018-05-25
Start Page: 1
Total Pages: 113
Sponsor: European Unions Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement no 308036.
DOI: 10.15479/AT:ISTA:th_1021
Notes: First of all, I want to thank my supervisor, Christoph. I could not have wished for a better supervisor. Ironically, before applying to Christoph’s group I had read his group web-page and was intrigued by the breadth and quality of his research, but for some reason assumed that Christoph is likely to be a tough person to work with. Luckily, I knew one of his students, who told me that my assumption is wrong. And it was, indeed, as wrong as it gets. I had absolutely positive experience of being a Christoph’s student. Thank you, Christoph, for granting me complete freedom in selecting research topics to work on and for always being available for long detailed discussions regarding my current progress and research obstacles I encountered. Moreover, thank you for introducing me to many bright researches, for gathering a group of extremely talented people, who have taught me a lot, and for maintaining an excellent atmosphere in the group. I also would like to express my gratitude to my committee members, Vittorio and Vladimir. Thank you for providing me with valuable feedback during my qualification exam, for taking time to share your expertise during the course of my PhD and for your thoughtful comments regarding the text of my thesis. Also, I want to additionally thank Vittorio for hosting and supervising me as an intern in his Google Research team. It was a great experience. Perhaps, during my PhD I’ve spent uncountable number of hours with my friends and former office mates, Alex Z, Harald and Michal. I have very vivid memories of our joint activities, such as participating in programming contests, playing football and solving mathematical riddles. Thank you, guys! I also want to acknowledge former and current group members, Georg, Csaba, Emelie, Asya, Mary, Amélie, Nikola, Tomas, Neel, Viktoriia, Saeid, Sylvestre, Nathaniel, Dmitriy, Eela, Chris, Vladislav, Mayu, Sameh for the all fun time we have spent together, and in particular, for our table soccer games. I would like thank to Amélie, with whom I closely collaborated while working on some results from this thesis. Amélie, I was deeply impressed by your skills in doing research, coding, debugging, writing and I have learned a lot from you. I am endlessly grateful to my parents for their support of my decision to pursue a PhD degree and to relocate to a foreign country. Finally, I would like thank my wife, Lera, with whom iv I have shared the most memorable moments and who helped me to withstand the pressure of being a PhD student. This thesis was partially funded by the European Research Council under the European Unions Seventh Framework Programme (FP7/2007-2013)/ERC grant agreement no 308036. I also gratefully acknowledge the support of NVIDIA Corporation with the donation of the GPUs used for this research.
Open access: yes (repository)
IST Austria Authors
Related IST Austria Work