Abstract
Visual attributes, from simple objects (e.g., backpacks, hats) to soft-biometrics (e.g., gender, height, clothing) have proven to be a powerful representational approach for many applications such as image description and human identification. In this paper, we introduce a novel method to combine the advantages of both multi-task and curriculum learning in a visual attribute classification framework. Individual tasks are grouped after performing hierarchical clustering based on their correlation. The clusters of tasks are learned in a curriculum learning setup by transferring knowledge between clusters. The learning process within each cluster is performed in a multi-task classification setup. By leveraging the acquired knowledge, we speed-up the process and improve performance. We demonstrate the effectiveness of our method via ablation studies and a detailed analysis of the covariates, on a variety of publicly available datasets of humans standing with their full-body visible. Extensive experimentation has proven that the proposed approach boosts the performance by 4%–10%.
1. Introduction
Vision as reception. Vision as reflection. Vision as projection. –Bill Viola, note 1986
When we are interested in providing a description of an object or a human, we tend to use visual attributes to accomplish this task. For example, a laptop can have a wide screen, a silver color, and a brand logo, whereas a human can be tall, female, wearing a blue t-shirt and carrying a backpack. Visual attributes in computer vision are equivalent to the adjectives in our speech. We rely on visual attributes since (i) they enhance our understanding by creating an image in our head of what this object or human looks like; (ii) they narrow down the possible related results when we want to search for a product online or when we want to provide a suspect description; (iii) they can be composed in different ways to create descriptions; (iv) they generalize well as with some finetuning they can be applied to recognize objects for different tasks; and (v) they are a meaningful semantic representation of objects or humans that can be understood by both computers and humans. However, effectively predicting the corresponding visual attributes of a human given an image remains a challenging task [1]. In reallife scenarios, images might be of low-resolution, humans might be partially occluded in cluttered scenes, or there might be significant pose variations.
6. Conclusion
Given a set of tasks that need to be learned we sought to find an answer to how we can learn them effectively and what would be the optimal way in terms of performance, speed, and simplicity. Learning each task separately, although very simple, lacks in terms of performance since it does not exploit the information from other tasks. Learning all tasks at the same time in a multitask classification scenario is relatively fast, easy to implement, and employs knowledge from other tasks to boost the classification performance. Curriculum learning is a learning scheme in which samples or tasks are not treated as equally easy or hard, but are instead presented to the model in a meaningful way so as to increase generalization and performance. Since learning a large number of tasks one at a time is computationally expensive, we opted for learning clusters of tasks in a curriculum. In each cluster of visual attributes, we proposed to learn the corresponding tasks in a multi-task classification setup.
