OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in an Open World

Zhun Zhong1      Linchao Zhu2      Zhiming Luo3      Shaozi Li3      Yi Yang2      Nicu Sebe1
1. University of Trento            2. University of Technology Sydney            3. Xiamen University

Examples of (a) directly using MixUp among unlabeled samples and (b) the proposed OpenMix. Due to the uncertainty of pseudo-labels of unlabeled samples, their mixed labels may still have low confidence. In OpenMix, the prior knowledge (area of high confidence) leads the mixed label to have high (exactly true) confidence in old classes and medium (reliable) confidence in new classes.

Abstract


In this paper, we tackle the problem of discovering new classes in unlabeled visual data given labeled data from disjoint classes. Existing methods typically first pre-train a model with labeled data, and then identify new classes in unlabeled data via unsupervised clustering. However, the labeled data that provide essential knowledge are often underexplored in the second step. The challenge is that the labeled and unlabeled examples are from non-overlapping classes, which makes it difficult to build a learning relationship between them. In this work, we introduce OpenMix to mix the unlabeled examples from an open set and the labeled examples from known classes, where their nonoverlapping labels and pseudo-labels are simultaneously mixed into a joint label distribution. OpenMix dynamically compounds examples in two ways. First, we produce mixed training images by incorporating labeled examples with unlabeled examples. With the benefit of unique prior knowledge in novel class discovery, the generated pseudo-labels will be more credible than the original unlabeled predictions. As a result, OpenMix helps preventing the model from overfitting on unlabeled samples that may be assigned with wrong pseudo-labels. Second, the first way encourages the unlabeled examples with high class-probabilities to have considerable accuracy. We introduce these examples as reliable anchors and further integrate them with unlabeled samples. This enables us to generate more combinations in unlabeled examples and exploit finer object relations among the new classes. Experiments on three classification datasets demonstrate the effectiveness of the proposed OpenMix, which is superior to state-of-the-art methods in novel class discovery.

Method Overview



The pipeline of the proposed method. (a) We first initialize the model on the labeled data (L ce). (b) Then, we learn the unsupervised clustering model for discovering new classes in unlabeled data, by pseudo-pair learning (L ppl ), pseudo-label learning (L pll ) and learning with the proposed OpenMix (L opm ).

Results




Materials



Paper


Codes

Citation

@inproceedings{zhong2021openmix,
      title={OpenMix: Reviving Known Knowledge for Discovering Novel Visual Categories in An Open World},
      author={Zhong, Zhun and Zhu, Linchao and Luo, Zhiming and Li, Shaozi and Yang, Yi and Sebe, Nicu},
      booktitle={Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2021}
}

}