Neighborhood Contrastive Learning for Novel Class Discovery

Zhun Zhong1      Enrico Fini1      Subhankar Roy1,3      Zhiming Luo2      Elisa Ricci1,3      Nicu Sebe1
1. University of Trento            2. Xiamen University            3. Fondazione Bruno Kessler

Illustration of novel class discovery (NCD) and the proposed neighborhood contrastive learning (NCL).


In this paper, we address Novel Class Discovery (NCD), the task of unveiling new classes in a set of unlabeled samples given a labeled dataset with known classes. We exploit the peculiarities of NCD to build a new framework, named Neighborhood Contrastive Learning (NCL), to learn discriminative representations that are important to clustering performance. Our contribution is twofold. First, we find that a feature extractor trained on the labeled set generates representations in which a generic query sample and its neighbors are likely to share the same class. We exploit this observation to retrieve and aggregate pseudo-positive pairs with contrastive learning, thus encouraging the model to learn more discriminative representations. Second, we notice that most of the instances are easily discriminated by the network, contributing less to the contrastive loss. To overcome this issue, we propose to generate hard negatives by mixing labeled and unlabeled samples in the feature space. We experimentally demonstrate that these two ingredients significantly contribute to clustering performance and lead our model to outperform state-of-the-art methods by a large margin (e.g., clustering accuracy +13% on CIFAR-100 and +8% on ImageNet).

Method Overview

The proposed neighborhood contrastive learning framework for novel class discovery. Given training images sampled from the labeled and the unlabeled data, we forward them into the network to obtain corresponding representations. For the labeled data, the CE loss, CS loss and the proposed NCL loss are calculated with the ground-truth labels. For the unlabeled data, BCE loss and CS loss are computed to optimize the new classifier while the NCL loss is proposed to learn discriminative representation. CE: cross-entropy, BCE: binary cross-entropy, CS: consistency, NCL: neighborhood contrastive learning, HNG: hard negative generation.






      title={Neighborhood Contrastive Learning},
      author={Zhong, Zhun and Fini, Enrico and Roy, Subhankar and Luo, Zhiming and Ricci, Elisa and Sebe, Nicu},
      booktitle={Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},