Exploiting Instance-Label Dynamics through Reciprocal Anchored Contrastive Learning for Few-Shot Relation Extraction

Academic Background

In the field of Natural Language Processing (NLP), Relation Extraction (RE) is a fundamental task aimed at identifying and extracting relationships between entities in text. However, traditional supervised learning methods rely heavily on large amounts of annotated data, and in practical applications, the scarcity of annotated data severely limits model performance. To address this challenge, Few-Shot Relation Extraction (FSRE) has emerged, aiming to train models with limited annotated data so that they can accurately identify entity relationships with few samples.

In recent years, Pre-trained Language Models (PLMs) have made significant progress in FSRE tasks, especially when combined with Contrastive Learning (CL), which effectively leverages the dynamic relationships between instances and labels. However, existing methods still fall short in fully utilizing instance-label pairs to extract rich semantic representations. To this end, this paper proposes a framework based on Reciprocal Anchored Contrastive Learning (RACL), aiming to further enhance the performance of FSRE tasks through multi-view contrastive learning.

Source of the Paper

This paper is co-authored by Yanglei Gan, Qiao Liu, Run Lin, Tian Lan, Yuxiang Cai, Xueyi Liu, Changlin Li, and Yan Liu from the School of Computer Science and Engineering at the University of Electronic Science and Technology of China. The paper was published in 2025 in the journal Neural Networks, titled Exploiting Instance-Label Dynamics through Reciprocal Anchored Contrastive Learning for Few-Shot Relation Extraction.

Research Process and Results

1. Research Process

a) Design of the Reciprocal Anchored Contrastive Learning Framework

The core idea of the RACL framework is to enhance the model’s understanding of semantic relationships through reciprocal anchored contrastive learning between instances and labels. Specifically, RACL adopts a symmetric contrastive objective, combining instance-level and label-level contrastive losses to promote the unity and consistency of the representation space. The framework is divided into two stages: pre-training and fine-tuning.

• Pre-training Phase: RACL uses two independent BERT models as the label encoder and sentence encoder, respectively. By processing sentence-label pairs, the model generates high-dimensional semantic representations. The pre-training tasks include Reciprocal Contrastive Learning (RCL) and Masked Language Modeling (MLM). RCL optimizes the representation space by maximizing the cosine similarity of correct sentence-label pairs while minimizing the similarity of incorrect pairs.

• Fine-tuning Phase: In the fine-tuning phase, RACL combines the sentence and label representations obtained from pre-training to generate hybrid prototypes for relation classification. By introducing a symmetric contrastive loss, RACL further optimizes the discriminability of the prototypes, enabling them to better distinguish semantically similar relations.

b) Datasets and Experimental Setup

RACL was tested on two benchmark datasets: FewRel 1.0 and FewRel 2.0. FewRel 1.0 contains 70,000 instances and 100 relation types, while FewRel 2.0 builds upon FewRel 1.0 by introducing a test set from the biomedical domain and adding a “None of the Above” (NOTA) category. The experiments were conducted under four few-shot settings: 5-way-1-shot, 5-way-5-shot, 10-way-1-shot, and 10-way-5-shot.

2. Main Results

a) Few-Shot Relation Extraction Performance

RACL achieved significantly better results than existing methods on both the FewRel 1.0 and FewRel 2.0 datasets. On the FewRel 1.0 test set, RACL achieved the highest accuracy in the 5-way-1-shot, 5-way-5-shot, and 10-way-5-shot settings, with rates of 95.59%, 96.82%, and 96.19%, respectively. In the cross-domain testing of FewRel 2.0, RACL led in the 5-way-1-shot and 10-way-1-shot settings with accuracies of 81.80% and 72.48%, demonstrating its strong domain adaptation capabilities.

b) Effectiveness of Reciprocal Anchored Contrastive Learning

By comparing the feature distributions of different pre-training methods, RACL exhibited more compact and consistent clustering, indicating its ability to better align instance and label representations. Additionally, RACL maintained high performance when combined with other pre-training methods such as MAPRE and LPD, further validating the unique advantages of its reciprocal anchored contrastive learning.

c) Zero-Shot Relation Extraction Performance

In the Zero-Shot Relation Extraction (ZSRE) task, RACL also performed exceptionally well. On the FewRel 1.0 validation set, RACL achieved accuracies of 73.50% and 58.90% in the 5-way-0-shot and 10-way-0-shot settings, respectively, significantly outperforming other methods.

3. Conclusions and Significance

The RACL framework significantly improves the performance of few-shot relation extraction tasks by introducing reciprocal anchored contrastive learning. Its core contributions include: - Multi-View Contrastive Learning: RACL captures semantic relationships more effectively through reciprocal anchored contrastive learning between instances and labels, enhancing the model’s discriminative ability. - Symmetric Contrastive Loss: By introducing a symmetric contrastive loss, RACL ensures the consistency of instance and label representations, strengthening the model’s generalization capabilities. - Cross-Domain Adaptation: RACL demonstrated outstanding performance in the cross-domain testing of FewRel 2.0, showcasing its robustness in complex scenarios.

4. Research Highlights

• Innovative Method: RACL is the first to apply reciprocal anchored contrastive learning to few-shot relation extraction tasks, significantly improving model performance through multi-view contrastive learning.
• Broad Applicability: RACL not only excels in few-shot settings but also demonstrates strong adaptability in zero-shot and cross-domain tasks.
• Open-Source Code and Models: The research team has made the pre-training code and models of RACL publicly available, providing valuable resources for future research.

Summary

The RACL framework provides a novel and efficient solution for few-shot relation extraction tasks through reciprocal anchored contrastive learning. Its multi-view contrastive learning strategy and symmetric contrastive loss design significantly enhance the model’s semantic understanding and generalization capabilities. In the future, RACL is expected to be applied to more NLP tasks, further advancing the field of few-shot learning.