Connecting Embeddings Based on Multiplex Relational Graph Attention Networks for Knowledge Graph Entity Typing

Computer Science Information Science Artificial Intelligence
Knowledge Graph Entity Typing Graph Attention Networks Embedding Learning Multiplex Relational Graph

Using Connection Embeddings Based on Multi-Relational Graph Attention Networks for Entity Typing in Knowledge Graphs

Research Background

Today, knowledge graphs (KGs) are garnering increasing research interest in various AI-related fields driven by KGs. Large-scale knowledge graphs provide rich and efficient structured information, serving as core data resources for multiple intelligent applications, such as question-answering systems and web searches. Typically, KGs contain numerous entity typing instances presented as tuples ((e, t)), where (e) is the entity and (t) is its hierarchical type. Despite the significant successes of modern knowledge graphs (like Freebase, YAGO, and Google Knowledge Graph), their coverage still remains far from complete and comprehensive. For instance, in the FB15k dataset, 10% of entities with the /music/artist type lack the /people/person type. This critical issue prompts extensive research on knowledge graph entity typing (KGET), which aims to predict missing entity typing instances in KGs, an important subtask of knowledge graph completion (KGC). KGET greatly aids many type-involving downstream algorithms, such as knowledge representation learning, entity linking, relation extraction, and question-answering systems.

Source of the Paper

This paper is authored by Yu Zhao, Han Zhou, Anxiang Zhang, Ruobing Xie, Qing Li (IEEE Member), and Fuzhen Zhuang (IEEE Member). The authors are from the following institutions: Financial Technology Innovation Center, Key Laboratory of Financial Intelligence and Financial Engineering of Sichuan Province, Southwestern University of Finance and Economics, Baidu Inc., School of Computer Science at Carnegie Mellon University, WeChat Search Product Center, Tencent, Institute of Artificial Intelligence, and School of Computer Science and Engineering at Beihang University. This paper was published in the IEEE Transactions on Knowledge and Data Engineering journal in May 2023.

Research Details

Workflow

The research method of this paper involves constructing a heterogeneous relational graph (HRG) and proposing a multi-relational graph attention network (MRGAT) to learn on the HRG, followed by using a connection embedding model (ConnectE) for entity type reasoning. The specific workflow includes the following key steps:

  1. Constructing a Heterogeneous Relational Graph (HRG): To effectively integrate existing heterogeneous structural information in the KG (such as entity typing tuples and entity relationship triples), an HRG containing three different but related semantic subgraphs is constructed. Each subgraph encodes a type of relationship between entities and entity types: entity relationship graph (G_ER), entity type graph (G_E2T), and type relationship graph (G_TRT).

  2. Learning with Multi-Relational Graph Attention Network (MRGAT): MRGAT is used to learn on all subgraphs of the HRG separately, namely: RGAT_ER, RGAT_E2T, and RGAT_TRT. These models can effectively capture neighborhood information by aggregating features of surrounding nodes.

  3. Entity Type Reasoning: Entity type prediction is performed using the ConnectE model on the learned embeddings. ConnectE implements two unique knowledge-driven type reasoning mechanisms, ensuring that the reasoning aligns with the entity typing instances and the triple knowledge in the KG.

Experiments and Results

Experimental results indicate that compared to various state-of-the-art baseline methods, the proposed model achieves significant and consistent improvements in entity type prediction and classification tasks on two benchmark datasets.

Research Findings

  1. Entity Type Prediction: This task aims to infer the missing hierarchical type in incomplete entity typing instances (e.g., (entity, type=?)). The proposed model optimizes this process by encoding the surrounding node features through MRGAT and performing type reasoning using ConnectE. Compared to other state-of-the-art methods, the proposed model achieves the best performance on multiple evaluation metrics.

    • Evaluation Metrics: The model is evaluated using ranking criteria, where the correct type ranking higher indicates superior model performance. Evaluation metrics include Mean Reciprocal Rank (MRR) and Hits@N.
    • Results: The proposed model significantly outperforms traditional models in terms of MRR and Hits@N metrics on the FB15k and YAGO43k datasets, especially excelling in Hits@1.

  2. Entity Type Classification: This task involves constructing artificial negative samples to validate whether the model correctly identifies entity typing instances. The ConnectE model considerably improves over traditional models, with an accuracy boost of about 3.5% on the YAGO43k dataset. Using MRGAT to integrate neighborhood information further enhances classification accuracy.

Research Significance and Value

The main contributions of this research are as follows:

  1. Constructing a Novel Heterogeneous Relational Graph: It effectively integrates existing heterogeneous structural information in the KG, including entity typing tuples and entity relationship triples, providing rich data support for embedding learning.
  2. Proposing the Multi-Relational Graph Attention Network (MRGAT): MRGAT learns on different subgraphs of the HRG, thus fully utilizing neighborhood information and enhancing the performance of representation learning.
  3. Proposing the Connection Embedding Model (ConnectE): Based on the learned embeddings, ConnectE improves entity type prediction accuracy by combining reasoning mechanisms of entity typing instances and relational knowledge.

Conclusion

The method using connection embeddings based on multi-relational graph attention networks exhibits outstanding performance in the field of entity typing. It holds promising potential for various practical applications in knowledge graph completion, knowledge representation learning, and more. In the future, the study plans to incorporate external information and propose a new heterogeneous relational graph attention network (HRGAT) to further enhance the model’s generalizability and accuracy.