Deep Relational Graph Infomax for Knowledge Graph Completion

Computer Science Information Science Artificial Intelligence
knowledge graph embedding mutual information maximization graph attention networks knowledge graph completion artificial intelligence

Knowledge Graph (KG) embedding technology is an important research topic in the field of artificial intelligence, mainly used for knowledge acquisition and extension of knowledge graphs. In recent years, although many graph embedding methods have been proposed, these methods typically focus only on the semantic information of the knowledge graph, ignoring its inherent structural information. Thus, despite the Graph Convolutional Networks (GCNs) being able to capture some structural information, they still face the issue of information insufficiency due to the incompleteness of knowledge graphs. To overcome this problem, this study proposes a new model called Deep Relational Graph Infomax (DRGI), which fully utilizes both structural and semantic information through Mutual Information (MI) maximization.

This paper is jointly written by Shuang Liang, Jie Shao, Dongyang Zhang, Jiasheng Zhang from the Future Media Center of the University of Electronic Science and Technology of China, and Bin Cui from the Department of Computer Science at Peking University. It was published in the 2021 IEEE Transactions on Knowledge and Data Engineering journal.

Research Background

A knowledge graph is a semantic network describing entities and their relationships. A typical representation of a knowledge graph is a triple, such as the triple (h, r, t) formed by connecting entity h and t through relationship r. Although existing knowledge graphs already contain a vast number of entities and relationships, there is still a significant amount of incompleteness compared to real-world knowledge. Therefore, the extension of knowledge graphs (knowledge graph completion, inferring missing links) is particularly important.

Current knowledge graph embedding methods, such as linear models or convolutional neural network models, mainly focus on the semantic information in triples, while ignoring structural information. Although GCN methods can utilize structural information of the graph, they still face the problem of information insufficiency caused by the incompleteness of knowledge graphs. Experiments have shown that many knowledge graphs are not connected graphs. For instance, as shown in Figure 2, although Einstein and Nolan have many neighbors, they still do not have a direct relationship. Therefore, relying solely on GCN methods makes it difficult to capture comprehensive structural information.

Research Methods

To address this issue, we propose the DRGI model, which consists of two identical Adaptive Relational Graph Attention Networks (ARGAT), used respectively for capturing semantic information and structural information. The model further uses the Mutual Information (MI) maximization method, aiming to simultaneously capture the graph’s structural and semantic information.

Workflow

  1. Data Preparation: We first convert the input knowledge graph into two types, one for capturing structural information and the other for capturing semantic information.

  2. Encoders:

    • The first encoder captures the graph’s structural information through the ARGAT model. Specifically, use the MI maximization method to maximize between global representations and local representations, thus learning more comprehensive graph structural information.
    • The second encoder captures the graph’s semantic information through supervised learning on known triples.

  3. Decoder: We use a convolutional network as the decoder to compute scores for all candidate entities, used for predicting missing entities.

ARGAT Model

ARGAT utilizes cross-correlation and relationship attention mechanisms, enabling adaptive updating of entity and relationship representations. Specific steps include: 1. For each target node, first combine the characteristics of the neighboring nodes with the relationship characteristics. 2. Calculate an attention score for each neighboring node concerning the target node. 3. Aggregate the information of nodes and edges based on these scores. 4. Dynamically update the hidden representations of edges to achieve adaptive representation learning.

Mutual Information Maximization

The purpose of using MI maximization is to enhance the ability to capture the graph’s global structural information. The specific method is: 1. Calculate MI between local representations and global representations of the graph. 2. Summarize the local representation into a global representation through a readout function. 3. Use a discriminator to judge local-global pairs, treating related pairs as positive samples and unrelated pairs as negative samples, and train the model by maximizing MI.

Experimental Results

We conducted comprehensive experiments on multiple datasets, including FB15k, FB15k-237, WN18, WN18RR, Alyawarra Kinship, and UMLS. Compared with various existing methods, our DRGI model performs excellently on all metrics, particularly on the FB15k-237 and WN18RR datasets, where DRGI outperforms all other models, demonstrating faster convergence speeds and better prediction performance.

Key Contributions

  1. Innovative Model: Proposes a new graph embedding model DRGI, combining structural and semantic information through MI maximization.
  2. Adaptive Encoder: Introduces the ARGAT encoder, achieving adaptive learning of entities and relationships using cross-correlation and relationship attention mechanisms.
  3. Comprehensive Experiments: Experimental results on multiple benchmark datasets demonstrate the effectiveness and efficiency of DRGI, especially in terms of performance on low-degree entities.

Conclusion

The new model DRGI proposed in this paper effectively solves the issue of insufficient structural information in knowledge graphs. Through the MI maximization method, DRGI can not only capture complete structural information but also has faster convergence speeds and better prediction performance. This research provides a new perspective and method for the extension of knowledge graphs and demonstrates the importance of integrating structural and semantic information.

In future work, they will further optimize the complexity of the model and consider how to apply this model to open domain situations, to enhance knowledge graphs by extracting information from other sources such as texts.