Knowledge Graph Completion by Jointly Learning Structural Features and Soft Logical Rules

Computer Science Information Science Artificial Intelligence
knowledge graph completion knowledge graph embedding graph attention neural networks link prediction soft logical rules

In recent years, Knowledge Graphs (KG) have been widely used in many artificial intelligence tasks. Knowledge graphs represent entities and their relationships using triplets consisting of a head entity, a relation, and a tail entity. For example, the triplet (h = Paris, r = capital_of, t = France) represents a common-sense fact about the real world. Although knowledge graphs have become an important resource for various downstream AI applications such as intelligent question answering, entity disambiguation, semantic network search, and fact verification, existing knowledge graphs are not perfect and often suffer from missing relationships or errors. To address these issues, Knowledge Graph Embedding (KGE) has become a primary task, which aims to predict missing facts by learning the structural information and latent logical rules in the graph.

Research Background

Although existing KGE models have improved the completeness of knowledge graphs to a certain extent, they still face two unresolved challenges: (i) how to simultaneously utilize local structural features of entities and potential soft logical rules to learn more expressive entity and relation embeddings; (ii) how to integrate these two learning processes into a unified model to achieve better performance.

To solve these problems, this paper proposes a novel KGE model named JSSKGE, which can jointly learn the local structural features of entities and soft logical rules. Experiments on multiple datasets have shown that it outperforms the current state-of-the-art methods.

Source of the Paper

The authors of this paper, Weidong Li, Rong Peng, and Zhi Li, are from the School of Computer Science at Wuhan University and the School of Computer and Information Engineering at Guangxi Normal University, respectively. This study was published on August 30, 2021, in Volume 35, Issue 3 of IEEE Transactions on Knowledge and Data Engineering.

Research Workflow

This study includes several main steps, each supported by specific experiments or data processing methods:

1. Structural Feature Learning

First, Graph Attention Networks (GATs) are used to aggregate the local structural information of nodes. This network is specifically designed for handling graph-structured data and can automatically learn the importance of each node’s neighbors, thereby more accurately representing the structural features of entities.

2. Utilization of Soft Logical Rules

Soft logical rules embedded in the knowledge graph are used as experts to further refine the embeddings of entities and relations. Unlike hard logical rules, soft logical rules allow for some exceptions and can be automatically obtained using association rule mining tools such as AMIE, thereby avoiding the high cost of manual rule construction.

3. Joint Learning

By jointly learning the graph attention neural network and soft logic rules, embeddings that contain more information are obtained for predicting new facts. The proposed JSSKGE model integrates the structural information of entities and soft logical rules, and experimental results on four commonly used datasets outperform current state-of-the-art methods.

Experiments and Results

Extensive experimental evaluations of JSSKGE were conducted on four typical datasets, including FB15k, WN18, FB15k-237, and WN18RR. These datasets cover knowledge graphs of different domains and scales:

  • FB15k: A subset of the Freebase knowledge graph, containing a large number of facts about movies, actors, sports, and awards.
  • WN18: A subset of WordNet, primarily containing content related to intuitive dictionary use and synonym sets.
  • FB15k-237 and WN18RR: Subsets of FB15k and WN18, respectively, aimed at eliminating test leakage issues, making the models more challenging and reliable.

Model Evaluation Metrics

Commonly used metrics for evaluating the final performance of models include Mean Rank (MR), Mean Reciprocal Rank (MRR), and Hits@K (the proportion of prediction results in the top K). Specifically, filtered evaluation metrics are adopted to avoid abnormal prediction rankings due to related candidate entities present in the training or validation set.

Experimental Results

Results on the FB15k and WN18 datasets show that the JSSKGE model outperforms most existing models on multiple metrics such as Hits@10, Hits@3, and Hits@1, especially excelling on the WN18 dataset. Additionally, on the more challenging FB15k-237 and WN18RR datasets, the JSSKGE model still demonstrates high robustness and effectiveness:

  • FB15k-237: On the MRR, Hits@10, Hits@3, and Hits@1 metrics, the JSSKGE model outperforms various benchmark models, verifying the feasibility of jointly learning structural features and soft logical rules.
  • WN18RR: Although the AnyBURL-EXT model performs better on this dataset, the JSSKGE model still shows significant advantages in competing with other models.

Analysis of Parameter Influence

Experiments adjusting hyperparameters further verify the relative importance of structural feature learning and soft logical rules in the JSSKGE model. Results show that using either method alone cannot achieve optimal results, and only joint learning can fully leverage the advantages of both.

Visualization of Attention Weights

Visualization of the attention weight changes in the graph attention layer of the model shows that the model increasingly focuses on certain specific entity connections to optimize prediction results, further validating the important role of the attention mechanism in the learning process.

Conclusion

The proposed JSSKGE model effectively improves the expressiveness and prediction accuracy of knowledge graph embeddings by jointly learning structural features and soft logical rules. However, the model needs improvement in computational cost and space resource usage. Future work will continue to explore more efficient structural feature learning methods and introduce more complex logical rules to further enhance model performance.