Textbook in PDF format This book explores various aspects of knowledge graph reasoning to solve different tasks, encompassing first, traditional symbolic methods for knowledge graph reasoning; second, recent developments in neural-based knowledge graph reasoning techniques; and third, cutting-edge advancements in neural-symbolic hybrid approaches to knowledge graph reasoning. The authors focus on the model and algorithm design aspect and study knowledge graphs from two perspectives: background knowledge graph and input query. Knowledge graph reasoning, which aims to infer and discover new knowledge from existing information in the knowledge graph, has played an important role in many real-world applications, such as question answering and recommender systems. A new trend in knowledge graph reasoning is the combination of neural models with symbolic knowledge graphs, allowing for the design of models that are not only efficient and accurate, but also interpretable. In this book, the authors study the application of neural-symbolic knowledge reasoning to different tasks from two perspectives: the input query and the background knowledge graph. Artificial Intelligence (AI) has been transforming the way we live, work, and interact with the world. Among the many emerging directions of AI, Neural-Symbolic AI has gained significant attention in recent years. By combining the representational power of deep learning with the logical rigor of symbolic reasoning, it promises the next generation of AI systems that are more explainable, trustworthy, and versatile. Such systems are expected to revolutionize a wide range of high-impact applications, including code generation, question answering, and drug discovery. To fully unleash the potential of neural-symbolic reasoning, a fundamental challenge lies in how symbolic knowledge is represented and integrated with neural models. Among various approaches, knowledge graphs (KGs) have emerged as a powerful and versatile framework for organizing and connecting real-world information. Acknowledge graph represents a collection of facts in the form of triples, each consisting of a subject, predicate, and object. This structured representation captures semantic relationships among entities and concepts, enabling machines to perform reasoning and inference over interconnected knowledge. Knowledge graphs have been successfully applied in diverse areas such as search engines, recommender systems, and knowledge graph question answering (KGQA). Despite the superb performance in many tasks, large language models (LLMs) bear the risk of generating hallucination or even wrong answers when confronted with tasks that demand the accuracy of knowledge. The issue becomes even more noticeable when addressing logic queries that require multiple logic reasoning steps. On the other hand, knowledge graph (KG) based question answering methods are capable of accurately identifying the correct answers with the help of knowledge graph, yet its accuracy could quickly deteriorate when the knowledge graph itself is sparse and incomplete. It remains a critical challenge on how to integrate knowledge graph reasoning with LLMs in a mutually beneficial way so as to mitigate both the hallucination problem of LLMs as well as the incompleteness issue of knowledge graphs. In this chapter, we introduce ‘Logic-Query-of-Thoughts’ (LGOT) which combines LLMs with knowledge graph based logic query reasoning. LGOT seamlessly combines knowledge graph reasoning and LLMs, effectively breaking down complex logic queries into easy to answer subquestions. Through the utilization of both knowledge graph reasoning and LLMs, it successfully derives answers for each subquestion. By aggregating these results and selecting the highest quality candidate answers for each step, LGOT achieves accurate results to complex questions. Large language models (LLMs) have exhibited remarkable performance across various natural language processing (NLP) tasks, including question answering, machine translation, text generation, recommender system and so on. Recent increases in the model size have further enhanced the learning capabilities of LLMs, propelling their performance to new heights. Notable examples of these advanced models include GPT-4 and Llama 2, among others. The primary goal of knowledge graph reasoning is to derive meaningful insights from the available data—discovering hidden patterns, explaining existing knowledge, or inferring new information from known facts. Reasoning over KGs using neural network models naturally embodies the spirit of neural-symbolic AI, where symbolic structures guide logical reasoning while neural networks contribute generalization and learning capabilities. Together, these developments form a comprehensive framework for understanding and advancing neural-symbolic reasoning on knowledge graphs, contributing toward the long-term goal of building AI systems that are not only intelligent but also transparent, interpretable, and trustworthy. Preface Introduction Introduction Literature Review Knowledge Graph Reasoning Accurate Query Answering with Symbolic Reasoning Over Complete KGs Symbolic Reasoning for Inconsistency Detection Over Complete KG Accurate Query Answering with Neural Symbolic Reasoning Over Incomplete KG Accurate Query Answering with LLMs Over Incomplete KG Ambiguous Query Answering with Neural Symbolic Reasoning Over Incomplete KG Ambiguous Entity Matching with Neural Symbolic Reasoning Over Incomplete KG Dynamic Query Answering with Neural Symbolic Reasoning Over Incomplete KG Summary Conclusion and Future Directions References