EXplainable Neural-Symbolic Learning X-NeSyL methodology to fuse deep learning representations with expert knowledge graphs: The MonuMAI cultural heritage use case

Neurosymbolic AI: the 3rd wave Artificial Intelligence Review

For example, deep learning systems are trainable from raw data and are robust against outliers or errors in the base data, while symbolic systems are brittle with respect to outliers and data errors, and are far less trainable. It is therefore natural to ask how neural and symbolic approaches can be combined or even unified in order to overcome the weaknesses of either approach. Traditionally, in neuro-symbolic AI research, emphasis is on either incorporating symbolic abilities in a neural approach, or coupling neural and symbolic components such that they seamlessly interact [2]. In the history of the quest for human-level artificial intelligence, a number of rival paradigms have vied for supremacy. Symbolic artificial intelligence was dominant for much of the 20th century, but currently a connectionist paradigm is in the ascendant, namely machine learning with deep neural networks. However, both paradigms have strengths and weaknesses, and a significant challenge for the field today is to effect a reconciliation.

The issue is that in the propositional setting, only the (binary) values of the existing input propositions are changing, with the structure of the logical program being fixed. It has now been argued by many that a combination of deep learning with the high-level reasoning capabilities present in the symbolic, logic-based approaches is necessary to progress towards more general AI systems [9,11,12]. With this paradigm shift, many variants of the neural networks from the ’80s and ’90s have been rediscovered or newly introduced. Benefiting from the substantial increase in the parallel processing power of modern GPUs, and the ever-increasing amount of available data, deep learning has been steadily paving its way to completely dominate the (perceptual) ML.

Access Paper:

Other non-monotonic logics provided truth maintenance systems that revised beliefs leading to contradictions. Marvin Minsky first proposed frames as a way of interpreting common visual situations, such as an office, and Roger Schank extended this idea to scripts for common routines, such as dining out. Cyc has attempted to capture useful common-sense knowledge and has “micro-theories” to handle particular kinds of domain-specific reasoning. We note that this was the state at the time and the situation has changed quite considerably in the recent years, with a number of modern NSI approaches dealing with the problem quite properly now.

However, interestingly, even the modern idea of deep learning was not originally described as bound to the neural networks only, but rather universally to “methods modeling hierarchical composition of useful concepts”, that are reused in different inference paths of target variables from the input samples [15]. We believe that our results are the first step to direct learning representations in the neural networks towards symbol-like entities that can be manipulated by high-dimensional computing. Such an approach facilitates fast and lifelong learning and paves the way for high-level reasoning and manipulation of objects.

Future directions

While the particular techniques in symbolic AI varied greatly, the field was largely based on mathematical logic, which was seen as the proper (“neat”) representation formalism for most of the underlying concepts of symbol manipulation. With this formalism in mind, people used to design large knowledge bases, expert and production rule systems, and specialized programming languages for AI. For other AI programming languages see this list of programming languages for artificial intelligence. Currently, Python, a multi-paradigm programming language, is the most popular programming language, partly due to its extensive package library that supports data science, natural language processing, and deep learning.

• Interestingly, we note that the simple logical XOR function is actually still challenging to learn properly even in modern-day deep learning, which we will discuss in the follow-up article.
• Examples of common-sense reasoning include implicit reasoning about how people think or general knowledge of day-to-day events, objects, and living creatures.
• By neural we mean approaches based on artificial neural networks—sometimes called connectionist or subsymbolic approaches—and in particular this includes deep learning, which has provided very significant breakthrough results in the recent decade, and is fueling the current general interest in AI.
• A more flexible kind of problem-solving occurs when reasoning about what to do next occurs, rather than simply choosing one of the available actions.
• Limitations were discovered in using simple first-order logic to reason about dynamic domains.
• DOLCE is an example of an upper ontology that can be used for any domain while WordNet is a lexical resource that can also be viewed as an ontology.

Graphplan takes a least-commitment approach to planning, rather than sequentially choosing actions from an initial state, working forwards, or a goal state if working backwards. Satplan is an approach to planning where a planning problem is reduced to a Boolean satisfiability problem. Forward chaining inference engines are the most common, and are seen in CLIPS and OPS5. Backward symbolic learning chaining occurs in Prolog, where a more limited logical representation is used, Horn Clauses. Programs were themselves data structures that other programs could operate on, allowing the easy definition of higher-level languages. This section provides an overview of techniques and contributions in an overall context leading to many other, more detailed articles in Wikipedia.

(3) We discuss the applications of neural-symbolic learning systems and propose four potential future research directions, thus paving the way for further advancements and exploration in this field. The concept of neural networks (as they were called before the deep learning “rebranding”) has actually been around, with various ups and downs, for a few decades already. It dates all the way back to 1943 and the introduction of the first computational neuron [1]. Stacking these on top of each other into layers then became quite popular in the 1980s and ’90s already. However, at that time they were still mostly losing the competition against the more established, and better theoretically substantiated, learning models like SVMs.

Metadata that augments network input is increasingly being used to improve deep learning system performances, e.g. for conversational agents. Metadata are a form of formally represented background knowledge, for example a knowledge base, a knowledge graph or other structured background knowledge, that adds further information or context to the data or system. In its simplest form, metadata can consist just of keywords, but they can also take the form of sizeable logical background theories. Neuro-symbolic lines of work include the use of knowledge graphs to improve zero-shot learning.

Neurosymbolic AI in Cybersecurity: Bridging Pattern Recognition and Symbolic Reasoning

DOLCE is an example of an upper ontology that can be used for any domain while WordNet is a lexical resource that can also be viewed as an ontology. YAGO incorporates WordNet as part of its ontology, to align facts extracted from Wikipedia with WordNet synsets. A key component of the system architecture for all expert systems is the knowledge base, which stores facts and rules for problem-solving.[52]

The simplest approach for an expert system knowledge base is simply a collection or network of production rules. Production rules connect symbols in a relationship similar to an If-Then statement. The expert system processes the rules to make deductions and to determine what additional information it needs, i.e. what questions to ask, using human-readable symbols. For example, OPS5, CLIPS and their successors Jess and Drools operate in this fashion.

Such transformed binary high-dimensional vectors are stored in a computational memory unit, comprising a crossbar array of memristive devices. A single nanoscale memristive device is used to represent each component of the high-dimensional vector that leads to a very high-density memory. The similarity search on these wide vectors can be efficiently computed by exploiting physical laws such as Ohm’s law and Kirchhoff’s current summation law. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy.

Section 5 discusses the future research directions, after which Section 6 concludes this survey. In this line of effort, deep learning systems are trained to solve problems such as term rewriting, planning, elementary algebra, logical deduction or abduction or rule learning. These problems are known to often require sophisticated and non-trivial symbolic algorithms.