Do neuro-symbolic systems improve the performance of deep neural networks?

Shalom Lappin

A neuro-symbolic AI model adds a symbolic representational component or a rule system to a deep neural network (DNN). A variety of such models have been developed in the past fifteen years, involving different sorts of architectures for integrating symbolic content into the computational operations of DNNs. The following are examples of such systems. Tree structures have been added to LSTMs and first generation transformers for NLP tasks. Researchers have added inductive logic programming to a CNN to infer rules that apply to the visual features that the CNN generates, for complex image recognition. LLMs have been enriched with modules for encoding natural language descriptions of NP-hard planning problems in Python, and then applying symbolic solvers to the formalised versions of these problems. I will consider the architectures of these models, and look at their effectiveness compared to non-symbolically enriched counterpart DNNs. The currently available evidence for the success of these models in significantly improving performance for the tasks to which they are applied is, at best, equivocal. This is particularly the case with grammar enriched DNNs. Much additional work is needed to clarify the properties and effectiveness of neuro-symbolic systems, and I will briefly describe possible lines of future research.