A Knowledge-Based System for Neural Network Design and Explanation

An SPP Project (1994-1996)

Problem Description

The project addresses the intersection of two distinct but related problems. The first problem emerged from an analysis of the state of the art in symbolic/connectionist integration; it consists in devising a new metaprocessing integration scheme which will usefully exploit the reflective capacities of symbolic approaches. The second problem is internal to the connectionist domain and can be stated as follows: Although neural network (NN) technology has been growing steadily since the 1980s, it still faces difficulties that limit its utility in many areas. First, NN development remains a highly empirical process: efficient networks have been obtained mainly through painstaking manual experimentation and often reflect idiosyncrasies of the application task. Second, NNs tend to behave as black boxes: they have no way of explaining why a certain output has been generated from the input data.

This project will attempt to solve both problems concomitantly. On the one hand, intensive NN research over the past decade has given rise to a corpus of collective expertise which might be harnessed to establish NN design on more principled grounds, thereby facilitating NN analysis and explanation. On the other hand, by formalizing this expertise into a symbolic knowledge base (KB) and by incorporating it on the metalevel of a neural network simulator, we will have an effective platform for investigating open problems and issues in symbolic/connectionist integration.

Project Objectives

The goal of this project is to build a precompetitive hybrid system comprising: (1) an environment for the modular construction and composition of neural networks (2) a two-level knowledge-based system: a metalevel theory and methodology of neural networks coupled with a baselevel KB, a domain theory or model needed to adapt NN structure and operation to the application task.

In this framework, the neural net theory is the rule-based representation of three essential aspects of NN development-namely NN design, explanation and learning-seen as the sucessive compilation, extraction and refinement of domain knowledge under the supervision of a general NN theory:

1. NN design as knowledge compilation: Connectionist researchers have come up with increasingly sophisticated techniques for introducing domain-specific knowledge into neural nets. Such techniques include determining the number of hidden layers and/or units, prewiring or pruning explicitly chosen connections, choosing the adequate learning procedure, and designing a modular, possibly heterogeneous, network architecture adapted to the application problem. In other words, designing an NN consists in using relevant knowledge from the application domain to determine the appropriate network structure and learning algorithm. The first objective of this project is to automate this process which remains the object of careful handcrafting by human designers.

2. NN explanation as knowledge extraction: The problem of NN explanation can be stated as follows: how can a network's dynamics and final topology be interpreted in the context of background knowledge to justify its output, preferably in domain-specific terms? This view of explanation in neural nets is predicated on the possibility of attributing a posteriori an explicit meaning to hidden nodes (had these not been given a predefined meaning during network configuration). This possibility is far from evident, and one of the open issues to be resolved in this project is defining precisely those conditions in which hidden units operate as a dynamically changing but humanly intelligible encoding of the network's given task. The analysis of network states in view of extracting internal knowledge and explaining network inferences will be the second major objective of the proposed project.

3. NN learning as knowledge assimilation: If we assume that all learning but the most elementary requires a minimum of background knowledge, the learning cycle in NNs can be described as follows. On the one hand, a preexisting domain theory guides the search for empirical regularities; on the other hand, newly discovered regularities can be used to augment or revise domain theory. Knowledge assimilation refers to the latter aspect of this dialectic. It can be defined as the incremental integration of automatically learned knowledge into a given domain theory or model. Knowledge assimilation is an extremely difficult problem even in pure symbolic knowledge based systems. Project research on this aspect is limited to determining consistency between knowledge generated by neural nets and the current domain model. If a discrepancy is detected, the system alerts the user who will then assume full responsibility for revising domain theory.

Application to Financial Risk Evaluation and Management

The financial domain was chosen as a testbed for symbolic/connectionnist integration because of the number and diversity of application problems that require both the dynamic adaptativity offered by NNs and a fair amount of prior domain-specific knowledge. Examples of financial problems which have been investigated using neural networks are foreign exchange rate prediction, mortgage underwriting and bankruptcy prediction. To our knowledge, however, there is as yet no system that integrates symbolic and connectionist processing in view of decision making in finance or economics.

Within the financial domain, we are working in the specific area of financial risk evaluation and management for the following reasons. First, this area offers a variety of application problems which have been investigated using symbolic or connectionist approaches, and for which a hybrid approach might prove more effective. Second, it is an area in which our financial partners, UNICIBLE, have considerable experience and can offer working domain models to support neural network design and operation. The specific application problem that is currently addressed is credit evaluation. UNICIBLE has agreed to let us avail of the expertise and databases of their risk managers in the banking domain.