In recent years, the use of biologically inspired methods such as the evolutionary algorithm have been increasingly employed to solve and analyze complex computational problems. BELBIC (Brain Emotional Learning Based Intelligent Controller) is one such controller which is proposed by Caro Lucas, Danial Shahmirzadi and Nima Sheikholeslami and adopts the network model developed by Moren and Balkenius to mimic those parts of the brain which are known to produce emotion (namely, the amygdala, orbitofrontal cortex, thalamus and sensory input cortex).
A model is a simplified description of a phenomenon. It brings to life some aspects of this phenomenon while overlooking others. What aspects are kept in the model and what are overlooked greatly depends on the topic of study. Thus, the nature of a model depends on the purpose the investigator plans to carry out. A computational model is one which can be mathematically analyzed, tested and simulated using computer systems. To construct a computational model of emotional learning in the brain requires a thorough analysis of the amygdala and the orbitofrontal cortex and the interaction between them: In mammals, emotional responses are processed in a part of the brain called the limbic system which lies in the cerebral cortex. The main components of the limbic system are the amygdala, orbitofrontal cortex, thalamus and the sensory cortex. The amygdala is an almond shaped area which is placed such that it can communicate with all other cortices within the limbic system. The primary affective conditioning of the system occurs within the amygdala. That is, the association between a stimulus and its emotional consequence takes place in this region. It has been suggested that learning takes place in two fundamental steps. First, a particular stimulus is correlated with an emotional response. This stimulus can be an endless number of phenomena from observing a face, to detecting a scent, hearing a noise, etc. Second, this emotional consequence shapes an association between the stimulus and the response. This analysis is quite influential in part because it was one of the first to suggest that emotions play a key part in learning. In more recent studies, it has been shown that the association between a stimulus and its emotional consequence take place in the amygdala. "In this region, highly analyzed stimulus representations in the cortex are associated with an emotional value. Therefore, emotions are properties of stimuli". The task of the amygdala is thus to assign a primary emotional value to each stimulus that has been paired with a primary reinforcer – the reinforcer is the reward and punishment that the mammal receives. This task is aided by the orbitofrontal complex. "In terms of learning theory, the amygdala appears to handle the presentation of primary reinforcement, while the orbitofrontal cortex is involved in the detection of omission of reinforcement." The first thing we notice in the computational model developed by Moren and Balkenius is that quite a number of interacting learning systems exist in the brain that deal with emotional learning. The computational model is presented below where: Th : Thalamus CX : Sensory Cortex A : Input structures in the amygdala E : Output structures in the amygdala O : Orbitofrontal Cortex Rew/Pun : External signals identifying the presentation of reward and punishment CR/UR : conditioned response/unconditioned response V : Associative strength from cortical representation to the amygdala that is changed by learning W : Inhibitory connection from orbitofrontal cortex to the amygdala that is changed during learning This image shows that the sensory input enters through the thalamus TH. In biological systems, the thalamus takes on the task of initiating the process of a response to stimuli. It does so by passing the signal to the amygdala and the sensory cortex. This signal is then analyzed in the cortical area – CX. In biological systems, the sensory cortex operates by distributing the incoming signals appropriately between the amygdala and the orbitofrontal cortex. This sensory representation in CX is then sent to the amygdala A, through the pathway V. This is the main pathway for learning in this model. Reward and punishment enter the amygdala to strengthen the connection between the amygdala and the pathway. At a later stage if a similar representation is activated in the cortex, E becomes activated and produces an emotional response. O, the orbitofrontal cortex, operates based on the difference between the perceived (i.e. expected) reward/punishment and the actual received reward/punishment. This perceived reward/punishment is the one that has been developed in the brain over time using learning mechanisms and it reaches the orbitofrontal cortex via the sensory cortex and the amygdala. The received reward/punishment on the other hand, comes courtesy of the outside world and is the actual reward/punishment that the specie has just obtained. If these two are identical, the output is the same as always through E. If not, the orbitofronal cortex inhibits and restrains emotional response to make way for further learning. So the path W is only activated in such conditions.Refrence: link
The two-process model of learning described by Mowrer (1973) separates learning into first a stimulus-emotional system that evaluates incoming stimuli, and a second learning system that uses this evaluation as a reinforcer for stimulus-response learning. Among the advantages of this approach is that the motivation to respond and the response itself are cleanly separated (Rolls, 1986). We believe that the amygdalo-orbitofrontal system implements the evaluative functionality of such a system. We have attempted to capture these features in a computational model suitable for comparisons between neurophysiological data and simulations. We hope that this approach will enable us to attain a clearer understanding both of the functions of the amygdala and of the limitations of the model; this would have been difficult to accomplish with a model that is not testable in simulation. The model described below is of course only a part of a complete emotional system; specifically, this model does not fully address context, configurational stimuli or higher-order conditioning. An extension with an accompanying hippocampal model and a revised orbitofrontal model will be described in the next chapter. As discussed in chapter 1, a motivation for having an emotional system instead of only a goal generating system is that emotions are a more flexible way to generate motivations. Whereas a goal-directed system tends to specify what to be done, and in some cases how to do it, an emotional system only points out good or bad features, leaving the system to devise ways of coping with the situation. As part of this investigation, we have implemented a computational model of the amygdala 1 and the orbitofrontal cortex, and tested it in simulation. This is not a detailed physiological model, even though it shares its larger-scale structure with that of the real amygdalo-orbitofrontal system; instead, the aim is to make use of neurophysiological data to construct a functional model of emotional processing as part of a general learning system. This system can in turn be used as a learning component in autonomous systems.
The model is divided into two parts, conceptually corresponding to the amygdala and the orbital frontal cortex, respectively. Of course, these areas are complex, and we have not in any way attempted to capture all of their functionality. The amygdaloid part receives inputs from the thalamus and from cortical areas, while the orbital part receives inputs from the cortical areas and the amygdala only.
This part of the model attempts to capture a few aspects of learning in the amygdala. It is very much like the Rescorla-Wagner model discussed in chapter 2, with nodes learning associations controlled by a reinforcement signal defined as a difference between expected and actual reinforcement.
The second part of the system is a model of the orbitofrontal cortex. This area is, as we have seen, responsible for inhibiting inappropriate reactions from other areas, including the amygdala. The OFC needs much the same data as the amygdala: stimuli and reinforcer. In addition, this design presupposes a data path from the amygdala to the OFC indicating the current emotional evaluation of the amygdala to the present stimuli. This is so the OFC will have a target to inhibit. The OFC model is similar to the amygdala model; it also adapts its output according to the sensory data S and the reinforcer R. An added wrinkle is that the output is used to inhibit the amygdala, so the OFC is using the output of the amygdala as another parameter to determine the level at which the inhibition should be applied. Note that we are not arguing that it is inhibitory at the synaptic level. Instead, what we are saying is that the functional effect of this connection is such that it acts to block the response of the amygdala.
Refrence: Emotion and Learning: A Computational Model of the Amygdala (Jan Mor´en) - Lund University Cognitive Studies 93 - 2002 full-text: link
> Developed by: Morteza Karimian Kelishadrokhi
> Send your feedback on: M.Karimian2010 [at] Yahoo.com
> Download: Brain Emotional Learning Algorithm (BEL ).rar
The brain emotional learning control paradigm has been gathering increased interest by the control systems design community. Hence, this toolbox can be used by the researcher to reduce the development time of BELBIC based controllers. Comments and suggestions can be sent to email@example.com
> version 1.0 (337 KB) by João Paulo Coelho
> Brain Emotional Learning Toolbox
> Refrence: mathworks