----------------------------------
How to use bp-ct (October 1999, August 2000)
----------------------------------

This HOWTO is maintained by Pietro Terna, University of Torino, 
Department of economics and finance, <pietro.terna@unito.it>

bp-ct is a Swarm (version 1.3.1 or 1.4.1 or 2.0.1 or 2.1.1) code, handling 
simple artificial neural networks, trained with the error 
backpropagation (BP) technique. The code applies the Environment-
Rules-Agents (ERA) scheme and it is also capable of generating 
automatically the neural network inputs and targets, mainly to 
apply the Cross Target (CT) technique.

A general introduction to bp-ct, and to the ideas behind it (with 
ERA and CT frameworks), is contained in: Terna, P. (2000), 
Economic Experiments with Swarm: a Neural Network Approach to the 
Self-Development of Consistency in Agents' Behavior, in F. Luna 
and B. Stefansson (eds.), Economic Simulations in Swarm: Agent-
Based Modelling and Object Oriented Programming. Dordrecht and 
London: Kluwer Academic.

------------------------
A few words introduction
------------------------

The CT method has the goal of building agents based upon 
artificial neural networks (that self develop their rules while 
acting and learning). Anyway, adopting the ERA scheme we can 
interchange the techniques in an easy way, so we are planning for 
the future to use Production Systems, Classifier Systems, Genetic 
Algorithms.

With the ERA protocol, we use a specialized class of objects, 
named RuleMaster, to manage the behavior of the agents. A 
ruleMaster (ruleMaster = an instance of RuleMaster) receives all 
the necessary data from each agent, which owns a dataWarehouse 
object. To allow agent heterogeneity, we can have several 
ruleMaster objects. To simplify the code design, the interaction 
with the environment is allowed only to the agents. Another 
specialized class of objects (the RuleMaker class) handles the 
development of the rules.

The code allows us to use simultaneously any number of agents (an 
agent = a neural network) and can be used: (i) as a simple BP 
artificial neural networks training program; (ii) as a BP program 
using internally generated inputs and targets, without a true CT 
scheme or (iii) with the CT scheme.

The bp-ct code shows cases (i) and (ii); the companion ct-hayek 
code is a true CT application, that shows the (iii) case.

NB in this package RuleMaster and RuleMaker are named 
BPCTRuleMaster and BPCTRuleMaker

----------------------
How to use the program
----------------------

-------------------------------------------------------------
(i) bp-ct as a BP artificial neural networks training program
   (Subsection 6.1 of the paper quoted above)
-------------------------------------------------------------

In this case we must provide the files containing training and 
verification data. In the bp-ct package we have, as an example, a 
data.training file and a data.verification file, both containing 
four lines with all the possible inputs and desired outputs 
(targets) of an OrXor function, which uses as input two logical 
values (0=F; 1=T) and as output the Or and the Xor logical 
results. In this example the data set and the validation set are 
exactly coincident, because we are limited to these four 
possibilities, but normally we are employing a verification set 
other than the training one (think, as an example, to the case of 
time series forecasting with neural networks).

Using neural networks, if we apply in the output layer a 
transformation function like the logistic, output and target 
values must be mapped in the range 0-1 (or, better, 0.1-0.9, to 
avoid the necessity of huge negative or positive inputs to the 
function to obtain 0 or 1 as outputs); traditionally this is done 
also for the input values, as a largely adopted convention 
(mapping them in the range 0-1). In our package, this modification 
from the external metrics to the internal one is done via a 
minmax.data file - also needed in the cases (ii) and (iii) - 
containing, in each row, four values: rows identify the inputs and 
then the outputs-targets of the neural network function; the four 
values in each row are the min and the max in the external metrics 
and the min and max (normally 0-1 for the input lines and 0.1-0.9 
for the output-target lines) in the internal one (see the example 
in the package: two rows refer to inputs and have the 0 1 0 1 
form; two refer to the outputs-targets and have the 0 1 0.1 0.9 
form).

To begin using the program, run bp-ct and set the probes to 
ModelSwarm and to ObserverSwarm (the two windows labeled 
ModelSwarm and ObserverSwarm) in the following way:

ModelSwarm

                      useEO_EP 0
                   agentNumber 2
               inputNodeNumber 2
              hiddenNodeNumber 4
              outputNodeNumber 2
patternNumberInVerificationSet 4
    patternNumberInTrainingSet 4
epochNumberInEachTrainingCycle 100
                   epochGroups 0
       agentsAreDisplayingData 0
           readWeightsFromFile 0

ObserverSwarm

    displayErrorGraphInVerificationSet 1
        displayErrorGraphInTrainingSet 1
numberOfTheAgentA_ToBeObservedDirectly 1
                stopAtEpochGroupNumber 10

Explanations:

NB if you change a value in a box displayed in a probe window, 
remember to 'enter' it, by pressing the enter button on the 
keyboard.

(a) useEO_EP can be used only in the (iii) case; see the paper 
cited above to read about the use of this switch in the ct-hayek 
companion code;

(b) agentNumber can be 1 or > 1; in our case we train 
simultaneously two neural networks on the specific task of 
approximating the OrXor function; being the initial parameters 
(the weights, in the neural networks jargon) randomly chosen, the 
results can be slightly different; the possibility of using 
several agent/neural networks is mainly important in type (iii) 
applications;

(c) inputNodeNumber, is here set to 2 as required by the OrXor 
problem; we suggest not to use this setting interactively, but to 
modify it in modelSwarm.m;

(d) hiddenNodeNumber, is here set to 4 to allow more flexibility 
to the neural network (the choice of the right number of hidden 
nodes is always a puzzle); the same suggestion presented above 
applies here;

(e) outputNodeNumber, see (c);

(f) patternNumberInVerificationSet, is here set to 4, accordingly 
to the data.verification file content;

(g) patternNumberInTrainingSet, is here set to 4, accordingly to 
the data.training file content;

(h) epochNumberInEachTrainingCycle is the number of learning steps 
for each cycle run of the program; an epoch is a learning run upon 
the complete training set (with four patterns in our example); 
choosing here epochNumberInEachTrainingCycle = 100 we obtain 100*4 
training steps in each cycle (a cycle is reported as an Epoch 
Group in the label of the X axis of the graphic windows);

(i) epochGroups is set by the program, to display the number of 
epoch groups done until now;

(j) agentsAreDisplayingData, if set to 1 it produces a report on 
the active window; set to 0 to avoid the report;

(k) readWeightsFromFile, if 1 the program reads previously 
generated weights from files (see below); set to 0 to avoid 
reading.

(l) displayErrorGraphInVerificationSet (0 = no; 1 = yes);
(m) displayErrorGraphInTrainingSet (0 = no; 1 = yes);
[The program generates two windows that report error measures, one 
related to training data and the other related to verification 
data; the errors reported are: (1) the conventional 
backpropagation error (sum of the errors in all the output units 
in all the patterns, i.e. epoch error, divided by the number of 
patterns; this measure is also conventionally divided by 2), 
labeled as Mean/Min/Max bp error; (2) the proportional error (this 
error is the ratio obtained dividing the absolute value of the 
difference between each target and each output by half the 
difference between the max potential value and the min potential 
value; this is a raw measure of the error as a proportion of half 
the range of the output values; we have max error = 2; min error = 
0), labeled as Mean/Min/Max prop error. NB: errors in training and 
verification sets are slightly different - particularly in the 
initial epochs - also if the content of the data.training and 
data.verification files are coincident, because the training set 
error evaluation is done while weights are changing.]

(n) numberOfTheAgentA_ToBeObservedDirectly, where we show the 
behavior (in this case the outputs of the neural network vs. the 
targets) of a conventional agent named A (or B, C etc. if we 
provide more windows in ObserverSwarm.m); if we set this parameter 
to 1, we link the agent A with the internal agent 1, etc.; 
choosing 0 we avoid displaying any agent;

(o) stopAtEpochGroupNumber, if set to 0 the program never stops 
(we have to stop it pressing the stop or quit button); if set to a 
value greater than 0, the program stops after n epochGroups and 
save the network parameters (weights). Pressing start again the 
program restarts and never stops nor saves the weights again.

We can save and reuse the trained weights (neural network 
parameters): as seen, if in the ObserverSwarm probe we set a value 
of stopAtEpochGroup > 0, the programs save the weights (and the 
output matrixes, in external metrics); in the successive run, if 
we set in the ModelSwarm probe readWeightsFromFile = 1, the 
program starts reading the previously saved weights.

A remark: the labelling of the X axis is a little intricate; it 
should show the product of the number of epoch groups times the 
number of pattern in the verification set, but, because of an 
internal problem, the labels are shifted of one unit backward: so, 
the last pattern the network has seen is labeled as the product 
above minus 1, the second last as the product above minus 2 and so 
on. We can simply read it as results after n epoch groups, related 
to each pattern in the verification set (in the error window 
related to the training set, the interpretation of the X axis 
label become a little bit more complicated: we have to divide the 
X value - plus 1 - by the number of patterns in the verification 
set to see the number of epoch groups).

To clean the files containing the saved weights, the output 
matrixes and the series reported in the agent's windows from the 
directory in use, run cleanDos or cleanUnix; pay attention in 
modifying the lines of these files, mainly in cleanUnix, because 
the wildcard effect, particularly in Linux/Unix, can be 
devastating (check the modification effects with 'ls' before using 
'rm').

To modify the k parameter in the logistic function f(x) = 1 / (1 + 
e^(-k x)), used as transformation function in our neural network, 
simply correct it in the transf.setup file. This file is read by 
the transFunc object (see the TransFunc.m file).

The bp.setup file contains 
weightRange                 0.3
  (random range in weight Generation, from -0.3 to 0.3)
eps                         0.6
  (learning rate, see BPCTRuleMaker.m))
alpha                       0.9
  (momentum, see BPCTRuleMaker.m; this value is 0.0 in ct-hayek app.)
usingRandomOrderInLearning  1
  (if 1, patterns are used in random order in each epoch; if 0, we
  keep unchanged the sequence)
longTermLearningInCT_OnlyWithCompleteTrainingSet 1
  (see (ii) case)
useOutputsAsTargetsInCT_RelearningScheme 0
  (see (iii) case)

We can easily modify this free form file, delimited with @begin / 
@end

------------------------------------------------------------
(ii) bp-ct as a BP program with internally generated inputs
     and targets, without the CT scheme  (Subsection 6.2 of the
     paper quoted above)
------------------------------------------------------------

General considerations introduced in case (i) apply also here.

In this case we must generate, internally and randomly, the 
logical inputs and targets necessary to train the neural network; 
the pattern generated as 'one pattern' verification set is then 
used as the last in the training set [see (f), (g) below].

To use the program, run bp-ct and set the probes to ModelSwarm and 
to ObserverSwarm (the two windows labeled ModelSwarm and 
ObserverSwarm) in the following way:

NB
patternNumberInVerificationSet = -1 and patternNumberInTrainingSet 
< 0 are mandatory in the cases (ii) and (iii).


ModelSwarm

                      useEO_EP 0
                   agentNumber 3
               inputNodeNumber 2
              hiddenNodeNumber 4
              outputNodeNumber 2
patternNumberInVerificationSet -1
    patternNumberInTrainingSet -10
epochNumberInEachTrainingCycle 100
                   epochGroups 0
       agentsAreDisplayingData 0
           readWeightsFromFile 0

ObserverSwarm

    displayErrorGraphInVerificationSet 1
        displayErrorGraphInTrainingSet 1
numberOfTheAgentA_ToBeObservedDirectly 1
                stopAtEpochGroupNumber 0

We concentrate only on the differences with the case (i) and the 
letters are used in reference to it.

(f) patternNumberInVerificationSet is here set to -1, which means 
internal pattern generation (here without adopting the CT scheme, 
but this is only a matter concerning how the patterns are 
logically built); this setting also means that we use the last 
generated pattern as unique pattern of the verification set and as 
the last pattern in the training set (or also as the unique 
pattern, if the parameter (g) is = -1); [NB if 
patternNumberInVerificationSet is < 0 it must be equal to -1;]

(g) patternNumberInTrainingSet is set to -10 to obtain a moving 
training set, built with the last 10 patterns generated in the 
verification phase;

(h) epochNumberInEachTrainingCycle = 100 repeats one hundred times 
the learning upon the 10 patterns of each training cycle.

In the bp.setup file we have to set to 1 (yes) or 0 (no) the 
parameter longTermLearningInCT_OnlyWithCompleteTrainingSet to 
choose whether to start using the moving training set before its 
completeness (which is obtained only in the cycle corresponding to 
its dimension); choosing 1, until the completeness is obtained, 
the training set contains only the last pattern generated in the 
verification phase.

--------------------------------------------------------------
(iii) bp-ct as a BP program using internally generated  inputs
      and  targets,  with  the  CT  scheme  (Section 7 of  the
      paper quote above); see also the companion code ct-hayek
--------------------------------------------------------------


General considerations introduced in cases (i) and (ii) apply also 
here.

In this case, we must generate, internally and applying the CT 
scheme, the logical inputs and targets required to train the 
neural network; the pattern generated as 'one pattern' 
verification set is then used as the last (or unique) pattern in 
the training set.

We refer here to ct-hayek code: running it as it is, leaving 
unchanged the values contained in the ModelSwarm and ObserverSwarm 
probes, we obtain the first figure of Subsection 7.2 of the paper. 
The other figures can be generated on the basis of the file 
PaperFigures.txt included in the ct-hayek package. (Observe the 
'Valid Prices and Mean Quantity' figure, which is introduced in 
ObserverSwarm.m to display the results related to the 
environment.)

In order to build CT, the use of values of the previous cycles may 
also be necessary; at the starting day, we refer to the contents 
of the init.val file (here consumerInit.val and producerInit.val) 
that report (in order) initial values of inputs and outputs. If we 
omit these files, the code produces some warning, but runs anyway.

In the ct-hayek code the useEO_EP message is used as a switch, in 
the following way [EO = External Objective; EP = External 
Proposal]: 1 = no EO/EP; 2 = some of them; 3 = all. See the file 
PaperFigures.txt and the captions of the figures to understand how 
this works; the files ConsumerInterface.m and ProducerInterface.m 
contain the code used to generate and to choose EO/EP.

In the CT scheme the verification phase can be interpreted as an 
acting phase (see the paper); previous cycle values necessary to 
build targets are extracted from the training set (whose last or 
unique line is a copy of the previous cycle acting-verification 
data).

If we adopt patternNumberInTrainingSet = -1 we have light or short 
term leaning; if we choose patternNumberInTrainingSet < -1 and 
epochNumberInEachTrainingCycle > 1 we have full or long term 
learning. [epochNumberInEachTrainingCycle has to be used carefully 
in CT scheme, in principle avoiding (if we are not sure of what we 
are doing) the following situation:
epochNumberInEachTrainingCycle > 1, when
patternNumberInVerificationSet = -1 and
patternNumberInTrainingSet = -1  ]

In bp.setup, useOutputsAsTargetsInCT_RelearningScheme is a 0/1 
switch; if 0, we use CT targets (normal option) in the repeated 
learning, that is when patternNumberInTrainingSet < -1 and 
epochNumberInEachTrainingCycle > 1; if 1, we use the verification 
phase outputs as targets (this may be useful in particular 
applications).

--------------------------
Building a new application
--------------------------

To build a new application, as ct-hayek, we have to modify the 
following files (starting form those contained in the bp-ct 
package):

main.m
ObserverSwarm.h and ObserverSwarm.m
ModelSwarm.h e ModelSwarm.m
SpecificInterface.h e SpecificInterface.m (also changing the names 
of these files, as in ct-hayek, where we have ConsumerInterface.h, 
ConsumerInterface.m, ProducerInterface.h, ProducerInterface.m).

In the bp-ct package, the lines

// +++++++ start
// +++++++ end

are delimiting the lines of code that we have to change to build a 
different application; the same in ct-hayek package, where the 
modifications with respect to bp-ct package are delimited with the 
lines reported above.
