This section describes how to use NN4. I typical compile with modest optimization and reduce range, IO checking etc. I have found it to be almost twice as fast. Anyhow, NN4 is command line program, so start it in CMD box on windows. (Being in Free Pascal can be compiled for other OS). I find it best to arrange folders where each folder relates to experiment of some nature. And in the folder have NN4 and of course, at least one configuration file with weights and one data file and perhaps log file if logging. The three modes of NN4 are: (1) randomiseweights, (2) run and (3) learn. Command lien would look like this:
nn4 aprelu l3 n2,5,1, v4 wcfg.txt fdata.txt glog.txt tyes e0.001 c10000 s10000 r0.1 m0.5 b1 d0.1 k-1,1 olearn
full explanation of each parameter can be found on help page, NN$ with no parameters.
The key is to run NN4 in 3 different modes: (1) rnd – randomises weights, (2) run – which uses current weights to run through data file and show errors and (3) learn – which loops using defiend activaiton fucniton to learn weigths based on prvoided data.
Rnd objective will write weights in v# precision (v4 or v8, dont forget v otherwise you get integers), to file specified after parameter “w” and weights are generated with native OS random function in range of RNDMIN (inclusive) and RNDMAX (exclusive) specified by paramter “k”. RNDMIN must be < RNDMAX. The weights are created based on network defintion, e.g. layers after parameter “l” and neurons count after n#,#,… Limits are based on machine memory – certainly values of 100 – 200 are possible with 10 – 20 deep layers.
Typical usage would be prepare cfg to match model you want and data you have (data would have been normalised etc). randomize, and then run and observer outcomes. now you have starting point. now learn… and again etc and then run to see new outcomes.
you can while learning start with one data line, and display at every cycle. give it a try. below are samples of config file and samples of a data file in precision 4.
44 ASCII char number of delimiter used here after. 32=space or 44=comma (,) comma useful for xls
3000000,Max cycles to run before terminating learning, common values 1k – 100k
10000,How often to display progress in learn mode
0.00100000,Max error allowed before terminating learning, common values 1 (for large data sets, or even more) to 0.00000001
4,Precision to use in display, either 4 or 8
3,Number of layers, including first input and last output, example 3, min=3 max=20
3,7,1, Number of neurons per layer, including first as leftmost and last as right most, example 2 3 1
0.00100000,Learning rate, common values 0.000001 – 1.0
0.80000001,Momentum, common values 0.001 – 1.0 – recommend start 0.5 – 0.9
0.50000000,Sigmoid Beta values 0.1 – 100.0 – recommend between 1 and 2 for swish or 1 for Sigmoid
Sigmoid, Activation Function value one of these: Sigmoid, TanH, SoftPlus or Swish
-38.4422,-50.0333,34.6250,29.3865,42.4321,30.5373,-25.4449,-12.9188,-28.1659,40.2626,12.1451,32.0000,5.7353,11.7389,-13.1852,12.8091,40.5617,-0.0667,32.0000,-38.0355,7.5241,-8.4397,10.7501,-22.4326,-30.2322,-57.8663,6.8967,18.0125,
-9.3369,15.6179,16.0000,-3.5997,-14.3672,8.6653,-4.7546,-11.4359,
when config is written back the comments you see above are refreshed.
** Note if using notepad like i do to tweak config file – make sure you close it. Otherwise, at next tweak you will ave prior versions of weights.
Explanation of each parameters is found in various sections of this site.
The above data file example is from mixed logic gates experiment. Input one is one of 5 values in the range -1 to 1. (1) -0.8 is gate AND, (2) -0.4 is OR etc. Input 2 and 3 are Gate inputs and Target 0 is output for the gate. Actual is Neural Net Output. Neural net used is config file above.
As you can see Actuals are not too far from Targets. But see more under experiments page.
Here is once more complete sequence of usage
First we randomise the weights (assume i have prepared config file neural net type / size for the problem at hand)
Now we can notepad the cfg.txt to see random weights and other paramters. Note i set display after every 10 cycles (on full dataset) and stop after max 100 cycles.
Now lets prepare data file. see below. i am simulating 5 different logic gate types.
AND gate is -0.8 in first column, followed by
0,0 = 0
0,1 = 0
1,0 = 0
1,1 = 1
OR gate is -0.4 followed by
0,0 = 0
0,1 = 1
1,0 = 1
1,1 = 1
NAND gate is 0.0 followed by
0,0 = 1
0,1 = 1
1,0 = 1
1,1 = 0
XOR gate is +0.4 followed by
0,0 = 0
0,1 = 1
1,0 = 1
1,1 = 0
NOR gate is +0.8 followed by
0,0 = 1
0,1 = 0
1,0 = 0
1,1 = 0
20, count of lines
titles
-0.8,0,0,0,
-0.8,0,1,0,
-0.8,1,0,0,
-0.8,1,1,1,
-0.4,0,0,0,
-0.4,0,1,1,
-0.4,1,0,1,
-0.4,1,1,1,
0,0,0,1,
0,0,1,1,
0,1,0,1,
0,1,1,0,
0.4,0,0,0,
0.4,0,1,1,
0.4,1,0,1,
0.4,1,1,0,
0.8,0,0,1,
0.8,0,1,0,
0.8,1,0,0,
0.8,1,1,0,
Notice how data file got tidied up
Now lets run 10 cycles of learning
Note how the overall error is decreasing, and below is after another RUN command to updated data file, what data file looks like.
and reconfigure cfg.txt to run 10,000 cycles showing every 1,000….(learning again will continue with weight saved after prior learn so it simply continues to learn and need not start again unless randomise weights is run again)
Data file looks like this now (dont forget to do RUN). Note how where target is zero, the actual are slowly getting there and where it is one, high values are being retained.
And after another 100,000 runs data file looks as below. Below are 3 pictures, edited cfg.txt for cycles, then runs screen, in about 3 seconds further 100k cycles of backpropagation and finally new data file after run command.
NN4 -> NNx 