by now you probably realize NN4 has no way to prepare and normalize values from raw data input. Things have to be prepared for NN4 modeling. Meaning input values if using Sigmoid should be between -1 and 1. Or if you want to model lists like Monday to Sunday then you probably need a way to map to -1..+1 range and of course back.
This page provides those formulas and i found it easiest to work in Excel. So i use excel to hold raw data, i do conversion to range of neural net on Excel (using formulas described below) and once NN4 is finished, i get data back into Excel and use reverse formals to see if results make sense.
As said several time, aim was guts of Neural Networks rather then data scrubbing code. But fact that big data and AI is 90% data scrubbing and 10% fun of Neural Nets.
Linear Mapping from real world to 0..1 or -1..+1 and back
from GSCE maths you will recall line equation being
y = mx +a (and this is used to go from real world to NN domain)
x = (y-a)/m (and this to go from NN to real world)
and
m is gradient or (Ymax – Ymin) / (Xmax – Xmin)
a is intercept, once we have m, for y=0, a=-mx (or for x=0, a=y)
Example, Oxford weather experiment has years 1850 to 2019.
Xmin = 1850 and should be Ymin=0
Xmax = 2019 and should be Ymax=1
m = 1 / (2019 – 1850) = 0.0059
a = -0.0059 * 1850 = -10.92
lets check
for x = 1850: y = 0.0059 * 1850 – 10.92 = 0
for x = 2019: y=0.0059 * 2019 – 10.92 = 1
for x = 1970: y =0.0059 * 1970 – 10.92 = 0.7030
For range -1 .. +1 simply rework m and a as per above.
And mapping a set to 0..1
Example could be to map week days Monday through to Sunday on scale 0 .. 1. E.g. assigning each week day discreet value. Assume position count starts at zero for our unique list of items. See below. Lets call this position in list.
0 = Monday
1 = Tuesday
2=Wednesday
…..
6 = Sunday
Count is 7.
Mapping to Y min =0 and Y max = 1.
Y increment = (Y max – Y min) / (Count -1 )
Position to NN is then = Y min + (Position * Y increment)
and NN back to Position = Round ( (Y value – Y min) / Y increment )
I have built Excel template that i keep reusing for various experiments with above formulas so i can quickly convert values.
Incidentally, i also have in Excel a concatenate function to make up my data strings. Then i copy paste as values and export as csv.
After renaming to txt so notepad can open, i have ready made data.txt for NN4 to use.
NN4 -> NNx 