Added header comments

Added big README :)
Added gif stuff
Fixed the remaining NeuralNetDemo names and changed to FCNeuralNet

Author: Josiah Laivins

.gitignore

@@ -57,3 +57,22 @@ test/Makefile
 data/steering_dataset/training/images/
 data/steering_dataset/training/steering_angles.csv
 /logs/
+/cmake_install.cmake
+/CMakeCache.txt
+/FCNeuralNet
+/FCNeuralNet.cbp
+/Makefile
+/test/googletest-src/
+/test/cmake_install.cmake
+/test/test
+/FCNeuralNet_autogen/
+/res/
+CMakeFiles/
+bin/
+lib/
+models/
+res/
+test/CMakeFiles/
+test/googletest-build/
+test/googletest-download/
+test/test_autogen/

CMakeLists.txt

@@ -1,3 +1,18 @@
+#
+# FCNeuralNet
+# Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+#
+# This software is provided 'as-is', without any express or implied warranty.
+#
+# In no event will the author(s) be held liable for any damages arising from the use of this software.
+#
+# Permission is granted to anyone to use this software for any purpose,
+# including commercial applications, and to alter it and redistribute it freely,
+# subject to the following restrictions:
+# 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+# 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+# 3. This notice may not be removed or altered from any source distribution.
+#
 cmake_minimum_required(VERSION 3.13)
 project(FCNeuralNet)
 

README.md

@@ -65,13 +65,18 @@ Once done pull the repo from github:
 
 `cmake CMakeLists.txt` 
 or as I found out using my Mac, I needed to specify the direct path to 
-the correct cmake (default was 13.12.X), exeample path goes to the currect one:
+the correct cmake (default was 13.12.X), example path goes to the correct one:
 `/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" /Users/jlaivins/CLionProjects/FCNeuralNet`
 
 `make`
 
 Then Hurray! You installed it! This CMakeLists file also is fancy in that it will install
-Google Test for you! 
+Google Test for you! Maybe in the future I can move the Opencv install to the cmake. 
+
+As a note, I did not use windows because of the non-linux inputs and just generally not liking the operating system. 
+If you have issues / trying to install on windows, I can try the best I can to help, but honestly Linux/iOS seems of 
+more interest to me to support. Of course, if you get it to run on Windows, let me know the commands / issues / links 
+you used and I can add them to this README.
 
 ## Running
 
@@ -99,11 +104,45 @@ Feel free to go into `/src/executables/NeuralNetRun.cpp` and reference the `trai
 performance can be improved by changing the bin size (make larger), adding extra layers (more advanced features?),
 and adding more neurons.
 
+You might be interested in the output layer proposals. In the `train` and `run` method, there is commented out code 
+which runs either is done, will show a window with the raw outputs (kind of cool) (but also kind of ugly...).
+```c++
+std::printf("\nShowing weight output comparison");
+cv::Mat dst;
+cv::normalize(nn.layers[0].w, dst, 0, 1, cv::NORM_MINMAX);
+dst = dst.reshape(std::get<2>(imageDimension), std::get<0>(imageDimension));
+cv::imshow("result",dst);
+cv::waitKey(0);
+```
+
+Below is kind of what it looks like.
+
+|   |
+|:---:|
+| ![this is something](res/activations.png)  |
+| |
+| *Fig 4: The actual model outputs (Left) have noise due to weight outputs. They gradually become clearer the more epochs are run. The input values (Right) have a gaussian distribution applied to them (hence instead of a single steering angle value, its a range.)*  |
+
 `./FCNeuralNet -runNN`
 Also free to go into `/src/executables/NeuralNetRun.cpp` and reference the `run` method. However, this method
 will produce all the final runs for the visualizer so I would not really want to change much.
 
-`./FCNeuralNet -runVis`
+`./FCNeuralNet -runVis` 
+Directs to `/src/executables/VisualizationBoard.cpp` and you can reference the `run` method. However, unless you 
+plan to make changes to maybe how the code initializes, I would recommend using a Python approach that I plan to go 
+with. This visualization object selects the most recent `log` directory, and loads the most recent model from the 
+`models` directory, so I would recommend you write a python script / notebook that reads directly from there. 
+* Note, this will live read the graphs, but to load a model (get the nice video) you need to restart the visualizer.
+
+Aaaand the end result should be something like this:
+![](res/training_animation_smaller.gif)
+
+|   |
+|:---:|
+| ![this is something](res/activations.png)  |
+| |
+| *Fig 4: The actual model outputs (Left) have noise due to weight outputs. They gradually become clearer the more epochs are run. The input values (Right) have a gaussian distribution applied to them (hence instead of a single steering angle value, its a range.)*  |
+
 
 
 ## Project Structure
@@ -156,8 +195,21 @@ will produce all the final runs for the visualizer so I would not really want to
 ```
 
 ## Contribution guidelines
-
-## For more information
+I am most likely going to start a new repo building on this. There are a lot of things I plan to change such as:
+- Move NeuralNetRun methods into their own objects / make part of the NeuralNet object
+- Drop QT and use Python Jupyter or a Tensorboard log file maker (which means I could keep the project 100% C++)
+- Add Dropout Layers
+- Add bias (probably the first thing I would add besides better logging)
+- Add Convolutional Layers
+- Add ReLu and LeakyReLu activation functions to experiment with.
+- Remove "FCNeuralNet" constants from the writers and dataset objects. Use CMake to insert them as Macros
+    - Initially tried doing this, but did not want to spend a whole day working on something that really did not seem 
+    like as big of an issue and the 1000 other things I needed to work in.
+- Add more datasets
+    - I want to add MNIST and FashionMNIST
+    
+Feel free to do PRs / Forks of this repo. Let me know if there are any changes. If they are big,
+I might go ahead and jump start a new repo. For questions the issues tracker is always good.
 
 ## License
 

main.cpp

@@ -1,3 +1,18 @@
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
 
 #include <cstdio>
 #include <string>

src/executables/NeuralNetRun.cpp

@@ -1,6 +1,19 @@
-//
-// Created by Laivins, Josiah on 2019-03-16.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
+
 
 #include <iostream>
 #include "../utils/DebugHelpers.h"

src/executables/NeuralNetRun.h

@@ -1,14 +1,27 @@
-//
-// Created by Laivins, Josiah on 2019-03-16.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
 
-#ifndef NEURALNETDEMO_NEURALNETRUN_H
-#define NEURALNETDEMO_NEURALNETRUN_H
+
+#ifndef FCNEURALNET_NEURALNETRUN_H
+#define FCNEURALNET_NEURALNETRUN_H
 
 namespace nn {
     int train(int argc, char **argv);
 
     int run(int argc, char **argv);
 }
 
-#endif //NEURALNETDEMO_NEURALNETRUN_H
+#endif //FCNEURALNET_NEURALNETRUN_H

src/executables/VisualizationBoard.cpp

@@ -1,6 +1,19 @@
-//
-// Created by Laivins, Josiah on 2019-03-15.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
+
 
 #include <cstdio>
 #include <QApplication>

src/executables/VisualizationBoard.h

@@ -1,12 +1,25 @@
-//
-// Created by Laivins, Josiah on 2019-03-16.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
 
-#ifndef NEURALNETDEMO_VISUALIZATIONBOARD_H
-#define NEURALNETDEMO_VISUALIZATIONBOARD_H
+
+#ifndef FCNEURALNET_VISUALIZATIONBOARD_H
+#define FCNEURALNET_VISUALIZATIONBOARD_H
 
 namespace boardViz {
     int run(int argc, char *argv[]);
 }
 
-#endif //NEURALNETDEMO_VISUALIZATIONBOARD_H
+#endif //FCNEURALNET_VISUALIZATIONBOARD_H

src/layers/Dimension.h

@@ -1,13 +1,26 @@
-//
-// Created by Laivins, Josiah on 2019-03-05.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
 
-#ifndef NEURALNETDEMO_DIMENSION_H
-#define NEURALNETDEMO_DIMENSION_H
+
+#ifndef FCNEURALNET_DIMENSION_H
+#define FCNEURALNET_DIMENSION_H
 
 struct Dimension {
     int currentLayerSize;
     int nextLayerSize;
 };
 
-#endif //NEURALNETDEMO_DIMENSION_H
+#endif //FCNEURALNET_DIMENSION_H

src/layers/Layer.cpp

@@ -1,7 +1,19 @@
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
 
-//
-// Created by Laivins, Josiah on 2019-03-07.
-//
 
 #include <utility>
 #include <string>

src/layers/Layer.h

@@ -1,9 +1,21 @@
-//
-// Created by Laivins, Josiah on 2019-03-07.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
 
-#ifndef NEURALNETDEMO_LAYER_H
-#define NEURALNETDEMO_LAYER_H
+#ifndef FCNEURALNET_LAYER_H
+#define FCNEURALNET_LAYER_H
 
 
 #include <opencv2/core/mat.hpp>
@@ -91,4 +103,4 @@ class Layer {
 };
 
 
-#endif //NEURALNETDEMO_LAYER_H
+#endif //FCNEURALNET_LAYER_H

src/nn/NeuralNet.cpp

@@ -1,6 +1,19 @@
-//
-// Created by Laivins, Josiah on 2019-03-05.
-//
+/**
+ * FCNeuralNet
+ * Created by Laivins, Josiah https://josiahls.github.io/ on 2019-03-08
+ *
+ * This software is provided 'as-is', without any express or implied warranty.
+ *
+ * In no event will the author(s) be held liable for any damages arising from the use of this software.
+ *
+ * Permission is granted to anyone to use this software for any purpose,
+ * including commercial applications, and to alter it and redistribute it freely,
+ * subject to the following restrictions:
+ * 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required.
+ * 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software.
+ * 3. This notice may not be removed or altered from any source distribution.
+ */
+
 
 #include "NeuralNet.h"