Excellent post. Followed your github script, and got simiiar results.

One thing I did notice is that you didn't standardise the images to be of a set size before feeding it to the neural network.

Any reason why you didn't do this?