This post gives a working sample of training neutral net with cnn in keras. CNN setup is based on Yan LeCun configuration. model = Sequential() model.add(Conv2D( 32 , ( 3 , 3 ), input_shape =input_shape)) model.add(Activation( 'relu' )) model.add(MaxPooling2D( pool_size =( 2 , 2 ))) model.add(Conv2D( 32 , ( 3 , 3 ))) model.add(Activation( 'relu' )) model.add(MaxPooling2D( pool_size =( 2 , 2 ))) model.add(Conv2D( 64 , ( 3 , 3 ))) model.add(Activation( 'relu' )) model.add(MaxPooling2D( pool_size =( 2 , 2 ))) model.add(Flatten()) model.add(Dense( 64 )) model.add(Activation( 'relu' )) model.add(Dropout( 0.5 )) model.add(Dense( 1 )) model.add(Activation( 'sigmoid' )) model.compile( loss = 'binary_crossentropy' , optimizer = 'rmsprop' , metrics =[ 'accuracy' ]) validation_generator = test_datagen.flow_from_directory( validation_data_dir, target_size =(img_wid