World Modeling with Neural Networks Sucks (WMNNS)

As many researchers may have felt so, neural networks always make me frustrated. It has no ability to generalize the sequential data. Human has elaborate world model learned from his own experiences, but neural networks does not. Experiences are perceived as a sequential data, yet neural networks cannot cope with them at all. Neural networks sucks.

 

Of course the main cause of my rage is onto lack of my own ability to making use of neural networks great, not onto neural networks itself. However as many of many experiments go, a bunch of disappointment toward neural networks arouse.

 

In a traditional manner, there will never be found the breakthrough for online sequential learning. That's reasonable, cause the neural networks are so fuckin adaptive that it cannot generalize anything with just a single data in every single backward step. They fall in love with just seen one single data, and forget everything about the good things shared before with her right after observing newcoming sexy single data. That's why the things like weird mini-batch or disgusting replay-buffer are required to train neural network in a well-behaved way.

 

It cannot even generalize the visual dynamics of Pong! The autoencoder's poorness make the Pong dynamics model unable to predict even one future step of visual state. It does not capture a simple white ball stably. But yeah, all I have to do is not to blame neural networks, but to learn more. Just my deeply frustrated emotions are leaved here.