Is there an intuitive explanation why some neural networks have more than one fully connected layers?

2017-12-25 07:54:09

I have searched online, but am still not satisfied with answers like this and this.

My intuition is that fully connected layers are completely linear. That means no matter how many FC layers are used, the expressiveness is always limited to linear combinations of previous layer. But mathematically, one FC layer should already be able to learn the weights to produce the exactly the same behavior. Then why do we need more? Did I miss something here?

There is a nonlinear activation function inbetween these fully connected layers. Thus the resulting function is not simply a linear combination of the nodes in the previous layer.

  • There is a nonlinear activation function inbetween these fully connected layers. Thus the resulting function is not simply a linear combination of the nodes in the previous layer.

    2017-12-25 09:48:33