Sleep changes our architecture

The idea, concept of neural networks came from the unbelievable structure of human brains. any species brain to be precise. billion or even more cells called neurons, that process and transfer electricity like signals from one part of brain to another that “somehow” simulate thoughts, ideas, visualizations everything in our mind.

Brain as a Neural Network

We use this brain of hours to make, break and change the entire world around us. But for survival, we need rest. More like our brain and body needs rest. Have you ever wondered why?

Rest vs sleep

Sleeping is a means of rest. Meditation is another means of rest which is unpopular in it’s true form. I want to mention this clearly that sleeping and day/night cycle are completely independent of each other. It just so happened to be an evolutionary trait that nights were too dark to do anything other than rest and our bodies synced our brain with this concept of sleeping at night. Keep in mind, it’s rest that is a necessity, not sleeping every night.

I guess our body needs to heal

Yes. Our bodies heal and fix, and repair and strengthens itself during rest. That is the most efficient way to stay healthy compared to the alternative where te body tries to fix itself while in use. But that does not answer the question of why rest completely. If a person is already pretty healthy, there is not much a body needs rest for. Yes, your cells will regenerate even when you are not taking rest. It’s the brain that needs rest for growth.

Neural network training

This part I am still thinking about. It is an interesting thought I got, and it feels to me like it might be related to something big.
Our brains are closest analogous to neural networks. Obviously much more and unknown than that, hence closest. Neural networks undergo training after which they are used for inference, to perform certain task, achive some goal. Training represents network weights, parameters and hyperparameters getting updated. There is also the architecture of the neural network which also justifies how well a network can perform.

So according to me - speculation - the brain runs its inference throughout the day whenever we are active. There is near to no training that happens during ineference time. All data is stored in L1, L2 cache that is the prefrontal cortex that stores short term memory. Including concepts and new context that one learn, all of it is just stored and if required retrieved from the cache. Magic happens when you take rest. The brain takes all the data in the short term memory and starts training on it to make, update, delete neuron connections to update our parameters and weights. Neurons might get closer, synpases might strengthen (long-term potentiation), existing structure might completely change for major architecture changes by even making new neuron connections called synaptogenesis rewiring itself based on requirement. During training, it also feeds in the long term memory, that is the L3 cache or maybe the ROM. Logically, while rest we are actually learning the concepts we learn throughout the day, we are actually remembering what we tried to remember and we are adapting to our changing enironment. We wake up with a newer better model of ourselves every-single-day.

Backing it up

This idea has some interesting side-effects as well. It proves why babies sleep so much, why rest becomes less and less necessary as we age and as the brain decays.

Food for thought : Chances are while the brain is training, even training has its outputs. These outputs are thoughts, visualizations, hallucinations which we humans call dreams. Not really meaningful in any way, but they represent part of what the brain is trying to learn.

The more you rest, the more you learn