It's actually quite simple - the harder you work your body, the more time it takes to recover and recuperate from training. It is also important to know that although you stimulate growth by training, it is during the resting period that the actual muscle growth (adaptation) takes place.
Have you ever hit a plateau and no matter what you did, you couldn't overcome this plateau? Then you decide to take a week off on active rest, and you come back even stronger than before? Why is this? It's simple. You allowed your body time to recuperate fully thus allowing you to become stronger!
One of the most common mistakes I see nowadays is the tendency for beginners to want to overtrain. They think that more is more when, in actuality, more is less.
I hear how lifters brag how they spend 3 hours in the gym. I can almost assure that if you spent 3 hours in the gym training at all-out intensity, you would have to be wheeled out of the gym.
I know that for my training partner and myself, 70 minutes or so is enough to make me ill! Anything more and we're wasting our time.
Also, different muscle groups recover at different rates. There isn't a set number for the recovery rate of different muscle groups, but the usual principle is that it takes larger muscle groups longer to recover as opposed to smaller muscle groups.
You have to train and discover how fast your different muscle groups recover. Then you can design a program which gives you optimum recovery time to ensure optimal growth.