Adenosine is a substance that accumulates in the brain during your waking hours and so builds sleep pressure — your natural, growing desire to sleep. During sleep, adenosine is broken down again, so that you wake up fresh. Caffeine blocks the receptors and postpones that pressure, but doesn't take it away.
The longer you are awake, the more adenosine accumulates and the stronger the desire for sleep. That is homeostatic sleep pressure. Together with your circadian rhythm it determines when and how deeply you sleep.
A long, active day builds a lot of sleep pressure. A long afternoon sleep takes it away again, making it harder to fall asleep in the evening. A twenty-minute nap doesn't; that is exactly where the difference sits.
Caffeine resembles adenosine in shape and occupies the same receptors, without producing the sleepy effect. Your sleep pressure doesn't disappear, it simply isn't felt for a while. As soon as the caffeine wears off, the accumulated pressure arrives after all.
Because caffeine has a half-life of five to six hours, a cup at four in the afternoon is still a quarter active at eleven at night. You won't notice that in falling asleep, but you will in the depth of your night.
A common rule of thumb is eight to ten hours before bedtime. Anyone going to sleep at eleven drinks their last cup around midday. That feels early, and it is precisely why most people have never noticed the difference.
Short no, long yes. Twenty to thirty minutes gives recovery without materially reducing sleep pressure. Ninety minutes takes out a whole cycle and you feel that in the evening.