Oh yeah, you know how people put on music to relax and go to sleep (with the music set on a timer)? It doesn't work, it keeps me awake.. I analyze every detail, think about how they made a certain sound or how something is placed in the mix...
That said, at the same time I pretty much have to listen to music while working, otherwise my brain is like 1/2 as functional. No clue why.