The music is what gives the game it's charm. Well not only music also the sound.
If it's get stuck in your head then it means it is irritating or just to freaking good to forget(Still remember Final fantasy VII Music when you beat some monsters xD)
Also The music is often used to set up a mood in the game.
A scary game don't get that scary when you are playing without sound on(Even though we are talking to music. Music is sound so
).
A happy game doesn't get happy if you don't really show it with music.
Myself I see that the music has to be in the game. otherwise the game looses a bit of feeling. simple enough.
If you take a while and think about it. What is the thing you most often can remember from a game. Myself I remember most of the music to most games.