One of my favorite gaming memories is being given the soundtrack to my 7th Sea game in college. One of the things my GM did was play ambient music throughout the night to set the scene.
It made each game distinct and changed the mood depending on what was being played. While the group would play around and talk before the session, everyone shut up and paid attention the second we heard, "The Autumn Wind is a pirate..." We knew it was time to get into character.
I think music was a huge reason we were able to get so deeply into the game. Rob made the environment immersive, even with minimum props. When it came time for my epic duel with my rival, he played the song from the Mask of Zorro where they trained. Now, I can't hear the song without remembering pacing back and forth through his living room while trading insults with my rival.
Once the game ended, we were all handed a soundtrack to the game with a specific song on it for each of our characters. Guess which one was mine.
I've also seen background music done wrong where a random song comes on that breaks the mood or the GM takes too long to find the "prefect" song. Like any other tool in a GM's arsenal, it needs to add to the game, not detract from it. If you can't easily find the song, skip it and move on. Ambient Mixer (https://www.ambient-mixer.com) is a great one because there are tons of premade sounds or you can customize your own.
When I'm working on a new character concept, I like to take the time to think of what kind of music they listen to or if they have a personal theme song.
What's your personal or your favorite character's theme song?