If I create a new chat room, how will others know about it to come to the room? Activity on many of the current rooms seems to be low, and since it's not appropriate to announce things like this in the non-meta Q&A section, I'm wondering how people will know about new chat rooms. Thanks for your help.
As chat rooms are not private (except for mods), the only real reasons for creating a new chat room on a low chat traffic site like Music.SE are for mods (to discuss something in private), or for specific commentary on a post (to keep things segregated)
So first off, my guidance would be to not create a new chat room. Using the main chat should be appropriate for pretty much everything, and will help increase activity - which as you have pointed out isn't huge currently so would be even worse if diluted across many rooms.
If you do have a good reason, once you create the room, simply use the URL.
Remember, people can also see the chat rooms for each site from the main chat page, so any that you create will also be seen by mods, 10k'ers and anyone who tends to access chat through that route.