I had this annoying bug that the library of Rhythmbox was empty after I restarted it.
After a bit searching I discovered that it is caused by moving your library from temporary storage to your harddisk.
This can be from a SD-card and symlinked to your home directory or from an encrypted partition symlinked to your home directory.
You can get some more information at this blog post or the bug database.
I didn’t care much about my entrys in the database so I just deleted it:
After that Rhythmbox builds your database new and should find all your music files.