The good Reverend is one of the classic psychobilly/punk rockabilly dudes. There are albums, including five released after it, but the compilation Holy Roller is pretty much the alpha and omega. You'll get all the classic stuff.
As far as George Michael, you're probably fine with just the hits. If you want to go further, Faith is the one album of his I was able to listen to at some point.