Qui-Gon is right. Though the soundtrack to "The Cave" was okay, it's not nearly as good as the CAVE publications of the last half-decade. I think Akai Katana Shin makes the cut, and obviously DDP SDOJ does. And then there were two Mushihimesama albums, one of which was arranged by Ryu Umemoto (RIP).
Also, I love Qui-Gon's list. Great, great choices. Especially happy to see that Corpse Party representation. That US 2disc release with Blood Drive for Vita was a huge money-saver for collectors and the music is just so memorable, it's ridiculous.
See, this's exactly why I make topics like this, so I can find out about new stuff I never would've otherwise discovered!
Any of their albums/soundtracks that you'd recommend in particular?
You mean from the publisher, CAVE? ... most of them. If you can find it at a good price, I'd start with the Deathsmiles OST and any Dodonpachi album, OST or arrange. Mushihimesama albums are also very good, I highly recommend the "Double Arrange Album" if you can get it.
Akai Katana Shin is great, but don't expect to pay less than $60 for it.
Some of CAVE's "less-than-great" offerings to probably avoid are Cave no Uta (vocal collection), and unless you have a taste for it, MuchiMuchiPork and PinkSweets are weird. Ibara is good though!