Large libraries crash often (10,000+ mp3s) and it's a resource hog as well. I prefer to use Vox although since Coppertino took over, it's been high on resources as well. OS X badly needs a solid MP3 player that focuses only on music.
iTunes is a complete mess.
Use QuickTime. It plays audio files quickly and efficiently. Although there isn't support for creating libraries, playlists, etc.
Use get info on any audio file and change open with to QT, then click change all…