Unless you go back to a time when the processors did the rendering for the GUI --at least, back when a GT120 was plenty and Apple was not feuding with Nvidia-- don't skimp on your video card. It might be painful, maybe even almost prohibitly so, however, in the end, the "cheap" today is the e-trash of tomorrow. It is somewhat like buying a plastic windows box on the argument that they are "cheaper" vs a used Mac; the plastic gets toasted and falls apart and the windows license sticker leaves a little bit of itself behind wherever it goes vs a 4,1 single processor --a Mac mini has a lot of the same cost problems as a laptop: "small, reliable, or cheap, pick any two... or so I've been told"-- cMP that can be upgraded well beyond what a Mac mini can for ~600 dollars. Take your time, ask around, maybe, someone you know just upgraded their video card and will lend you their old one --some are metal capable, they just needed more vram, as I did-- and do so for free. If that doesn't work, ask some PC gamers you might know, you never know, you might be just the excuse they need to upgrade and they seem, at least, the ones I know, to be either about to upgrade or upgrading.
I would also look in thrift stores, people die all the time whilst leaving behind something that the family either wants to get rid of asap or doesn't know what to do with and a Ralph Lauren down comforter was a dry cleaning away for less than 50 bucks! Flashy fans are a dead giveaway as are front facing fan controls, etc... grab it, throw it in your car, pay whatever they ask, at worst, you get a fairly new motherboard and processor with a hard drive or two. You might have to go to the store several times a week for a month or so but you never know what you might find; original NordicTrack for 20 bucks! Sure, you can do Craig's list etc... but people selling there probably know what they are selling and are not in any kind of hurry to get it out of the house. Sounds cold hearted but your situation, like more than a few people out there right now, demands a different kind of action.
Right now, I would not buy an Nvidia card to put into a Mac and I am not all that certain that I would take one if someone gave it to me for free. I have a feeling that Apple has had it with Nvidia and --outside their professional cards-- what reason do they have to make anything UNIX compatible? Their main market is PC gamers and they do not have AMDs processor market to fall back on so there is no upside for Nvidia to fighting Apple unless their lawyers have nothing better to do. If you MUST do Nvidia, ask around here, someone in your state probably just upgraded their video card. I would offer my old one but I have had more than a few bad turns with Nvidia chipped cards so ATI [old enough that it takes work to say A-M-D]
One last thing: I presume that you are using the Mac mini because it saves electricity for things that do not need MacPro processing power? That is how I use --most of the time ;-)-- my unibody MBP. My setup is different because I do not have the ability to use an eGPU but I control the cMP remotely --within the same network or plugged directly into ethernet ports-- and let it sleep whenever I do not need the cMP and it will run headless without a problem.