Even XP can be annoying if you have too much running in the background.
For real-time applications I used to set up a dual-boot for XP, a "normal" and a striped-down version without anti-virus, servers, even network. Then I was sure nothing would steal CPU cycles or freeze the application regularly for a few ms.
It wasn't for games but the same technique could be used in your case with Vista if you don't want to disable a whole set of features permanently just because you play now and then.
Don't blindly disable the protections as removing the firewall while you are connected to the network, it's suicidal (unless you are behind a router, and even so...). There are other attacks than running virus-infected files you have downloaded
