If it won't degrade the signal too much for you and are willing to set 2 switches, you could try stacking switchboxes. I've got 4 of them used that way: the first 3 give me 9 console connections and feed into the fourth which determines which of the other 3 boxes are active. I picked up all the boxes at thrifts for about $5 each and still see them fairly often.
Vintage Extron hardware FTW! Seriously. (And oddly, this is my second reference to this hardware today.) An example is linked below but you might need to do a bit of research and careful description reading before buying.
Extron was/is a pro-grade A/V equipment maker and this old Composite/Component hardware is out dated and, thus, pretty cheap. It is excellent, though.
From my observation when I was hunting my desired type of switch, I think these things were made-to-order, so you will need to pay attention to the number of inputs vs. outputs. Regardless, most of these Switches tend to have more inputs than outputs, so something like an 8x4 might suit your needs, but you can find these in 8x8, 12x8 or even as high as 16x16 configurations, which is a bit ludacris.
You will also want to be sure that it comes with the correct audio output blocks. If not, you may need to hunt some down. Mine came with most of what I needed, but I did find a guy who made some homemade ones that I bought to finish off my unit. Anyway, it does take work to setup and use Extron equipment, but it's not hard. The biggest requirement is time. You can't beat the price though. This stuff is dirt cheap for what you get.
Example of what you might be looking for below. Note, with patience, even a model like this might be "expensive". I got mind for something like $75, shipped.