Using a Patchbay as amp switcher ?

Yeah I know this might sound like a bad idea because amp switchers use a dummy load....but

In my home studio I have several amps.

Most are tube amps.

At the moment I choose the amp I want to use, take out the speakercable from the Marshall and put it in the Carvin in the back (running to a Torpedo Capture X) and than connect the guitar.

My amps are never on when not used.

At the moment I always have to dive behind the amps with a flashlight and change the speaker output to feed the Capture X.

I'm installing a Patchbay for outboard gear at the moment and when installing it just hit me.

Why not use one output of the Patchbay for the Torpedo X and use 5 inputs for the amps. Ofcourse use the proper cabling to patch.

One part of my brain tells me bad idea because... I've never seen anyone do it.

The other part of my brain tells me that as long as I don't run the amps without load (same risk I run now, expect with the patch panel I can visually see which amp is connected. there should be no problem as long as I use THRU on the patchbay.

Any ideas?

I would make life here a lot easier, and buying an amp switcher is too expensive, I rather spend that on vintage gear:D