I'm a bit two-sided about this.
I understand why the one prefers to keep them vanity slots.
Vanity slots were, as said earlier, for vanity purpose only. I like to have the mining helmet to carry along with my armor set, but I don't use the mining helmet I won't put it in the vanity slot so I won't be confused to have a mining helmet as vanity. Adding options to hide armor that simple as that sounds silly to me because you don't swap armors that frequent. The familiar set will do well enough.
Back to the accessoires. I personally think you should already be grateful that there's the option to quick-swap them on the first place, as it was not intended as a second set of accessoires. Eventually the vanity could also be moved to the equipment slots so people won't abuse it as extra accessoire pocket space.
But I also understand why you want to have them disabled.
I also use a few slots to quick swap as well, mostly items that are not visible on the character. (like the greedy ring). But the fact that not all accessoires are visible on the character, it makes it oddly specific which one is more vanity-convienient than the other. And now that the Accessoire quick swap is in the game already, there's not really a reason to make them more annoying to use.
Though, imagine if it gets implemented, I can imagine that the next suggestion would be these suggesting another column for 'real' vanity equipment, because they'll likely use the old vanity column for their second accessoire's then.
That thought is not something I'm looking forward for. Just saying that.
I'm not supporting the suggestion, neither am I really against it.
At the end I don't think I'll care too much if it gets implemented. It's been abused for so long already.