First of all, this suggestion is regarding leaving and rejoining the world, so not sure how an inventory toggle would work with that unless you're implying you set it ahead of time before you leave the world?
Also, as I explained in an earlier response, being able to choose whether you warp back to spawn or not is even worse! That would allow you to still get a free recall potion back to spawn the way it works now, but at your discretion whenever you feel like it. At least right now, being able to get a free recall back to spawn has the one downside that it happens every time and you cannot control it, even if you wanted to stay wherever you were.
also... I kind of just realized this "mechanic" negatively impacts builders and explorers, while providing an abusable cheese mechanic for everyone else...