Defense is not for tanking everything, it is for reducing the damage you take from being chipped.
If you get hit by the Destroyer's head, The Eye of Cthulhu charging, or the Moon Lord's phantasmal death ray, you're gonna take a ton of damage regardless.
Defense is more for the rest of the things in the fight. Let's say you're a mage using Orichalcum Armour fighting The Destroyer on Expert as an example.
Your armour stat will be 27 at minimum, more likely 59 or so with Ironskin and Warding accessories, and let's round that down to 44 due to Expert using 75% of your armour value.
So that's 44 less damage you're taking from everything.
You get hit by the Destroyer's Head, that's 280 damage reduced to 236, you've taken a bit less damage, but it's still a high damage hit you must avoid at all costs.
You get hit by the Destroyer's lasers, that's 72 damage reduced to 28, which is way more manageable.
You get hit by the Probe lasers, that's 88 damage reduced to 44.
Of course there's still the roughly 15% damage variation, so you could take 15% more, 15% less, and if you use some luck bonuses such as the Galaxy Pearl, Gnomes, or Luck Potions, you're likely to take a bit less damage. You could even use Endurance potions for 10% less damage taken, etc. It all adds up.
Defense isn't there for you to rely on as a way to win every fight, it's there for you to rely on for when your dodging skills fail you or you fall into situations where you MUST take damage.
If you're struggling with the Destroyer, it helps to know that the Destroyer shoots out a probe at a low chance when you hit its glowing segments, which then stop glowing. So if you hit the destroyer a bunch, stop to destroy the probes so you don't get swarmed. You can also build arenas with walls so the Destroyer doesn't have a line of sight to shoot you with, and that works on the probes also. Melee has a pretty easy time against the probes since they like to come into melee range and get knockback stunned.
tl;dr always use ironskin, try to use endurance potions, and learn better arena building + don't rely on defense more than your dodging skills