Unpopular opinion here, but the Angler is my favorite town NPC. I find that his daily fishing quests act as nice breaks from other activities, making the gameplay feel more varied. I like to weave his quests between other Terraria activities such as building or boss fighting.
His rewards are also very good, as having the full set of Fishing Armor, a Angler Tackle Bag and some fishing-related potions alongside the Golden Fishing Rod with some 50% bait power bait makes it trivial to catch large quantities of high-tier potion fish. He also gives good-looking furniture items and the Fin Wings are also very useful at the start of hardmode.
I personally have done at least over 600 fishing quests across all of my playthroughs (I have gotten Supreme Helper Minion 3 times on different accounts/platforms). I think that many people just grind the angler using the bed to skip time, instead of doing it organically while building or progressing with the playthrough in the down-time, which makes them feel bored at the Angler quests.