First off, for me this question can be very broad and opinion-based.
You could give a game the rarest drops off all items, making their enemy AI act perfect so that there's barely a chance to avoid them.
Or things that makes the game more unforgiving than hard. If such a game relies so much on luck that it's not fun to play them either.
(Take games like "unfair mario", "I wanna be the guy" or 10+ hour permadeath mode for example)
Now, for my opinion the most difficult, yet 'fair' challenges would be final challenges in Crypt of the Necrodancer. Such like the Coda run. Which was a combination of the 3 hardest character runs. Even the developers believed it to be impossible. Yet with enough time and determination, you'll get it through.
There's also the All character low% run in the same game, but that sticks more to the "Too long and unfair" challenges of my list.