While they do have reached a "limit" on liquids and paints, this has NOTHING to do with the engine. Consider, the engine doesn't have any inbuilt liquids or paints, it doesn't even have a concept of blocks or items inbuilt. All it does is provide a basic framework with an update and draw loop and graphics and i/o support.
The "limit" reached is due to the way Terraria stores blocks in a world, which i explained before in another post about paints, but here it is again: ReLogic has designated 5 bits of the storage to storing tile paint, which happen to be able to represent 2^5 = 32 different values and thus the game has 31 paints and a "no paint" value. Similarly, 2 bits were designated to store the liquid type of a block, which is able to store 2^2 = 4 values, one for each liquid that currently exists ("no liquid" is stored by setting the liquid amount to zero).
"Why not just use more bits for these?" you may ask, and the answer is that this would either explode the amount of memory used by the game (because there could be up to 8400*2400 = over 20 million tiles at once, so every byte matters), or degrade performance significantly due to complicated bit operations being necessary.
TLDR: "The engine" is mostly responsible for graphics capabilities such as shaders, and liquid types, paints, or trees, are developer-implemented features and not affected by graphics capabilities of the engine.
This also means that no, tModLoader doesn't have a better engine, they simply expand the block data storage. Sad to see people spread pure speculation as facts.