I'm also experiencing this.
My steam friend is hosting a game via steam, and has other players on the server. When I join, I consistently will lose connection within 5-10 minutes of playtime on the server, and am presented with the Terraria menu and "Lost Connection" displayed, no other info is present.
I have verified integrity of my game files, and no issues are present there.
I have verified that my game version (1.4.1.2) matches the version that the host is on.
I have verified that all tmodloader files are deleted, and the mod manager is uninstalled.
Tried rebooting steam, pc, and network, all to no avail.