I had only built a small path of asphalt on the top of the dungeon. Didn't destroy the main entrance, but a bit on the back of the building.It should be possible. I've played on a tiny world that has cultists.
If your dungeon enterance has been drastically modified in some way. That might effect the cultists from appearing however.