If the npcs are spaced out enough with your builds then it probably won't come with any negatives. Just none of the added benefits. and you can take it into account for future builds.As a builder, I complain precisely because now my buildings will make the NPС unhappy. I feel punished for building cool houses, but now they are not in the right place or there is a wrong neighbor. And now they will always remind me of this.
Note that this does not in any way call for improving the quality of buildings. This introduces arbitrage conditions on where they are located and which NPCs should be nearby. And apparently the distance from other NPCs