Upon review, it looks like MOST bookshelves in the game do not have a flat/even top, and so this issue is present, more or less, on about 3/4ths of the bookshelves. There isn't really a simple way to fix this except by redoing the sprites for every single affected bookshelf to be "perfectly flat on top", and I don't think its worth changing the design of 20+ Bookshelves for this purpose, it would make them more boring.
Meanwhile, to prevent books from being placed on top would just be limiting for the shelves that do work normally, so there isn't much that can be done here. Limitation of the system, I suppose.