The PPU is using the address pointer internally when drawing the picture, that's the same pointer as you are using when you write to the palette memory.

You should disable the PPU when you want to access memory through it, if it isn't for a small amount of time. Then it could be enough during the vertical blanking period.
But you have to disable it if you want to do it in the middle of the screen.