I think its usually pretty safe to just right click on top of one of your own pieces to remove arrows. Since it never legal to capture your own pieces this would never result in you accidentally moving your own piece.
How to remove arrows without moving a piece

Oh that's nice! Not easy to always remember to click your own pieces after arrow drawing. Also, clicking your own piece sometimes just makes an odd premove.

I also have this problem. I rarely click consciously, making it hard to learn it the right way. I'm looking forward to the patches

just fyi, you can just left-click anywhere on the board (preferably an empty square) to erase all arrows/marked squares.
I like using arrows and square markings when playing but too often I end up accidentally moving a piece when I click the left mouse button to remove the markings. (This obviously means I've clicked on a piece before creating arrows.) I've played a little over 800 games elsewhere without ever having this problem. I thought that since I'm using the setting "drag pieces" to move them this wouldn't be a problem, but apparently "drag pieces" doesn't work the way I expected.
So anyways, what is the correct technique here? Never click on a piece before creating arrows? Is there something I can change in the settings? Or is it simply go back to another site?