I want a retreat hotkey , they run away from whatever they were attackin or attacked by, but are still "following" you. and until you press stop all actions they will continue to continuosly run away.
What is your rep with Unity? That's why the probably attack. Also, the "don't attack unless hit" only applies to other players, not hostile NPCs.