“Flags or painted markings” that may be distracting or hazardous to drivers are being removed “strictly on safety grounds”, the Isle of Wight Council has said.