This happens to very passing transwomen frequently. I suspect some of the people trying to call you ‘sir’ are mistaking you for a transwomen (due to your height and lower voice) and trying to enforce their ideology on you.