In short, not following the correct shaving M.O. can be a recipe for both ingrown hairs and skin irritation. Ahead, the best things you can do to help ensure a super smooth (and safe) shave. If an ingrown does pop up, Dr. Farhang recommends opting for an in-grown treatment that contains either tea tree oil or benzoyl peroxide, both of which can help alleviate inflammation with added antibacterial benefits.