Gentle exfoliation of the skin can help get rid of ingrown hairs. Sebaceous cysts are caused by a blocked sebaceous gland (oil gland in the skin). They commonly occur in the vulva and appear as a small, hard lump, which is generally painless. Sebaceous cysts do not require treatment unless they cause discomfort. May 2, - "Women come into my office worried about [genital bumps] all the time, and many jump to the What's inside of them differs. of the small hair follicles around the vulva, which can create bumps and lumps," says Dweck.