There are two main reasons why some men get swelling under the nipple areas: either there is growth of breast tissue, or there is thicker fat or muscle layer underneath. In the former condition, the nipple areas will be painful and swollen, and there may be discharges from the nipples. This happens in normal pubertal stage, but can also be caused by diseases like testicular or brain tumours, liver diseases, thyroid gland hyperactivity, certain chromosome abnormalities and side-effects of some drugs. On the other hand, if the condition is due to thicker layer of fat or muscle underneath the nipples and there is no abnormal growth of breast tissue, the nipples will not be painful and this is not a disease. You should see a doctor to ascertain the reason behind your condition.