You can also try applying cool compresses or ice packs (a bag of frozen vegetables works nicely) and taking nonsteroidal anti-inflammatory medication such as ibuprofen.

http://www.babycenter.com/404_im-not-breastfeeding-will-my-breast-milk-dry-up-on-its-own_11730.bc