Having a low sex drive sucks, and talking about it with your doctor or partner can make the experience even more unbearable.

But here's the thing: More often than not, low libido is the consequence of mostly preventable sexual health problems—and that means you have a good chance to right the issue before it gets any worse.

We talked to Birgit M. Fisher, Ph.D., a clinical psychologist who specializes in the treatment of sexual disorders, and psychiatrist Gary Martz, M.D., about what could potentially stall your sex drive—and how you can get back on track in the sack.

1. Certain medications

Anti-depressants have a bad rap for messing with mojo. But more often than not, medical side effects are not to blame. Most people actually have something called hypoactive sexual desire disorder, which is just another name for low sex drive. Not every guy wants sex all the time (despite what society has us believe). Everyone has different libidos, and different biological sex drives and desires. So if you're not always in the mood, that doesn’t mean something is automatically "wrong" with you.

However, certain medications can alter your sex drive. Be wary about meds used to lower testosterone (like those used to treat prostate cancer), selective serotonin re-uptake inhibitors (like paroxetine and fluoxetine), and serotonin and norepinephrine re-uptake inhibitors (like venlafaxine). Some others include lithium, benzodiazephines, antihypertensives, cardiovascular drugs, and lipid-lowering meds like gemfibrozil.

And while there isn’t a whole lot of evidence linking supplements with lower sex drives, scientists have found people who take St. John’s Wort notice a lower libido. It’s always good to check the labels on anything you take, or ask your doctor or pharmacist if you’re concerned.

2. Alcohol and caffeine

Booze and caffeine can lower your sex drive, but it depends on how much and how often you're consuming both. You also need to factor in any underlying psychiatric issues.

Because caffeine is a stimulant, it can aggravate underlying anxiety conditions, which can lessen your desire for sex. Hitting the bottle too hard is also bad for libido, erectile function, and the quality of your orgasms.

3. Too much and too little exercise

Everyone knows too little exercise can lower sex drives, but too much can have the same effect. Excessive exercise can lead to conditions akin to eating disorders and body dysmorphic disorder, which can have a negative impact on your sex drive.

Overall, being fit helps people feel sexy, so moderate exercise is good. Yoga has been known to increase sexual interest and enhance sexual pleasure. Yoga may even stimulate genital blood flow, which can improve weak erections and orgasms, according to a review published in the Journal of Sex and Marital Therapy. And obviously yoga is a huge de-stresser, which can stave off low sex drives as well.

4. Aggressive porn use

If you're addicted to porn and your masturbation habits are out of control, you can over-stimulate levels of dopamine, the neurotransmitter that triggers sexual pleasure. If this happens over and over, it becomes harder for your brain to respond appropriately. This can be reversed by avoiding pornography and sticking to a healthier masturbation pattern. It might sound easy, but men often have a hard time self-regulating this behavior, which is why many seek professional help.

5. Emotional imbalance

If you’re in a rough patch with your partner, it could be the reason your sex drive has taken a nose dive. When your libido plummets, it could be your body's way of protecting itself. If you sense a wall, identify what’s bothering you and talk to your partner about it, or seek professional help. You have to address the issues. Avoiding the conversation might make your partner feel rejected or even suspicious you’re being unfaithful.