Most of us scoop a few spoonfuls of sugar into our daily dose of hot caffeine and think nothing of it. It’s not something we see adding to our calorie count each day.

And plenty of us are happy to reach for the coffee additives, too. More than half of American adults who drink coffee daily, and 67% of them put sugar, cream, milk, or other calorie-rich ingredients into it, according to a new analysis of 12 years of data from the National Health and Nutrition Examination Survey. And of the 26% of Americans who sip tea daily, around 33% of them add flavorings and fixings.

The bad news? This daily barrage of extra gunk is loaded with energy and fat, but lacks much nutrition. Sure, you get some calcium from the dairy products (about 22mg per day), but considering you need about 1,300mg of calcium every day, that creamer in your cuppa is pretty insignificant.

What isn't insignificant: The extra calories. “Compared with adding nothing to one’s tea, drinking tea with caloric add-ins increased daily caloric intake by more than 43 calories, on average, with nearly 85% of those added calories coming from sugar,” said study author Ruopeng An, Ph.D., a University of Illinois kinesiology and community health professor. It may not seem like much, but over time, those sugary calories can add up—plus your sweet tooth can lead to other indulgences.

On the flip side, people who drink black coffee get about 69 fewer calories on average every day.

Our advice? If you're trying to drop pounds, get your morning caffeine fix without the extras. And if you absolutely have to satisfy a craving, save the sugar blast until after your workout, when it'll do less damage and might actually enhance protein uptake to your muscles. We recommend these HIIT barn-burners to get your metabolism roaring.