Is it really just the calories?

I hear all that you should avoid carbs to lose weight, from the Atkins diet to the south beach and this guy called Gary Taubes. But isn't it really the calories? Now they are saying that it's not carbs but calories that cause weight gain. For instance if someone was to eat a 100 calorie apple they shoud get the same results of weight loss if they were to get the 100 calories from a donut. I believe this but many do not, why? Don't you believe it's really just the calories and we should stop going on about the stupid atkins and gary taubes?