It’s easy to understand why the term “comfort wine”, or even comfort drink, isn’t in common use: nobody wants to claim that alcohol is comforting (though, in moderation, it can be). Of course, many non-alcoholic drinks are comforting, not least tea and hot chocolate (I’d argue that coffee doesn’t come into that category, though, because it’s more a stimulant, but lovers of lattes and flat whites would no doubt disagree).
To be comforting, a drink needs to be familiar and consistent, hence the popularity of brands such as Coca-Cola and PG Tips. It may even recall your childhood: I still love Rose’s lime juice cordial, for instance, which I can first remember tasting aged about two; these days, however, I prefer it in a gimlet.