Seems like there’s a natural association between the two:
Metaphors such as icy stare depict social exclusion using cold-related concepts; they are not to be taken literally and certainly do not imply reduced temperature. Two experiments, however, revealed that social exclusion literally feels cold. Experiment 1 found that participants who recalled a social exclusion experience gave lower estimates of room temperature than did participants who recalled an inclusion experience. In Experiment 2, social exclusion was directly induced through an on-line virtual interaction, and participants who were excluded reported greater desire for warm food and drink than did participants who were included. These findings are consistent with the embodied view of cognition and support the notion that social perception involves physical and perceptual content. The psychological experience of coldness not only aids understanding of social interaction, but also is an integral part of the experience of social exclusion.
Source: “Cold and Lonely: Does Social Exclusion Literally Feel Cold?” Chen-Bo Zhong. 2008; Psychological Science