You may be describing a condition called cold hyperalgesia. Some studies suggest this reduced tolerance to cold may actually help predict who will get better and who won't. Patients who have had an acute whiplash and suffer from increased sensitivity to cold are more likely to still have symptoms six months up to two years later.
There is actually a group of symptoms that predict the future result of whiplash injury. Besides cold intolerance, there's loss of neck motion, high pain and disability at the time of the accident, and older age.
How and why this all happens remains unclear. It's possible that damage to the receptors that sense temperature and movement occur with the trauma of the neck injury. Many scientists are actively studying the problem to find a way to prevent it from happening.