You've just described a problem called fear avoidance beliefs. The more a person fears pain, the less he or she does and the more disabled they become. Some of this response is natural. It can be based on past experience or "pain memory."
Studies show that most people fear a level of pain far greater than anything they actually experience. Many other studies show that physical activity and exercise is the best medicine for back pain.
Just being aware of the problem is the first step. You may want to seek help if you are unable to talk yourself through increasing your activity. Sometimes patients need behavioral help from a counselor or psychologist. Others seek the services of a physical therapist to help them with a rehab program.