Exercises should not bring on pain, even after having a baby. You didn't describe your exercise program for us to evaluate, but here are some guidelines.
First of all, make sure your doctor gives you the go-ahead. There may be medical reasons to wait before beginning an exercise program.
It's important to breathe while doing exercises. Holding your breath increases the pressure inside the abdomen. Exercises should be started slowly with only a few repetitions. Build gradually by adding a few more each day. Pain one or two days after starting a new exercise may be a signal that you are doing too many too soon.
After pregnancy the deep abdominal muscles and the low back muscles are often unbalanced. There are some specific exercises for this called stabilization exercises. A physical therapist can prescribe these for you based on your body and the condition of your muscles, ligaments, and bones.
Smoking and obesity can add to painful back problems. Good nutrition and getting enough rest and liquids are also very important in regaining your health and strength after pregnancy.