how soon would you start to take out bitch for exercising after pups, they 2 weeks old now. She just been in garden but getting restless now.What are chances of her bringing infection back home to pups if she goes out.

There is a chance but then so is there from the people coming and going.  You can only take all reasonable precautions such as wiping her feet and undercarriage with disinfectant (not a phenol based one such as Dettol) try to stick to areas where any other dogs using it are more likely to be well kept and vaccinated and hope for the best. 
I don't think it is fair to deny them the opportunity to see the world again when they are ready nor is a restless, unhappy bitch likely to make a good and patient mother.

My bitches get taken out when they're ready - usually at about 4 or 5 days. The risk of infection is no greater than you going in and out of the house, especially if you have other dogs as well. I've never had any problems from this. Obviously you'd wash any mud off her tummy when you got back, and you could wash her feet too.