
Did you report the men to the police? I certainly would have - I have done in fact when some years ago a similar situation arose on one of my OoH excursions, I was definitely followed and when reported to the police discovered someone else had been similarly stalked a week earlier but thankfully neither of us incurred injury or had our dogs targetted :)
As for the garden, much will depend on the height of your boundaries. I have mainly walls, 6' high, and a short 'L' shaped run of 5' high fencing but three sides of the garden are either inaccessible because of taller shrubs or the rear wall of our house. It's not impossible to get over a 6' wall or fence but even in a large garden unlikely to be able to do so undetected if you're busy in the general area of the house nearest the garden - certainly in the unlikely even of someone trying to escape with a dog under their arm I'd think it would be very difficult and that without the dog objecting :)
If you don't feel the garden to be sufficiently secure then raise the height of your boundaries and plant some prickly shrubs around them too with perhaps coarse gravel/shingle so that anyone entering uninvited will not be able to do so quietly :)
HTH, Teri