After taking my puppy to the vet I found out he has an umbilical hernia. Should I have been made aware of this before I bought him. Do I have any rights?

What did the vet tell you ? is it a late closure or a true hernia that may need surgery, or is it just going to remain a small bump ? Hernias are not uncommon in many breeds, most are tiny and never need to be touched they will not affect the pup at all and can be forgotten. Did you get the puppy to show/breed/pet. All these points may be relevant to your question.
What sort of "Rights" are you looking for ? Do you want to give the pup back and have another ? Do you want money off ? Do you want to give the pup back and have a refund ?
Have you spoken to the breeder yet ? Maybe she will refund you your money and take the pup back if you are within the times stipulated on the contract of sale. I tell new owners if hernias are present at point of sale and give 2 weeks to visit a vet and refund in full if a problem is found. I also will take a pup back at any time for any reason, but will not necessarily offer a refund then.
Lots of possible answers for your question really.
Aileen

If it a delayed closure they are extremely common. I had had them in pups I have bred and they have always got smaller a dog grows and never been a problem. I have seen no hereditary pattern to them either.....I think a lot of it is caused by not supporting pups bum when picked up and the gravity pulls on the not very strong opening.... Since I have been supporting all my pups bums when handled I have not had a single delayed closure.

I find the vets class any bump as a true hernia.... If it is a delayed closure I would not blame the breeder.

Hernia can also pop out in a pup e.g it might have not been there when you bought the puppy. I had this happen with a pup I bred, 100% sure he did not have a hernia when he left me.
Nothing to worry about if it is just a small bubble.

I actually have a girl I bred myself( no relation to the pup above) Perfect bellybutton as a pup and going forwards. She is now 5 years old and about 6 months ago I noticed a bubble on her bellybutton. No idea when it has appeared but can't be too long time ago as would have noticed it when grooming/giving belly rubs.
I can only think she must have had a tiny delayed closure bubble ( so small it was not noticeable) and maybe with little more fat on, it now shows! it is so small and slightly raised most people might not even notice it but it is definately there and closed up, not a hole etc

This may explain what is meant by a delayed closure:
http://www.showdogsupersite.com/kenlclub/breedvet/umbilical.html