I think you should definitely take him to the vet to rule out physical reasons.
He may be in physical pain in some way and this should be ruled out before you can consider anything else.
Ask the vet to give him a thorough check up.
You may not want to hear this, but castration is unlikely to solve the problem. Occasionally castration can help in aggression cases but it is by no means a guaranteed conclusion. You are more likely to have the same problems, but your dog having been castrated too.
You say that he has growled at you and your husband 3 times, but then you only tell us about the time when he was sleeping on the bed - what happened the other times?
Personally, I would review some handling socialisation - feeding him treats with one hand, and with the other hand touching him all over, looking in his ears, stroking his tail, touching each foot and so on. If he is uneasy at any stage of this, back off and reduce what you are doing and proceed very slowly, feeding loads of treats to make this a positive experience. I would also review teaching him to accept having his collar grabbed, by just touching the collar and feeding. Holding the collar and feeding, pulling the collar a bit and feeding, tugging him and feeding and so on, over and over. And repeat the process with the car harness on. You'd need to do both these things, every day, indefinitely for now - and try to do them in different places, not just in the same room. Once you have done it all successfully, get your husband to do the same.
This suggestion is just based on the little info you have given us here. If you don't see any improvement, then go back to your vet and ask for a referral to an APBC registered behaviourist (they only see people on vet referrals). There is a list here:
http://www.apbc.org.uk/members.php