
Start again from scratch, I'm afraid. You'll have to pretend he's an 8-week baby, and take him outside at least once every hour and stay with him until he performs outside. Have you trained hm to pee on command? If so, use that cue-word outside instead, and never, ever indoors. Whenever he toilets outside, give him plenty of praise, and ignore the inevitable messes inside. It will take a long time for him to unlearn his training; that peeing inside is now not acceptable.
Puppy mats make life very difficult for both dog and owner, I'm afraid. It's much easier to start the right way - that outside only is for toileting - from the very beginning.
ETA: don't allow him access to the area where the mat used to be. It will still (to him) smell like his toilet area (clean it thoroughly with a biological washing liquid solution then wipe down with surgical spirit or white vinegar to remove any traces of odour) so he will want to use that area even though there is no longer a mat there.