How annoying !
I would guess - and it is a guess - that it is your word against his if there were no witnesses or you didn't get it in writing.
Sometimes you have to bite the bullet and be nice to them, and appeal to their better nature to get the job done quick. If you go in all guns blazing they could either give it back to you half done and refuse to finish it, or take even longer to finish it