Honestly, something along those lines is a good reply

I should be more direct even when it's nasty