That would make sense *if* it trimmed some other active process, that process might take a few milliseconds to recover the paged out RAM that is in active use.
BUT you saying the hiccup happens even when it does nothing makes me wonder... the SmartTrim dedicated thread is lean and mean, so that's not the issue.
The logging, that must be it. You can try disabling, just to see. I've tried to make the log as efficient as possible, but... we'll see. Maybe coincidence right now.
EDIT: Also, security software overhead could increase during SmartTrim activity and/or logging, so that could exacerbate any disruption.