I still see merit in having atleast one of them. Whether it be through donation troops or just building one yourself. Even if the unit only manages to kill one building, if it's a high value target like a mortar, anti-tank tower, or an anti-air building then that unit has payed for itself I feel. Basically compare a sabotuer to a living demolition tactic, for the cheap price of sub 100 oil you can essentially be casting multiple demolition tactics on an entire cell of buildings if they aren't properly set up with traps.
If you're Roman I'd probably outright say there is no reason to not have one, since you have the extra troop space to utilize atleast one of them and still have a normal army composition. Shame that castles attack the little guy.