If you're raiding a few in a row, then it's good to have a couple backups for the cool-down. I'm just upping the castle to get the 3rd now. Plus you can tend to keep one active for defense and still keep a couple in reserve for raiding. Plus the castle upgrades are really good from a HP perspective - they add a couple of thousand each upgrade, which makes them really hard to attack.