I noticed it too and can ruin some attacks... My guess is the cannon AI prioritizes the higher HP structures such as mills and storages. Also, I noticed defenders march right past my cannons - not complaining but it did create some anxious moments. You don't want to take your cannons away...