Then don't place one giant rectangle around your whole base. Good base designs split up their defense in small enclosed sections with walls. In any case, you're only medieval age, so defense is pretty much useless.
Just focus on upgrading and not keeping too much loot before you log. Defense really starts to matter Enlightenment and up. When I was in Gunpowder Age, I had a solid base design and kept low medal count and would probably go a week without someone attacking me even with filled storages.
Would be happy to help you if you wouldn't mind sharing your layout.