It's a fair price in my opinion. If you don't want to work a bit for your extra workers then leave them. I find it challenging and rewarding to pursue such buildings. Just enjoy the game and have fun attacking. It would take less than an hour to get 3 mil so i don't see the problem to be honest.