Don't worry about it. You are expected to be able to walk over most of your targets at this power level. When you run into Deathlords (which I'm planning on doing, by the way), you will need every Charm you can get your hands on to come out alive. As I said before, this is more a nation building game than a low level hack and slash game.