You mostly get evil alignment by capturing towns or doing the evil objectives.
Besides I noticed starving villagers can give you a big amount of decrease. Defeating enemy troops outside of their influence shouldn't give you that much of evil alignment points.
