Every time you do damage to the ghosts, the combat text informs you that they don't really want to fight. It's a good indication to do something that, uh, doesn't involve fighting with your combat move. This could be surrendering, as Malidica says, but "Defend" is also a perfectly cromulent option.