This concept is close to the one I have for a roleplay alt character of mine, so I have an outline of that build planned. No idea how viable it will be late-game, though.
All charmer Perks are focused on inflicting Resolve damage and sustaining your party's Resolve and all the class damage dealing powers are spells. So if you want to optimize weapon-based DPS or defensive stats like armor and evasion, you will have to class-swap into a warrior or a thief. If you value roleplaying more you can stay a charmer.
Additionally, if you want most of your damage coming from the actual swashbuckling, your best bet are Song of Courage, Bolstering Dance or Duelist stance in the At-Wll slot, Blinding Beuty combined with a damage dealing Power of your choice from warrior or thief class kit in your Recharge slots and eventually Aria of Splendor in your Encounter slot. That way you don't need Willpower and its derivative stats, so you can focus on Resolve, Agility and either Strengths/Cunning for damage or Toughness for HP. Can balance all three non-essential stats equally. Combined with a decent healer this build will hopefully live for long enough to buff allies and deal decent damage over the course of the fight despite lacking good alpha-strike oprions,