Before the quest, Calise takes you out to dinner, and at the end you have the option of declining to extend the date after getting a missive inviting you and Calise to the party. Declining makes it where she isn't in your party for the Feast of Heroes. I understand the confusion. When I first played this quest, I thought declining Calise was declining the quest itself. My thinking was whether I get the quest or not rides on going with Calise. But on another playthrough, I deduced that the invitation was for the both of you separately, but was presented as a joint offer because the 2 of you are obviously on a date.