How are you planning to celebrate Valentine’s Day? If it’s up to your local retailers, you’ll be showered with flowers, eating chocolates, wearing red and looking longingly into your partner’s eyes for the evening before a passionate end to your night together. In reality, you may be lucky if you even get a card. While many couples happily celebrate their love on February 14, others find the holiday a source of contention—especially when it means more to one partner than another.

Mutually deciding not to celebrate doesn’t always work, either. According to a just-released survey by Bill Me Later and Ipsos Insight, while many couples will tell each other they don’t need to exchange gifts, 20 percent of men said that when they agreed not to exchange gifts, their Valentine was upset when she didn’t receive a present. Maybe that’s why the survey also found that 8 million people had bought Valentine’s Day gifts for themselves.

While you can’t always expect to feel the same way about holidays as your partner—arguments arise whether you’re decorating a Christmas tree or planning a trip for President’s Day weekend—shirking Valentine’s day when you know your partner has some romantic expectations can cause a particularly deep wound. "Not celebrating could signify a lack of love, caring, commitment or recommitment," says Dr. Diana Kirschner, a New York City-based psychologist.

She suggests having 10-minute "listening sessions" with your partner where you take turns talking about how you feel about the holiday while the other simply listens. And make sure you’re clear about what you want. "Say something like, 'I would really love it if you got me those tiny yellow daisies that remind me of when we met,'" says Kirschner. Your mate isn’t a mind reader.

Once you know how you both feel about the holiday, make some sort of compromise. "Having a small, symbolic celebration should be easy for both of you," says Kirschner. "Go to the restaurant where you had your first date, buy a box of her favorite chocolates or write each other love letters."

If the conspicuous day remains a dispute, though, it might be indicative of other problems in your relationship since you aren’t sharing a win-win attitude. If this is the case, it’s especially important to start listening to your spouse and getting to know them all over again. Once you understand their point of view, Kirschner says, you can start to move in the direction of making your partner happy.

However you end up celebrating, don’t let Valentine’s Day be your one and only romantic day of the year. Says Kirschner, "I know couples who basically celebrate it in some small way every day."