Refunding the payment and cancelling the membership are two separate actions. Most of the time you'll want to do both.
Refund the payment
Go to Accounting, then Transactions. Find the membership purchase in the list and select Refund. The amount goes back to the card the customer paid with. See Accounting transactions (payment log) for more on this page.
Cancel the membership
A refund does not cancel the membership on its own. To end it, go to Customers, search for the customer, and click their name to open their customer card. In the Memberships section, open More actions and choose Cancel membership. You can cancel at the end of the paid period or immediately. For more on ending a membership, see How can I cancel a membership.