Can I leave a challenge?

Yes. Open the recording scree and click on the boxing glove icon . At the end of the rankings you'll see a red button Leave Challenge – tap that. Keep in mind that every time you rejoin a challenge, you have to start the related goal from zero.