Are there any limits on Challenges?

Yes: max 3 pending outgoing challenges per team, teams must be within 100 km of each other, no duplicate challenges while one is already pending or accepted, game date must be within 7 days, pending challenges expire after 12 hours, and both teams need at least 5 players to accept.