I went ahead and connected to the postgres DB and deleted them manually. I suspect that because they were not able to join the team, their users were created but not associated to the team. Then when trying to get in, they were going in via the "team" URL, which is why they would get invalid authorization (as they were not part of the team).
Probably opening the team would have worked. Thanks