“I chose to be unvaccinated, and that was my choice,” Irving said on Wednesday