The men's players who visited the Oval Office have also earned the ire of many.