Some already fear that there is a backlash coming and think it should be headed off