“We have much less protection over our right to vote than most people think.”