With a time limit, and the matching pair of + and - mod points, I don't see a problem (other than implementation) - people would not be coming back a day later to recoup all their mod points to spend on something else. The current situation in the event of a mistake would benefit neither the modder nor the moddee, since it would not be an accurate representation of the modders opinion of a post, and cost a point which could have been spent better elsewhere.
Should the modder lose out further for voluntarily correcting their mistake, or the moddee be punished (if the post is modded down) if they do not?
Also, I'd really rather see no change if the choice is between a confirmation box and nothing at all. The intent was for such a change to be a minor tweak, not another addition to the interface, to inconvenience everyone for the benefit of the few (or the one)