My answer:
The question is inherently loaded with the word "should", and it is this point in which we should start our investigation. The word should carries with it a certain gravity, it implies not only that we are judging the driver, but that there is an answer to the question. What is the "right" thing to do? This suggestion of objectivity is essentially incorrect. To believe in the word should, we must justify an absolute morality, either from God, or from societal values or an inherent (for example genetic) moral code. Yet no such morality exists, as far as we are aware.
Let me first tackle the question of God. It is true that if God exists that we are committed to His values, and therefore have an obligation to save as many lives as possible. Yet even so the situation is not clear. What if the five people were sinners and the one a righteous man? Should the thousands at Sodom and Gomorrah die for the good of the righteous elsewhere? Likewise, if God created the situation, and if He is aware of what our action in this situation will be, is there not a certain fatalism to the entire scenario? Are we to believe that God, like the majority of us, shares a utilitarian value in this instance?
The utilitarian ethic cannot hold true in this scenario, for the consequential view cannot be properly quantified. We may kill a man who was to cure cancer at the expense of five criminals - we simply do not have the information to make a proper decision. Therefore to claim that one was "right" in killing the one person is self-righteous at best. The objective morality in this situation cannot be established without an impossible understanding of the facts at hand, and all future consequences, which as we know is impossible. One thousand years from now, it may turn out that killing the five was the right thing to do - but we have a very limited understanding of the situation in which to decide.
Therefore all that can exist is a subjective morality, or a morality which the majority of persons share. We either believe that the man has the authority to make such a choice or that he has no authority to make such a choice. In the second, all actions can be considered immoral. "Who are we to make such a decision?" you may ask. No man has the right to dictate another man's life. Yet the decision presents itself nonetheless, it is inescapable. We may therefore concede that the driver, in this instance, has the authority to make a decision. He is, we would hope, qualified as a trolley driver, and therefore holds responsibility over what occurs.
Will society judge the driver for killing the one or the five? We don't normally hold people accountable for accidents, yet we may judge them based on our subjective opinion, ala Kant. The movie I, Robot springs to mind, where the main character was angry at a robot for making a moral decision not to save a child, based on sheer mathematics. The protagonist felt that basing such a decision on pure logical probability was unethical in itself. The reality of the outcome of the situation is that the only person who will judge the driver is the driver himself, and he therefore holds responsibility for his own actions. He may decide to kill five adults rather than harm one child, he may judge the persons based on how they look or who they are. Perhaps he would hit five workers to avoid hitting one non-affiliated citizen. Either way, there is no "should" about the question, and therefore I believe the driver is entitled to do whichever decision he wants to do, and will offer him no blame either way, for being the victim of such an unfortunate situation.