It's due to what's known as the "Equation of Time" -- that our way of keeping time assumes constants that aren't exactly constant.
Earth's path around the sun isn't a circle, it's an ellipse. We orbit faster around the sun in winter when we're closest, and slowest when were furthest away in summer. These little changes mean the sun is not at its highest point in the sky when we say it's noon -- it can be off a few minutes.
For More Information: