I was always under the impression that the pitcher had to have a foot in contact with the rubber as he was releasing the ball.
The current rules in the Definitions section says:
The pitcher’s PIVOT FOOT is that foot which is in contact with the pitcher’s plate as he delivers the pitch.
A rule book from the early '50s says: Rule 8.05 If there is a runner, or runners, it is a balk when--
(e) The pitcher delivers the ball to the batter while his pivot foot is back of, or not in contact with the pitcher's plate.
The way I interpret the above is that Capps' delivery can't be legal, but then I don't understand the way umpires interpret some of the rules anyway.