its not a case of speeding it up to soon, if its failing its to fast, but it should be done at the fastest speed that allows it to be correctly exicuted.. In he real world great speed and less than perfect techneque has a habit of trumping great technique but ponderous speed.
in my fast,drilling, when it gets really fast i have,a bad habit of abandoning the selected technique and doing a more deeply conditioned movement, like I'm supposed to trap the arm and pull, but i hit them with an elbow instead, the end result is good but its not what I'm supposed to be doing, but that movement pattern is what,comes once I'm running on auto. This annoys the instructor but I'm happy as i have an effective move programmed in for any street attacks