Some conditioning on the heavy bag or the makiwari isn't going to be detrimental. If anything, using it as feedback to see if you are punching with good mechanics can be helpful.
This results in your knuckles getting calcified, which makes them harder, but there comes a point where too much isn't good for you.
Unfortunately, some people go too far in such training, and end up destroying the nerves in the hands. Once this happens, everyday tasks can become difficult. Masutatsu Oyama, for example, was well-known for his toughness, and that his knuckles were capable of withstanding blows from a metal hammer.
However, people who knew him personally, will tell you that Oyama's hands would shake uncontrollably due to the accumulation of damages.
If you do engage in toughening excercises, make darn sure you take care of your hands, before and after. Don't overdo it. If you notice that your hands start trembling when you hold a cup of coffee, then stop right away.