A lot of work and effort went into getting TKD into the olympics. Regardless of my personal opinion of the olympics and the IOC, to have put decades of work into getting in and staying in, only to have it all undone, well, yes, that is a serious blow.
It would damage the reputation of the art, cause greater schism and animosity between taekwondoin, and would probably leave a lot of people angry and bitter. Ramifications for athletes are fairly obvious.
As for taekwondo out here in suburbia? Probably would continue along, business as usual.
Daniel