You truly can do whatever you like. However, if you KNOW you have a weakness, it would be a good idea to get that weakness taken care of in the event (however unlikely) that someone could exploit it and do you harm.
Obviously, the chances of a skilled grappler attacking you is rare, however you're also a man. If you were a woman, those chances increase exponentially, and should be seriously taken into consideration.
Which leads us back to your post; Certainly if you practice whatever and you do it just for fun, you definitely don't have anything to prove to anyone. The problem comes around when someone who legitimately is looking for a martial art to defend themselves (especially women) and they take up a style that is clearly not going to protect them on the most basic level. I'm pretty confident that I could hold my own versus a boxer/striker, or a skilled wrestler, mainly because I've fought against both many times. Some of them were highly skilled individuals who were trying to knock me unconscious. I've also been rolling against highly skilled grapplers for the better part of a decade now, and many of them are former collegiate wrestlers who are also highly ranked in Bjj or highly experienced in MMA (or both). Again, doing that type of consistent fighting has given me a level of confidence in my ability to handle a wide variety of unarmed threats should they happen to come my way. My ability to survive two self defense situations in the last couple of years only enhances that confidence.
That never happened in the previous styles I took. In my previous MAs we did a lot of kata and light sparring. We did drills full of unrealistic responses to unrealistic attacks. Fortunately, I only got beat by a boxer in the safety of my dojo, not the street. If it had been the latter, I shudder to think the level of damage he could have done to me because I lacked the tools to defend myself on even the most basic level.
In short, if I had studied a martial art for years and said that any highly skilled karate guy could walk up and beat the crap out of me, I'd have a serious case of buyer's remorse.