I will clarify a bit. Standing side on presents your opponent with a smaller front-on target, blocking (deflecting) a straight punch to the head may only mean moving your arm 2 or 3 inches. If they want to hit you in the head or body then they have to go around your arms and your elbow protects your ribs from front kicks (ever copped an elbow to the top of the foot or the shin?, Very painful, not much good for the elbow either). To hit the front of your body with a kick or hand strike they have to come from the side, which takes longer. When standing side on one hand is in front of the other, which gives you two lines of defense, if the front hand is trapped or knocked away you have the rear hand as backup. If you are standing completely front on, usually both hands will be roughly the same distance in front (I see it all the time in MMA fights, not taking a dig, just an observation), this means that both hands can be trapped or knocked away at the same time and also leaves a convenient gap between the arms for those one-two punch combinations to get through which just isn't there when you are standing side on. So not "nothing to worry about", just less.
I used to spar in this type of stance. I also used more of a boxing type stance as well. I had success with both. When I started to spar in Kyokushin, I found myself in that side stance at one point, mainly due to force of habit. I ate more than my share of kicks to the leg because of that stance. I was told by my teacher, as well as my class mates, that I needed to change that stance, as it wouldn't allow me to have much success with blocking the low line roundhouse kick.
Now, I'm not saying this is a bad stance, that's not the case at all. I'm simply saying that in that situation, it didn't work for me. I assume that you use this side stance yourself? If so, is it the only one you use or do you have others? Are you able to deal with leg kicks from that side stance?