I think the prejudice mentioned may come from something I am used to from the study of Japanese Martial arts, Aikido most notably. Aikido talks slot about your ki and that of the opponent. People see the context in which it is noted traditionally in teaching and then see the no touch stuff and dismiss it. Let me give an Aikido example.
There is a term for a series of throws that translates, roughly, into "breath throw." In some more traditional descriptions they speak about inhaling deeply as your opponent attacks, to absorb their ki, and then exhale upon the executing the throw to use their ki to power it. That is bollocks. What you are actually doing is this. In inhaling before you execute the throw and exhaling as it is executed you are loosening the diaphragm so you can have unhindered motion. If your diaphragm was contracted during the throw you would not be able to move as swiftly or smoothly. Then you are using the opponent's momentum (aka ki) to provide the primary motive power for the throw.
You have other similar things such as proper breathing generating ki energy to allow you to fight with a more clear mind. Again bollocks. What one is doing there is, use a natural physical reaction to your benefit. In fight or flight the dump of hormones causes the heart rate to rise. This can (short form because it's really complicated) reduce cognitive function, cause tunnel vision, auditory exclusion due to the fact the body thinks it needs to send more oxygen to power fight or flight vs thought, and cause a progressive degredation of fine motor skills much of this produced by the elevated heart rate.
This elevated heart rate (again all the follows is short form, it's really complicated) also causes a rise in respiration as more oxygen is needed due to more blood being pumped through the body. The thing is you can reduce the rise heart rate, to an extent, with proper breathing. By doing "tactical breathing", in some MA circles it is called dantian breathing, you bring in more oxygen per breath than the rapid breathing the hormone dump tries to force on you. This extra oxygen per breath means the heart doesn't have to pump as fast in order to provide the oxygen required by the fight or flight response. This keeping the heart rate down means that you reduced the effects on cognition, vision, hearing and fine motor skills.
TL;DR. Ki was used in the past to explain things that people observed but could not quantify due to the state of the science of the day. If one perpetuates any of the more "mystical" abilities today though and simply says "ki" instead of the quantifiable causes, similar to the examples I just provided, expect things to be dismissed. It's not prejudice, it's people knowing there are quantifiable answers to why things happen and when confronted with mysticism saying "show me the numbers.". This is called logic.