It was the Japanese that introduced the "kata shouldn't change" mindset. The Japanse were in pre WWII mindset of military conforming doctrine.
Your post had a lot of good stuff in it, especially about kata being more complicated than some may think, but I'd like to make a few points.
The Japanese standardized kata into a curriculum, true, but I think this began a little before their militarization, done in the interests of teaching effectively in a public-school environment starting during the mid 1920's to early 1930's.
It's important to understand that the original purpose of kata was for the transmission of combat techniques. It was really a
synthesis of a master's concepts. It has been said that each of the old kata formed the basis of a complete fighting style, in and of itself. I believe this is true, but probably not complete, rather a compilation of representative techniques of a particular master's personal fighting style.
Kata is a structured action pattern that spans multiple levels of muscle memory that is purposely repeated in order for the brain to gain and engrain embodied knowledge.
Kata certainly helps with this, however, repetition to perfect karate technique execution is not the purpose of kata. That's what drilling is for. What is not often done, however, is taking the technique/series
out of the kata and drilling them individually, solo and with a resisting partner. After all, these individual techniques or short combos are really what kata is composed of. This kind of drilling puts some of the purpose back into kata.
Kata can show how these techniques are used in dynamic combinations, but its main purpose is conceptual. In other words, a framework of ideas to work off of. This infers that kata can not only (and should) be done as taught but can also be utilized as platform for further non-standard exploration.