I think it's essential. If you don't grow and improve, both in terms of physical fitness/ability and in terms of personal character, from studying martial arts.... what have you gained, really? And pushing yourself, and being pushed, are how you grow.
Edit: Also, being pushed isn't always about physical exercise. It can also be about personal growth - dealing with your own ego or insecurities or how you relate to other people. Confronting those things in ourselves can be much harder than doing 10 rounds of sparring or 300 roundhouse kicks.
Exactly. And you're especially right about confronting difficult challenges that might not be physical. I had that problem myself but I've worked on it and I keep working on it.