Oyama himself wasn't that small and there's footage of him lifting weights.
In one of his books he says that lifting weighs is the best way of getting strong for younger people (also that strength is the most important part between strength, speed and technique).
Also apparently his mountain...