Why do the times matter to the swimmer. If your trying your best knowing the tines doesn't matter you do your best and your best us either fast enough or its not. If knowing the time males you swim harder then your not trying your best.
And if knowing the times is what you need to swim your best than that's what you need. If a swimmer performs better when they know the required times than they need to know the times. Knowing the times is an entirely psychological effect but a psychological effect can be a very strong one with performance.