I've found knowing where to look is a HUGE deal to learn. Learn how to ignore the little things that make you cautious, let your body take over, and stay smooth. You can process information a lot easier looking far ahead. If you get caught up and look further down it will slow you down tremendously.
I've only been back riding for three years yet I've improved tremendously. I love getting "man, you've gotten so much faster!" The little things you learn to improve add up.