Gizmodo has a great article concerning YouTube upload speeds getting better in the very near future. Like, next week in fact. For the past year, YouTube has been employing "Sliced Bread," where the various improvements in resolution aren't separate files (which is what they were before April 2012). The "sliced bread" can load on demand should your internet connection be up to the task, and with no interruption in playback. Now, more improvements are coming which will address the loading of the watch page and player, plus some mobile enhancements in which Android users may already be familiar.
A Faster YouTube Is Coming: The Race for 200 ms
The brain perceives anything longer than 200-250 ms as a delay, so YouTube is working to cut down the speed between the time you click and the time the video plays back.
The faster YouTube is explained in this Gizmodo video...on Vimeo:
The other part of the equation is doing less. In this case, it means that if the player is already loaded and you click to another video, there's no point in re-loading the player again, which is what YouTube does every time you click on a new video.
The next part might be a bit controversial. YouTube is going to try to figure out which video you are likely to click on next and start loading it before you click on it. So, in the recommendations and suggested boxes, selected videos will pre-load so that when you click on it, it goes straight to the video, which is neat in theory. But it's going to be nearly impossible for YouTube to get this right most of the time (I'm sure other times it will seem like they're reading your mind). But it can be something useful when it comes to watching videos in a playlist, where the most likely video you watch will be the next one in line. They show that here:
Gizmodo mentions that this doesn't mean you're going to suddenly lose pre-roll ads or anything, so you'll still have that delay, but at least you'll get to that point faster, and if the ad is skippable, all the better.
On to mobile improvements: there is already a feature on the Android YouTube app that allows you to pre-load videos for later watching. All the videos you pre-load will have little green arrows on them and you can watch those instantly. Where this is awesome, and they demonstrate in the next video, is that you can even be on a slow network like Edge and the video can play in full HD and you can skip from one thing to the next without a problem. You do need the internet to play these videos, still. They won't let you have it pre-loaded on your phone to watch without a network. This is because of two things: to make sure the video is still available, and, they still need to show you some ads. The ads they want to show you require an internet connection.
The Android feature should be on iOS devices soon. The desktop speed improvements should be coming by next week, at or around July 4.