IT does speed up. For our main site, only php and DB are now on server, All images (including css images from template), Javascripts, CSS files are loading from S3 and for the past 2 weeks, they are loading from cloudfront.
There is a huge improvement. Our server load has gone down drastically.
Most of the elements are now external links.
Downside, with cloudfront, if you replace any file, it takes hours to reflect the change, but if you add a new file, it gets propogated instantly and you get to download that file from your nearest server.
Give it a test, there are many online services for speed / loading time checking services.
Arvind