Using avatars direct is fine we designed it for large scale use, we added the proxy config cause some people had concerns about tracking (not that we even store logs for avatars but yeah, concerns)
You are going to have to punch through a lot more than avatars though:
that button is broken cause gravatars are not punched through, look up by IP in admin is broken as well.
Also every onebox is not pretty like it should be:
^^^ This times out over and over causing a mini DOS, which is far from ideal.
It should look like:
Other major issue, you got to put a CDN in front of your S3 (just add cloudfront to the mix or whatever), S3 is not a CDN loading images is slow and expensive. (once you add the CDN via site settings be sure to do a full rebake)