Archive

Posts Tagged ‘cloudfront’

CDN Latency Compared

March 21st, 2009
3 comments

I compared some puplar commercial CDN’s global latency by using just-ping.

The raw test result you can downloaded from here.

Result Summary:

.

 Network Average Median

.

 Akamai 9.67 2.9

.

 Aol CDN 9.22 4

.

 Panthercdn 62.36 12.7

.

 LimeLight 58.04 13.2

.

 Mosso Cloud Files 56.81 13.5

.

 Amazon Cloudfront 62.82 18.6

.

 Google Homepage 53.53 23.15

.

 Cachefly 54.57 28.2

.

 Google Ajax Library 54.96 28.5

.

 Homemade CDN 76.31 29

.

 Yahoo Homepage 82.77 38.4

.

 Google App Engine 76.03 42.8

.

 US East 130.11 96.9

.

 SimpleCDN 142.84 100.8

.

 US West 156.32 165.4

Chart:

CDN Latency Comparison

Notes:

  1. Akamai
  2. Aol CDN is served by Akamai
  3. Panther Express
  4. Limelight Networks
  5. Mosso is served by Limelight
  6. Amazon CloudFront Images on this page are served by CloudFront
  7. Google
  8. CacheFly
  9. Google AJAX Libraries
  10. This page is served by my own homemade CDN, you can test speed here
  11. Yahoo
  12. Google App Engine
  13. Single location ip in New Jersey USA listed here for comparison purpose
  14. SimpleCDN
  15. Single location ip in California USA listed here for comparison purpose

Disclamer: I am not affiliated to any company mentioned above. Nor do I endorse the accuracy of these results.

Bookmark and Share  
 

Yejun Web , , , , ,

How to copy selected files to cloudfront

February 24th, 2009
2 comments

If you use my WordPress CDN plugin and amazon cloudfront, you may have problem putting files into s3 storage. Here is a simple way without using any commercial tool if you are using Linux.

First download s3sync . Extract it to somewhere. In this example I used my home directory.

mkdir ~/.s3conf

Edit ~/.s3conf/s3config.yml, which should looks like this

aws_access_key_id: your s3accesskey
aws_secret_access_key: your secret key

Enter wordpress directory

cd wordpress
find * -type f -readable  \( -name \*.css -o -name \*.js -o \
    -name \*.png -o -name \*.jpg -o -name \*.gif -o -name \*.jpeg \) \
    -exec ~/s3sync/s3cmd.rb -v put bucket:prefix/{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 \;

Change bucket to your real bucket name. If you don’t need any prefix, do not include slash. Adjust cache-control header. ~/s3sync/s3cmd.rb should point to where you extracted s3sync.

Update 1: Don’t forget install mime-types if you Linux distro didn’t install it by default. Check whether /etc/mime.types exists.

Update 2: s3cmd.rb does not set content-type at all. I think python version does. Any way I wrote a script to redo everything.

#!/bin/sh
 
BUCKET=
#Set your bucket
PREFIX=
#If you want to use prefix set it like PREFIX=blog/
 
find * -type f -readable  -name \*.css -exec ~/s3sync/s3cmd.rb -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:text/css \;
 
find * -type f -readable  -name \*.js -exec ~/s3sync/s3cmd.rb -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:application/x-javascript \;
 
find * -type f -readable  -name \*.png -exec ~/s3sync/s3cmd.rb -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/png \;
 
find * -type f -readable  -name \*.gif -exec ~/s3sync/s3cmd.rb -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/gif \;
 
find * -type f -readable  \( -name \*.jpg -o -name \*.jpeg \) -exec ~/s3sync/s3cmd.rb -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/jpeg \;

Update 3: I just realized cloudfront does not gzip files. So I rewrote my script to force gzip encoding on css and js files.

#!/bin/sh
 
BUCKET=
#Your bucket
PREFIX=
#If you want to use prefix set it like PREFIX=blog/ 
S3CMD=/home/user/s3sync/s3cmd.rb
#Your absolute path to s3cmd.rb
 
find * -type f -readable  -name \*.css -exec sh -c "gzip -9 -c {} > /tmp/s3tmp && \
        $S3CMD -v put $BUCKET:$PREFIX{} /tmp/s3tmp x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:text/css Content-Encoding:gzip" \;
 
find * -type f -readable  -name \*.js -exec sh -c "gzip -9 -c {} > /tmp/s3tmp && \
        $S3CMD -v put $BUCKET:$PREFIX{} /tmp/s3tmp x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:application/x-javascript Content-Encoding:gzip" \;
 
find * -type f -readable  -name \*.png -exec $S3CMD -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/png \;
 
find * -type f -readable  -name \*.gif -exec $S3CMD -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/gif \;
 
find * -type f -readable  \( -name \*.jpg -o -name \*.jpeg \) -exec $S3CMD -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/jpeg \;

Update 4: 4/1/2009
I added function to copy a single file or single directory.

#!/bin/sh
if [[  -n $1 ]]; then
LOC=$1
else
LOC="*"
fi
 
BUCKET=
#Your bucket
PREFIX=
#If you want to use prefix set it like PREFIX=blog/ 
S3CMD=/home/user/s3sync/s3cmd.rb
#Your absolute path to s3cmd.rb
 
find $LOC -type f -readable  -name \*.css -exec sh -c "gzip -9 -c {} > /tmp/s3tmp && \
        $S3CMD -v put $BUCKET:$PREFIX{} /tmp/s3tmp x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:text/css Content-Encoding:gzip" \;
 
find $LOC -type f -readable  -name \*.js -exec sh -c "gzip -9 -c {} > /tmp/s3tmp && \
        $S3CMD -v put $BUCKET:$PREFIX{} /tmp/s3tmp x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:application/x-javascript Content-Encoding:gzip" \;
 
find $LOC -type f -readable  -name \*.png -exec ${S3CMD} -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/png \;
 
find $LOC -type f -readable  -name \*.gif -exec ${S3CMD} -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/gif \;
 
find $LOC -type f -readable  \( -name \*.jpg -o -name \*.jpeg \) -exec ${S3CMD} -v put $BUCKET:$PREFIX{} {} \
    x-amz-acl:public-read Cache-Control:max-age=604800 Content-Type:image/jpeg \;

For example if you saved this script to a file name cloudfront:

cd wordpress
cloudfront wp-content/uploads

Without any command line argument this script will upload all file under current directory.

Bookmark and Share  
 

Yejun Linux, Web , , , , ,