Archive

Posts Tagged ‘ziproxy’

Using Ziproxy to block ads and save bandwidth

June 19th, 2010

Recently AT&T ended unlimited wireless data plan for iphone and ipad and starbucks will offer free wifi. If you are an iphone or ipad user, you might be wondering how to save bandwidth and secure wifi access. If you also own a Linux server with fast internet access, here is how.

Table of Content

  1. Ziproxy
  2. Ad Block
  3. PPTP VPN
  4. Client Configure

1. Ziproxy

Ziproxy is free opensource non-cache proxy to reduce webpage size by recompress image files.
I run a simple test on some of my favorite websites by measuring full page loading size. These data are measured by Firefox, so the flash is included. Images have visible compression artifact at normal viewing distance, but that’s the price you have to pay for saving bandwidth. Ads are stripped away in proxy testing, see next section.


Ziproxy Test

Install ziproxy or from package manager.
Modify /etc/ziproxy/ziproxy.conf

Port = 3128
RunAsUser = "nobody"
RunAsGroup = "nobody"
PreemptNameRes = true
ImageQuality = {65,65,65,65}
URLReplaceDataCT = "/etc/ziproxy/replace_ct.list"
URLReplaceDataCTList = {"image/jpeg", "image/gif", "image/png", "application/x-shockwave-flash"}

Don’t use VPN end point address here, because it doesn’t exist before VPN established. Depends on how you setup VPN, you can either limit access from iptables or binding address. You can also start ziproxy from ppp if-up script to avoid binding address problem.
65% compression has a good balance between quality and size. URLReplaceDataCT will be used for adblocking to further reduce bandwidth usage.

Update: I did some real test on iPad. I used the traffic number on VPN interface to measure the total bandwidth usage to visit all above website once. I also measured multiple times to make sure data convergent. Here is the result
No Proxy: TX 13.3MB RX 1.6MB
With Ziproxy: TX 8.2MB RX 1.5MB
Total: 35% Bandwidth save before VPN overhead.

2. Ad Block

There are several ways to block ads.
First, VPN configure of your iPad or iPhone supports proxy autoconfigure. You can use a special proxy.pac file to block ads. It will redirect urls to a fake proxy.

You can also chain privoxy to your ziproxy.

Another way is url block list used by ziproxy
/etc/ziproxy/replace_ct.list

http://*.yimg.com/*/promotions/*
http://*.doubleclick.net/*
http://*.2mdn.net/*
http://*.serving-sys.com/*
http://*.adtechus.com/*
http://*.eyewonder.com/*
http://*.atdmt.com/*
http://*.fbcdn.net/ads*/*
http://*.advertising.com/*
http://*.smartadserver.com/*
http://*.mediaplex.com/*
http://*.zedo.com/*
http://*.tumri.com/*
http://*yieldmanager*
http://*.checkm8.com/*
http://*.pointroll.com/*
http://*.tubemogul.com/*
http://*.intellitxt.com/*
http://*.crispwireless.com/*
http://*.condenet.com/*
http://*.eyereturn.com/*
http://*.linkstorm.net/*
http://*.optmd.com*
http://*.tcgmsrv.net*
http://*.videoegg.com*
http://adserver.*
http://ads.*
http://ad.*
http://*/ads/*
http://*adsfile*
http://*adfile*
http://*ad_file*
http://*ads_file*
http://*banner*
http://*banners*
http://*sponsor*
http://*sponsor*
http://*/adRequest/*
http://*Advertising*
http://*/adx/*
http://*marketing*
http://*adspace*
http://*adimage*
http://*ad_*
http://*AdManager*
http://*adscontent*
http://*adcontent*

These are urls I found in my testing website.

3. PPTP VPN

Poptop is a PPTP server. Most linux should include it in ppp package.
Here is some sample configure, you can use
/etc/pptpd.conf

option /etc/ppp/options.pptpd
localip 192.168.x.1
remoteip 192.168.x.234-238,192.168.x.245

/etc/ppp/options.pptpd

name pptpd
refuse-pap
refuse-chap
refuse-mschap
require-mschap-v2
require-mppe-128

/etc/ppp/chap-secrets

user pptpd password *

If you want to enable “Send All Traffic” option on your iDevice, you also need to enable nat and ipv4 forwarding. This is only needed if you want to use application which does not support normal proxy. I use shorewall for NAT setup.

As I mentioned in previous section, you can enable ad block in VPN setting with proxy.pac file.

4. Client Configure

Here is example setting for vpn and proxy.
Client VPN and Proxy Setting

If you don’t need ad block. Here is a simple proxy.pac, or you can use manual proxy setting.

function FindProxyForURL(url, host) {
         if ( url.substring(0, 6) == "https:" ) { return "DIRECT"; }
         return "PROXY 192.168.x.1:3128; DIRECT";
}
Bookmark and Share  
 

Yejun Linux , , , , , , , , , , , ,