Another light reading

what is recycle bandwidth ?.When you are running ISP, have many clients behind and meanwhile cost of bandwidth is expensive, and your customer ask faster access because of broadband era. Here solution:
1. Recycle bandwidth
Most of user access the same static content, like youtube, mp3, ebook, etc. This contents are static, why u don’t save this content in a big storage. Cost of storage right now Rp 3 /MB, or Rp. 3000 /GB. Compared by buying bandwidth  at 1200-1500 USD/Mbps (at this year 2008). Let’s say 1Mbps is equal to 128kB/s. I will equal to 128kB x 60 (seconds) x 60 (minute) x 12 (hours) x 30 (Days) = 165888 MB (with asumption 50% utilization during work hour) and Let’s convert 1200 USD to IDR (9200/USD) = Rp. 11.040.000,- and divide this cost by 165888MB, it will be Rp 66/MB.
So ?. It’s better to saving the content (technically called cache) rather than buying bandwidth.

2. Cache most popular site that cacheable.
Look at www.alexa.com,  check it out top 500 sites  categorized global, or  by country. The top ten sites is yahoo, google, friendster, myspace, youtube, rapidshare, etc…. Some sites can’t be cached because of dynamic content like yahoo, google, friendster,etc. But some sites have static content like rapidshare, youtube, 4shared.com, easy-share, download.com etc. You should focus to recycle bandwidth for those sites. Because this site typical hungry bandwidth. How many people access the same video clip britney spears, bon jovi, peterpan (indonesia), ungu (indo), nidji (indo), etc….

I think you got my point now.

download.com/cnet.com is website that have collection freeware/shareware software. But unfortunately, this url link contains ? character. So standard squid will not cache this. You should tune this. And I done it.

so far now we have 90 files from download.com already cached.

Thanks for God, our youtube accelerator already got 50K over video.  So far it takes 500GByte space of harddrive. I hope I could hit 100K or 500K. Do u want ?.

after one week, doing testing, trial, and development. we get following result
1. Running squid for caching youtube content, and compiled also for supporting tproxy, and redirector program to modify url of youtube. Thanks to Andrian Cadd for your inspiration.

1209046095.276 24476 202.169.224.44 TCP_HIT/200 10006381 GET http://cache.googlevideo.com/get_video?video_id=qH3T3cNGhp<truncated> - NONE/- video/flv
1209060914.258 26915 202.169.224.44 TCP_HIT/200 10006377 GET http://ash-v62.ash.youtube.com/get_video?video_id=qH3T3cNGhp8<truncated> - NONE/- video/flv
1209061024.407 23590 202.169.224.44 TCP_HIT/200 11048089 GET http://ash-v105.ash.youtube.com/get_video?video_id=iZ_O7Od74Ec<truncated> - NONE/- video/flv
1209061175.322 23562 202.169.224.44 TCP_HIT/200 10286647 GET http://chi-v91.chi.youtube.com/get_video?video_id=EGtTblTR0YU<truncated>

Why caching youtube is difficult ? we should know basic work of squid/common proxy.
1. Common proxy/squid doesn’t cache dynamic content, url of dynamic content embed “cgi-bin” or “?”, this is not static content but output result may vary depend on parameter passed to url.
2. Especially youtube, youtube issue different signature for same video ID even they output the same content and size. So, squid will have different key md5 for the same object.

Reference:
- mailing list of squid
- squidproxy.wordpress.com

hari Rabu, 16 April 2008…. 7.5 jt entries object cache. bisa gak tembus 10 jt ? berapa lama ? Ada yg punya lebih banyak lagi ?

Internal Data Structures:
7578888 StoreEntries
79803 StoreEntries with MemObjects
78318 Hot Object Cache Items
7546313 on-disk objects
Request Hit Ratios:     5min: 42.8%, 60min: 41.3%
Byte Hit Ratios:        5min: 27.0%, 60min: 27.3%
Request Memory Hit Ratios:      5min: 11.5%, 60min: 10.6%
Request Disk Hit Ratios:        5min: 43.8%, 60min: 42.7%
Number of clients accessing cache:      1021
Number of HTTP requests received:       12129585
Number of ICP messages received:        0
Number of ICP messages sent:    0
Number of queued ICP replies:   0
Number of HTCP messages received:       0
Number of HTCP messages sent:   0
Request failure ratio:   0.00
Average HTTP requests per minute since start:   6813.8

Requirement:
- TV Hauppauge WinTV PVR 500 (installed)
- Pentium 4 or compatible
1. Install Fedora Core 7

2. install rpm -ivh http://livna.rpm.org/livna-release-7.rpm

3. yum install vlc

Apabila anda memakai FC 8, point 2 menjadi
install rpm -ivh http://livna.rpm.org/livna-release-8.rpm

langkah berikutnya sama, yaitu yum install vlc.

Cara di atas, adalah cara yang praktis, cepat dan tepat. Daripada anda mendownload source dan mengcompile satu persatu paketnya.

Berikutnya instalasi driver hauppauge,
download driver hauppauge dari www.ivtvdriver.org/index.php/Download, ada 2 versi driver untuk versi kernel yang berbeda.

• version 0.10.6 for kernels >= 2.6.18 and <= 2.6.21.x, and
• version 1.0.3 for kernels >= 2.6.22 and <= 2.6.23

Untuk mengetahui versi kernel linux anda, ketik uname -a.

#wget http:/dl.ivtvdriver.org/ivtv/archive/0.10.x/ivtv-0.10.6.tar.gz
#tar xzvf ivtv-0.10.6.tar.gz
#cd ivtv-0.10.6
#make
#make install

Setelah itu download firmwarenya,

#wget http://dl.ivtvdriver.org/ivtv/firmware/firmware.tar.gz
#tar xzvf firmware.tar.gz
#cp v4l-* /etc/firmware

Aktifkan drivernya,

#modprobe ivtv

here simple way to mitigating DoS attack.
Let’s assume your linux PC is as gateway / core router, or maybe your linux has ethernet interface that act as SPAN/Port monitoring of your backbone. How to setup port monitoring in Catalyst, search in google ?. And assume you have block IP 202.aaa.bbb.ccc/20

Install ipband,
install httpd/apache (let’s say your document root is /var/www/html)

type following command in root prompt

#ipband eth0 -m 32 -a 15 -r 300 -f “udp && dst net 202.aaa.bbb.ccc/20″ -L “202.aaa.bbb.ccc/20″ -w /var/www/html/ipband.html

After typing above command, point your browser to http://<your linux ip>/ipband.html

Every 15 seconds, ipband will calculate average throughput rate of udp packet of each IP (/32) . Ask your tech support guys to monitor this web. How to make automation to send sms if you suffered flood ?. For example: if any /32 ip has throughput rate of udp traffic beyond 512kbps/seconds for more than 10 seconds, than send sms to your mobile and make blakchole of this traffic and make automation bgp blackhole by advertising this victim IP to your uplink provider.

This is not free, I will don’t put my program at here. Contact me if you want this.

If you’running famous portal, i think speed up access website become main concern. Here there are some ways to speed up the website (from cheapest to most expensive):
1. Use mod_gzip / mod_deflate
2. use reverse proxy in the front of your website
3. put your content at CDN like akamai

I will teach you how to speed up website using option 1. Modul of Gzip/Deflate is doing compression the html content (on runtime) before sent to client. Not just html could be compressed, but also your image, css, shockwave, etc. This proces also called encoding. W3C issues specification http 1.1, one of additional feature is encoding. The web server 1.1 issue header response Accept-Encoding: gzip/deflate to browser/client to indicate that content is transmitted in gzip format. So client/browser need to unzip before display it out. This is easiest one and everyadmin could do this easily. This article is not really something new.
Check it out your httpd.conf, if you’re running fedora please find out mod_deflate.so in apache library. If so, check also in httpd.conf you uncomment
LoadModule deflate_module modules/mod_deflate.so
Activating this module not automatically your server compress your html.

Read the rest of this entry »

Hi guys,
in Docsis there is modulation profile to determine what kind modulation should be sent by modem to cmts. Modulation profile of docsis 1.0/1.1 using qpsk and 16QAM, and for docsis 2.0 using qpsk, 8qam, 16qam, 32 qam, and 64 qam. This modulation determine how much bandwidth could be carried by your upstream. Without clean hfc plant, impossible using higher order modulation (like 32qam or 64qam). Less cascade amplifier and less total number of amplifier, more your chance to using higher modulation. Ok, let’s say you have many cascade and number of amplifier, so your possibility just using qpsk or 16QAM. Let’s optimize this profile to achieve less packet loss, less uncorrectable packet. How ?, let’s play with T bytes, come in…
Read the rest of this entry »

Microsoft sedang membuat project dengan membuat windows companion ke dalam sebuah stick usb flash drive.  Sehingga kemana-mana setting windows dan data bahkan OSnya sendiri cukup disimpan di dalam usb flash drive, dan kita tidak perlu pusing masalah mobilitas. Semua email, isi My document, wallpaper, dan  Windows sendiri tercakup di dalam usb flash.  Pemakai tinggal colok ke terminal komputer maka serasa pakai komputer sendiri. Project ini dinamakan startkey. Sebetulnya ide ini tidak terlalu mustahil, karena usb flash disk dapat diinstall OS sendiri, kemudian PC kita tinggal boot dari USB flash. Daripada bawa laptop yang rentan terhadap goncangan, makan tempat, dan mahal tentunya. Tetapi kecepatan akses harddisk masih lebih cepat daripada flash disk.

Read the rest of this entry »