PCI Express Throughput

x1 : 0.5GBps (4gbps)

x2 : 1GBps (8gbps)

x4 : 2GBps (16gbps)

x8 : 4GBps (32gbps)

x16: 8GBps (64gbps)

x32: 16GBps (128gbps)

Hacking ASUS WL520GU, Installing openWRT

Last week I tried to hack the Asus WL520GU wifi router, after reading this article - Building a Wifi Radio. Though you do not need to have a serial console to hack it, It makes all the sense to have one, just to know what is going on in your router. I bought some 0.1" male header on-line and soldered in the right place as shown in the mightyOhm article. I hit a road block when i found that i needed a serial driver to connect Asus Router to serial cable.

Router gives out 3.3V TTL signals. Whereas a RS232-serial port needs +/-12V signalling. If you connect a serial cable straight to router's header, it will not work. You need a chip to convert TTL level to RS232 level signals. The article mentioned about driver FTDI-232-3V3 which I could not find on-line in India. I tried this PC-MCU Serial Link (USB) converter, which is available on-line robokits.

This PC-MCU has a Female-header that does not work as signals order is different. You need to solder wires and 0.1" 4pin-female header as per the order given in the mightOhm article. Just leave the VCC pin as the driver chip gets power from USB. Connecting Gnd,Tx,Rx lines is enough to make it work. After you prepare the driver cable and connect to header and other end to USB of your laptop. Device Driver needs to be installed in order for the laptop to recognize the usb device as a serial COM port. After inserting the chord and driver, I get a new COM4 device to work with. I used Putty to connect -115200-8N1.

The mightyOhm article provides a Kamikaze build, Flashing procedure is straight forward. It was simple and trouble free to install kamikaze. Soon I realized that the kamikaze runs old 2.4 kernel. Sure enough, I had to upgrade to Backfire, which i did immediately.

miniHOWTO: Install FC14 in a Headless Server

I was trying to install a desktop centric Linux Distro like FC14 into a ATCA Blade based on intel. I and My co-worker tried to install as mentioned in my previous post.
FC14 did install but with minimal packages. It did not have basic utils like ftp. Need less to say, yum did not work. I let him work on it and crack it, which he did over a Sunday afternoon.

Following was his report:

--------------------

OS:- FC14x86_84

Mode of Installation:- Serial Console ( Using Minicom for installing ) and USB-DVD.

First boot the system using DVD drive and when the installation menu comes press Tab for editing the input commands. Then type,

linux console=tty0 console=ttyS0,115200n8 ( Enter)

After installation I assigned an IP and issued the command yum update I had to face to problem like yum was not working and I was getting the Error,

[root@localhost~]# yum update

Error Cannt retrieve repository Metadata ( respond.xml) for Repository:Fedora. Please verify its Path and try again.

Then I Google and found out some solution,

Problem was with the Python, it does not handle https proxy well. So I change all the https present in *.repo files in /etc/yum.repos.d/ to http

Here’s the way to do it… edit your repository files /etc/yum.repos.d/fedora.repo and /etc/yum.repos.d and /fedora-updates.repo by commenting all the lines starting with term mirrorlist and uncommenting all the lines starting with term baseurl.

After that my yum started working fine for me.

Referenece URL:-

http://rishabhsays.wordpress.com/2010/01/24/error-cannot-retrieve-repository-metadata-repomd-xml-for-repository-fedora/

--------------------
Thanks: S.Dhananjaya.

"ஜெ.சுயநலம் இல்லாதவர் " சாருவின் கருத்து

"ஜெயலலிதா சுயநலம் இல்லாதவர் என்றே நான் நினைக்கிறேன். கருணாநிதியின் குடும்பத்தில் சுமார் 500 பேர் இருப்பார்கள். தமிழகத்தின் அத்தனை துறைகளும் அவர்கள் கையில்தான் இருந்தன; இருக்கின்றன. ஜெயலலிதா ஒரே ஆள். உலகில் தனித்து வாழும் மனிதர்கள் சுயநலம் கருத வேண்டிய அவசியமே இல்லை." - சாரு நிவேதிதா

என்ன கொடுமை சார் இது?

HOWTO: Install centos 5.4 OR RHEL 5.5 over serial console

I had to install a processor blade which could boot from USB but it didnot have VGA & Keyboard. Serial port was the only option. I tried to do like this.

It did not work. Later I fugured, I do not need to modify the ISO. I could do it so easily. The procedure is roughly like below.

1) Insert your bootable DVD and connect your DVD-Drive to USB.

2) Make the BIOS boot from USB Drive.

3) Prepare your putty/minicom suitably. 115200-8-n-1 worked for me.

4) You will see following menu.

5) Select F4 to type in command line arguments to kernel.

6) Type in

boot:linux console=tty0 console=ttyS0,115200n8

7) press ENTER and you will see booting proceeding with logs showing in your console.

Happy Exploring.

Buying time?

The nifty trades around 5663 today, reflecting the P/E of sub 22. Analysis shows that the market had traded below 22 only on 30% of the days last year. So, based on 2010 data, the Nifty is correcting from expensive region.

Is it cheap enough at current levels? Are market headed further south? I do not know. But, I think it makes sense to start accumulating from these levels.

Packet Length Distribution in LAN - II

After running the experiment for about 10 days, the table looks like this...

Pkt Histogram
==============
Length %of total packets
=========================
0-63 21.28
64-127 0.69
128-191 5.43
192-255 0.83
256-319 0.56
320-383 0.03
384-447 0.04
448-511 0.06
512-575 0.07
576-639 0.07
640-703 0.05
704-767 0.05
768-831 0.05
832-895 0.03
896-959 0.04
960-1023 0.03
1024-1087 0.03
1088-1151 0.04
1152-1215 0.04
1216-1279 0.1
1280-1343 0.02
1344-1407 1.42
1408-1471 0.02
1472-1535 69.01

Watson, What's next? - Ultimate knowledge economy.

IBM has built a new super computer program called Watson and it is capable of playing Jeopardy quiz game against humans and winning them hands down. This amazing super computer is a result of deepQA project which "understands" human natural language and answers the question from its knowledge built again by processing millions of articles written in natural language. So, this is how the Watson plays the game. Just like other human contestants, it listens to the "answer" on select topic, digitizes the voice, process and understand the hint. Searches its knowledge base, comes up with correct "question" before others and presses the button. Then when asked, Watson would 'speak out' the question in natural language. If the "question" was correct contestants get points else they get to loose points.

In other words, computer program can read and understand any printed book, news paper archive and correlate events hence build its 'knowledge'. Here is an interesting video by lead scientist David Ferrucci of deepQA project. What interested me at first is that Watson is improving its answering accuracy since three years and now it can beat any grandmasters of Jeopardy game. Another piece of detail is Watson is built by networking what seems to be server-grade computers. It runs 3000 Cores simultaneously. It is NOT a keyword matching search engine.

Ok, now let me speculate little bit here what might happen next:

1) Immediate: Dawn of cog-engine
IBM might roll out its next version of "search engine" whose results are far better than Google's. They might probably call it "Cog Engine" than search engine. Who knows, Google might already be working on something similar already. It is not IBM or google, point here is "search" as we know it today might change dramatically.


2)Immediate to near future: Worthless "Research"
"Research" would loose its meaning. Anyone who has access to Watson type programs could produce the results in a day or two what today's researcher would produce with years of effort. So, it is imperative that either human researcher find something really complex to research on OR they research on improving watson algo itself!


3) Immediate to near future: Hand-held Watson
You would be able to conduct the research/learning/QA from your mobile devices. Mobile phones would be so powerful that they could easily do the "knowledge retrival" part. "Knowledge gathering" part may be done by relatively powerful servers on the cloud.

4) Far future: Download F-16 flying skill
Some one will eventually figure out how human brain analyses the natural language and makes decision vis-a-vis Watson. Watson is made to mimic the brain in the first place any way. Once that is done, the "knowledge" can be freely moved across machines and humans. Think about it, computers can learn a subject and move the "knowledge" to humans' brains and vice versa.

5) Digitize the skill:
Logical next step would be to digitize the "skill set". Till today, humans acquire knowledge individually. That means, One has to learn a subject by reading books, practising the art etc basically by putting in time and effort. He or she eventually becomes expert in the subject. Then He/She dies. Only way to pass on the expertise or knowledge to other people is by writing a book or articles or by giving speech and so on. So, there is no "continuity" in the knowledge gaining process. Now, if one can digitize and pass on the "expertise" to others as he/she left it without needing to write a book and others to read and work on it to get to that point.
Imagine you could download Linus Torvalds' operating system knowledge and Larry Page/Sergey Brin's analytical knowledge and Steve Job's product finish and marketing right into your brain!
No one would ever need to write a book, article, papers, exams and so on. you could directly submit your brain map for evaluation when you graduate out of college! Implications are profound.

6) Cog-market: sell your skill for the stay at ski resort
Imagine you have knowledge market, where various skill sets in various maturity level gets traded like stocks in today's stock/commodity market. one could learn any language in a jiffy. There could be some rating agency that grades the quality people's knowledge. So, people will be become Hardware that constantly work and improve the humanity's software - knowledge system that is.

7) Far into future: The Big Leap
We humans, have taken 10,000 years since we organized ourselves as civilization to develop steam engine (invention in mid of 18th century)and hence industrial revolution. It just took us less than 50years since we got our practical computer to get to where we are now. Previously, it would take decades to conceptualizing a product , developing a product to seeing the product in market. Now-a-days, it is the matter of years some time within a year.
Rightly, we are already on exponential growth trajectory in improving our collective cognizance. This is primarily because we are building stuff on the knowledge we have gained so far. The predictions which I have dared to make, if comes true... if at all, then the exponential growth is not just going to grow faster. It is going to be a complete game changer. our growth path would be hyper-exponential or hyper-hyperbolic.

The exponential growth seems very impressive given where we started from. But in absolute terms, humanity is still sub Type-1 Kardashev scale. We are at 0.7 in kardashev scale while aliens/extra terrestrials are expected to be of Type-II or even Type-III civilizations. Ultimately this hyper-hyperbolic growth would help propel our civilization from the Type-I into Type-II.

I would think it would all or most of it happen in next 3, 4 or 5 decades of time. We are going to live in interesting times. The Chinese curse, "May you live in interesting times" is more truer for this generation than anybody else.

mini Howto: write your own iptables target module

Howto Extend the iptables

I searched the existing iptables target/match modules whether they would help me collect the packet length statistics of LAN. I set out to use iptables infrastructure because iptables offered a nice packet management framework. I did not find a readily available target so, I decided to write my own.
What i found was it is surprisingly easy to write your own target modules.

Design:
My target module should inspect the length of the packets it receives. Iptables rules can be setup to send packets from specific source/destination/port number/application/time of day etc to the newly written module. The module, builds a array of 32 integers, which is modifiable at compile-time. The array elements act as a packet counter. array[0] would count the packets whose length varies from 0-63 and so on. My sampling step size is 64, which is again programmable at compile-time.

I used proc entry in order to access these counters from user space. For time being I have avoided netlink socket(elegant solution) and kprintf(beginner solution) both would work.

The module has entry and exit functions. Entry function creates proc entry and registers a target function. Write the target function itself and fill up the xt_target structure. Use this structure to register with iptables as a new target. Exit function de-registers the target and removes the proc entry.

The target function gets the skb pointer to work on. Yes, now we can do anything with the packet. The target function gets called by iptables with the packet (skb) as an argument and target module's argument as typed in iptables command as second argument. To keep things simple, I have decided to have my module without second argument. so, it just receives the packet.I get the packet length and quantize the length into 32 steps, and increase the appropriate counter.

To export the constructed histogram, I have written the output function which is invoked whenever user attempts to read the /proc/pkt-hist file. This function just prints out the array.

PS: Extending the iptables involves writing actual target or match modules which does the job in kernel space AND writing supporting user space library to parse the arguments and let the iptables know what to do when a target is typed in the command line for e.g
iptables -I FORWARD -s 192.168.x.y/32 -d 0.0.0.0/0 -j PHIST


Steps:
1) Make sure you have latest kernel code downloaded and netfilter option enabled. I had linux-2.6.32. Also make sure the /usr/src/linux points to /usr/src/linux-2.6.32/ where i have the linux source is available.

2) Get the iptables source code and untar it. Do ./configure;make;make install - usual procedure. I had iptables-1.4.3.2.

3) Implement the target: easiest way to write your own target is to take a reference from existing target modules. go into /usr/src/linux/net/netfilter/. You will find all the match and target modules supported by that netfilter(kernel). All match modules are in named in small letters while targets are written in caps. for eg, xt_mark.c is a match module, while xt_MARK.c is a target module.
I have copied the xt_NOTRACK.c which is a very simple target module and fits my bill. I have removed all code inside the target function and written my own. Renamed all the functions, structures appropriately. In entry function i have created /proc/pkt-hist file. I have also added required call back code for /proc/pkt-hist.

4) Modify KConfig: Now, in /usr/src/linux/net/netfilter/ open Kconfig in vi editor. search for NOTRACK, you will find
" config NETFILTER_XT_TARGET_NOTRACK
tristate '"NOTRACK" target support'
depends on IP_NF_RAW || IP6_NF_RAW
depends ....
"
If you recollect, this is what appears in "make menuconfig" program. So, copy-paste this entire section into Kconfig as below:
config NETFILTER_XT_TARGET_PHIST
tristate '"PHIST" target support'
depends on NF_CONNTRACK
help
The PHIST target collects the packet distribution
in pre-set steps.

If you want to compile it as a module, say M here and read
. If unsure, say `N'.


5) Implement user space library: go into /usr/src/iptables-1.4.3.2/extensions
and you will find libxt_NOTRACK.c, clone the same into libxt_PHIST.c and modify the names of functions appropriately.

6) compile and fly
Compile Target:
In /usr/src/linux/net/netfilter/, edit Makefile. Search for NOTRACK. you will find the following
obj-$(CONFIG_NETFILTER_XT_TARGET_NOTRACK) += xt_NOTRACK.o
Now, clone this line for PHIST. see below:
obj-$(CONFIG_NETFILTER_XT_TARGET_PHIST) += xt_PHIST.o

Now, At /usr/src/linux/ do make modules;make module_install. you will find xt_PHIST.ko in /usr/src/linux/net/netfilter/
Compile iptables userspace: repeat step (2). you will find libxt_PHIST.so in /usr/src/iptables-1.4.3.2/extensions/

7) Now try typing
iptables -I FORWARD -s 192.168.x.y/32 -d 0.0.0.0/0 -j PHIST
Now, iptables finds the xt_PHIST.ko and loads it and send the packet satifsying the rule to the target code which is just written.

So, the target module worked like a charm may be due to it is not doing much. But it was a breeze to write a new target. I guess writing match module wont be entirely different from what is described here.

Packet Length Distribution in LAN

I this post, I have copy-pasted from internet (ofcourse, citing the author) the packet distribution in internet. I always wanted to do that experiment myself. Last week, finally i could do just that. This is just a first experiment where I have captured all my own packets going into LAN, not purely internet.

Setup was similar to what i have done in my previous post. Replace "LAN cloud" with my PC and "wan cloud" with my office servers and internet. Linux box is a dual NIC-PC which runs FC10. I have configured the "Linux box" as a bridge. I wrote a iptables target module and passed all packets emerging from my PC through the module.

The Target module I wrote, looked and sampled the packet length of all packetes it receives and built a histogram. The histogram was built with the step size of 64bytes. After running it for couple of days, the result looks like below:

Pkt Histogram
==============
Length %of total packets
=========================
[ 0- 63]=18.91%
[ 64- 127]=0.84%
[ 128- 191]=5.76%
[ 192- 255]=0.47%
[ 256- 319]=0.8%
[ 320- 383]=0.05%
[ 384- 447]=0.08%
[ 448- 511]=0.09%
[ 512- 575]=0.13%
[ 576- 639]=0.17%
[ 640- 703]=0.09%
[ 704- 767]=0.06%
[ 768- 831]=0.05%
[ 832- 895]=0.03%
[ 896- 959]=0.03%
[ 960-1023]=0.03%
[1024-1087]=0.04%
[1088-1151]=0.04%
[1152-1215]=0.04%
[1216-1279]=0.2%
[1280-1343]=0.02%
[1344-1407]=0.62%
[1408-1471]=0.03%
[1472-1535]=71.44%
[1536-1599]= 0%
[1600-1663]= 0%
[1664-1727]= 0%
[1728-1791]= 0%
[1792-1855]= 0%
[1856-1919]= 0%
[1920-1983]= 0%
[1984-2047]= 0%

Now, I am tempted to the same exercise on the packet flowing purely into internet from my office. I will do the same and report in coming days. so, keep visiting this page or just subscribe to the RSS feed of my page.

PS:
My Work-PC is also a Linux machine running FC13. The packet captured reflects regular web browsing -http/https, email, and NFS (my home is mounted on nfs) and NIS.

Coming up next: How to write a iptables target module... Stay tuned.

An Amateur's Analysis of Nifty 50

How to time the market

Is it a right time to enter market? is there a potential upside from here? If you ask these questions to value-investors, they would say look at P/E. If P/E is high, the stock is said to be expensive and if low, the stock is said to be cheap. Buy a stock when it is cheap and sell it when it is expensive. simple, right? But the problem is how to establish if a P/E number is 'high' or 'low'? Pundits would ask you to take a look at the industry in which the stock in question belong and compare it with its peers.

To me, any single stock is too much of a risk. Even if you are an employee in a company there is a high probability that you wont know what is going on 'in-side' the company. Remember Satyam/LIC HF/Enron? Nor I have time to construct and prune a portfolio. It is too much of an effort. So, I would put my money into index ETF, which i think is a simple investment strategy that would not drain my time at the same time, gets me decent returns by the virtue of equity exposure. My favourite vehicle Niftybees is quoting about 589,which would be 1/10th of index value (with tracking error of about 1%). I don't want to blindly buy niftybees to get just "Rupee cost averaging". I want to restraint from buying whenever market is too expensive (hence build up cash) and resume to do SIP when it is not. As on 29 Nov 2010, nifty is trading at 5830 with current year's P/E of 23.

So, the question is, Does it make sense to put more money into this ETF or should i wait on the sidelines? Let us see if we could analyse the nifty index and try to arrive some answers.

Before I start, Let me tell you I am going to treat nifty index as a stock. I have downloaded the Nifty Index P/E from www.nseindia.com site from Jan 2002 till today.

At first cut, I get following

Jan 2006 - today
Average: 18.1 Min:10.7 Max:28.3 Std Dev: 3.8
This shows that over a long term, P/E has moved between the band of 11 and 28. Very wide band indeed.

Jan - Dec 2008
Average: 18.7 Min: 10.7 Max: 28.3 Std Dev: 4

Jan - Dec 2009
Average:18.7 Min:13 Max:23.2 Std Dev:3.7

Jan 2010 - 26 Nov 2010
Average:22.8 Min:20 Max:26 Std Dev:1.5

This shows me that today's P/E of 23.3 is more than average p/e so market is expensive. This also tells me that overall, 2010 valuations are stretched comparing long term.

Let us drill deeper. I plotted p/e distribution across the time line.
Key Take-away 1: Over the period of last 3 years, the is progressively become more expensive.
Key take-away 2: In 2010, mostly PE of 23 is sustained.

Now, 23.3 is very expensive if i compare last 7-8 years. But, it is close to average for the year. So, what will i do? probably I will stay on cash, waiting for correction to kick-in. Or I can invest little and keep the remaining in cash, look for right P/E level to enter.

I would probably do the later.

Disclaimer:
* I am not a qualified financial advisor nor a money manager. I manage my own money.
* This is not an advert for Niftybees - I do own some Niftybees.
* Opinion expressed here are mine it could be out-right foolish OR extremely genius, depending on how much you know on this subject.
* Calculations shown here may be prone to errors (i call bugs), after all I am a software engineer. So, trust this at your own risk.
* Nifty is a benchmark index provided by Nationsal Stock Exchange (NSE), India. It comprises of top 50 stocks in terms of free float listed on NSE. These 50 stocks accounts for 50%+ in free float market capitalization.
* Used Openoffice's Calc tool to crunch the numbers.

MS Office is online now! Will it change Google Docs for better?

Disclaimer: I am a Linux programmer, user and evangelist. I use open office and MS Office for work related documentation. When I bought a PC for my home use, I was very reluctant to buy ms office for the personal use as it was too pricey. Office is more expensive than MS Win7 by almost 1.6x to 2x. So, I chose not to buy the MS Office.

The Google docs service is around for quite some time now. It is a useful tool for non-m$ users like me. I could open email attachments online convert one doc format to other. So, I do not need to spend money just to open my mail attachments.

But the usefulness stops there. I don't think anyone can do some serious 'office work' on Google docs. For example, how do you do simple function like 'save as'? You will have to do 'make a copy' of a doc. Then delete the prev copy. I feel it is just good enough for very basic document/spreadsheet editing.

Recently Google added realtime doc sharing which could probably compete with plethora of web meeting tools like teamviewer, cisco webex etc. This is probably one 'new' feature that got added in so long time.

Today, this news item drew my attention. If ms office is offered online, that will give good competition to g.docs. as a result it is reasonable to expect Google would spend more money on the project and get some useable software. So, welcome Microsoft...

How to Ratelimit Physical Port (L2)

If you have a LAN (PC or Network)and you want to restrict the amount of data rate the LAN receives/transmits, with tc (Traffic Control) command you can achieve this very easily. All you need is that you need to make sure entire traffic passes throug a Linux machine in Bridge configuration.

You might want to type in the following command at your terminal.

# tc qdisc add dev eth0 root tbf rate 0.5mbit burst 5kb latency 70ms minburst 1540

This implements a 512kbps 'pipe'. In other words, entire LAN segment shown in the diagram gets to share 512kbps.

This command also implements bucket size of 5KBytes.

Packets would wait for 70milli seconds before they get dropped.

Applications
1) This is useful in lab environments where a DUT is to be tested with restricted bandwidth. DUT could be a base station simulator or any other network element.
2) In case you want to simulate the congestion or choked bandwidth within LAN, this comes handy to simulate a smaller pipe.

Whats inside UTStarcom's BSNL Branded WA3002GA ADSL2+Router

I have a ADSL modem at my disposal at my office. I am going to test my learnings from my personal Broadcom Beetel router in my home.

This ADSL modem is supplied by BSNL. This device has got 4 LAN Ethernet ports and has a Wifi-AP built in. Wan side, it connects to a ADSL enabled POTS line.

I have logged into GUI. surprise surprise, I saw the exactly the same look and feel of my Airtel -supplied-beetel modem. Of course It shows some wifi menu, vpn config menu and more capable firewall menus. Apart from these same font, same color, same table everything same. I checked the bottom of device, it shows it is made by UTStarcom and manufactured in China. My home modem has No info though. It should be either Beetel modem is also manufactured by UTStarcom OR Broadcom chip vendor has supplied the same code base to these OEMs.

I ventured telent-ing into the box. I see 'some' difference here though. I see fixed menu showing. In my home modem, you have to press ? to get the list of commands supported. I typed 'sh' to get a unrestricted busybox shell -so that is same. Just like beetel modem this box's software bulid doesn't support 'ls' command so used 'echo *' to list the files. Software sports 2.6.8 kernel and gcc version 3.4.2, uClibc library. The software version of the box has 'bsnl' in it. It suggests me that code base for both BSNL and Beetel modems are same and separately built.

The Hardware is has got some more muscle

Wifi Chipset,
4 LAN ports
14MB RAM
3MB Flash

It makes sense as my office is using business class ADSL .

I have one more ADSL+everything kinda box with me. it is imported!! I am going to work on it and make it work with BSNL Line. Hope it works, Watch this space for updates.

Tearing apart ADSL Router - Beetel 110BX1 ADSL Router

Looks like the bcm96338 based modems are widely used by Indian ISPs not just Airtel.Little bit more google helped me break into the box. Thanks to http://www.cyberciti.biz/tips/hacking-beetel-220x-adsl-router-broadcom-bcm6338.html for the tips.

I found 2 major flaws in the ADSL-router software.

Hack#1: In web browser, key in http://192.168.1.1/main.html Bingo! It opens the web interface No Login/Password asked!! Must be a developer's page, overlooked during release!!

Hack#2: After logging in to telnet, type 'sh' it opens helplessly a Busybox shell coming out of the fixed menu, revealing entire system. so much for 'hardening'!!

Tip: there is no ls.use echo * instead.

In the last post I had mentioned that I am going to find our CPU and Linux Info. Once you got the entire shell these are the least of the info you can find.

Just for the record, The system has IPTables and ebtables, runs linux 2.6.8, gcc version 3.4.2. Looks like the vendor has not been spending $$ on software lately.


Syaonara.

Sneak Peek into my ADSL Router - Beetel 110BX1 ADSL Router

I have been using Airtel's DSL connection for some time now. I always wanted to see what the are all the capabilities of my "Modem" sitting on my desktop. The Modem reads Beetel 110Bx1 ADSL2 + Router. Today finally I got down to the device and took a look at the beast.
It is a simple device is running (hardened) Linux.

Hardware
- 6MB RAM
- 1.4MB Flash (fully utilized)
- Broadcom chipset, revealed during telnet login.
BCM96338 ADSL Router
Login:
- Single core CPU
- one RJ45 for LAN and One RJ11 for ADSL input line.

Software
- Telnetd , Login:admin, pwd:password
- Httpd (microhttpd, which is very light weight) Login:admin, pwd:password
All config are possible thru web interface.
- No SSH
- Shell is hardened - exposes very limited set of commands.
- Linux contains bridge utilities(brctl is available) The device is actually 'Bridge'.
- Runs a NAT
- Runs simpler form of ACL on Firewall
- Being Linux, it has QoS which is marking of packets with DSCP Values for selected source/destination IP/Port combination.
- It Also has DMZ and D-NAT.
- DHCP and DynDNS client.
- My modem says it runs "3.12L.01.A2pB023k.d20k_rc2" version.
- NTP Client
- as i suspected it runs Busybox v1.0
- All these are packed in 21 processes! (plus my telnet process)



Considering the software functionality and corresponding hardware muscle, It means there's no flab in system. The developers have definitely done a good job. This box says, it is imported by Bharti Teletech Ltd. Too bad it doesn't show who is the original manufacturer even worse country of origin too.

Kudos, whoever developed this 'Router' which works just fine, doesn't use too much resources and still provides all needed s/w features.

What I do not know now is the CPU speed, make Linux version. Let me see if i can break in and find out this.

Average Ethernet Packet size

As on Y2000, the packet size distribution looked something like this...

Downstream Upstream
---------- -------------
0 - 64 14.68 58.49
65 - 127 13.87 29.73
128 - 255 7.25 1.72
256 - 511 6.44 3.98
512 - 1023 13.59 3.37
1024 - 1518 44.17 2.70

Reported by:
Steve Dispensa
Sr. Network Architect
Sprint Broadband Wireless Group

Anatomy of initrd image

Lets say, you have a Linux Installer CD/Live CD, and you want to know what your system does immediately after booting kernel. initrd image is the place you might want to look at.
Initrd is actually a zipped , cpio archive of the hierarchy of directory structure. It contains init script aka mother of all processes. copy your initrd image into tmp folder. Then issue the following command.

$gunzip initrd.gz
$cpio -id < inird

This will create the entire directory structure. you will find 'init' as executable. If you are using a Linux-installer CD OR Live DVD chances are the initrd image uses busybox.

Happy Exploring.

QuickHowto: Building Real time Linux using RTAI

I have always been interested in Real-Time Kernels and O.S.es. I have started looking for Real time version of Linux almost immediately since I started working on Linux. There are multiple solutions out there to address the problem of making regular Linux-kernel into RTOS. While scanning through them I came acorss RTAI (Real Time Application Interface). This is Free, comes with good docmentation and RTAI addresses the problem neatly. Did I mention that RTAI is actively developed? Check out www.rtai.org

I have decided to build my own free usable realtime linux that could run on regular PC. This is what i did:

1) get the latest kernel image
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.tar.bz2

2) Download latest or one of the rtai package from www.rtai.org. My firefox firefox 3.6 didnt let me open this site.
So I tried in Firefox 2.0 and 2.0 complained about certificate but it let me open the page after i asked the browser to ignore.
I downloaded rtai-3.8.tar.gz

3) Now, untar the Linux kernel and rtai images. you will find directories linux-2.6.23/ and rtai-3.8/

4) Explore the direcotry rtai-3.8/ There are kernel patches for 2.6.20,2.6.22,2.6.23 under
rtai-3.8/base/arch/i386/patches

5)Patch the kernel.
$cd linux-2.6.23/ $patch -p1

Compile the Linux Kernel

6) Now, configure the kernel.
$ make menuconfig
Make sure the loadable kernel module is enabled.

7)compile the kernel with 'make' The linux kernel successfully compiles.

Compile RTAI - Kernel

8) inside rtai-3.8/ directory,
$make menuconfig
Under General -> Change the Linux directory and final installation directories to point to appropriate directories.
Under Machine (x86) -> make sure the number of CPUs in your machine is given correctly.
$ make
this will compile the RTAI referring your patched-kernel source.
$ make install

Testing
Now, you have RTAI as well as Linux kernel. You can either do "make install" the kernel. Or you can use test this realtime kernel with just initrd, which i will do next.
Create your own initrd image. I am going to just follow the steps explained my previous post. Basically, this should result in a system directory structre based on busybox. Create a linux system directory structure. additionally create realtime/ directory under usr/. Then Copy all realtime/ folder contents installed form previous step into it.
So your "root" directory structure should look something like this.

[root@sm initramfs-rtai3.8]# ls -lhrt
total 36K
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 sys
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 sbin
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 proc
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 newroot
-rw-r--r-- 1 root root 20 2010-03-08 17:31 initramfs.igz
-rwxr-xr-x 1 root root 1.2K 2010-03-08 17:31 init
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 etc
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 bin
drwxr-xr-x 3 root root 4.0K 2010-03-08 17:31 usr

Then from inside your directory strcuture issue folllowing command

[root@sm initramfs-rtai3.8]# find . | cpio -H newc -o > ../initramfs.-rtai3.8.cpio
This creats your own initrd image. So, you have initrd , kernel are ready. Recollect that we have copied all "realtime" directory into initrd. Once these two are ready you can test it with qemu.

# qemu -kernel /mnt/lfs1/linux-2.6.23/arch/i386/boot/bzImage -initrd initramfs.-rtai3.8.cpio -append "root=/dev/ram" /dev/cdrom

This opens a new PC-emulator window and loads kernel and initrd image, runs the kernel.


Reference:
* This is a very good and latest tutorial on howto compile kernel. http://www.cyberciti.biz/tips/compiling-linux-kernel-26.html
* A good Howto on howto create initramfs image is here. http://jootamam.net/howto-initramfs-image.htm It works for me.

In-Memory Linux

Aim is to create my own Linux distribution which runs completely runs in-memory. The Linux should be minimal at the same time functional.

In order to achieve this, you need 2 things: 1. Kernel of your choice 2. Initrd image

Kernel
1) Download the kernel of your choice. using
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.tar.bz2
2) unzip the bz2 image
''tar -jxvf linux-2.6.23.tar.bz2''
3) make sure the following is enabled in kernel configuration
General Setup --->

[* ] Initial RAM filesystem and RAM disk (initramfs/initrd) support

() Initramfs source file(s)
4) just make the kernel. Need not do make install. The bzImage is available in ''linux-2.6.23/arch/i386/boot/bzImage''


Prepare Initrd image

Initrd image is nothing but a small filesystem that contains mini linux system - shell and all supported commands etc. You can put your own programs also into this.
Most lightweight linux systems use busybox as their all-in-one utility program. busybox is a single binary that acts as common programs like ls,pwd,time,date etc. So, download busybox from www.busybox.net and compile it using make menuconfig & make commands. you will find busybox executable in the busybox folder itself. Make sure to configure the busybox to link statically all "applets". Applets are the individual programs.

1) in your work directory, create a directory called initramfs
2) create bin sbin etc proc sys newroot directories under initramfs directory.
''$ mkdir -p initramfs/{bin,sbin,etc,proc,sys,newroot}
$ touch initramfs/etc/mdev.conf ''
3) copy the busybox executable into ''initramfs/bin/''
4) create softlink called sh in bin.
'' $ ln -s busybox initramfs/bin/sh''
5) in initramfs directory create a ''init'' script file. This is the script that is run as pid=1 by Kernel.
~~~~~~~
''#!/bin/sh

#Mount things needed by this script
mount -t proc proc /proc
mount -t sysfs sysfs /sys

#Disable kernel messages from popping onto the screen
echo 0 > /proc/sys/kernel/printk

#Clear the screen
clear

#Create all the symlinks to /bin/busybox
busybox --install -s

#Create device nodes
mknod /dev/null c 1 3
mknod /dev/tty c 5 0
mdev -s

#Function for parsing command line options with "=" in them
# get_opt("init=/sbin/init") will return "/sbin/init"
get_opt() {
echo "$@" | cut -d "=" -f 2
}

#Defaults
init="/sbin/init"
root="/dev/hda1"

#Process command line options
for i in $(cat /proc/cmdline); do
case $i in
root=*)
root=$(get_opt $i)
;;
init=*)
init=$(get_opt $i)
;;
esac
done

#Mount the root device
mount "${root}" /newroot

#Check if $init exists and is executable
if [[[ -x "/newroot/${init}" ]] ; then
#Unmount all other mounts so that the ram used by
#the initramfs can be cleared after switch_root
umount /sys /proc

#Switch to the new root and execute init
exec switch_root /newroot "${init}"
fi

#This will only be run if the exec above failed
echo "Failed to switch_root, dropping to a shell"
exec sh''

~~~~~~~~~
6) create softlinks for mount,echo and clear in initramfs/sbin
''$ln -s initramfs/bin/busybox initramfs/sbin/mount
$ln -s initramfs/bin/busybox initramfs/sbin/echo
$ln -s initramfs/bin/busybox initramfs/sbin/clear''

7) inside initramfs, execute following command to create a cpio archive
''$ cd initramfs
$ find . | cpio -H newc -o > ../initramfs.cpio
$ cd ..
$ cat initramfs.cpio | gzip > initramfs.igz
''
Now, Initrd image is ready.

Testing

I. Using a emulator - QEMU
qemu is a PC emulator. You can download the qemu from internet. Once installed, You can test your images using following command.

''qemu -kernel /boot/bzImage -initrd /boot/initramfs.igz -append "root=/dev/ram" my_iso/rawimage''

II. Booting your PC with newly created images
Copy newly built kernel image and initramfs.igz image into /boot directory
Create new entry in grub/menu.lst
Restart your PC and choose the new kernel to boot with.
The PC should start and provide you a prompt. Now the PC runs completely on RAM. Go ahead explore the system.

In a typical Server/Desktop environment, Initrd contains specific programs that will further load necessary applications even do second stage of booting. In embedded
systems initrd IS the filesystem. Initrd provides simple "container" of your own programs that boot loader iteslf loads as part of bootprocess.

Happy Exploring.