miniHOWTO: Install FC14 in a Headless Server

I was trying to install a desktop centric Linux Distro like FC14 into a ATCA Blade based on intel. I and My co-worker tried to install as mentioned in my previous post.
FC14 did install but with minimal packages. It did not have basic utils like ftp. Need less to say, yum did not work. I let him work on it and crack it, which he did over a Sunday afternoon.

Following was his report:

--------------------

OS:- FC14x86_84

Mode of Installation:- Serial Console ( Using Minicom for installing ) and USB-DVD.

First boot the system using DVD drive and when the installation menu comes press Tab for editing the input commands. Then type,

linux console=tty0 console=ttyS0,115200n8 ( Enter)

After installation I assigned an IP and issued the command yum update I had to face to problem like yum was not working and I was getting the Error,

[root@localhost~]# yum update

Error Cannt retrieve repository Metadata ( respond.xml) for Repository:Fedora. Please verify its Path and try again.

Then I Google and found out some solution,

Problem was with the Python, it does not handle https proxy well. So I change all the https present in *.repo files in /etc/yum.repos.d/ to http

Here’s the way to do it… edit your repository files /etc/yum.repos.d/fedora.repo and /etc/yum.repos.d and /fedora-updates.repo by commenting all the lines starting with term mirrorlist and uncommenting all the lines starting with term baseurl.

After that my yum started working fine for me.

Referenece URL:-

http://rishabhsays.wordpress.com/2010/01/24/error-cannot-retrieve-repository-metadata-repomd-xml-for-repository-fedora/

--------------------
Thanks: S.Dhananjaya.

HOWTO: Install centos 5.4 OR RHEL 5.5 over serial console

I had to install a processor blade which could boot from USB but it didnot have VGA & Keyboard. Serial port was the only option. I tried to do like this.

It did not work. Later I fugured, I do not need to modify the ISO. I could do it so easily. The procedure is roughly like below.

1) Insert your bootable DVD and connect your DVD-Drive to USB.

2) Make the BIOS boot from USB Drive.

3) Prepare your putty/minicom suitably. 115200-8-n-1 worked for me.

4) You will see following menu.

5) Select F4 to type in command line arguments to kernel.

6) Type in

boot:linux console=tty0 console=ttyS0,115200n8

7) press ENTER and you will see booting proceeding with logs showing in your console.

Happy Exploring.

mini Howto: write your own iptables target module

Howto Extend the iptables

I searched the existing iptables target/match modules whether they would help me collect the packet length statistics of LAN. I set out to use iptables infrastructure because iptables offered a nice packet management framework. I did not find a readily available target so, I decided to write my own.
What i found was it is surprisingly easy to write your own target modules.

Design:
My target module should inspect the length of the packets it receives. Iptables rules can be setup to send packets from specific source/destination/port number/application/time of day etc to the newly written module. The module, builds a array of 32 integers, which is modifiable at compile-time. The array elements act as a packet counter. array[0] would count the packets whose length varies from 0-63 and so on. My sampling step size is 64, which is again programmable at compile-time.

I used proc entry in order to access these counters from user space. For time being I have avoided netlink socket(elegant solution) and kprintf(beginner solution) both would work.

The module has entry and exit functions. Entry function creates proc entry and registers a target function. Write the target function itself and fill up the xt_target structure. Use this structure to register with iptables as a new target. Exit function de-registers the target and removes the proc entry.

The target function gets the skb pointer to work on. Yes, now we can do anything with the packet. The target function gets called by iptables with the packet (skb) as an argument and target module's argument as typed in iptables command as second argument. To keep things simple, I have decided to have my module without second argument. so, it just receives the packet.I get the packet length and quantize the length into 32 steps, and increase the appropriate counter.

To export the constructed histogram, I have written the output function which is invoked whenever user attempts to read the /proc/pkt-hist file. This function just prints out the array.

PS: Extending the iptables involves writing actual target or match modules which does the job in kernel space AND writing supporting user space library to parse the arguments and let the iptables know what to do when a target is typed in the command line for e.g
iptables -I FORWARD -s 192.168.x.y/32 -d 0.0.0.0/0 -j PHIST


Steps:
1) Make sure you have latest kernel code downloaded and netfilter option enabled. I had linux-2.6.32. Also make sure the /usr/src/linux points to /usr/src/linux-2.6.32/ where i have the linux source is available.

2) Get the iptables source code and untar it. Do ./configure;make;make install - usual procedure. I had iptables-1.4.3.2.

3) Implement the target: easiest way to write your own target is to take a reference from existing target modules. go into /usr/src/linux/net/netfilter/. You will find all the match and target modules supported by that netfilter(kernel). All match modules are in named in small letters while targets are written in caps. for eg, xt_mark.c is a match module, while xt_MARK.c is a target module.
I have copied the xt_NOTRACK.c which is a very simple target module and fits my bill. I have removed all code inside the target function and written my own. Renamed all the functions, structures appropriately. In entry function i have created /proc/pkt-hist file. I have also added required call back code for /proc/pkt-hist.

4) Modify KConfig: Now, in /usr/src/linux/net/netfilter/ open Kconfig in vi editor. search for NOTRACK, you will find
" config NETFILTER_XT_TARGET_NOTRACK
tristate '"NOTRACK" target support'
depends on IP_NF_RAW || IP6_NF_RAW
depends ....
"
If you recollect, this is what appears in "make menuconfig" program. So, copy-paste this entire section into Kconfig as below:
config NETFILTER_XT_TARGET_PHIST
tristate '"PHIST" target support'
depends on NF_CONNTRACK
help
The PHIST target collects the packet distribution
in pre-set steps.

If you want to compile it as a module, say M here and read
. If unsure, say `N'.


5) Implement user space library: go into /usr/src/iptables-1.4.3.2/extensions
and you will find libxt_NOTRACK.c, clone the same into libxt_PHIST.c and modify the names of functions appropriately.

6) compile and fly
Compile Target:
In /usr/src/linux/net/netfilter/, edit Makefile. Search for NOTRACK. you will find the following
obj-$(CONFIG_NETFILTER_XT_TARGET_NOTRACK) += xt_NOTRACK.o
Now, clone this line for PHIST. see below:
obj-$(CONFIG_NETFILTER_XT_TARGET_PHIST) += xt_PHIST.o

Now, At /usr/src/linux/ do make modules;make module_install. you will find xt_PHIST.ko in /usr/src/linux/net/netfilter/
Compile iptables userspace: repeat step (2). you will find libxt_PHIST.so in /usr/src/iptables-1.4.3.2/extensions/

7) Now try typing
iptables -I FORWARD -s 192.168.x.y/32 -d 0.0.0.0/0 -j PHIST
Now, iptables finds the xt_PHIST.ko and loads it and send the packet satifsying the rule to the target code which is just written.

So, the target module worked like a charm may be due to it is not doing much. But it was a breeze to write a new target. I guess writing match module wont be entirely different from what is described here.

Anatomy of initrd image

Lets say, you have a Linux Installer CD/Live CD, and you want to know what your system does immediately after booting kernel. initrd image is the place you might want to look at.
Initrd is actually a zipped , cpio archive of the hierarchy of directory structure. It contains init script aka mother of all processes. copy your initrd image into tmp folder. Then issue the following command.

$gunzip initrd.gz
$cpio -id < inird

This will create the entire directory structure. you will find 'init' as executable. If you are using a Linux-installer CD OR Live DVD chances are the initrd image uses busybox.

Happy Exploring.

QuickHowto: Building Real time Linux using RTAI

I have always been interested in Real-Time Kernels and O.S.es. I have started looking for Real time version of Linux almost immediately since I started working on Linux. There are multiple solutions out there to address the problem of making regular Linux-kernel into RTOS. While scanning through them I came acorss RTAI (Real Time Application Interface). This is Free, comes with good docmentation and RTAI addresses the problem neatly. Did I mention that RTAI is actively developed? Check out www.rtai.org

I have decided to build my own free usable realtime linux that could run on regular PC. This is what i did:

1) get the latest kernel image
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.tar.bz2

2) Download latest or one of the rtai package from www.rtai.org. My firefox firefox 3.6 didnt let me open this site.
So I tried in Firefox 2.0 and 2.0 complained about certificate but it let me open the page after i asked the browser to ignore.
I downloaded rtai-3.8.tar.gz

3) Now, untar the Linux kernel and rtai images. you will find directories linux-2.6.23/ and rtai-3.8/

4) Explore the direcotry rtai-3.8/ There are kernel patches for 2.6.20,2.6.22,2.6.23 under
rtai-3.8/base/arch/i386/patches

5)Patch the kernel.
$cd linux-2.6.23/ $patch -p1

Compile the Linux Kernel

6) Now, configure the kernel.
$ make menuconfig
Make sure the loadable kernel module is enabled.

7)compile the kernel with 'make' The linux kernel successfully compiles.

Compile RTAI - Kernel

8) inside rtai-3.8/ directory,
$make menuconfig
Under General -> Change the Linux directory and final installation directories to point to appropriate directories.
Under Machine (x86) -> make sure the number of CPUs in your machine is given correctly.
$ make
this will compile the RTAI referring your patched-kernel source.
$ make install

Testing
Now, you have RTAI as well as Linux kernel. You can either do "make install" the kernel. Or you can use test this realtime kernel with just initrd, which i will do next.
Create your own initrd image. I am going to just follow the steps explained my previous post. Basically, this should result in a system directory structre based on busybox. Create a linux system directory structure. additionally create realtime/ directory under usr/. Then Copy all realtime/ folder contents installed form previous step into it.
So your "root" directory structure should look something like this.

[root@sm initramfs-rtai3.8]# ls -lhrt
total 36K
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 sys
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 sbin
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 proc
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 newroot
-rw-r--r-- 1 root root 20 2010-03-08 17:31 initramfs.igz
-rwxr-xr-x 1 root root 1.2K 2010-03-08 17:31 init
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 etc
drwxr-xr-x 2 root root 4.0K 2010-03-08 17:31 bin
drwxr-xr-x 3 root root 4.0K 2010-03-08 17:31 usr

Then from inside your directory strcuture issue folllowing command

[root@sm initramfs-rtai3.8]# find . | cpio -H newc -o > ../initramfs.-rtai3.8.cpio
This creats your own initrd image. So, you have initrd , kernel are ready. Recollect that we have copied all "realtime" directory into initrd. Once these two are ready you can test it with qemu.

# qemu -kernel /mnt/lfs1/linux-2.6.23/arch/i386/boot/bzImage -initrd initramfs.-rtai3.8.cpio -append "root=/dev/ram" /dev/cdrom

This opens a new PC-emulator window and loads kernel and initrd image, runs the kernel.


Reference:
* This is a very good and latest tutorial on howto compile kernel. http://www.cyberciti.biz/tips/compiling-linux-kernel-26.html
* A good Howto on howto create initramfs image is here. http://jootamam.net/howto-initramfs-image.htm It works for me.

In-Memory Linux

Aim is to create my own Linux distribution which runs completely runs in-memory. The Linux should be minimal at the same time functional.

In order to achieve this, you need 2 things: 1. Kernel of your choice 2. Initrd image

Kernel
1) Download the kernel of your choice. using
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.23.tar.bz2
2) unzip the bz2 image
''tar -jxvf linux-2.6.23.tar.bz2''
3) make sure the following is enabled in kernel configuration
General Setup --->

[* ] Initial RAM filesystem and RAM disk (initramfs/initrd) support

() Initramfs source file(s)
4) just make the kernel. Need not do make install. The bzImage is available in ''linux-2.6.23/arch/i386/boot/bzImage''


Prepare Initrd image

Initrd image is nothing but a small filesystem that contains mini linux system - shell and all supported commands etc. You can put your own programs also into this.
Most lightweight linux systems use busybox as their all-in-one utility program. busybox is a single binary that acts as common programs like ls,pwd,time,date etc. So, download busybox from www.busybox.net and compile it using make menuconfig & make commands. you will find busybox executable in the busybox folder itself. Make sure to configure the busybox to link statically all "applets". Applets are the individual programs.

1) in your work directory, create a directory called initramfs
2) create bin sbin etc proc sys newroot directories under initramfs directory.
''$ mkdir -p initramfs/{bin,sbin,etc,proc,sys,newroot}
$ touch initramfs/etc/mdev.conf ''
3) copy the busybox executable into ''initramfs/bin/''
4) create softlink called sh in bin.
'' $ ln -s busybox initramfs/bin/sh''
5) in initramfs directory create a ''init'' script file. This is the script that is run as pid=1 by Kernel.
~~~~~~~
''#!/bin/sh

#Mount things needed by this script
mount -t proc proc /proc
mount -t sysfs sysfs /sys

#Disable kernel messages from popping onto the screen
echo 0 > /proc/sys/kernel/printk

#Clear the screen
clear

#Create all the symlinks to /bin/busybox
busybox --install -s

#Create device nodes
mknod /dev/null c 1 3
mknod /dev/tty c 5 0
mdev -s

#Function for parsing command line options with "=" in them
# get_opt("init=/sbin/init") will return "/sbin/init"
get_opt() {
echo "$@" | cut -d "=" -f 2
}

#Defaults
init="/sbin/init"
root="/dev/hda1"

#Process command line options
for i in $(cat /proc/cmdline); do
case $i in
root=*)
root=$(get_opt $i)
;;
init=*)
init=$(get_opt $i)
;;
esac
done

#Mount the root device
mount "${root}" /newroot

#Check if $init exists and is executable
if [[[ -x "/newroot/${init}" ]] ; then
#Unmount all other mounts so that the ram used by
#the initramfs can be cleared after switch_root
umount /sys /proc

#Switch to the new root and execute init
exec switch_root /newroot "${init}"
fi

#This will only be run if the exec above failed
echo "Failed to switch_root, dropping to a shell"
exec sh''

~~~~~~~~~
6) create softlinks for mount,echo and clear in initramfs/sbin
''$ln -s initramfs/bin/busybox initramfs/sbin/mount
$ln -s initramfs/bin/busybox initramfs/sbin/echo
$ln -s initramfs/bin/busybox initramfs/sbin/clear''

7) inside initramfs, execute following command to create a cpio archive
''$ cd initramfs
$ find . | cpio -H newc -o > ../initramfs.cpio
$ cd ..
$ cat initramfs.cpio | gzip > initramfs.igz
''
Now, Initrd image is ready.

Testing

I. Using a emulator - QEMU
qemu is a PC emulator. You can download the qemu from internet. Once installed, You can test your images using following command.

''qemu -kernel /boot/bzImage -initrd /boot/initramfs.igz -append "root=/dev/ram" my_iso/rawimage''

II. Booting your PC with newly created images
Copy newly built kernel image and initramfs.igz image into /boot directory
Create new entry in grub/menu.lst
Restart your PC and choose the new kernel to boot with.
The PC should start and provide you a prompt. Now the PC runs completely on RAM. Go ahead explore the system.

In a typical Server/Desktop environment, Initrd contains specific programs that will further load necessary applications even do second stage of booting. In embedded
systems initrd IS the filesystem. Initrd provides simple "container" of your own programs that boot loader iteslf loads as part of bootprocess.

Happy Exploring.