December 20, 2010

How to make sendmail use hosts file as well as the DNS for name resolutions ?

Subject: Configure Sendmail for name resolutions using hosts file and DNS server.

To make the Sendmail look for the hosts file as well as the DNS for name resolutions:

1) create a service.switch file in /etc/mail folder.
   
2) vi the service.switch file and add the below entry

    hosts    file    dns

    This will query the hosts file before querying the DNS server.


3) execute the below command

    # kill -HUP `head -1 /var/run/sendmail.pid`

    This will bring the changes into effect

November 1, 2010

Exclude folder using TAR command during backup process

Subject: Tar command
 
In tar command one can exclude the so called not so required folder & sub-folders during backup by:

[root@lino ~] # tar -cvf backup.tar projects/ --exclude="<folder path to be excluded>"

For excluding multiple folders:

[root@lino ~] # tar -cvf backup.tar projects/ --exclude="<folder-1 path to be excluded>" --exclude="<folder-2 path to be excluded>"

-c    creates a new archive  file
-v    verbosely lists files processed
-f    use archive file or device F (default "-", meaning stdin/stdout)
--exclude    will exclude the folders and files

September 22, 2010

Optimizing SQUID - (Part 2)

Subject: Squid's cache_swap_low & cache_swap_high disk space watermarks

In continuation to my testing of squid for optimization [refer Optimizing SQUID - (Part 1) ] , I then used Squid's configuration directive cache_swap_low and cache_swap_high. Thus performance of squid does improves but probably it won't be  noticeable. But, it definately reduces the chances  of squid's slow down or performance degradation.

The cache_swap_low and cache_swap_high directives controls the replacement of cache objects stored on disk. The values are in percentage (%) of the cache size (i.e.sum of all the cache_dir sizes).

I have set the following configurationg for the directives in squid.conf

cache_swap_low  60

cache_swap_high 65

So, until and unless disk usage is below the 60 %, replacement does not take place and, it begins only when it is above the low water mark. Replacement is more aggressive when disk utilization is close to the high water mark.

This could help in replacing the stale cache records as well.

August 14, 2010

Optimizing SQUID - (Part 1)

Subject: Use Ramdisk for Caching

(I have been working on an issue related to squid slowing down gradually thus slowing down internet access by network users. As it made me more enthusiastic what could be the reason or could be there an alternative way to resolve this. Thus, I can across the so called RAMDISK and tried to use it with squid.

I tried my way to use RAMDISK with SQUID. I found squid was doing good....so here I blogged......to get the better view of my result.


INTRODUCTION:

RAM disk is a section of the RAM (memory) which is configured as a disk drive. Data stored in RAM disk can be accessed faster than the data stored in a hard disk drive.

Usually, Squid is configured to cache its information of visited websites in the hard disk. Using a RAM Disk for storing this information will enable the squid to access the cache more faster than the one stored in cache. Though, data stored in the RAM DISK are TEMPORARY (i.e. data will be stored only until the system is up and running), I recommend to have multiple cache directories storage: one in RAMDISK and another as well as in Hard-disk.



I. CREATING RAMDISK for SQUID cache storage:

1) Check if ramdisk has been created?

root@lino90:~# ls -l /dev/ram*

By default only ram0 - ram15 are usable

root@lino90:~# ls -l /dev/ram*

lrwxrwxrwx 1 root root 4 Jul 20 12:37 /dev/ram -> ram1
brw-r----- 1 root disk 1, 0 Jul 20 12:38 /dev/ram0
brw-r----- 1 root disk 1, 1 Jul 20 12:36 /dev/ram1
brw-r----- 1 root disk 1, 10 Jul 20 12:36 /dev/ram10
brw-r----- 1 root disk 1, 11 Jul 20 12:36 /dev/ram11
brw-r----- 1 root disk 1, 12 Jul 20 12:36 /dev/ram12
brw-r----- 1 root disk 1, 13 Jul 20 12:36 /dev/ram13
brw-r----- 1 root disk 1, 14 Jul 20 12:36 /dev/ram14
brw-r----- 1 root disk 1, 15 Jul 20 12:36 /dev/ram15
brw-r----- 1 root disk 1, 2 Jul 20 12:36 /dev/ram2
brw-r----- 1 root disk 1, 3 Jul 20 12:36 /dev/ram3
brw-r----- 1 root disk 1, 4 Jul 20 12:36 /dev/ram4
brw-r----- 1 root disk 1, 5 Jul 20 12:36 /dev/ram5
brw-r----- 1 root disk 1, 6 Jul 20 12:36 /dev/ram6
brw-r----- 1 root disk 1, 7 Jul 20 12:36 /dev/ram7
brw-r----- 1 root disk 1, 8 Jul 20 12:36 /dev/ram8
brw-r----- 1 root disk 1, 9 Jul 20 12:36 /dev/ram9
lrwxrwxrwx 1 root root 4 Jul 20 12:37 /dev/ramdisk -> ram0

2) To Check the size of ramdisk:

root@lino90:~# dmesg | grep RAMDISK
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
root@lino90:~#

This shows that the default ramdisk size is 4MB.

3) To increase the size of the ramdisk for eg. 16MB
A command has to be passed to the kernel during booting. This can be done in the grub.conf  by  passing  an entry in ramdisk_size=[enter the size in 1024-byte blocks]

root@lino90:~# vi /etc/grub.conf
     
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You do not have a /boot partition. This means that
# all kernel and initrd paths are relative to /, eg.
# root (hd0,0)
# kernel /boot/vmlinuz-version ro root=/dev/sda1
# initrd /boot/initrd-version.img
# boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux Server (2.6.18-8.el5)
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-8.el5 ro root=LABEL=/1 rhgb quiet ramdisk_size=16000
initrd /boot/initrd-2.6.18-8.el5.img

Save the file and reboot. After the reboot, confirm with the below command to check the new size of the ramdisk

root@lino90:~# dmesg | grep RAMDISK
RAMDISK driver initialized: 16 RAM disks of 16000K size 1024 blocksize
root@lino90:~#

4) Format the ramdisk as a journaling file system eg.ext2 file system. In case if only one ramdisk, we will use /dev/ram0

root@lino90:~# mke2fs -m 0 /dev/ram0

mke2fs 1.41.3 (12-Oct-2008)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
16384 inodes, 65536 blocks
0 blocks (0.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=67108864
8 block groups
8192 blocks per group, 8192 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
8193, 24577, 40961, 57345

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 29 mounts or
80 days, whichever comes first. Use tune2fs -c or -i to override.

5) mounting ramdisk

a) Create a directory (In case of squid you can create the directory where the squid cache is stored)

root@lino90:~# mkdir /var/spool/squid_ram

b) mount the ramdisk to the above directory

root@lino90:~ # mount /dev/ram0 /var/spool/squid_ram

c) To verify the ramdisk mount

root@lino90:~# mount | grep ram0
OR
root@lino90:~# df -h | grep ram0

d) Change the ownership as per squid requirement

root@lino90:~# chown squid:squid -R /var/spool/squid_mem

e) Change the permission as per squid requirement

root@lino90:~# chmod 755 -R /var/spool/squid_mem

f) To check the details of the new ramdisk

root@lino90:~# tune2fs -l /dev/ram0

g) To create and Automount ramdisk at boot time add the below entry in rc.local

root@lino90:~# vi /etc/rc.local

/sbin/mke2fs -q -m 0 /dev/ram0
/bin/mount /dev/ram0 /var/spool/squid_mem
/bin/chown squid:squid /var/spool/squid_mem
/bin/chmod 0755 /var/spool/squid_mem
/usr/sbin/squid -z

Save the rc.local and exit.

II. CONFIGURING SQUID TO USE RAMDISK

1) Adding Ramdisk for cache store in addition to the existing cache store.

root@lino90:~# vi /etc/squid/squid.conf

2) Search for cache_dir entry in your squid.conf. Your orginal entry could be like below:

#
#Default:
cache_dir aufs /var/spool/squid 1000 20 256

Add the RAMDISK mount directory in the squid.conf

#
#Default:
cache_dir aufs /var/spool/squid_mem 45 100 100
cache_dir aufs /var/spool/squid 1000 20 256

The setting of "cache_dir aufs /var/spool/squid_mem 45 100 100" is ideally for a 50 MB of ramdisk. In this case we have configured 45MB out of 50MB randisk for cache store, leaving 5MB unused. The  number of irst level sub-directories created will be 100 and number of second level sub-directories created will be 100

Here, you need to check if squid can create the defined number of 1st level and 2nd level of cache directoies in the RAMDISK by running the below command

root@lino90:~# squid -z

In case if it is unable to create it will display an error

root@lino90:~# squid -z
Creating Swap Directories
FATAL: Failed to make swap directory /var/spool/squid_mem/77/4B: (28) No space left on device
Squid Cache (Version 2.6.STABLE6): Terminated abnormally.
CPU Usage: 0.157 seconds = 0.000 user + 0.157 sys
Maximum Resident Size: 0 KB
Page faults with physical i/o: 0

Then, in this case you will have to modify the number of cache directories (a balance number of cache directories)

Run the command squid -z, until you only get the below message

root@lino90:~# squid -z
2010/07/21 14:01:17
Creating Swap Directories
root@lino90:~#

Once Done start the squid service.

root@lino90:~ /etc/init.d/squid start

Note: Disable the squid service startup during bootup, as this will fail because at that time ramdisk will not be mounted.

root@lino90:~ chkconfig squid off


End Note: I scheduled a regular backup of the /var/spool/squid_mem, so the same can be restored in case if the system restarts. This, I have configured to load on booting.