Free web based Solaris 10 course from SUN

In the BOSUG mailing list, Venky has mentioned about this free web-based Solaris 10 course from Sun Microsystems themselves:
http://www.sun.com/training/catalog/courses/WS-245.xml

You may need to register for free in the Sun Website for this. But it is more than worth it. Don't miss this golden chance to get an overview of Solaris 10 (and hence OpenSolaris). I didn't yet finish listening it (yeah it is voice enabled presentation made in Macromedia Breeze) but couldn't wait to post about the links. Below are the contents of the course.

Module 1: DTrace
Module 2: Extreme Network Performance
Module 3: Solaris Containers
Module 4: Solaris Containers for Linux Applications
Module 5: Solaris ZFS
Module 6: Solaris 10 Predictive Self Healing
Module 7: Unparalleled Security

Don't miss it guys and spread the news. I am already late, but better late than never :) Enjoy

Creating PostgreSQL datase with custom encoding

To create a database with custom encoding, su as postgres user su - postgres and login into PostgreSQL terminal.

psql template1 postgres

and execute the following SQL command to create the db:

create database username_dbname with owner=username encoding='ENCODING_NAME';

If you get the "CREATE DATABASE" message and no errors, all is ok and you can logout from the terminal by inputing '\q'.

Google enters into Domain Business!!!

Google is into acquisition spree. Where are they heading to ? No more research driver projects eh ? They are trying to make money now by all means possible, by their revolutionary search alogrithm , GFS and many other hot technologies and yes many pet projects of their team. Enough of technology, we are gonna buy Bill Gates soon. Way to go Google. Here is the hot news!!!

Google said it has signed an agreement to resell Web site addresses held by domain name registrar services GoDaddy.com and eNom beginning Friday. Registration has been integrated in Google Apps for Your Domain.

Registration fees are $10 annually. The service will support .com, .org, .net, .biz, and .info domains. The Mountain View, Calif., company said the service also includes the ability to create an administrative account to manage the site and a configuration tool to ensure the Google Apps is available on the new site.

Google Apps for Your Domain includes Gmail, calendar, shared calendaring, Google Talk instant messaging, Google Page Creator, and the Start Page for creating a home page.

The Web site of Bellevue, Wash.-based eNom says domain names to resellers sell for between $6.95 and $8.95 each annually. Mobile domain names, .Mobi, also are available from the site. Scottsdale, Ariz., GoDaddy, sells domains to resellers between $7 and $9 per name, plus an annual fee and additional charges for hosting and security certificates, according to its site.

If it seems a little odd that Google would offer to resell domain names, that’s likely because its engineers are afforded the luxury to explore other types of applications.

Similar to other innovative companies, Google allows intellectual freedom to its engineers of up to 20 percent of their working hours, estimates Guzman & Co. senior equity analyst Philip Remek. “Google probably has about 100 pet projects that will never generate revenue,” he said. “Then you have applications like Google Earth that on first glance looks like a toy, but when linked to local and mobile based search could become a powerful tool.”

As the Internet evolves, some of Google’s pet projects could turn into revenue-generating applications, Mr. Remek said.

Shares of Google rose $1.11 to $483.23 in mid-morning trading. (Not much as it raised when they took over GooTube :-))

Original Article link here

I hate MicroSoft

Nothing new..But I hate MicroSoft. Normally I don't like writing flames, but I am not in a good mood now, after the day turned out to be worst after fighting with my ISP. So have to express my anger here to get my head cool. So here it goes,

I knew that RTR Ready to Run Software, withdrew the support for MS FrontPage extensions. I never used FrontPage, but customers does and I must admit that it is hard to solve their issues without access to FP program. A customer was asking to install FP on his bare server. He was specific about the FP5 version. Ended up at RTR site to read the faq,

#Where can I get the latest Unix/Linux FrontPage Server Extensions?
The FPSE kits are no longer available for download.
 
#I really need the Unix/Linux FPSE kit, can you please send me a copy?
No, unfortunately, end-of-life really means end-of-life.

#May I distribute the FPSE kit(s) that I have previously downloaded?
No.

Searched for a few more, but all I could find is rpm based fp pack inside the plesk's rpm packages. They had everyone removed the file from every google cache.

Though I admit that FP extensions was one of the packages with only security holes, I don't understand why Microsoft has to withdraw the support from Unix completely instead of announcing a EOL ? Looks like MicroSoft was frustrated with the popularity of Unix boxes over Windows. Or did they buy RTR ? I know lots of customers uses FP extensions on Linux, for it's ease of use and support for FP components. Plesk already withdraw the FP support in fresh installation of the panel on Linux. When is cPanel going to ?

I always have a feeling that Anti Piracy cell of Microsoft is doing a great job in spreading Linux and other FOSS. Microsoft, please do more piracy raids, especially at small and medium business. Let me get more clients wanting to convert Windows boxes to Linux and convert their MS Office and Excel docs to sxw and sxi formats.

Ask the IT Managers to pay you per computer connected to the Windows server, and they will consult with us and replace windows server to be replaced with Samba, released under GPL (v2 or v3 ?:-P). Praise the Anti Piracy department of M$ for spreading Linux and other FOSS.

Inside the Linux boot process

Wow..Why did I miss this article :-( No one can explain the linux boot process better. Tim Jones did an excellent job with his article published at ibm.com developerworks, in explaining the linux boot processes. If you are a linux learner at a novice level, this is a must to read. Quoting some linux jokes here, before I give you the link to that article,

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Windows: "Where do you want to go today?"
Linux: "Where do you want to go tomorrow?"
FreeBSD: "Are you guys coming, or what?"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Macs are for those who don't want to know why their computer works.
Linux is for those who want to know why their computer works.
DOS is for those who want to know why their computer doesn't work.
Windows is for those who don't want to know why their computer doesn't work.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


The process of booting a Linux® system consists of a number of stages. But whether you're booting a standard x86 desktop or a deeply embedded PowerPC® target, much of the flow is surprisingly similar. This article explores the Linux boot process from the initial bootstrap to the start of the first user-space application. Along the way, you'll learn about various other boot-related topics such as the boot loaders, kernel decompression, the initial RAM disk, and other elements of Linux boot.

Read more here

Dada Mail Mailing lists v 2.9.2 - Restoring lists and fixing DBM errors

I always recommend Dada Mail lists for any customers having issues with mailman, Most serious hosting companies, limit the emails send out by the customers per hour. In that cases, if your mailing list has more than 1000 customers, and the limit per hour is set to 300 mails per hour, the extra 700 mails will be bounced.

This is where DaDa Mails comes into picture. Off all the other excellent features, what I like is it's flexibility to configure number of emails to be sent at a certain interval (which can be in secs, mins and hours). Now a days, many other one way mailing list application have such feature, like phplist and so on. Does Mailman have that feature I may have missed ?

Okay anyways the issue arised when a DaDa Mail installation suddenly started showing internal server errors. Checked the log file /usr/local/apache/logs/error_log to see the errors displayed as below,

No DBM package was successfully found or installed at /usr/lib/perl5/5.8.0/AnyDBM_File.pm line 15.
Compilation failed in require at admin.cgi line 21.
BEGIN failed--compilation aborted at admin.cgi line 21. 

How did that happen ? A working installation suddenly reports that there is no DBM package. Only reason which I could not confirm though was that perl version was 5.8.0, which was not a desired version on a relatively new cPanel server. Now I had to install the DBM packages. AnyDBM_File.pm was reading @ISA = qw(NDBM_File DB_File GDBM_File SDBM_File ODBM_File), and none of the DBM packages was existing on the server.

CPAN didn't had a version in it's tree for perl version 5.8.0, so download and executed the versions for V 5.8.0

wget -rnp http://search.cpan.org/src/JHI/perl-5.8.0/ext/GDBM_File/
wget -rnp http://search.cpan.org/src/JHI/perl-5.8.0/ext/NDBM_File/
wget -rnp http://search.cpan.org/src/JHI/perl-5.8.0/ext/SDBM_File/
wget -rnp http://search.cpan.org/src/JHI/perl-5.8.0/ext/ODBM_File/

Did a make and make install on all of them and internal server error is now gone to be replaced with another error, while trying to login in administration panel of DaDa Mail using the pass which is present in plain text in DADA/Config.pm. Error was as below

[Wed Dec 13 06:09:57 2006] mail.cgi: Dada Mail 2.9.2 warning! List settings db empty!  List setting DB Possibly corrupted! at /DADA/MailingList/Settings/Db.pm line 115.
[Wed Dec 13 06:09:57 2006] mail.cgi: Dada Mail 2.9.2 warning! no listshortname saved in list settings db! List settings DB Possibly corrupted! at /DADA/MailingList/Settings/Db.pm line 118.
[Wed Dec 13 06:09:57 2006] mail.cgi: listshortname in db, 'announcements' does not match saved list shortname: '' at /DADA/MailingList/Settings/Db.pm line 121.
BAD List name "" No such file or directory at /DADA/Template/HTML.pm line 145
[Wed Dec 13 06:09:57 2006] [error] [client 192.168.1.1] Premature end of script headers: /home/example/public_html/cgi-bin/dada/mail.cgi

Only option left to me was to restore the list. Accessed the restoring list URL for the DaDa mail at http://example.com/cgi-bin/dada/mail.cgi?f=restore_lists (Replace example.com with your_domain.com). Asked for the admin password and on correct entry , welcomed by Restore screen.. Restore done and all is well :-)

Limiting apache connections per IP

There are many cases, where on a shared hosting environment, one of the sites may be getting slashdotted or dugg for various reasons. That is where mod_limitipconn comes to help. There are many such modules available. This situation arised on apache 1.3 server for me, and so the patch is meant for 1.3 server only.

Installation
~~~~~~~~~~~~~~~~~

wget http://dominia.org/djao/limit/mod_limitipconn-0.04.tar.gz
wget ftp://ftp.opennet.ru/pub/web/modules/limits/mod_limitipconn-0.04-vhost.patch
tar xvzf mod_limitipconn-0.04.tar.gz
cd mod_limitipconn-0.04/
patch -p1 < ../mod_limitipconn-0.04-vhost.patch

This was a cPanel server and apache root was at /usr/local/apache. So used the following commands

/usr/local/apache/bin/apxs -c mod_limitipconn.c
/usr/local/apache/bin/apxs -i -a -n limitipconn mod_limitipconn.so

The above two commands, should have made a backup copy of your existing httpd.conf and added two new lines,

LoadModule limitipconn_module libexec/mod_limitipconn.so
AddModule mod_limitipconn.c

The apxs command need not be issued, you can simple edit the Makefile and change the first line which says,

APXS = apxs
to
APXS = /usr/local/apache/bin/apxs

and then run the normal make and make install thing.

Configuration
~~~~~~~~~~~~~

For the Vhost you have to limit, say, you want to limit access to http://www.example.com/gallery/ (which may be having lots of controversial pics) use this inside it's virtual host section, like

<Location /gallery>
        MaxConnPerUid 25
        MaxConnPerIP 2
</Location>

What it essentially means 25 connections to gallery and each can access or view two images at a time. There are more directives for limitconnip module which you can find from the code. After the addition the Vhost section may look like below,

<Virtualhost 192.168.1.102>
ServerAlias example.com
ServerAdmin webmaster@example.com
DocumentRoot /home/example/public_html
BytesLog domlogs/example.com-bytes_log
ServerName www.example.com

<IfModule mod_limitipconn.c>
        <Location /gallery>
        MaxConnPerUid 25
        MaxConnPerIP 2
        </Location>
</IfModule>

User example
Group example
CustomLog /usr/local/apache/domlogs/example.com combined
ScriptAlias /cgi-bin/ /home/example/public_html/cgi-bin/
</VirtualHost>

Other options to check are mod_vhost_limit , mod_throttle , mod_bandwidth, mod_curb and mod_cband. There may be few more such bandwidth throttling modules available. If you find better ones, do suggest. I will be interested in learning them.

Sorting IPs in bash scripting

Oh yeah..This is something I had my head banging for a few minutes, till I figured out the -t option of sort. Okay..I had a list of IPs around 20000 today, for which I had to sort to find any patterns so that I can block the subnet in the firewall itself. I ended up with

sort -u -n -t. -k 1,1 -k 2,2 -k 3,3 -k 4,4 /etc/eximblacks

-k and -t were the key options you need to look in the menu. There are more than one way to do it :-).

That helped me. Hope it may help you at some point of time.

Few RPM Management Hacks

I always have to look at the man page or search for the cpio options (I usually dont use cpio formats) whenever I wanted the files of a rpm to be extracted without installing it. I thought of putting together a few commands which may help some of you,

Extracting files from an RPM archive

rpm2cpio RPM_file | cpio -idmv

The above command will create the rpm directory structure inside the present working directory (pwd) and the entire rpm will be extracted inside the subdirectory.

If you just want to extract the binaries in the folder which rpm may install in /bin/ , /sbin/ , /usr/local/bin/ , /usr/local/sbin then append a few more options / words to the command like below,

rpm2cpio RPM_file | cpio -idmv '*/bin/*' '*/sbin/*'

Rebuilding RPM database

Often I have experienced situations where a yum upgrade on RPM based OSes, break the rpm database. Sometimes a reboot usually fixes it, but in case if you want to fix it rightaway without rebooting it, try rebuilding the RPM db by following the below steps,


cd /var/lib
tar cvzf rpmdb.tar.gz ./rpm
rm -f /var/lib/rpm/__db.00?
rpm --rebuilddb

There are many complex situations where rpm --rebuilddb can't help, where you have to use other tools to rebuild DB, the harder way involving, /usr/lib/rpm/rpmdb_dump and /usr/lib/rpm/rpmdb_verify.

Verifying the RPM integrity

If you suspect your machine is hacked and your sshd or netstat or ps or fuser is been replaced by the hacker by their own versions, you can use rpm -V RPM_name to check the integrity of the files installed by the RPM.

rpm -Vf /bin/netstat

The above command will verify the integrity of all the files installed by the RPM package which installed the netstat also. If all are fine, you wont get any output, otherwise similar to below can appear.


#rpm -Vf /usr/sbin/sshd
S.5....T c /etc/ssh/sshd_config
S.5....T /usr/sbin/sshd


Notations are below (if 5 is there in the flag, it means md5 differs. More details, man rpm)

5 — MD5 checksum
S — file size
L — symbolic link
T — file modification time
D — device
U — user
G — group
M — mode (includes permissions and file type)
? — unreadable file

Remember this is a very basic test and can help you get the kiddies who is on the learning track, but not the professional hackers, who know their job.

Restoring original attributes of an rpm

Often users messes up the permissions of important files and folders, here for a test I messed up the ownership and permissions of files. Go through it for the steps on restoring the file attributes.


[root@hackcity ~]# rpm -qf /etc/ssh/sshd_config
openssh-server-3.9p1-8.RHEL4.12
[root@hackcity ~]# ls -l /etc/ssh/sshd_config
-rw------- 1 root root 3027 Aug 14 05:07 /etc/ssh/sshd_config
[root@hackcity ~]# chmod 777 /etc/ssh/sshd_config
[root@hackcity ~]# chown hacktech.hacktech /etc/ssh/sshd_config
[root@hackcity ~]# ls -l /etc/ssh/sshd_config
-rwxrwxrwx 1 hacktech hacktech 3027 Aug 14 05:07 /etc/ssh/sshd_config
[root@hackcity ~]# rpm --setperms openssh-server-3.9p1-8.RHEL4.12
[root@hackcity ~]# ls -l /etc/ssh/sshd_config
-rw------- 1 hacktech hacktech 3027 Aug 14 05:07 /etc/ssh/sshd_config
[root@hackcity ~]# rpm --setugids openssh-server-3.9p1-8.RHEL4.12
[root@hackcity ~]# ls -l /etc/ssh/sshd_config
-rw------- 1 root root 3027 Aug 14 05:07 /etc/ssh/sshd_config


Pretty useful RPM options

I will use sshd packages for listing those options,

#rpm -q --whatrequires openssh
openssh-clients-3.9p1-8.RHEL4.12
openssh-askpass-gnome-3.9p1-8.RHEL4.12
openssh-askpass-3.9p1-8.RHEL4.12
openssh-server-3.9p1-8.RHEL4.12

rpm -qf --stats /etc/ssh/sshd_config
openssh-server-3.9p1-8.RHEL4.12

   total:             1      0.000000 MB      0.135489 secs
   digest:            2      0.029817 MB      0.006168 secs
   signature:         1      0.000000 MB      0.092698 secs
   dbget:             6      0.094604 MB      0.005985 secs

Well that's it. If you like RPM read more at http://www.rpm.org/max-rpm/ . BTW Alien is a good program to convert between the rpm, dpkg, stampede slp, and slackware tgz file formats

Booting from your USB/Pen/Thumb Drive

I just read a post from Joe (I guess he is a manager at SUN - if so, a good one to support the developers :-)) mentioning about the visual media coverage of Anil Gulecha's and Moinak Ghosh's achievement of booting live Solaris from USB. Watch the video below

In this light, I thought of mentioning a few USB bootable live CDs in linux and mention a few links also, mainly for my own future reference. Linux is the only OS, I know atleast a bit of, even I am playing with OpenSolaris, not yet as an admin. Is there a hosting based GUI control panel available for OpenSolaris ? I could find only Zomos (though the site says only v9 is supported, but both arch SPARC and x86). If there is a good and "non-technical user" friendly one, I bet OpenSolaris is gonna compete with Linux in hosting industry in next few years.

PenDriveLinux This is one good discussion site and the Live Linux I would recommend is Slax So here goes a few URLs,

Boot Knoppix from USB
Knoppix Remastering toolkit FAQ can be read here
BootUSB from debian Wiki
Googling for a HOWTO boot from USB can fetch you this link

cPanel mail quota mismatch

cPanel decided that they want to use maildir format for it's mailboxes and started converting all mboxes to maildir. There are 100's of issues (I love these issues and work arounds :)) involved with the conversion, like failing to convert some large mboxes of more than 1GB+, quotas mismatch and so on and on.. But I must agree, that they did a pretty good job with the /scripts/convert2maildir script which uses /usr/local/cpanel/3rdparty/mb2md/mb2md . Actual project page is at http://batleth.sapienti-sat.org/projects/mb2md/.

When customers use the Show Disk Space used cPanel actually reads the quota from a filename inside the mail directory usually located at /home/username/mail/domainname.com/emailaccount/maildirsize and prints out in a -h format (human readable format). After the conversion, around 50% of the accounts will show the wrong quota usage. The following command fixes the issue for you and let cPanel recalculate the disk usage.

find /home -name "maildirsize" -exec rm -f {} \;

For those of who like xargs use,

find /home -name "maildirsize" | xargs rm -f

or even

find /home -name "maildirsize" -delete

Replace /home with /home/username/mail for dealing with one particular cPanel account.

The OpenSolaris Quiz - FOSS.IN/2006

These are OpenSolaris Quiz questions presented in FOSS.IN/2006 These answers are mine and disclaimers apply here. I am putting it here, since I found the questions good enough to make me interested toward OpenSolaris

Edit: Please consider the questions only and not the answers. Attended the BOSUG, and Ananth helped in confirming a few answers. Some went over my head though

The OpenSolaris Quiz - FOSS.IN/2006

1. Which Computer Scientist who co-founder the Apache HTTP server servers as a board member of the OpenSolaris CAB.

Roy Thomas Fielding



2. On a Quad core, dual processor machine, how many times can the DTrace probe 'profile:::tick-17hz' fire in a span of 3 seconds



17



3. What does this Dtrace one-liner do ?

proc:::signal-send { printf("%s - %s". args[2], args[1] - }



Trace all the signals sent to all the processes running on the system.



4. 256 Quadrillion Zetabytes is a significant number as regard ZFS. This is the amount of data required to fillup a ZFS filesystem. How many digits are there in that number ? (when expressed as bytes)



39

5. Expand COW in the context of ZFS.



Copy on Write.



6. FreeBSD = Ktrace, Linux = strace , Solaris = ?



truss (dtrace is way too advanced :-))



7. Which binary / libc function enables seamless execution of 64 and 32 bit binaries on a 64 bit OpenSolaris machine ?



isaexec.



8. How do you encrypt a file with AES algorithm using the OpenSolarius Cryptographic Frameword ?



encrypt -a aes -i file.txt



9. Which is the distro wholly created by the Bangalore OpenSolaris community ?



Belenix



10. How many privileges does OpenSolaris have by default ?



48



11. Which OpenSolaris project is underway to introduce virtualization in networking ?



CrossBow.



12. What is the latest Source Code Management System used to OpenSolaris development ?



Merucrial / Hg



13. Write down a minimal fool proof "C" function which takes a (void *) pointer as the argument and returns :

0 - if it is an invalid userland pointer

1 - If the pointer points to a valid userland address backed up by a page. No core dumps, No signal handling. (Elegant Solutions = ++ points)



shh...shh...Answer is something I may be able to figure out after studying C :-D



14. Write a DTrace script to print the absolute path of ALL files being opened by processes running in a system.

Note : Your script should not throw any kind of errors when being executed.



dtrace -n 'syscall::open*:entry { printf("%s %s",execname,copyinstr(arg0)); }'



15. Draw an approximate diagram that explains the code-flow when a system callis called from a kernel thread

Syscall ==> Kernel panic :-)

16. Give three expansions for the acronym BFU which would be valid in the OpenSolaris World.



a) Blinding Fast Upgrade

b) Big Fucking Make

c) Bonwick-Faulkner Upgrade



17. Assuming you could completely populate a zpool to its theoretical limit

Qa) Find the approximate energy required to do the same (in Electron Volt)



7.488 x 10⁴⁶



Qb) Find the Mass Equivalnet for the energy



136 billion kg



18. You want to find out how many minor faults occur from the time your kernel loads till the time you get your login prompt during boot. How would you go about doing it ?



dnk again. It must be a dtrace script. I will try to write one once I install OpenSolaris.



19. SMF automatically manages dependencies betweekn services.

a. What is the ideal data structure to store the dependency information ?



Sorted Tree / Graph



b. What is the best algorithm to find the order in which the serices should be started ?



Topology Sort



20. Using any or all of the new and old technologies in OpenSolaris, write about a really wild or cool project that you would like to implement.



I hardly know of any technology of OpenSolaris in detail :-(. I am looking for clustering and building a NAS using ZFS. I got some idea, but will try to do a bit after going home only.

I missed FOSS.in for the last years

Yes that's true. Being a delegate there for last 2 days of the India's premium FOSS event conducted from Nov 24 - Nov 26th, I am now realizing what I missed in the last few years. Chances of meeting FOSS's who's who. Oh wait!!..for anyone who dont know what is FOSS it stands for "Free and Open Source Software". We had talks by Rasmus Lerdorf, founder of php (till v2), Tim Pritlove (all in all of Chaos Computer Club), Luke Kanies (man behind the PuPPeT) and Harald Welte (gpl-violations.org and the diamond sponsor of FOSS.in). On the eve of 25th, there was also a interesting panel discussion by the India's who's who of FOSS where, Karunakar (The linux localite), Kishore Bhargava (The Technological Evangelist), Atul Chitnis (one and only toolz), Frederick Noronha (Foss media guy), Arun Sharma (The FreeBSD India) , Sirtaj Singh Kang (wah taj!!, the KDE India Inc.) and K.Dakshinamurthy. And the great job of moderation was done by Sudhakar Thaths Chandra (Thaths). Talk on past and future 10 years of linux in India was so interesting and in the question session, (which was stopped by IISc guys) the pioneers of FOSS movement in India gave tit for tat replies for all of us, who speaks a lot, and works a bit in spreading FOSS.

The most lively hall was of a MNC who are pioneers in technology, but lacks the marketing. The enthusiasm I could see in each and every person in SUN's hall in spreading the knowledge and the work. The real FOSS activists :-). I missed their BoF's :-(. I believe the entire BOSUG team was there. Most of them know what they are doing. Sanjeeva, Shivakumar, Sheshadri all were of great help in explaining the concepts. I was more interested in hearing about details on ZFS. And yeah I registered to BOSUG list. Anil Gulecha, who is a third year computer science student at JSS Academy, Bangalore and the person who put a live bootable Belenix into a thumbdrive, delivered excellent talk on booting Belenix (Ingeniously Indian!!) from USB and explaining the bootup calls like livecd, liveUSB and why they prefer USB to CD and so on. Unfortunately, I couldn't bootup the liveCD they provided. Need to spend some time on it, or have to get the laptop to next BOSUG Meet. BTW AFAIK Belenix is the first OpenSolaris distribution to be able to run from a USB drive and it supports profiling too :-) They say it gets you the desktop in 60-90 sec which is awesome.

I am a guy who have been using Linux for some time, about 6 years, and if I remember right, I first installed linux RH 7.1 after 6 months when I first touched a computer. Believe it didnt fascinate me much. I installed Linux on a 2Gb partition of 20Gb disk and the 64MB RAM, and linux GUI (XF86 ??) was damn slow and I couldn't do anything there which resulted in formatting the disk space. It was only after a few months later, I learned about multiple virtual terminals at Ctrl + Alt + F1 --> F7 and from that point, I switched to Linux. I loved the b/w screen and could get HCF modem work there in linux, fixed a few issues of other's HD's where HD was not at all detected in windows and at one point or the other, I completely switched to linux unknowingly.

I believe it's the time I play with OpenSolaris and explore the latest technologies. I will surely attend the next BOSUG. I will go back to Cochin and talk with Bejoy Sir more on that. He and Prajeev Sir are my inspirations. They are the known best in Solaris, both SCNA :-) I need to read a lot about dtrace and ZFS. I will explain about them, as I understand in this blog at a latter point. Long live FOSS.IN.

NB: Next blog is the questions of OpenSolaris Quiz conducted at FOSS.in 2006 and my answers to it, as I think.

Meanwhile visit foss.in and view the snaps at,

http://www.flickr.com/photos/tags/foss.in
http://www.flickr.com/photos/tags/fossin
http://www.flickr.com/photos/tags/fossin2006

Symlinks in Unix OSes

Around a year back, this was actually tip or quite a new information to me. I was really shocked to know that I didn’t know the fact. :-( Well the information is the symbolic link (the soft links) have the same size as the number of characters in it’s target

Creating the soft link,

ln -s /hdb9/SuSe9.3/SUSE-9.3-Prof-i386-CD1.iso Suse1.iso

ls -l Suse1.iso
lrwxrwxrwx 1 guest guest 40 May 1 01:21 Suse1.iso -> /hdb9/SuSe9.3/SUSE-9.3-Prof-i386-CD1.iso

wc - (to take input from stdin)
/hdb9/SuSe9.3/SUSE-9.3-Prof-i386-CD1.iso (Ctrl + D here to get the results)
0 1 40 -

Signing off..

Upgrading kernel in CentOS

A piece of cake for sysadmins. A old tutorial, I wrote, but pasting now, in case it helps someone in any way.

Make sure that you have enough space for your kernel in /boot before you install the new kernel. At least 15 MB!!

Why should I update the kernel ?
================================

Three reasons mainly,

1) For better driver support
2) For better performance, stability and reliability.
3) For security reasons.

The default kernels coming with OS version less than CentOS 4.3 and CentOS 3.7 are exploitable. For the details of CentOS 4 bugs, refer (CentOS = Free RHEL)

https://rhn.redhat.com/errata/RHSA-2006-0617.html
http://www.securityfocus.com/bid/18992/info
http://www.cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2006-3745

How do I know my current kernel version installed
=================================================

1) Login to server as user root and issue the command `uname -r`

OR

2) `rpm -q kernel` command issued as root, will return all the types and versions of kernels installed on your server.

Which is my boot loader - LILO or GRUB ?
=======================================

How do I know whether my bootloader is grub or lilo ?

Unless you specifically changed the bootloader, it is GRUB by default on most of newer RPM based OS, like CentOS 3.x , CentOS 4.x, RHEL. Though this question was unnecessary since then, for those who have played with installing both lilo and grub on your servers may have some doubt on the present bootloader.

Assuming you installed the bootloader to the MBR and your harddisk is /dev/hda,
you could run (as root):

dd if=/dev/hda bs=512 count=1 2>&1 | grep GRUB
if = Input File (Everything in a UNIX system is a file!!! even harddisk :-), ofcourse with a few exceptions)
bs = bytes
count = 1st block

If this returns:

Binary file (standard input) matches

then you're using GRUB. (Basically you're just searching the MBR to see if the word 'GRUB' shows up inside.) If it doesn't say anything, then you're not using GRUB and instead LILO. You may need to confirm that by 'grep LILO' instead.

You can identify the device name from fdisk -l command. Device name will be /dev/hd[a-h] for IDE disks, /dev/sd[a-p] for SCSI disks. My desktop, shows the fdisk -l output as below, which means my device name would be /dev/hdc and for knowing the bootloader, I would actually issue the command listed below.

Disk /dev/hdc: 60.0 GB, 60011642880 bytes
255 heads, 63 sectors/track, 7296 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes ...o/p continued

dd if=/dev/hdc bs=512 count=1 2>&1 | grep GRUB

Where do I get the latest RedHat kernels ?
===========================================

There are a few places where you can get the later versions.

http://people.redhat.com/~jbaron/rhel4/RPMS.kernel/

The above URL is updated even before, the repositories of CentOS are updated.

http://mirror.centos.org/
http://www.centos.org/modules/tinycontent/index.php?id=13

Before proceeding further, you have to detemine the OS version by login to server via ssh and issuing the command

cat /etc/redhat-release

If the output of command is CentOS release 3.x (Final) , then you have to descend to the directory centos-3 and then the correct 3.x directory. (x can be 5, 6 or 7 depending on the version available at the time of your order)

Again, if the output is CentOS release 4.x (Final), then you have to click the centos-4 directory and once inside, proceed to appropriate versions, depending on value of x in 4.x.

Entering the updates/i386/RPMS/ directory , you will see lots of rpms alphabetically ordered. Look for kernel RPMs and you will see lots of RPMs, as below

for CentOS release 4.x (Final) OSes, inside the http://mirror.centos.org/centos-4/4.3/updates/i386/RPMS/

kernel-2.6.9-34.0.1.EL.i586.rpm 24-May-2006 17:29 10M
kernel-2.6.9-34.0.1.EL.i686.rpm 24-May-2006 17:15 10M
kernel-2.6.9-34.0.2.EL.i586.rpm 07-Jul-2006 22:26 10M
kernel-2.6.9-34.0.2.EL.i686.rpm 07-Jul-2006 22:27 10M
kernel-2.6.9-42.0.2.EL.i586.rpm 23-Aug-2006 05:01 11M
kernel-2.6.9-42.0.2.EL.i686.rpm 23-Aug-2006 05:03 11M
kernel-2.6.9-42.EL.i586.rpm 12-Aug-2006 13:05 11M
kernel-2.6.9-42.EL.i686.rpm 12-Aug-2006 13:06 11M


and it repeats for kernel-devel, kernel-hugemem, kernel-smp and kernel-doc directories.

It is the highest numbered kernel which is of interest to us. You may also look
at the timestamps, the latest kernel in the list is that uploaded on 23-Aug-2006, kernel-2.6.9-42.0.2.EL.i686.rpm.

Now the question which is the architecture, you should choose. 99.99%, it is the i686 rpm. A output of the command 'uname -a' will show the architectures supported like,


Linux cookie.cuckoos.com 2.6.9-34.0.1.EL #1 Wed May 24 07:40:56 CDT 2006 i686 i686 i386 GNU/Linux


Time to Upgrade
==============

I now know which is the RPM to be downloaded (here, kernel-2.6.9-42.0.2.EL.i686.rpm). I have downloaded it using wget using the command

cd /usr/src/redhat
wget -c http://mirror.centos.org/centos-4/4.3/updates/i386/RPMS/kernel-2.6.9-42.0.2.EL.i686.rpm
rpm -ivh kernel-2.6.9-42.0.2.EL.i686.rpm

Use the -i argument with the rpm command to keep the old kernel. If the -U option is used to upgrade the kernel package, it will overwrite the currently installed kernel.

If the system is a multi-processor system, install the kernel-smp packages as well



cd /usr/src/redhat/
wget -c http://mirror.centos.org/centos-4/4.3/updates/i386/RPMS/kernel-smp-2.6.9-42.0.2.EL.i686.rpm
rpm -ivh kernel-smp-2.6.9-42.0.2.EL.i686.rpm


Configuring the BootLoader (GRUB)
================================

On installing the kernel rpm, (using -ivh option), it configures the GRUB boot loader to boot the newly installed kernel. However, it does not configure the boot loader to boot the new kernel by default. If you have used rpm -Uvh, option instead of ivh, it will configure the bootloader config, at /boot/grub/grub.conf (a symlink/shortcut from /etc/grub.conf also exists) to boot the new kernel as well.

Now open up the file /boot/grub/grub.conf in your favorite editor and confirm that it contains a title section with the same version as the kernel package just
installed, like that of below



# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda3
# initrd /initrd-version.img
#boot=/dev/sda
default=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.9-42.0.2.EL)
root (hd0,0)
kernel /vmlinuz-2.6.9-42.0.2.EL ro root=LABEL=/
initrd /initrd-2.6.9-42.0.2.EL.img
title CentOS (2.6.9-34.0.2.EL)
root (hd0,0)
kernel /vmlinuz-2.6.9-34.0.2.EL ro root=LABEL=/
initrd /initrd-2.6.9-34.0.2.EL.img



If you don't have a separate /boot partition, the paths to the kernel and initrd image are relative to the / partition, like kernel /boot/vmlinuz-2.6.9-42.0.2.EL ro root=LABEL=/

Notice that the default is not set to the new kernel. To configure GRUB to boot
the new kernel by default, change the value of the default variable to the title section number for the title section that contains the new kernel. The count starts with 0. For example, if the new kernel is the second title section, set default to 1. However in our case, it is the first title and the default must be changed to 0.

Cross your fingers and reboot. If it's all good, it will come back.

FailSafe mode
=============

Since the kernel upgradation is via RPM, you dont have to worry about the boot failure. Still mentioning the grub's failsafe feature. Though Grub don't have a perfect failsafe boot mechanism as lilo, you can use it's 'failsafe' feature for
booting the newly installed kernel for once and editing grub.conf after successful reboot. And in case the newly installed kernel doesn't come back, we can simply remote reboot the server and it will boot back to the default kernel as in grub.conf


[root@server ~]# grub

Screen will get refreshed and you will get grub shell as below



GNU GRUB version 0.95 (640K lower / 3072K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB lists possible command completions. Anywhere else TAB lists the possible completions of a device/filename.]

grub>

In the grub shell, enter these
grub> savedefault --default=0 --once
grub> quit


--default=0 is the first title, default=1 is the second title and so on as explained earlier.

Once the server is successfully rebooted, login and then do a 'uname -a' to make sure that the new kernel is loaded and if it did, edit the grub.conf and make the newly installed kernel as default.

Tables in blogger and some good VI (ViM) Commands

Someone someday said something about VI - If I remember right, it was : Either learn Vi or learn unix. You can't do both in a life time. VI itself is a unix , millions of command or shorcuts hidden inside vi. When I say VI, I meant VIM. I joined orkut community of VI and is copying many of the useful (to me) commands discussed there to my blog. Copyleft wrongs belong Orkut VI community.

BTW I must say it was VI which helped me to join all the pretty tables to a single line table, to avoid the huge space just above the table in a matter of secs. The command I used was :1,$join!

Normal/Command Mode
-------------------

cw	Delete a word and put in insert mode
dw	Delete a word
~	Change the case letters
guu	lowercase line
gUU	uppercase line
ga	display hex,ascii value of character under cursor
u	Undo
Ctrl + R	Redo
Alt + W	Move word by word
G or ]]	Go to last line
gg or [[	Go to first line
$	Go to the end of line
0	Go to the beginning of line
ggdG or [[dG or [[d]]	Delete the entire lines of a file
^g	Display current line number on the bottom (every detail:-))
cc	Cut the line, leaving a blank line there
dd	Delete the line (similar to cc, but wont leave a blank line)
yy	Yank/Copy a line
p	Paste the line just cc'ed or dd'ed or yy'ed
.	Yes, that's a dot. (Period). Repeats the last command executed in normal mode
/word	Search for the word, 'word' in the document
%	To match appropriate brace close for an open brace
J	Join 2 lines. To join more than one line (say 10) press : 10 shift+j
D	Delete from cursor to end of line
X	backspace
x	Delete character under cursor
ma	Mark the current line as "line a"
mb	Mark the current line as "line b"
'a	Return to the line marked "a"
d'a or y'a	delete or copy the line marked a
/\<\d\{4}\>	Search for exactly 4 digit numbers
/\<\a\{4}\>	Search for 4 letter words
/first\_s*second/i	Search for first followed second on a new line
/bugs\(\_.\)*bunny	bugs followed by bunny anywhere in file
/^\n\{3}	Find 3 empty lines

Ex Mode

-----------

By Ex mode I mean the executable mode, where the command starts with a colon [ : ]

:help	Vi's inbuilt help
:%s/OlD/new/g	Replace 'OlD' with 'new' word all over the document / file
:%s/OlD/new/gi	Same results as above, but case insensitive replacements
:s/old/new/g	Replace 'old' word with 'new' word in the line at which cursor is currently pointed at
:2,5s/old/new/g	Replace old with new from line number two to five
:%s/\r/\r/g	Turn DOS returns ^M into real returns. I usally do a search and replace of Ctrl+v Ctrl+m.
:%s/^\(.*\)\n\1$/\1/	Delete duplicate lines, which are together
:shell	Escape to shell to do anything then exit returning to vi
:q	Quit :-D
:x, :wq	Save and Quit
ZZ	Quit equivalent to :wq!
:ma a	ma is the command and a is the argument. Marks the point where cursor is present as a. (But it explicitly wont show it). Now if you want to copy somthin from some other point to the point 'a' , keep the cursor at the other point and type y'a (in escape mode) (y can be replaced by d and so on).
:r!cmd	r followed by shell command, reads the command o/p and paste in a line just below cursor.
:rew!	rew! is for rewind. Clear all the buffers and the files to initial state of editing.
:set nu	Display line numbers
:%! nl -ba	Enough display, really number the lines
:sp	it splits your screen and ^W - to move between windows
:X	prompts for an encryption key. After writing your key, if you save your document it will be encrypted and no one else (but you and vim) can read your documents. If you reopen the file, VIM will ask for the key. If you want to disable encryption, just type :set key=
:%!xxd	view in Hex format
%!xxd -r	Revert to Normal format from Hex format
:g/^\s*$/d	Delete all blank lines
:v/./.,/./-1join	compress empty lines (two or three lines to 1)
:s/\(.*\)\ \(.*\)/\2 \1/g	Shift the last word of the line to the first, first to second and so on. %s - will do it for entire file

That's all for now. I will add more as I learn more. I need to learn the basics of advanced VI, playing with the registers.

Dictionary attack spamming on cpanel servers

Never again..It was bad and tough to fight against the Dictionary attack spamming. Either the server load will rise or

Dictionary attack
-----------------

From wikipedia :

Spammers may also use a form of dictionary attack in order to harvest e-mail addresses. For example, a spammer may send messages to adam@example.com, betty@example.com, carl@example.com, etc. Any addresses to which messages are delivered, as opposed to being bounced back, the spammer can then add to his or her sending list.

My issue was that, the attacker was spamming the domain in such a way that in matter of seconds, the cPanel server, which had the setting of smtp_accept_max = 150 will show

Connection refused from [xx.xx.xx.xx] : too many connections

not allowing other legit customers to send and receive emails. The spammers were targetting only one domain and I didnt have to go for some automate application.

RBLs were not working as expected and had to end up using acl_smtp_connect (Exim 4.53), It was dropping after a connection is made.

acl_smtp_connect = check_host

Touched two files, /etc/eximwhites and /etc/eximblacks and just after begin acl, added this

check_host:
accept
hosts = /etc/eximwhites

deny
log_message = match eximblacks
hosts = /etc/eximblacks
accept

I could configure ACL to use the RBL in the connect, but 50% of many spammer IPs being used was not on any RBL lists. So executed two piped commands as below to start with,

For adding server Ips to white list

ifconfig | grep 'inet addr' | cut -d ":" -f 2 | cut -d " " -f 1 >> /etc/eximwhites

To add spammer's IP to black list. Replace example.com with the domain under attack.

grep example.com /var/log/exim_mainlog | grep "rejected RCPT" | cut -d "[" -f 2 | cut -d "]" -f 1 | sort | uniq >> /etc/eximblacks

Do appropriate greps. This one worked for me.

wohoo. eximblacks file had 800+ IPs all on a sudden. And those IPs were rejected at the time of connection itself, before the SMTP banner or greeting sending the "550 administrative prohibition".

Cleared the log and finally restarted exim and made sure that there are no errors reported in the /var/log/exim_mainlog. All was fun after that, problem solved and no more connection refused errors. This is not a perfect solution, but worked for me.

Update : This solution again worked for me today, but this time, I had to execute the script every 15s to get things under control. There were 20000 IPs and it took me an hour. I need a better solution. Any suggestions ?

ImageMagick and errors

ImageMagick always causes trouble with versions. Either we have to upgrade, downgrade or install some other packages like PerlMagick to get it work. This time it was the usage of Image::Magick perl module and was getting the errors.

Can't load '/usr/lib/perl5/site_perl/5.8.7/i686-linux/auto/Image/Magick/Magick.so' for module Image::Magick: libMagick.so.10: cannot open shared object file: No such file or directory at /usr/lib/perl5/5.8.7/i686-linux/DynaLoader.pm line 230.

I dont know of a proper solution to this. But the steps I did, solve the issue. ldconfig (ldconfig scans a running system and sets up the symbolic links that are used to load shared libraries properly. It also creates a cache (/etc/ld.so.cache) which speeds the loading of programs which use shared libraries.) was not considering /usr/local/lib where the ImageMagick was installed and once it was fixed everything was working.

edited /etc/ld.so.conf
Added /usr/local/lib as a new line and saved the file.
Ran ldconfig :-)

That did the trick.

MySQL - LOAD DATA INFILE and cPanel

I wanna convert this blog as a NoteKeeper, a Dairy , a Friend , a NewsPaper and everything else I feels at that very moment. As my father always say I start a job with great enthu, but will burn out soon. I bet him for this blog!!!

So at this time it is about the LOAD DATA INFILE thing for which I had to spend around 10 mins to figure what is wrong with the command. The server do allow LOAD DATA INFILE thing, but the client was getting Access Denied (well actually better thing compared to Not allowed on this server error). I confirmed that we do allow it, by taking mysql prompt and entering show variables and it showed

local_infile ON

Happy now..but why the issue of access denied. Ctrl + C and Ctrl + Ved a small code
<?php

if ($db = mysql_connect('localhost','db_user','password')) {
print "connected!";
} else {
print "could not connect" . mysql_error;
}

if (mysql_select_db('db_name', $db)) {
print "
database accessed!";
} else {
print "could not access database!" . mysql_error();
}

$query = "LOAD DATA INFILE '/home/username/csv_data_to_import.txt'
INTO TABLE table_name
FIELDS TERMINATED BY ','
ENCLOSED BY '\"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES";
if (mysql_query($query, $db)) {
echo "
wohoo! loaded!";
} else {
print "
failed!" . mysql_error();
}
?>

Added a LOCAL keyword and it started working. Now the question was why :-P I was puzzled till I carefully read the documentation at http://dev.mysql.com/doc/refman/4.1/en/load-data.html which read

Note that, in the non-LOCAL case, these rules mean that a file named as ./myfile.txt is read from the server's data directory, whereas the file named as myfile.txt is read from the database directory of the default database. For example, if db1 is the default database, the following LOAD DATA statement reads the file data.txt from the database directory for db1, even though the statement explicitly loads the file into a table in the db2 database:

which basically meant, if I have to use LOAD DATA INFILE (without LOCAL keyword, I have to copy the csv to /var/lib/mysql/db_name or to the temp database directory (?) and use the csv location appopriately. Anyways the story ended fine. The prince married the princess and happily lived thereafter.

First one for previewing the template

I thought I would make my first post a grand one just like the karimeen (Pearl Spot Fish) at Grand Hotel :-P But for previewing my blog template , they say, I would need to have atleast one post. So here it is. I will add blogger emoticons as well. Let me find and do some touchings before I start blogging..

I hope this blogging wont become my yet another flash in the pan attempt :-)

BTW I am using the template from BlogSpot Templates