Cron

The software utility Cron is a time-based job scheduler in Unix-like computer operating systems. People who set up and maintain software environments use cron to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals. It typically automates system maintenance or administration—though its general-purpose nature makes it useful for things like connecting to the Internet and downloading email at regular intervals.

Cron is driven by a crontab (cron table) file, a configuration file that specifies shell commands to run periodically on a given schedule. The crontab files are stored where the lists of jobs and other instructions to the cron daemon are kept. Users can have their own individual crontab files and often there is a system wide crontab file (usually in /etc or a subdirectory of /etc) that only system administrators can edit.

For Scheduling Repetitive Jobs: crontab

You can schedule routine system administration tasks to execute daily, weekly, or monthly by using the crontab command.
Daily crontab system administration tasks might include the following:

• Removing files more than a few days old from temporary directories
• Executing accounting summary commands
• Taking snapshots of the system by using the df and ps commands
• Performing daily security monitoring
• Running system backups

Weekly crontab system administration tasks might include the following:

• Rebuilding the catman database for use by the man -k command
• Running the fsck -n command to list any disk problems

Monthly crontab system administration tasks might include the following:

• Listing files not used during a specific month
• Producing monthly accounting reports

Controlling Access to the crontab Command

You can control access to the crontab command by using two files in the /etc/cron.d directory: cron.deny and cron.allow. These files permit only specified users to perform crontab command tasks such as creating, editing, displaying, or removing their own crontab files.
The cron.deny and cron.allow files consist of a list of user names, one user name per line.
These access control files work together as follows:

• If cron.allow exists, only the users who are listed in this file can create, edit, display, or remove crontab files.
• If cron.allow does not exist, all users can submit crontab files, except for users who are listed in cron.deny.
• If neither cron.allow nor cron.deny exists, superuser privileges are required to run the crontab command.

Superuser privileges are required to edit or create the cron.deny and cron.allow files.

Note:

• Check for the job errors in /var/spool/mail/username file.
• Escape percent (%) sign with backslash in command line while scheduling a job.

Print paragraph between semicolons containing a pattern

Below two awk snippets can do this for you:

• awk 'function dothis(){ if(v && s==2){ print p} if(s==2)s=v=p="" }{ dothis(); s += /;/ ; v += /XXXX/; s ? p = length(p) ? p ORS $0 : $0 : "" }END{dothis()}' file

   

• awk '($0 ~ /;/){set++;} set{v=v?v ORS $0:$0} (v ~ /XXXX/){value=1} {if(set>1 && value==1){print v;v="";value="";set=""}}'  filename 

Extract rpm/jar/tar/iso in Windows 7

Windows 7-Zip from All Programs Menu

How about extracting .rpm/iso/tar/jar files on windows 7 machine?

Once of the best utilities i have come across in recent years in 7-Zip. I was particularly interested in jar, tar, iso and rpm files extraction. Other than these four  extensions, 7-Zip supports a number of other compression and non-compression archive formats (both for packing and unpacking) including 7z, ZIP, GZip, bzip2, xz and WIM. The utility also supports unpacking APM, ARJ, CHM, cpio, DEB, FLV, JAR, LHA/LZH, LZMA, MSLZ, onepkg, RAR, RPM, smzip, SWF, XAR and Z archives and CramFS, DMG, FAT, HFS, ISO, MBR, NTFS, SquashFS, UDF and VHD disk images.

7-Zip can open some MSI files, allowing access to the meta-files within along with the main contents. Some Microsoft CAB (LZX compression) and NSIS (LZMA) installer formats can be opened. Similarly, some Microsoft executable programs (.EXEs) which are self-extracting archives or otherwise contain archived content (e.g., some setup files) may be opened as archives.

The utility can be downloaded from here:
http://www.7-zip.org/download.html

fuser and pfiles: lsof equivalent in Solaris

Lets talk about lsof. It displays information about files which are opened by running(active) processes. An open file may be a regular file like text file or any script, directory, NFS file, block special file, character special file, shared library, regular pipe, named pipe, symbolic link, socket, anything. In Unix, we consider everything is a file, so you can think of how incredibly smart utility lsof is. Generally it is supplied with Linux.

But the question is, what is its Solaris equivalent? 

After some hit and misses i have managed to run an equivalent command. fuser. The fuser syntax that worked for me is :

sudo fuser -vm / 2>&1 | awk '$3 ~ /f|F/' | less

Another one is pfiles. It is not necessarily supplied with the default package.

pfiles /proc/*

Run fsck on a filesystem to repair it..

fsck -y -o b=32 /filesystem

This will repair the corrupted file system to its best and restore the data in /filesystem/lost+found directory. Data in this directory will have "#inode name" nomenclature. From there the data can be copied back to the filesystem. Ofcourse the techie who is repairing the FS must have prior knowledge about the directory structure in filesystem otherwise he would not be able to decide where to copy which file from lost+found directory. 

Note: This is system administrator's work, so please do not run mount/umount/fsck sort of commands with minimal knowledge. The fsck command can have other parameters as well. So depending on the need of the work, one can decide the exact fsck command. Above fsck command is most generic one i came across and applied while **working on real time issues**. Also please keep in mind that the filesystem has to be unmounted first (umount /filesystem) then run fsck and then mount (mount /filesystem /mountpoint) the filesystem back if properly repaired. Then only you will be able to see the files for restore in /lost+found directory.

Mount ISO image in Linux

Getting a lot of emails regarding mounting ISOs in linux. Please use the following steps in order to mount the image to FS.

Procedure to mount ISO images in Linux.

1) Login as a root user:
$ su -

2) Create a mount point:
# mkdir -p /mnt/disk

3) Mount iso file called mountthisdisk.iso:
# mount -o loop mountthisdisk.iso /mnt/disk

4) Verify it:
# cd /mnt/disk
# ls -l

In case you have not noticed "loop" in the mount command, let me tell you that A loop device is a pseudo-device that makes a file accessible as a block device. Loop devices are often used for CD ISO images and floppy disc images. Mounting a file containing a filesystem via such a loop mount makes the files within that filesystem accessible. They appear in the mount point directory using above commands.

xargs with cp and xargs with mv command

Find with xargs and mv and cp, On public demand :)
=======================================
find . -type f  | xargs -I {} mv {} /target_directory_path/
find . -type f  | xargs -I {} cp -p {} /target_directory_path/

Count the number of occurances of a pattern in a file using awk

awk '{ for (i=1;i<=NF;i++) if ( $i == "word" ) count++ } END{print count}' filename

This will print the count of the occurance of  a pattern "word" in a file.

How to compare the values of a column in awk in a same file and consecutive lines..

If one would like to compare the values of 2nd column of consecutive lines of same file in such a way so that if the difference between first value and second value is more than 100 it should print complete line else ignore line.

Input File:

=======

ABC 2500

ABCD 123

XYZ 122

WXYZ 2565

Desired Output:

==========

ABC 2500 (i.e. difference between 2500 and 123 is greater than 100 here)

XYZ 122 (i.e. difference between 122 and 2565 is greater than 100 here)

Command:

=======

awk 'NR % 2 != 0 {a=$1; b=$2} NR % 2 == 0 {if (b - $2 > 100 || $2 - b > 100){print a,b}}' inputfile

Count the number of files in each directory recursively

find ./ -type d -exec sh -c "echo -n {} ' ' ; ls -l {} | wc -l" \;

Use awk in following cases

Joining two files parallely using awk:
========================
awk 'NR==FNR{a[NR]=$0; next} {print a[FNR], $0}' file1.txt file2.txt

Changing extension of all .txt to .sh:
========================
for i in *.txt;do mv "$i" "${i%.txt}".sh;done

Print 3 lines before and 5 lines after pattern match abcd in a file a.txt

In Linux:

awk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=3 a=5 s="abcd" a.txt

In SOLARIS:

nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=3 a=5 s="abcd" a.txt

ftp in UNIX

ftp -n 10.200.120.12 << EOF
quote user username
quote pass password
cd /path/to/directory/
prompt off
put dump.txt
exit
EOF

Adding 30 days to current date using PERL inside SHELL

#! /usr/bin/ksh

dt1=`perl -l -e '$date=(localtime(time + (30 * 24 * 60 *60))); print $date;'`

dt=`echo $dt1|awk '{if($3<10) print $1" "$2" 0"$3" "$4" "$5;else print $0}'`

echo $dt|awk '{print $5"-"$2"-"$3"23:59:59"}'|sed -e 's/Jan/01/g'\;'s/Feb/02/g'\;'s/Mar/03/g'\;'s/Apr/04/g'\;'s/May/05/g'\;'s/Jun/06/g'\;'s/Jul/07/g'\;'s/Aug/08/g'\;'s/Sep/09/g'\;'s/Oct/10/g'\;'s/Nov/11/g'\;'s/Dec/12/g'

Print 3 lines before pattern match using sed and awk ..

To reverse a file using awk and print next three lines after pattern match (Including pattern):

awk '{ a[i++] = $0 } END { for (j=i-1; j>=0;) print a[j--] }' abcd.txt |sed -n '/MANU/{p;n;p;n;p;n;p;}' >abcd_new.txt

(This can be used when you want to print 3 lines before pattern match, including pattern. Logic: reverse a file and print 3 lines after pattern match.)

How to reverse a file using awk:

awk '{ a[i++] = $0 } END { for (j=i-1; j>=0;) print a[j--] }' abcd.txt

How to reverse a file using sed:

sed -n '1!G;h;$p' abcd.txt

Print one line before pattern match using sed in UNIX.

sed -n -e '/manu/{x;p;x;p;}' -e h test.txt

Adding one month (30-31 days) in UNIX SOLARIS

#! /usr/bin/ksh

d=`date +%d`

m=`date +%m`

y=`date +%Y`

if [ $m%2 -eq 0 ]

then

d1=`expr $d + 30`

d=expr $d1 - 30`

else

d1=`expr $d + 31`

d=expr $d1 - 31`

fi

if [ $m -eq 12 ]

then

y=`expr $y + 1`

m=0

fi

m=`expr $m + 1`

if [ $m -lt 10 ]

then

m=0$m

fi

echo "$y-$m-$d"

exit 0

sort a delimited file on the basis of a column

sort -t":" -k3 filename.txt

(The above will sort a file filename.txt which is ":" delimited on the basis of third column.)

Print every 6th line of a file ..

The below awk command will print every 6th line of a file test.txt starting from line number one.

awk '{if(NR%6==1) print $0}' test.txt

(Note: Print every nth line using awk.)

Sed Options