We're going to look at five of these more obscure tools that could make your life easier. We'll briefly look at each command and its flags and options but don't forget to check each command's man page for all the possible flags and options.
The first tool we're going to look at is
logsave. The logsave command saves the output of a command to a log file:
$ logsave /var/log/partsize df –h
Here, the logsave command saves the output of the df command to the file /var/log/partsize file – it will also add a timestamp and output the results of the command to standard out.
Doesn't sound overly interesting, does it? Well the special magic of
logsave is that if the file, in our case /var/log/partsize, does not exist then
logsave will queue the data in memory and wait for the file to appear. This is perfect for tracking the output of commands run doing the boot process when the partition you are writing to is not yet mounted. I use
logsave quite a bit during automatic provisioning and bootstrapping to save command output that might otherwise be lost.
The next tool is the diff3 command. You may have used the diff command to generate differences between files.
$ diff3 dog cat mouse ==== 1:1c This is a dog 2:1c This is a cat 3:1c This is a mouse
The diff3 command displays the different string from each of the three files. You can also specify
– as one of the files. This will feed in standard input to compare against the other files.
You can also use the
–A flag to incorporate changes in all the files and highlight any conflicts:
$ diff3 -A dog cat mouse
This is a cat
This is a mouse
You've probably already used
ps to list processes on your hosts. But you may not have heard of
pstree -- a command that displays your processes in a tree. Each process either branches out from the init process, from a PID you can specify, or from a user you can specify. Each process and its sub-processes are displayed as you can see below:
pstree command also has some command flags you can use, for example the –a flag will display the command line used for the process and –p will add the process ID of the process to the tree.
The nl command is very little known but performs a nifty little function – it adds line numbers to output sent to the command. Let's take a simple example and run the nl command against a file called dog.
$ nl dog
1 This is a dog
2 This is not a dog
You can see
nl has taken the contents of the dog file and outputted it prefixed with line numbers. You can then pipe this output to a new file.
$ nl dog > dog_numbered
We can then display the contents of the file with our new numbered lines:
$ cat dog_numbered
1 This is a dog
2 This is not a dog
The last command we're going to look at is
split. The split command splits files into pieces usually based on size. I use it to split large log files into more manageable pieces, for example:
$ split --bytes=1m /tmp/largelogfile /tmp/smallerfile
The split command has options for dividing files by bytes and numbers of lines, here we've divided the largelogfile into one million byte chunks by specifying the
--bytes=1m option (you can specify b for bytes, k for kilobytes, g for gigabytes, etc).
We could divide the file by numbers of lines like so:
$ split --lines=1000 /tmp/largelogfile /tmp/smallerfile
Here the file would be divided into 1,000 line chunks.
In our examples each chunk will be stored in /tmp and the filename will be prefixed with the name smallerfile. The first file would be smallerfileaa, and the second smallerfileab, etc. You can also use the
–d flag to number the chunks numerically instead of using alphabetic increments. Lastly, instead of specifying a file you can use
– to indicate that
split should take input from standard input and divide that into separate files.
ABOUT THE AUTHOR: James Turnbull works for the National Australia Bank as the manager of the CERT (Computer Emergency Response Team). He is an experienced infrastructure architect with a background in Linux/Unix, AS/400, Windows, and storage systems. He has been involved in security consulting, infrastructure security design, SLA and service definition and has an abiding interest in security metrics and measurement. James is also involved in the Free and Open Source Software community as a developer and contributor.
This was first published in August 2009