This Content Component encountered an error

Five Linux commands you have (probably) never heard of

Learn about five of the more rarely used Linux commands (logsave, diff3, pstree, nl and split) that are useful, with a description of each command's flags and options. These commands work well for file management, tracking, and output and process analytics.

Linux has a huge collection of command-line tools ranging from tools you use everyday like ls, cd and ps to more obscure tools that you might not have used or even heard of.

We're going to look at five of these more obscure tools that could make your life easier. We'll briefly look at each command and its flags and options but don't forget to check each command's man page for all the possible flags and options.

logsave
The first tool we're going to look at is logsave. The logsave command saves the output of a command to a log file:

$ logsave /var/log/partsize df –h

Here, the logsave command saves the output of the df command to the file /var/log/partsize file – it will also add a timestamp and output the results of the command to standard out.

Doesn't sound overly interesting, does it? Well the special magic of logsave is that if the file, in our case /var/log/partsize, does not exist then logsave will queue the data in memory and wait for the file to appear. This is perfect for tracking the output of commands run doing the boot process when the partition you are writing to is not yet mounted. I use logsave quite a bit during automatic provisioning and bootstrapping to save command output that might otherwise be lost.

diff3
The next tool is the diff3 command. You may have used the diff command to generate differences between files. diff3 performs the same function for three files instead of two, but lacks some of diff's more sophisticated options. So let's take a simple example. We have three files: cat, dog, and mouse. They each contain a string. We can compare them all with diff3 like so:

$ diff3 dog cat mouse
====
1:1c
  This is a dog
2:1c
  This is a cat
3:1c
  This is a mouse

The diff3 command displays the different string from each of the three files. You can also specify as one of the files. This will feed in standard input to compare against the other files.

You can also use the –A flag to incorporate changes in all the files and highlight any conflicts:

$ diff3 -A dog cat mouse
1a
||||||| cat
This is a cat
=======
This is a mouse
>>>>>>> mouse
.
0a
<<<<<<< dog
.

pstree
You've probably already used ps to list processes on your hosts. But you may not have heard of pstree -- a command that displays your processes in a tree. Each process either branches out from the init process, from a PID you can specify, or from a user you can specify. Each process and its sub-processes are displayed as you can see below:

The pstree command also has some command flags you can use, for example the –a flag will display the command line used for the process and –p will add the process ID of the process to the tree.

nl
The nl command is very little known but performs a nifty little function – it adds line numbers to output sent to the command. Let's take a simple example and run the nl command against a file called dog.

$ nl dog
1 This is a dog
2 This is not a dog

You can see nl has taken the contents of the dog file and outputted it prefixed with line numbers. You can then pipe this output to a new file.

$ nl dog > dog_numbered

We can then display the contents of the file with our new numbered lines:

$ cat dog_numbered
1 This is a dog
2 This is not a dog

split
The last command we're going to look at is split. The split command splits files into pieces usually based on size. I use it to split large log files into more manageable pieces, for example:

$ split --bytes=1m /tmp/largelogfile /tmp/smallerfile

The split command has options for dividing files by bytes and numbers of lines, here we've divided the largelogfile into one million byte chunks by specifying the --bytes=1m option (you can specify b for bytes, k for kilobytes, g for gigabytes, etc).

We could divide the file by numbers of lines like so:

$ split --lines=1000 /tmp/largelogfile /tmp/smallerfile

Here the file would be divided into 1,000 line chunks.

In our examples each chunk will be stored in /tmp and the filename will be prefixed with the name smallerfile. The first file would be smallerfileaa, and the second smallerfileab, etc. You can also use the –d flag to number the chunks numerically instead of using alphabetic increments. Lastly, instead of specifying a file you can use to indicate that split should take input from standard input and divide that into separate files.

ABOUT THE AUTHOR: James Turnbull works for the National Australia Bank as the manager of the CERT (Computer Emergency Response Team). He is an experienced infrastructure architect with a background in Linux/Unix, AS/400, Windows, and storage systems. He has been involved in security consulting, infrastructure security design, SLA and service definition and has an abiding interest in security metrics and measurement. James is also involved in the Free and Open Source Software community as a developer and contributor.
 

This was first published in August 2009

Dig deeper on Linux administration tools

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchDataCenter

SearchServerVirtualization

SearchCloudComputing

SearchEnterpriseDesktop

Close