Tip

Search and replace on steroids: Do more with these sed commands

On the surface, the UNIX Stream Editor (sed) may appear to be just a mild-mannered text editor. But beneath that calm exterior resides brawny functionality, featuring advanced capabilities to help manage your data center’s resources with just a few keystrokes. This article will unlock some of the potent functions in sed.

The case for sed
So why sed? Ordinary editors like vi are capable of making modifications to text files as well. The difference is that sed is much more efficient at handling many files simultaneously; the ability to filter text in a pipeline is unknown with other editors. The default behavior of sed is to walk through its input files line-by-line, apply its commands to these lines and write the result to the standard output. To perform these commands, sed uses regular expressions. Let's have a look at some examples that are applied to the file users that you can see in Listing 1:

my-computer:~> cat users

lori:x:1006:100::/home/lori:/bin/bash

linda:x:1007:100::/home/linda:/bin/bash

lydia:x:1008:100::/home/lydia:/bin/bash

lisa:x:1009:100::/home/lisa:/bin/bash

leonora:x:1010:100:/home/leonora:/bin/bash

Requires Free Membership to View

Listing 1
To start, you can just display a few lines from the example file, as in the following command that shows the first two lines and exits next:

sed 2q users

Much more useful is the following command that prints all lines containing the string or:

sed -n /or/p users

In this example, consider -n a mandatory option that is followed by the or search string. After this string, the p command prints the result. In this example, you've been looking for the string or. Sed uses regular expressions, the powerful search patterns you can use on Linux and UNIX environments, to make your searches more flexible. Some examples in which regular expressions are used are below:

sed -n /^or/p users: shows all lines that don't contain the text or.

sed -n /./p users: shows all lines that contain at least one character.

sed -n /\./p users: shows all lines that contain a dot.

Just printing lines isn't what makes sed so powerful — you can also substitute characters. The syntax is summarized in the following command:

sed s/leo/lea/g users

This command replaces the string leo with the stringlea and writes the results to the standard output. Writing it to the standard output is very secure, but it doesn't apply any changes to the file. To make changes, add the -i option to the command:

sed -i s/leo/lea/g users

The changes are applied immediately, which is useful if you know what you are doing. If you’re still shaky with sed, send the results to standard output first so that you can check it before writing it.

You can unleash the full power of sed through its shell scripting functionality. For example, if you have four files, named file1, file2, file3 and file4 in the current directory and you need to replace the text one in each file with the text ONE. The following scripting line would perform this task:

for i in file[1-4]; do sed -i s/one/ONE/g $i; done

Imagine the power of sed in a data center: For example, you can change all configuration files that contain the ID of a storage device that has just been replaced or you can modify a template file to make sure that the name of a placeholder service is replaced by the real name of the service you are using. The possibilities with sed are unlimited.

About the author: Sander van Vugt is an independent trainer and consultant living in the Netherlands. Van Vugt is an expert in Linux high availability, virtualization and performance and has completed several projects that implement all three. Sander is also a regular speaker on many Linux conferences all over the world. He is also the writer of various Linux-related books, such as Beginning the Linux Command LineBeginning Ubuntu Server Administration and Pro Ubuntu Server Administration.

 

This was first published in January 2012

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

Disclaimer: Our Tips Exchange is a forum for you to share technical advice and expertise with your peers and to learn from other enterprise IT professionals. TechTarget provides the infrastructure to facilitate this sharing of information. However, we cannot guarantee the accuracy or validity of the material submitted. You agree that your use of the Ask The Expert services and your reliance on any questions, answers, information or other materials received through this Web site is at your own risk.