Ask the Expert

Using AWK to convert a text file in Unix

My Unix box will receive a comma-delimited text file via FTP on a periodic basis. I will need to process (convert) this text file into another text file for further processing by an app.

For this text file conversion I will need to:

    Requires Free Membership to View

a) extract certain fields from source file
b) embed a text string at front of each rec
The above will require some IF-THEN-ELSE logic.

The new text file will have space(s) between fields.

What is the "best" common Unix utility to use for this task; e.g., shell scripting, Perl, Python, AWK, a 3-GL language (e.g., FORTRAN, C/C++), etc. (I have been wanting to get into Python some). Is Python generally standard with Unix installs or require a separate purchase or download?

Any other options you can think of with assumption that, for example, a relation database is not available?

Thank You!
I like AWK the best for this kind of thing. You can execute simple type AWK programs right at the Unix prompt. Here is an example of something that might work for you:

awk '/string/ {"yourtextstring"  print $2 }' < test_input_file > output_file

This would first search for all records in the file that have the /string/. Then it would would extract all lines with a character sequence matching that pattern, parse each line into fields separated by blanks, put "yourteststring" in the first field, and output the second field in the file. Play around with this a bit. The general Unix awk man page is a good start. Here's a nice little link worth looking at: http://www.engr.utk.edu/ecc/unix/man/awk.php

If you want to set-up flow control using if-then-else logic, here is your format at a high level.

if (condition) 
  { commands1 } 
 [ else 
  { commands2 } ] 

Though there is much you can do at the beginner's level, you will learn awk best by trying many different things with examples. If you want a good book to help you, not only with awk, but sed, another strong text manipulation tool, check out this link, from O'Reilly. They have really good Unix books, and this is one of many: http://www.oreilly.com/catalog/sed2/

Good luck!

This was first published in July 2003

There are Comments. Add yours.

 
TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: