Awk Reading the Script From a File

In the previous mail, we talked near sed command and we saw many examples of using it in text processing and we saw how it is good in this, only information technology has some limitations. Sometimes you need something powerful, giving you more control to process data. This is where awk command comes in. The awk command or GNU awk in specific provides a scripting language for text processing. With awk scripting linguistic communication, y'all can brand the following: a) Define variables, b) Employ string and arithmetics operators, c) Utilize control menstruation and loops, d) Generate formatted reports. Really, you tin process log files that comprise maybe millions of lines to output a readable study that you lot can do good from.

Awk Options

The awk command is used like this:

$ awk options programme file

Awk can take the post-obit options:

-F fs     To specify a file separator.

-f file     To specify a file that contains awk script.

-five var=value     To declare a variable.

We will meet how to process files and print results using awk.

Read AWK Scripts

To define an awk script, use braces surrounded past single quotation marks like this:

$ awk '{print "Welcome to awk command tutorial "}'

If you type anything, it returns the same welcome string nosotros provide.

To cease the programme, press The Ctrl+D. Looks catchy, don't panic, the all-time is yet to come up.

Using Variables

With awk, you can process text files. Awk assigns some variables for each data field found:

• $0 for the whole line.
• $i for the commencement field.
• $2 for the second field.
• $n for the nth field.

The whitespace graphic symbol like space or tab is the default separator between fields in awk.

Check this example and come across how awk processes it:

$ awk '{print $1}' myfile

The above example prints the starting time give-and-take of each line.

Sometimes the separator in some files is not space nor tab just something else. You can specify information technology using –F option:

$ awk -F: '{impress $ane}' /etc/passwd

This command prints the first field in the passwd file. Nosotros utilise the colon as a separator because the passwd file uses information technology.

Using Multiple Commands

To run multiple commands, separate them with a semicolon similar this:

$ echo "Hullo Tom" | awk '{$2="Adam"; print $0}'

The beginning command makes the $2 field equals Adam. The second command prints the unabridged line.

Reading The Script From a File

You can type your awk script in a file and specify that file using the -f option.

Our file contains this script:

{print $1 " home at " $6}

$ awk -F: -f testfile /etc/passwd

Here we print the username and his domicile path from /etc/passwd, and surely the separator is specified with capital -F which is the colon.

You tin can your awk script file like this:

{

text = " home at "

impress $1 $6

}

$ awk -F: -f testfile /etc/passwd

Awk Preprocessing

If you need to create a title or a header for your upshot or so. You lot tin can use the Brainstorm keyword to reach this. Information technology runs before processing the information:

$ awk 'BEGIN {print "Report Title"}'

Let'due south utilize it to something we tin can meet the upshot:

$ awk 'Begin {print "The File Contents:"}

{print $0}' myfile

Awk Postprocessing

To run a script after processing the information, utilise the END keyword:

$ awk 'BEGIN {impress "The File Contents:"}

{print $0}

END {print "File footer"}' myfile

This is useful, you tin use it to add together a footer for example.

Let'south combine them together in a script file:

Brainstorm {

print "Users and thier corresponding dwelling"

impress " UserName \t HomePath"

print "___________ \t __________"

FS=":"

}

{

print $1 " \t " $half dozen

}

END {

print "The end"

}

First, the top section is created using Begin keyword. And then we define the FS and print the footer at the end.

$ awk -f myscript /etc/passwd

Built-in Variables

We saw the data field variables $ane, $ii $3, etc are used to extract data fields, we too deal with the field separator FS.

But these are not the just variables, there are more built-in variables.

The following list shows some of the built-in variables:

FIELDWIDTHS     Specifies the field width.

RS Specifies the record separator.

FS Specifies the field separator.

OFS  Specifies the Output separator.

ORS  Specifies the Output separator.

By default, the OFS variable is the space, you can set the OFS variable to specify the separator you need:

$ awk 'Begin{FS=":"; OFS="-"} {print $i,$vi,$7}' /etc/passwd

Sometimes, the fields are distributed without a fixed separator. In these cases, FIELDWIDTHS variable solves the trouble.

Suppose we accept this content:

1235.96521

927-viii.3652

36257.8157

$ awk 'BEGIN{FIELDWIDTHS="3 4 3"}{impress $1,$2,$3}' testfile

Look at the output. The output fields are 3 per line and each field length is based on what we assigned past FIELDWIDTH exactly.

Suppose that your data are distributed on dissimilar lines like the following:

Person Name

123 Loftier Street

(222) 466-1234

Another person

487 Loftier Street

(523) 643-8754

In the above example, awk fails to process fields properly because the fields are separated past new lines and not spaces.

Y'all need to set the FS to the newline (\north) and the RS to a bare text, and then empty lines will exist considered separators.

$ awk 'Brainstorm{FS="\n"; RS=""} {print $one,$three}' addresses

Awesome! we can read the records and fields properly.

More than Variables

There are some other variables that help y'all to get more than information:

ARGC     Retrieves the number of passed parameters.

ARGV Retrieves the command line parameters.

ENVIRON Array of the trounce environment variables and corresponding values.

FILENAME The file name that is processed by awk.

NF     Fields count of the line being candy.

NR    Retrieves total count of candy records.

FNR The record which is candy.

IGNORECASE     To ignore the graphic symbol case.

You can review the previous post shell scripting to know more than near these variables.

Let's test them.

$ awk 'Brainstorm{print ARGC,ARGV[ane]}' myfile

The ENVIRON variable retrieves the shell surroundings variables like this:

$ awk '

BEGIN{

print ENVIRON["PATH"]

}'

You tin can use fustigate variables without ENVIRON variables like this:

$ echo | awk -v home=$Dwelling house '{impress "My home is " home}'

The NF variable specifies the concluding field in the tape without knowing its position:

$ awk 'Brainstorm{FS=":"; OFS=":"} {print $1,$NF}' /etc/passwd

The NF variable can be used as a information field variable if yous type it like this: $NF.

Let'southward take a expect at these two examples to know the divergence betwixt FNR and NR variables:

$ awk 'Begin{FS=","}{impress $1,"FNR="FNR}' myfile myfile

In this example, the awk command defines two input files. The same file, but processed twice. The output is the first field value and the FNR variable.

Now, check the NR variable and see the difference:

$ awk '

BEGIN {FS=","}

{print $1,"FNR="FNR,"NR="NR}

Finish{print "Total",NR,"processed lines"}' myfile myfile

The FNR variable becomes i when comes to the second file, only the NR variable keeps its value.

User Divers Variables

Variable names could exist anything, but information technology can't begin with a number.

You tin can assign a variable as in vanquish scripting like this:

$ awk '

Brainstorm{

examination="Welcome to LikeGeeks website"

print test

}'

Structured Commands

The awk scripting linguistic communication supports if conditional argument.

The testfile contains the following:

ten

15

6

33

45

$ awk '{if ($1 > thirty) print $1}' testfile

Just that elementary.

You should utilise braces if yous want to run multiple statements:

$ awk '{

if ($1 > 30)

{

x = $1 * three

impress 10

}

}' testfile

Or blazon them on the same line and separate the if statement with a semicolon like this:

While Loop

You can use the while loop to iterate over data with a condition.

cat myfile

124 127 130

112 142 135

175 158 245

118 231 147

$ awk '{

sum = 0

i = 1

while (i < 5)

{

sum += $i

i++

}

average = sum / 4

impress "Average:",boilerplate

}' testfile

The while loop runs and every time it adds i to the sum variable until the i variables becomes 4.

You can leave the loop using suspension command like this:

$ awk '{

tot = 0

i = i

while (i < five)

{

tot += $i

if (i == 3)

break

i++

}

boilerplate = tot / three

print "Boilerplate is:",average

}' testfile

The for Loop

The awk scripting language supports the for loops:

$ awk '{

total = 0

for (var = i; var < 5; var++)

{

full += $var

}

avg = total / 3

print "Average:",avg

}' testfile

Formatted Printing

The printf command in awk allows you lot to impress formatted output using format specifiers.

The format specifiers are written similar this:

%[modifier]control-letter

This list shows the format specifiers y'all can use with printf:

c              Prints numeric output equally a cord.

d             Prints an integer value.

due east             Prints scientific numbers.

f               Prints float values.

o             Prints an octal value.

s             Prints a text string.

Here we apply printf to format our output:

$ awk 'BEGIN{

x = 100 * 100

printf "The result is: %e\n", 10

}'

Here is an example of printing scientific numbers.

Nosotros are non going to try every format specifier. You know the concept.

Born Functions

Awk provides several built-in functions similar:

Mathematical Functions

If you dearest math, you can use these functions in your awk scripts:

sin(x) | cos(x) | sqrt(10) | exp(x) | log(10) | rand()

And they can be used normally:

$ awk 'Brainstorm{10=exp(5); print x}'

Cord Functions

There are many string functions, you can check the list, merely we will examine one of them as an instance and the rest is the same:

$ awk 'BEGIN{x = "likegeeks"; print toupper(ten)}'

The function toupper converts grapheme case to upper instance for the passed string.

User Defined Functions

Yous tin can define your function and utilize them like this:

$ awk '

role myfunc()

{

printf "The user %s has home path at %s\n", $one,$6

}

BEGIN{FS=":"}

{

myfunc()

}' /etc/passwd

Here nosotros define a role chosen myprint, then we use it in our script to print output using printf function.

I hope you lot like the mail.

Thank you.

likegeeks.com

gossettnerisfamands.blogspot.com

Source: https://www.getgnu.org/gnulinux/gnulinux-ipuclari/30-examples-for-awk-command-in-text-processing.html

Share this post