awk on A Scripter's Notes

How to remove duplicate lines using awk?

Thu, 20 Mar 2014 16:11:24 -0400

If you type echo "Hi\nHow\nHi\nAre\nHi\nYou?\nAre", you will get this in your terminal:

Hi
How
Hi
Are
Hi
You?
Are

Here’s how we can remove the duplicate lines using awk ..

echo "Hi\nHow\nHi\nAre\nHi\nYou?\nAre" | awk '\!x[$0]++'

The above will give this output:

Hi
How
Are
You?

The escape char \ is required for ! in tcsh.

This is how that awk snippet works:

Initially the x array will be empty.
When $0 is Hi, x[$0]=x[Hi]=0. So !x[Hi] will be True and it will be printed out.
After that the x[Hi] becomes 1 because of the ++ increment operator.
Next time when $0==Hi, as x[Hi]==1, !x[Hi] will be False and so $0 won’t be printed out.

One liner if-else in awk

Tue, 18 Mar 2014 16:00:53 -0400

Tiny snippets showing if/else use in awk ..

echo abc:def | awk -F: '{ if ( $2 ) {print $2} else {print} }'

Prints def.

echo abc | awk -F: '{ if ( $2 ) {print $2} else {print} }'

Prints abc.

Writing aliases with optional arguments in tcsh

Wed, 12 Mar 2014 11:46:49 -0400

Some times I would need to define an alias in tcsh which can have optional arguments. tcsh doesn’t seem to support that directly.

Here’s how I solve that problem.

If you have an alias alias test 'echo \!:1*' and if you run test abc def, you will get the output abc def.

!:1* prints out all the arguments starting from argument 1 till the last where even argument 1 is optional. If that argument doesn’t exist, the variable will be assigned a null value.

But tcsh will not complain about it – the * after !:1 is the beauty. On the other hand, if I have an alias alias test2 'echo \!:1', and if I run test – with zero arguments – tcsh will give an error.

So extending that, I have the below alias defined to grab an argument of any index.

alias opt_args 'set arg1 = `echo \!:1* | awk '"'"'{ print $1 }'"'"'`; \\
 echo -n "Arg num 1 = $arg1 "; \\
 set arg2 = `echo \!:2* | awk '"'"'{ print $1 }'"'"'`; \\
 echo -n "Arg num 2 = $arg2 "; \\
 set arg3 = `echo \!:3* | awk '"'"'{ print $1 }'"'"'`; \\
 echo -n "Arg num 3 = $arg3 "; \\
 echo ""; \\
 '

You can test this alias by running these commands:

opt_args abc
opt_args abc def
opt_args abc def ghi
opt_args abc def ghi jkl

Get current directory name without path

Tue, 04 Mar 2014 13:48:01 -0500

This post shows how to get the current directory name without the preceeding path, using awk or rev+cut or the boring basename.

pwd | awk -F/ '{print $NF}'

rev and cut

pwd | rev | cut -d/ -f 1 | rev

basename

basename `pwd`