Passing Variables to Awk

Introduction to Passing Variables to AWK

AWK is one of the most famous commands to use and manipulate the string based files which can be parsed into fields using delimiters and the functionality can even extended with passing variables to awk.

It is so much convenient to work with AWK due to its easiness and understandable scripting to accomplish some certain tasks such as filtering fields.

The special parameter -v is used to pass bash variables or variables inside a script . The general syntax will look like the following. The most important thing is that for each variable that needs to be passed into awk script, they all need to be passed with repetitive -v use. awk -v var_name=value [-v var_name2=value2] 'AWK script' input_file

In a sample input.txt file, say the content looks like as the following. If delimited with space, first column will be the count, second field is the user agent and third field is the agent version.

$ cat input.txt
30 okhttp 5.0.0-alpha.2
50 Boto3 1.28.25
54 aws-sdk-java 1.11.380
145 aws-sdk-java 1.12.581
160 Boto3 1.24.84
184 akka-http 10.2.10
184 aws-cli 1.22.34
184 aws-cli 2.9.3
184 aws-sdk-dotnet-coreclr 3.7.107.0
184 aws-sdk-go 1.38.15

Passing Variables to AWK Basic Examples

Lets first do some practice with AWK using this file

Print the lines where the count is more than 100

$ awk '$1 > 100 {print}' input.txt
145 aws-sdk-java 1.12.581
160 Boto3 1.24.84
184 akka-http 10.2.10
184 aws-cli 1.22.34
184 aws-cli 2.9.3
184 aws-sdk-dotnet-coreclr 3.7.107.0
184 aws-sdk-go 1.38.15

Print only the user agent field where the count is more than 100

$ awk '$1 > 100 {print $2}' input.txt
aws-sdk-java
Boto3
akka-http
aws-cli
aws-cli
aws-sdk-dotnet-coreclr
aws-sdk-go

Print the agent version where the count is more than 100 and user agent is aws-sdk-java

$ awk ‘$1 > 100 && $2 == “aws-sdk-java” {print $3}’ input.txt
1.12.581

This time lets set the aws-sdk-java with passing variable to awk

$ a=”aws-sdk-java”
$ awk -v var1=$a ‘$1 > 100 && $2 == var1 {print $3}’ input.txt
1.12.581

Passing the variables to awk can be a part of many different examples and use cases. The above examples demonstrates a basic usage and syntax which hopefully helpful when anyone needs it.

Passing Variables to AWK with Script Examples

We can also give a different examples for scripting it. Lets here try to make one using a for loop to iterate over some user agents.

$ for x in $(cat input.txt | awk '{print $2}' | sort -u); do awk -v var1=$x '$1 > 100 && $2 == var1 {print $2" has "$1" count of the version "$3}' input.txt ; done
akka-http has 184 count of the version 10.2.10
aws-cli has 184 count of the version 1.22.34
aws-cli has 184 count of the version 2.9.3
aws-sdk-dotnet-coreclr has 184 count of the version 3.7.107.0
aws-sdk-go has 184 count of the version 1.38.15
aws-sdk-java has 145 count of the version 1.12.581
Boto3 has 160 count of the version 1.24.84

One another example could be a bash script as follows. This time I will use two variables and the variables will be provided with the script execution.

$ cat pass-variable.sh
#!/bin/bash
x=$1
y=$2
awk -v var1=$x -v var2=$y '$1 > var2 && $2 == var1 {print $2" has "$1" count of the version "$3}' input.txt

$ ./pass-variable.sh aws-cli 50
aws-cli has 184 count of the version 1.22.34
aws-cli has 184 count of the version 2.9.3

$ ./pass-variable.sh Boto3 50
Boto3 has 160 count of the version 1.24.84

Thanks for reading!