blog/content/AWK cheatsheet.md

tags

tags
AI notes coding-notes docs public

1. Basic Syntax

The standard structure of an awk command:

Bash

awk [options] 'BEGIN { initialization } pattern { action } END { finalization }' file

commands

• BEGIN { ... }: Runs once before the first line is read. Great for setting variables or printing headers.

• pattern: A condition (regex or math) that must be met for the action to run.

• { action }: The command(s) executed on each line that matches the pattern.

• END { ... }: Runs once after the last line is processed. Great for printing totals or summaries.

---

2. Common Command-Line Flags

Flag	Description	Example
-F	Defines the field separator (default is space/tab)	awk -F"," '{print $1}' file.csv
-v	Assigns an external variable to use inside AWK	awk -v max=100 '$1 > max' file
-f	Reads AWK commands from a script file	awk -f script.awk input.txt
-i inplace	Edits the file in place (requires GNU awk)	gawk -i inplace '{print $1}' file

---

3. Built-in Variables

Variable	Meaning	Example Use Case
$0	The entire current line	print $0
$1, $n	The 1st, 2nd... _n_th field (column)	print $2, $4
NR	Number of Records (Current line number)	NR > 1 (Skip headers)
NF	Number of Fields (Total columns in current line)	print $NF (Print the last column)
FS	Input Field Separator (Same as -F)	BEGIN { FS=":" }
OFS	Output Field Separator	BEGIN { OFS="," }
RS	Input Record Separator (Default: newline)	BEGIN { RS="" } (Paragraph mode)
ORS	Output Record Separator (Default: newline)	BEGIN { ORS="\n\n" } (Double space)

---

4. Operators & Conditionals

Relational & Logical Operators:

• ==, !=, <, >, <=, >= (Standard math comparisons)

• ~ (Matches regular expression)

• !~ (Does not match regular expression)

• && (AND), || (OR), ! (NOT)

Control Structures (used inside action blocks):

# If/Else Statement
{ if ($1 > 50) print "Pass"; else print "Fail" }

# For Loop
{ for (i=1; i<=NF; i++) print $i }

# While Loop
{ i=1; while (i<=NF) { print $i; i++ } }

---

5. Essential Built-in Functions

Function	Description	Example
length(str)	Returns the length of a string	length($0) > 80
tolower(str)	Converts string to lowercase	print tolower($1)
toupper(str)	Converts string to uppercase	print toupper($1)
substr(s,p,l)	Extracts substring from s starting at p with length l	print substr($1, 1, 3)
gsub(r,s,t)	Globally replaces regex r with string s in target t	gsub(/apple/, "orange", $0)

---

6. Handy Everyday One-Liners

Printing & Filtering:

• Print the last column of every line:

  Bash

  awk '{ print $NF }' file.txt
  

• Print lines longer than 80 characters:

  Bash

  awk 'length($0) > 80' file.txt
  

• Print lines matching "Error" or "Warning":

  Bash

  awk '/Error|Warning/' file.log
  

Math & Summaries:

• Sum the values in the first column:

  Bash

  awk '{ sum += $1 } END { print sum }' numbers.txt
  

• Calculate the average of the first column:

  Bash

  awk '{ sum += $1 } END { print sum / NR }' numbers.txt
  

• Count the number of empty lines:

  Bash

  awk '/^$/ { count++ } END { print count }' file.txt
  

Text Manipulation:

• Remove duplicate consecutive lines (simulates uniq):

  Bash

  awk 'a != $0; { a = $0 }' file.txt
  

• Number each line (simulates cat -n):

  Bash

  awk '{ print NR, $0 }' file.txt
  

---