The awk is a powerful Linux command line tool, that can process the input data as columns.
In the following note i will show how to print columns by numbers – first, second, last, multiple columns etc.
I will show how to change the default field separator in awk.
At the end of this article i will also show how to print or exclude specific columns or even ranges of columns using awk. (more…)
The grep, egrep, sed and awk are the most common Linux command line tools for parsing files.
From the following article you’ll learn how to match multiple patterns with the OR, AND, NOT operators, using grep, egrep, sed and awk commands from the Linux command line.
I’ll show the examples of how to find the lines, that match any of multiple patterns, how to print the lines of a file, that match each of provided patterns and how to find and print the lines, that do not match a pattern (negative matching). (more…)
From the following article, you’ll learn how to print lines between two patterns in bash.
I’ll show how to to extract and print strings between two patterns using sed and awk commands.
I’ve created a file with the following text.
It’ll be used in the examples below, to print text between strings with patterns.
I Love Linux ***** BEGIN ***** BASH is awesome BASH is awesome ***** END ***** I Love Linux
Lets say we need to print only strings between two lines that contain patterns ‘BEGIN’ and ‘END’.
With the sed command, we can specify the starting pattern and the ending pattern, to print the lines between strings with these patterns. The syntax and the example are shown below.
Syntax:
sed -n '/StartPattern/,/EndPattern/p' FileName
| Option | Description |
|---|---|
| -n, –quiet, –silent | Suppress automatic printing of pattern space |
| p | Print the current pattern space |
Example:
sed -n '/BEGIN/,/END/p' info.txt ***** BEGIN ***** BASH is awesome BASH is awesome ***** END *****
Similar to the sed command, we can specify the starting pattern and the ending pattern with the awk command.
Syntax:
awk '/StartPattern/,/EndPattern/' FileName
Example:
awk '/BEGIN/,/END/' info.txt ***** BEGIN ***** BASH is awesome BASH is awesome ***** END *****
With the help of grep command it is quite easy just to look at the active setting of configuration files, omitting comments.
The following command will display the content of the file somefile.conf, omitting blank lines and the lines started with #:
$ grep -v '^\s*$\|^\s*\#' somefile.conf
Also, you can pipe the output to a new file:
$ grep -v '^\s*$\|^\s*\#' somefile.conf > newfile.conf
To make filtering easier, you can add an alias for ‘clean config’:
$ alias cf='grep -v '^\s*$\|^\s*\#'
And use it as follows:
$ cf /etc/ntp.conf driftfile /var/lib/ntp/drift restrict default kod nomodify notrap nopeer noquery restrict -6 default kod nomodify notrap nopeer noquery restrict 127.0.0.1 restrict -6::1 server 0.centos.pool.ntp.org server 1.centos.pool.ntp.org server 2.centos.pool.ntp.org includefile /etc/ntp/crypto/pw keys /etc/ntp/keys
Use one of the following commands to remove blank lines from a file.
1. Using the grep command:
$ grep -v "^$" file.txt
2. Using the sed command:
$ sed '/^$/d' file.txt
3. Using the awk command:
$ awk '/./' file.txt
4. Using the tr command:
$ tr -s '\n' < file.txt
You also can pipe the output of each command to a new file, like follows:
$ grep -v "^$" input.txt > output.txt