1. Introduction
Bash does not have built-in support for handling JSON data directly. However, various tools can assist in parsing and manipulating JSON files within Bash scripts. In this guide, we will use jq, a powerful command-line utility specifically designed for working with JSON data in Bash.
jq is a lightweight and flexible command-line tool that allows us to extract, filter, and transform JSON data easily. It provides a simple yet effective way to work with JSON files in Bash scripts. With its expressive syntax, jq simplifies JSON manipulation tasks. Its ability to handle complex JSON structures and extract specific data elements makes it an ideal choice for reading JSON files in Bash scripts.
2. Extracting keys and values
Before we can start reading the data, we’re first going to need a JSON file. For the first example we will be using the following JSON file named “example.json”.
{
"name": "Mark",
"age": 25,
"email": "[email protected]",
"address": {
"city": "London",
"country": "England"
},
"hobbies": ["running", "cooking", "writing"],
"is_student": true
}
Let’s first take a look at the jq command and how it works. We can either pipe the JSON contents to the jq command or we can directly access it using the jq command.
$ cat example.json | jqor$ jq '.' example.json
Both of these commands will print out the entire contents of our JSON file.
Now let’s take a look at how we can extract certain values. Let’s say for example we wan’t to get the value of the “name” key. In that case we can run the following command.
$ jq '.name' example.json“Mark”We can get a basic understanding of how the command works at this point. We call the command with the jq keyword, after that we specify, in quotes, which key we want to read preceded by a dot. In the first example we wanted to read everything so we simply put a dot in quotes without specifying which key to read.
If we wan’t to get the output as a raw string we can use the -r flag, which would look like.
$ jq -r '.name' example.json
MarkNow, we can use all of this to write a simple bash script.
#!/bin/bash
input_file="example.json"
data_age=`jq -r '.age' "$input_file"`
echo $data_age
The input_file variable contains the name of our JSON file and for the data_age variable we specified our command inside backticks. In bash backticks (`) are used as a command substitution, meaning anything inside them will be treated as a command that should be run, and will be replaced with the commands output. Finally, we print out our variable using echo.
$ chmod +x script.sh
$ ./script.sh
25Moving on, let’s take a look at how to grab both the key and it’s value. In this example we will grab each key and it’s value, then store them in their variables. Here is a bash script that can do that.
#!/bin/bash
input_file="example.json"
mapfile -t json_data < <(jq -r "to_entries[] | \"\(.key)=\(.value|tostring)\"" "$input_file")
for pair in "${json_data[@]}"; do
IFS='=' read -r key value <<< "$pair"
echo "$key: $value"
done
The mapfile command reads each line of the output produced by jq and stores it in the json_data array. The way we do this is by running the jq command inside the <( ) brackets and piping it to the mapfile command using another < character. Process substitution is similar to the command substitution we used in the previous script.
The to_entries function converts the JSON object into an array of key-value pairs. The [] operator is used to iterate over each element in that array. After that we specify that we want to store them as key=value pairs, making sure to convert the final value to a string.
Then, the for loop iterates through the array, extracting the key and value from each pair, and prints them on separate lines with a colon separating them. In essence, the script extracts and displays the key-value pairs from the JSON file, providing a human-readable representation of the data.
3. Looping through JSON data
Although we used loops in the previous script, here we have a different application for them. Instead of using loops to grab each key-value pair, we will be using them to iterate through objects inside a parent object (employees) of our new JSON file.
{
"employees": [
{
"name": "John Doe",
"age": 30,
"email": "[email protected]"
},
{
"name": "Jane Smith",
"age": 25,
"email": "[email protected]"
}
]
}
#!/bin/bash
input_file="example.json"
jq -r '.employees[] | "\(.name), \(.email)"' "$input_file" |
while IFS= read -r employee_info; do
echo "Employee Info: $employee_info"
done
We use a loop to iterate through the array of "employees" in the JSON file. By utilizing jq and a loop, we can extract and process each employee's "name" and "email" fields. The jq expression '.employees[] | "\(.name), \(.email)"' iterates through the "employees" array and formats the output as "name, email". The loop then reads each line, representing an employee, and prints their information using echo.
In the following example we are going to see how we can perform actions on each element, by using the employees’ age.
#!/bin/bash
input_file="example.json"
jq -r '.employees[] | "\(.name), \(.email), \(.age)"' "$input_file" |
while IFS= read -r employee_info; do
name=$(echo "$employee_info" | cut -d ',' -f1)
email=$(echo "$employee_info" | cut -d ',' -f2)
age=$(echo "$employee_info" | cut -d ',' -f3)
age_in_days=$((age * 365))
echo "Employee Name: $name"
echo "Employee Email: $email"
echo "Employee Age: $age years ($age_in_days days)"
echo "--------------------------------"
done
We extend the previous script to include the "age" field for each employee. After extracting the "name", "email", and "age" using jq, we calculate the employee's age in days by multiplying their age by 365. The loop prints the employee's name, email, and age in years and days, providing a more detailed view of each employee's information.
4. Conditional processing with JSON data
Finally, we are going to look at how we can do conditional statements with the extracted JSON data. In the example below we are classifying each employee by the department they work in and printing that data out. In the case that an employee doesn’t work in either of the three specified departments (engineering, hr, sales), we are simply going to print out that they work in an unknown department. Once again we have modified our JSON file to fit the example.
{
"employees": [
{
"name": "John Doe",
"age": 30,
"department": "Engineering"
},
{
"name": "Jane Smith",
"age": 25,
"department": "HR"
},
{
"name": "Michael Johnson",
"age": 35,
"department": "Sales"
}
]
}
#!/bin/bash
input_file="example.json"
jq -r '.employees[] | "\(.name),\(.department)"' "$input_file" |
while IFS= read -r employee_info; do
name=$(echo "$employee_info" | cut -d ',' -f1)
department=$(echo "$employee_info" | cut -d ',' -f2)
department=$(echo "$department" | awk '{$1=$1};1')
case "$department" in
Engineering)
echo "$name works in the Engineering department."
;;
HR)
echo "$name works in the Human Resources department."
;;
Sales)
echo "$name works in the Sales department."
;;
*)
echo "$name works in an unknown department."
;;
esac
done
In this script, we use conditional processing with jq and a loop to categorize employees based on their "department". The jq expression '.employees[] | "\(.name),\(.department)"' extracts the "name" and "department" fields for each employee. The loop processes each employee's data, removing leading spaces from the department, and uses a case statement to print a custom message based on their department. This allows us to easily classify employees into different departments based on the JSON data.
Comments