jq Tutorial: The Complete Guide to Command-Line JSON Processing

What is jq?

jq is a lightweight, command-line JSON processor. Just as sed, awk, and grep are essential tools for working with plain text, jq is purpose-built for slicing, filtering, mapping, and transforming JSON data. It takes JSON input, applies one or more filters, and produces JSON (or plain text) output.

Whether you need to analyze an API response, parse JSON log files, modify a configuration file, or extract specific fields from a large dataset, jq lets you accomplish these tasks with concise one-liner commands — no need to write complex scripts in Python or Node.js. It has become an indispensable tool for DevOps engineers, backend developers, data engineers, and anyone who works with JSON on the command line.

jq is open-source, written in C, and has zero runtime dependencies, making it fast and easy to install on virtually any system.

Installing jq

jq can be installed on most operating systems using standard package managers. Here are the installation commands for the most common platforms:

macOS (Homebrew)

brew install jq

Ubuntu / Debian

sudo apt-get update
sudo apt-get install jq

Windows (Chocolatey)

choco install jq

Windows (Scoop)

scoop install jq

After installation, verify that jq is working correctly by checking the version:

$ jq --version
jq-1.7.1

Basic Filters

The core concept of jq is the filter. You pass JSON data to jq along with a filter expression, and jq produces the filtered output. Let's use the following sample JSON for the examples in this section:

{
  "name": "Alice Johnson",
  "age": 30,
  "address": {
    "city": "New York",
    "district": "Manhattan"
  },
  "hobbies": ["reading", "coding", "hiking"]
}

Identity Filter: .

The simplest jq filter is the dot (.), which outputs the input unchanged. This is most commonly used to pretty-print compressed JSON.

echo '{"name":"Alice","age":30}' | jq '.'

Output:

{
  "name": "Alice",
  "age": 30
}

Field Access: .key

Use dot notation to extract the value of a specific field from an object.

echo '{"name":"Alice","age":30}' | jq '.name'
# Output: "Alice"

echo '{"name":"Alice","age":30}' | jq '.age'
# Output: 30

Nested Field Access: .key.nested

Chain dots together to access fields in nested objects.

cat data.json | jq '.address.city'
# Output: "New York"

Array Iteration: .[]

The .[] filter iterates over all elements of an array, outputting each one individually.

cat data.json | jq '.hobbies[]'
# Output:
# "reading"
# "coding"
# "hiking"

Array Index: .[n]

Access a specific element by its zero-based index. You can also use slice notation to extract a range of elements.

cat data.json | jq '.hobbies[0]'
# Output: "reading"

cat data.json | jq '.hobbies[1:3]'
# Output: ["coding", "hiking"]

Pipes and Chaining

The | operator in jq works similarly to the Unix pipe. It takes the output of the filter on the left and feeds it as input to the filter on the right. This allows you to chain multiple filters together to build powerful data processing pipelines.

# Extract the "name" field from each element of an array
echo '[{"name":"Alice","age":25},{"name":"Bob","age":30}]' | jq '.[] | .name'
# Output:
# "Alice"
# "Bob"

# Wrap the results back into an array
echo '[{"name":"Alice","age":25},{"name":"Bob","age":30}]' | jq '[.[] | .name]'
# Output: ["Alice", "Bob"]

You can chain as many filters as needed. Each pipe passes the result of one transformation to the next, making it easy to build complex queries incrementally.

Built-in Functions

jq includes a rich set of built-in functions that make it easy to work with JSON data. Here are some of the most commonly used ones:

length

Returns the length of an array, the number of keys in an object, or the length of a string.

echo '["a","b","c"]' | jq 'length'
# Output: 3

echo '{"x":1,"y":2}' | jq 'length'
# Output: 2

keys and values

keys returns an array of an object's keys (sorted alphabetically). values returns an array of its values.

echo '{"name":"Alice","age":30}' | jq 'keys'
# Output: ["age", "name"]

echo '{"name":"Alice","age":30}' | jq 'values'
# Output: ["Alice", 30]

type

Returns the data type of a value as a string.

echo '42' | jq 'type'
# Output: "number"

echo '"hello"' | jq 'type'
# Output: "string"

echo '{"a":1}' | jq 'type'
# Output: "object"

select

Filters values based on a boolean condition. If the condition is true, the value passes through; otherwise, it is suppressed. Think of it as the equivalent of SQL's WHERE clause.

# Filter users older than 28
echo '[{"name":"Alice","age":25},{"name":"Bob","age":30},{"name":"Charlie","age":35}]' \
  | jq '.[] | select(.age >= 28)'
# Output:
# {"name":"Bob","age":30}
# {"name":"Charlie","age":35}

map

Applies a filter to every element of an array and returns the transformed array.

echo '[1, 2, 3, 4, 5]' | jq 'map(. + 10)'
# Output: [11, 12, 13, 14, 15]

echo '[{"name":"Alice","age":25},{"name":"Bob","age":30}]' \
  | jq 'map({name: .name})'
# Output: [{"name":"Alice"},{"name":"Bob"}]

sort_by

Sorts an array of objects by a specified field.

echo '[{"name":"Charlie","age":35},{"name":"Alice","age":25},{"name":"Bob","age":30}]' \
  | jq 'sort_by(.age)'

group_by

Groups array elements by a specified field, returning an array of arrays.

echo '[{"dept":"dev","name":"A"},{"dept":"hr","name":"B"},{"dept":"dev","name":"C"}]' \
  | jq 'group_by(.dept)'
# Output: [[{"dept":"dev","name":"A"},{"dept":"dev","name":"C"}],[{"dept":"hr","name":"B"}]]

unique

Removes duplicate values from an array.

echo '[1,2,2,3,3,3]' | jq 'unique'
# Output: [1, 2, 3]

Conditional Filtering

The select function is one of jq's most powerful features for conditional filtering. You can combine it with string functions, comparison operators, and boolean logic to create sophisticated filters.

# Filter by string containment
echo '[{"name":"frontend-app"},{"name":"backend-api"},{"name":"frontend-lib"}]' \
  | jq '.[] | select(.name | contains("frontend"))'
# Output:
# {"name":"frontend-app"}
# {"name":"frontend-lib"}

# Combine multiple conditions with "and" / "or"
echo '[{"name":"Alice","age":25,"active":true},{"name":"Bob","age":30,"active":false}]' \
  | jq '.[] | select(.age >= 25 and .active == true)'
# Output: {"name":"Alice","age":25,"active":true}

Array Operations

jq provides powerful capabilities for transforming and manipulating arrays.

flatten

Flattens nested arrays into a single-level array.

echo '[[1,2],[3,[4,5]]]' | jq 'flatten'
# Output: [1, 2, 3, 4, 5]

Array Construction: [.[] | ...]

Iterate over an array, transform each element, and collect the results back into a new array.

echo '[{"name":"Alice","score":85},{"name":"Bob","score":92}]' \
  | jq '[.[] | {student: .name, grade: (if .score >= 90 then "A" else "B" end)}]'
# Output: [{"student":"Alice","grade":"B"},{"student":"Bob","grade":"A"}]

add

Sums the elements of a numeric array or concatenates strings/arrays.

echo '[1,2,3,4,5]' | jq 'add'
# Output: 15

echo '[["a","b"],["c","d"]]' | jq 'add'
# Output: ["a","b","c","d"]

Real-World Examples

Let's look at some practical scenarios where jq shines in day-to-day development work.

Processing API Responses: GitHub Repositories

# Fetch a user's repos from the GitHub API, extract key fields,
# sort by stars descending, and show the top 5
curl -s "https://api.github.com/users/octocat/repos" \
  | jq '[.[] | {name: .name, stars: .stargazers_count, language: .language}]
        | sort_by(.stars)
        | reverse
        | .[0:5]'

This command fetches the repository list from GitHub, extracts only the name, star count, and language for each repo, sorts them by stars in descending order, and outputs the top 5 results.

Parsing Log Files

# Filter JSON-formatted logs for ERROR level entries only
cat app.log | jq -c 'select(.level == "ERROR") | {time: .timestamp, msg: .message}'

The -c flag outputs compact (single-line) JSON, which is useful for piping results into other tools or for writing back to log files.

Reshaping Data Structures

echo '{
  "firstName": "John",
  "lastName": "Doe",
  "contacts": {
    "email": "john@example.com",
    "phone": "555-1234"
  }
}' | jq '{
  fullName: (.firstName + " " + .lastName),
  email: .contacts.email
}'
# Output: {"fullName":"John Doe","email":"john@example.com"}

Useful jq One-Liners

Here is a collection of practical jq one-liners that you can use in your daily workflow:

# Pretty-print JSON
cat data.json | jq '.'

# Extract specific fields as CSV
jq -r '.[] | [.name, .email] | @csv' users.json

# Remove null values from an object
jq 'del(.[] | nulls)' data.json

# Merge two JSON files (deep merge)
jq -s '.[0] * .[1]' file1.json file2.json

# Convert an object to an array of key-value pairs
echo '{"a":1,"b":2,"c":3}' | jq 'to_entries'

# Convert key-value pairs back to an object
echo '[{"key":"a","value":1},{"key":"b","value":2}]' | jq 'from_entries'

# Count elements matching a condition
jq '[.[] | select(.status == "active")] | length' users.json

# Sum a numeric array
echo '[1,2,3,4,5]' | jq 'add'
# Output: 15

# Join a string array with a delimiter
echo '["a","b","c"]' | jq 'join(",")'
# Output: "a,b,c"

# Get unique values from a field across an array of objects
jq '[.[].category] | unique' products.json

# Convert JSON to a tab-separated format
jq -r '.[] | [.name, .age] | @tsv' users.json

# Recursively find all keys named "id"
jq '.. | .id? // empty' data.json

Conclusion

jq is the most powerful command-line tool for processing JSON data. Starting from basic filters like . and .key, you can compose complex data transformations by chaining filters with pipes. Built-in functions like select, map,sort_by, group_by, and unique give you the ability to filter, reshape, and aggregate JSON data with remarkable ease.

Whether you are analyzing API responses, parsing application logs, performing data migrations, or simply exploring JSON datasets, jq should be a core part of your command-line toolkit. The more you practice, the more natural jq expressions will become — and the more time you will save.

Want to test jq expressions right in your browser without installing anything? Try the tool below:

jq Playground — Run jq filters directly in your browser and see results instantly. Practice and experiment with jq without any installation.