Skip to main content
DevOps Insightsguides & answers

jq · troubleshooting

jq: parse error: Invalid numeric literal

jq was given something that is not JSON, usually an HTML error page or a plain-text message from an API that failed. The related Cannot iterate over null means the JSON was valid but did not contain the field you looped over.

Reproduced on jq 1.7.1, . Where the text says “reproduced”, the behaviour was run on that version, not taken from documentation.

First published

New to this? Read what is actually happening first, then come back to the steps.

On this page · 8 sections

Before you start

  • The command that produced the input (often curl), and the ability to see what it actually returned.

Run this first

step 1 of 2
curl -s -o body.out -w '%{http_code}\n' URL

Look for A status of 400 or above

Shows the HTTP status and keeps the body, which the pipe into jq was hiding.

Work out which cause you have

A few questions to narrow the list. Every answer ends in a command that confirms or rules the cause out — this cannot see your cluster, so nothing here is a certainty until you have checked.

Narrow it down

0 answered · nothing is sent anywhere

Which error does jq print?

Cause space

3 of 3 still possible

  • The API returned an error page instead of JSONCommon
  • Valid JSON, but the field you iterate is missingCommon
  • The command never produces JSONOccasional

Nothing ruled out yet. Answer the question above and the branches your answer eliminates will strike through here.

Or diagnose it manually

In this order. The first command usually contains the whole answer.

  1. Step 1

    curl -s -o body.out -w '%{http_code}\n' URL

    Shows the HTTP status and keeps the body, which the pipe into jq was hiding.

    Look for A status of 400 or above

  2. Step 2

    head -c 300 body.out

    Shows what jq was really given.

    Look for <html, an error message, or JSON without the field

Every cause, and how to fix it

Ordered by how often each one turns out to be the answer.

The API returned an error page instead of JSON

Common

A 4xx or 5xx response, a login redirect or a proxy error page. curl prints it like any other response unless told to fail.

Confirm

curl -s -o body.out -w '%{http_code}\n' URL; head -c 300 body.out

A status of 400 or above, and HTML or text where JSON should be

Fix

  • Make the pipeline fail on HTTP errors: set -o pipefail, then curl -sf URL | jq ….
  • Or capture the status first (curl -s -o body.out -w '%{http_code}'), check it, and only then run jq on body.out, which also keeps the error body for the log.

Valid JSON, but the field you iterate is missing

Common

An error object, an empty result or a different API version returns JSON without .items, so .items[] has nothing to loop over.

Confirm

curl -s URL | jq 'keys'

Keys such as message or error instead of the field you expected

Fix

  • Handle the absence on purpose: .items[]? to skip it, or (.items // [])[] to treat it as empty.
  • Use jq -e when the result must exist: it exits 1 when the last output is null or false, so the script stops.

The command never produces JSON

Occasional

A CLI printing a table, a log line before the JSON, or a progress bar on stdout.

Confirm

your-command | head -c 200

Anything before the first { or [

Fix

  • Ask the tool for JSON output (often --output json or -o json) and send its logs to stderr.

Understanding it properly

Skip this if you are mid-incident — the working part of the page is above. Worth reading afterwards, because understanding the mechanism is what stops the next one.

In plain terms

In a pipeline like curl … | jq, jq only sees what curl printed. When the API returns an error page, a login page or a rate-limit message instead of JSON, jq tries to read HTML as numbers and stops. The real problem is the HTTP response, which the pipeline has hidden.

Words used on this page

pipefail
A bash option that makes a pipeline fail when any command in it fails, not just the last one.
curl -f
Makes curl fail (exit 22) on HTTP responses of 400 and above instead of printing the error page.
?, //
jq operators: .items[]? skips a missing field instead of failing, and .items // [] substitutes an empty list.
What is actually happening

Reproduced with jq 1.7.1 on macOS and jq 1.7 on Ubuntu. An HTML page piped to jq gave "jq: parse error: Invalid numeric literal at line 1, column 16" and exit code 5; plain text such as Not Found gave the same error. JSON that lacked the field, such as an API's rate-limit message, gave "jq: error (at <stdin>:1): Cannot iterate over null (null)" for .items[].

Against a test server answering 502 with an HTML page, curl -s … | jq '.items' showed only jq's parse error, and the pipeline's exit code was jq's. With set -o pipefail and curl -sf, the pipeline exited 22, curl's code for an HTTP error, so the real failure was visible. curl --fail-with-body still passed the HTML to jq.

How to tell this is your problem
WhereWhat you see
jq, input is HTML or textjq: parse error: Invalid numeric literal at line 1, column 16
jq, field missingjq: error (at <stdin>:1): Cannot iterate over null (null)

How to know it is actually fixed

  • jq gets JSON and the script fails loudly, with the HTTP status, when the API does not return it.

Stopping it happening again

  • Start scripts with set -euo pipefail and use curl -f, so an HTTP error stops the script at the request, not three steps later.
  • Write jq filters that state what to do when a field is missing, instead of assuming it is there.

Sources

Behaviour described here is drawn from official documentation. Where a figure could not be confirmed on an official page it is attributed in the text rather than stated as canonical.

Did this get you to an answer?

No text box on purpose — please do not paste production logs anywhere

Something wrong or out of date? Report it

Where to go next