URL Encode from the Command Line: curl, jq, Python and Bash Examples

Let curl encode the query for you

If the encoded value is going into a curl request anyway, do not encode it separately. --data-urlencode encodes a name=value pair, and -G (--get) moves the data from a POST body into the query string:

v='R&B café/50% off?'
curl -G --data-urlencode "q=$v" --data-urlencode "page=2" https://api.example.com/search
# requests: /search?q=R%26B+caf%C3%A9%2F50%25+off%3F&page=2

curl 7.87.0 and later also have --url-query, which appends to the query string without turning the request into a GET, so it can be combined with a JSON body. A leading + passes a value through unencoded when you already have it encoded:

curl --url-query "q=$v" https://api.example.com/search
# /search?q=R%26B+caf%C3%A9%2F50%25+off%3F

curl --url-query "+q=already%20encoded" https://api.example.com/search
# /search?q=already%20encoded

Two details from the curl manual are easy to miss. Only the part after the first = is encoded; the name is expected to be encoded already. And curl writes spaces as +, the form-encoding convention, which every query-string parser accepts but which is wrong for path segments.

Encode a string without sending a request

When you need the encoded text itself, for a path segment, a log line or a configuration file, jq is the cleanest tool if it is installed. Its @uri filter encodes everything except the RFC 3986 unreserved characters and uses %20 for spaces:

jq -rn --arg v "$v" '$v|@uri'
# R%26B%20caf%C3%A9%2F50%25%20off%3F

# from a pipe: -R reads raw text, -s slurps it into one string
printf %s "$v" | jq -sRr @uri

There is also a well-known curl trick that prints the encoded value without contacting anyone: ask for the effective URL of a request to a dummy host and discard the error. It works, but the output still has the dummy prefix and + for spaces, so jq or Python is clearer in scripts.

Encoding the value yourself is also the only correct option for path segments. A file name such as Q3 report/final.pdf placed in /files/{name} must become Q3%20report%2Ffinal.pdf: the space as %20, because + means a literal plus in a path, and the slash encoded, because an unencoded / creates an extra path segment and the server looks for a directory that does not exist. curl's own options only touch the query string, so build the path with jq or Python and pass the finished URL to curl.

Python and Node one-liners

Python 3 is present on nearly every Linux distribution and on macOS with the developer tools, which makes it the most portable choice. Pass the value as an argument, not by interpolating it into the code, so quotes in the value cannot break the command:

python3 -c 'import sys, urllib.parse; print(urllib.parse.quote(sys.argv[1], safe=""))' "$v"
# R%26B%20caf%C3%A9%2F50%25%20off%3F

# decode (unquote_plus also turns + back into a space)
python3 -c 'import sys, urllib.parse; print(urllib.parse.unquote_plus(sys.argv[1]))' 'R%26B+caf%C3%A9'
# R&B café

node -e 'console.log(encodeURIComponent(process.argv[1]))' "$v"

safe="" matters: by default quote() leaves / unencoded, which is right for paths and wrong for a value that will sit in a query string. Use quote_plus instead of quote if the receiving side expects + for spaces, as HTML forms do.

Pure Bash, when nothing else is installed

Minimal containers and embedded systems sometimes have neither jq nor Python. A Bash function can encode by converting the string to bytes with od and keeping only unreserved characters:

urlencode() {
  local hex out="" byte
  for hex in $(printf %s "$1" | od -An -v -tx1); do
    byte=$(printf "\\x$hex")
    case $byte in
      [a-zA-Z0-9.~_-]) out+=$byte ;;
      *) out+="%${hex^^}" ;;
    esac
  done
  printf '%s\n' "$out"
}

urlencode 'R&B café/50% off?'
# R%26B%20caf%C3%A9%2F50%25%20off%3F

Working on bytes is the important part. Many snippets online loop over characters and use printf '%02X' "'$c", which gives the code point of a character instead of its UTF-8 bytes, or fails differently on glibc and musl. We tested the version above on Debian and Alpine with identical output. It needs Bash 4 for ${hex^^}; the /bin/bash that ships with macOS is 3.2, so use the Python line there.

The trailing newline trap

echo adds a newline, and every encoder faithfully encodes it as %0A. The resulting URL looks right in a terminal, because the newline is invisible at the end, but the server receives a different value:

echo "$v" | python3 -c 'import sys, urllib.parse; print(urllib.parse.quote(sys.stdin.read(), safe=""))'
# R%26B%20caf%C3%A9%2F50%25%20off%3F%0A      <- note %0A

printf %s "$v" | python3 -c 'import sys, urllib.parse; print(urllib.parse.quote(sys.stdin.read(), safe=""))'
# R%26B%20caf%C3%A9%2F50%25%20off%3F

Use printf %s when piping, or pass the value as an argument. The same applies to values read from files: $(cat file) strips trailing newlines, but curl --data-urlencode name@file does not, and the curl manual says so explicitly.

PowerShell and Windows

On Windows, PowerShell can call the .NET encoder directly. EscapeDataString follows RFC 3986, so spaces become %20 and / is encoded:

[uri]::EscapeDataString('R&B café/50% off?')
# R%26B%20caf%C3%A9%2F50%25%20off%3F

[uri]::UnescapeDataString('R%26B%20caf%C3%A9')
# R&B café

curl.exe, which ships with Windows 10 and 11, supports the same --data-urlencode options as on Linux. Call it as curl.exe; in Windows PowerShell 5.1, plain curl is an alias for Invoke-WebRequest.

Whichever method you use, check a tricky value in a URL encoder and decoder once, and keep it as a test case for the script.

Sources and further reading

RFC 3986 defines unreserved characters and percent-encoding of UTF-8 bytes. The curl manual documents --data-urlencode, --get and --url-query (added in curl 7.87.0), including the encoding rules for names and file contents. The jq manual describes the @uri format string. The Python urllib.parse documentation covers quote, quote_plus and the safe parameter, and Microsoft's documentation describes Uri.EscapeDataString.

Frequently asked questions

How do I URL encode a string in Bash?

Use jq -rn --arg v "$v" '$v|@uri' or a Python one-liner with urllib.parse.quote. If neither is available, a Bash function that converts the string to bytes with od and percent-encodes everything except unreserved characters works on any Linux.

How do I URL encode query parameters with curl?

Use -G with --data-urlencode "name=value", or --url-query "name=value" in curl 7.87.0 and later. curl encodes the value part and appends it to the query string.

Why does my encoded string end with %0A?

The input had a trailing newline, usually from echo. Use printf %s instead of echo, or pass the value as an argument rather than through a pipe.

Does curl encode spaces as + or %20?

--data-urlencode and --url-query encode spaces as +, the form-encoding convention that query-string parsers accept. If you need %20, encode the value with jq or Python and pass it with a leading + in --url-query.

How do I URL decode on the command line?

Use python3 -c 'import sys, urllib.parse; print(urllib.parse.unquote_plus(sys.argv[1]))' with the encoded text. unquote_plus also converts + back to a space; use unquote if + should stay as it is.

How do I URL encode in PowerShell?

Call [uri]::EscapeDataString('text'), which follows RFC 3986 and encodes spaces as %20. [uri]::UnescapeDataString reverses it.