Query String Parser Guide: Repeated Keys, Brackets, Empty Values and Why Parsers Disagree

The grammar, and what it leaves undefined

A query string is everything after ? and before #. RFC 3986 only says which characters may appear there. The key=value&key=value convention comes from HTML forms, and the WHATWG URL Standard defines it precisely as application/x-www-form-urlencoded: split on &, split each piece at the first =, replace + with a space, then percent-decode.

What no standard defines is what a repeated key means, whether tags[] is an array, or whether a.b=1 is nested. Those are conventions of individual languages and libraries, and that is where parsers diverge.

One query string, four parsers

This is real output for a=1&a=2&b[]=x&b[]=y&c[k]=v&empty=&flag&sp=a+b%20c.

PHP parse_str / $_GET
  {"a":"2", "b":["x","y"], "c":{"k":"v"}, "empty":"", "flag":"", "sp":"a b c"}

Python urllib.parse.parse_qs (defaults)
  {'a': ['1','2'], 'b[]': ['x','y'], 'c[k]': ['v'], 'sp': ['a b c']}
  -> empty and flag are GONE

JavaScript URLSearchParams
  get('a') = '1'   getAll('a') = ['1','2']   get('flag') = ''   get('b[]') = 'x'

Node.js querystring.parse
  { a: ['1','2'], 'b[]': ['x','y'], 'c[k]': 'v', empty: '', flag: '', sp: 'a b c' }

Three disagreements matter. For repeated keys, PHP keeps the last value, URLSearchParams.get returns the first, and Python and Node return all of them. Brackets only mean arrays and objects in PHP and in libraries like qs; everywhere else, b[] is just a key name with brackets in it. And blank values are silently dropped by Python's parse_qs unless you pass keep_blank_values=True, so a ?debug flag disappears without error.

Repeated keys: arrays by convention

There are three common ways to send a list, and the server must expect the one the client uses.

?tag=a&tag=b          repeated key: URLSearchParams, Python, Go, Java servlets, Spring
?tag[]=a&tag[]=b      bracket suffix: PHP, Rails, qs (Express), jQuery.param
?tag=a,b              comma-joined: OpenAPI style=form, explode=false; needs manual split

The failure is quiet. Send ?tag=a&tag=b to PHP and $_GET['tag'] is just 'b'. Send ?tag[]=a to a Python service and the key is literally tag[]. When filters "only apply the last value", check which convention each side uses.

// JavaScript: build and read repeated keys
const p = new URLSearchParams();
['a', 'b'].forEach((t) => p.append('tag', t));
p.toString();          // 'tag=a&tag=b'
p.getAll('tag');       // ['a', 'b']

from urllib.parse import parse_qs
parse_qs('tag=a&tag=b&debug', keep_blank_values=True)
# {'tag': ['a', 'b'], 'debug': ['']}

Nested objects with brackets

PHP and the qs library turn filter[status]=paid&filter[total][gt]=100 into nested objects. That is convenient for complex filters, and dangerous when the depth and size are not limited.

Express used qs with this "extended" parsing by default until Express 5, which switched the default query parser to "simple", so brackets no longer create objects unless you opt in. Code upgraded from Express 4 that reads req.query.filter.status gets undefined after the upgrade.

When nested parsing is on, a key such as __proto__[isAdmin]=1 can try to modify object prototypes, a vulnerability class called prototype pollution. Current qs versions block it by default, but hand-written nested parsers are often vulnerable. Validate the parsed query against a schema instead of passing it straight into merges or database queries.

Empty values, missing values and plus signs

?q= (empty value), ?q (no =) and a missing q are three different states, but most parsers collapse the first two into an empty string, and Python drops both by default. If your API distinguishes "filter by empty" from "no filter", document it and test each parser in the chain.

In form-encoded query strings, + means a space. ?q=c++ arrives as c (c plus two spaces), and a phone number +44… loses its plus. Clients must encode a literal + as %2B. URLSearchParams and encodeURIComponent do this; string concatenation does not.

The semicolon is a historical separator. Python's parse_qs used to split on ; as well as &, until a security fix in Python 3.9.2 and 3.8.8 (CVE-2021-23336) removed that because proxies and back ends disagreed about where parameters started. Modern parsers treat semi=1;d=2 as one value, 1;d=2.

HTTP parameter pollution

When two components parse the same query differently, an attacker can exploit the gap. A classic case: a WAF or signature check reads the first amount, and the application reads the last one.

/transfer?amount=10&to=alice&amount=10000
  gateway (first value wins) -> checks amount=10, allows it
  PHP back end (last wins)   -> transfers 10000

Defences: reject requests with unexpected repeated keys for parameters that must be single-valued; make every layer use the same parser; and include the whole canonical query in anything you sign, such as webhook signatures or presigned URLs, rather than selected parameters.

Limits and robustness

Parsers need limits, because a query string is attacker-controlled input. PHP stops after max_input_vars variables (1000 by default) and max_input_nesting_level levels. qs limits parameters to 1000 and nesting depth to 5 by default. Servers also cap URL length: many reject request lines over 8 KB, and browsers and CDNs have their own limits. Very large or deeply nested queries should be a POST body instead.

Percent-decoding can fail on malformed input such as %E0%A4 or a bare %. Decide whether to reject the request with a 400 or keep the raw value, and do it the same way everywhere.

Sources and further reading

RFC 3986 defines the query component. The WHATWG URL Standard defines application/x-www-form-urlencoded parsing and URLSearchParams. The Python urllib.parse documentation covers keep_blank_values and the CVE-2021-23336 separator change. The PHP manual documents parse_str and max_input_vars, the qs README lists its defaults, and the Express 5 migration guide describes the query-parser change. OWASP describes HTTP parameter pollution.

Frequently asked questions

How do I parse a query string in JavaScript?

Use `new URLSearchParams(location.search)` or `new URL(href).searchParams`. `get(key)` returns the first value, `getAll(key)` returns every value for a repeated key, and `+` is decoded as a space.

How are repeated query parameters handled?

It depends on the parser. PHP keeps only the last value unless the key ends in `[]`; Python's `parse_qs`, Node's `querystring` and Java servlets return all values; `URLSearchParams.get` returns the first. Use `getAll` or a list-aware API when multiple values are allowed.

Why does parse_qs drop my empty parameter?

Python's `parse_qs` and `parse_qsl` ignore blank values by default. Pass `keep_blank_values=True` to keep `?q=` and `?flag` as empty strings.

How do I send an array in a query string?

Either repeat the key (`?id=1&id=2`), which most languages understand, or use brackets (`?id[]=1&id[]=2`) for PHP, Rails and `qs`. Choose the form your server parses, or document both.

Why does + become a space in my query parameter?

Query strings are decoded with form rules, where `+` means a space. Encode a literal plus as `%2B`, for example with `encodeURIComponent` or `URLSearchParams`.

What is HTTP parameter pollution?

An attack that sends the same parameter several times so that different components, such as a gateway and an application, read different values. Reject unexpected duplicates and use one parser consistently.