Recent chats
ASKGEEZ Q&A GUIDETechnology & devices

How can I check for duplicate keys while parsing a JSON object?

Quick answer

Python’s standard JSON decoder accepts repeated object names by default and keeps only the last value, so loading directly into a dictionary can hide duplicates. Pass an object_pairs_hook that checks the ordered key/value pairs and raises an error when a key repeats. If no duplicate is found, the hook can return a dictionary as usual. The hook also checks nested objects. [S1]

LET’S WORK THROUGH IT

The question, the details, the next step.

Question

I load a JSON file with json.loads() and get a dictionary. How can I tell whether the original object repeated a key?

Geez

A dictionary made by the usual decoder may no longer show every key occurrence: Python’s JSON decoder accepts repeated object names by default and uses the last name/value pair. To catch repeats while parsing, provide object_pairs_hook. It receives an object’s decoded pairs as an ordered list, so your hook can check for repeated names before creating a dictionary. [S1]

A simple policy is to reject the JSON when any object contains a repeated key. That avoids silently choosing a value. Here is a small example:

import json

class DuplicateKeyError(ValueError):
    pass

def reject_duplicate_keys(pairs):
    result = {}
    for key, value in pairs:
        if key in result:
            raise DuplicateKeyError(f"duplicate JSON key: {key!r}")
        result[key] = value
    return result

text = '{"name": "Ada", "name": "Grace"}'

try:
    value = json.loads(text, object_pairs_hook=reject_duplicate_keys)
except DuplicateKeyError as exc:
    print(f"Invalid JSON object: {exc}")
else:
    print(value)

For this sample, parsing raises DuplicateKeyError rather than returning a dictionary that obscures the repeated name. If the input has no duplicate keys, the hook returns ordinary dictionaries and value remains a usable parsed result. This example reports the key name, not its location in the original text.

Follow-up question

Does that catch a duplicate inside a nested object too, or only keys in the outer object?

Geez

It checks nested objects too. The decoder invokes object_pairs_hook for decoded objects, so the same check applies at each level; a duplicate in any one of them raises the error. [S1]

For example, this has different keys at the top level but repeats mode inside settings:

text = '{"settings": {"mode": "quiet", "mode": "loud"}}'
value = json.loads(text, object_pairs_hook=reject_duplicate_keys)

The assignment to value does not complete normally: the hook raises DuplicateKeyError while decoding the nested object. With the try/except shown earlier, you can handle that failure in the same way as an outer-object duplicate. The basic hook does not tell you which nesting path contained the duplicate; it only reports the key.

Follow-up question

I want the valid case to continue through my program. What should my calling code do, and what should I expect if it finds a repeat?

Geez

Keep the parsing call inside a try/except, then use the parsed value only in the success branch. For example:

try:
    data = json.loads(text, object_pairs_hook=reject_duplicate_keys)
except DuplicateKeyError as exc:
    print(f"Please fix the JSON: {exc}")
except json.JSONDecodeError as exc:
    print(f"The JSON could not be parsed: {exc}")
else:
    # Continue with data only after parsing succeeds.
    print(data)

The first handler covers your duplicate-key policy; the second handles a different problem: input that is not valid JSON syntax. The exception messages shown are useful for a small script, but you can replace the print calls with the error-handling approach that fits your program. If you need to identify a precise object path or source position, this minimal hook does not provide that information. The Python documentation describes the hook and the default repeated-name behavior; it does not decide whether your application should reject duplicates or what message to show. [S1]

Follow-up question

Can you give me a compact checklist for adding this to an existing JSON loader without accidentally accepting a duplicate?

Geez

Use the checklist below to add the hook and keep duplicate handling separate from ordinary parse failures. The key design choice is to reject a repeat before building the dictionary; on success, continue with the returned value. The example detects the repeated key but does not provide its object path or exact position in the input. The appropriate rejection policy and user-facing message are choices for your application.

A practical checklist

  • Check each key before adding it to a new dictionary; raise a distinct error on repeats.

  • Pass the hook as object_pairs_hook to the JSON loading call.

  • Handle duplicate-key errors separately from malformed-JSON errors if needed.

  • Use the parsed value only after the load succeeds.

  • Check both top-level and nested duplicates, plus duplicate-free input.

A few more useful answers

Why can’t I check for duplicates after `json.loads()` returns a dictionary?

The standard decoder accepts repeated object names by default and keeps only the last name/value pair, so the earlier occurrence may no longer be visible in the dictionary. Check the pairs during decoding with object_pairs_hook. [S1]

Does the hook automatically tell me the path to a duplicate key?

No. The example detects the repeated key and raises an error, but does not track the object’s path or the key’s position in the input.

What does this approach do to the JSON input?

It validates while decoding; it does not edit or rewrite the input. The hook returns dictionaries for objects without duplicate keys, or raises when it finds one.

Sources

References used for this guide.

  1. [S1] Python json module: object_pairs_hook and repeated names
    docs.python.org · Checked

YOUR QUESTION. YOUR DETAILS.

Let’s work through your situation.

Get a tailored answer or help with your next task. Starts at $1, with a quote before you pay.

Ask your own question
← Back to Geez at Work