Recent chats
ASKGEEZ Q&A GUIDETechnology & devices

How can I detect short or extra fields in CSV rows?

Quick answer

Read the header with Python’s csv.reader and use its number of fields as the expected count. Then check each parsed row and report any whose field count differs. Use the CSV parser rather than splitting lines on commas, since quoted fields can contain commas or newlines. The example only reads the file and reports irregular rows; it does not alter the input.

LET’S WORK THROUGH IT

The question, the details, the next step.

Question

What’s a reliable way to find CSV rows with too few or too many fields?

Geez

Use Python’s csv.reader to parse the file, count the header fields, and compare that count with every later row. The reader returns each parsed record as a list of fields, so len(row) gives you the count to check. [S1]

import csv

path = "input.csv"  # Replace with your file path.

with open(path, newline="", encoding="utf-8") as file:
    reader = csv.reader(file)
    header = next(reader, None)

    if header is None:
        print("The file is empty.")
    else:
        expected = len(header)
        print(f"Expected fields per data row: {expected}")

        for record_number, row in enumerate(reader, start=2):
            if len(row) != expected:
                print(
                    f"Record {record_number}: expected {expected} fields, "
                    f"found {len(row)}; source line(s) through {reader.line_num}"
                )

This assumes the first parsed record is a header. The script reads and reports; it does not write to the file. The encoding value is an example—use the encoding appropriate for your file.

Follow-up question

Why not just split each line on commas? And what does the reported line number mean?

Geez

A comma in a CSV file is not always a field boundary. For example, a properly quoted field might contain a comma: "Miller, Ana" can be one field. Quoted fields can also contain line breaks. The CSV parser applies delimiter and quote settings, whereas splitting raw text on commas cannot reliably distinguish those cases. [S1]

There are two useful numbering ideas in the example. record_number counts parsed records, with the header as record 1, so the first data record is 2. reader.line_num is the number of source lines consumed so far; it is not the parsed record number. A quoted record spanning multiple physical lines can make those counts differ. [S1]

The message says “source line(s) through” to avoid implying that every record occupies exactly one physical line. If you need to identify a record by position in the parsed CSV, use its record number; if you need a physical-line reference, the reader’s line counter provides the number of lines consumed.

Follow-up question

Can you show what the output might look like, and how do I decide whether to skip the header?

Geez

Suppose, as an example, the header has three fields and the first data row has two. The report could look like this:

Expected fields per data row: 3
Record 4: expected 3 fields, found 2; source line(s) through 4
Record 7: expected 3 fields, found 4; source line(s) through 7

Those numbers are illustrative, not results from a real file. In the code, next(reader, None) consumes the first parsed record as the header. That is appropriate when your CSV has a header row. If your file has no header, do not treat its first record as column names: set the expected field count from a known schema or another explicit assumption, then check every parsed row starting with the first.

If the file may use a different delimiter or quote style, configure the reader’s dialect to match the file. The example does not infer a schema or determine whether a row’s values are meaningful; it only detects a difference in field count. [S1]

Follow-up question

How should I use the report without accidentally changing or mislabeling the file?

Geez

Keep validation separate from any cleanup step. Review the reported record and line references against the original file, then decide what correction is appropriate. This example opens the input for reading and prints findings; it contains no write operation, so it does not repair, remove, or rewrite rows.

Before relying on a report, confirm that the correct file was selected, that the header assumption is right, and that the parser settings fit the CSV. A field-count match is not proof that every value is valid: a row can have the expected number of fields and still contain missing, misplaced, or unexpected content. The check establishes only whether each parsed row’s field count matches the header’s count.

A practical checklist

  • Confirm whether the first parsed record is a header.

  • Set the expected field count from the header or a known schema.

  • Use csv.reader with settings that match the file; do not split raw lines on commas.

  • Compare len(row) with the expected count for each parsed record.

  • Report parsed-record numbers and, when useful, reader.line_num separately.

  • Review flagged records in the original file before making any edits.

A few more useful answers

Does a matching field count prove that a CSV row is valid?

No. This check only compares the number of parsed fields with the expected count. It does not validate field contents or confirm that values are in the right columns.

Why can the source line count differ from the record number?

A quoted field can contain a newline, so one parsed record may span multiple physical source lines. reader.line_num counts source lines consumed, not parsed records. [S1]

Will this script fix rows with extra or missing fields?

No. It only reports field-count mismatches and does not write to the CSV. Review the original records before deciding how to handle them.

Sources

References used for this guide.

  1. [S1] Python 3.14 csv — CSV File Reading and Writing
    docs.python.org · Checked

YOUR QUESTION. YOUR DETAILS.

Let’s work through your situation.

Get a tailored answer or help with your next task. Starts at $1, with a quote before you pay.

Ask your own question
← Back to Geez at Work