Recent chats
ASKGEEZ Q&A GUIDETechnology & devices

Why does a unittest pass alone but fail with the whole test suite?

Quick answer

A test that passes alone but fails in the suite may depend on state another test changed, assume a particular execution order, or leave something behind. Check what the failing test reads and changes, then make its starting state explicit. Use setUp() to create fresh per-test data where practical, and clean up changes reliably. unittest orders test methods by name, so avoid relying on another test to prepare state. [S1]

LET’S WORK THROUGH IT

The question, the details, the next step.

Question

My test passes when I run just that method, but the whole suite fails. Where should I start?

Geez

First capture the exact failure: the assertion or exception and the test name. Then ask whether the test reads or changes anything that can outlive it, such as a module-level list or dictionary, a class attribute, a file, or an environment setting. These are things to inspect, not proof of the cause.

Run the failing test by itself, then run the suite and note whether the same test fails. Look for an earlier test that changes the same state. Each TestCase should be self-contained and runnable alone or with other cases. unittest orders test methods by name, so a test that silently relies on another test running first is fragile. [S1]

Follow-up question

Can you show how shared state creates that problem?

Geez

Suppose a module contains a shared list and two tests append to it. The second test may expect an empty list when run alone, but encounter the first test’s value in the suite. Here is a small illustrative example:

import unittest

items = []

class ItemTests(unittest.TestCase):
    def test_add_one(self):
        items.append("one")
        self.assertEqual(items, ["one"])

    def test_add_two(self):
        items.append("two")
        self.assertEqual(items, ["two"])

Both tests assume the list starts empty, but neither establishes that condition. This example illustrates the risk; it is not a recommendation to use global state in tests.

Follow-up question

What is a minimal way to isolate the tests?

Geez

Create fresh test data in setUp(), which unittest runs for each test. [S1] Replace the shared list with an instance attribute:

import unittest

class ItemTests(unittest.TestCase):
    def setUp(self):
        self.items = []

    def test_add_one(self):
        self.items.append("one")
        self.assertEqual(self.items, ["one"])

    def test_add_two(self):
        self.items.append("two")
        self.assertEqual(self.items, ["two"])

Each method now starts with its own empty list. For state that cannot simply be recreated, identify what the test changes and restore it in cleanup. unittest documents that tearDown() runs even if the test method fails, provided setUp() succeeded. [S1] Avoid relying on a later test to perform cleanup.

Follow-up question

How do I narrow down the cause when the failure is less obvious?

Geez

Compare the conditions around the failure rather than changing several tests at once. Check whether the test uses a shared fixture or class or module attribute, whether it assumes a particular prior test, and whether it cleans up after modifying state. If useful, temporarily run the failing test with a suspected predecessor, then without it, to check that hypothesis. Keep the smallest reproducing case and make its starting state explicit.

A practical investigation note could look like this:

Failing test: test_add_two
State it reads: module-level items list
State it changes: appends "two"
Expected starting state: empty list
Possible source: another test appended without resetting
Isolation change to try: create a fresh list in setUp()

This narrows the investigation; it does not establish that every suite-only failure comes from shared state. The documentation supports checking independence, ordering, and cleanup, but does not identify the cause of a particular failure. [S1]

A practical checklist

  • Record the failing test name and the complete failure message.

  • List shared values, fixtures, files, or settings the test reads or changes.

  • Check whether the test assumes another test runs first; unittest methods are ordered by name. [S1]

  • Create fresh per-test state in setUp() where practical. [S1]

  • Restore changed state in cleanup, including when the test assertion fails.

A few more useful answers

Does unittest always run tests in the same order?

The unittest documentation says test methods are ordered by name. Avoid making one test depend on another having prepared state. [S1]

Is tearDown() enough to fix a suite-only failure?

Only if incomplete cleanup is the cause and cleanup restores the relevant state. For ordinary test data, creating a fresh value in setUp() may be simpler. unittest documents that tearDown() runs after a test method even if it fails, provided setup succeeded. [S1]

What if the test uses a shared class or module fixture?

Inspect whether the fixture is changed and whether its lifecycle leaves state for other tests. Shared fixtures are separate from per-test setup and can break isolation if used carelessly. [S1]

Sources

References used for this guide.

  1. [S1] Python unittest: Test independence, fixtures, order, and cleanup
    docs.python.org · Checked

YOUR QUESTION. YOUR DETAILS.

Let’s work through your situation.

Get a tailored answer or help with your next task. Starts at $1, with a quote before you pay.

Ask your own question
← Back to Geez at Work