Quick Tips
3 min

Debugging Flaky Tests

Quick strategies to identify and fix flaky tests that pass and fail randomly

...
debuggingflaky-teststestingtroubleshooting

Identifying Flaky Tests

1. Run Tests Multiple Times

# Run test 10 times
for i in {1..10}; do npm test -- tests/flaky.spec.js; done
 
# Use test runner's repeat feature
npm test -- --repeat-each=10 tests/flaky.spec.js

2. Common Causes

Race Conditions

// FLAKY ❌
test('data loads', async () => {
  fetchData();
  expect(getData()).toBeDefined(); // May fail!
});
 
// FIXED ✅
test('data loads', async () => {
  await fetchData();
  expect(getData()).toBeDefined();
});

Improper Waiting

// FLAKY ❌
await page.click('button');
await page.waitForTimeout(1000); // Arbitrary wait
 
// FIXED ✅
await page.click('button');
await page.waitForSelector('.result'); // Wait for condition

Test Order Dependency

// FLAKY ❌
let userId;
 
test('create user', () => {
  userId = createUser();
});
 
test('update user', () => {
  updateUser(userId); // Depends on previous test!
});
 
// FIXED ✅
test('update user', () => {
  const userId = createUser(); // Independent
  updateUser(userId);
});

Shared State

// FLAKY ❌
const cache = {};
 
test('test 1', () => {
  cache.value = 'test1';
  expect(cache.value).toBe('test1');
});
 
test('test 2', () => {
  expect(cache.value).toBeUndefined(); // May fail!
});
 
// FIXED ✅
test('test 1', () => {
  const cache = {};
  cache.value = 'test1';
  expect(cache.value).toBe('test1');
});

3. Debugging Techniques

Add Retry Logic (Temporary)

test('flaky test', async () => {
  await retry(async () => {
    const result = await fetchData();
    expect(result).toBe('expected');
  }, { retries: 3 });
});

Add Logging

test('debug flaky test', async () => {
  console.log('Before action:', getState());
  await performAction();
  console.log('After action:', getState());
  expect(getState()).toBe('expected');
});

Take Screenshots on Failure

test('ui test', async ({ page }) => {
  try {
    await page.click('button');
    await expect(page.locator('.result')).toBeVisible();
  } catch (error) {
    await page.screenshot({ path: 'failure.png' });
    throw error;
  }
});

4. Tools for Detection

// Playwright has built-in retry
test.describe.configure({ retries: 2 });
 
// Jest can repeat tests
test.each([1, 2, 3])('attempt %i', (attempt) => {
  // Test code
});

Quick Fixes

  1. Add explicit waits instead of timeouts
  2. Make tests independent - each test sets up its own data
  3. Clean up state after each test
  4. Use stable selectors (data-testid instead of class names)
  5. Mock time-dependent code (dates, timers)
  6. Increase timeouts only as last resort

Prevention

// Good: Deterministic
test('sorts users alphabetically', () => {
  const users = [
    { name: 'Charlie' },
    { name: 'Alice' },
    { name: 'Bob' },
  ];
  const sorted = sortUsers(users);
  expect(sorted[0].name).toBe('Alice');
});
 
// Good: Isolated
beforeEach(async () => {
  await resetDatabase();
  await seedTestData();
});
 
// Good: Explicit waits
await page.waitForSelector('.loaded');
await page.waitForLoadState('networkidle');

Remember: A flaky test is worse than no test - fix them immediately!

Comments (0)

Loading comments...