Why do some Flutter widget tests fail if executed together but pass individually?

Issue

When having multiple tests in the same file, and running one test after the other. Some kinds of tests are failing, but the same tests pass when run individually.

This is my test file at the moment, it’s kinda long though:

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:rec/Components/Inputs/text_fields/DniTextField.dart';
import 'package:rec/Helpers/Validators.dart';

import '../../test_utils.dart';

void main() {
  testWidgets('DniTextField works with invalid DNI', (WidgetTester tester) async {
    var key = GlobalKey<DniTextFieldState>();
    var formKey = GlobalKey<FormState>();
    var onChangedResult;

    var widget = Form(
      key: formKey,
      child: DniTextField(
        key: key,
        onChange: (String value) {
          onChangedResult = value;
        },
        validator: (s) => Validators.verifyIdentityDocument(s),
      ),
    );
    await tester.pumpWidget(
      TestUtils.wrapPublicRoute(widget),
    );
    await tester.pumpAndSettle();

    // Test that widget has at least rendered
    TestUtils.widgetExists(widget);

    // Enter text into field with
    var widgetFinder = find.byWidget(widget);
    await tester.tap(widgetFinder);
    await tester.showKeyboard(widgetFinder);
    await tester.enterText(widgetFinder, 'invaliddni');

    await tester.pumpAndSettle();

    expect(onChangedResult, 'invaliddni');

    expect(formKey.currentState.validate(), false);
  });

  testWidgets('DniTextField works with valid DNI', (WidgetTester tester) async {
    var key = GlobalKey<DniTextFieldState>();
    var formKey = GlobalKey<FormState>();
    var onChangedResult;

    var widget = Form(
      key: formKey,
      child: DniTextField(
        key: key,
        onChange: (String value) {
          onChangedResult = value;
        },
        validator: (s) => Validators.verifyIdentityDocument(s),
      ),
    );
    await tester.pumpWidget(
      TestUtils.wrapPublicRoute(widget),
    );
    await tester.pumpAndSettle();

    // Enter text into field
    var widgetFinder = find.byType(DniTextField);
    await tester.tap(widgetFinder);
    await tester.showKeyboard(widgetFinder);
    await tester.enterText(widgetFinder, '80008000k');

    await tester.pumpAndSettle();

    expect(onChangedResult, '80008000k');

    expect(formKey.currentState.validate(), true);
  });

  testWidgets('DniTextField works with valid DNI with trailing space', (WidgetTester tester) async {
    var key = GlobalKey<DniTextFieldState>();
    var formKey = GlobalKey<FormState>();
    var onChangedResult;

    var widget = Form(
      key: formKey,
      child: DniTextField(
        key: key,
        onChange: (String value) {
          onChangedResult = value;
        },
        validator: (s) => Validators.verifyIdentityDocument(s),
      ),
    );
    await tester.pumpWidget(TestUtils.wrapPublicRoute(widget));
    await tester.pumpAndSettle();

    // Enter text into field
    var widgetFinder = find.byWidget(widget);
    await tester.tap(widgetFinder);
    await tester.showKeyboard(widgetFinder);
    await tester.enterText(widgetFinder, '80008000k ');

    await tester.pumpAndSettle();

    // The value emitted by the field, should be free of trailing whitespace
    expect(onChangedResult, '80008000k');

    expect(formKey.currentState.validate(), true);
  });
}

If I run this like this:

$ flutter test test/Components/inputs/dni_text_field_test.dart

The first test passes, but the next 2 don’t, spitting out the following error:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
The finder "zero widgets with the given widget
(DniTextField-[LabeledGlobalKey<DniTextFieldState>#0fb86]) (ignoring offstage widgets)" (used in a
call to "tap()") could not find any matching widgets.

When the exception was thrown, this was the stack:
#0      WidgetController._getElementPoint (package:flutter_test/src/controller.dart:902:7)
#1      WidgetController.getCenter (package:flutter_test/src/controller.dart:841:12)
#2      WidgetController.tap (package:flutter_test/src/controller.dart:273:18)
#3      main.<anonymous closure> (file:///[masked]/test/Components/inputs/dni_text_field_test.dart:99:18)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)

If I then comment out the first test, then the second one passes but the third one does not. As I said the test pass if I run them individually.

I can’t find any information about this, not even one thing. Maybe there is someone here that can guide me in the right direction. I’m a bit of a noob with Flutter widget testing, so I might be missing some important things.

The only solution I’ve found for now is having each test in a separate file. It’s not ideal though, would be much better to contain related tests in the same file. Flutter has examples showing that multiple tests per file is allowed as it should be.

Flutter Version: 2.4.0-5.0.pre.87

Solution

Okay I found the solution and reason, fixed thanks to this issue and this SO question.

The main issue was the use of Localizations that loaded from JSON files. The solution was to wrap each test with tester.runAsync()

testWidgets('widget test 2', (WidgetTester tester) async {
  await tester.runAsync(() async {
    // tests
  });
});

Answered By – Keff

Answer Checked By – Jay B. (FlutterFixes Admin)