Me: “Hmm… No… No the code is good, it’s the compiler that’s wrong.”
runs again
I often do this, but I always hit Ctrl-S before running it again. Shamefully, this probably works about 10% of the time. Does that technically count as changing nothing?
Ever work?
All the time. Causes include:
- Test depends on an external system (database, package manager)
- Race conditions
- Failing the test cleared bad state (test expects test data not to be in the system and clears it when it exits)
- Failing test set up unknown prerequisite (Build 2 tests depends on changes in Build 1 but build system built them out of order)
- External forces messing with the test runner (test machine going to sleep or running out of resources)
We call those “flaky tests” and only fail a build if a given test cannot pass after 2 retries. (We also flag the test runs for manual review)
And the worst part is when it actually does and you have no fucking idea what went wrong before.
Sponsored by QA gang. Gotta make sure it’s a 5/5 issue and not just a frequent issue