Me: “Hmm… No… No the code is good, it’s the compiler that’s wrong.”
runs again
Somehow higher than 0% success rate.
I often do this, but I always hit Ctrl-S before running it again. Shamefully, this probably works about 10% of the time. Does that technically count as changing nothing?
Sponsored by QA gang. Gotta make sure it’s a 5/5 issue and not just a frequent issue
Ever work?
All the time. Causes include:
- Test depends on an external system (database, package manager)
- Race conditions
- Failing the test cleared bad state (test expects test data not to be in the system and clears it when it exits)
- Failing test set up unknown prerequisite (Build 2 tests depends on changes in Build 1 but build system built them out of order)
- External forces messing with the test runner (test machine going to sleep or running out of resources)
We call those “flaky tests” and only fail a build if a given test cannot pass after 2 retries. (We also flag the test runs for manual review)