How to run tests repeatedly in Xcode

Sometimes, unit tests don't give you the same result each time. The reasons why are beyond the scope of this quick tip, but the impact they have on developer time is immense.

Flaky tests can cause you to waste time having to re-run CI jobs. I've seen this a lot, and it costs both time and money. It delays your merges, and if you're not merging code you're not shipping features. Flaky tests also mess with the sense of security that you're meant to get from automated tests. They're meant to give you confidence that your code is behaving properly. If your tests give unreliable results, then is that a problem with the code or the test? It then costs more time to solve the problem and fix the flaky test.

Once you think you've got a solution to fix your unreliable test, how do you prove it fixes the problem and gets rid of the flakiness? Xcode has a handy tool which lets you re-run tests repeatedly: - Right-click on the test diamond beside the test name. - Select Run "testName" repeatedly.... - Choose your maximum repetitions, configure some other properties, and off you go.

I like to run tests 1000 times to prove the fix for an unreliable test. Unit tests should run quite quickly, so this isn't time-consuming. And it's easy for randomness to slip through at any amount of tests, but 1000 feels like a good amount to give me confidence in my fix.