Build verification tests are sometimes called the rolling build or the nightly build. On a regular or a continuous basis, the build service compiles and tests the software that has been checked into the source tree. If a test fails—or worse, if the source doesn’t compile—then the service sends plaintive emails to everyone.
Source control helps team members avoid overwriting each other’s work, and lets a team of people work on a single body of software. As soon as you have installed TFS and created a team project, you can use Visual Studio to create a source tree and add code to it, and assign permissions to other users. But to make sure that the code does what is expected of it, you must set up regular builds. Typically, you will have more than one set up: a continuous (“rolling”) build that runs most of the unit tests; and a nightly build that runs more extensive tests, including performance and load tests, and automated systems tests (which we will discuss in the next chapter).
If you’re a test professional, you won’t need us to tell you this, but just to confirm the point: The only way to deliver quality software on time is to never let code be checked in without good test coverage; to run the tests all the time during development; and never to let bugs go unfixed. Anyone who has been around a while knows of projects where they let a moraine of untested code be pushed back towards the end of the project, and knows the pain that caused.
So these are the rules about making changes to the source:
- Check in all your code at least every week, and preferably more often. Plan your development tasks so that you can contribute a small but complete extension or improvement to the system at each check-in.
- Before checking in a change, use the Get Latest Version command (on the shortcut menu of your project in Solution Explorer) to integrate updates that have been made by other team members while you were working on your changes. Rebuild everything and run the unit tests.
- Use the Run All Impacted Tests command, which determines what tests are affected by changes that you have made or imported. (See Streamline Testing Process with Test Impact Analysis on MSDN.) Your changes can affect tests that you didn’t write, and changes made by others can affect tests you have written.
To use this command, you must initialize a baseline when you check out code. (See the MSDN topic How to: Identify the Test Impact of Code Changes During Development.)
- Switch on Code Coverage and check that at least 80% of your code has been exercised by the tests. If not, use the coloring feature to find code that has not been used. Write more tests.
- Do not check in your changes unless 80% coverage has been achieved and the tests all pass. Some instances of lower coverage are allowed. For example, where code is generated, it is sometimes reasonable to take coverage of one generated item as verification of another. But if you propose to make an exception, the proposal must be reviewed by a colleague with a skeptical personality.
To enforce this rule, create a testing check-in policy.
- If the rolling build breaks after you checked in code, you (and everyone else) will get a notification by email. If your changes might be the cause of the problem, undo your check-in. Before working on any other code, check in a fixed version. If it turns out that the fix will take some time, reopen the work item related to this change; you can no longer claim it is complete.
- Don’t go home immediately after checking in. You don’t want to come back in the morning to find everyone’s mailbox full of build failures.
This doesn’t apply if your team uses gated check-ins, where your code isn’t actually integrated into the main body of the source until the tests pass on an auxiliary build server.