Unit test patterns, Episode III : Avoid the combinatorial – cheat!

This is the third post in a series on unit test patterns.

• Unit test patterns, Episode I : A foundation emerges
• Unit test patterns, Episode II : Inheritance strikes back

The previous two patterns have been my go-to guys for almost everything I have to do, and they work well to pressure me to better understand the problem domain and to decompose complex processes into simple ones. However, there have been a few occasions when I cheated a bit and intentionally didn’t specify the behavior for every situation within a process.

For example, in the last post I listed the possible situations for a process that invites a user given an email address and an account number…

• Given a valid email address and a valid account number
• Given an invalid email address and a valid account number
• Given a valid email address and an invalid account number
• Given an invalid email address and an invalid account number

This is only validating 2 items and yet I have 4 possible combinations. If I had a large number of items, well, I trust you see the exponential problem (specifically, I think this one is 2^n,  but that’s an oversimplification). Perhaps you also see the out: we don’t have to care about every situation. True, I want any validation routine to be user friendly and return all the errors, not just the first one caught. But simply returning a list of individual errors can provide that without needing a dedicated error for each combination of wrong-doings. I know, duh. But the point is that I have a collection of independent scenarios whose combined situations I don’t feel a need to specify behavior for.

In the example, this means that if the invite user process, given an invalid email address, returns a collection of errors containing one for “invalid email address,” I am trusting it will still give me that error if the process was also given an invalid account number. Yeah, you see the discomfort I have. The worst situation would be to pass validation with just the right combination of invalid datum, but to be fair, something should have to be messed up royally for that to happen. Theoretically, if I could write specs for each independent situation, I could then programmatically combine them for each possible combination – let the computer do the heavy lifting. Maybe I’ll do that some day, but 2^n can get big pretty fast (and I see that it’s sometimes more like 3^n).

I’m going to refactor the InviteUserProcess interface a bit from the last post. Instead of having each field accepted as a separate parameter, I’m combining them into one “InviteUserForm” object. I’ll be adding to this object later; otherwise, I probably wouldn’t bother with such a small set of data. More importantly, this will simplify the spec and the production code as well.

So instead of…

InviteUserProcess.Execute(String emailAddress, String accountNumber)

I’ll have…

InviteUserProcess.Execute(InviteUserForm form)

Also, the InviteUserProcess previously took two dependencies– an IValidateEmailAddressProcess and an IValidateAccountProcess– but now I’ll replace it with a single IValidateInviteUserFormProcess that returns a list of errors– an empty list meaning no errors, i.e., the input data is valid. That fast forwards me to looking at that new form validation process. Whereas the consumer of this process (InviteUserProcess) now has only two scenarios– a valid form or an invalid form– the validation process alone bares the full responsibility of dealing with all the different validation scenarios. To begin specifying the behavior of this process, I’ll define a “standard situation” where the form has all valid data…

namespace Validate_invite_user_form_process_specs
{
   public class ValidateFormProcessSpec
   {
      protected ValidateFormProcess ValidateFormProcess;
      protected InviteUserForm InviteUserForm;
      protected List<InviteUserError> ExpectedErrors;
      protected IEnumerable<InviteUserError> Errors;

      public virtual void SetupTestFixture()
      {
         InviteUserForm = new InviteUserForm
                              {
                                 EmailAddress = "test@example.com",
                                 AccountNumber = "1234567890"
                              };
         ValidateFormProcess = new ValidateFormProcess();
      }
   }
}

So I’m pulling out my inheritance pattern and setting up protected members (accessible from child classes), and instantiating an InviteUserForm that should pass validation. That form will serve as my “standard.” Notice that the SetupTestFixture() method is virtual (override-able).

Now I create a sub class for each “alternate situation” I want to spec. I’ll start by specifying that the all-valid situation returns an empty errors collection.

[TestFixture]
public class Given_a_valid_form : ValidateFormProcessSpec
{
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
      Errors = ValidateFormProcess.Execute( InviteUserForm );
   }

   [Test]
   public void The_errors_collection_should_be_empty()
   {
      CollectionAssert.IsEmpty( Errors );
   }
}

To create a spec for what should happen if the form contains an invalid email address, I tweak just that field…

[TestFixture]
public class Given_an_invalid_email_address : ValidateFormProcessSpec
{
   private List
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
      InviteUserForm.EmailAddress = "invalidEmail"; // tweaked the InviteUserForm for this particular situation: an invalid email address
      ExpectedErrors = new List<InviteUserError> { InviteUserError.InvalidEmailAddress };
      Errors = ValidateFormProcess.Execute( InviteUserForm );
   }

   [Test]
   public void The_errors_collection_should_contain_an_error_for_invalid_email()
   {
      CollectionAssert.AreEquivalent( ExpectedErrors, Errors ); // defined errors with an enum
   }
}

Because I know that the base form is fully valid, it’s very easy to setup a specification for just an invalid email. By using NUnit’s CollectionAssert.AreEquivalent, I can easily specify that no other errors should exist. I probably wouldn’t test for “Given_an_invalid_email_and_an_invalid_account_number”. I don’t like leaving situations unspec’ed, but as the number of input fields grow, you can see how this shortcut can save considerable time.

If I wanted to add an optional phone number to the InviteUserForm, an empty phone number is technically invalid, but it’s still acceptable. So I’d test for the invalid case like above, but I may also test for the no-phone-number case just to be sure that I don’t get that invalid error.

[TestFixture]
public class Given_no_phone_number : ValidateFormProcessSpec
{
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
   }

   [Test]
   [TestCase( "" )]
   [TestCase( null )]
   public void The_errors_collection_should_not_contain_an_error_for_invalid_phone_number( String phoneNumber )
   {
      InviteUserForm.PhoneNumber = phoneNumber;
      Errors = ValidateFormProcess.Execute( InviteUserForm );
      CollectionAssert.DoesNotContain( Errors, InviteUserError.InvalidPhoneNumber );
   }
}

That’s pretty much it. I wanted to explore how a process may have a set of data that combine to form too many scenarios to test under normal conditions. If this code sent people to the moon, I’d probably have a more comprehensive solution. But since I don’t, this works. I’ve used the validation requirements in this series’ example code, and I really have done validation this way, but I also do validation other ways I like better. Don’t use this as a template for all your validation needs.

To summarize this pattern:

• One base class to define a “standard” situation of a process
• Multiple sub-classes that alter the “standard” to fit individual, alternate situations and that specify behavior for those
• Not specifying behavior for every combination of alternate situations if they’re not related