Unit test patterns, Episode III : Avoid the combinatorial – cheat!

This is the third post in a series on unit test patterns.

The previous two patterns have been my go-to guys for almost everything I have to do, and they work well to pressure me to better understand the problem domain and to decompose complex processes into simple ones. However, there have been a few occasions when I cheated a bit and intentionally didn’t specify the behavior for every situation within a process.

For example, in the last post I listed the possible situations for a process that invites a user given an email address and an account number…

  • Given a valid email address and a valid account number
  • Given an invalid email address and a valid account number
  • Given a valid email address and an invalid account number
  • Given an invalid email address and an invalid account number

This is only validating 2 items and yet I have 4 possible combinations. If I had a large number of items, well, I trust you see the exponential problem (specifically, I think this one is 2^n,  but that’s an oversimplification). Perhaps you also see the out: we don’t have to care about every situation. True, I want any validation routine to be user friendly and return all the errors, not just the first one caught. But simply returning a list of individual errors can provide that without needing a dedicated error for each combination of wrong-doings. I know, duh. But the point is that I have a collection of independent scenarios whose combined situations I don’t feel a need to specify behavior for.

In the example, this means that if the invite user process, given an invalid email address, returns a collection of errors containing one for “invalid email address,” I am trusting it will still give me that error if the process was also given an invalid account number. Yeah, you see the discomfort I have. The worst situation would be to pass validation with just the right combination of invalid datum, but to be fair, something should have to be messed up royally for that to happen. Theoretically, if I could write specs for each independent situation, I could then programmatically combine them for each possible combination – let the computer do the heavy lifting. Maybe I’ll do that some day, but 2^n can get big pretty fast (and I see that it’s sometimes more like 3^n).

I’m going to refactor the InviteUserProcess interface a bit from the last post. Instead of having each field accepted as a separate parameter, I’m combining them into one “InviteUserForm” object. I’ll be adding to this object later; otherwise, I probably wouldn’t bother with such a small set of data. More importantly, this will simplify the spec and the production code as well.

So instead of…

InviteUserProcess.Execute(String emailAddress, String accountNumber)

I’ll have…

InviteUserProcess.Execute(InviteUserForm form)

Also, the InviteUserProcess previously took two dependencies– an IValidateEmailAddressProcess and an IValidateAccountProcess– but now I’ll replace it with a single IValidateInviteUserFormProcess that returns a list of errors– an empty list meaning no errors, i.e., the input data is valid. That fast forwards me to looking at that new form validation process. Whereas the consumer of this process (InviteUserProcess) now has only two scenarios– a valid form or an invalid form– the validation process alone bares the full responsibility of dealing with all the different validation scenarios. To begin specifying the behavior of this process, I’ll define a “standard situation” where the form has all valid data…

namespace Validate_invite_user_form_process_specs
{
   public class ValidateFormProcessSpec
   {
      protected ValidateFormProcess ValidateFormProcess;
      protected InviteUserForm InviteUserForm;
      protected List<InviteUserError> ExpectedErrors;
      protected IEnumerable<InviteUserError> Errors;

      public virtual void SetupTestFixture()
      {
         InviteUserForm = new InviteUserForm
                              {
                                 EmailAddress = "test@example.com",
                                 AccountNumber = "1234567890"
                              };
         ValidateFormProcess = new ValidateFormProcess();
      }
   }
}

So I’m pulling out my inheritance pattern and setting up protected members (accessible from child classes), and instantiating an InviteUserForm that should pass validation. That form will serve as my “standard.” Notice that the SetupTestFixture() method is virtual (override-able).

Now I create a sub class for each “alternate situation” I want to spec. I’ll start by specifying that the all-valid situation returns an empty errors collection.

[TestFixture]
public class Given_a_valid_form : ValidateFormProcessSpec
{
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
      Errors = ValidateFormProcess.Execute( InviteUserForm );
   }

   [Test]
   public void The_errors_collection_should_be_empty()
   {
      CollectionAssert.IsEmpty( Errors );
   }
}

To create a spec for what should happen if the form contains an invalid email address, I tweak just that field…

[TestFixture]
public class Given_an_invalid_email_address : ValidateFormProcessSpec
{
   private List
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
      InviteUserForm.EmailAddress = "invalidEmail"; // tweaked the InviteUserForm for this particular situation: an invalid email address
      ExpectedErrors = new List<InviteUserError> { InviteUserError.InvalidEmailAddress };
      Errors = ValidateFormProcess.Execute( InviteUserForm );
   }

   [Test]
   public void The_errors_collection_should_contain_an_error_for_invalid_email()
   {
      CollectionAssert.AreEquivalent( ExpectedErrors, Errors ); // defined errors with an enum
   }
}

Because I know that the base form is fully valid, it’s very easy to setup a specification for just an invalid email. By using NUnit’s CollectionAssert.AreEquivalent, I can easily specify that no other errors should exist. I probably wouldn’t test for “Given_an_invalid_email_and_an_invalid_account_number”. I don’t like leaving situations unspec’ed, but as the number of input fields grow, you can see how this shortcut can save considerable time.

If I wanted to add an optional phone number to the InviteUserForm, an empty phone number is technically invalid, but it’s still acceptable. So I’d test for the invalid case like above, but I may also test for the no-phone-number case just to be sure that I don’t get that invalid error.

[TestFixture]
public class Given_no_phone_number : ValidateFormProcessSpec
{
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
   }

   [Test]
   [TestCase( "" )]
   [TestCase( null )]
   public void The_errors_collection_should_not_contain_an_error_for_invalid_phone_number( String phoneNumber )
   {
      InviteUserForm.PhoneNumber = phoneNumber;
      Errors = ValidateFormProcess.Execute( InviteUserForm );
      CollectionAssert.DoesNotContain( Errors, InviteUserError.InvalidPhoneNumber );
   }
}

That’s pretty much it. I wanted to explore how a process may have a set of data that combine to form too many scenarios to test under normal conditions. If this code sent people to the moon, I’d probably have a more comprehensive solution. But since I don’t, this works. I’ve used the validation requirements in this series’ example code, and I really have done validation this way, but I also do validation other ways I like better. Don’t use this as a template for all your validation needs.

To summarize this pattern:

  • One base class to define a “standard” situation of a process
  • Multiple sub-classes that alter the “standard” to fit individual, alternate situations and that specify behavior for those
  • Not specifying behavior for every combination of alternate situations if they’re not related

Unit test formats, Episode II : Inheritance strikes back

This is the second post in a series on unit test patterns.

Referring to the base pattern from the previous post, I broke a test or specification down into three parts:

  • a process
  • a situation
  • and expectations

In this post, I expand the base pattern to specify behavior for different situations (the second bullet point) of the same process. There are many ways to specify behavior for different situations, including using frameworks like NSpec or SpecFlow that have more complex syntax. I don’t mind saying that I’m not there yet, but I’m eying them. So instead, I’ll do this with NUnit and…inheritance…

An aside: inheritance has had an interesting rise and fall in popularity. And I agree with the “market’s correction.” It seems that there was a point where OO was about objects and inheritance and little else. And when all us developers jumped out of the gate with nothing but “IS A” in our heads, well, when all you’ve got is a hammer… Anyway, good or bad, I’m using inheritance and so far it’s helping, not hurting.

Returning to the UserInvitationManager class of the last post, I now forget what that class does, exactly. What about user invitations does it manage? Blast my absent-mindedness! Well, I can look at the public methods and get a much better idea– InviteUser()– ah, ok; but I don’t like that I had to do that. Not at all. I want to rename that guy to something just in-my-face obvious: InviteUserProcess. Of course it’s interfaced, so there’s also an IInviteUserProcess now. You could also call the interface just IInviteUser. And I’ll just replace the redundancy of InviteUserProcess.InviteUser() with InviteUserProcess.Execute() or .Run(), .Invoke(), etc.

Now, InviteUserProcess is a very specific name. I’ve kind of locked myself in here. I’m limited to pretty much that one function and I don’t have the freedom to put a lot of other user invitation-related methods on it like I did with the UserInvitationManager…GREAT!!! In fact, I like that so well that I’ll go ahead and update the names of the dependencies as well, e.g., from IUserInvitationSender to ISendUserInvitationProcess.

Anyway, this renames the specification file and its namespace, but I think I can also eliminate that When_ namespace since the new name defines the “when”. Here’s the last post’s specification code with these changes…

namespace UserManagerSpecs
namespace InviteUserProcessSpecs
{
 namespace When_inviting_a_user_to_join_an_account
 {
      [TestFixture]
      public class Given_an_email_address_and_an_account_number
      {
         [Test]
 public void Then_the_user_invitation_manager_should_pre_register_that_email_address_to_the_account()
         public void This_process_should_preregister_that_email_address_to_the_account()
         {

That defines only one situation: given an email address and an account number. The different situations that are the focus of this post are going to arise out of an increased awareness that just any ol’ string won’t do for either the email address or the account number. We need to validate those two strings. Now, if there’s anything I hate doing, it’s validation, but there’s no sense crying about it; it has to be done. So here are the situations…

  • Given a valid email address and a valid account number
  • Given an invalid email address and a valid account number
  • Given a valid email address and an invalid account number
  • Given an invalid email address and an invalid account number

In the first situation, my expectations/desired behavior is super simple– I want the process to 1) preregister that email address and 2) send the invitation. Oh, in the last version, we assumed the values were fine so we didn’t have anything to return. The real world is a messy place, ain’t it? Anway, this adds one more expectation: 3) let the consumer of this process know that it worked. On the flip side, if either parameters are invalid, the consumer of this process is going to want to know something went wrong and what that was, and I certainly don’t want to preregister the email address or send the invitation in such a case. So, here I list these expected behaviors…

  • Given a valid email address and a valid account number
    – The process should preregister the email address with the account
    – The process should send an invitation to the email address
    – The process should return a success notice
  • Given an invalid email address and a valid account number
    – The process should not preregister anything
    – The process should not send an invitation
    – The process should return an invalid email notice
  • Given a valid email address and an invalid account number
    – The process should not preregister anything
    – The process should not send an invitation
    – The process should return an invalid account notice
  • Given an invalid email address and an invalid account number
    – The process should not preregister anything
    – The process should not send an invitation
    – The process should return an invalid email and account notice

So I haven’t written any code yet, but I’ve got an excellent idea of the problem domain I’m up against and how the system should behave. Writing these specifications for the behavior has drawn all that out.

The InviteUserProcess has gained the new responsibilities of knowing how to recognize these two parameters as valid. I could load up this process with some regex’s or something, but that would be harder to test. Because it will be easier to test, I’ll imagine there’s an IValidateEmailAddressProcess and an IValidateAccountProcess, and then stub them. That’s actually quite a bit easier. My specifications will not have to simultaneously deal with defining expectations for what to do because of an invalid email address and what makes for an invalid email address. Or what makes for an invalid account number. I’m dropping that extra responsibility (think the ‘S’ in SOLID). Note: “Because it will be easier to test, I’ll…” is the effect of the driving nature of TDD/BDD, and here I see how that drive pushes me toward better code. Remember, TDD is not just about having an automated test, it’s also about improving how you code (and other things, too).

Now that I know my situations and expected behaviors, I’ll break out my inheritance pattern for building the spec classes. The inheritance pattern creates a base class which defines all the common parties to greatly reduce redundant setup code for the derived classes. I’ll also bother giving this base class a name that will satisfy the “IS A” criteria, “InviteUserProcessSpec,” though that may be a bit superfluous.

public class InviteUserProcessSpec
{
   // System under test
   protected InviteUserProcess inviteUserProcess;

   // Dependencies
   protected IValidateEmailAddressProcess validateEmailProcess = MockRepository.GenerateMock<IValidateEmailAddressProcess>();
   protected IValidateAccountProcess validateAcctProcess = MockRepository.GenerateMock<IValidateAccountProcess>();
   protected IPreregisterEmailProcess preregisterEmailProcess = MockRepository.GenerateMock<IPreregisterEmailProcess>();
   protected ISendUserInvitationProcess sendInvitationProcess = MockRepository.GenerateMock<ISendUserInvitationProcess>();

   // Parameters
   protected String emailAddress = "doesntMatter";
   protected String accountNumber = "doesntMatterEither";

   // Return value
   protected InviteUserResponse response;

   public virtual void SetupTestFixture()
   {
       inviteUserProcess = new InviteUserProcess( validateEmailProcess, validateAcctProcess, preregisterEmailProcess, sendInvitationProcess );
   }
}

You can see that, as I mentioned in the previous post, I’ve still got four kinds of players: the system under test, the dependencies, the parameters, and the return value. You may also notice that in this class I’ve introduced a return value: InviteUserResponse. This is just an enum defined with the different response cases (e.g. Successful, InvalidEmailAddress, etc.). It could have been as simple as a string with a message or more complex like a full blown object.

And finally, a specification for a given situation, which “is a” InviteUserProcessSpec, looks like this. Notice that I setup for the defined situation in the TestFixtureSetup. It’s here where I actually make the “Given” true.

[TestFixture]
public class Given_a_valid_email_address_and_a_valid_account_number : InviteUserProcessSpec
{
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
      validateEmailProcess.Stub( validateProc => validateProc.Execute( emailAddress ) ).Return( true ); // Given a valid email
      validateAcctProcess.Stub( validateProc => validateProc.Execute( accountNumber ) ).Return( true ); // and a valid account
      response = inviteUserProcess.Execute( emailAddress, accountNumber ); // For this spec, I can act here once for all asserts
   }

   [Test]
   public void The_process_should_pre_register_the_email_address_with_the_account()
   {
      preregisterEmailProcess.AssertWasCalled( registerProc => registerProc.Execute( emailAddress, accountNumber ) );
   }

   [Test]
   public void The_process_should_send_an_invitation_to_the_email_address()
   {
      sendInvitationProcess.AssertWasCalled( sendProc => sendProc.Execute( emailAddress ) );
   }

   [Test]
   public void The_process_should_return_successful()
   {
      Assert.AreEqual( InviteUserResponse.Successful, response ); // checking the returned enum value
   }
}

So this should look very similar to the bullet points from above. Notice how using the base class to hold the basics really cleans up the spec. I wouldn’t do this for just one situation, but we’ve got a handful of situations here. I’ll do one more situation for illustration.

[TestFixture]
public class Given_an_invalid_email_address_and_a_valid_account_number : InviteUserProcessSpec
{
   [TestFixtureSetUp]
   public override void SetupTestFixture()
   {
      base.SetupTestFixture();
      validateEmailProcess.Stub( validateProc => validateProc.Execute( emailAddress ) ).Return( false ); // Given an INvalid email
      validateAcctProcess.Stub( validateProc => validateProc.Execute( accountNumber ) ).Return( true ); // and a valid account
      response = inviteUserProcess.Execute( emailAddress, accountNumber );
   }

   [Test]
   public void The_process_should_not_pre_register_anything()
   {
      preregisterEmailProcess.AssertWasNotCalled( registerProc => registerProc.Execute( Arg<String>.Is.Anything, Arg<String>.Is.Anything ) );
   }

   [Test]
   public void The_process_should_not_send_an_invitation()
   {
      sendInvitationProcess.AssertWasNotCalled( sendProc => sendProc.Execute( Arg<String>.Is.Anything ) );
   }

   [Test]
   public void The_process_should_return_invalid_email()
   {
      Assert.AreEqual( InviteUserResponse.InvalidEmail, response );
   }
}

You can see that for the negative assertions (the process should not…) I’m using NUnit’s Arg<> class. Since this post is about the inheritance pattern and not NUnit syntax, I’ll just say that this guy can be very powerful.

So there it is, multiple situations tamed with inheritance.  And as far as that pattern goes, my big take away is noticing the leanness of the spec classes, including how the individual [Test] methods are just one assertion line. When I can easily get away with that, I do it. Sometimes I can’t, so I “act” within the [Test] method. When I see that the spec classes are simple, then I feel like the system code is getting well written, meaning that it’s understandable, it’s maintainable, it’s delectable! Or so I think today. “Therefore let him who thinks he stands take heed lest he fall.” So I’ll see what tomorrow holds. 🙂

Unit test patterns, Episode I : A foundation emerges

Learning TDD has been a lot of work. I’ll start by saying that. If you’re new to TDD, you’re probably floundering around a bit. I did. A lot. I think the main, underlying issue is to “see the layers,” specifically just the one layer you’re writing a test for. Before TDD, I saw an application as more of a big unit with a few large layers. This really causes a struggle when writing a test. TDD really pressures you (drives you ;-)) to see the small, incremental steps. It’s seeing those little steps that’s the trick. If you bite off too much of an application to unit test because you’re a power coder that “get’s things done,” then writing that test becomes like trying to put your shoes on without untying them. It will frustrate you to no end. I’ve been there and done that.  But don’t get rid of the laces. Just untie the shoes. This is what kept me from giving up on TDD– something about it just felt right and smarter folks than me were doing it. Now, I’m no lemming, blindly following all the latest fads, but, well, I’m no lemming blindly following the status quo either. And to be fair, TDD is far too old to be a fad.

And I’m certainly still learning– I’m young to TDD– just barely over a year old now. I had tried it a few times before and couldn’t quite get off the ground, so I took a weekend at home and locked myself in a room until I wrote some successful tests. That’s when I finally got started and I’ve used it on my most recent project at work: a new website (ASP.Net MVC). Perhaps someday I’ll write about how to start a project with TDD, but for now I just want to share a few testing patterns I’m currently using. Feel free to critique; I’m on a journey– “Give instruction to a wise man, and he will be still wiser; Teach a just man, and he will increase in learning.”

So, about those patterns. This post will look at the basic one which is all about

  • a process
  • a situation
  • and expectations

This fits well with Arrange, Act, and Assert for the mechanics of the tests/specs, and all the formats share this overarching design or structure which ends up looking like this

namespace Process
{
   public class Situation
   {
      public void Expectation()
      {}
   }
}

I like the BDD naming convention and totally favor a perspective of “specifying behavior” rather than “passing tests.” In the past, I’ve used “When” to declare the process and “Given” to declare the situation. Because of this, I typically reorder the classic “Given-When-Then” naming pattern to “When-Given-Then.”

I’ll take a recent real world (and common) situation for an example: a web site where a user who has an account may invite others to join that account. I’ll call this a UserInvitationManager (for this post).

I feel using the “When” first to declare the activity/process establishes the context quickly. So…

namespace When_inviting_a_user_to_join_an_account
{
}

The situation is limited to just the input parameters in this case, so…

namespace When_inviting_a_user_to_join_an_account
{
   [TestFixture]
   public class Given_an_email_address_and_an_account_number
   {
   }
}

Before I move on to the “Then,” I set up for those Thens. This can be most of the work, actually, as you’ll see. Most of the time, I put the Act(ing) into the TestFixtureSetup method rather than inside the Test method. I do this so that each TestFixture specifies the behavior for one situation and has the chance to easily Assert many things after the single Act(ion). By no means am I married to this. If a situation serves me better to re-Act before every Assert(ion), I’ll do it in an instant. But otherwise, I’ll Act once, which keeps the Assert(ion)s very simple.

So to set up, I first need to define and initialize the actors involved, including the system under test, or in other words, the process for which I’m specifying behavior. And like I said, I’m skipping some of the “TDD-how-to,” so I won’t go into how I came up with the dependencies for the UserInvitationManager. But I will go ahead and Act at the end of the setup.

namespace When_inviting_a_user_to_join_an_account
{
   [TestFixture]
   public class Given_an_email_address_and_an_account_number
   {
      UserInvitationManager invitationMgr;
      IUserInvitationSender invitationSender;
      IAccountManager accountMgr;

      String emailAddress;
      String accountNumber;

      [TestFixtureSetup]
      public void SetupTestFixture()
      {
         // Arrange
         invitationSender = MockRepository.GenerateMock<IUserInvitationSender>();
         accountMgr = MockRepository.GenerateMock<IAccountManager>();
         invitationMgr = new UserInvitationManager(invitationSender, accountMgr);
         emailAddress = "newUserEmail@doesntmatter.stilldoesntmatter";
         accountNumber = "doesntMatterEither";

         //Act
         invitationMgr.InviteUser(emailAddress, accountNumber);
      }
   }
}

So, a lot of code, relatively speaking, just got added. For me, arranging is defining your parties and intializing them.

I generally define four groups of parties:

  • system under test (the UserInvitationManager)
  • dependencies (the invitationSender and accountMgr)
  • parameters (the emailAddress and accountNumber)
  • result (this action returned no result, but if it did, I would have set the result variable during the Act)

Then I initialize them.

To keep things separated in the above example, I initialized them in the setup method, but most times I initialize what I can on definition to reduce unnecessary loc.

IAccountManager accountMgr = MockRepository.GenerateMock();

Purely a preference. Perhaps a TDD puritan would refuse, but once I got used to this format, I didn’t need to always separate the definition and intialization, and I found that it cleans up some stuff for other setup efforts, like stubbing dependencies.

Now we get to the “Thens.” And, thanks to Ayende and RhinoMocks, they’re short and sweet, as, I think, they should be. Also notice one Assert per Test. Yeah, I’m generally one of those.

namespace When_inviting_a_user_to_join_an_account
{
   [TestFixture]
   public class Given_an_email_address_and_an_account_number
   {
      UserInvitationManager invitationMgr;
      IUserInvitationSender invitationSender;
      IAccountManager accountMgr;

      String emailAddress;
      String accountNumber;

      [TestFixtureSetup]
      public void SetupTestFixture()
      {
         // Arrange
         invitationSender = MockRepository.GenerateMock<IUserInvitationSender>();
         accountMgr = MockRepository.GenerateMock<IAccountManager>();
         invitationMgr = new UserInvitationManager(invitationSender, accountMgr);
         emailAddress = "newUserEmail@doesntmatter.stilldoesntmatter";
         accountNumber = "doesntMatterEither";

         //Act
         invitationMgr.InviteUser(emailAddress, accountNumber);
      }

      [Test]
      public void Then_the_user_invitation_manager_should_pre_register_that_email_address_to_the_account()
      {
         accountMgr.AssertWasCalled(acctMgr => acctMgr.PreRegisterByEmailAddress(accountNumber, emailAddress));
      }

      [Test]
      public void Then_the_user_invitation_manager_should_send_an_invitation_to_the_given_email_address()
      {
         invitationSender.AssertWasCalled(sender => sender.SendInvitation(emailAddress));
      }
   }
}

Pretty basic, but again:

  • One [TestFixture] per situation as generally defined (“Given”) by parameters. Sometimes, the situation is additionally defined by a result from calling a dependency.
  • Usually one Act(ion) per situation. Sometimes this is not possible or preferable, and when it’s not, I don’t cry about it.
  • One Assert(ion) per [Test] method.

That’s the foundational pattern that has emerged for me and it’s good for many of my specifications. In a future post, I’ll actually change up how I name some of these elements, like the Whens and the Thens and even the system under test. See the previous post for a hint. But upon this basic structure I can easily define more complex situations. So, it’s been a little over a year for TDD/BDD and me, and this is where I am…headed.

Now read Unit test patterns, Episode II : Inheritance strikes back.