As a contractor, I have now appeared at no less than six different companies beaming with pride that they have "unit tests". And, combined with that great pride is concern that these "unit tests" are turning out to be a hassle. "Other people talk about how great unit testing is, but this is really becoming painful. The tests take 45 minutes to run, and for every little change to the code, we break about seven tests!"
What these folks have is a massive pile of functional tests. They fell into the popular trap of thinking that because they run the tests with JUnit, they must be unit tests. 90% of their problems could have been resolved with just a wee bit of vocabulary.
Unit Test:
The smallest amount of testable code. Often a single method/function,
sans the use of other methods or classes. Fast! Thousands of unit tests can run in ten seconds or less!
A unit test NEVER uses:
Regression Suite:
A collection of tests that can be run all at once. An example would
be that all tests put in a certain directory would all be run by JUnit. A developer could run a unit test
regression suite 20 times per day. Or, they might run a functional test regression suite twice a month.
Functional Test:
Bigger than a unit and smaller than a full component test.
Usually exercising several methods/functions/classes working together. Permitted to take its sweet time:
hundreds of tests could take hours to run. Most functional tests are part of a functional test regression suite.
Usually run from JUnit.
Integration Test:
Testing two or more components working together.
Sometimes part of a regression suite.
Component Test:
Running one component by itself. Frequently done by QA, managers,
XP customers, etc. This sort of test is not part of a regression suite and it is not run by JUnit.
Component Acceptance Test (C.A.T.):
A component test run in front of a crowd of people
as part of a formal process. The people in the room collectively decide if the component has met required criteria.
System Test:
All components being run together.
System Acceptance Test (S.A.T.):
A system test run in front of a crowd of people
as part of a formal process. The people in the room collectively decide if the system has met required criteria.
Stress Tests:
Another program loads a component, some components, or possibly the entire
system. I have seen some small stress tests worked into some regression functional tests - a pretty smart way to test
some concurrency code!
Mock:
A brain dead piece of code used in unit and functional tests to make sure that
a piece of code you are attempting to test does not use some other bit of production code. A Mock class typically
overrides all of the public methods in a production class and is inserted somewhere that might try to use the
production class. Sometimes a Mock class implements an interface and replaces production code that implements the same
interface.
Shunt:
Kinda like a mock class that extends production code, only the purpose is NOT to
override ALL of the methods, but just enough so you can exercise some production methods, while mocking the rest of the
production methods. Especially useful if you want to test a class that might try to use I/O. Your shunt can override
the I/O methods while testing the non I/O methods.
Don't get me wrong. Functional tests have great value. I think apps that are well tested would have a regression suite of functional tests and a collection of non-regression functional tests. Usually, for every pound of production code, I would like to see about two pounds of unit tests and two ounces of functional tests (a little bit goes a long ways). The problem I see in too many shops is zero unit tests and a pound of functional tests.
The following two images demonstrate classes using classes that use classes. There are functional tests that exercise these classes working together. Fixing a bug in one class breaks many functional tests ....
I have seen this happen many times. In one case, a small change broke 47 tests. Meetings were held to decide if the bug should just be left in the code! In the end, it was decided to turn off all tests until the time could be set aside to fix all of the tests. Months passed. Things got mighty stinky ....
The solution is to use unit tests instead:
The result is that the project is more flexible.
"By writing only functional tests, I write less test code and exercise more production code!" True! But at the price of making your project brittle. Plus, there are some finer points of your application that will be a lot harder to test without using unit tests. The best coverage and flexibility can only be found through a mix of unit and functional testing that is heavy on unit testing and light on functional testing.
"My business logic is all of these classes working together, so testing just one method is pointless." I'm suggesting that you test all of the methods - just separately. Further, I am not suggesting that you have zero functional tests - they have their value.
"I don't mind if my unit test suite takes a few minutes to run." But do the other people on your team mind? Does your team lead mind? Your manager? If it take a few minutes instead of a few seconds, do you still run the full suite a dozen times a day? At what point do people stop running the tests at all?
"A unit test is anything run by JUnit." It is true that in our industry, the term "unit test" is subjective. I think that what I'm expressing here is the most popular interpretation of the term "unit test".
Here is a really obvious form of unit testing. Trying all sorts of wacky stuff on one method that doesn't depend on other methods.
public void testLongitude()
{
assertEquals( "-111.44" , Normalize.longitude( "111.44w" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44W" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44 w" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44 W" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44 w" ) );
assertEquals( "-111.44" , Normalize.longitude( "-111.44w" ) );
assertEquals( "-111.44" , Normalize.longitude( "-111.44W" ) );
assertEquals( "-111.44" , Normalize.longitude( "-111.44 w" ) );
assertEquals( "-111.44" , Normalize.longitude( "-111.44 W" ) );
assertEquals( "-111.44" , Normalize.longitude( "-111.44" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44-" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44 -" ) );
assertEquals( "-111.44" , Normalize.longitude( "111.44west" ) );
// ...
}
Sure, any putz can unit test that sort of thing. But most business logic uses other business logic:
public class FarmServlet extends ActionServlet
{
public void doAction( ServletData servletData ) throws Exception
{
String species = servletData.getParameter("species");
String buildingID = servletData.getParameter("buildingID");
if ( Str.usable( species ) && Str.usable( buildingID ) )
{
FarmEJBRemote remote = FarmEJBUtil.getHome().create();
remote.addAnimal( species , buildingID );
}
}
}
Not only is this calling other business logic, it's calling an application server! Possibly across a network! Thousands of these are gonna take more than ten seconds. Plus, changes in the EJB stuff could break my tests here! So a mock object needs to be introduced.
If I could just mock out all of the EJB stuff, I'd be sitting pretty. Hmmmm .... If the code were to somehow get my mock FarmEJBRemote, I'd be in fat city.
First, to create the mock. If FarmEJBRemote were a class, I would extend it and override all the methods. But since it happens to be an interface, I'll just make a fresh class and implement all the methods:
public class MockRemote implements FarmEJBRemote
{
String addAnimal_species = null;
String addAnimal_buildingID = null;
int addAnimal_calls = 0;
public void addAnimal( String species , String buildingID )
{
addAnimal_species = species ;
addAnimal_buildingID = buildingID ;
addAnimal_calls++;
}
}
The mock is dumb. Really dumb. It just carries data between my unit test and the code I am trying to exercise.
Does this class make you .... uncomfortable? It should. There are two things about this sort of class that bugged me the first time I was exposed to it: The class attributes are not private, and they have underscores in them. The first time I saw a mock object like this I was told "Your unit test code doesn't go into production, so it can cut a few corners." I dunno ... I want to only write first class code all the time! Not even an hour had passed and I needed to mock java.sql.Connection. 40 methods! Getters and setters for every parameter, return value and counter for every method? .... hmmmm .... thinking this through a little .... the reason we make the attributes private is for the sake of encapsulation - to hide how things are done on the inside so that we can change our business logic later without breaking a bunch of stuff that decided to tap into our innards. But that doesn't really apply to a mock, does it? By definition, a mock has zero business logic. Further, it doesn't really have anything that it didn't just copy from somebody else. All mock objects everywhere could easily be 100% generated at build time! ... So I sometimes still feel a little queasy about this, but in the end I always end up re-convincing myself that this is the best way. So it is still "first class code all the time" - it just smells a little off. But it smells better than if I did it the other way.
Now I need to get the code to take in my mock object instead of firing up some application server. So here's that snippet of code again and I've highlighted the line of code where I want to use my mock.
public class FarmServlet extends ActionServlet
{
public void doAction( ServletData servletData ) throws Exception
{
String species = servletData.getParameter("species");
String buildingID = servletData.getParameter("buildingID");
if ( Str.usable( species ) && Str.usable( buildingID ) )
{
FarmEJBRemote remote = FarmEJBUtil.getHome().create();
remote.addAnimal( species , buildingID );
}
}
}
First, let's separate that out a bit from the rest of the herd ...
public class FarmServlet extends ActionServlet
{
private FarmEJBRemote getRemote()
{
return FarmEJBUtil.getHome().create();
}
public void doAction( ServletData servletData ) throws Exception
{
String species = servletData.getParameter("species");
String buildingID = servletData.getParameter("buildingID");
if ( Str.usable( species ) && Str.usable( buildingID ) )
{
FarmEJBRemote remote = getRemote()
remote.addAnimal( species , buildingID );
}
}
}
This is gonna hurt a little .... I will now extend my production class and override getRemote() so I can foist my mock into this operation. So I'll need to make one little change ...
public class FarmServlet extends ActionServlet
{
FarmEJBRemote getRemote()
{
return FarmEJBUtil.getHome().create();
}
public void doAction( ServletData servletData ) throws Exception
{
String species = servletData.getParameter("species");
String buildingID = servletData.getParameter("buildingID");
if ( Str.usable( species ) && Str.usable( buildingID ) )
{
FarmEJBRemote remote = getRemote()
remote.addAnimal( species , buildingID );
}
}
}
If you are a good OO engineer, you should be hopping mad right about now! Oh sure, violating encapsulation in unit test code is mighty uncomfortable, but violating encapsulation in production code JUST AIN'T DONE! (in case you missed it, I took out the keyword "private", making the method "package private" - now, anything in the same package can see that method) Again, a long winded explanation might help smooth things over a bit. I'm going to save that for the forums and say for now: be ever vigilant about first class encapsulation in your production code ... but ... once in a while ... you might consider trading a dollar's worth of encapsulation for twenty dollars worth of testability. And to salve your pain/shame a bit, you might add a comment:
public class FarmServlet extends ActionServlet
{
//exposed for unit testing purposes only!
FarmEJBRemote getRemote()
{
return FarmEJBUtil.getHome().create();
}
public void doAction( ServletData servletData ) throws Exception
{
String species = servletData.getParameter("species");
String buildingID = servletData.getParameter("buildingID");
if ( Str.usable( species ) && Str.usable( buildingID ) )
{
FarmEJBRemote remote = getRemote()
remote.addAnimal( species , buildingID );
}
}
}
Now I just write the class so I can return the mock:
class FarmServletShunt extends FarmServlet
{
FarmEJBRemote getRemote_return = null;
FarmEJBRemote getRemote()
{
return getRemote_return;
}
}
Note the weird name: "Shunt". I'm not certain, but I think this word comes from electrical engineering/tinkering and refers to using a wire to temporarily complete a circuit. At first it sounded really stupid to me, but after a while I got used to it.
A shunt is kinda like a mock, except you don't override all of the methods. This way, you mock some methods while testing others. A unit test could end up with several shunts all overriding the same class, each testing different parts of the class. Shunts are usually inner classes.
Now for the grand finale! The actual unit test code!
public class TestFarmServlet extends TestCase
{
static class FarmServletShunt extends FarmServlet
{
FarmEJBRemote getRemote_return = null;
FarmEJBRemote getRemote()
{
return getRemote_return;
}
}
public void testAddAnimal() throws Exception
{
MockRemote mockRemote = new MockRemote();
FarmServletShunt shunt = new FarmServletShunt();
shunt.getRemote_return = mockRemote();
// just another mock to make
MockServletData mockServletData = new MockServletData();
mockServletData.getParameter_returns.put("species","dog");
mockServletData.getParameter_returns.put("buildingID","27");
shunt.doAction( mockServletData );
assertEquals( 1 , mockRemote.addAnimal_calls );
assertEquals( "dog" , mockRemote.addAnimal_species );
assertEquals( 27 , mockRemote.addAnimal_buildingID );
}
}
Now that the basic framework is in place, I just need to add lots of assertions.
TestFarmServlet vs. FarmServletTest: You gotta join one camp or the other. The folks from the latter camp make a damn good point: "FarmServletTest" sounds more like a noun and thus more OO. I am in the former camp. I have become addicted to my IDE and enjoy the way it can complete class names for me. When I get to the point that I have a rich suite of tests, and my test class names all end with "Test", then my IDE makes twice as many suggestions as I want it to. When my test class names all start with "Test", my IDE makes exactly the right number of suggestions.
This is the sort of directory structure I see all the time:
projectname
/src // production source
/java // java source. First directory in here is "com"
/db // database DDL (or .sql) files.
/test // test source
/unit // every class in this directory that starts with
// "Test" is run as a unit test
/functional // every class in this directory that starts with
// "Test" is run as a functional test
/nonregression // run stuff here manually during development
/lib // jar files and the like that the project uses
/production // stuff that will need to be copied into production
/development // stuff that is used only during development
// and should not find its way to production
/build // this dir does not go in version control.
// frequently deleted and re-created by ant
/gen-src // if your app generates any source, it goes in here
// (generated source never goes in VC!)
/classes // compiled production classes
/testclasses // compiled unit test code, functional test code,
// non-regression test code, mocks, shunts, etc.
/dist // the final artifacts destined for production.
// Usually a war file, ear file or jar file.
build.xml // ant build file
Folks put their unit test stuff in the same package as the production code they are testing - BUT! in a different directory structure. So if you are testing com.javaranch.Str.java found in src/java/com/javaranch/Str.java, you might have a test class com.javaranch.TestStr.java found in test/unit/com/javaranch/TestStr.java.
This is just a quick overview of this topic. But once you have your head wrapped around this, you're ready for