Small and Simple Web Applications - the Friki Way (Part 5)

Frank Carver, August 2003

Abstract

This article is the fifth of a series which laments the bloated and unmaintainable state of so many J2EE web applications and looks at ways to keep web applications small, simple and flexible. The series uses the author's Friki software as a case study, and discusses ways to design and build a powerful, flexible and usable web application in less space than a typical gif image.

This article carries on working through the user requirements and finally deploys a working web application, even though it doesn't have all the features of a real Wiki yet.

Introduction

If you've read the first, second, third and fourth articles, you should be aware that the aim of this project is to develop a small, simple and understandable "Wiki" (editable web site) application. We've considered and decided to defer the relatively heavy decision of how to store and retrieve pages by introducing a Java interface, decided on and implemented a simple "template" system to display pages, and are building a supporting test suite and an automated build script as we go along. Last session we created our first web application "war" file.

First, let's recap what our "customer" wants, and what we have got so far:

  1. each page may be viewed using it's own unique URL
  2. page content may contain links to other pages by name DONE
  3. links to nonexistent pages will be marked with a "?" DONE
  4. page content may be created and edited using just a browser
  5. at least 100 different pages can be stored

What do we write next?

We seem to be making serious progress; carrying on with this we now need to decide which of the remaining stories is the one to go for. I'm still not sure about story five, and story four seems to need story one to already be in place if we want it to make much sense. So story one is the one to go for. Note at this point how this process of re-evaluating our priorities at each point has led to a somewhat unexpected order of development. We've learned as we have gone along

Third task: each page may be viewed using it's own unique URL

Usually, we start by adding a new test. Up until now the tests have been a few lines of code run by JUnit, but if we want to test that we can fetch pages from a URL it's not quite as simple as that. Time to stop and think a little.

There is a really strong temptation about now to just press on with developing the "simple" servlet code we need to see something on a browser screen. It's a great feeling when you get to that stage, and the desire to run these last few steps is almost overwhelming. But fight that temptation. We need tests that cover all of our code, especially code that's hard to refactor like web pages and config files.

So, we are agreed that we need tests for the web interface. Luckily, because we haven't written the code yet, we are free to build an interface that's as easy to test as possible. How about we first test for the existence of a specific page URL, then we test that it has the correct contents. This is a new departure for our test cases, so I think a new test fixture class is in order:

AllTests.java

...

public class AllTests extends TestCase
{
    public static TestSuite suite()
    {
        TestSuite ret = new TestSuite();

        ret.addTest(new TestSuite(EmptyTest.class));
        ret.addTest(new TestSuite(TemplateTest.class));
        ret.addTest(new TestSuite(LinkTest.class));
        ret.addTest(new TestSuite(WebPageTest.class));

        return ret;
    }
}

That won't compile until we create the class so we'll start with a simple one:

WebPageTest.java

package tests;

import junit.framework.*;

public class WebPageTest extends TestCase
{
    public void testEmpty()
    {
    }
}

Compile and run this by typing "ant". Does it work? Sort of. We now have 12 passing tests, but the most recent one doesn't actually test anything. Still, it's a start.

Testing a Web Application

There are several tools for testing web applications, but the one I use most often is HTTPUnit. Strictly it's a bit more powerful than we need for just testing the existence of a web page, but I'll bend my own rule here because I know we'll need to test the content and navigation of pages later.

To use the HTTPUnit test tools to our project we need to add some "jar" files to a class path. To avoid version conflicts with other applications, I suggest you add the jar files to just this project for the moment. If you find HTTPUnit useful, you can always add them to a shared class path later.

  1. Download the latest HTTPUnit from sourceforge
  2. Unzip the downloaded archive to a temporary area somewhere
  3. Make a new directory in the "friki" project area next to "src" and "build", call it "lib"
  4. Copy the files httpunit/lib/httpunit.jar, httpunit/jars/nekohtml.jar and httpunit/jars/xercesImpl.jar to the new project "lib" directory.

To update our Ant build script to make these new facilities available to our unit tests, we need to add this new "lib" directory to the test class path declaration:

build.xml

...

  <path id="testclasspath">
    <pathelement location="build/delivery/classes"/>
    <pathelement location="build/tests/classes"/>
    <fileset dir="lib">
      <include name="*.jar"/>
    </fileset>
    <pathelement path="${java.class.path}"/>
  </path>

...

Now we just need to code our first real web application test case:

WebPageTest.java

package tests;

import junit.framework.*;
import com.meterware.httpunit.*;

public class WebPageTest extends TestCase
{
    WebConversation http;
    WebResponse response;

    public void setUp()
    {
        http = new WebConversation();
    }

    public void testApplicationPresent()
        throws Exception
    {
        response = http.getResponse(
            new GetMethodWebRequest("http://localhost:8080/frikidemo"));

        assertEquals("application root page should return a code of 200 (success)",
            200, response.getResponseCode());
    }
}

Note that this code is actually pretty simple. The "setUp" method creates a HTTPUnit "Conversation" class which manages all the HTTP requests and responses for us. Our initial test method creates a "GET" request for a particular URL and fetches the response. Then we are free to use regular JUnit assert methods to check all sorts of things about the response. In this case all we are looking for is a HTTP 200 status. code indicating that the page exists.

However, unless you have already built and deployed a web application called "frikidemo" to a server running on port 8080 of your local machine, then this test fails. If you look in the test log you should see something like:

    Testcase: testApplicationPresent took 2.125 sec
        Caused an ERROR
    Connection refused: connect
    java.net.ConnectException: Connection refused: connect
That's cool, though, we haven't built anything to pass the test yet.

The first stage in making a web application to pass our test is to get a server in place to run our shiny new application. As this is both a readily identifiable step in the process, and something it's easy to get wrong (or just plain forget) later, I suggest we add a specific test for it. That way, if/when something else fails we'll easily be able to tell if it is a problem with the server or with our application.

WebPageTest.java

...
    public void testServerPresent()
        throws Exception
    {
        response = http.getResponse(
            new GetMethodWebRequest("http://localhost:8080/"));

        assertEquals("server root page should return a code of 200 (success)",
            200, response.getResponseCode());
    }
...

This may not feel much like progress. We now have two failing tests instead of one!

    [junit] Tests run: 13, Failures: 0, Errors: 2, Time elapsed: 3.891 sec
The good point of this, though, is that we can address the first problem of getting the server going without forgetting what that is for. Those nagging test failure messages have become a kind of "to do" list.

Making it Work, Part 1: Installing a Local Web Server

Unfortunately, I don't really have the space here to cover installing a web server. Many of you will already be familiar with doing this, or have a local server running already. If this is all new, I recommend downloading and installing the Resin or Tomcat servers.

Come back here when "testServerPresent" passes!

Making it Work, Part 2: Deploying a Web Application

So now we have a server, but one test still fails. If you look at the test log now, you should see something like:

    Testcase: testApplicationPresent took 1.265 sec
        Caused an ERROR
    Error on HTTP request: 404 Not Found [http://localhost:8080/friki]
    com.meterware.httpunit.HttpNotFoundException: Error on HTTP request:
        404 Not Found [http://localhost:8080/friki]
This is a good sign. It shows that the server is running and has correctly responded with the correct HTTP error code (404) for an attempt to access an unknown page. Even better, it shows that HTTPUnit is working, and has correctly recognized the 404 as a "Not Found" error code.

To make this test work we need to "deploy" our web application to the server. You may recall that last session we built a "war" file for just this purpose. The simplest way to "deploy" a war file to a server on the same machine is just to copy the file to the correct location. Long ago we decided to automate all of this, so we need to add some code to our Ant build file to do this for us:

build.xml

...
  <target name="deploy-war" depends="build-war">
    <copy file='frikidemo.war' todir='c:/temp/webapps'/>
  </target>
...

Great. Works for me. I run ant test and all the tests pass.

But I bet it doesn't work for you. I happen to have configured my web server to use the directory c:\temp\webapps for its "war" files, but your server is almost guaranteed to be different. You may be running different server software, you may have configured it differently, you may be running on a system where c:\temp\webapps doesn't even make sense.

We could just require everyone to tinker with their Ant build file before deploying the application, but that seems very clumsy. Luckily Ant has a way round this. Ant can look in an external "properties" file for stuff like this, and we can then gather together anything that might vary between installations in one easily-editable place.

Add a "property file" declaration to the build file:

build.xml

<project name="friki" default="build" basedir=".">

  <property file="local.properties"/>

  <path id="classpath">
    <pathelement location="build/delivery/classes"/>
    <pathelement path="${java.class.path}"/>
  </path>
...

Update the deploy-war target to use a property instead of a literal directory name:

build.xml

...
  <target name="deploy-war" depends="build-war">
    <copy file='frikidemo.war' todir='${deploy.dir}'/>
  </target>
...

And finally, create a new file in the same directory as build.xml:

local.properties

deploy.dir=your real deployment "webapps" directory

If all went well, you should now be able to type

    ant deploy-war test
And all 13 tests should pass.

Looking at Building and Testing Again

You may have noticed something weird going on while you were following on with that last section. Although we have Ant build targets for each of our operations (clean, build, test, build-war, deploy-war etc.) they don't seem to work together very well. If you run ant clean before ant build-war it doesn't work. If you run ant build before ant deploy-war it tries to run some tests that need the application to be deployed. We've got in a tangle.

In a previous session we decided that our eventual goal was to be able to type just ant and have the build script do everything, in the right order. We need to sort it out. One problem is that we have added a bunch of new targets without relating them to each other. Another (and trickier) problem is that some of our unit tests just need our application code, but others need the application to have already been built and deployed.

We could just move all the testing until after the application is built and deployed, but this would really slow us down. Imagine if every time we make even a trivial change to one of our classes we had to completely build all the classes, package everything into a war file, deploy the war file to the server, and wait for the server to open and install the new application. It would be much easier not to bother with testing, wouldn't it?

A better solution is to split the unit tests into two groups. The first group are the tests which just need the code. These should run as soon as possible, and stop the build if they fail. There's no point building and deploying an application if we already know that some of the code is wrong. The second group of tests are the ones which require a running server with a deployed application.

Step One. Split the Unit Tests

Replace our existing AllTests.java with two new files:

LocalTests.java

package tests;

import junit.framework.*;

public class LocalTests extends TestCase
{
    public static TestSuite suite()
    {
        TestSuite ret = new TestSuite();

        ret.addTest(new TestSuite(EmptyTest.class));
        ret.addTest(new TestSuite(TemplateTest.class));
        ret.addTest(new TestSuite(LinkTest.class));

        return ret;
    }
}

RemoteTests.java

package tests;

import junit.framework.*;

public class RemoteTests extends TestCase
{
    public static TestSuite suite()
    {
        TestSuite ret = new TestSuite();

        ret.addTest(new TestSuite(WebPageTest.class));

        return ret;
    }
}

Step Two. Split the Test Build Scripts

Replace our existing "test" target with two new ones:

build.xml

...
  <target name="local-test">
    <junit fork="yes" haltonfailure="yes" printsummary="yes" dir="build/tests">
      <jvmarg value="-Djava.compiler=NONE"/>
      <test name="tests.LocalTests"/>
      <classpath refid="testclasspath"/>
      <formatter type="plain"/>
    </junit>
  </target>

  <target name="remote-test">
    <junit fork="yes" haltonfailure="yes" printsummary="yes" dir="build/tests">
      <jvmarg value="-Djava.compiler=NONE"/>
      <test name="tests.RemoteTests"/>
      <classpath refid="testclasspath"/>
      <formatter type="plain"/>
    </junit>
  </target>
...

Step Three. Reorganise the Target dependencies

We really only want two "public" targets. One (clean) to remove anything generated by the build process such as class files, war file and so on. One (build) to completely compile, package, deploy and test the application:

build.xml

...
  <target name="build" depends="compile,local-test,deploy-war,remote-test"/>
...

As a final test, type ant clean build and watch all the targets roll by successfully.

Back to the task: each page may be viewed using it's own unique URL

We now have a sweet setup to automatically build, deploy and test our application, so we really ought to get back to make some progress with the user goal that we are working on. As developers we know we've made huge progress, but until our customer can see something working, we might as well have stayed in bed.

As always, we will start with a test. But to write a test which fetches a page from a URL we really need to we need to decide what URL to fetch. Different Wiki software addresses this problem in different ways. Some pass the page name as an argument to the code like http://www.ugh.com/wiki/show?page=SomePage Some encode the page in the path like http://www.ugh.com/wiki/pages/SomePage?action=show Some have long URLs with session ids and user preferences and stuff. It doesn't seem to make a lot of difference, so for this I'll just choose the one that seems the simplest for the moment - passing the page name as an argument.

WebPageTest.java

...
    public void testExamplePage()
        throws Exception
    {
        response = http.getResponse(
            new GetMethodWebRequest("http://localhost:8080/frikidemo/show?page=ExamplePage"));

        assertEquals("example page should return a code of 200 (success)",
            200, response.getResponseCode());
    }
...

Sure enough, our test fails. We have no page to show:

    Testcase: testExamplePage took 0.031 sec
        Caused an ERROR
    Error on HTTP request: 404 Not Found [http://localhost:8080/frikidemo/show?page=ExamplePage]
    com.meterware.httpunit.HttpNotFoundException: Error on HTTP request:
        404 Not Found [http://localhost:8080/frikidemo/show?page=ExamplePage]

Note that we have changed the name of the test suite classes, so you will need to look in the file TEST-tests.RemoteTests.txt to see this error.

Let's build a quick servlet class which does nothing except tell us which page we requested:

ShowServlet.java

package friki;

import java.io.IOException;
import java.io.Writer;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class ShowServlet
    extends HttpServlet
{
    public void doGet(HttpServletRequest req, HttpServletResponse res)
        throws ServletException, IOException
    {
        String name = req.getParameter("page");
        Writer writer = res.getWriter();
        writer.write(
            "<html><head><title>Page " + name + "</title></head>" +
            "<body>This is Page '" + name + "'</body></html>");
        writer.flush();
    }
}

We also need to tell our web application to map this servlet to the "show" URL. For this we need to create a web application configuration file WEB-INF/web.xml. Create a new directory in friki/src called "files". Create a new directory inside that called "WEB-INF", and create the following file in that new directory:

web.xml

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE web-app
    PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
    "http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>
  <servlet>
    <servlet-name>show</servlet-name>
    <servlet-class>friki.ShowServlet</servlet-class>
  </servlet>

  <servlet-mapping>
    <servlet-name>show</servlet-name>
    <url-pattern>/show</url-pattern>
  </servlet-mapping>

</web-app>

However, we also need to include it in our war archive, which needs a bit more build code. Add the following lines at the start of the "build-war" target to copy anything from the "files" directory to where we make our war file from:

build.xml

...
  <target name="build-war">
    <mkdir dir='build/war/'/>
    <copy todir='build/war'>
      <fileset dir='src/files'/>
    </copy>
    <mkdir dir='build/war/WEB-INF/classes'/>
    <copy todir='build/war/WEB-INF/classes'>
      <fileset dir='build/delivery/classes'/>
    </copy>
    <jar jarfile='frikidemo.war'>
      <fileset dir="build/war"/>
    </jar>
  </target>
...

Now, finally, we have a web application which does something!. It should pass all the tests, and if you send your browser to http://localhost:8080/frikidemo/show?page=ugh you should see "This is Page 'ugh'".

If you like, you can think about what other unit tests might be appropriate at this point, but I reckon it's a good place to stop for the moment. I think we've met our next goal. Do you?

How are We Doing?

We still haven't made a Wiki yet! But we have just about completed another of our user goals, and have built, deployed and even run a web application that we have built from scratch. Best of all, we didn't abandon our commitment to testing everything (even things that seemed easy to write and hard to test). We can say with confidence that whatever we do next, it won't sneakily break what we have written so far.

Next session we will press on with the customer goals and see if we can get to something actually useful to give to our customer as a first real release of the software.