As you know, we've recently migrated JavaRanch from a variant of Universal Bulletin Board (Perl) to JForum (Java.) Naturally, I wanted to write an article talking about how we managed it.
Who are we
JavaRanch is a website run by volunteers throughout the world. A number of us worked on the forum migration project, including developers in Japan, India, Switzerland, Germany, England, and in the states of New York, Maryland, Texas, Colorado and California in the USA, Many other people made suggestions and tested.
How do we communicate
Mainly through the "Moderators Only" forum. In addition to plotting world domination, we've been quite busy in there talking about the new forum software. In fact, the Moderators Only forum has been on the new software since the end of November; we've been busy making sure the new software is ready for use. We also communicated some through e-mail for more targetted discussions amongst the core developers on the project. I find that being in different time zones is helpful. A surprising number of things get solved "while one is asleep."
Well that's all and well, but what happens when we need to communicate synchronously? For the two data migration/deployment weekends, we used an instant message session. This allowed the three people involved to do things simultaneously and manage dependencies.
Technologies in JForum
- Servlets
- Freemarker template language – While this serves the same purpose as a JSP (view) it uses a separate language that forces the Java code to stay out of the view. If we were writing software from scratch we probably would have used JSPs/EL since we had to go and learn Freemarker to use it.
- JDBC
- Java 5/J2EE 1.4 – This was an interesting one. We backported it to Java 1.4/J2EE 1.3 and then decided to go with the later runtime after all.
Tools we used
- Multiple development environments – A few of us ran the software locally on our own machines along with our own database. We also had a sandbox environment for integration and testing. And of course we have a production environment.
- Subversion (SVN) so we don't lose our work. It was down for a week and let me tell you that I really missed it! We did operate under the "if it isn't in source control, it doesn't exist" philosophy, which worked well. A bit of uncommitted work was lost early on, but we were able to have a stable production base.
- Ant build file to minimize number of steps in deployment and avoid the need to upload loose files. Very early on, we released inconsistent class files (rather than a jar) which broke the application in our sandbox. Needless to say we now use jars!
- JUnit – Some of the new classes/major functionality got thorough unit tests. The rest of the enhancements were manually tested. This was a strategy of managing the risk. Some of the new code was quite complex and really needed the tests. These tests found things many, many times. Whereas minor modifications could be manually tested without a huge effort. Not ideal, but a decent balance for "legacy software" - modified by volunteers.
- Password protected wiki pages to track internal requests along with what was done (and in what environment). This helped us prioritize and manage requests from other moderators.
Our long road to release
- If you've been around a while, you know that new forum software gets talked about regularly and nothing appears to happen. The reason is that migrating to new software is a lot or work! Our major milestones for this project were:
- January 2008 (yes, last year) – Our server is straining under load. Temporary solution is to buy a more recent server. Long term solution is to migrate to software we can actually change to manage problems.
- January 2008 – "This time is going to be different." We are going to pick a software package this time and go with it.
- January/February 2008 – JForum seems the closest to what we want. One of our sheriffs installed it on our sandbox and proved you can get up and running with JForum very quickly. Unfortunately, migrating our data (anywhere) and adding features we view as critical are a lot more work.
- February 2008 – Fundraiser to buy a new server since migrating to new software will take too long to fix the problem.
- February 2008 – Let's go with JForum. Initial commit to SVN so we can start making changes to it.
- February/March 2008 – JForum seems easy enough to extend. Let's go with it.
- March-October 2008 – A lot of work got done on extending JForum and the data migration. We stopped a few times for holidays, vacations and nice weather. However, we never lost momentum and that's the important thing.
- November 29 2008 (Thanksgiving weekend) – We cut over the private Moderators Only forum to JForum. This let us beta test by using the software "for real" amongst the moderators. It also represented the "point of no return" where we were really committing to the project going live no matter what. This was the first time we couldn't decide not to continue without losing data. We also used this day as a dry run for a full cutover.
- January 3, 2009 (weekend after New Year) – Public cutover. We made it.
Development process
The process evolved over time. Some roles:
- Project Manager – We realized we needed one before starting. Being a bunch of developers, this wasn't something anyone was overly enthusiastic about. We did get a volunteer project manager, although we didn't call it that.
- Deployer – We all deployed to our sandbox environment so everyone knew how and we didn't have a "bottleneck" person. We did have one person do the deployments to production and a team of three for the two cutover weekends.
- Developers – Of course!
- Tester – We solicited a rancher than we were thinking of asking to become a bartender to do some independent beta testing for us. He did a great job and is now a bartender – thanks Amit!
Legacy data
A side effect of working with volunteers is that the most interesting works gets done faster, either right after it is thought of or right before it is needed. Whereas the less interesting work -like migrating users and threads- dragged on for months. Granted, this was hard and a lot of work. There were more pauses in it than some other work though.
Summary
We did it! We did it! We did it! I'm proud and excited that a project of this size succeeded with only volunteers on it.