Testing Plug-ins with Fragments

As Eclipse plug-in and Rich Client Platform developers, we face unique challenges in how we structure and execute our unit tests. In this article, I suggest an approach to unit testing based on Eclipse fragments that can help us overcome these challenges. If you find yourself frustrated with your current plug-in testing options, read on!

But before going into detail on a fragment based solution, let’s examine the current approaches and the pros and cons associated with each. The first approach is the one most of us start with as we learn the ropes of plug-in development: placing all code in a single plug-in.

Option 1: Place all code in a single plug-in

Placing all your code in a single plug-in is obviously the simplest approach. Usually, developers who go down this path choose to place their code in two different source directories: src and test.

In addition to being simple, this approach has the benefit of allowing all of your code to be loaded by the same classloader (each plug-in has its own classloader). This means that if you maintain the same package structure in your source and test folders, your test classes will have access to the non-public methods of the classes under test.

The downside to this approach is that it seriously complicates your build process. If you wish to run your tests as part of your build process (and who doesn’t), then you’ll need to include your test code in your deployable plug-ins. Your plug-ins will also need to declare dependencies on JUnit and any other test libraries you use (EasyMock, etc.). It’s possible to manually remove the classes and manifest entries after the build, but this is far from ideal.

Option 2: Place test code in a separate plug-in

To get around these build-related issues, many of us choose to place unit test code in a separate plug-in. Usually, each testable plug-in will have a corresponding test plug-in. This is the approach taken by most Eclipse projects, and it has the advantage of cleanly separating out test code and dependencies during the build process.

The disadvantage of this approach, however, is that your test classes will be loaded by a separate classloader and will therefore not have access to the non-public methods of the classes under test. In addition, your tests will not have access to classes in packages not exported by the plug-in under test.

Plug-ins that are properly designed will typically export only a small set of API packages, keeping most of the implementation packages hidden from the outside world. But it’s often useful to have access to these implementation packages during testing. A common workaround is to expose these packages as friends of the test plug-in. While this approach works, it pollutes your plug-in manifest with reference to test plug-ins. It also does nothing to solve the other issue of accessing non-public methods.

Option 3: Place test code in a fragment

Wouldn’t it be great if we could combine the classloading benefits of having a single plug-in with the clean separation of test code achieved with a separate plug-in? Well luckily for us, this is possible using Eclipse fragments.

A fragments looks much like a plug-in from the outside. It is represented as a separate project in your workspace, it contains a manifest that describes its contents, it is built and deployed as a jar. What makes a fragment different is that it contributes it’s resources at runtime to a single host plug-in. The classes in the fragment are therefore loaded by the host plug-in’s classloader.

By placing our unit tests in fragments, we can provide our tests access to the non-public methods of the classes under test. Also, because the source code and test code are actually part of the same plug-in, there are no issues related to non-exported packages. Test classes will have access to all packages in the plug-in, whether they are exported or not.

Workaround: Creating a master test suite

The main disadvantage to this fragment based approach is that it is difficult to aggregate unit tests into a master test suite. While it’s easy to create a test suite that includes the tests within a fragment, it’s not so easy to create a suite that includes the tests in multiple fragments.

The basic problem is that classes in fragments are not visible outside of the fragment itself. Any attempt to reference fragment classes in another plug-in will result in a compile error, so it’s not possible to assemble a master test suite in the standard way. But because the fragment classes are accessible via the host plug-in’s classloader at runtime, it is possible to create a master test suite using reflection. The code below shows how this can be done.

public class AllTests {

	public static Test suite() throws ClassNotFoundException {
		TestSuite suite = new TestSuite(
				"Master test suite.");

		suite.addTest(getTest("com.mycompany.myplugin1.AllTests"));
		suite.addTest(getTest("com.mycompany.myplugin2.AllTests"));
		return suite;
	}

	private static Test getTest(String suiteClassName) {
		try {
			Class clazz = Class.forName(suiteClassName);
			Method suiteMethod = clazz.getMethod("suite", new Class[0]);
			return (Test) suiteMethod.invoke(null, new Object[0]);
		} catch (Exception e) {
			throw new RuntimeException("Error", e);
		}
	}
}

Simply place this suite in a plug-in that declares dependencies on all testable plug-ins. You should then be able to run all of your tests with one click. While this code does add some complexity to the test suite, it is only necessary for the single master test suite class. The individual test suites in the fragments can be written normally.

UPDATE: For an alternative approach to running tests in fragments, see my post on Running Unit Tests for RCP and OSGi Applications.

UPDATE 01/12/2009: Another alternative is to add a Eclipse-ExtensibleAPI: true entry to the manifest of the plug-in being tested. This header makes your fragment classes available to other plug-ins during development, and eliminates the need for the reflection workaround above. Thanks to Martin Dilger for the suggestion.

Final thoughts

There are some other minor issues with a fragment based approach. One of the most annoying is that the JUnit view in Eclipse does not allow you to double-click on a test case to open the class. For some reason, though, clicking on a line in the Failure Trace section does work. Go figure.

Overall, however, I’ve found that the benefits of fragment based unit testing far outweigh the disadvantages. If you are satisfied with the more traditional approaches to unit testing plug-ins, then by all means keep doing what you’re doing. But if you find yourself frustrated with the limitations of these approaches, give fragments a try. You’ll be glad you did!