Connecting to Active Directory in Java: Still a Sorry State of Affairs

Three years ago, Kohsuke Kawaguchi, the man behind the Hudson/Jenkins continuous integration system, wrote two excellent blog posts (January 2008 and June 2008) on the absence of a good API for accessing Active Directory (AD) directly with Java. Kohsuke laid out the terrain, briefly described how he tackled the problem in the Hudson AD plugin and even gave advice on how an API could rectify the problem. I’m here to report that, after doing some research on the same topic recently, the situation has not improved. In fact, it’s a little worse.

In this post I’m just expounding on the research I did recently with trying to connect to AD from Java. The problem I was trying to solve was to simply authenticate a user against an AD domain and check their membership in a group.

LDAP vs. Talking to AD Directly

In Java, there really is only one widely available solution for communicating with AD: LDAP. Kohsuke wisely explains that Java cowers from connecting to proprietary systems such as AD in the name of portability. That’s generally a good thing, but it’s problematic if you want to solve a simple, constrained problem like mine. LDAP adds a thick layer of abstraction over a directory service that swallows up familiar AD terms like “groups”. Instead of being able to speak about a user’s group, with LDAP you have only generic attributes. Rather than just authenticating a user, LDAP forces a three step process that includes two, yes two, sets of user credentials.

The most cumbersome part of this process in Java is at the top of the trio of abstraction layers, the API for communicating with an LDAP compliant directory service. The Java Naming Directory Interface (JNDI) is the API built-in to Java SE for communicating with a directory service that speaks LDAP.

Current Solutions

Here’s where the story turns really grim. There are three options for working with AD in Java: the Java SE JNDI API, one of the open source LDAP APIs written in Java, or Kohsuke’s proposed solution. Presumably since Java SE has had a decent LDAP API for so long, there are very few other Java APIs out there. Of those that exist today, it seems that all of them are now defunct projects. Even worse, I could only find one API that works specifically with AD. Kohsuke describes his solution in his blog post, but it has problems as well.

JNDI API

In reality, using JNDI is not all that bad. And that is probably why there aren’t many other options. JNDI is very flexible, portable and can work with LDAP, which is ubiquitous. There’s also really good documentation on how to use it. The common critique of JNDI is that you have to write a lot of boiler-plate code. If you want an easy-to-use API for simple tasks, you’re stuck either writing your own or using one of the poor options below. As far as I could tell, people haven’t written much on JNDI since around 2004 or 2005, which means to me that people have resigned to using it as is.

Open Source LDAP APIs

Don’t get me wrong, the usual suspects in the Java Open Source space, Apache and Sun (now Oracle), both have APIs. However, they are in states that are not typical for these normally excellent contributors.

Oracle OpenDS

The best LDAP API for Java that I’ve found is Oracle’s OpenDS. OpenDS is actually an Open Source directory server project that was started by Sun, but includes a very nice SDK for working with LDAP. The SDK is robust and well written and this was the solution I ended up choosing. However, it appears that Oracle has pulled the plug on the project. The “downloads” link on the SDK homepage links to a page on the new java.net site here. As of 2/17/2011 is just a message stating that the page could not be found. A fellow WordPress blogger offers consensus that the OpenDS “Product has been discontinued. Oracle may offer commercial and supported products based on OpenDS in the future ”.  If you just need some decent Java code to connect to an AD server though, it shouldn’t be a big deal to just find a jar for the OpenDS SDK and use it.

Update (2/20/2011)

A company named ForgeRock has forked the OpenDS code and is offering similar services to what I assume Sun was previously offering. The new open source product is named OpenDJ, which ForgeRock claims will “meet Sun’s original product roadmap for OpenDS”. However, right now the offering is only the directory service server. I poked around on the site and couldn’t find an LDAP API download like OpenDS provided. The only download is for the entire server and doesn’t contain any LDAP API jars, as far as I could tell. I did find that ForgeRock has plans to release a stand-alone API soon though. Their roadmap on their wiki states that OpenDJ 2.5, which is scheduled for a Q1 2011 release, will have a “New LDAP Client SDK”.

Thanks to jhawk28 on Hacker News for the tip.

Apache LDAP API

Apache offers what is merely an “experimental” API. From a cursory look, it appears to be similar to the OpenDS SDK. The cool thing about Apache’s library is that they have a Java and a Groovy interface. This is something to watch for in the future, but I chose to stay clear of such a new offering, in case it goes defunct. The other down side is that there is scant documentation available on the API.

Spring LDAP (Added on 2/23/2011)

The venerable Spring framework provides an LDAP solution as well. I did research this one a bit in my search, but had to reject it without trying it out.  The main problem with Spring LDAP is that it has a lot of baggage. You have to have the entire core Spring framework in your classpath in order to use Spring LDAP. Since my application doesn’t use Spring, I didn’t want to add all that excess (and confuse other developers along the way, keep it up-to-date, etc…).

I can only speculate on another major problem. I’m guessing, that like the other API’s Spring LDAP has it’s own abstraction of the already abstract LDAP. So this doesn’t get you any closer to just working with AD. You have to try it out to see for yourself, but that’s my guess. Aside from that, I’m also guessing that it’s quite a nice API, since Spring tends to be so popular amongst enterprise Java development teams.

Novell JLDAP

Novell has a Java API that seems to have been popular once, although largely unusable now. The only working link I can find now is to a site that has jldap usage samples. The “Sample Home” and “Sample Information” links lead to a newer Novell developer site that doesn’t have anything about jldap there. I was able to find a link to the jldap CVS repository and download the source from there. You’ll see though that the code has not been touched in years. It’s also impossible to determine which branch has the most recent code. I tried to compile several branches, all with no luck. I searched around and found that you need a proprietary jar file that you get from some Novell product in order to compile it. I’d actually be surprised if the code would still compile, even if I could get the missing jar. It’s pretty clear that this project is not supported anymore and probably not worth the time to investigate.

All the Rest

There are some others listed on this page at the Apache LDAP API site. There’s some overlap with this list. To the best of my knowledge, the combination of this list and Apache’s is the entirety of Open Source LDAP API offerings in Java.

Proprietary APIs

I could only find a single active proprietary API for Java. Jespa, is a Java library that actually works directly with AD through the NTLM protocol, rather than using LDAP. I checked this library out and it’s very convenient. In fact, the API is just what I was looking for. I can call a simple authenticate method with a user name and password supplied and then check for group membership just as easily. The library seemed to do a good job of using AD specific terminology so it made the code read well. I found Jespa mentioned in several forums and postings on the net as the recommended way of solving my specific problem. In fact, there is even a Spring framework plugin for AD that is based on Jespa.

The problem with Jespa is it’s not free and it’s website is poorly designed. I work for a very profitable corporation, so the license fee is not a real problem. But, if they’re going to pay, they need to feel like they’re working with a professional company. Jespa’s website houses so little information about the company that produces it, Ioplex, that my company was not comfortable buying a license from them. There is a phone number listed, but no company information page and no street address. There’s also very little support information provided. I know that seems trite of us, but actually, it’s patently unprofessional to sell services without making potential customers feel safe in working with you.

Kohsuke’s Solution

This one is the germ of a quite sane way to talk to AD directly from Java. Basically, Kohsuke is using one of his own open source projects, com4j, to allow Java to talk to Windows native COM objects. This way you are interacting directly with AD. The main advantage, as Kohsuke points out several times, is that it can be made into a zero configuration solution. With LDAP you have to know and store an AD user/password that has enough authority to connect and read the directory just to be able to authenticate someone. With Kohsuke’s solution, you can just authenticate the person in one call (notice that Jespa does the same thing, although via NTLM).

Kohsuke’s solution isn’t packaged as an API, but it’s simple and described good enough in his blog entries (see above). Even better, he’s implemented a solution in the Hudson AD plugin, which is open source. I checked out the source and it was simple and easy to understand. The only problem here is the com4j project. It appears to be active on java.net, but there are no downloads available. So if you want a jar file for it you’ll have to dig it up. This link on java.net does provide the URL for the Subversion repository however, so you should be able to check it out and compile it into a jar.

Conclusion

If you’re looking for a simple, recent and well maintained Java solution to working intuitively with Active Directory, you’re out-of-luck. The only viable open soruce solution, OpenDS, has just been dropped by Oracle and the only decent licensed one, Jespa, is a little too shady and unknown to risk. Kohsuke railed three years ago about the bad situation in hopes that it would improve – it hasn’t.

I don’t claim that I did exhaustive research here, so if I’ve missed an important library or resource, please post it in the comments. If I have time, I will evaluate it and update this post.

Some Thoughts on Integrating SoapUI Functional Tests with Your Build

The development team I’m a part of at Ohio Mutual Insurance Group is about to develop our own web services for the first time and I want to make sure we test them properly. So, for about two months now I have been experimenting with using SoapUI to aid in developing our services.  One very important aspect of my experiment is to determine how difficult it is to run SoapUI functional tests from a Hudson CI job. In this blog are my rough thoughts on how well SoapUI functional tests integrate with the build process we use on Hudson.

Functional testing is not all SoapUI does, but it is one outstanding feature that I’m focusing on for now. Even though SoapUI has a solid interface for running functional tests, it doesn’t make any sense to have compilation, unit tests, and deployment automated but not functional tests.  It appears that the SoapUI developers feel the same way and have provided some help in this area. There are two ways to run SoapUI functional tests automatically: integrate with JUnit or use the SoapUI “testrunner” shell script.   Unfortunately, the official documentation on both is sparse (see the hyperlinks in the previous sentence)  and not helpful beyond a bare minimum of support.

Note: All mention of SoapUI in this entry is related to version 3.5.1.

Integrating with JUnit

The documentation for how to integrate with JUnit is awful.  It barely provides the necessary information for using JUnit and only offers part of the story.  If you really want to know how to run your SoapUI tests from JUnit you’re mostly stuck with using the SoapUI JavaDocs and your own experimentation (Intellisense is your friend here).  However, the one advantage that this approach has over using the testrunner script is that you can have more control over how and when tests are executed. If you need that control but want to automate the process of running the test cases, this is the way to do it. But, if you just need to automate the test case runs without control, the command line test runner is preferable. Here is some help with the JUnit method.

To run SoapUI tests from JUnit tests you have to use the SoapUI API.  This means that the SoapUI libraries need to be in your Classpath.  The main archive needed is the bin\soapui-3.5.1.jar file.  If they’re not already in your Classpath, you may also need some of the JARs in the lib directory.  At a minimum you will need the xmlpublic-2.4.0.jar because it contains the XMLException class that is thrown by some of the classes in the SoapUI API.

When you’ve got your Classpath setup you can use the SoapUI API in your JUnit tests.  There are several ways to run SoapUI functional tests with the SoapUI API.  You can just run all the test cases in all the test suites in a SoapUI project without control over the order (you choose whether to run them synchronously or asynchronously), run a whole test suite, or run individual test cases within a test suite.  For the second and third options, you can either blindly iterate over test suites and test cases or you can run test suites or test cases by name.  Which way you run the tests is based on how much control you need over the order of test execution.  When running a test suite you have control over whether the test cases in the suite are run synchronously or asynchronously.  The options that you have available reflect what you can do in SoapUI’s user interface.  The problem is digging through the JavaDocs to find out which methods on what objects to call.

Here is an example of a JUnit test that runs all the test cases in a SoapUI project in two different ways.  The lessControl method just runs every test case in the project in the simplest way whereas the fullControl method iterates over the test suites and the test cases in the project.

package com.omig.soapUISamples;

import java.util.List;
import org.junit.Test;
import com.eviware.soapui.impl.wsdl.WsdlProject;
import com.eviware.soapui.model.support.PropertiesMap;
import com.eviware.soapui.model.testsuite.TestCase;
import com.eviware.soapui.model.testsuite.TestRunner;
import com.eviware.soapui.model.testsuite.TestSuite;
import com.eviware.soapui.model.testsuite.TestRunner.Status;
import com.eviware.soapui.tools.SoapUITestCaseRunner;
import static org.junit.Assert.*;

public class SoapUITest {

@Test
public void lessControl() throws Exception {
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile(“SoapUIWorkspace/Sample-soapui-project.xml”);
runner.setPrintReport(true);  //Outputs a small table to stdout of test results.
runner.run();
}

@Test
public void fullControl() throws Exception {
WsdlProject project = new WsdlProject(“SoapUIWorkspace/Sample-soapui-project.xml”);
List<TestSuite> testSuites = project.getTestSuiteList();
for( TestSuite suite : testSuites ) {
List<TestCase> testCases = suite.getTestCaseList();
for( TestCase testCase : testCases ) {
System.out.println(“Running SoapUI test [" + testCase.getName() + "]“);
TestRunner runner2 = testCase.run(new PropertiesMap(), false);
assertEquals(Status.FINISHED, runner2.getStatus());
}
}
}
}

With the code above you’ve got a hint at how run SoapUI functional tests from a JUnit test.  To actually make this automatic you’ve got (at least) one more step.  In my situation we use Ant to script our software builds.  We then use Hudson to schedule the builds and make it all run automatically.  There are a lot of other ways you could do this but I’ll only show the Ant way here.  Any build tool that has support for JUnit will suffice.

Below is an example of an Ant task that runs the JUnit tests that are executing the SoapUI tests.  The JUnit output is being generated in XML format and sent to a folder named soapui-test-reports.

<path id="classpath">
    <!-- The path to the lib folder in the SoapUI installation.-->
    <fileset dir="${env.SOAPUI.LIB}"/>
    <!-- The path to the bin folder in the SoapUI installation.-->
    <fileset dir="${env.SOAPUI.BIN}"/>
</path>

<path id="soapui-workspace">
    <fileset dir="SoapUIWorkspace">
      <include name="**/*.xml"/>
    </fileset>
</path>

<target name="soapui-tests" depends="compile">
    <mkdir dir="../soapui-test-reports" />
    <junit printsummary="withOutAndErr"
           showoutput="true"
           haltonerror="on"
           haltonfailure="on"
           filtertrace="off"
           fork="no"
           forkmode="once">
      <classpath refid="classpath" />
      <classpath refid="soapui-workspace" />
      <formatter type="xml" />
      <batchtest todir="../soapui-test-reports">
        <fileset dir="tests">
                <include name="**/*SoapUI*.java"/>
        </fileset>
      </batchtest>
    </junit>
</target>

Testrunner Script

The documentation for the testrunner script is much clearer than for integrating with JUnit.  Integrating the testrunner.bat file with your build process is simple and straight-forward. An example of an Ant task to do this is shown below.  See SoapUI’s documentation for an explanation of the command line switches.

<target name="soapui-tests-cmdline" depends="compile">
        <exec executable="${env.SOAPUI.BIN}/testrunner.bat"
                  failonerror="yes"
                  failifexecutionfails="yes">
                <arg value="-rjaf"/>
                <arg path="../soapui-test-reports"/>
                <arg path="SoapUIWorkspace/Insured-Access-soapui-project.xml"/>
        </exec>
</target>

Which Method Is Best?

In my experience so far the command line testrunner tool is the better method for several reasons:

1. There’s less work because you do not have to write a JUnit test suite/case to run the SoapUI tests.
2. It creates a less fragile environment because writing a JUnit test to run the SoapUI tests makes it very difficult not to hard code the path to the SoapUI project file. You don’t have to worry about this with the command line because the input to script is parametrized.
3. The command line test runner can output “correct” JUnit output. The JUnit reports generated by the command line test runner split results by test suite then test case. This is ideal when the results are viewed graphically because you get a full count of actual test cases run and you can drill-down to the individual test cases to see its run status*.
4. To integrate with JUnit you have to use the SoapUI API, which is not documented well. There are Javadocs, but actual documentation is sparse in them.

* As far as I can tell it is difficult or impossible to do this by integrating with JUnit manually. To do so you would have to somehow generate one JUnit test for each test case in all the test suites. The SoapUI API allows you to easily iterate over the test suites and test cases, but there is no way in Java to dynamically generate methods for each test case.