soapUI Tip: Options for Refreshing a WSDL Definition

Here’s a quick tip for soapUI users that I have been stumbling over recently.  soapUI allows you to refresh an already defined service definition from an updated WSDL file.  But, it’s easy to blow away the data that’s in your SOAP test steps if you chose the wrong options.

First, here’s the specific task I’m talking about.  You have an existing soapUI workspace that has interface definitions already imported and you have at least one test suite using that interface.  Now you’ve made a change to the WSDL and you need to refresh your interface definition in soapUI and update the SOAP test steps in your test suite.  Clearly, you want to update the SOAP requests to match the new WSDL without losing the test data that already exists in the test steps.  You reload a WSDL definition by right-clicking on an interface definition in soapUI as shown below.

Here’s a rundown of the effect of some of the “Update Definition” parameter configurations.  I haven’t gone through every permutation of the parameters, just the ones that seem most useful.  I’ve portrayed the configurations in terms of actions you’re likely to want to take after changing your WSDL.

• You added a new operation to the WSDL and you want to add default requests for it.

• You made a change to an existing operation in your WSDL and you want to regenerate existing requests using the new schema without creating optional elements.  This configuration will cause you to lose the data in your existing requests.

• You made a change to an existing operation in your WSDL and you want to regenerate existing requests using the new schema and create optional elements.  This configuration will cause you to lose the data in your existing requests, unless the elements are optional.

• You made a change to an existing operation in your WSDL and you want to regenerate existing requests using the new schema and create optional elements.  This configuration will keep the existing data in tact.

• You made a change to an existing operation in your WSDL and you want to regenerate existing requests and test steps using the new schema without creating optional elements.  This configuration will keep the existing data in tact but will remove optional elements from the existing requests.

• You made a change to an existing operation in your WSDL and you want to regenerate existing requests and test steps using the new schema and maintain optional elements.  This configuration will keep the existing data and optional elements in tact.

• The last configuration is the safest bet for most refreshes.  It combines all of the above. That means that it picks up all WSDL changes and incorporates them into both default requests and your SOAP test steps.  None of your existing data will be overwritten.  It also opens up a window that lists exactly which items have been modified.

I have not mentioned the “Keep SOAP Headers” and “Create Backups” parameters because I have not used them yet.

I suggest playing around with the “Update Definition” feature a little before using it.  soapUI is a powerful tool, so it assumes you know what you want.  It accomplishes that by exposing panels with lots of parameters on them, like the “Update Definiton” panel.  If you get yourself into a bind, remember that you can just reload the project you’re working on by right-clicking on the project name and choosing “Reload Project”.  This option will reload the project into the workspace without saving the changes you just made.

Some Thoughts on Integrating SoapUI Functional Tests with Your Build

The development team I’m a part of at Ohio Mutual Insurance Group is about to develop our own web services for the first time and I want to make sure we test them properly. So, for about two months now I have been experimenting with using SoapUI to aid in developing our services.  One very important aspect of my experiment is to determine how difficult it is to run SoapUI functional tests from a Hudson CI job. In this blog are my rough thoughts on how well SoapUI functional tests integrate with the build process we use on Hudson.

Functional testing is not all SoapUI does, but it is one outstanding feature that I’m focusing on for now. Even though SoapUI has a solid interface for running functional tests, it doesn’t make any sense to have compilation, unit tests, and deployment automated but not functional tests.  It appears that the SoapUI developers feel the same way and have provided some help in this area. There are two ways to run SoapUI functional tests automatically: integrate with JUnit or use the SoapUI “testrunner” shell script.   Unfortunately, the official documentation on both is sparse (see the hyperlinks in the previous sentence)  and not helpful beyond a bare minimum of support.

Note: All mention of SoapUI in this entry is related to version 3.5.1.

Integrating with JUnit

The documentation for how to integrate with JUnit is awful.  It barely provides the necessary information for using JUnit and only offers part of the story.  If you really want to know how to run your SoapUI tests from JUnit you’re mostly stuck with using the SoapUI JavaDocs and your own experimentation (Intellisense is your friend here).  However, the one advantage that this approach has over using the testrunner script is that you can have more control over how and when tests are executed. If you need that control but want to automate the process of running the test cases, this is the way to do it. But, if you just need to automate the test case runs without control, the command line test runner is preferable. Here is some help with the JUnit method.

To run SoapUI tests from JUnit tests you have to use the SoapUI API.  This means that the SoapUI libraries need to be in your Classpath.  The main archive needed is the bin\soapui-3.5.1.jar file.  If they’re not already in your Classpath, you may also need some of the JARs in the lib directory.  At a minimum you will need the xmlpublic-2.4.0.jar because it contains the XMLException class that is thrown by some of the classes in the SoapUI API.

When you’ve got your Classpath setup you can use the SoapUI API in your JUnit tests.  There are several ways to run SoapUI functional tests with the SoapUI API.  You can just run all the test cases in all the test suites in a SoapUI project without control over the order (you choose whether to run them synchronously or asynchronously), run a whole test suite, or run individual test cases within a test suite.  For the second and third options, you can either blindly iterate over test suites and test cases or you can run test suites or test cases by name.  Which way you run the tests is based on how much control you need over the order of test execution.  When running a test suite you have control over whether the test cases in the suite are run synchronously or asynchronously.  The options that you have available reflect what you can do in SoapUI’s user interface.  The problem is digging through the JavaDocs to find out which methods on what objects to call.

Here is an example of a JUnit test that runs all the test cases in a SoapUI project in two different ways.  The lessControl method just runs every test case in the project in the simplest way whereas the fullControl method iterates over the test suites and the test cases in the project.

package com.omig.soapUISamples;

import java.util.List;
import org.junit.Test;
import com.eviware.soapui.impl.wsdl.WsdlProject;
import com.eviware.soapui.model.support.PropertiesMap;
import com.eviware.soapui.model.testsuite.TestCase;
import com.eviware.soapui.model.testsuite.TestRunner;
import com.eviware.soapui.model.testsuite.TestSuite;
import com.eviware.soapui.model.testsuite.TestRunner.Status;
import com.eviware.soapui.tools.SoapUITestCaseRunner;
import static org.junit.Assert.*;

public class SoapUITest {

@Test
public void lessControl() throws Exception {
SoapUITestCaseRunner runner = new SoapUITestCaseRunner();
runner.setProjectFile(“SoapUIWorkspace/Sample-soapui-project.xml”);
runner.setPrintReport(true);  //Outputs a small table to stdout of test results.
runner.run();
}

@Test
public void fullControl() throws Exception {
WsdlProject project = new WsdlProject(“SoapUIWorkspace/Sample-soapui-project.xml”);
List<TestSuite> testSuites = project.getTestSuiteList();
for( TestSuite suite : testSuites ) {
List<TestCase> testCases = suite.getTestCaseList();
for( TestCase testCase : testCases ) {
System.out.println(“Running SoapUI test [" + testCase.getName() + "]“);
TestRunner runner2 = testCase.run(new PropertiesMap(), false);
assertEquals(Status.FINISHED, runner2.getStatus());
}
}
}
}

With the code above you’ve got a hint at how run SoapUI functional tests from a JUnit test.  To actually make this automatic you’ve got (at least) one more step.  In my situation we use Ant to script our software builds.  We then use Hudson to schedule the builds and make it all run automatically.  There are a lot of other ways you could do this but I’ll only show the Ant way here.  Any build tool that has support for JUnit will suffice.

Below is an example of an Ant task that runs the JUnit tests that are executing the SoapUI tests.  The JUnit output is being generated in XML format and sent to a folder named soapui-test-reports.

<path id="classpath">
    <!-- The path to the lib folder in the SoapUI installation.-->
    <fileset dir="${env.SOAPUI.LIB}"/>
    <!-- The path to the bin folder in the SoapUI installation.-->
    <fileset dir="${env.SOAPUI.BIN}"/>
</path>

<path id="soapui-workspace">
    <fileset dir="SoapUIWorkspace">
      <include name="**/*.xml"/>
    </fileset>
</path>

<target name="soapui-tests" depends="compile">
    <mkdir dir="../soapui-test-reports" />
    <junit printsummary="withOutAndErr"
           showoutput="true"
           haltonerror="on"
           haltonfailure="on"
           filtertrace="off"
           fork="no"
           forkmode="once">
      <classpath refid="classpath" />
      <classpath refid="soapui-workspace" />
      <formatter type="xml" />
      <batchtest todir="../soapui-test-reports">
        <fileset dir="tests">
                <include name="**/*SoapUI*.java"/>
        </fileset>
      </batchtest>
    </junit>
</target>

Testrunner Script

The documentation for the testrunner script is much clearer than for integrating with JUnit.  Integrating the testrunner.bat file with your build process is simple and straight-forward. An example of an Ant task to do this is shown below.  See SoapUI’s documentation for an explanation of the command line switches.

<target name="soapui-tests-cmdline" depends="compile">
        <exec executable="${env.SOAPUI.BIN}/testrunner.bat"
                  failonerror="yes"
                  failifexecutionfails="yes">
                <arg value="-rjaf"/>
                <arg path="../soapui-test-reports"/>
                <arg path="SoapUIWorkspace/Insured-Access-soapui-project.xml"/>
        </exec>
</target>

Which Method Is Best?

In my experience so far the command line testrunner tool is the better method for several reasons:

1. There’s less work because you do not have to write a JUnit test suite/case to run the SoapUI tests.
2. It creates a less fragile environment because writing a JUnit test to run the SoapUI tests makes it very difficult not to hard code the path to the SoapUI project file. You don’t have to worry about this with the command line because the input to script is parametrized.
3. The command line test runner can output “correct” JUnit output. The JUnit reports generated by the command line test runner split results by test suite then test case. This is ideal when the results are viewed graphically because you get a full count of actual test cases run and you can drill-down to the individual test cases to see its run status*.
4. To integrate with JUnit you have to use the SoapUI API, which is not documented well. There are Javadocs, but actual documentation is sparse in them.

* As far as I can tell it is difficult or impossible to do this by integrating with JUnit manually. To do so you would have to somehow generate one JUnit test for each test case in all the test suites. The SoapUI API allows you to easily iterate over the test suites and test cases, but there is no way in Java to dynamically generate methods for each test case.