Hibernate.orgCommunity Documentation

Chapter 5. Configuration

5.1. Configuring TestNG to execute the TCK
5.2. Selecting the ValidationProvider
5.3. Selecting the DeployableContainer
5.3.1. Local test execution via BeanValidationLocalContainer
5.3.2. Remote test execution in managed JBoss AS 7
5.4. arquillian.xml

This chapter lays out how to configure the TCK harness by specifying the DeployableContainer, selecting the container interaction style and setting various other switches. Maven 2 is used in the configuration examples, mainly because it is the de facto standard build tool at the time of writing. However, the test harness is not dependent on a specific build tool and any other tool, for example Ant or Gradle, work as well.

If you have not made yourself familiar with the Arquillian documentation by now, this is a good time to do it. It will give you a deeper understanding of the different parts described in the following sections.

Tip

If you followed all the steps of Chapter 3, Installation you find the full pom.xml used in the following examples in the directory jsr349/ri/tck-runner.

The Bean Validation test harness is built atop TestNG, and it is TestNG that is responsible for selecting the tests to execute, the order of execution, and reporting the results. Detailed TestNG documentation can be found at testng.org.

The tck-tests.xml artifact provided in the TCK distribution must be run by TestNG (described by the TestNG documentation as "with a testng.xml file") unmodified for an implementation to pass the TCK. For testing purposes it is of course ok to modify the file.

<suite name="JSR-349-TCK" verbose="1">
    <test name="JSR-349-TCK">

        <method-selectors>
            <method-selector>
                <selector-class name="org.hibernate.beanvalidation.tck.util.IntegrationTestsMethodSelector"/>
            </method-selector>
        </method-selectors>

        <packages>
            <package name="org.hibernate.beanvalidation.tck.tests"/>
        </packages>
    </test>
</suite>

TestNG provides extensive reporting information. Depending on the build tool or IDE you use, the reporting will take a different format. Please consult the TestNG documentation and the tool documentation for more information.

Tip

In the example below the maven-dependency-plugin is used to extract the tck-tests.xml from the artifact jar file to make it available for the surefire configuration.

The most important configuration you have make in order to run the Bean Validation TCK is to specify your ValidationProvider you want to run your tests against. To do so you need to set the Java system property validation.provider to the fully specified class name of your ValidationProvider. In the examples below this is done via the systemProperties configuration of the maven-surefire-plugin. This property will be picked up by org.hibernate.beanvalidation.tck.util.TestUtil which will instantiate the Validator under test. This means the test harness does not rely on the service provider mechanism to instantiate the Bean Validation provider under test, partly because this selection mechanism is under test as well.

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <artifactId>hibernate-validator-parent</artifactId>
        <groupId>org.hibernate</groupId>
        <version>5.0.0-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

    <artifactId>hibernate-validator-tck-runner</artifactId>

    <name>Hibernate Validator TCK Runner</name>

    <properties>
        <tck.version>1.1.0-SNAPSHOT</tck.version>
        <tck.suite.file>${project.build.directory}/dependency/beanvalidation-tck-suite.xml</tck.suite.file>
        <arquillian.version>1.0.0.Final</arquillian.version>
        <validation.provider>org.hibernate.validator.HibernateValidator</validation.provider>
        <remote.debug/>
    </properties>

    <dependencies>
        <dependency>
            <groupId>javax.validation</groupId>
            <artifactId>validation-api</artifactId>
        </dependency>
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-validator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
        </dependency>
        <dependency>
            <groupId>org.hibernate.beanvalidation.tck</groupId>
            <artifactId>beanvalidation-tck</artifactId>
            <version>${tck.version}</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate.javax.persistence</groupId>
            <artifactId>hibernate-jpa-2.0-api</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.testng</groupId>
            <artifactId>arquillian-testng-container</artifactId>
            <version>${arquillian.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jboss.arquillian.protocol</groupId>
            <artifactId>arquillian-protocol-servlet</artifactId>
            <version>${arquillian.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <testResources>
            <testResource>
                <filtering>true</filtering>
                <directory>src/test/resources</directory>
            </testResource>
        </testResources>
        <plugins>
           <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <executions>
                    <execution>
                        <id>copy-tck-test-suite-file</id>
                        <phase>generate-test-sources</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>
                        <configuration>
                            <stripVersion>true</stripVersion>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>org.hibernate.beanvalidation.tck</groupId>
                                    <artifactId>beanvalidation-tck</artifactId>
                                    <type>xml</type>
                                    <classifier>suite</classifier>
                                    <overWrite>false</overWrite>
                                </artifactItem>
                            </artifactItems>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <suiteXmlFiles>
                        <suiteXmlFile>${tck.suite.file}</suiteXmlFile>
                    </suiteXmlFiles>
                    <argLine>-Xmx128m</argLine>
                    <forkMode>once</forkMode>
                    <systemProperties>
                        <property>
                            <name>validation.provider</name>
                            <value>${validation.provider}</value>
                        </property>
                    </systemProperties>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <executions>
                    <execution>
                        <id>generate-test-report</id>
                        <phase>test</phase>
                        <goals>
                            <goal>report-only</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <outputDirectory>${project.build.directory}/surefire-reports</outputDirectory>
                    <outputName>test-report</outputName>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

After setting the ValidationProvider you have to make a choice on the right DeployableContainer. Arquillian picks which container it is going to use to deploy the test archive and negotiate test execution using Java's service provider mechanism. Concretely Arquillian is looking for an implementation of the DeployableContainer SPI is on the classpath. In the following examples this choice is made through the use of Maven profiles adding the appropriate test dependencies.

Before trying to run the test harness against a real Java EE container it is recommended to run the test in a local JVM mode. The Bean Validation test harness ships with a DeployableContainer called BeanValidationLocalContainer which allows you to do so. All you need is the right dependency:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

   ...

    <profiles>
        <profile>
            <id>local</id>
            <activation>
                <property>
                    <name>incontainer</name>
                    <value>!true</value>
                </property>
            </activation>
            <properties>
                <arquillian.protocol>Local</arquillian.protocol>
            </properties>
            <dependencies>
                <dependency>
                    <groupId>org.hibernate.beanvalidation.tck</groupId>
                    <artifactId>standalone-container-adapter</artifactId>
                    <version>${tck.version}</version>
                </dependency>
            </dependencies>
        </profile>
    </profiles>
</project>

As you can see the local profile is activated by default. It adds the dependency org.hibernate.beanvalidation.tck:standalone-container-adapter which contains the DeployableContainer BeanValidationLocalContainer. If the test harness is executed with this dependency on the classpath the tests will run in the local JVM. The purpose of the property arquillian.protocol will be discussed in Section 5.4, “arquillian.xml”. Running the test harness is as easy as executing:

mvn test

Note

The local execution mode is just a convenience for developers. To pass the Bean Validation TCK the test harness has to pass in a Java EE container. The reason for this is that Bean Validation is part of the Java EE specification.

Tip

Arquillian will throw an exception in case it finds more than one DeployableContainer implementations on the classpath. If you want to be able to run in local as well as incontainer mode from the same project setup you will need to separate the dependencies. This is the main purpose of the profiles used in this example.

Now let's try to run the tests in a Java EE container, namely JBoss AS 7. For this we defined a new profile incontainer which can be activated via:

> mvn test -Dincontainer

In this case we are adding the dependency org.jboss.as:jboss-as-arquillian-container-managed which will enable a managed JBoss AS 7 as DeployableContainer. AS 7 could be already installed on the system or as in this case be downloaded and unpacked via the maven-dependency-plugin prior to the test run.

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

   ...

    <profiles>
        <profile>
            <id>local</id>
            <activation>
                <property>
                    <name>incontainer</name>
                    <value>!true</value>
                </property>
            </activation>
            <properties>
                <arquillian.protocol>Local</arquillian.protocol>
            </properties>
            <dependencies>
                <dependency>
                    <groupId>org.hibernate.beanvalidation.tck</groupId>
                    <artifactId>standalone-container-adapter</artifactId>
                    <version>${tck.version}</version>
                </dependency>
            </dependencies>
        </profile>
  
        <profile>
            <id>incontainer</id>
            <activation>
                <property>
                    <name>incontainer</name>
                </property>
            </activation>
            <properties>
                <arquillian.protocol>Servlet 3.0</arquillian.protocol>
            </properties>
            <dependencies>
                <dependency>
                    <groupId>org.jboss.as</groupId>
                    <artifactId>jboss-as-arquillian-container-managed</artifactId>
                    <version>${jbossas.version}</version>
                    <scope>test</scope>
                </dependency>
            </dependencies>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-dependency-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>unpack</id>
                                <phase>process-test-resources</phase>
                                <goals>
                                    <goal>unpack</goal>
                                </goals>
                                <configuration>
                                    <artifactItems>
                                        <artifactItem>
                                            <groupId>org.jboss.as</groupId>
                                            <artifactId>jboss-as-dist</artifactId>
                                            <version>${jbossas.version}</version>
                                            <type>zip</type>
                                            <overWrite>false</overWrite>
                                            <outputDirectory>${project.build.directory}</outputDirectory>
                                        </artifactItem>
                                    </artifactItems>
                                </configuration>
                            </execution>
                            <execution>
                                <id>copy-beanvalidation-api</id>
                                <phase>generate-test-sources</phase>
                                <goals>
                                    <goal>copy</goal>
                                </goals>
                                <configuration>
                                    <outputDirectory>
                                        ${project.build.directory}/jboss-as-${jbossas.version}/modules/javax/validation/api/main
                                    </outputDirectory>
                                    <stripVersion>true</stripVersion>
                                    <artifactItems>
                                        <artifactItem>
                                            <groupId>javax.validation</groupId>
                                            <artifactId>validation-api</artifactId>
                                        </artifactItem>
                                    </artifactItems>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <configuration>
                            <argLine>-Xmx1024m -Djava.util.logging.manager=org.jboss.logmanager.LogManager</argLine>
                            <forkMode>once</forkMode>
                            <suiteXmlFiles>
                                <suiteXmlFile>${tck.suite.file}</suiteXmlFile>
                            </suiteXmlFiles>
                            <systemPropertyVariables>
                                <arquillian.launch>incontainer</arquillian.launch>
                                <validation.provider>${validation.provider}</validation.provider>
                                <includeIntegrationTests>true</includeIntegrationTests>
                            </systemPropertyVariables>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>

The easiest way to debug the tests when they are executes in a remote container is to open a remote debugging session. In order to do so the JVM needs to be started with the right remote debugging options. Again this can be achieved via a profile:

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

   ...

    <profiles>
        <profile>
            <id>local</id>
            <activation>
                <property>
                    <name>incontainer</name>
                    <value>!true</value>
                </property>
            </activation>
            <properties>
                <arquillian.protocol>Local</arquillian.protocol>
            </properties>
            <dependencies>
                <dependency>
                    <groupId>org.hibernate.beanvalidation.tck</groupId>
                    <artifactId>standalone-container-adapter</artifactId>
                    <version>${tck.version}</version>
                </dependency>
            </dependencies>
        </profile>

        <profile>
            <id>incontainer-debug</id>
            <activation>
                <property>
                    <name>debug</name>
                </property>
            </activation>
            <properties>
                <remote.debug>-Xnoagent -Djava.compiler=NONE -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005</remote.debug>
            </properties>
        </profile>
  
        <profile>
            <id>incontainer</id>
            <activation>
                <property>
                    <name>incontainer</name>
                </property>
            </activation>
            <properties>
                <arquillian.protocol>Servlet 3.0</arquillian.protocol>
            </properties>
            <dependencies>
                <dependency>
                    <groupId>org.jboss.as</groupId>
                    <artifactId>jboss-as-arquillian-container-managed</artifactId>
                    <version>${jbossas.version}</version>
                    <scope>test</scope>
                </dependency>
            </dependencies>
            <build>
                <plugins>
                    <plugin>
                        <artifactId>maven-dependency-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>unpack</id>
                                <phase>process-test-resources</phase>
                                <goals>
                                    <goal>unpack</goal>
                                </goals>
                                <configuration>
                                    <artifactItems>
                                        <artifactItem>
                                            <groupId>org.jboss.as</groupId>
                                            <artifactId>jboss-as-dist</artifactId>
                                            <version>${jbossas.version}</version>
                                            <type>zip</type>
                                            <overWrite>false</overWrite>
                                            <outputDirectory>${project.build.directory}</outputDirectory>
                                        </artifactItem>
                                    </artifactItems>
                                </configuration>
                            </execution>
                            <execution>
                                <id>copy-beanvalidation-api</id>
                                <phase>generate-test-sources</phase>
                                <goals>
                                    <goal>copy</goal>
                                </goals>
                                <configuration>
                                    <outputDirectory>
                                        ${project.build.directory}/jboss-as-${jbossas.version}/modules/javax/validation/api/main
                                    </outputDirectory>
                                    <stripVersion>true</stripVersion>
                                    <artifactItems>
                                        <artifactItem>
                                            <groupId>javax.validation</groupId>
                                            <artifactId>validation-api</artifactId>
                                        </artifactItem>
                                    </artifactItems>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <configuration>
                            <argLine>-Xmx1024m -Djava.util.logging.manager=org.jboss.logmanager.LogManager</argLine>
                            <forkMode>once</forkMode>
                            <suiteXmlFiles>
                                <suiteXmlFile>${tck.suite.file}</suiteXmlFile>
                            </suiteXmlFiles>
                            <systemPropertyVariables>
                                <arquillian.launch>incontainer</arquillian.launch>
                                <validation.provider>${validation.provider}</validation.provider>
                                <includeIntegrationTests>true</includeIntegrationTests>
                            </systemPropertyVariables>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>
</project>

In this you would start the tests via (activation order is important):

mvn test -Ddebug -Dincontainer

The test execution would start but then be suspended until the JVM receives a remote debugging request on port 5005.

The next piece in the configuration puzzle is arquillian.xml. This xml file needs to be in the root of the classpath and is used to pass additional options to the selected container. Let's look at an example:

<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
        http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

    <!-- Need to set the default protocol and use resource filtering, because of https://issues.jboss.org/browse/ARQ-579 -->
    <defaultProtocol type="${arquillian.protocol}"/>

    <engine>
        <property name="deploymentExportPath">target/artifacts</property>
    </engine>

    <container qualifier="local" default="true">
        <protocol type="Local"/>   <!-- Takes no effect - ARQ-579 -->
    </container>

    <container qualifier="incontainer">
        <protocol type="Servlet 3.0"/> <!-- Takes no effect - ARQ-579 -->
        <configuration>
            <property name="jbossHome">${jbossTargetDir}</property>
            <property name="javaVmArguments">-Xmx1024m -XX:MaxPermSize=512m ${remote.debug} -Dvalidation.provider=${validation.provider}</property>
            <property name="allowConnectingToRunningServer">true</property>
        </configuration>
    </container>

</arquillian>

In this case we have two container nodes, local and incontainer. The selection of which configuration to chose at runtime is made by setting the system property arquillian.launch. If arquillian.launch is not specify the default is selected which in this case is local.

The most important container configuration option is the protocol type which determines how Arquillian communicates with the selected container. The most popular types are Local and Servlet 3.0. The protocol can be defined per container node, however, due to a bug only the defaultProtocol setting takes effect. This is also the reason for the arquillian.protocol property used in the pom examples. Depending on the chosen profile the appropriate protocol is selected and added to arquillian.xml via resource filtering.

Another interesting property is deploymentExportPath which is optional and instructs Arquillian to dump the test artifacts to the specified directory on disk. Inspection of the deployed artifacts can be very useful when debugging test failures.