Hibernate.orgCommunity Documentation
This chapter lays out how to configure the TCK harness by specifying
the DeployableContainer
, selecting the container
interaction style and setting various other switches. Maven 2 is used in the
configuration examples, mainly because it is the de facto standard build tool
at the time of writing. However, the test harness is not dependent on a
specific build tool and any other tool, for example Ant or Gradle, work as
well.
If you have not made yourself familiar with the Arquillian documentation by now, this is a good time to do it. It will give you a deeper understanding of the different parts described in the following sections.
If you followed all the steps of Chapter 3, Installation
you find the full pom.xml
used in the following
examples in the directory
jsr349/ri/tck-runner
.
The Bean Validation test harness is built atop TestNG, and it is TestNG that is responsible for selecting the tests to execute, the order of execution, and reporting the results. Detailed TestNG documentation can be found at testng.org.
The tck-tests.xml
artifact provided in the TCK
distribution must be run by TestNG (described by the TestNG documentation
as "with a testng.xml
file") unmodified for an
implementation to pass the TCK. For testing purposes it is of course ok to
modify the file.
<suite name="JSR-349-TCK" verbose="1"> <test name="JSR-349-TCK"> <method-selectors> <method-selector> <selector-class name="org.hibernate.beanvalidation.tck.util.IntegrationTestsMethodSelector"/> </method-selector> </method-selectors> <packages> <package name="org.hibernate.beanvalidation.tck.tests"/> </packages> </test> </suite>
TestNG provides extensive reporting information. Depending on the build tool or IDE you use, the reporting will take a different format. Please consult the TestNG documentation and the tool documentation for more information.
In the example below the
maven-dependency-plugin is used to extract the
tck-tests.xml
from the artifact jar file to make it
available for the surefire configuration.
The most important configuration you have make in order to run the
Bean Validation TCK is to specify your
ValidationProvider
you want to run your tests
against. To do so you need to set the Java system property
validation.provider to the fully specified class name
of your ValidationProvider
. In the examples below
this is done via the systemProperties configuration
of the maven-surefire-plugin. This property will be
picked up by
org.hibernate.beanvalidation.tck.util.TestUtil
which will instantiate the Validator
under test.
This means the test harness does not rely on the service provider
mechanism to instantiate the Bean Validation provider under test, partly
because this selection mechanism is under test as well.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>hibernate-validator-parent</artifactId>
<groupId>org.hibernate</groupId>
<version>5.0.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>hibernate-validator-tck-runner</artifactId>
<name>Hibernate Validator TCK Runner</name>
<properties>
<tck.version>1.1.0-SNAPSHOT</tck.version>
<tck.suite.file>${project.build.directory}/dependency/beanvalidation-tck-suite.xml</tck.suite.file>
<arquillian.version>1.0.0.Final</arquillian.version>
<validation.provider>org.hibernate.validator.HibernateValidator</validation.provider>
<remote.debug/>
</properties>
<dependencies>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>beanvalidation-tck</artifactId>
<version>${tck.version}</version>
</dependency>
<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.0-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.testng</groupId>
<artifactId>arquillian-testng-container</artifactId>
<version>${arquillian.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.protocol</groupId>
<artifactId>arquillian-protocol-servlet</artifactId>
<version>${arquillian.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<testResources>
<testResource>
<filtering>true</filtering>
<directory>src/test/resources</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-tck-test-suite-file</id>
<phase>generate-test-sources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>beanvalidation-tck</artifactId>
<type>xml</type>
<classifier>suite</classifier>
<overWrite>false</overWrite>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${tck.suite.file}</suiteXmlFile>
</suiteXmlFiles>
<argLine>-Xmx128m</argLine>
<forkMode>once</forkMode>
<systemProperties>
<property>
<name>validation.provider</name>
<value>${validation.provider}</value>
</property>
</systemProperties>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<executions>
<execution>
<id>generate-test-report</id>
<phase>test</phase>
<goals>
<goal>report-only</goal>
</goals>
</execution>
</executions>
<configuration>
<outputDirectory>${project.build.directory}/surefire-reports</outputDirectory>
<outputName>test-report</outputName>
</configuration>
</plugin>
</plugins>
</build>
</project>
After setting the ValidationProvider
you have
to make a choice on the right DeployableContainer
.
Arquillian picks which container it is going to use to deploy the test
archive and negotiate test execution using Java's service provider
mechanism. Concretely Arquillian is looking for an implementation of the
DeployableContainer
SPI is on the classpath. In the
following examples this choice is made through the use of Maven profiles
adding the appropriate test dependencies.
Before trying to run the test harness against a real Java EE
container it is recommended to run the test in a local JVM mode. The
Bean Validation test harness ships with a
DeployableContainer
called
BeanValidationLocalContainer
which allows you to
do so. All you need is the right dependency:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<profiles>
<profile>
<id>local</id>
<activation>
<property>
<name>incontainer</name>
<value>!true</value>
</property>
</activation>
<properties>
<arquillian.protocol>Local</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>standalone-container-adapter</artifactId>
<version>${tck.version}</version>
</dependency>
</dependencies>
</profile>
</profiles>
</project>
As you can see the local
profile is activated by default. It adds the dependency
org.hibernate.beanvalidation.tck:standalone-container-adapter
which contains the DeployableContainer
BeanValidationLocalContainer
. If the test harness
is executed with this dependency on the classpath the tests will run in
the local JVM. The purpose of the property
arquillian.protocol will be discussed in Section 5.4, “arquillian.xml”. Running the test harness is as
easy as executing:
mvn test
The local execution mode is just a convenience for developers. To pass the Bean Validation TCK the test harness has to pass in a Java EE container. The reason for this is that Bean Validation is part of the Java EE specification.
Arquillian will throw an exception in case it finds more than
one DeployableContainer
implementations on
the classpath. If you want to be able to run in local as well as
incontainer mode from the same project setup you will need to
separate the dependencies. This is the main purpose of the profiles
used in this example.
Now let's try to run the tests in a Java EE container, namely JBoss AS 7. For this we defined a new profile incontainer which can be activated via:
> mvn test -Dincontainer
In this
case we are adding the dependency
org.jboss.as:jboss-as-arquillian-container-managed
which will enable a managed JBoss AS 7 as
DeployableContainer
. AS 7 could be already
installed on the system or as in this case be downloaded and unpacked
via the maven-dependency-plugin prior to the test
run.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<profiles>
<profile>
<id>local</id>
<activation>
<property>
<name>incontainer</name>
<value>!true</value>
</property>
</activation>
<properties>
<arquillian.protocol>Local</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>standalone-container-adapter</artifactId>
<version>${tck.version}</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>incontainer</id>
<activation>
<property>
<name>incontainer</name>
</property>
</activation>
<properties>
<arquillian.protocol>Servlet 3.0</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-arquillian-container-managed</artifactId>
<version>${jbossas.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack</id>
<phase>process-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-dist</artifactId>
<version>${jbossas.version}</version>
<type>zip</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
<execution>
<id>copy-beanvalidation-api</id>
<phase>generate-test-sources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/jboss-as-${jbossas.version}/modules/javax/validation/api/main
</outputDirectory>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx1024m -Djava.util.logging.manager=org.jboss.logmanager.LogManager</argLine>
<forkMode>once</forkMode>
<suiteXmlFiles>
<suiteXmlFile>${tck.suite.file}</suiteXmlFile>
</suiteXmlFiles>
<systemPropertyVariables>
<arquillian.launch>incontainer</arquillian.launch>
<validation.provider>${validation.provider}</validation.provider>
<includeIntegrationTests>true</includeIntegrationTests>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
You find a list of available deployable containers in the Arquillian documentation. Refer to Container adapters.
Some of the tests can only run within a container (mainly due to
the way they are setup). These tests are annotated with
@IntegrationTest. Since these tests would fail in local execution mode
they can be excluded by setting the system property
includeIntegrationTests to
false
(or not specifying it at all). To pass the
TCK the property needs to be set to true
and all
integration tests need to run.
The in- an exclusion of integration tests is handled via a
TestNG specific IMethodSelector
, more
concretely IntegrationTestsMethodSelector
.
The easiest way to debug the tests when they are executes in a remote container is to open a remote debugging session. In order to do so the JVM needs to be started with the right remote debugging options. Again this can be achieved via a profile:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<profiles>
<profile>
<id>local</id>
<activation>
<property>
<name>incontainer</name>
<value>!true</value>
</property>
</activation>
<properties>
<arquillian.protocol>Local</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>standalone-container-adapter</artifactId>
<version>${tck.version}</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>incontainer-debug</id>
<activation>
<property>
<name>debug</name>
</property>
</activation>
<properties>
<remote.debug>-Xnoagent -Djava.compiler=NONE -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005</remote.debug>
</properties>
</profile>
<profile>
<id>incontainer</id>
<activation>
<property>
<name>incontainer</name>
</property>
</activation>
<properties>
<arquillian.protocol>Servlet 3.0</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-arquillian-container-managed</artifactId>
<version>${jbossas.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack</id>
<phase>process-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-dist</artifactId>
<version>${jbossas.version}</version>
<type>zip</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
<execution>
<id>copy-beanvalidation-api</id>
<phase>generate-test-sources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/jboss-as-${jbossas.version}/modules/javax/validation/api/main
</outputDirectory>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx1024m -Djava.util.logging.manager=org.jboss.logmanager.LogManager</argLine>
<forkMode>once</forkMode>
<suiteXmlFiles>
<suiteXmlFile>${tck.suite.file}</suiteXmlFile>
</suiteXmlFiles>
<systemPropertyVariables>
<arquillian.launch>incontainer</arquillian.launch>
<validation.provider>${validation.provider}</validation.provider>
<includeIntegrationTests>true</includeIntegrationTests>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
In this you would start the tests via (activation order is important):
mvn test -Ddebug -Dincontainer
The test execution would start but then be suspended until the JVM receives a remote debugging request on port 5005.
The next piece in the configuration puzzle is
arquillian.xml
. This xml file needs to be in the root
of the classpath and is used to pass additional options to the selected
container. Let's look at an example:
<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://jboss.org/schema/arquillian http://jboss.org/schema/arquillian/arquillian_1_0.xsd"> <!-- Need to set the default protocol and use resource filtering, because of https://issues.jboss.org/browse/ARQ-579 --> <defaultProtocol type="${arquillian.protocol}"/> <engine> <property name="deploymentExportPath">target/artifacts</property> </engine> <container qualifier="local" default="true"> <protocol type="Local"/> <!-- Takes no effect - ARQ-579 --> </container> <container qualifier="incontainer"> <protocol type="Servlet 3.0"/> <!-- Takes no effect - ARQ-579 --> <configuration> <property name="jbossHome">${jbossTargetDir}</property> <property name="javaVmArguments">-Xmx1024m -XX:MaxPermSize=512m ${remote.debug} -Dvalidation.provider=${validation.provider}</property> <property name="allowConnectingToRunningServer">true</property> </configuration> </container> </arquillian>
In this case we have two container nodes, local and incontainer. The selection of which configuration to chose at runtime is made by setting the system property arquillian.launch. If arquillian.launch is not specify the default is selected which in this case is local.
The most important container configuration option is the protocol
type which determines how Arquillian communicates with the selected
container. The most popular types are Local and
Servlet 3.0. The protocol can be defined per
container node, however, due to a bug only the
defaultProtocol setting takes effect. This is also
the reason for the arquillian.protocol property used
in the pom examples. Depending on the chosen profile the appropriate
protocol is selected and added to arquillian.xml
via
resource
filtering.
Another interesting property is deploymentExportPath which is optional and instructs Arquillian to dump the test artifacts to the specified directory on disk. Inspection of the deployed artifacts can be very useful when debugging test failures.
Copyright © 2009 - 2012 Red Hat, Inc.