Hibernate.orgCommunity Documentation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Copyright © 2009 - 2012 Red Hat, Inc.
This guide describes how to download, install, configure, and run the Technology Compatibility Kit (TCK) used to verify the compatibility of an implementation of JSR 349: Bean Validation 1.1.
The Bean Validation TCK is built atop Arquillian, a portable and configurable automated test suite for authoring unit and integration tests in a Java EE environment.
The Bean Validation TCK is provided under the Apache Public License 2.0.
This guide is for implementors of the Bean Validation specification to assist in running the test suite that verifies the compatibility of their implementation.
The Bean Validation TCK is based on the Bean Validation specification 1.1 (JSR 349). Information about the specification can be found on the JSR-349 JCP page.
If you are running the Bean Validation TCK for the first time, read Chapter 1, Introduction completely for the necessary background information about the TCK. Once you have reviewed that material, perform the steps outlined in the remaining chapters.
Chapter 1, Introduction gives an overview of the principles that apply generally to all Technology Compatibility Kits (TCKs), outlines the appeals process and describes the Bean Validation TCK architecture and components.
Chapter 2, Appeals Process explains the process to be followed by an implementor should they wish to challenge any test in the TCK.
Chapter 3, Installation explains where to obtain the required software for the Bean Validation TCK and how to install it.
Chapter 5, Configuration details the configuration of the test harness, how to create a TCK runner for the TCK test suite and the mechanics of how an in-container test is conducted.
Chapter 4, Reports explains the test reports that are generated by the TCK test suite and introduces the TCK audit report as a tool for measuring the completeness of the TCK in testing the JSR 349 specification and in understanding how test cases relate to the specification.
??? documents how the TCK test suite is executed. It covers both modes supported by the TCK, standalone and in-container, and shows how to dump the generated test artifacts to disk
This chapter explains the purpose of a TCK and identifies the foundation elements of the Bean Validation TCK.
A TCK, or Technology Compatibility Kit, is one of the three required pieces for any JSR (the other two being the specification document and the reference implementation). The TCK is a set of tools and tests to verify that an implementation of the technology conforms to the specification. The tests are the primary component, but the tools serve an equally critical role of providing a framework and/or set of SPIs for executing the tests.
The tests in the TCK are derived from assertions in the written
specification document. The assertions are itemized in an XML document
(tck-audit.xml
), where they each get assigned a
unique identifier, and materialize as a suite of automated tests that
collectively validate whether an implementation complies with the
aforementioned assertions, and in turn the specification. For a particular
implementation to be certified, all of the required tests must pass
(meaning the provided test suite must be run unmodified).
A TCK is entirely implementation agnostic. It should validate assertions by consulting the specification's public API.
The goal of any specification is to eliminate portability problems so long as the program which uses the implementation also conforms to the rules laid out in the specification.
Executing the TCK is a form of compatibility testing. It's important to understand that compatibility testing is distinctly different from product testing. The TCK is not concerned with robustness, performance or ease of use, and therefore cannot vouch for how well an implementation meets these criteria. What a TCK can do is to ensure the exactness of an implementation as it relates to the specification.
Compatibility testing of any feature relies on both a complete specification and a complete reference implementation. The reference implementation demonstrates how each test can be passed and provides additional context to the implementor during development for the corresponding assertion.
Java platform compatibility is important to different groups involved with Java technologies for different reasons:
Compatibility testing is the means by which the JCP ensures that the Java platform does not become fragmented as it is ported to different operating systems and hardware.
Compatibility testing benefits developers working in the Java programming language, enabling them to write applications once and deploy them across heterogeneous computing environments without porting.
Compatibility testing enables application users to obtain applications from disparate sources and deploy them with confidence.
Conformance testing benefits Java platform implementors by ensuring the same extent of reliability for all Java platform ports.
The Bean Validation specification goes to great lengths to ensure that programs written for Java EE are compatible and the TCK is rigorous about enforcing the rules the specification lays down.
The Bean Validation TCK is designed as a portable, configurable and automated test suite for verifying the compatibility of an implementation of JSR 349. The test suite is built atop TestNG and provides a series of extensions that allow runtime packaging and deployment of JEE artifacts for in-container testing (Arquillian).
Each test class in the suite acts as a deployable unit. The deployable units, or artifacts, are defined in a declarative way using annotations.
The declarative approach allows many of the tests to be executed in a standalone implementation of Bean Validation, accounting for a boost in developer productivity. However, an implementation is only valid if all tests pass using the in-container execution mode. The standalone mode is merely a developer convenience.
TODO - explain the reason for this - BV part of EE
This section lists the applicable requirements and specifications for the Bean Validation TCK.
Bean Validation API - The Java API defined in the Bean Validation specification and provided by the reference implementation.
Arquillian - The Bean Validation TCK requires Arquillian version 1.0.0.Final. The Harness is based on TestNG 5.x.
TCK Audit Tool - An itemization of the assertions in the specification documents which are cross referenced by the individual tests. Describes how well the TCK covers the specification.
Reference runtime - The designated reference runtimes for compatibility testing of the Bean Validation specification is the Sun Java Platform, Enterprise Edition (Java EE) 7 reference implementation (RI). See details at Java EE 7 (todo - check link)
The Bean Validation TCK includes the following components:
The test suite, which is a collection of TestNG tests, the TestNG test suite descriptor and supplemental resources that configure Bean Validation and other software components.
The TCK audit
(tck-audit.xml
) is used to list out the
assertions identified in the Bean Validation specification. It
matches the assertions to test cases in the test suite by unique
identifier and produces a coverage report.
The audit document is provided along with the TCK. Each assertion is defined with a reference to a chapter, section and paragraph from the specification document, making it easy for the implementor to locate the language in the specification document that supports the feature being tested.
TCK documentation accompanied by release notes identifying updates between versions.
While the Bean Validation TCK is rigourous about enforcing an implementation's conformance to the JSR 349 specification, it's reasonable to assume that an implementor may discover new and/or better ways to validate the assertions. This chapter covers the appeals process, defined by the Specification Lead, Red Hat Middleware LLC., which allows implementors of the JSR 349 specification to challenge one or more tests defined by the Bean Validation TCK.
The appeals process identifies who can make challenges to the TCK, what challenges to the TCK may be submitted, how these challenges are submitted, how and by whom challenges are addressed and how accepted challenges to the TCK are managed.
Following the recent adoption of transparency in the JCP, implementors are encouraged to make their appeals public, which this process facilitates. The JCP community should recognize that issue reports are a central aspect of any good software and it's only natural to point out shortcomings and strive to make improvements. Despite this good faith, not all implementors will be comfortable with a public appeals process. Instructions about how to make a private appeal are therefore provided.
Any implementor may submit an appeal to challenge one or more tests in the Bean Validation TCK. In fact, members of the JSR 349 Expert Group (EG) encourage this level of participation.
Any test case (e.g. @Test
method), test case
configuration (e.g. @Deployment, validation.xml), test entities,
annotations and other resources may be challenged by an appeal.
What is generally not challengeable are the assertions made by the specification. The specification document is controlled by a separate process and challenges to it should be handled through the JSR 349 EG by sending an e-mail to jsr349-comments@jcp.org. (todo verify email)
To submit a challenge, a new issue of type Bug should be created against BVTCK in the Hibernate JIRA instance. The appellant should complete the Summary, Component (TCK Appeal), Environment and Description Field only. Any communication regarding the issue should be added in the comments of the issue for accurate record.
To submit an issue in the Hibernate JIRA, you must have a (free) Jira member account. You can create a member account using the on-line registration.
If you wish to make a private challenge, you should follow the above procedure, setting the Security Level to Private. Only the issue reporter, TCK Project Lead and designates will be able to view the issue.
The challenges will be addressed in a timely fashion by the Bean Validation TCK Project Lead, as designated by Specification Lead, Red Hat Inc. or his/her designate. The appellant can also monitor the process by watching the issue filed against BVTCK.
The current TCK Project Lead is listed on the BVTCK Project Summary Page.
Accepted challenges will be acknowledged via the filed issue's comment section. Communication between the Bean Validation TCK Project Lead and the appellant will take place via the issue comments. The issue's status will be set to "Resolved" when the TCK project lead believes the issue to be resolved. The appellant should, within 30 days, either close the issue if they agree, or reopen the issue if they do not believe the issue to be resolved.
Resolved issue not addressed for 30 days will be closed by the TCK Project Lead. If the TCK Project Lead and appellant are unable to agree on the issue resolution, it will be referred to the JSR 349 specification lead or his/her designate.
This chapter explains how to obtain the TCK and supporting software and provides recommendations for how to install/extract it on your system.
You can obtain a release of the Bean Validation TCK project from the
from the download
page on the Hibernate Validator website. The Bean Validation TCK
is distributed as a ZIP file, which contains the TCK artifacts (the test
suite binary and source, the test suite descriptor, the audit source and
report), the TCK library dependencies in /lib
and
documentation in /doc
. The contents should look like (todo -
verify against final dist bundle):
artifacts/ changelog.txt docs/ lib/ license.txt src/
You can also download the source code from GitHub - https://github.com/beanvalidation/beanvalidation-tck.
The JSR 349 reference implementation (RI) project is named Hibernate Validator. You can obtain the Hibernate Validator release used as reference implementation from the Hibernate Validator download page.
Hibernate Validator is not required for running the Bean Validation TCK, but it can be used as a reference for familiarizing yourself with the TCK before testing your own Bean Validation implementation.
The TCK requires the following two Java runtime environments:
Java 6 or better
Java EE 7 or better (e.g., JBoss AS 7.x)
You should refer to vendor instructions for how to install the runtime.
The rest of the TCK software can simply be extracted. It's
recommended that you create a dedicated folder to hold all of the
jsr349-related artifacts. This guide assumes the folder is called
jsr349
. Extract the src
folder
of the TCK distribution into a sub-folder named tck
or
use the following git commands to
git clone git://github.com/beanvalidation/beanvalidation-tck tck git checkout 1.1.0.Final // TODO - add right version number
You can also check out the full Hibernate Validator source into a subfolder ri. This will allow you to run the TCK against Hibernate Validator.
git clone git://github.com/hibernate/hibernate-validator.git ri git checkout 4.2.0.Final // TODO - add right version number
The resulting folder structure is shown here:
jsr349/ ri/ tck/
Now lets have a look at one concrete test of the TCK, namely
ConstraintInheritanceTest
(found in
tck/tck/src/main/java/org/hibernate/beanvalidation/tck/tests/constraints/inheritance/ConstraintInheritanceTest.java
):
package org.hibernate.beanvalidation.tck.tests.constraints.inheritance; import java.lang.annotation.Annotation; import javax.validation.Validator; import javax.validation.constraints.NotNull; import javax.validation.metadata.BeanDescriptor; import javax.validation.metadata.PropertyDescriptor; import org.jboss.arquillian.container.test.api.Deployment; import org.jboss.arquillian.testng.Arquillian; import org.jboss.shrinkwrap.api.spec.WebArchive; import org.jboss.test.audit.annotations.SpecAssertion; import org.jboss.test.audit.annotations.SpecAssertions; import org.testng.annotations.Test; import org.hibernate.beanvalidation.tck.util.TestUtil; import org.hibernate.beanvalidation.tck.util.shrinkwrap.WebArchiveBuilder; import static org.testng.Assert.assertTrue; public class ConstraintInheritanceTest extends Arquillian { @Deployment public static WebArchive createTestArchive() { return new WebArchiveBuilder() .withTestClassPackage( ConstraintInheritanceTest.class ) .build(); } @Test @SpecAssertion(section = "3.3", id = "b") public void testConstraintsOnSuperClassAreInherited() { Validator validator = TestUtil.getValidatorUnderTest(); BeanDescriptor beanDescriptor = validator.getConstraintsForClass( Bar.class ); String propertyName = "foo"; assertTrue( beanDescriptor.getConstraintsForProperty( propertyName ) != null ); PropertyDescriptor propDescriptor = beanDescriptor.getConstraintsForProperty( propertyName ); // cast is required for JDK 5 - at least on Mac OS X Annotation constraintAnnotation = (Annotation) propDescriptor.getConstraintDescriptors() .iterator() .next().getAnnotation(); assertTrue( constraintAnnotation.annotationType() == NotNull.class ); } @Test @SpecAssertions({ @SpecAssertion(section = "3.3", id = "a"), @SpecAssertion(section = "3.3", id = "b") }) public void testConstraintsOnInterfaceAreInherited() { Validator validator = TestUtil.getValidatorUnderTest(); BeanDescriptor beanDescriptor = validator.getConstraintsForClass( Bar.class ); String propertyName = "fubar"; assertTrue( beanDescriptor.getConstraintsForProperty( propertyName ) != null ); PropertyDescriptor propDescriptor = beanDescriptor.getConstraintsForProperty( propertyName ); // cast is required for JDK 5 - at least on Mac OS X Annotation constraintAnnotation = (Annotation) propDescriptor.getConstraintDescriptors() .iterator() .next().getAnnotation(); assertTrue( constraintAnnotation.annotationType() == NotNull.class ); } }
Each test class is treated as an individual artifact (hence
the @Deployment
annotation on the class). In most tests
the created artifact is a standard Web
application Archive build via WebArchiveBuilder
which
in turn is a helper class of the TCK itself alleviating the creation of of
the artifact. All methods annotated with @Test
are
actual tests which are getting run. Last but not least we see the use of
the @SpecAssertion
annotation which creates the link between
the tck-audit.xml document and the actual test (see Section 1.1, “TCK Primer”).
Install Maven. You can find documentation on how to install Maven 2 in the Maven: The Definitive Guide book published by Sonatype.
Change to the
ri/hibernate-validator-tck-runner
directory.
Next, instruct Maven to run the TCK:
mvn test -Dincontainer
TestNG will report, via Maven, the outcome of the run, and
report any failures on the console. Details can be found in
target/surefire-reports/TestSuite.txt
.
This chapter covers the two types of reports that can be generated from the TCK, an assertion coverage report and the test execution results.
A specification can be distilled into a collection of assertions that define the behavior of the software. This section introduces the Bean Validation TCK coverage report, which documents the relationship between the assertions that have been identified in the JSR 349 specification document and the tests in the TCK test suite.
The structure of this report is controlled by the assertion document, so we'll start there.
The Bean Validation TCK developers have analyzed the JSR 349 specification document and identified the assertions that are present in each chapter. Here's an example of one such assertion found in section 2.1: “Every constraint annotation must define a message element of type String”
The assertions are listed in the XML file
tck-audit.xml
in the Bean Validation TCK
distribution. Each assertion is identified by the section of the
specification document in which it resides and assigned a unique
paragraph identifier to narrow down the location of the assertion
further. To continue with the example, the assertion shown above is
listed in the tck-audit.xml
file using this XML
fragment:
<section id="2.1.1" title="Constraint definition properties">
...
<assertion id="c">
<text>Every constraint annotation must define a message element of type String</text>
</assertion>
...
</section>
The strategy of the Bean Validation TCK is to write a test which
validates this assertion when run against an implementation. A test case
(a method annotated with @Test
) is correlated with an
assertion using the
@org.jboss.test.audit.annotations.SpecAssertion
annotation as follows:
@Test
@SpecAssertion(section = "2.1.1", id = "c")
public void testConstraintDefinitionWithoutMessageParameter() {
try {
Validator validator = TestUtil.getValidatorUnderTest();
validator.validate( new DummyEntityNoMessage() );
fail( "The used constraint does not define a message parameter. The validation should have failed." );
}
catch ( ConstraintDefinitionException e ) {
// success
}
To help evaluate the distribution of coverage for these assertions, the TCK provides a detailed coverage report. This report is also useful to help implementors match tests with the language in the specification that supports the behavior being tested.
The coverage report is an HTML report generated as part of the TCK project build. Specifically, it is generated by an annotation processor that attaches to the compilation of the classes in the TCK test suite.
You can find the source code for this processor in the GitHub repository https://github.com/jboss/jboss-test-audit
The report is written to the file
target/coverage-report/coverage-beanvalidation.html
.
The report itself has five sections:
Chapter Summary - List the chapters (that contain assertions) in the specification document along with total assertions, tests and coverage percentage.
Section Summary - Lists the sections (that contain assertions) in the specification document along with total assertions, tests and coverage percentage.
Coverage Detail - Each assertion and the test that covers it, if any.
Unmatched Tests - A list of tests for which there is no matching assertion (useful during TCK development).
Unversioned Tests - A list of
tests for which there is no @SpecVersion
annotation on the test class (useful during TCK development).
The coverage report is color coded to indicate the status of an assertion, or group of assertions. The status codes are as follows:
Covered - a test exists for this assertion
Not covered - no test exists for this assertion
Unimplemented - a test exists, but is unimplemented
Untestable - the assertion has been deemed untestable, a note, explaining why, is normally provided
For reasons provided in the tck-audit.xml
document and presented in the coverage report, some assertions are not
testable.
The coverage report does not give any indication as to whether the tests are passing. That's where the TestNG reports come in.
As you by now know, the Bean Validation TCK test suite is really just a TestNG test suite. That means an execution of the Bean Validation TCK test suite produces all the same reports that TestNG produces. This section will go over those reports and show you were to go to find each of them.
When the Bean Validation TCK test suite is executed during the Maven 2 test phase of the TCK runner project, TestNG is invoked indirectly through the Maven Surefire plugin. Surefire is a test execution abstraction layer capable of executing a mix of tests written for JUnit, TestNG, and other supported test frameworks.
Why is this relevant? It means two things. First, it means that you are going to get a summary of the test run on the commandline. Here's the output generated when the tests are run using standalone mode.
------------------------------------------------------- T E S T S ------------------------------------------------------- Running TestSuite Tests run: 258, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.062 sec Results : Tests run: 258, Failures: 0, Errors: 0, Skipped: 0
The number of tests executed, the execution time, and the output will differ when you run the tests using in-container mode as the Bean Validation TCK requires.
If the Maven reporting plugin that compliments Surefire is
configured properly, Maven will also generate a generic HTML test result
report. That report is written to the file test-report.html in the
target/surefire-reports
directory of the TCK runner
project. It shows how many tests were run, how many failed and the
success rate of the test run.
This chapter lays out how to configure the TCK harness by specifying
the DeployableContainer
, selecting the container
interaction style and setting various other switches. Maven 2 is used in the
configuration examples, mainly because it is the de facto standard build tool
at the time of writing. However, the test harness is not dependent on a
specific build tool and any other tool, for example Ant or Gradle, work as
well.
If you have not made yourself familiar with the Arquillian documentation by now, this is a good time to do it. It will give you a deeper understanding of the different parts described in the following sections.
If you followed all the steps of Chapter 3, Installation
you find the full pom.xml
used in the following
examples in the directory
jsr349/ri/tck-runner
.
The Bean Validation test harness is built atop TestNG, and it is TestNG that is responsible for selecting the tests to execute, the order of execution, and reporting the results. Detailed TestNG documentation can be found at testng.org.
The tck-tests.xml
artifact provided in the TCK
distribution must be run by TestNG (described by the TestNG documentation
as "with a testng.xml
file") unmodified for an
implementation to pass the TCK. For testing purposes it is of course ok to
modify the file.
<suite name="JSR-349-TCK" verbose="1"> <test name="JSR-349-TCK"> <method-selectors> <method-selector> <selector-class name="org.hibernate.beanvalidation.tck.util.IntegrationTestsMethodSelector"/> </method-selector> </method-selectors> <packages> <package name="org.hibernate.beanvalidation.tck.tests"/> </packages> </test> </suite>
TestNG provides extensive reporting information. Depending on the build tool or IDE you use, the reporting will take a different format. Please consult the TestNG documentation and the tool documentation for more information.
In the example below the
maven-dependency-plugin is used to extract the
tck-tests.xml
from the artifact jar file to make it
available for the surefire configuration.
The most important configuration you have make in order to run the
Bean Validation TCK is to specify your
ValidationProvider
you want to run your tests
against. To do so you need to set the Java system property
validation.provider to the fully specified class name
of your ValidationProvider
. In the examples below
this is done via the systemProperties configuration
of the maven-surefire-plugin. This property will be
picked up by
org.hibernate.beanvalidation.tck.util.TestUtil
which will instantiate the Validator
under test.
This means the test harness does not rely on the service provider
mechanism to instantiate the Bean Validation provider under test, partly
because this selection mechanism is under test as well.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>hibernate-validator-parent</artifactId>
<groupId>org.hibernate</groupId>
<version>5.0.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<artifactId>hibernate-validator-tck-runner</artifactId>
<name>Hibernate Validator TCK Runner</name>
<properties>
<tck.version>1.1.0-SNAPSHOT</tck.version>
<tck.suite.file>${project.build.directory}/dependency/beanvalidation-tck-suite.xml</tck.suite.file>
<arquillian.version>1.0.0.Final</arquillian.version>
<validation.provider>org.hibernate.validator.HibernateValidator</validation.provider>
<remote.debug/>
</properties>
<dependencies>
<dependency>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-validator</artifactId>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
</dependency>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>beanvalidation-tck</artifactId>
<version>${tck.version}</version>
</dependency>
<dependency>
<groupId>org.hibernate.javax.persistence</groupId>
<artifactId>hibernate-jpa-2.0-api</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.testng</groupId>
<artifactId>arquillian-testng-container</artifactId>
<version>${arquillian.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.jboss.arquillian.protocol</groupId>
<artifactId>arquillian-protocol-servlet</artifactId>
<version>${arquillian.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<testResources>
<testResource>
<filtering>true</filtering>
<directory>src/test/resources</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-tck-test-suite-file</id>
<phase>generate-test-sources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>beanvalidation-tck</artifactId>
<type>xml</type>
<classifier>suite</classifier>
<overWrite>false</overWrite>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<suiteXmlFiles>
<suiteXmlFile>${tck.suite.file}</suiteXmlFile>
</suiteXmlFiles>
<argLine>-Xmx128m</argLine>
<forkMode>once</forkMode>
<systemProperties>
<property>
<name>validation.provider</name>
<value>${validation.provider}</value>
</property>
</systemProperties>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<executions>
<execution>
<id>generate-test-report</id>
<phase>test</phase>
<goals>
<goal>report-only</goal>
</goals>
</execution>
</executions>
<configuration>
<outputDirectory>${project.build.directory}/surefire-reports</outputDirectory>
<outputName>test-report</outputName>
</configuration>
</plugin>
</plugins>
</build>
</project>
After setting the ValidationProvider
you have
to make a choice on the right DeployableContainer
.
Arquillian picks which container it is going to use to deploy the test
archive and negotiate test execution using Java's service provider
mechanism. Concretely Arquillian is looking for an implementation of the
DeployableContainer
SPI is on the classpath. In the
following examples this choice is made through the use of Maven profiles
adding the appropriate test dependencies.
Before trying to run the test harness against a real Java EE
container it is recommended to run the test in a local JVM mode. The
Bean Validation test harness ships with a
DeployableContainer
called
BeanValidationLocalContainer
which allows you to
do so. All you need is the right dependency:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<profiles>
<profile>
<id>local</id>
<activation>
<property>
<name>incontainer</name>
<value>!true</value>
</property>
</activation>
<properties>
<arquillian.protocol>Local</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>standalone-container-adapter</artifactId>
<version>${tck.version}</version>
</dependency>
</dependencies>
</profile>
</profiles>
</project>
As you can see the local
profile is activated by default. It adds the dependency
org.hibernate.beanvalidation.tck:standalone-container-adapter
which contains the DeployableContainer
BeanValidationLocalContainer
. If the test harness
is executed with this dependency on the classpath the tests will run in
the local JVM. The purpose of the property
arquillian.protocol will be discussed in Section 5.4, “arquillian.xml”. Running the test harness is as
easy as executing:
mvn test
The local execution mode is just a convenience for developers. To pass the Bean Validation TCK the test harness has to pass in a Java EE container. The reason for this is that Bean Validation is part of the Java EE specification.
Arquillian will throw an exception in case it finds more than
one DeployableContainer
implementations on
the classpath. If you want to be able to run in local as well as
incontainer mode from the same project setup you will need to
separate the dependencies. This is the main purpose of the profiles
used in this example.
Now let's try to run the tests in a Java EE container, namely JBoss AS 7. For this we defined a new profile incontainer which can be activated via:
> mvn test -Dincontainer
In this
case we are adding the dependency
org.jboss.as:jboss-as-arquillian-container-managed
which will enable a managed JBoss AS 7 as
DeployableContainer
. AS 7 could be already
installed on the system or as in this case be downloaded and unpacked
via the maven-dependency-plugin prior to the test
run.
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<profiles>
<profile>
<id>local</id>
<activation>
<property>
<name>incontainer</name>
<value>!true</value>
</property>
</activation>
<properties>
<arquillian.protocol>Local</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>standalone-container-adapter</artifactId>
<version>${tck.version}</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>incontainer</id>
<activation>
<property>
<name>incontainer</name>
</property>
</activation>
<properties>
<arquillian.protocol>Servlet 3.0</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-arquillian-container-managed</artifactId>
<version>${jbossas.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack</id>
<phase>process-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-dist</artifactId>
<version>${jbossas.version}</version>
<type>zip</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
<execution>
<id>copy-beanvalidation-api</id>
<phase>generate-test-sources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/jboss-as-${jbossas.version}/modules/javax/validation/api/main
</outputDirectory>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx1024m -Djava.util.logging.manager=org.jboss.logmanager.LogManager</argLine>
<forkMode>once</forkMode>
<suiteXmlFiles>
<suiteXmlFile>${tck.suite.file}</suiteXmlFile>
</suiteXmlFiles>
<systemPropertyVariables>
<arquillian.launch>incontainer</arquillian.launch>
<validation.provider>${validation.provider}</validation.provider>
<includeIntegrationTests>true</includeIntegrationTests>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
You find a list of available deployable containers in the Arquillian documentation. Refer to Container adapters.
Some of the tests can only run within a container (mainly due to
the way they are setup). These tests are annotated with
@IntegrationTest. Since these tests would fail in local execution mode
they can be excluded by setting the system property
includeIntegrationTests to
false
(or not specifying it at all). To pass the
TCK the property needs to be set to true
and all
integration tests need to run.
The in- an exclusion of integration tests is handled via a
TestNG specific IMethodSelector
, more
concretely IntegrationTestsMethodSelector
.
The easiest way to debug the tests when they are executes in a remote container is to open a remote debugging session. In order to do so the JVM needs to be started with the right remote debugging options. Again this can be achieved via a profile:
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
...
<profiles>
<profile>
<id>local</id>
<activation>
<property>
<name>incontainer</name>
<value>!true</value>
</property>
</activation>
<properties>
<arquillian.protocol>Local</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.hibernate.beanvalidation.tck</groupId>
<artifactId>standalone-container-adapter</artifactId>
<version>${tck.version}</version>
</dependency>
</dependencies>
</profile>
<profile>
<id>incontainer-debug</id>
<activation>
<property>
<name>debug</name>
</property>
</activation>
<properties>
<remote.debug>-Xnoagent -Djava.compiler=NONE -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=5005</remote.debug>
</properties>
</profile>
<profile>
<id>incontainer</id>
<activation>
<property>
<name>incontainer</name>
</property>
</activation>
<properties>
<arquillian.protocol>Servlet 3.0</arquillian.protocol>
</properties>
<dependencies>
<dependency>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-arquillian-container-managed</artifactId>
<version>${jbossas.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>unpack</id>
<phase>process-test-resources</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.jboss.as</groupId>
<artifactId>jboss-as-dist</artifactId>
<version>${jbossas.version}</version>
<type>zip</type>
<overWrite>false</overWrite>
<outputDirectory>${project.build.directory}</outputDirectory>
</artifactItem>
</artifactItems>
</configuration>
</execution>
<execution>
<id>copy-beanvalidation-api</id>
<phase>generate-test-sources</phase>
<goals>
<goal>copy</goal>
</goals>
<configuration>
<outputDirectory>
${project.build.directory}/jboss-as-${jbossas.version}/modules/javax/validation/api/main
</outputDirectory>
<stripVersion>true</stripVersion>
<artifactItems>
<artifactItem>
<groupId>javax.validation</groupId>
<artifactId>validation-api</artifactId>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<argLine>-Xmx1024m -Djava.util.logging.manager=org.jboss.logmanager.LogManager</argLine>
<forkMode>once</forkMode>
<suiteXmlFiles>
<suiteXmlFile>${tck.suite.file}</suiteXmlFile>
</suiteXmlFiles>
<systemPropertyVariables>
<arquillian.launch>incontainer</arquillian.launch>
<validation.provider>${validation.provider}</validation.provider>
<includeIntegrationTests>true</includeIntegrationTests>
</systemPropertyVariables>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>
In this you would start the tests via (activation order is important):
mvn test -Ddebug -Dincontainer
The test execution would start but then be suspended until the JVM receives a remote debugging request on port 5005.
The next piece in the configuration puzzle is
arquillian.xml
. This xml file needs to be in the root
of the classpath and is used to pass additional options to the selected
container. Let's look at an example:
<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://jboss.org/schema/arquillian http://jboss.org/schema/arquillian/arquillian_1_0.xsd"> <!-- Need to set the default protocol and use resource filtering, because of https://issues.jboss.org/browse/ARQ-579 --> <defaultProtocol type="${arquillian.protocol}"/> <engine> <property name="deploymentExportPath">target/artifacts</property> </engine> <container qualifier="local" default="true"> <protocol type="Local"/> <!-- Takes no effect - ARQ-579 --> </container> <container qualifier="incontainer"> <protocol type="Servlet 3.0"/> <!-- Takes no effect - ARQ-579 --> <configuration> <property name="jbossHome">${jbossTargetDir}</property> <property name="javaVmArguments">-Xmx1024m -XX:MaxPermSize=512m ${remote.debug} -Dvalidation.provider=${validation.provider}</property> <property name="allowConnectingToRunningServer">true</property> </configuration> </container> </arquillian>
In this case we have two container nodes, local and incontainer. The selection of which configuration to chose at runtime is made by setting the system property arquillian.launch. If arquillian.launch is not specify the default is selected which in this case is local.
The most important container configuration option is the protocol
type which determines how Arquillian communicates with the selected
container. The most popular types are Local and
Servlet 3.0. The protocol can be defined per
container node, however, due to a bug only the
defaultProtocol setting takes effect. This is also
the reason for the arquillian.protocol property used
in the pom examples. Depending on the chosen profile the appropriate
protocol is selected and added to arquillian.xml
via
resource
filtering.
Another interesting property is deploymentExportPath which is optional and instructs Arquillian to dump the test artifacts to the specified directory on disk. Inspection of the deployed artifacts can be very useful when debugging test failures.
One of the requirements of an implementation passing the TCK is for it to pass the Bean Validation signature test. This section describes how the signature file is generated and how to run it against your implementation.
You can obtain the Sigtest tool (at the time of writing the TCK uses version 2.1) from the Sigtest home page at https://sigtest.dev.java.net/. The user guide can be found at http://download.oracle.com/javame/test-tools/sigtest/2_1/sigtest2.1_usersguide.pdf. The downloadable package contains the jar files used in the commands below.
The TCK package contains the files
validation-api-java5.sig
,
validation-api-java6.sig
and
validation-api-java7.sig
and (in the
artifacts
directory) which were created using the
following command (using the corresponding Java version):
java -jar sigtestdev.jar Setup -classpath $JAVA_HOME/jre/lib/rt.jar:lib/validation-api-1.1.0.Final.jar -package javax.validation -filename validation-api-java6.sig
In order to pass the Bean Validation TCK you have to make sure that
your API passes the signature tests against
validation-api.sig.
To run the signature test use:
java -jar sigtest.jar Test -classpath $JAVA_HOME/jre/lib/rt.jar:lib/validation-api-1.1.0.Final.jar -static -package javax.validation -filename validation-api-java6.sig
You have to chose the right version of the signature file depending
on your Java version. In order to run against your own Bean Validation API
replace validation-api-1.1.0.Final.jar with your own API jar. You
should get the message "STATUS:Passed."
.
Just for fun (and to confirm that the signature test is working correctly), you can try the following:
1) Edit validation-api.sig
2) Modify one of the class signatures - in the following example we
change one of the constructors for ValidationException
- here's the original:
CLSS public javax.validation.ValidationException cons public ValidationException() cons public ValidationException(java.lang.String) cons public ValidationException(java.lang.String,java.lang.Throwable) cons public ValidationException(java.lang.Throwable) supr java.lang.RuntimeException
Let's change the default (empty) constructor parameter to one with a
java.lang.Integer
parameter instead:
CLSS public javax.validation.ValidationException cons public ValidationException(java.lang.Integer) cons public ValidationException(java.lang.String) cons public ValidationException(java.lang.String,java.lang.Throwable) cons public ValidationException(java.lang.Throwable) supr java.lang.RuntimeException
3) Now when we run the signature test using the above command, we should get the following errors:
Missing Constructors -------------------- javax.validation.ValidationException: constructor public javax.validation.ValidationException.ValidationException(java.lang.Integer) Added Constructors ------------------ javax.validation.ValidationException: constructor public javax.validation.ValidationException.ValidationException() STATUS:Failed.2 errors
Copyright © 2009 - 2012 Red Hat, Inc.