<?xml version="1.0" encoding="UTF-8"?>
<chapter id="ch.core-persistence"
         version="5.0"
         xsi:schemaLocation="http://docbook.org/ns/docbook http://www.docbook.org/xml/5.0/xsd/docbook.xsd http://www.w3.org/1999/xlink http://www.docbook.org/xml/5.0/xsd/xlink.xsd"
         xml:base="../" xmlns="http://docbook.org/ns/docbook" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xlink="http://www.w3.org/1999/xlink"
         xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:ns="http://docbook.org/ns/docbook">

  <title>Core Engine: Persistence and transactions</title>

  <para>jBPM allows the persistent storage of certain information. This chapter 
  describes these different types of persistence, and how to configure them.
  An example of the information stored is the process runtime state. Storing
  the process runtime state is necessary in order to be able to continue execution 
  of a process instance at any point, if something goes wrong. Also, the process 
  definitions themselves, and the history information (logs of current and previous 
  process states already) can also be persisted. 
  </para>

  <section>
    <title>Runtime State</title>

    <para>Whenever a process is started, a process instance is created, which
    represents the execution of the process in that specific context.  For example,
    when executing a process that specifies how to process a sales order, one process
    instance is created for each sales request.  The process instance represents
    the current execution state in that specific context, and contains all the
    information related to that process instance.  Note that it only contains
    the minimal runtime state that is needed to continue the execution of that
    process instance at some later time, but it does not include information
    about the history of that process instance if that information is no longer
    needed in the process instance.</para>

    <para>The runtime state of an executing process can be made persistent,
    for example, in a database.  This allows to restore the state of
    execution of all running processes in case of unexpected failure,
    or to temporarily remove running instances from memory and restore
    them at some later time.  jBPM allows you to plug in different
    persistence strategies.  By default, if you do not configure the
    process engine otherwise, process instances are not made persistent.</para>

    <para>If you configure the engine to use persistence, it will automatically store
    the runtime state into the database.  You do not have to trigger persistence yourself,
    the engine will take care of this when persistence is enabled.  Whenever you invoke the
    engine, it will make sure that any changes are stored at the end of that invocation, at
    so-called safe points.  Whenever something goes wrong and you restore the engine from
    the database, you also should not reload the process instances and trigger them manually
    to resume execution, as process instances will automatically resume execution if they are
    triggered, like for example by a timer expiring, the completion of a task that was 
    requested by that process instance, or a signal being sent to the process instance.  The
    engine will automatically reload process instances on demand.</para>
    
    <para>The runtime persistence data should in general be considered internal, meaning that you
    probably should not try to access these database tables directly and especially not try
    to modify these directly (as changing the runtime state of process instances without the
    engine knowing might have unexpected side-effects).  In most cases where information about
    the current execution state of process instances is required, the use of a history log
    is mostly recommended (see below).  In some cases, it might still be useful to for example
    query the internal database tables directly, but you should only do this if you know what
    you are doing.</para>

    <section>
      <title>Binary Persistence</title>

    <para>jBPM uses a binary persistence mechanism, otherwise known
    as marshalling, which converts the state of the process instance
    into a binary dataset. When you use persistence with jBPM, this 
    mechanism is used to save or retrieve the process instance state 
    from the database. The same mechanism is also applied to the 
    session state and any work item states.</para> 

    <para>When the process instance state is persisted, two things
    happen: 
    <itemizedlist>
      <listitem>First, the process instance information is transformed 
      into a binary blob. For performance reasons, a custom serialization 
      mechanism is used and not normal Java serialization.</listitem>
      <listitem>This blob is then stored, alongside other metadata about 
      this process instance. This metadata includes, among other things, 
      the process instance id, process id, and the process start date.</listitem>
    </itemizedlist>
    </para>

    <para>Apart from the process instance state, the session itself can 
    also store some state, such as the state of timer jobs, or the session 
    data that the any business rules would be evaluated over.  
    This session state is stored separately as a binary blob, along with 
    the id of the session and some metadata. You can always restore session 
    state by reloading the session with the given id.  The session id can 
    be retrieved using <code>ksession.getId()</code>.</para>

    <para>Note that the process instance binary datasets are usually 
    relatively small, as they only contain the minimal execution state 
    of the process instance.  For a simple process instance, this usually 
    contains one or a few node instances, i.e., any node that is currently 
    executing, and any existing variable values.</para>
    
    <para>As a result of jBPM using marshalling, the data model is both 
    simple and small:</para>

    <!-- width && scalefit for pdf's -->
    <ulink url="images/Chapter-Persistence/jbpm_schema.png">
      <figure>
        <title>jBPM data model</title>
        <mediaobject>
          <imageobject>
            <imagedata fileref="images/Chapter-Persistence/jbpm_schema_doc.png" format="PNG" 
                       align="center" width="100%" scalefit="1" />
          </imageobject>
        </mediaobject>
      </figure>
    </ulink>

    <para>The <code>sessioninfo</code> entity contains the state of the 
    (knowledge) session in which the jBPM process instance is running.</para>

    <table>
      <title>SessionInfo</title>
      <tgroup cols='3' align='left' colsep='1' rowsep='1'>
      <thead>
        <row>
          <entry>Field</entry>
          <entry>Description</entry>
          <entry>Nullable</entry>
        </row>
      </thead>
      <tbody>
        <row>
          <entry><code>id</code></entry>
          <entry>The primary key.</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>lastmodificationdate</code></entry>
          <entry>The last time that the entity was saved to the database</entry>
        </row>
        <row>
          <entry><code>rulesbytearray</code></entry>
          <entry>The binary dataset containing the state of the session</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>startdate</code></entry>
          <entry>The start time of the session</entry>
        </row>
        <row>
          <entry><code>optlock</code></entry>
          <entry>The version field that serves as its optimistic lock value</entry>
        </row>
      </tbody>
      </tgroup>
    </table>
    
    <para>The <code>processinstanceinfo</code> entity contains the state 
    of the jBPM process instance.</para>

    <table>
      <title>ProcessInstanceInfo</title>
      <tgroup cols='3' align='left' colsep='1' rowsep='1'>
      <thead>
        <row>
          <entry>Field</entry>
          <entry>Description</entry>
          <entry>Nullable</entry>
        </row>
      </thead>
      <tbody>
        <row>
          <entry><code>instanceid</code></entry>
          <entry>The primary key</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>lastmodificationdate</code></entry>
          <entry>The last time that the entity was saved to the database</entry>
        </row>
        <row>
          <entry><code>lastreaddate</code></entry>
          <entry>The last time that the entity was retrieved (read) from the database</entry>
        </row>
        <row>
          <entry><code>processid</code></entry>
          <entry>The name (id) of the process</entry>
        </row>
        <row>
          <entry><code>processinstancebytearray</code></entry>
          <entry>This is the binary dataset containing the state of the process instance</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>startdate</code></entry>
          <entry>The start time of the process</entry>
        </row>
        <row>
          <entry><code>state</code></entry>
          <entry>An integer representing the state of the process instance</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>optlock</code></entry>
          <entry>The version field that serves as its optimistic lock value</entry>
        </row>
      </tbody>
      </tgroup>
    </table>

    <para>The <code>eventtypes</code> entity contains information
    about events that a process instance will undergo or has undergone.</para>

    <table>
      <title>EventTypes</title>
      <tgroup cols='3' align='left' colsep='1' rowsep='1'>
      <thead>
        <row>
          <entry>Field</entry>
          <entry>Description</entry>
          <entry>Nullable</entry>
        </row>
      </thead>
      <tbody>
        <row>
          <entry><code>instanceid</code></entry>
          <entry>This references the <code>processinstanceinfo</code> primary 
          key and there is a foreign key constraint on this column.</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>element</code></entry>
          <entry>A text field related to an event that the process has undergone.</entry>
        </row>
      </tbody>
      </tgroup>
    </table>

    <para>The <code>workiteminfo</code> entity contains the state of a work item.</para>

    <table>
      <title>WorkItemInfo</title>
      <tgroup cols='3' align='left' colsep='1' rowsep='1'>
      <thead>
        <row>
          <entry>Field</entry>
          <entry>Description</entry>
          <entry>Nullable</entry>
        </row>
      </thead>
      <tbody>
        <row>
          <entry><code>workitemid</code></entry>
          <entry>The primary key</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>name</code></entry>
          <entry>The name of the work item</entry>
        </row>
        <row>
          <entry><code>processinstanceid</code></entry>
          <entry>The (primary key) id of the process: there is no foreign key constraint 
          on this field.</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>state</code></entry>
          <entry>An integer representing the state of the work item</entry>
          <entry>NOT NULL</entry>
        </row>
        <row>
          <entry><code>optlock</code></entry>
          <entry>The version field that serves as its optimistic lock value</entry>
        </row>
        <row>
          <entry><code>workitembytearay</code></entry>
          <entry>This is the binary dataset containing the state of the work item</entry>
          <entry>NOT NULL</entry>
        </row>
      </tbody>
      </tgroup>
    </table>

    </section>

    <section>
      <title>Safe Points</title>

    <para>The state of a process instance is stored at so-called "safe points"
    during the execution of the process engine.  Whenever a process instance
    is executing (for example when it started or continuing from a previous wait state,
    the engine executes the process instance until no more actions can be performed (meaning
    that the process instance either has completed (or was aborted), or that it has reached a
    wait state in all of its parallel paths). At that point, the engine
    has reached the next safe state, and the state of the process instance
    (and all other process instances that might have been affected) is stored
    persistently.</para>
    </section>

    <section>
      <title>Configuring Persistence</title>

    <para>By default, the engine does not save runtime data persistently.  This
    means you can use the engine completely without persistence (so not even
    requiring an in memory database) if necessary, for example for performance
    reasons, or when you would like to manage persistence yourself.  It is, however,
    possible to configure the engine to do use persistence by configuring it to do so.
    This usually requires adding the necessary dependencies, configuring a datasource
    and creating the engine with persistence configured.</para>
    
    <section>
      <title>Adding dependencies</title>

    <para>You need to make sure the necessary dependencies are available in the classpath
    of your application if you want to user persistence.  By default,
    persistence is based on the Java Persistence API (JPA) and can
    thus work with several persistence mechanisms.  We are using Hibernate by
    default.</para>

    <para>If you're using the Eclipse IDE and the jBPM Eclipse plugin, you should make sure
    the necessary jars are added to your jBPM runtime directory.  You don't really need to do
    anything (as the necessary dependencies should already be there) if you are using the jBPM
    runtime that is configured by default when using the jBPM installer, or if you downloaded
    and unzipped the jBPM runtime artefact (from the downloads) and pointed the jBPM plugin to
    that directory.</para>
    
    <para>If you would like to manually add the necessary dependencies to your project, first of all,
    you need the jar file <filename>jbpm-persistence-jpa.jar</filename>,
    as that contains code for saving the runtime state whenever necessary.
    Next, you also need various other dependencies, depending on the
    persistence solution and database you are using.  For the default
    combination with Hibernate as the JPA persistence provider and using an H2
    in-memory database and Bitronix for JTA-based transaction management, the
    following list of additional dependencies is needed:
      <itemizedlist>
        <listitem>jbpm-test (org.jbpm)</listitem>
        <listitem>jbpm-persistence-jpa (org.jbpm)</listitem>
        <listitem>drools-persistence-jpa (org.drools)</listitem>
        <listitem>persistence-api (javax.persistence)</listitem>
        <listitem>hibernate-entitymanager (org.hibernate)</listitem>
        <listitem>hibernate-annotations (org.hibernate)</listitem>
        <listitem>hibernate-commons-annotations (org.hibernate)</listitem>
        <listitem>hibernate-core (org.hibernate)</listitem>
        <listitem>commons-collections (commons-collections)</listitem>
        <listitem>dom4j (dom4j)</listitem>
        <listitem>jta (javax.transaction)</listitem>
        <listitem>btm (org.codehaus.btm)</listitem>
        <listitem>javassist (javassist)</listitem>
        <listitem>slf4j-api (org.slf4j)</listitem>
        <listitem>slf4j-jdk14 (org.slf4j)</listitem>
        <listitem>h2 (com.h2database)</listitem>
      </itemizedlist>
    </para>
    
    </section>

    <section>
      <title>Configuring the engine to use persistence using <code>JBPMHelper</code></title>

    <para>You need to configure the jBPM engine to use persistence, usually simply by using
    the appropriate constructor when creating your session.  There are various ways to create
    a session (as we have tried to make this as easy as possible for you and have several
    utility classes for you, depending for example if you are trying to write a process junit
    test).</para>
    
    <para>The easiest way to do this is to use the <code>jbpm-test</code> module that allows you to easily
    create and test your processes.  The <code>JBPMHelper</code> class has a method to create a session,
    and uses a configuration file to configure this session, like whether you want to use persistence,
    the datasource to use, etc.  The helper class will then do all the setup and configuration
    for you.</para>
    
    <para>To configure persistence, create a <filename>jBPM.properties</filename> file and configure the following properties
    (note that the example below are the default properties, using an H2 in-memory database with persistence
    enables, if you are fine with all of these properties, you don't need to add new properties file, as it
    will then use these properties by default):</para>
    
    <programlisting># for creating a datasource
persistence.datasource.name=jdbc/jbpm-ds
persistence.datasource.user=sa
persistence.datasource.password=
persistence.datasource.url=jdbc:h2:tcp://localhost/~/jbpm-db
persistence.datasource.driverClassName=org.h2.Driver

# for configuring persistence of the session
persistence.enabled=true
persistence.persistenceunit.name=org.jbpm.persistence.jpa
persistence.persistenceunit.dialect=org.hibernate.dialect.H2Dialect

# for configuring the human task service
taskservice.enabled=true
taskservice.datasource.name=org.jbpm.task
taskservice.transport=mina
taskservice.usergroupcallback=org.jbpm.task.service.DefaultUserGroupCallbackImpl
    </programlisting>
    
    <para>If you want to use persistence, you must make sure that the datasource (that you specified
    in the <filename>jBPM.properties</filename> file) is initialized correctly.  This means that the database itself must
    be up and running, and the datasource should be registered using the correct name.  If you would like
    to use an H2 in-memory database (which is usually very easy to do some testing), you can use the 
    <code>JBPMHelper</code> class to start up this database, using:</para>
    
    <programlisting language="java">JBPMHelper.startH2Server();
    </programlisting>

    <para>To register the datasource (this is something you always need to do, even if you're not using H2
    as your database, check below for more options on how to configure your datasource), use:</para>
    
    <programlisting language="java">JBPMHelper.setupDataSource();
    </programlisting>

    <para>Next, you can use the <code>JBPMHelper</code> class to create your session (after creating your knowledge base,
    which is identical to the case when you are not using persistence):</para>
    
    <programlisting language="java">StatefulKnowledgeSession ksession = JBPMHelper.newStatefulKnowledgeSession(kbase);
    </programlisting>
    
    <para>Once you have done that, you can just call methods on this ksession (like <code>startProcess</code>)
    and the engine will persist all runtime state in the created datasource.</para>

    <para>You can also use the <code>JBPMHelper</code> class to recreate your session (by restoring its state
    from the database, by passing in the session id (that you can retrieve using <code>ksession.getId()</code>)):</para>

    <programlisting language="java">StatefulKnowledgeSession ksession = 
    JBPMHelper.loadStatefulKnowledgeSession(kbase, sessionId);
    </programlisting>
    
    </section>
    
    <section>
      <title>Manually configuring the engine to use persistence</title>
    
    <para>You can also use the <code>JPAKnowledgeService</code> to create your knowledge session.  This
    is slightly more complex, but gives you full access to the underlying configurations.  You can create
    a new knowledge session using <code>JPAKnowledgeService</code> based on a knowledge base, a
    knowledge session configuration (if necessary) and an environment.  The environment 
    needs to contain a reference to your Entity Manager Factory.  For example:</para>

    <programlisting language="java">// create the entity manager factory and register it in the environment
EntityManagerFactory emf =
    Persistence.createEntityManagerFactory( "org.jbpm.persistence.jpa" );
Environment env = KnowledgeBaseFactory.newEnvironment();
env.set( EnvironmentName.ENTITY_MANAGER_FACTORY, emf );

// create a new knowledge session that uses JPA to store the runtime state
StatefulKnowledgeSession ksession =
    JPAKnowledgeService.newStatefulKnowledgeSession( kbase, null, env );
int sessionId = ksession.getId();

// invoke methods on your method here
ksession.startProcess( "MyProcess" );
ksession.dispose();</programlisting>

    <para>You can also use the <code>JPAKnowledgeService</code> to recreate
    a session based on a specific session id:</para>

    <programlisting language="java">
// recreate the session from database using the sessionId
ksession = JPAKnowledgeService.loadStatefulKnowledgeSession(
    sessionId, kbase, null, env );</programlisting>

    <!--
    <para>You can also configure your engine to use persistence by simply specifying this in your session
    configuration.  There are various ways to do this, but using a simple drools.session.conf file in a
    META-INF directory on your classpath is probably the easiest way (check the documentation for other
    ways of configuring your session, for example by providing a KnowledgeSessionConfiguration when first creating your
    session).  In this config file you need to do two things: tell the engine that it needs to use
    a command service underneath (as commands are used to determine safe points during the execution of
    the engine), and use the JPA-based implementations of 3 internal components (the process instance
    manager, the work item manager and the signal manager), as these components will then be able to
    look up the necessary information from persistence using JPA.  The drools.session.conf file should
    thus look like this:</para>

    <programlisting>
drools.commandService = org.drools.persistence.session.SingleSessionCommandService
drools.processInstanceManagerFactory = org.drools.persistence.processinstance.JPAProcessInstanceManagerFactory
drools.workItemManagerFactory = org.drools.persistence.processinstance.JPAWorkItemManagerFactory
drools.processSignalManagerFactory = org.drools.persistence.processinstance.JPASignalManagerFactory</programlisting>
    -->

    <para>Note that we only save the minimal state that is needed to continue
    execution of the process instance at some later point.  This means, for example, that
    it does not contain information about already executed nodes if that information
    is no longer relevant, or that process instances that have been completed or
    aborted are removed from the database.  If you want to search for history-related
    information, you should use the history log, as explained later.</para>

    <para>You need to add a persistence configuration to your classpath to 
    configure JPA to use Hibernate and the H2 database (or your own preference), called
    <filename>persistence.xml</filename> in the META-INF directory, as shown below.
    For more details on how to change this for your own configuration, we refer to
    the JPA and Hibernate documentation for more information.</para>

    <programlisting language="xml">&lt;?xml version="1.0" encoding="UTF-8" standalone="yes"?&gt;
&lt;persistence
  version="1.0"
  xsi:schemaLocation=
    "http://java.sun.com/xml/ns/persistence
     http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd
     http://java.sun.com/xml/ns/persistence/orm
     http://java.sun.com/xml/ns/persistence/orm_1_0.xsd"
  xmlns:orm="http://java.sun.com/xml/ns/persistence/orm"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="http://java.sun.com/xml/ns/persistence"&gt;

  &lt;persistence-unit name="org.jbpm.persistence.jpa" transaction-type="JTA"&gt;
    &lt;provider&gt;org.hibernate.ejb.HibernatePersistence&lt;/provider&gt;
    &lt;jta-data-source&gt;jdbc/jbpm-ds&lt;/jta-data-source&gt;
    &lt;mapping-file&gt;META-INF/JBPMorm.xml&lt;/mapping-file&gt;
    &lt;class&gt;org.drools.persistence.info.SessionInfo&lt;/class&gt;
    &lt;class&gt;org.jbpm.persistence.processinstance.ProcessInstanceInfo&lt;/class&gt;
    &lt;class&gt;org.drools.persistence.info.WorkItemInfo&lt;/class&gt;

    &lt;properties&gt;
      &lt;property name="hibernate.dialect" value="org.hibernate.dialect.H2Dialect"/&gt;
      &lt;property name="hibernate.max_fetch_depth" value="3"/&gt;
      &lt;property name="hibernate.hbm2ddl.auto" value="update"/&gt;
      &lt;property name="hibernate.show_sql" value="true"/&gt;
      &lt;property name="hibernate.transaction.manager_lookup_class"
                value="org.hibernate.transaction.BTMTransactionManagerLookup"/&gt;
    &lt;/properties&gt;
  &lt;/persistence-unit&gt;
&lt;/persistence&gt;</programlisting>

    <para>This configuration file refers to a data source called "jdbc/jbpm-ds".
    If you run your application in an application server (like for example JBoss AS),
    these containers typically allow you to easily set up data sources using some
    configuration (like for example dropping a datasource configuration file in the
    deploy directory).  Please refer to your application server documentation to know
    how to do this.</para>
     
    <para>For example, if you're deploying to JBoss Application Server v5.x, you can 
    create a datasource by dropping a configuration file in the deploy directory, 
    for example:</para>
    
    <programlisting language="xml">&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;datasources&gt;
  &lt;local-tx-datasource&gt;
    &lt;jndi-name&gt;jdbc/jbpm-ds&lt;/jndi-name&gt;
    &lt;connection-url&gt;jdbc:h2:tcp://localhost/~/test&lt;/connection-url&gt;
    &lt;driver-class&gt;org.h2.jdbcx.JdbcDataSource&lt;/driver-class&gt;
    &lt;user-name&gt;sa&lt;/user-name&gt;
    &lt;password&gt;&lt;/password&gt;
  &lt;/local-tx-datasource&gt;
&lt;/datasources&gt;</programlisting>

    <para>If you are however executing in a simple Java environment, you can use the 
    <code>JBPMHelper</code> class to do this for you (see above) or the following code
    fragment could be used to set up a data source (where we are using the H2 in-memory 
    database in combination with Bitronix in this case).</para>

    <programlisting language="java">PoolingDataSource ds = new PoolingDataSource();
ds.setUniqueName("jdbc/jbpm-ds");
ds.setClassName("bitronix.tm.resource.jdbc.lrc.LrcXADataSource");
ds.setMaxPoolSize(3);
ds.setAllowLocalTransactions(true);
ds.getDriverProperties().put("user", "sa");
ds.getDriverProperties().put("password", "sasa");
ds.getDriverProperties().put("URL", "jdbc:h2:tcp://localhost/~/jbpm-db");
ds.getDriverProperties().put("driverClassName", "org.h2.Driver");
ds.init();</programlisting>

    <!--para>After adding the necessary dependencies and the configuration file to your project, you can simply
    use the StatelessKnowledgeSession just the way you used to do.  The engine will underneath translate your
    invocations to commands that will persist the state of the engine after each successful execution of a
    command.  For example, the following code snippet shows how to create a session and start a process.  Note
    that this snippet does not show anything about persistence (as the config file configures all that for you,
    and the engine takes care of it automatically), so the example looks just like normal Drools code.  You
    can however always destroy your session and create a new one (or one session could continue the work
    that was started in another session), as the runtime state is persisted safely in a database, and can be
    retrieved whenever necessary.</para>

    <programlisting language="java">
  StatefulKnowledgeSession session = kbase.newStatefulKnowledgeSession();
  long processInstanceId = session.startProcess("org.drools.test.TestProcess").getId();
  session.dispose();</programlisting-->
    </section>

    </section>

    <section>
      <title>Transactions</title>

    <para>The jBPM engine supports JTA transactions. It also supports local transactions 
    <emphasis>only</emphasis> when using Spring. It does not support pure local transactions 
    at the moment. For more information about using Spring to set up persistence, please 
    see the Spring chapter in the Drools integration guide.</para>

    <para>Whenever you do not provide transaction boundaries inside your application,
    the engine will automatically execute each method invocation on the engine
    in a separate transaction.  If this behavior is acceptable, you don't need to do
    anything else.  You can, however, also specify the transaction boundaries yourself.
    This allows you, for example, to combine multiple commands into one transaction.</para>

    <para>You need to register a transaction manager at the environment before
    using user-defined transactions.  The following sample code uses the Bitronix
    transaction manager.  Next, we use the Java Transaction API (JTA)
    to specify transaction boundaries, as shown below:</para>

    <programlisting language="java">// create the entity manager factory and register it in the environment
EntityManagerFactory emf =
    Persistence.createEntityManagerFactory( "org.jbpm.persistence.jpa" );
Environment env = KnowledgeBaseFactory.newEnvironment();
env.set( EnvironmentName.ENTITY_MANAGER_FACTORY, emf );
env.set( EnvironmentName.TRANSACTION_MANAGER,
         TransactionManagerServices.getTransactionManager() );

// create a new knowledge session that uses JPA to store the runtime state
StatefulKnowledgeSession ksession =
    JPAKnowledgeService.newStatefulKnowledgeSession( kbase, null, env );

// start the transaction
UserTransaction ut =
  (UserTransaction) new InitialContext().lookup( "java:comp/UserTransaction" );
ut.begin();

// perform multiple commands inside one transaction
ksession.insert( new Person( "John Doe" ) );
ksession.startProcess( "MyProcess" );

// commit the transaction
ut.commit();</programlisting>

    <para>Note that, if you use Bitronix as the transaction manager, you should also add
    a simple <filename>jndi.properties</filename> file in you root classpath to register the
    Bitronix transaction manager in JNDI.  If you are using the jbpm-test module, this is
    already included by default.  If not, create a file named <filename>jndi.properties</filename>
    with the following content:</para>

    <programlisting>java.naming.factory.initial=bitronix.tm.jndi.BitronixInitialContextFactory</programlisting>

    <para>If you would like to use a different JTA transaction manager, you can change the
    <filename>persistence.xml</filename> file to use your own transaction manager.  For example,
    when running inside JBoss Application Server v5.x, you can use the JBoss transaction manager.
    You need to change the transaction manager property in <filename>persistence.xml</filename> to:</para>
    
    <programlisting>&lt;property name="hibernate.transaction.manager_lookup_class"
             value="org.hibernate.transaction.JBossTransactionManagerLookup" /&gt;
    </programlisting>

      <section>
        <title>Container managed transaction</title>
        
        <para>Special consideration need to be taken when embedding jBPM inside an application that
        exucutes in Container Managed Tansaction (CMT) mode, for instance EJB beans. This especially 
        applies to application servers that does not allow accessing UserTransaction instance from
        JNDI when being part of container managed transation, e.g. WebSphere Application Server.
        Since default implementation of transaction manager in jBPM is based on UserTransaction 
        to get transaction status which is used to decide if transaction should be started or not, 
        in environments that prevent accessing UserTrancation it won't do its job. 
        To secure proper execution in CMT environments a dedicated transaction manager implementation
        is provided:</para>
        <programlisting>org.jbpm.persistence.jta.ContainerManagedTransactionManager</programlisting>
        
        <para>This transaction manager expects that transaction is active and thus will always
        return ACTIVE when invoking getStatus method. Operations like begin, commit, rollback
        are no-op methods as transaction manager runs under managed transaction and can't affect it.
        </para>
        <note>
            <para>To make sure that container is aware of any exceptions that happened during
            process instance execution, user needs to ensure that exceptions thrown by the 
            engine are propagated up to the container to properly rollback transaction.</para>
        </note>
        
        <para>To configure this transaction manager following must be done:</para>
        <itemizedlist>
        <listitem>Insert transaction manager and peristence context manager into environment prior
        to creating/loading session
        <programlisting>
        Environment env = EnvironmentFactory.newEnvironment();
        env.set(EnvironmentName.ENTITY_MANAGER_FACTORY, emf);
        env.set(EnvironmentName.TRANSACTION_MANAGER, new ContainerManagedTransactionManager());
        env.set(EnvironmentName.PERSISTENCE_CONTEXT_MANAGER, new JpaProcessPersistenceContextManager(env));
        </programlisting>
        </listitem>
        <listitem>configure JPA provider (example hibernate and WebSphere)
        <programlisting>
        &lt;property name="hibernate.transaction.factory_class" value="org.hibernate.transaction.CMTTransactionFactory"/&gt;
        &lt;property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.WebSphereExtendedJTATransactionLookup"/&gt;
        </programlisting>
        </listitem>
        </itemizedlist>
        
        <para>With following configuration jBPM should run properly in CMT environment.</para>
      </section>
    </section>

    <section>
      <title>Persistence and concurrency</title>

      <para>Please see the <emphasis>Multi-threading</emphasis> section of
      <quote><link linkend='ch.core-basics' endterm="ch.core-basics.title"/></quote>
      for more information.</para>

    </section>

  </section>

  <section>
    <title>Process Definitions</title>

    <para>Process definition files are usually written in an XML format.  These
    files can easily be stored on a file system during development.  However,
    whenever you want to make your knowledge accessible to one or more engines
    in production, we recommend using a knowledge repository that (logically)
    centralizes your knowledge in one or more knowledge repositories.</para>

    <para>Guvnor is a Drools sub-project that does exactly that. It 
    consists of a repository for storing different kinds of knowledge, as 
    well a web application that allows users to view and update the information 
    in the repository. It not only stores process definitions but also 
    can hold rule definitions, object models, and much more.</para>
    
    <para>Easy programmatic retrieval of knowledge packages is possible 
    either using WebDAV or by using a knowledge agent. The knowledge
    agent will automatically download the information from Guvnor, for example, 
    during the creation of a knowledge base.</para>

    <para>Check out the Drools Guvnor documentation for more information on how to
    do this.</para>

  </section>

  <section>
    <title>History Log</title>

    <para>In many cases it will be useful (if not necessary) to store information
    <emphasis>about</emphasis> the execution of process instances, so that 
    this information can be used afterwards. For example, sometimes we 
    want to verify which actions have been executed for a particular process 
    instance, or in general, we want to be able to monitor and analyze 
    the efficiency of a particular process.</para>
    
    <para>However, storing history information in the runtime database 
    can result in the database rapidly increasing in size, not to mention 
    the fact that monitoring and analysis queries might influence the performance 
    of your runtime engine. This is why process execution history information 
    can be stored separately.</para>

    <para>This history log of execution information is created based on 
    events that the the process engine generates during execution. This is 
    possible because the jBPM runtime engine provides a generic mechanism 
    to listen to events. The necessary information can easily be extracted 
    from these events and then persisted to a database. Filters can also be 
    used to limit the scope of the logged information.</para>

    <section>
      <title>The Business Activity Monitoring data model</title>

      <para>The jbpm-bam module contains an event listener that stores
      process-related information in a database using JPA or Hibernate directly.
      The data model itself contains three entities, one for process instance 
      information, one for node instance information, and one for (process) 
      variable instance information.</para>
     
      <!-- width && scalefit for pdf's -->
      <figure>
        <title>Business Activity Monitoring data model</title>
        <mediaobject>
          <imageobject>
            <imagedata fileref="images/Chapter-Persistence/bam_schema.png" format="PNG" 
                       align="center" width="100%" scalefit="1" />
          </imageobject>
        </mediaobject>
      </figure>
 
      <para>The <code>ProcessInstanceLog</code> table contains the basic
      log information about a process instance.</para>

      <table>
        <title>ProcessInstanceLog</title>
        <tgroup cols='3' align='left' colsep='1' rowsep='1'>
        <thead>
          <row>
            <entry>Field</entry>
            <entry>Description</entry>
            <entry>Nullable</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry><code>id</code></entry>
            <entry>The primary key and id of the log entity</entry>
            <entry>NOT NULL</entry>
          </row>
          <row>
            <entry><code>end_date</code></entry>
            <entry>When applicable, the end date of the process instance</entry>
          </row>
          <row>
            <entry><code>processid</code></entry>
            <entry>The name (id) of the process</entry>
          </row>
          <row>
            <entry><code>processinstanceid</code></entry>
            <entry>The process instance id</entry>
            <entry>NOT NULL</entry>
          </row>
          <row>
            <entry><code>start_date</code></entry>
            <entry>The start date of the process instance</entry>
          </row>
        </tbody>
        </tgroup>
      </table>

      <para>The <code>NodeInstanceLog</code> table contains more information about which 
      nodes were actually executed inside each process instance. Whenever a node instance 
      is entered from one of its incomming connections or is exited through one of its outgoing 
      connections, that information is stored in this table.</para>

      <table>
        <title>NodeInstanceLog</title>
        <tgroup cols='3' align='left' colsep='1' rowsep='1'>
        <thead>
          <row>
            <entry>Field</entry>
            <entry>Description</entry>
            <entry>Nullable</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry><code>id</code></entry>
            <entry>The primary key and id of the log entity</entry>
            <entry>NOT NULL</entry>
          </row>
          <row>
            <entry><code>log_date</code></entry>
            <entry>The date of the event</entry>
          </row>
          <row>
            <entry><code>nodeid</code></entry>
            <entry>The node id of the corresponding node in the process definition</entry>
          </row>
          <row>
            <entry><code>nodeinstanceid</code></entry>
            <entry>The node instance id</entry>
          </row>
          <row>
            <entry><code>nodename</code></entry>
            <entry>The name of the node</entry>
          </row>
          <row>
            <entry><code>processid</code></entry>
            <entry>The id of the process that the process instance is executing</entry>
          </row>
          <row>
            <entry><code>processinstanceid</code></entry>
            <entry>The process instance id</entry>
            <entry>NOT NULL</entry>
          </row>
          <row>
            <entry><code>type</code></entry>
            <entry>The type of the event (0 = enter, 1 = exit)</entry>
            <entry>NOT NULL</entry>
          </row>
        </tbody>
        </tgroup>
      </table>

      <para>The <code>VariableInstanceLog</code> table contains information about changes 
      in variable instances. The defaul is to only generate log entries when (after) a variable 
      changes. It's also possible to log entries before the variable (value) changes.</para>

      <table>
        <title>VariableInstanceLog</title>
        <tgroup cols='3' align='left' colsep='1' rowsep='1'>
        <thead>
          <row>
            <entry>Field</entry>
            <entry>Description</entry>
            <entry>Nullable</entry>
          </row>
        </thead>
        <tbody>
          <row>
            <entry><code>id</code></entry>
            <entry>The primary key and id of the log entity</entry>
            <entry>NOT NULL</entry>
          </row>
          <row>
            <entry><code>log_date</code></entry>
            <entry>The date of the event</entry>
          </row>
          <row>
            <entry><code>processid</code></entry>
            <entry>The id of the process that the process instance is executing</entry>
          </row>
          <row>
            <entry><code>processinstanceid</code></entry>
            <entry>The process instance id</entry>
            <entry>NOT NULL</entry>
          </row>
          <row>
            <entry><code>value</code></entry>
            <entry>The value of the variable at the time that the log is made</entry>
          </row>
          <row>
            <entry><code>variableid</code></entry>
            <entry>The variable id in the process definition</entry> 
          </row>
          <row>
            <entry><code>variableinstanceid</code></entry>
            <entry>The id of the variable instance</entry>
          </row>
        </tbody>
        </tgroup>
      </table>
        
    </section>

    <section>
      <title>Storing Process Events in a Database</title>

      <para>To log process history information in a database like this, you
      need to register the logger on your session (or working memory) like
      this:</para>

      <programlisting language="java">
StatefulKnowledgeSession ksession = ...;
JPAWorkingMemoryDbLogger logger = new JPAWorkingMemoryDbLogger(ksession);

// invoke methods one your session here

logger.dispose();</programlisting>

      <para>Note that this logger is like any other audit logger, which means
      that you can add one or more filters by calling the method
      <code>addFilter</code> to ensure that only relevant information is
      stored in the database.  Only information accepted by all your filters will
      appear in the database.  You should dispose the logger when it is no
      longer needed.</para>

      <para>To specify the database where the information should be stored,
      modify the file <filename>persistence.xml</filename> file to include
      the audit log classes as well (ProcessInstanceLog, NodeInstanceLog and
      VariableInstanceLog), as shown below.</para>
      
    <programlisting language="xml">&lt;?xml version="1.0" encoding="UTF-8" standalone="yes"?&gt;
&lt;persistence
  version="1.0"
  xsi:schemaLocation=
    "http://java.sun.com/xml/ns/persistence
     http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd
     http://java.sun.com/xml/ns/persistence/orm
     http://java.sun.com/xml/ns/persistence/orm_1_0.xsd"
  xmlns:orm="http://java.sun.com/xml/ns/persistence/orm"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="http://java.sun.com/xml/ns/persistence"&gt;

  &lt;persistence-unit name="org.jbpm.persistence.jpa"&gt;
    &lt;provider&gt;org.hibernate.ejb.HibernatePersistence&lt;/provider&gt;
    &lt;jta-data-source&gt;jdbc/processInstanceDS&lt;/jta-data-source&gt;
    &lt;class&gt;org.drools.persistence.info.SessionInfo&lt;/class&gt;
    &lt;class&gt;org.jbpm.persistence.processinstance.ProcessInstanceInfo&lt;/class&gt;
    &lt;class&gt;org.drools.persistence.info.WorkItemInfo&lt;/class&gt;
    &lt;class&gt;org.jbpm.process.audit.ProcessInstanceLog&lt;/class&gt;
    &lt;class&gt;org.jbpm.process.audit.NodeInstanceLog&lt;/class&gt;
    &lt;class&gt;org.jbpm.process.audit.VariableInstanceLog&lt;/class&gt;

    &lt;properties&gt;
      &lt;property name="hibernate.dialect" value="org.hibernate.dialect.H2Dialect"/&gt;
      &lt;property name="hibernate.max_fetch_depth" value="3"/&gt;
      &lt;property name="hibernate.hbm2ddl.auto" value="update"/&gt;
      &lt;property name="hibernate.show_sql" value="true"/&gt;
      &lt;property name="hibernate.transaction.manager_lookup_class"
                value="org.hibernate.transaction.BTMTransactionManagerLookup"/&gt;
    &lt;/properties&gt;
  &lt;/persistence-unit&gt;
&lt;/persistence&gt;</programlisting>      

      <para>All this information can easily be queried and used in a lot of
      different use cases, ranging from creating a history log for one
      specific process instance to analyzing the performance of all instances
      of a specific process.</para>
      
      <para>This audit log should only be considered a default implementation.
      We don't know what information you need to store for analysis afterwards,
      and for performance reasons it is recommended to only store the relevant
      data.  Depending on your use cases, you might define your own data model
      for storing the information you need, and use the process event listeners
      to extract that information.</para>
    </section>

  </section>

</chapter>
