  <chapter id="cache_loaders">
    <title>Cache Loaders</title>

    <para>JBoss Cache can use a <emphasis>cache loader</emphasis> to back up the in-memory cache to a backend datastore. If JBoss Cache is configured with a cache loader, then
    the following features are provided: <itemizedlist>
        <listitem>Whenever a cache element is accessed, and that element is not in
        the cache (e.g. due to eviction or due to server restart), then the cache loader transparently
        loads the element into the cache if found in the backend
        store.</listitem>

        <listitem>Whenever an element is modified, added or removed, then that
        modification is persisted in the backend store via the cache loader. If
        transactions are used, all modifications created within a transaction
        are persisted. To this end, the cache loader takes part in the two
        phase commit protocol run by the transaction manager.</listitem>
      </itemizedlist></para>

    <para>Currently, the cache loader API looks similar to the TreeCache API.
    In the future, they will both implement the <emphasis>same</emphasis>
    interface. The goal is to be able to form hierarchical cache topologies,
    where one cache can delegate to another, which in turn may delegate to yet
    another cache.</para>

    <para>As of JBossCache 1.3.0, you can now define several cache loaders, in
    a chain. The impact is that the cache will look at all of the cache
    loaders in the order they've been configured, until it finds a valid,
    non-null element of data. When performing writes, all cache loaders are
    written to (except if the ignoreModifications element has been set to true
    for a specific cache loader. See the configuration section below for
    details.</para>

    <para>The cache loader interface is defined in
    org.jboss.cache.loader.CacheLoader as follows (edited for brevity):</para>

    <programlisting>public interface CacheLoader extends Service {

   /**
    * Sets the configuration. Will be called before {@link #create()} and {@link #start()}
    * @param props A set of properties specific to a given CacheLoader
    */
   void setConfig(Properties props);

   void setCache(TreeCache c);


   /**
    * Returns a list of children names, all names are &lt;em&gt;relative&lt;/em&gt;. Returns null if the parent node is not found.
    * The returned set must not be modified, e.g. use Collections.unmodifiableSet(s) to return the result
    * @param fqn The FQN of the parent
    * @return Set&lt;String&gt;. A list of children. Returns null if no children nodes are present, or the parent is
    * not present
    */
   Set getChildrenNames(Fqn fqn) throws Exception;


   /**
    * Returns the value for a given key. Returns null if the node doesn't exist, or the value is not bound
    */
   Object get(Fqn name, Object key) throws Exception;


   /**
    * Returns all keys and values from the persistent store, given a fully qualified name.
    * 
    * NOTE that the expected return value of this method has changed from JBossCache 1.2.x 
    * and before!  This will affect cache loaders written prior to JBossCache 1.3.0 and such 
    * implementations should be checked for compliance with the behaviour expected.
    *  
    * @param name
    * @return Map&lt;Object,Object&gt; of keys and values for the given node. Returns null if the node is not
    * found.  If the node is found but has no attributes, this method returns an empty Map.
    * @throws Exception
    */
   Map get(Fqn name) throws Exception;



   /**
    * Checks whether the CacheLoader has a node with Fqn
    * @return True if node exists, false otherwise
    */
   boolean exists(Fqn name) throws Exception;


   /**
    * Inserts key and value into the attributes hashmap of the given node. If the node does not exist, all
    * parent nodes from the root down are created automatically
    */
   void put(Fqn name, Object key, Object value) throws Exception;

   /**
    * Inserts all elements of attributes into the attributes hashmap of the given node, overwriting existing
    * attributes, but not clearing the existing hashmap before insertion (making it a union of existing and
    * new attributes)
    * If the node does not exist, all parent nodes from the root down are created automatically
    * @param name The fully qualified name of the node
    * @param attributes A Map of attributes. Can be null
    */
   void put(Fqn name, Map attributes) throws Exception;

   /**
    * Inserts all modifications to the backend store. Overwrite whatever is already in
    * the datastore.
    * @param modifications A List&lt;Modification&gt; of modifications
    * @throws Exception
    */
   void put(List modifications) throws Exception;

   /** Removes the given key and value from the attributes of the given node. No-op if node doesn't exist */
   void remove(Fqn name, Object key) throws Exception;

   /**
    * Removes the given node. If the node is the root of a subtree, this will recursively remove all subnodes,
    * depth-first
    */
   void remove(Fqn name) throws Exception;

   /** Removes all attributes from a given node, but doesn't delete the node itself */
   void removeData(Fqn name) throws Exception;


   /**
    * Prepare the modifications. For example, for a DB-based CacheLoader:
    * &lt;ol&gt;
    * &lt;li&gt;Create a local (JDBC) transaction
    * &lt;li&gt;Associate the local transaction with &lt;code&gt;tx&lt;/code&gt; (tx is the key)
    * &lt;li&gt;Execute the coresponding SQL statements against the DB (statements derived from modifications)
    * &lt;/ol&gt;
    * For non-transactional CacheLoader (e.g. file-based), this could be a null operation
    * @param tx            The transaction, just used as a hashmap key
    * @param modifications List&lt;Modification&gt;, a list of all modifications within the given transaction
    * @param one_phase     Persist immediately and (for example) commit the local JDBC transaction as well. When true,
    *                      we won't get a {@link #commit(Object)} or {@link #rollback(Object)} method call later
    */
   void prepare(Object tx, List modifications, boolean one_phase) throws Exception;

   /**
    * Commit the transaction. A DB-based CacheLoader would look up the local JDBC transaction asociated
    * with &lt;code&gt;tx&lt;/code&gt; and commit that transaction&lt;br/&gt;
    * Non-transactional CacheLoaders could simply write the data that was previously saved transiently under the
    * given &lt;code&gt;tx&lt;/code&gt; key, to (for example) a file system (note this only holds if the previous prepare() did
    * not define one_phase=true
    */
   void commit(Object tx) throws Exception;

   /**
    * Roll the transaction back. A DB-based CacheLoader would look up the local JDBC transaction asociated
    * with &lt;code&gt;tx&lt;/code&gt; and roll back that transaction
    */
   void rollback(Object tx);

   /**
    * Fetch the entire state for this cache from secondary storage (disk, DB) and return it as a byte buffer.
    * This is for initialization of a new cache from a remote cache. The new cache would then call
    * storeEntireState()
    * todo: define binary format for exchanging state
    */
   byte[] loadEntireState() throws Exception;

   /** Store the given state in secondary storage. Overwrite whatever is currently in storage */
   void storeEntireState(byte[] state) throws Exception;
}
</programlisting>

<para>
	<emphasis role="bold">NOTE:</emphasis> the contract defined by the CacheLoader interface has changed 
	from JBoss Cache 1.3.0 onwards, specifically with the <literal>get(Fqn fqn)</literal> method.  
	Special care must be taken with custom CacheLoader implementations to ensure this new contract is 
	still adhered to.  See the javadoc above on this method for details, or visit 
	<ulink url="http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossCacheCacheLoaders">this wiki page</ulink> 
	for more discussion on this.
</para>

   <para>CacheLoader implementations that need to support partial state transfer should
   also implement the subinterface org.jboss.cache.loader.ExtendedCacheLoader:
   </para>
   
   <programlisting>public interface ExtendedCacheLoader extends CacheLoader
{
   /**
    * Fetch a portion of the state for this cache from secondary storage 
    * (disk, DB) and return it as a byte buffer.
    * This is for activation of a portion of new cache from a remote cache. 
    * The new cache would then call {@link #storeState(byte[], Fqn)}.
    * 
    * @param subtree Fqn naming the root (i.e. highest level parent) node of
    *                the subtree for which state is requested.
    *                
    * @see org.jboss.cache.TreeCache#activateRegion(String)
    */
   byte[] loadState(Fqn subtree) throws Exception;
   
   /**
    * Store the given portion of the cache tree's state in secondary storage. 
    * Overwrite whatever is currently in secondary storage.  If the transferred 
    * state has Fqns equal to or children of parameter <code>subtree</code>, 
    * then no special behavior is required.  Otherwise, ensure that
    * the state is integrated under the given 'subtree'. Typically
    * in the latter case 'subtree' would be the Fqn of the buddy 
    * backup region for a buddy group; e.g.
    * 
    * If the the transferred state had Fqns starting with "/a" and
    * 'subtree' was "/_BUDDY_BACKUP_/192.168.1.2:5555" then the
    * state should be stored in the local persistent store under
    * "/_BUDDY_BACKUP_/192.168.1.2:5555/a"
    * 
    * @param state   the state to store
    * @param subtree Fqn naming the root (i.e. highest level parent) node of
    *                the subtree included in 'state'.  If the Fqns  
    *                of the data included in 'state' are not 
    *                already children of 'subtree', then their
    *                Fqns should be altered to make them children of 
    *                'subtree' before they are persisted.
    */   
   void storeState(byte[] state, Fqn subtree) throws Exception;
   
   /**
    * Sets the {@link RegionManager} this object should use to manage 
    * marshalling/unmarshalling of different regions using different
    * classloaders.
    *
    * NOTE: This method is only intended to be used by the TreeCache instance 
    * this cache loader is associated with.
    * 
    * @param manager    the region manager to use, or <code>null</code>.
    */
   void setRegionManager(RegionManager manager);

}
   </programlisting>

<para>
	<emphasis role="bold">NOTE:</emphasis> If a cache loader is used along with
	buddy replication, the cache loader must implement <literal>ExtendedCacheLoader</literal>
	unless its <literal>FetchPersistentState</literal> property is set to false.
</para>

<para>
	<emphasis role="bold">NOTE:</emphasis> the contract defined by the <literal>ExtendedCacheLoader</literal> interface has changed 
	from JBoss Cache 1.4.0 onwards, specifically with the requirement that data passed to <literal>storeState</literal> 
	method be integrated under the given subtree, even if that data didn't originate in that subtree.  This
	behavior is necessary to properly support buddy replication. Special care must be taken with custom 
	ExtendedCacheLoader implementations to ensure this new contract is still adhered to. 
</para>

    <section>
      <title>The CacheLoader Interface</title>

      <para>The interaction between JBoss Cache and a CacheLoader
      implementation is as follows. When
      <literal>CacheLoaderConfiguration</literal> (see below) is non-null, an
      instance of each configured <literal>cacheloader</literal> is created
      when the cache is created. Since <literal>CacheLoader</literal> extends
      <literal>Service</literal>,</para>

      <programlisting>public interface Service {
   void create() throws Exception;

   void start() throws Exception;

   void stop();

   void destroy();
}
</programlisting>

      <para><literal>CacheLoader.create()</literal> and
      <literal>CacheLoader.start()</literal> are called when the cache is
      started. Correspondingly, <literal>stop()</literal> and
      <literal>destroy()</literal> are called when the cache is
      stopped.</para>

      <para>Next, <literal>setConfig()</literal> and
      <literal>setCache()</literal> are called. The latter can be used to
      store a reference to the cache, the former is used to configure this
      instance of the CacheLoader. For example, here a database CacheLoader
      could establish a connection to the database.</para>

      <para>The CacheLoader interface has a set of methods that are called
      when no transactions are used: <literal>get()</literal>,
      <literal>put()</literal>, <literal>remove()</literal> and
      <literal>removeData()</literal>: they get/set/remove the value
      immediately. These methods are described as javadoc comments in the
      above interface.</para>

      <para>Then there are three methods that are used with transactions:
      <literal>prepare()</literal>, <literal>commit()</literal> and
      <literal>rollback()</literal>. The <literal>prepare()</literal> method
      is called when a transaction is to be committed. It has a transaction
      object and a list of modfications as argument. The transaction object
      can be used as a key into a hashmap of transactions, where the values
      are the lists of modifications. Each modification list has a number of
      <literal>Modification</literal> elements, which represent the changes
      made to a cache for a given transaction. When
      <literal>prepare()</literal> returns successfully, then the CacheLoader
      <emphasis>must</emphasis> be able to commit (or rollback) the
      transaction successfully.</para>

      <para>Currently, the TreeCache takes care of calling prepare(), commit()
      and rollback() on the CacheLoaders at the right time. We intend to make
      both the TreeCache and the CacheLoaders XA resources, so that instead of
      calling those methods on a loader, the cache will only enlist the loader
      with the TransactionManager on the same transaction.</para>

      <para>The <literal>commit()</literal> method tells the CacheLoader to
      commit the transaction, and the <literal>rollback()</literal> method
      tells the CacheLoader to discard the changes associated with that
      transaction.</para>

      <para>The last two methods are <literal>loadEntireState()</literal> and
      <literal>storeEntireState()</literal>. The first method asks the
      CacheLoader to get the entire state the backend store manages and return
      it as a byte buffer, and the second tells a CacheLoader to replace its
      entire state with the byte buffer argument. These methods are used for
      scenarios where each JBossCache node in a cluster has its own local data
      store, e.g. a local DB, and - when a new node starts - we have to
      initialize its backend store with the contents of the backend store of
      an existing member. See below for deails.</para>
      
      <para>The <literal>ExtendedCacheLoader</literal> methods are also
      related to state transfer.  The <literal>loadState(Fqn)</literal> method is called
      when the cache is preparing a partial state transfer -- that is, the
      transfer of just the portion of the cache loader's state that is rooted
      in the given Fqn.  The <literal>storeState(byte[], Fqn)</literal> method
      is then invoked on the cache loader of the node that is receiving the
      state transfer.  Partial state transfers occur when the cache's
      <literal>activateRegion()</literal> API is used and during the 
      formation of buddy groups if buddy replication is used.</para>
    </section>
    
    <section>
      <title>Configuration via XML</title>

      <para>The CacheLoader is configured as follows in the JBossCache XML
      file:</para>

      <programlisting>    &lt;!-- ==================================================================== --&gt;
    &lt;!-- Defines TreeCache configuration                                      --&gt;
    &lt;!-- ==================================================================== --&gt;

    &lt;mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=TreeCache"&gt;

        &lt;!-- New 1.3.x cache loader config block --&gt;
        &lt;attribute name="CacheLoaderConfiguration"&gt;
            &lt;config&gt;
                &lt;!-- if passivation is true, only the first cache loader is used; the rest are ignored --&gt;
                &lt;passivation&gt;false&lt;/passivation&gt;
                &lt;!-- comma delimited FQNs to preload --&gt;
                &lt;preload&gt;/&lt;/preload&gt;
                &lt;!-- are the cache loaders shared in a cluster? --&gt;
                &lt;shared&gt;false&lt;/shared&gt;

                &lt;!-- we can now have multiple cache loaders, which get chained --&gt;
                &lt;!-- the 'cacheloader' element may be repeated --&gt;
                &lt;cacheloader&gt;
                    &lt;class&gt;org.jboss.cache.loader.JDBCCacheLoader&lt;/class&gt;
                    &lt;!-- same as the old CacheLoaderConfig attribute --&gt;
                    &lt;properties&gt;
                        cache.jdbc.driver=com.mysql.jdbc.Driver
                        cache.jdbc.url=jdbc:mysql://localhost:3306/jbossdb
                        cache.jdbc.user=root
                        cache.jdbc.password=
                    &lt;/properties&gt;
                    &lt;!-- whether the cache loader writes are asynchronous --&gt;
                    &lt;async&gt;false&lt;/async&gt;
                    &lt;!-- only one cache loader in the chain may set fetchPersistentState to true.
                        An exception is thrown if more than one cache loader sets this to true. --&gt;
                    &lt;fetchPersistentState&gt;true&lt;/fetchPersistentState&gt;
                    &lt;!-- determines whether this cache loader ignores writes - defaults to false. --&gt;
                    &lt;ignoreModifications&gt;false&lt;/ignoreModifications&gt;
                    &lt;!-- if set to true, purges the contents of this cache loader when the cache starts up.
                    Defaults to false.  --&gt;
                    &lt;purgeOnStartup&gt;false&lt;/purgeOnStartup&gt;
                &lt;/cacheloader&gt;

            &lt;/config&gt;
        &lt;/attribute&gt;

    &lt;/mbean&gt;
</programlisting>

      <para><emphasis role="bold">Note:</emphasis> In JBossCache releases
      prior to 1.3.0, the cache loader configuration block used to look like
      this. Note that this form is <emphasis role="deprecated">DEPRECATED</emphasis> and you will have to replace
      your cache loader configuration with a block similar to the one
      above.</para>

      <para>
        <programlisting>    &lt;!-- ==================================================================== --&gt;
    &lt;!-- Defines TreeCache configuration                                      --&gt;
    &lt;!-- ==================================================================== --&gt;

    &lt;mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=TreeCache"&gt;
       &lt;attribute name="CacheLoaderClass"&gt;org.jboss.cache.loader.bdbje.BdbjeCacheLoader&lt;/attribute&gt;
       &lt;!-- attribute name="CacheLoaderClass"&gt;org.jboss.cache.loader.FileCacheLoader&lt;/attribute --&gt;
       &lt;attribute name="CacheLoaderConfig" replace="false"&gt;
         location=c:\\tmp\\bdbje
       &lt;/attribute&gt;
       &lt;attribute name="CacheLoaderShared"&gt;true&lt;/attribute&gt;
       &lt;attribute name="CacheLoaderPreload"&gt;/&lt;/attribute&gt;
       &lt;attribute name="CacheLoaderFetchTransientState"&gt;false&lt;/attribute&gt;
       &lt;attribute name="CacheLoaderFetchPersistentState"&gt;true&lt;/attribute&gt;
       &lt;attribute name="CacheLoaderAsynchronous"&gt;true&lt;/attribute&gt;
    &lt;/mbean&gt;
</programlisting>
      </para>

      <para>The <literal>CacheLoaderClass</literal> attribute defines the
      class of the CacheLoader implementation. (Note that, because of a bug in
      the properties editor in JBoss, backslashes in variables for Windows
      filenames might not get expanded correctly, so replace="false" may be
      necessary).</para>

      <para>The currently available implementations shipped with JBossCache
      are:</para>

      <itemizedlist>
        <listitem>
          <para><literal>FileCacheLoader</literal>, which is a simple
          filesystem-based implementation. The
          <literal>&lt;cacheloader&gt;&lt;properties&gt;</literal> element
          needs to contain a "location" property, which maps to a directory where
          the file is located (e.g., "location=c:\\tmp").</para>
        </listitem>

        <listitem>
          <para><literal>BdbjeCacheLoader</literal>, which is a CacheLoader
          implementation based on the Sleepycat DB Java Edition. The
          <literal>&lt;cacheloader&gt;&lt;properties&gt;</literal> element needs
           to contain a "location" property, which maps to a directory,where the
          database file for Sleepycat resides (e.g., "location=c:\\tmp").</para>
        </listitem>

        <listitem>
          <para><literal>JDBCCacheLoader</literal>, which is a CacheLoader
          implementation using JDBC to access any relational database. The
          <literal>&lt;cacheloader&gt;&lt;properties&gt;</literal> element
          contains a number of properties needed to connect to the
          database such as username, password, and connection URL. See the
          section on JDBCCacheLoader for more details.</para>
        </listitem>

        <listitem>
          <para><literal>LocalDelegatingCacheLoader</literal>, which enables
          loading from and storing to another local (same VM) TreeCache.</para>
          </listitem>

        <listitem>
          <para><literal>TcpDelegatingCacheLoader</literal>, which enables
          loading from and storing to a remote (different VM) TreeCache using
          TCP as the transport mechanism. This CacheLoader is available in 
          JBossCache version 1.3.0 and above.</para>
        </listitem>

        <listitem>
          <para><literal>ClusteredCacheLoader</literal>, which allows querying
          of other caches in the same cluster for in-memory data via the same
          clustering protocols used to replicate data. Writes are <emphasis
          role="bold">not</emphasis> 'stored' though, as replication would
          take care of any updates needed. You need to specify a property
          called "<literal>timeout</literal>", a long value telling the cache
          loader how many milliseconds to wait for responses from the cluster
          before assuming a null value.  For example, "<literal>timeout =
          3000</literal>" would use a timeout value of 3 seconds. This
          CacheLoader is available in JBossCache version 1.3.0 and
          above.</para>
        </listitem>
      </itemizedlist>

      <para>Note that the Sleepycat implementation is much more efficient than
      the filesystem-based implementation, and provides transactional
      guarantees, but requires a commercial license if distributed with an
      application (see http://www.sleepycat.com/jeforjbosscache for
      details).</para>

      <para>An implementation of CacheLoader has to have an empty constructor
      due to the way it is instantiated.</para>

      <para>The <literal>properties</literal> element defines a configuration
      specific to the given implementation. The filesystem-based
      implementation for example defines the root directory to be used,
      whereas a database implementation might define the database URL, name
      and password to establish a database connection. This configuration is
      passed to the CacheLoader implementation via
      <literal>CacheLoader.setConfig(Properties)</literal>. Note that
      backspaces may have to be escaped. <emphasis>Analogous to the
      <literal>CacheLoaderConfig</literal> attribute in pre-1.3.0
      configurations.</emphasis></para>

      <para><literal>preload</literal> allows us to define a list of nodes, or
      even entire subtrees, that are visited by the cache on startup, in order
      to preload the data associated with those nodes. The default ("/") loads
      the entire data available in the backend store into the cache, which is
      probably not a good idea given that the data in the backend store might
      be large. As an example, <literal>/a,
      /product/catalogue</literal> loads the subtrees <literal>/a</literal>
      and <literal>/product/catalogue</literal> into the cache, but nothing
      else. Anything else is loaded lazily when accessed. Preloading makes
      sense when one anticipates using elements under a given subtree
      frequently. <emphasis>Note that preloading loads all nodes and
      associated attributes from the given node, recursively up to the root
      node</emphasis>. <emphasis>Analogous to the
      <literal>CacheLoaderPreload</literal> attribute in pre-1.3.0
      configurations.</emphasis></para>

      <para><literal>fetchPersistentState</literal> determines whether or not
      to fetch the persistent state of a cache when joining a cluster. Only
      one configured cache loader may set this property to true; if more than
      one cache loader does so, a configuration exception will be thrown when
      starting your cache service. <emphasis>Analogous to the
      <literal>CacheLoaderFetchPersistentState</literal> attribute in
      pre-1.3.0 configurations.</emphasis></para>

      <para><literal>async</literal> determines whether writes to the cache
      loader block until completed, or are run on a separate thread so writes
      return immediately. If this is set to true, an instance of
      <literal>org.jboss.cache.loader.AsyncCacheLoader</literal> is
      constructed with an instance of the actual cache loader to be used. The
      <literal>AsyncCacheLoader</literal> then delegates all requests to the
      underlying cache loader, using a separate thread if necessary. See the
      Javadocs on <literal>org.jboss.cache.loader.AsyncCacheLoader</literal>
      for more details. If unspecified, the <literal>async</literal> element
      defaults to <emphasis>false</emphasis>. <emphasis>Analogous to the
      <literal>CacheLoaderAsynchronous</literal> attribute in pre-1.3.0
      configurations.</emphasis></para>

      <para><emphasis role="bold">Note on using the <literal>async</literal>
      element:</emphasis> there is always the possibility of dirty reads since
      all writes are performed asynchronously, and it is thus impossible to
      guarantee when (and even if) a write succeeds. This needs to be kept in
      mind when setting the <literal>async</literal> element to true.</para>

      <para><literal>ignoreModifications</literal> determines whether write
      methods are pushed down to the specific cache loader. Situations may
      arise where transient application data should only reside in a file
      based cache loader on the same server as the in-memory cache, for
      example, with a further shared JDBC cache loader used by all servers in
      the network. This feature allows you to write to the 'local' file cache
      loader but not the shared JDBC cache loader. This property defaults to
      <literal>false</literal>, so writes are propagated to all cache loaders
      configured.</para>

      <para>
          <literal>purgeOnStatup</literal> empties the specified cache loader
          (if <literal>ignoreModifications</literal> is <literal>false</literal>)
           when the cache loader starts up.
      </para>
    </section>

    <section>
      <title>Cache passivation</title>

      <para>A CacheLoader can be used to enforce node passivation and
      activation on eviction in a TreeCache.</para>

      <para><emphasis>Cache Passivation</emphasis> is the process of removing
      an object from in-memory cache and writing it to a secondary data store
      (e.g., file system, database) on eviction. <emphasis>Cache
      Activation</emphasis> is the process of restoring an object from the
      data store into the in-memory cache when it's needed to be used. In both
      cases, the configured CacheLoader will be used to read from the data
      store and write to the data store.</para>

      <para>When the eviction policy in effect calls evict() to evict a node
      from the cache, if passivation is enabled, a notification that the node
      is being passivated will be emitted to the tree cache listeners and the
      node and its children will be stored in the cache loader store. When a
      user calls get() on a node that was evicted earlier, the node is loaded
      (lazy loaded) from the cache loader store into the in-memory cache. When
      the node and its children have been loaded, they're removed from the
      cache loader and a notification is emitted to the tree cache listeners
      that the node has been activated.</para>

      <para>To enable cache passivation/activation, you can set
      <literal>passivation</literal> to true. The default is false. You set it
      via the XML cache configuration file. The XML above shows the
      <literal>passivation</literal> element when configuring a cache loader.
      When passivation is used, only the first cache loader configured is
      used. All others are ignored.</para>
    </section>

    <section>
      <title>CacheLoader use cases</title>

      <section>
        <title>Local cache with store</title>

        <para>This is the simplest case. We have a JBossCache instance, whose
        mode is <literal>LOCAL</literal>, therefore no replication is going
        on. The CacheLoader simply loads non-existing elements from the store
        and stores modifications back to the store. When the cache is started,
        depending on the <literal>preload</literal> element, certain data can
        be preloaded, so that the cache is partly warmed up.</para>

        <para>When using PojoCache, this means that entire POJOs can be
        stored to a database or a filesystem, and when accessing fields of a
        POJO, they will be lazily loaded using the CacheLoader to access a
        backend store. This feature effectively provides simple persistency
        for any POJO.</para>
      </section>

      <section>
        <title>Replicated caches with all nodes sharing the same store</title>

        <para>The following figure shows 2 JBossCache nodes sharing the same
        backend store:</para>

        <figure>
          <title>2 nodes sharing a backend store</title>

          <mediaobject>
            <imageobject>
              <imagedata fileref="images/SharedCacheLoader.gif" />
            </imageobject>
          </mediaobject>
        </figure>

        <para>Both nodes have a CacheLoader that accesses a common shared
        backend store. This could for example be a shared filesystem (using
        the FileCacheLoader), or a shared database. Because both nodes access
        the same store, they don't necessarily need state transfer on
        startup.<footnote>
            <para>Of course they can enable state transfer, if they want to
            have a warm or hot cache after startup.</para>
          </footnote>Rather, the<literal>FetchInMemoryState</literal>
        attribute could be set to false, resulting in a 'cold' cache, that
        gradually warms up as elements are accessed and loaded for the first
        time. This would mean that individual caches in a cluster might have
        different in-memory state at any given time (largely depending on
        their preloading and eviction strategies).</para>

        <para>When storing a value, the writer takes care of storing the
        change in the backend store. For example, if node1 made change C1 and
        node2 C2, then node1 would tell its CacheLoader to store C1, and node2
        would tell its CacheLoader to store C2.</para>
      </section>

      <section>
        <title>Replicated caches with only one node having a store</title>

        <para />

        <figure>
          <title>2 nodes but only one accesses the backend store</title>

          <mediaobject>
            <imageobject>
              <imagedata fileref="images/OnlyOneCacheLoader.gif" />
            </imageobject>
          </mediaobject>
        </figure>

        <para>This is a similar case as the previous one, but here only one
        node in the cluster interacts with a backend store via its
        CacheLoader. All other nodes perform in-memory replication. A use case
        for this is HTTP session replication, where all nodes replicate
        sessions in-memory, and - in addition - one node saves the sessions to
        a persistent backend store. Note that here it may make sense for the
        CacheLoader to store changes asynchronously, that is
        <emphasis>not</emphasis> on the caller's thread, in order not to slow
        down the cluster by accessing (for example) a database. This is a
        non-issue when using asynchronous replication.</para>
      </section>

      <section>
        <title>Replicated caches with each node having its own store</title>

        <figure>
          <title>2 nodes each having its own backend store</title>

          <mediaobject>
            <imageobject>
              <imagedata fileref="images/LocalCacheLoader.gif" />
            </imageobject>
          </mediaobject>
        </figure>

        <para>Here, each node has its own datastore. Modifications to the
        cache are (a) replicated across the cluster and (b) persisted using
        the CacheLoader. This means that all datastores have exactly the same
        state. When replicating changes synchronously and in a transaction,
        the two phase commit protocol takes care that all modifications are
        replicated and persisted in each datastore, or none is replicated and
        persisted (atomic updates).</para>

        <para>Note that currently JBossCache is <emphasis>not</emphasis> an
        XAResource, that means it doesn't implement recovery. When used with a
        TransactionManager that supports recovery, this functionality is not
        available.</para>

        <para>The challenge here is state transfer: when a new node starts it
        needs to do the following:</para>

        <orderedlist>
          <listitem>
            <para>Tell the coordinator (oldest node in a cluster) to send it
            the state</para>
          </listitem>

          <listitem>
            <para>The coordinator then needs to wait until all in-flight
            transactions have completed. During this time, it will not allow
            for new transactions to be started.</para>
          </listitem>

          <listitem>
            <para>Then the coordinator asks its CacheLoader for the entire
            state using <literal>loadEntireState()</literal>. It then sends
            back that state to the new node.</para>
          </listitem>

          <listitem>
            <para>The new node then tells its CacheLoader to store that state
            in its store, overwriting the old state. This is the
            <literal>CacheLoader.storeEntireState()</literal> method</para>
          </listitem>

          <listitem>
            <para>As an option, the transient (in-memory) state can be
            transferred as well during the state transfer.</para>
          </listitem>

          <listitem>
            <para>The new node now has the same state in its backend store as
            everyone else in the cluster, and modifications received from
            other nodes will now be persisted using the local
            CacheLoader.</para>
          </listitem>
        </orderedlist>

        <para />
      </section>

      <section>
        <title>Hierarchical caches</title>

        <para>If you need to set up a hierarchy within a single VM, you can
        use the <literal>LocalDelegatingCacheLoader</literal>. This type of
        hierarchy can currently only be set up programmatically. The code
        below shows how a first-level cache delegates to a local second-level
        cache:</para>

        <programlisting>TreeCache firstLevel, secondLevel;
LocalDelegatingCacheLoader cache_loader;

// create and configure firstLevel
firstLevel=new TreeCache();

// create and configure secondLevel
secondLevel=new TreeCache();

// create DelegatingCacheLoader
cache_loader=new LocalDelegatingCacheLoader(secondLevel);

// set CacheLoader in firstLevel
firstLevel.setCacheLoader(cache_loader);

// start secondLevel
secondLevel.startService();

// start firstLevel
firstLevel.startService();

</programlisting>

        <para>If you need to set up a hierarchy across VMs but within a
        cluster, you can use the <literal>RpcDelegatingCacheLoader</literal>,
        which delegates all cache loading requests from non-coordinator caches
        to the cluster's coordinator cache. The coordinator cache is the first
        cache in the cluster to come online. Note that if the coordinator
        cache leaves the cluster for any reason, the second cache in the
        cluster to come online becomes the coordinator and so on. The XML
        below shows how to configure a cluster using
        <literal>RpcDelegatingCacheLoader</literal>:</para>

        <programlisting>&lt;!-- ==================================================================== --&gt;
&lt;!-- Defines TreeCache configuration                                      --&gt;
&lt;!-- ==================================================================== --&gt;

&lt;mbean code="org.jboss.cache.TreeCache" name="jboss.cache:service=TreeCache"&gt;
...
&lt;attribute name="CacheLoaderConfiguration"&gt;
    &lt;config&gt;
        &lt;passivation&gt;false&lt;/passivation&gt;
        &lt;preload&gt;/some/stuff&lt;/preload&gt;
        &lt;cacheloader&gt;
            &lt;class&gt;org.jboss.cache.loader.RpcDelegatingCacheLoader&lt;/class&gt;
            &lt;!-- whether the cache loader writes are asynchronous --&gt;
            &lt;async&gt;false&lt;/async&gt;
            &lt;!-- only one cache loader in the chain may set fetchPersistentState to true.
                 An exception is thrown if more than one cache loader sets this to true. --&gt;
            &lt;fetchPersistentState&gt;false&lt;/fetchPersistentState&gt;
            &lt;!-- determines whether this cache loader ignores writes - defaults to false. --&gt;
            &lt;ignoreModifications&gt;false&lt;/ignoreModifications&gt;
            &lt;!-- if set to true, purges the contents of this cache loader when the cache starts up.
            Defaults to false.  --&gt;
            &lt;purgeOnStartup&gt;false&lt;/purgeOnStartup&gt;
        &lt;/cacheloader&gt;
    &lt;/config&gt;
&lt;/attribute&gt;
...
&lt;/mbean&gt;</programlisting>

        <para>Note that currently (JBossCache 1.3.0) this cache loader is not
        well supported, and has not been tested. We suggest to use
        TcpDelegatingCacheLoader instead (see next).</para>

        <para />
      </section>




      <section>
        <title>TcpDelegatingCacheLoader</title>

        <para>This cache loader allows to delegate loads and stores to another
        instance of JBossCache, which could reside (a)in the same address
        space, (b) in a different process on the same host, or (c) in a
        different process on a different host. Option (a) is mostly used for
        unit testing, and the envisaged use is (b) and (c).</para>

        <para>A TcpDelegatingCacheLoader talks to a remote TcpCacheServer,
        which can be a standalone process, or embedded as an MBean inside
        JBoss. The TcpCacheServer has a reference to another JBossCache, which
        it can create itself, or which is given to it (e.g. by JBoss, using
        dependency injection).</para>

        <para>The TcpDelegatingCacheLoader is configured with the host and
        port of the remote TcpCacheServer, and uses this to communicate to
        it.</para>

        <para />

        <para>An example set of a TcpCacheServer running inside of JBoss is
        shown below:</para>

        <programlisting>&lt;server&gt;

   &lt;classpath codebase="./lib" archives="jboss-cache.jar"/&gt;

   &lt;mbean code="org.jboss.cache.loader.tcp.TcpCacheServer" name="jboss.cache:service=TcpCacheServer"&gt;
      &lt;depends optional-attribute-name="Cache"
               proxy-type="attribute"&gt;jboss.cache:service=TreeCache&lt;/depends&gt;
      &lt;attribute name="BindAddress"&gt;${jboss.bind.address:localhost}&lt;/attribute&gt;
      &lt;attribute name="Port"&gt;7500&lt;/attribute&gt;
      &lt;attribute name="MBeanServerName"&gt;&lt;/attribute&gt;
      &lt;!--&lt;attribute name="CacheName"&gt;jboss.cache:service=TreeCache&lt;/attribute&gt;--&gt;
   &lt;/mbean&gt;

&lt;/server&gt;         </programlisting>

        <para>The BindAddress and Port define where its server socket is
        listening on, and an existing JBossCache MBean is injected into it
        (assigned to 'Cache'). This means that all requests from the
        TcpDelegatingCacheLoader will be received by this instance and
        forwarded to the JBossCache MBean.</para>

        <para>Note that there is also a 'Config' attribute which points to a
        config XML file for JBossCache. If it is set, then the TcpCacheServer
        will create its own instance of JBossCache and configure it according
        to the Config attribute.</para>

        <para>The client side looks as follow:</para>

        <programlisting>&lt;attribute name="CacheLoaderConfiguration"&gt;
   &lt;config&gt;
       &lt;cacheloader&gt;
           &lt;class&gt;org.jboss.cache.loader.tcp.TcpDelegatingCacheLoader&lt;/class&gt;
               &lt;properties&gt;
                   host=localhost
                   port=7500
               &lt;/properties&gt;
       &lt;/cacheloader&gt;
   &lt;/config&gt;
&lt;/attribute&gt;</programlisting>

        <para>This means this instance of JBossCache will delegate all load
        and store requests to the remote TcpCacheServer running at
        localhost:7500.</para>

        <para>A typical use case could be multiple replicated instance of
        JBossCache in the same cluster, all delegating to the same
        TcpCacheServer instance. The TcpCacheServer might itself delegate to a
        database via JDBCCacheLoader, but the point here is that - if we have
        5 nodes all accessing the same dataset - they will load the data from
        the TcpCacheServer, which has do execute one SQL statement per
        unloaded data set. If the nodes went directly to the database, then
        we'd have the same SQL executed multiple times. So TcpCacheServer
        serves as a natural cache in front of the DB (assuming that a network
        round trip is faster than a DB access (which usually also include a
        network round trip)).</para>

        <para>To alleviate single point of failure, we could combine this with
        a ChainingCacheLoader, where the first CacheLoader is a
        ClusteredCacheLoader, the second a TcpDelegatingCacheLoader, and the
        last a JDBCacheLoader, effectively defining our cost of access to a
        cache in increasing order of cost.</para>

        <para />
      </section>

	      <section>
	        <title>RmiDelegatingCacheLoader</title>

	        <para>Similar to the TcpDelegatingCacheLoader, the RmiDelegatingCacheLoader uses RMI as a method of communicating with a remote cache.</para>

	        <para>An RmiDelegatingCacheLoader talks to a remote RmiCacheServer,
	        which is a standalone process. The RmiCacheServer has a reference to another JBossCache, which it can create itself, or which is given to it (e.g. by JBoss, using dependency injection).</para>

	        <para>The RmiDelegatingCacheLoader is configured with the host, port of the remote RMI server and the bind name of the RmiCacheServer, and uses this to communicate.</para>

	        <para />

	        <para>An example set of an RmiCacheServer running inside of JBoss is
	        shown below:</para>

	        <programlisting>&lt;server&gt;

	   &lt;classpath codebase="./lib" archives="jboss-cache.jar"/&gt;

	   &lt;mbean code="org.jboss.cache.loader.rmi.RmiCacheServer" name="jboss.cache:service=RmiCacheServer"&gt;
	      &lt;depends optional-attribute-name="Cache"
	               proxy-type="attribute"&gt;jboss.cache:service=TreeCache&lt;/depends&gt;
		  &lt;!-- the address and port of the RMI server. --&gt;
	      &lt;attribute name="BindAddress"&gt;${jboss.bind.address:localhost}&lt;/attribute&gt;
	      &lt;attribute name="Port"&gt;1098&lt;/attribute&gt;
		  &lt;attribute name="BindName"&gt;MyRmiCacheServer&lt;/attribute&gt;
	      &lt;attribute name="MBeanServerName"&gt;&lt;/attribute&gt;
	      &lt;!--&lt;attribute name="CacheName"&gt;jboss.cache:service=TreeCache&lt;/attribute&gt;--&gt;
	   &lt;/mbean&gt;

	&lt;/server&gt;         </programlisting>

	        <para>The BindAddress and Port should point to an already-running RMI server and the BindName is the name the object is bound to in the RMI server.  An existing JBossCache MBean is injected into it
	        (assigned to 'Cache'). This means that all requests from the
	        TcpDelegatingCacheLoader will be received by this instance and
	        forwarded to the JBossCache MBean.</para>

	        <para>Note that there is also a 'Config' attribute which points to a
	        config XML file for JBossCache. If it is set, then the RmiCacheServer
	        will create its own instance of JBossCache and configure it according
	        to the Config attribute.</para>

	        <para>The client side looks as follow:</para>

	        <programlisting>&lt;attribute name="CacheLoaderConfiguration"&gt;
	   &lt;config&gt;
	       &lt;cacheloader&gt;
	           &lt;class&gt;org.jboss.cache.loader.RmiDelegatingCacheLoader&lt;/class&gt;
	               &lt;properties&gt;
	                   host=localhost
	                   port=1098
					   name=MyRmiCacheServer	
	               &lt;/properties&gt;
	       &lt;/cacheloader&gt;
	   &lt;/config&gt;
	&lt;/attribute&gt;</programlisting>

	        <para>This means this instance of JBossCache will delegate all load
	        and store requests to the remote RmiCacheServer running as MyRmiCacheServer on an RMI server running on localhost:1098.</para>

	        <para>Very similar use case scenarios that apply to TcpDelegatingCacheLoaders above apply to RmiDelegatingCacheLoaders as well.</para>

	        <para />
	      </section>
	    </section>


    <section>
      <title>JDBC-based CacheLoader</title>

      <para>JBossCache is distributed with a JDBC-based CacheLoader
      implementation that stores/loads nodes' state into a relational database.
      The implementing class is
      <literal>org.jboss.cache.loader.JDBCCacheLoader</literal>.</para>

      <para>The current implementation uses just one table. Each row in the table
      represents one node and contains three columns: <itemizedlist>
          <listitem>column for FQN (which is also a primary key
          column)</listitem>

          <listitem>column for node contents (attribute/value
          pairs)</listitem>

          <listitem>column for parent FQN</listitem>
        </itemizedlist></para>
        
        <para>FQN's are stored as strings. Node content is stored
      as a BLOB. <emphasis>WARNING:</emphasis> TreeCache does not impose any
      limitations on the types of objects used in FQN but this implementation of
      CacheLoader requires FQN to contain only objects of type
      <literal>java.lang.String</literal>. Another limitation for FQN is its
      length. Since FQN is a primary key, its default column type is
      <literal>VARCHAR</literal> which can store text values up to some
      maximum length determined by the database.  FQN is also subject to any
      maximum primary key length restriction imposed by the database.</para>

      <para>See <ulink
      url="???">http://wiki.jboss.org/wiki/Wiki.jsp?page=JDBCCacheLoader</ulink>
      for configuration tips with specific database systems.</para>

      <section>
        <title>JDBCCacheLoader configuration</title>

        <section>
          <title>Table configuration</title>

          <para>Table and column names as well as column types are
          configurable with the following properties. <itemizedlist>
              <listitem><emphasis>cache.jdbc.table.name</emphasis> - the name
              of the table. The default value is 'jbosscache'.</listitem>

              <listitem><emphasis>cache.jdbc.table.primarykey</emphasis> - the
              name of the primary key for the table. The default value is
              'jbosscache_pk'.</listitem>

              <listitem><emphasis>cache.jdbc.table.create</emphasis> - can be
              true or false. Indicates whether to create the table during startup.
              If true, the table is created if it doesn't already exist. The
              default value is true.</listitem>

              <listitem><emphasis>cache.jdbc.table.drop</emphasis> - can be
              true or false. Indicates whether to drop the table during shutdown.  The 
              default value is true.</listitem>

              <listitem><emphasis>cache.jdbc.fqn.column</emphasis> - FQN
              column name. The default value is 'fqn'.</listitem>

              <listitem><emphasis>cache.jdbc.fqn.type</emphasis> - FQN column
              type. The default value is 'varchar(255)'.</listitem>

              <listitem><emphasis>cache.jdbc.node.column</emphasis> - node
              contents column name. The default value is 'node'.</listitem>

              <listitem><emphasis>cache.jdbc.node.type</emphasis> - node
              contents column type. The default value is 'blob'.  This type must specify
              a valid binary data type for the database being used.</listitem>
            </itemizedlist></para>
        </section>

        <section>
          <title>DataSource</title>

          <para>If you are using JBossCache in a managed environment (e.g., an
          application server) you can specify the JNDI name of the DataSource
          you want to use. <itemizedlist>
              <listitem><emphasis>cache.jdbc.datasource</emphasis> - JNDI name
              of the DataSource. The default value is
              'java:/DefaultDS'.</listitem>
            </itemizedlist></para>
        </section>

        <section>
          <title>JDBC driver</title>

          <para>If you are <emphasis>not</emphasis> using DataSource you have
          the following properties to configure database access using a JDBC
          driver. <itemizedlist>
              <listitem><emphasis>cache.jdbc.driver</emphasis> - fully
              qualified JDBC driver name.</listitem>

              <listitem><emphasis>cache.jdbc.url</emphasis> - URL to connect
              to the database.</listitem>

              <listitem><emphasis>cache.jdbc.user</emphasis> - user name to
              connect to the database.</listitem>

              <listitem><emphasis>cache.jdbc.password</emphasis> - password to
              connect to the database.</listitem>
            </itemizedlist></para>
        </section>

        <section>
          <title>Configuration example</title>

          <para>Below is an example of a JDBC CacheLoader using Oracle as
          database. The CacheLoaderConfiguration XML element contains an
          arbitrary set of properties which define the database-related
          configuration.</para>

          <para>
            <programlisting>&lt;attribute name="CacheLoaderConfiguration"&gt;
    &lt;config&gt;
        &lt;passivation&gt;false&lt;/passivation&gt;
        &lt;preload&gt;/some/stuff&lt;/preload&gt;
        &lt;cacheloader&gt;
            &lt;class&gt;org.jboss.cache.loader.JDBCCacheLoader&lt;/class&gt;
            &lt;!-- same as the old CacheLoaderConfig attribute --&gt;
            &lt;properties&gt;
                cache.jdbc.table.name=jbosscache
                cache.jdbc.table.create=true
                cache.jdbc.table.drop=true
                cache.jdbc.table.primarykey=jbosscache_pk
                cache.jdbc.fqn.column=fqn
                cache.jdbc.fqn.type=varchar(255)
                cache.jdbc.node.column=node
                cache.jdbc.node.type=blob
                cache.jdbc.parent.column=parent
                cache.jdbc.driver=oracle.jdbc.OracleDriver
                cache.jdbc.url=jdbc:oracle:thin:@localhost:1521:JBOSSDB
                cache.jdbc.user=SCOTT
                cache.jdbc.password=TIGER
            &lt;/properties&gt;
            &lt;!-- whether the cache loader writes are asynchronous --&gt;
            &lt;async&gt;false&lt;/async&gt;
            &lt;!-- only one cache loader in the chain may set fetchPersistentState to true.
                 An exception is thrown if more than one cache loader sets this to true. --&gt;
            &lt;fetchPersistentState&gt;true&lt;/fetchPersistentState&gt;
            &lt;!-- determines whether this cache loader ignores writes - defaults to false. --&gt;
            &lt;ignoreModifications&gt;false&lt;/ignoreModifications&gt;
            &lt;!-- if set to true, purges the contents of this cache loader when the cache starts up.
            Defaults to false.  --&gt;
            &lt;purgeOnStartup&gt;false&lt;/purgeOnStartup&gt;
        &lt;/cacheloader&gt;
    &lt;/config&gt;
&lt;/attribute&gt;</programlisting>
          </para>

          <para>As an alternative to configuring the entire JDBC connection,
          the name of an existing data source can be given:</para>

          <programlisting>&lt;attribute name="CacheLoaderConfiguration"&gt;
    &lt;config&gt;
        &lt;passivation&gt;false&lt;/passivation&gt;
        &lt;preload&gt;/some/stuff&lt;/preload&gt;
        &lt;cacheloader&gt;
            &lt;class&gt;org.jboss.cache.loader.JDBCCacheLoader&lt;/class&gt;
            &lt;!-- same as the old CacheLoaderConfig attribute --&gt;
            &lt;properties&gt;
                cache.jdbc.datasource=java:/DefaultDS
            &lt;/properties&gt;
            &lt;!-- whether the cache loader writes are asynchronous --&gt;
            &lt;async&gt;false&lt;/async&gt;
            &lt;!-- only one cache loader in the chain may set fetchPersistentState to true.
                 An exception is thrown if more than one cache loader sets this to true. --&gt;
            &lt;fetchPersistentState&gt;true&lt;/fetchPersistentState&gt;
            &lt;!-- determines whether this cache loader ignores writes - defaults to false. --&gt;
            &lt;ignoreModifications&gt;false&lt;/ignoreModifications&gt;
            &lt;!-- if set to true, purges the contents of this cache loader when the cache starts up.
            Defaults to false.  --&gt;
            &lt;purgeOnStartup&gt;false&lt;/purgeOnStartup&gt;
        &lt;/cacheloader&gt;
    &lt;/config&gt;
&lt;/attribute&gt;</programlisting>
        </section>
      </section>
    </section>
  </chapter>