# Managing Data with Infinispan 10.0

## 1. Eviction and Data Container

Infinispan supports eviction of entries, such that you do not run out of memory. Eviction is typically used in conjunction with a cache store, so that entries are not permanently lost when evicted, since eviction only removes entries from memory and not from cache stores or the rest of the cluster.

Infinispan supports storing data in a few different formats. Data can be stored as the object iself, binary as a byte[], and off-heap which stores the byte[] in native memory.

| Passivation is also a popular option when using eviction, so that only a single copy of an entry is maintained - either in memory or in a cache store, but not both. The main benefit of using passivation over a regular cache store is that updates to entries which exist in memory are cheaper since the update doesn’t need to be made to the cache store as well.

| Eviction occurs on a local basis, and is not cluster-wide. Each node runs an eviction thread to analyse the contents of its in-memory container and decide what to evict. Eviction does not take into account the amount of free memory in the JVM as threshold to starts evicting entries. You have to set `size` attribute of the eviction element to be greater than zero in order for eviction to be turned on. If size is too large you can run out of memory. The `size` attribute will probably take some tuning in each use case.

### 1.1. Enabling Eviction

Eviction is configured by adding the `<memory />` element to your `<*-cache />` configuration sections or using MemoryConfigurationBuilder API programmatic approach.

All cache entry are evicted by piggybacking on user threads that are hitting the cache.

#### 1.1.1. Eviction strategy

Strategies control how the eviction is handled.

The possible choices are

`NONE`

Eviction is not enabled and it is assumed that the user will not invoke evict directly on the cache. If passivation is enabled this will cause aa warning message to be emitted. This is the default strategy.

`MANUAL`

This strategy is just like <b>NONE</b> except that it asssumes the user will be invoking evict directly. This way if passivation is enabled no warning message is logged.

`REMOVE`

This strategy will actually evict "old" entries to make room for incoming ones.

Eviction is handled by Caffeine utilizing the TinyLFU algorithm with an additional admission window. This was chosen as provides high hit rate while also requiring low memory overhead. This provides a better hit ratio than LRU while also requiring less memory than LIRS.

`EXCEPTION`

This strategy actually prevents new entries from being created by throwing a `ContainerFullException`. This strategy only works with transactional caches that always run with 2 phase commit, that is no 1 phase commit or synchronization optimizations allowed.

#### 1.1.2. Eviction types

Eviction type applies only when the size is set to something greater than 0. The eviction type below determines when the container will decide to remove entries.

`COUNT`

This type of eviction will remove entries based on how many there are in the cache. Once the count of entries has grown larger than the `size` then an entry will be removed to make room.

`MEMORY`

This type of eviction will estimate how much each entry will take up in memory and will remove an entry when the total size of all entries is larger than the configured `size`. This type does not work with `OBJECT` storage type below.

#### 1.1.3. Storage type

Infinispan allows the user to configure in what form their data is stored. Each form supports the same features of Infinispan, however eviction can be limited for some forms. There are currently three storage formats that Infinispan provides, they are:

`OBJECT`

Stores the keys and values as objects in the Java heap Only `COUNT` eviction type is supported.

`BINARY`

Stores the keys and values as a byte[] in the Java heap. This will use the configured marshaller for the cache if there is one. Both `COUNT` and `MEMORY` eviction types are supported.

`OFF-HEAP`

Stores the keys and values in native memory outside of the Java heap as bytes. The configured marshaller will be used if the cache has one. Both `COUNT` and `MEMORY` eviction types are supported.

| Both `BINARY` and `OFF-HEAP` violate equality and hashCode that they are dictated by the resulting byte[] they generate instead of the object instance.

#### 1.1.4. More defaults

By default when no `<memory />` element is specified, no eviction takes place, `OBJECT` storage type is used, and a strategy of `NONE` is assumed.

In case there is an memory element, this table describes the behaviour of eviction based on information provided in the xml configuration ("-" in Supplied size or Supplied strategy column means that the attribute wasn’t supplied)

Supplied size | Example | Eviction behaviour
--- | --- | ---
- | `<memory />` | no eviction as an object
- | `<memory> <object strategy="MANUAL" /> </memory>` | no eviction as an object and won’t log warning if passivation is enabled
> 0 | `<memory> <object size="100" /> </memory>` | eviction takes place and stored as objects
> 0 | `<memory> <binary size="100" eviction="MEMORY"/> </memory>` | eviction takes place and stored as a binary removing to make sure memory doens’t go higher than 100
> 0 | `<memory> <off-heap size="100" /> </memory>` | eviction takes place and stored in off-heap
> 0 | `<memory> <off-heap size="100" strategy="EXCEPTION" /> </memory>` | entries are stored in off-heap and if 100 entries are in container exceptions will be thrown for additional
0 | `<memory> <object size="0" /> </memory>` | no eviction
< 0 | `<memory> <object size="-1" /> </memory>` | no eviction

### 1.2. Expiration

Similar to, but unlike eviction, is expiration. Expiration allows you to attach lifespan and/or maximum idle times to entries. Entries that exceed these times are treated as invalid and are removed. When removed expired entries are not passivated like evicted entries (if passivation is turned on).

| Unlike eviction, expired entries are removed globally - from memory, cache stores, and cluster-wide.

By default entries created are immortal and do not have a lifespan or maximum idle time. Using the cache API, mortal entries can be created with lifespans and/or maximum idle times. Further, default lifespans and/or maximum idle times can be configured by adding the <expiration /> element to your `<*-cache />` configuration sections.

When an entry expires it resides in the data container or cache store until it is accessed again by a user request. An expiration reaper is also available to check for expired entries and remove them at a configurable interval of milliseconds.

You can enable the expiration reaper declaratively with the `reaper-interval` attribute or programmatically with the `enableReaper` method in the `ExpirationConfigurationBuilder` class.

| The expiration reaper cannot be disabled when a cache store is present. When using a maximum idle time in a clustered cache, you should always enable the expiration reaper. For more information, see Clustered Max Idle.

#### 1.2.1. Difference between Eviction and Expiration

Both Eviction and Expiration are means of cleaning the cache of unused entries and thus guarding the heap against `OutOfMemory` exceptions, so now a brief explanation of the difference.

With eviction you set maximal number of entries you want to keep in the cache and if this limit is exceeded, some candidates are found to be removed according to a choosen eviction strategy (LRU, LIRS, etc…​). Eviction can be setup to work with passivation, which is eviction to a cache store.

With expiration you set time criteria for entries to specify how long you want to keep them in the cache.

lifespan

  Specifies how long entries can remain in the cache before they expire. The default value is `-1`, which is unlimited time.

maximum idle time

  Specifies how long entries can remain idle before they expire. An entry in the cache is idle when no operation is performed with the key. The default value is `-1`, which is unlimited time.

### 1.3. Expiration details

1. Expiration is a top-level construct, represented in the configuration as well as in the cache API.

2. While eviction is local to each cache instance , expiration is cluster-wide . Expiration `lifespan` and `maxIdle` values are replicated along with the cache entry.

3. Maximum idle times for cache entries require additional network messages in clustered environments. For this reason, setting `maxIdle` in clustered caches can result in slower operation times.

4. Expiration lifespan and `maxIdle` are also persisted in CacheStores, so this information survives eviction/passivation.

#### 1.3.1. Maximum Idle Expiration

Maximum idle expiration has different behavior in local and clustered cache environments.

##### Local Max Idle

In non-clustered cache environments, the `maxIdle` configuration expires entries when:

- accessed directly (`Cache.get`).

- iterated upon (`Cache.size`).

- the expiration reaper thread runs.

##### Clustered Max Idle

In clustered environments, nodes in the cluster can have different access times for the same entry. Entries do not expire from the cache until they reach the maxium idle time for all owners across the cluster.

When a node detects that an entry has reached the maximum idle time and is expired, the node gets the last time that the entry was accessed from the other owners in the cluster. If the other owners indicate that the entry is expired, that entry is not returned to the requester and removed from the cache.

The following points apply to using the `maxIdle` configuration with clustered caches:

- If one or more owner in the cluster detects that an entry is not expired, then a `Cache.get` operation returns the entry. The last access time for that entry is also updated to the current time.

- When the expiration reaper finds entries that might be expired with the maximum idle time, all nodes update the last access time for those entries to the most recent access time before the `maxIdle` time. In this way, the reaper prevents invalid expiration of entries.

- Clustered transactional caches do not remove entries that are expired with the maximum idle time on `Cache.get` operations. These expired entries are removed with the expiration reaper thread only, otherwise deadlocking can occur.

- Iteration across a clustered cache returns entries that might be expired with the maximum idle time. This behavior ensures performance because no remote invocations are performed during the iteration. However this does not refresh any expired entries, which are removed by the expiration reaper or when accessed directly (`Cache.get`).

| Clustered caches should always use the expiration reaper with the `maxIdle` configuration. When using `maxIdle` expiration with exception-based eviction, entries that are expired but not removed from the cache count towards the size of the data container.

#### 1.3.2. Configuration

Eviction and Expiration may be configured using the programmatic or declarative XML configuration. This configuration is on a per-cache basis. Valid eviction/expiration-related configuration elements are:

    <!-- Eviction -->
    <memory>
       <object size="2000"/>
    </memory>
    <!-- Expiration -->
    <expiration lifespan="1000" max-idle="500" interval="1000" />

Programmatically, the same would be defined using:

    Configuration c = new ConfigurationBuilder()
                   .memory().size(2000)
                   .expiration().wakeUpInterval(5000l).lifespan(1000l).maxIdle(500l)
                   .build();

#### 1.3.3. Memory Based Eviction Configuration

Memory based eviction may require some additional configuration options if you are using your own custom types (as Infinispan is normally used). In this case Infinispan cannot estimate the memory usage of your classes and as such you are required to use `storeAsBinary` when memory based eviction is used.

    <!-- Enable memory based eviction with 1 GB/> -->
    <memory>
       <binary size="1000000000" eviction="MEMORY"/>
    </memory>

    Configuration c = new ConfigurationBuilder()
                   .memory()
                   .storageType(StorageType.BINARY)
                   .evictionType(EvictionType.MEMORY)
                   .size(1_000_000_000)
                   .build();

#### 1.3.4. Default values

Eviction is disabled by default. Default values are used:

- size: -1 is used if not specified, which means unlimited entries.

- 0 means no entries, and the eviction thread will strive to keep the cache empty.

Expiration lifespan and maxIdle both default to -1, which means that entries will be created immortal by default. This can be overridden per entry with the API.

#### 1.3.5. Using expiration

Expiration allows you to set either a lifespan or a maximum idle time on each key/value pair stored in the cache. This can either be set cache-wide using the configuration, as described above, or it can be defined per-key/value pair using the Cache interface. Any values defined per key/value pair overrides the cache-wide default for the specific entry in question.

For example, assume the following configuration:

    <expiration lifespan="1000" />

    // this entry will expire in 1000 millis
    cache.put("pinot noir", pinotNoirPrice);

    // this entry will expire in 2000 millis
    cache.put("chardonnay", chardonnayPrice, 2, TimeUnit.SECONDS);

    // this entry will expire 1000 millis after it is last accessed
    cache.put("pinot grigio", pinotGrigioPrice, -1,
              TimeUnit.SECONDS, 1, TimeUnit.SECONDS);

    // this entry will expire 1000 millis after it is last accessed, or
    // in 5000 millis, which ever triggers first
    cache.put("riesling", rieslingPrice, 5,
              TimeUnit.SECONDS, 1, TimeUnit.SECONDS);

### 1.4. Expiration designs

Central to expiration is an ExpirationManager.

The purpose of the ExpirationManager is to drive the expiration thread which periodically purges items from the DataContainer. If the expiration thread is disabled (wakeupInterval set to -1) expiration can be kicked off manually using ExprationManager.processExpiration(), for example from another maintenance thread that may run periodically in your application.

The expiration manager processes expirations in the following manner:

1. Causes the data container to purge expired entries

2. Causes cache stores (if any) to purge expired entries

## 2. Persistence

Persistence allows configuring external (persistent) storage engines complementary to the default in memory storage offered by Infinispan. An external persistent storage might be useful for several reasons:

- Increased Durability. Memory is volatile, so a cache store could increase the life-span of the information store in the cache.

- Write-through. Interpose Infinispan as a caching layer between an application and a (custom) external storage engine.

- Overflow Data. By using eviction and passivation, one can store only the "hot" data in memory and overflow the data that is less frequently used to disk.

The integration with the persistent store is done through the following SPI: CacheLoader, CacheWriter, AdvancedCacheLoader and AdvancedCacheWriter (discussed in the following sections).

These SPIs allow for the following features:

- Alignment with JSR-107. The CacheWriter and CacheLoader interface are similar to the the loader and writer in JSR 107. This should considerably help writing portable stores across JCache compliant vendors.

- Simplified Transaction Integration. All necessary locking is handled by Infinispan automatically and implementations don’t have to be concerned with coordinating concurrent access to the store. Even though concurrent writes on the same key are not going to happen (depending locking mode in use), implementors should expect operations on the store to happen from multiple/different threads and code the implementation accordingly.

- Parallel Iteration. It is now possible to iterate over entries in the store with multiple threads in parallel.

- Reduced Serialization. This translates in less CPU usage. The new API exposes the stored entries in serialized format. If an entry is fetched from persistent storage for the sole purpose of being sent remotely, we no longer need to deserialize it (when reading from the store) and serialize it back (when writing to the wire). Now we can write to the wire the serialized format as read from the storage directly.

### 2.1. Configuration

Stores (readers and/or writers) can be configured in a chain. Cache read operation looks at all of the specified `CacheLoader` s, in the order they are configured, until it finds a valid and non-null element of data. When performing writes all cache `CacheWriter` s are written to, except if the `ignoreModifications` element has been set to true for a specific cache writer.

| Implementing both a CacheWriter and CacheLoader Store providers should implement both the `CacheWriter` and the `CacheLoader` interfaces. Stores that do this are considered both for reading and writing (assuming `read-only=false`) data.

This is the configuration of a custom (not shipped with infinispan) store:

    <local-cache name="myCustomStore">
       <persistence passivation="false">
          <store
             class="org.acme.CustomStore"
             fetch-state="false" preload="true" shared="false"
             purge="true" read-only="false" segmented="true">

             <write-behind modification-queue-size="123" thread-pool-size="23" />

             <property name="myProp">${system.property}</property>
          </store>
       </persistence>
    </local-cache>

Parameters that you can use to configure persistence are as follows:

`connection-attempts`

  Sets the maximum number of attempts to start each configured CacheWriter/CacheLoader. If the attempts to start are not successful, an exception is thrown and the cache does not start.

`connection-interval`

  Specifies the time, in milliseconds, to wait between connection attempts on startup. A negative or zero value means no wait between connection attempts.

`availability-interval`

  Specifies the time, in milliseconds, between availability checks to determine if the PersistenceManager is available. In other words, this interval sets how often stores/loaders are polled via their `org.infinispan.persistence.spi.CacheWriter#isAvailable` or `org.infinispan.persistence.spi.CacheLoader#isAvailable` implementation. If a single store/loader is not available, an exception is thrown during cache operations.

`passivation`

  Enables passivation. The default value is `false` (boolean).

  This property has a significant impact on Infinispan interactions with the loaders. See Cache Passivation for more information.

`class`

  Defines the class of the store and must implement `CacheLoader`, `CacheWriter`, or both.

`fetch-state`

  Fetches the persistent state of a cache when joining a cluster. The default value is `false` (boolean).

  The purpose of this property is to retrieve the persistent state of a cache and apply it to the local cache store of a node when it joins a cluster. Fetching the persistent state does not apply if a cache store is shared because it accesses the same data as the other stores.

  This property can be `true` for one configured cache loader only. If more than one cache loader fetches the persistent state, a configuration exception is thrown when the cache service starts.

`preload`

  Pre-loads data into memory from the cache loader when the cache starts. The default value is `false` (boolean).

  This property is useful when data in the cache loader is required immediately after startup to prevent delays with cache operations when the data is loaded lazily. This property can provide a "warm cache" on startup but it impacts performance because it affects start time.

  Pre-loading data is done locally, so any data loaded is stored locally in the node only. The pre-loaded data is not replicated or distributed. Likewise, Infinispan pre-loads data only up to the maximum configured number of entries in eviction.

`shared`

  Determines if the cache loader is shared between cache instances. The default value is `false` (boolean).

  This property prevents duplicate writes of data to the cache loader by different cache instances. An example is where all cache instances in a cluster use the same JDBC settings for the same remote, shared database.

`segmented`

  Configures a cache store to segment data. The default value is `false` (boolean).

  If `true` the cache store stores data in buckets. The `hash.numSegments` property configures how many buckets there are for storing data.

  Depending on the cache store implementation, segmenting data can cause slower write operations. However, performance improves for other cache operations. See Segmented Stores for more information.

`read-only`

  Prevents new data from being persisted to the cache store. The default value is `false` (boolean).

`purge`

  Empties the specified cache loader at startup. The default value is `false` (boolean). This property takes effect only if the `read-only` property is set to `false`.

`max-batch-size`

  Sets the maximum size of a batch to insert of delete from the cache store. The default value is #{AbstractStore-maxBatchSize}.

  If the value is less than `1`, no upper limit applies to the number of operations in a batch.

`write-behind`

  Asynchronously persists data to the cache store. The default value is `false` (boolean). See Asynchronous Write-Behind for more information.

| You can define additional attributes in the `properties` section to configure specific aspects of each cache loader, such as the `myProp` attribute in the previous example. Other cache loaders with more complex configurations also include additional properties. See the following JDBC cache store configuration for examples.

The preceding configuration applies a generic cache store implementation. However, the default Infinispan store implementation has a more complex configuration schema, in which the `properties` section is replaced with XML attributes:

    <persistence passivation="false">
       <!-- note that class is missing and is induced by the fileStore element name -->
       <file-store
               shared="false" preload="true"
               fetch-state="true"
               read-only="false"
               purge="false"
               path="${java.io.tmpdir}">
          <write-behind thread-pool-size="5" />
       </file-store>
    </persistence>

The same configuration can be achieved programmatically:

    ConfigurationBuilder builder = new ConfigurationBuilder();
    builder.persistence()
          .passivation(false)
          .addSingleFileStore()
             .preload(true)
             .shared(false)
             .fetchPersistentState(true)
             .ignoreModifications(false)
             .purgeOnStartup(false)
             .location(System.getProperty("java.io.tmpdir"))
             .async()
                .enabled(true)
                .threadPoolSize(5)

### 2.2. Cache Passivation

A CacheWriter can be used to enforce entry passivation and activation on eviction in a cache. Cache passivation is the process of removing an object from in-memory cache and writing it to a secondary data store (e.g., file system, database) on eviction. Cache activation is the process of restoring an object from the data store into the in-memory cache when it’s needed to be used. In order to fully support passivation, a store needs to be both a CacheWriter and a CacheLoader. In both cases, the configured cache store is used to read from the loader and write to the data writer.

When an eviction policy in effect evicts an entry from the cache, if passivation is enabled, a notification that the entry is being passivated will be emitted to the cache listeners and the entry will be stored. When a user attempts to retrieve a entry that was evicted earlier, the entry is (lazily) loaded from the cache loader into memory. When the entry has been loaded a notification is emitted to the cache listeners that the entry has been activated. In order to enable passivation just set passivation to true (false by default). When passivation is used, only the first cache loader configured is used and all others are ignored.

#### 2.2.1. Limitations

Due to the unique nature of passivation, it is not supported with some other store configurations.

- Transactional store - Passivation writes/removes entries from the store outside the scope of the actual Infinispan commit boundaries.

- Shared store - Shared store requires entries always being in the store for other owners. Thus passivation makes no sense as we can’t remove the entry from the store.

#### 2.2.2. Cache Loader Behavior with Passivation Disabled vs Enabled

When passivation is disabled, whenever an element is modified, added or removed, then that modification is persisted in the backend store via the cache loader. There is no direct relationship between eviction and cache loading. If you don’t use eviction, what’s in the persistent store is basically a copy of what’s in memory. If you do use eviction, what’s in the persistent store is basically a superset of what’s in memory (i.e. it includes entries that have been evicted from memory). When passivation is enabled, and with an unshared store, there is a direct relationship between eviction and the cache loader. Writes to the persistent store via the cache loader only occur as part of the eviction process. Data is deleted from the persistent store when the application reads it back into memory. In this case, what’s in memory and what’s in the persistent store are two subsets of the total information set, with no intersection between the subsets. With a shared store, entries which have been passivated in the past will continue to exist in the store, although they may have a stale value if this has been overwritten in memory.

The following is a simple example, showing what state is in RAM and in the persistent store after each step of a 6 step process:

Operation | Passivation Off | Passivation On, Shared Off | Passivation On, Shared On
--- | --- | --- | ---
Insert keyOne | Memory: keyOne Disk: keyOne | Memory: keyOne Disk: (none) | Memory: keyOne Disk: (none)
Insert keyTwo | Memory: keyOne, keyTwo Disk: keyOne, keyTwo | Memory: keyOne, keyTwo Disk: (none) | Memory: keyOne, keyTwo Disk: (none)
Eviction thread runs, evicts keyOne | Memory: keyTwo Disk: keyOne, keyTwo | Memory: keyTwo Disk: keyOne | Memory: keyTwo Disk: keyOne
Read keyOne | Memory: keyOne, keyTwo Disk: keyOne, keyTwo | Memory: keyOne, keyTwo Disk: (none) | Memory: keyOne, keyTwo Disk: keyOne
Eviction thread runs, evicts keyTwo | Memory: keyOne Disk: keyOne, keyTwo | Memory: keyOne Disk: keyTwo | Memory: keyOne Disk: keyOne, keyTwo
Remove keyTwo | Memory: keyOne Disk: keyOne | Memory: keyOne Disk: (none) | Memory: keyOne Disk: keyOne

### 2.3. Cache Loaders and transactional caches

When a cache is transactional and a cache loader is present, the cache loader won’t be enlisted in the transaction in which the cache is part. That means that it is possible to have inconsistencies at cache loader level: the transaction to succeed applying the in-memory state but (partially) fail applying the changes to the store. Manual recovery would not work with caches stores.

### 2.4. Write-Through And Write-Behind Caching

Infinispan can optionally be configured with one or several cache stores allowing it to store data in a persistent location such as shared JDBC database, a local filesystem, etc. Infinispan can handle updates to the cache store in two different ways:

- Write-Through (Synchronous)

- Write-Behind (Asynchronous)

#### 2.4.1. Write-Through (Synchronous)

In this mode, which is supported in version 4.0, when clients update a cache entry, i.e. via a Cache.put() invocation, the call will not return until Infinispan has gone to the underlying cache store and has updated it. Normally, this means that updates to the cache store are done within the boundaries of the client thread.

The main advantage of this mode is that the cache store is updated at the same time as the cache, hence the cache store is consistent with the cache contents. On the other hand, using this mode reduces performance because the latency of having to access and update the cache store directly impacts the duration of the cache operation.

Configuring a write-through or synchronous cache store does not require any particular configuration option. By default, unless marked explicitly as write-behind or asynchronous, all cache stores are write-through or synchronous. Please find below a sample configuration file of a write-through unshared local file cache store:

    <persistence passivation="false">
       <file-store fetch-state="true"
                   read-only="false"
                   purge="false" path="${java.io.tmpdir}"/>
    </persistence>

#### 2.4.2. Write-Behind (Asynchronous)

In this mode, updates to the cache are asynchronously written to the cache store. Infinispan puts pending changes into a modification queue so that it can quickly store changes.

The configured number of threads consume the queue and apply the modifications to the underlying cache store. If the configured number of threads cannot consume the modifications fast enough, or if the underlying store becomes unavailable, the modification queue becomes full. In this event, the cache store becomes write-through until the queue can accept new entries.

This mode provides an advantage in that cache operations are not affected by updates to the underlying store. However, because updates happen asynchronously, there is a period of time during which data in the cache store is inconsistent with data in the cache.

The write-behind strategy is suitable for cache stores with low latency and small operational cost; for example, an unshared file-based cache store that is local to the cache itself. In this case, the time during which data is inconsistent between the cache store and the cache is reduced to the lowest possible period.

The following is an example configuration for the write-behind strategy:

    <persistence passivation="false">
       <file-store fetch-state="true"
                   read-only="false"
                   purge="false" path="${java.io.tmpdir}">
       <!-- start write-behind configuration -->
       <write-behind modification-queue-size="123" thread-pool-size="23" />
       <!-- end write-behind configuration -->
       </file-store>
    </persistence>

#### 2.4.3. Segmented Stores

You can configure stores so that data resides in segments to which keys map. See Key Ownership for more information about segments and ownership.

Segmented stores increase read performance for bulk operations; for example, streaming over data (`Cache.size`, `Cache.entrySet.stream`), pre-loading the cache, and doing state transfer operations.

However, segmented stores can also result in loss of performance for write operations. This performance loss applies particularly to batch write operations that can take place with transactions or write-behind stores. For this reason, you should evaluate the overhead for write operations before you enable segmented stores. The performance gain for bulk read operations might not be acceptable if there is a significant performance loss for write operations.

| Loss of data can occur if the number of segments in a cache store are not changed gracefully. For this reason, if you change the `numSegments` setting in the store configuration, you must migrate the existing store to use the new configuration. The recommended method to migrate the cache store configuration is to perform a rolling upgrade. The store migrator supports migrating a non-segmented cache store to a segmented cache store only. The store migrator does not currently support migrating from a segmented cache store.

| Not all cache stores support segmentation. See the appropriate section for each store to determine if it supports segmentation. If you plan to convert or write a new store to support segmentation, see the following SPI section that provides more details.

### 2.5. Filesystem based cache stores

A filesystem-based cache store is typically used when you want to have a cache with a cache store available locally which stores data that has overflowed from memory, having exceeded size and/or time restrictions.

| Usage of filesystem-based cache stores on shared filesystems like NFS, Windows shares, etc. should be avoided as these do not implement proper file locking and can cause data corruption. File systems are inherently not transactional, so when attempting to use your cache in a transactional context, failures when writing to the file (which happens during the commit phase) cannot be recovered.

### 2.6. Single File Store

The single file cache store keeps all data in a single file. The way it looks up data is by keeping an in-memory index of keys and the positions of their values in this file. This results in greater performance compared to old file cache store. There is one caveat though. Since the single file based cache store keeps keys in memory, it can lead to increased memory consumption, and hence it’s not recommended for caches with big keys.

In certain use cases, this cache store suffers from fragmentation: if you store larger and larger values, the space is not reused and instead the entry is appended at the end of the file. The space (now empty) is reused only if you write another entry that can fit there. Also, when you remove all entries from the cache, the file won’t shrink, and neither will be de-fragmented.

These are the available configuration options for the single file cache store:

- `path` where data will be stored. (e.g., `path="/tmp/myDataStore"`). By default, the location is `Infinispan-SingleFileStore`.

- `max-entries` specifies the maximum number of entries to keep in this file store. As mentioned before, in order to speed up lookups, the single file cache store keeps an index of keys and their corresponding position in the file. To avoid this index resulting in memory consumption problems, this cache store can bounded by a maximum number of entries that it stores. If this limit is exceeded, entries are removed permanently using the LRU algorithm both from the in-memory index and the underlying file based cache store. So, setting a maximum limit only makes sense when Infinispan is used as a cache, whose contents can be recomputed or they can be retrieved from the authoritative data store. If this maximum limit is set when the Infinispan is used as an authoritative data store, it could lead to data loss, and hence it’s not recommended for this use case. The default value is `-1` which means that the file store size is unlimited.

#### 2.6.1. Segmentation support

The single file cache store supports segmentation and creates a separate instance per segment. Segmentation results in multiple directories under the configured directory, where each directory is a number that represents the segment to which the data maps.

#### 2.6.2. Configuration

The following examples show single file cache store configuration:

    <persistence>
       <file-store path="/tmp/myDataStore" max-entries="5000"/>
    </persistence>

    ConfigurationBuilder b = new ConfigurationBuilder();
    b.persistence()
        .addSingleFileStore()
        .location("/tmp/myDataStore")
        .maxEntries(5000);

### 2.7. Soft-Index File Store

The Soft-Index File Store is an experimental local file-based. It is a pure Java implementation that tries to get around Single File Store’s drawbacks by implementing a variant of B+ tree that is cached in-memory using Java’s soft references - here’s where the name Soft-Index File Store comes from. This B+ tree (called Index) is offloaded on filesystem to single file that does not need to be persisted - it is purged and rebuilt when the cache store restarts, its purpose is only offloading.

The data that should be persisted are stored in a set of files that are written in append-only way - that means that if you store this on conventional magnetic disk, it does not have to seek when writing a burst of entries. It is not stored in single file but set of files. When the usage of any of these files drops below 50% (the entries from the file are overwritten to another file), the file starts to be collected, moving the live entries into different file and in the end removing that file from disk.

Most of the structures in Soft Index File Store are bounded, therefore you don’t have to be afraid of OOMEs. For example, you can configure the limits for concurrently open files as well.

#### 2.7.1. Segmentation support

The Soft-Index file store supports segmentation and creates a separate instance per segment. Segmentation results in multiple directories under the configured directory, where each directory is a number that represents the segment to which the data maps.

#### 2.7.2. Configuration

Here is an example of Soft-Index File Store configuration via XML:

    <persistence>
        <soft-index-file-store xmlns="urn:infinispan:config:store:soft-index:8.0">
            <index path="/tmp/sifs/testCache/index" />
            <data path="/tmp/sifs/testCache/data" />
        </soft-index-file-store>
    </persistence>

Programmatic configuration would look as follows:

    ConfigurationBuilder b = new ConfigurationBuilder();
    b.persistence()
        .addStore(SoftIndexFileStoreConfigurationBuilder.class)
            .indexLocation("/tmp/sifs/testCache/index");
            .dataLocation("/tmp/sifs/testCache/data")

#### 2.7.3. Current limitations

Size of a node in the Index is limited, by default it is 4096 bytes, though it can be configured. This size also limits the key length (or rather the length of the serialized form): you can’t use keys longer than size of the node - 15 bytes. Moreover, the key length is stored as 'short', limiting it to 32767 bytes. There’s no way how you can use longer keys - SIFS throws an exception when the key is longer after serialization.

When entries are stored with expiration, SIFS cannot detect that some of those entries are expired. Therefore, such old file will not be compacted (method AdvancedStore.purgeExpired() is not implemented). This can lead to excessive file-system space usage.

### 2.8. JDBC String based Cache Store

A cache store which relies on the provided JDBC driver to load/store values in the underlying database.

Each key in the cache is stored in its own row in the database. In order to store each key in its own row, this store relies on a (pluggable) bijection that maps the each key to a String object. The bijection is defined by the Key2StringMapper interface. Infinispans ships a default implementation (smartly named DefaultTwoWayKey2StringMapper) that knows how to handle primitive types.

| By default Infinispan shares are not stored, meaning that all nodes in the cluster will write to the underlying store upon each update. If you wish for an operation to only be written to the underlying database once, you must configure the JDBC store to be shared.

| The JDBC string-based cache store does not support segmentation. Support will be available in a future release.

#### 2.8.1. Connection management (pooling)

In order to obtain a connection to the database the JDBC cache store relies on a ConnectionFactory implementation. The connection factory is specified programmatically using one of the connectionPool(), dataSource() or simpleConnection() methods on the JdbcStringBasedStoreConfigurationBuilder class or declaratively using one of the `<connectionPool />`, `<dataSource />` or `<simpleConnection />` elements.

Infinispan ships with three ConnectionFactory implementations:

- PooledConnectionFactoryConfigurationBuilder is a factory based on Agroal, which is configured via the PooledConnectionFactoryConfiguration or by specifying a properties file via `PooledConnectionFactoryConfiguration.propertyFile`. Properties must be specified with the prefix "org.infinispan.agroal.". An example `agroal.properties` file is shown below:

    org.infinispan.agroal.metricsEnabled=false

    org.infinispan.agroal.minSize=10
    org.infinispan.agroal.maxSize=100
    org.infinispan.agroal.initialSize=20
    org.infinispan.agroal.acquisitionTimeout_s=1
    org.infinispan.agroal.validationTimeout_m=1
    org.infinispan.agroal.leakTimeout_s=10
    org.infinispan.agroal.reapTimeout_m=10

    org.infinispan.agroal.metricsEnabled=false
    org.infinispan.agroal.autoCommit=true
    org.infinispan.agroal.jdbcTransactionIsolation=READ_COMMITTED
    org.infinispan.agroal.jdbcUrl=jdbc:h2:mem:PooledConnectionFactoryTest;DB_CLOSE_DELAY=-1
    org.infinispan.agroal.driverClassName=org.h2.Driver.class
    org.infinispan.agroal.principal=sa
    org.infinispan.agroal.credential=sa

- ManagedConnectionFactoryConfigurationBuilder is a connection factory that can be used within managed environments, such as application servers. It knows how to look into the JNDI tree at a certain location (configurable) and delegate connection management to the DataSource.

- SimpleConnectionFactoryConfigurationBuilder is a factory implementation that will create database connection on a per invocation basis. Not recommended in production.

The `PooledConnectionFactory` is generally recommended for stand-alone deployments (i.e. not running within AS or servlet container). `ManagedConnectionFactory` can be used when running in a managed environment where a `DataSource` is present, so that connection pooling is performed within the `DataSource`.

#### 2.8.2. Sample configurations

Below is a sample configuration for the JdbcStringBasedStore. For detailed description of all the parameters used refer to the JdbcStringBasedStore.

    <persistence>
       <string-keyed-jdbc-store xmlns="urn:infinispan:config:store:jdbc:9.2" shared="true" fetch-state="false" read-only="false" purge="false">
          <connection-pool connection-url="jdbc:h2:mem:infinispan_string_based;DB_CLOSE_DELAY=-1" username="sa" driver="org.h2.Driver"/>
          <string-keyed-table drop-on-exit="true" create-on-start="true" prefix="ISPN_STRING_TABLE">
             <id-column name="ID_COLUMN" type="VARCHAR(255)" />
             <data-column name="DATA_COLUMN" type="BINARY" />
             <timestamp-column name="TIMESTAMP_COLUMN" type="BIGINT" />
          </string-keyed-table>
       </string-keyed-jdbc-store>
    </persistence>

    ConfigurationBuilder builder = new ConfigurationBuilder();
    builder.persistence().addStore(JdbcStringBasedStoreConfigurationBuilder.class)
          .fetchPersistentState(false)
          .ignoreModifications(false)
          .purgeOnStartup(false)
          .shared(true)
          .table()
             .dropOnExit(true)
             .createOnStart(true)
             .tableNamePrefix("ISPN_STRING_TABLE")
             .idColumnName("ID_COLUMN").idColumnType("VARCHAR(255)")
             .dataColumnName("DATA_COLUMN").dataColumnType("BINARY")
             .timestampColumnName("TIMESTAMP_COLUMN").timestampColumnType("BIGINT")
          .connectionPool()
             .connectionUrl("jdbc:h2:mem:infinispan_string_based;DB_CLOSE_DELAY=-1")
             .username("sa")
             .driverClass("org.h2.Driver");

Finally, below is an example of a JDBC cache store with a managed connection factory, which is chosen implicitly by specifying a datasource JNDI location:

    <string-keyed-jdbc-store xmlns="urn:infinispan:config:store:jdbc:9.2" shared="true" fetch-state="false" read-only="false" purge="false">
       <data-source jndi-url="java:/StringStoreWithManagedConnectionTest/DS" />
       <string-keyed-table drop-on-exit="true" create-on-start="true" prefix="ISPN_STRING_TABLE">
          <id-column name="ID_COLUMN" type="VARCHAR(255)" />
          <data-column name="DATA_COLUMN" type="BINARY" />
          <timestamp-column name="TIMESTAMP_COLUMN" type="BIGINT" />
       </string-keyed-table>
    </string-keyed-jdbc-store>

    ConfigurationBuilder builder = new ConfigurationBuilder();
    builder.persistence().addStore(JdbcStringBasedStoreConfigurationBuilder.class)
          .fetchPersistentState(false)
          .ignoreModifications(false)
          .purgeOnStartup(false)
          .shared(true)
          .table()
             .dropOnExit(true)
             .createOnStart(true)
             .tableNamePrefix("ISPN_STRING_TABLE")
             .idColumnName("ID_COLUMN").idColumnType("VARCHAR(255)")
             .dataColumnName("DATA_COLUMN").dataColumnType("BINARY")
             .timestampColumnName("TIMESTAMP_COLUMN").timestampColumnType("BIGINT")
          .dataSource()
             .jndiUrl("java:/StringStoreWithManagedConnectionTest/DS");

| Apache Derby users If you’re connecting to an Apache Derby database, make sure you set dataColumnType to BLOB: `<data-column name="DATA_COLUMN" type="BLOB"/>`

### 2.9. Remote store

The `RemoteStore` is a cache loader and writer implementation that stores data in a remote Infinispan cluster. In order to communicate with the remote cluster, the `RemoteStore` uses the HotRod client/server architecture. HotRod bering the load balancing and fault tolerance of calls and the possibility to fine-tune the connection between the RemoteCacheStore and the actual cluster. Please refer to Hot Rod for more information on the protocol, client and server configuration. For a list of RemoteStore configuration refer to the javadoc . Example:

#### 2.9.1. Segmentation support

The `RemoteStore` store supports segmentation because it can publish keys and entries by segment, allowing for more efficient bulk operations.

Segmentation is only supported when the remote server supports at least protocol version 2.3 or newer.

| Ensure the number of segments and hash are the same between the store configured cache and the remote server otherwise bulk operations will not return correct results.

#### 2.9.2. Sample Usage

    <persistence>
       <remote-store xmlns="urn:infinispan:config:store:remote:8.0" cache="mycache" raw-values="true">
          <remote-server host="one" port="12111" />
          <remote-server host="two" />
          <connection-pool max-active="10" exhausted-action="CREATE_NEW" />
          <write-behind />
       </remote-store>
    </persistence>

    ConfigurationBuilder b = new ConfigurationBuilder();
    b.persistence().addStore(RemoteStoreConfigurationBuilder.class)
          .fetchPersistentState(false)
          .ignoreModifications(false)
          .purgeOnStartup(false)
          .remoteCacheName("mycache")
          .rawValues(true)
    .addServer()
          .host("one").port(12111)
          .addServer()
          .host("two")
          .connectionPool()
          .maxActive(10)
          .exhaustedAction(ExhaustedAction.CREATE_NEW)
          .async().enable();

In this sample configuration, the remote cache store is configured to use the remote cache named "mycache" on servers "one" and "two". It also configures connection pooling and provides a custom transport executor. Additionally the cache store is asynchronous.

### 2.10. Cluster cache loader

The ClusterCacheLoader is a cache loader implementation that retrieves data from other cluster members.

#### 2.10.1. ClusterCacheLoader

It is a cache loader only as it doesn’t persist anything (it is not a Store), therefore features like fetchPersistentState (and like) are not applicable.

A cluster cache loader can be used as a non-blocking (partial) alternative to stateTransfer : keys not already available in the local node are fetched on-demand from other nodes in the cluster. This is a kind of lazy-loading of the cache content.

| The cluster cache loader does not support segmentation.

    <persistence>
       <cluster-loader remote-timeout="500"/>
    </persistence>

    ConfigurationBuilder b = new ConfigurationBuilder();
    b.persistence()
        .addClusterLoader()
        .remoteCallTimeout(500);

For a list of ClusterCacheLoader configuration refer to the javadoc .

| The ClusterCacheLoader does not support preloading (`preload=true`). It also won’t provide state if `fetchPersistentSate=true`.

### 2.11. Command-Line Interface cache loader

The Command-Line Interface (CLI) cache loader is a cache loader implementation that retrieves data from another Infinispan node using the CLI. The node to which the CLI connects to could be a standalone node, or could be a node that it’s part of a cluster. This cache loader is read-only, so it will only be used to retrieve data, and hence, won’t be used when persisting data.

#### 2.11.1. CLI Cache Loader

The CLI cache loader is configured with a connection URL pointing to the Infinispan node to which connect to. Here is an example:

| The Command-Line Interface (CLI) cache loader does not support segmentation.

    <persistence>
       <cli-loader connection="jmx://1.2.3.4:4444/MyCacheManager/myCache" />
    </persistence>

    ConfigurationBuilder b = new ConfigurationBuilder();
    b.persistence()
        .addStore(CLInterfaceLoaderConfigurationBuilder.class)
        .connectionString("jmx://192.0.2.0:4444/MyCacheManager/myCache");

### 2.12. RocksDB Cache Store

Infinispan supports using RocksDB as a cache store.

#### 2.12.1. Introduction

RocksDB is a fast key-value filesystem-based storage from Facebook. It started as a fork of Google’s LevelDB, but provides superior performance and reliability, especially in highly concurrent scenarios.

#### 2.12.2. Segmentation support

The RocksDB cache store supports segmentation and creates a separate column family per segment, which substantially improves lookup performance and iteration. However, write operations are a little slower when the cache store is segmented.

| You should not configure more than a few hundred segments. RocksDB is not designed to have an unlimited number of column families. Too many segments also significantly increases startup time for the cache store.

##### Sample Usage

The RocksDB cache store requires 2 filesystem directories to be configured - each directory contains a RocksDB database: one location is used to store non-expired data, while the second location is used to store expired keys pending purge.

    Configuration cacheConfig = new ConfigurationBuilder().persistence()
                                    .addStore(RocksDBStoreConfigurationBuilder.class)
                                    .build();
    EmbeddedCacheManager cacheManager = new DefaultCacheManager(cacheConfig);

    Cache<String, User> usersCache = cacheManager.getCache("usersCache");
    usersCache.put("raytsang", new User(...));

#### 2.12.3. Configuration

It is also possible to configure the underlying rocks db instance. This can be done via properties in the store configuration. Any property that is prefixed with the name `database` will configure the rocks db database. Data is now stored in column families, these can be configured independently of the database by setting a property prefixed with the name `data`.

Note that you do not have to supply properties and this is entirely optional.

##### Sample Programatic Configuration

    Properties props = new Properties();
    props.put("database.max_background_compactions", "2");
    props.put("data.write_buffer_size", "512MB");

    Configuration cacheConfig = new ConfigurationBuilder().persistence()
                                    .addStore(RocksDBStoreConfigurationBuilder.class)
                                    .location("/tmp/rocksdb/data")
                                    .expiredLocation("/tmp/rocksdb/expired")
            .properties(props)
                                    .build();

Parameter | Description
--- | ---
location | Directory to use for RocksDB to store primary cache store data. The directory will be auto-created if it does not exit.
expiredLocation | Directory to use for RocksDB to store expiring data pending to be purged permanently. The directory will be auto-created if it does not exit.
expiryQueueSize | Size of the in-memory queue to hold expiring entries before it gets flushed into expired RocksDB store
clearThreshold | There are two methods to clear all entries in RocksDB. One method is to iterate through all entries and remove each entry individually. The other method is to delete the database and re-init. For smaller databases, deleting individual entries is faster than the latter method. This configuration sets the max number of entries allowed before using the latter method
compressionType | Configuration for RocksDB for data compression, see CompressionType enum for options
blockSize | Configuration for RocksDB - see documentation for performance tuning
cacheSize | Configuration for RocksDB - see documentation for performance tuning

##### Sample XML Configuration

infinispan.xml

    <local-cache name="vehicleCache">
       <persistence>
          <rocksdb-store path="/tmp/rocksdb/data">
             <expiration path="/tmp/rocksdb/expired"/>
             <property name="database.max_background_compactions">2</property>
             <property name="data.write_buffer_size">512MB</property>
          </rocksdb-store>
       </persistence>
    </local-cache>

#### 2.12.4. Additional References

Refer to the test case for code samples in action.

Refer to test configurations for configuration samples.

### 2.13. JPA Cache Store

The implementation depends on JPA 2.0 specification to access entity meta model.

In normal use cases, it’s recommended to leverage Infinispan for JPA second level cache and/or query cache. However, if you’d like to use only Infinispan API and you want Infinispan to persist into a cache store using a common format (e.g., a database with well defined schema), then JPA Cache Store could be right for you.

Things to note

- When using JPA Cache Store, the key should be the ID of the entity, while the value should be the entity object.

- Only a single `@Id` or `@EmbeddedId` annotated property is allowed.

- Auto-generated ID is not supported.

- Lastly, all entries will be stored as immortal entries.

| The JPA cache store does not support segmentation.

#### 2.13.1. Sample Usage

For example, given a persistence unit "myPersistenceUnit", and a JPA entity User:

persistence.xml

    <persistence-unit name="myPersistenceUnit">
            ...
    </persistence-unit>

User entity class

User.java

    @Entity
    public class User implements Serializable {
            @Id
            private String username;
            private String firstName;
            private String lastName;

            ...
    }

Then you can configure a cache "usersCache" to use JPA Cache Store, so that when you put data into the cache, the data would be persisted into the database based on JPA configuration.

    EmbeddedCacheManager cacheManager = ...;

    Configuration cacheConfig = new ConfigurationBuilder().persistence()
                .addStore(JpaStoreConfigurationBuilder.class)
                .persistenceUnitName("org.infinispan.loaders.jpa.configurationTest")
                .entityClass(User.class)
                .build();
    cacheManager.defineCache("usersCache", cacheConfig);

    Cache<String, User> usersCache = cacheManager.getCache("usersCache");
    usersCache.put("raytsang", new User(...));

Normally a single Infinispan cache can store multiple types of key/value pairs, for example:

    Cache<String, User> usersCache = cacheManager.getCache("myCache");
    usersCache.put("raytsang", new User());
    Cache<Integer, Teacher> teachersCache = cacheManager.getCache("myCache");
    teachersCache.put(1, new Teacher());

It’s important to note that, when a cache is configured to use a JPA Cache Store, that cache would only be able to store ONE type of data.

    Cache<String, User> usersCache = cacheManager.getCache("myJPACache"); // configured for User entity class
    usersCache.put("raytsang", new User());
    Cache<Integer, Teacher> teachersCache = cacheManager.getCache("myJPACache"); // cannot do this when this cache is configured to use a JPA cache store
    teachersCache.put(1, new Teacher());

Use of `@EmbeddedId` is supported so that you can also use composite keys.

    @Entity
    public class Vehicle implements Serializable {
            @EmbeddedId
            private VehicleId id;
            private String color;        ...
    }

    @Embeddable
    public class VehicleId implements Serializable
    {
            private String state;
            private String licensePlate;
            ...
    }

Lastly, auto-generated IDs ﻿(e.g., `@GeneratedValue`) is not supported. When putting things into the cache with a JPA cache store, the key should be the ID value!

#### 2.13.2. Configuration

##### Sample Programmatic Configuration

    Configuration cacheConfig = new ConfigurationBuilder().persistence()
                 .addStore(JpaStoreConfigurationBuilder.class)
                 .persistenceUnitName("org.infinispan.loaders.jpa.configurationTest")
                 .entityClass(User.class)
                 .build();

Parameter | Description
--- | ---
persistenceUnitName | JPA persistence unit name in JPA configuration ﻿(persistence.xml) that contains the JPA entity class
entityClass | JPA entity class that is expected to be stored in this cache. Only one class is allowed.

##### Sample XML Configuration

    <local-cache name="vehicleCache">
       <persistence passivation="false">
          <jpa-store xmlns="urn:infinispan:config:store:jpa:7.0"
             persistence-unit="org.infinispan.persistence.jpa.configurationTest"
             entity-class="org.infinispan.persistence.jpa.entity.Vehicle">
                    />
       </persistence>
    </local-cache>

Parameter | Description
--- | ---
persistence-unit | JPA persistence unit name in JPA configuration ﻿(persistence.xml) that contains the JPA entity class
entity-class | Fully qualified JPA entity class name that is expected to be stored in this cache. Only one class is allowed.

#### 2.13.3. Additional References

Refer to the test case for code samples in action.

Refer to test configurations for configuration samples.

### 2.14. Custom Cache Stores

If the provided cache stores do not fulfill all of your requirements, it is possible for you to implement your own store. The steps required to create your own store are as follows:

1. Write your custom store by implementing one of the following interfaces:

   - `org.infinispan.persistence.spi.AdvancedCacheWriter`

   - `org.infinispan.persistence.spi.AdvancedCacheLoader`

   - `org.infinispan.persistence.spi.CacheLoader`

   - `org.infinispan.persistence.spi.CacheWriter`

   - `org.infinispan.persistence.spi.ExternalStore`

   - `org.infinispan.persistence.spi.AdvancedLoadWriteStore`

   - `org.infinispan.persistence.spi.TransactionalCacheWriter`

   - `org.infinispan.persistence.spi.SegmentedAdvancedLoadWriteStore`

2. Annotate your store class with the `@Store` annotation and specify the properties relevant to your store, e.g. is it possible for the store to be shared in Replicated or Distributed mode: `@Store(shared = true)`.

3. Create a custom cache store configuration and builder. This requires extending `AbstractStoreConfiguration` and `AbstractStoreConfigurationBuilder`. As an optional step, you should add the following annotations to your configuration - `@ConfigurationFor`, `@BuiltBy` as well as adding `@ConfiguredBy` to your store implementation class. These additional annotations will ensure that your custom configuration builder is used to parse your store configuration from xml. If these annotations are not added, then the `CustomStoreConfigurationBuilder` will be used to parse the common store attributes defined in `AbstractStoreConfiguration` and any additional elements will be ignored. If a store and its configuration do not declare the `@Store` and `@ConfigurationFor` annotations respectively, a warning message will be logged upon cache initialisation.

   If you wish for your store to be segmented, where it will craete a different store instance per segment, instead of extending `AbstractStoreConfiguration` you should extend `AbstractSegmentedStoreConfiguration`.

4. Add your custom store to your cache’s configuration:

   1. Add your custom store to the ConfigurationBuilder, for example:

          Configuration config = new ConfigurationBuilder()
                      .persistence()
                      .addStore(CustomStoreConfigurationBuilder.class)
                      .build();

   2. Define your custom store via xml:

          <local-cache name="customStoreExample">
            <persistence>
              <store class="org.infinispan.persistence.dummy.DummyInMemoryStore" />
            </persistence>
          </local-cache>

#### 2.14.1. HotRod Deployment

A Custom Cache Store can be packaged into a separate JAR file and deployed in a HotRod server using the following steps:

1. Follow Custom Cache Stores, steps 1-3>> in the previous section and package your implementations in a JAR file (or use a Custom Cache Store Archetype).

2. In your Jar create a proper file under `META-INF/services/`, which contains the fully qualified class name of your store implementation. The name of this service file should reflect the interface that your store implements. For example, if your store implements the `AdvancedCacheWriter` interface than you need to create the following file:

   - `/META-INF/services/org.infinispan.persistence.spi.AdvancedCacheWriter`

3. Deploy the JAR file in the Infinispan Server.

### 2.15. Store Migrator

Infinispan 9.0 introduced changes to internal marshalling functionality that are not backwardly compatible with previous versions of Infinispan. As a result, Infinispan 9.x and later cannot read cache stores created in earlier versions of Infinispan. Additionally, Infinispan no longer provides some store implementations such as JDBC Mixed and Binary stores.

You can use `StoreMigrator.java` to migrate cache stores. This migration tool reads data from cache stores in previous versions and rewrites the content for compatibility with the current marshalling implementation.

#### 2.15.1. Migrating Cache Stores

To perform a migration with `StoreMigrator`,

1. Put `infinispan-tools-10.0.jar` and dependencies for your source and target databases, such as JDBC drivers, on your classpath.

2. Create a `.properties` file that contains configuration properties for the source and target cache stores.

   You can find an example properties file that contains all applicable configuration options in migrator.properties.

3. Specify `.properties` file as an argument for `StoreMigrator`.

4. Run `mvn exec:java` to execute the migrator.

See the following example Maven `pom.xml` for `StoreMigrator`:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>

        <groupId>org.infinispan.example</groupId>
        <artifactId>jdbc-migrator-example</artifactId>
        <version>1.0-SNAPSHOT</version>

        <dependencies>
            <dependency>
                <groupId>org.infinispan</groupId>
                <artifactId>infinispan-tools</artifactId>
                <!-- Replace ${version.infinispan} with the
                version of Infinispan that you're using. -->
                <version>${version.infinispan}</version>
            </dependency>

            <!-- ADD YOUR REQUIRED DEPENDENCIES HERE -->
        </dependencies>

        <build>
            <plugins>
                <plugin>
                    <groupId>org.codehaus.mojo</groupId>
                    <artifactId>exec-maven-plugin</artifactId>
                    <version>1.2.1</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>java</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <mainClass>StoreMigrator</mainClass>
                        <arguments>
                            <argument><!-- PATH TO YOUR MIGRATOR.PROPERTIES FILE --></argument>
                        </arguments>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </project>

#### 2.15.2. Store Migrator Properties

All migrator properties are configured within the context of a source or target store. Each property must start with either `source.` or `target.`.

All properties in the following sections apply to both source and target stores, except for `table.binary.*` properties because it is not possible to migrate to a binary table.

##### Common Properties

Property | Description | Example value | Required
--- | --- | --- | ---
type | JDBC_STRING | JDBC_BINARY | JDBC_MIXED | LEVELDB | ROCKSDB | SINGLE_FILE_STORE | SOFT_INDEX_FILE_STORE | JDBC_MIXED | TRUE
cache_name | The name of the cache associated with the store | persistentMixedCache | TRUE
segment_count | How many segments this store will be created with. If not provided store will not be segmented. (supported as target only - JDBC not yet supported) | null | FALSE

It should be noted that the segment_count property should match how many segments your cache will be using. That is that it should match the `clustering.hash.numSegments` config value. If these do not match, data will not be properly read when running the cache.

##### JDBC Properties

Property | Description | Example value | Required
--- | --- | --- | ---
dialect | The dialect of the underlying database | POSTGRES | TRUE
marshaller.type | The marshaller to use for the store. Possible values are: - `LEGACY` Infinispan 8.2.x marshaller. Valid for source stores only. - `CURRENT` Infinispan 9.x marshaller. - `CUSTOM` Custom marshaller. | CURRENT | TRUE
marshaller.class | The class of the marshaller if type=CUSTOM | org.example.CustomMarshaller |
marshaller.externalizers | A comma-separated list of custom AdvancedExternalizer implementations to load `[id]:<Externalizer class>` | `25:Externalizer1,org.example.Externalizer2` |
connection_pool.connection_url | The JDBC connection url | `jdbc:postgresql:postgres` | TRUE
connection_pool.driver_class | The class of the JDBC driver | org.postrgesql.Driver | TRUE
connection_pool.username | Database username |  | TRUE
connection_pool.password | Database password |  | TRUE
db.major_version | Database major version | 9 |
db.minor_version | Database minor version | 5 |
db.disable_upsert | Disable db upsert | false |
db.disable_indexing | Prevent table index being created | false |
table.`<binary|string>`.table_name_prefix | Additional prefix for table name | tablePrefix |
table.`<binary|string>`.`<id|data|timestamp>`.name | Name of the column | id_column | TRUE
table.`<binary|string>`.`<id|data|timestamp>`.type | Type of the column | VARCHAR | TRUE
key_to_string_mapper | TwoWayKey2StringMapper Class | `org.infinispan.persistence.keymappers. DefaultTwoWayKey2StringMapper` |

##### LevelDB/RocksDB Properties

Property | Description | Example value | Required
--- | --- | --- | ---
location | The location of the db directory | /some/example/dir | TRUE
compression | The compression type to be used | SNAPPY |

##### SingleFileStore Properties

Property | Description | Example value | Required
--- | --- | --- | ---
location | The directory containing the store’s .dat file | /some/example/dir | TRUE

##### SoftIndexFileStore Properties

Property | Description | Example value | Required
--- | --- | --- | ---
location | The location of the db directory | /some/example/dir | TRUE
index_location | The location of the db’s index | /some/example/dir-index | Target Only

### 2.16. SPI

The following class diagram presents the main SPI interfaces of the persistence API:

Figure 1. Persistence SPI

Some notes about the classes:

- ByteBuffer - abstracts the serialized form of an object

- MarshalledEntry - abstracts the information held within a persistent store corresponding to a key-value added to the cache. Provides method for reading this information both in serialized (ByteBuffer) and deserialized (Object) format. Normally data read from the store is kept in serialized format and lazily deserialized on demand, within the MarshalledEntry implementation

- CacheWriter and CacheLoader provide basic methods for reading and writing to a store

- AdvancedCacheLoader and AdvancedCacheWriter provide operations to manipulate the underlaying storage in bulk: parallel iteration and purging of expired entries, clear and size.

- SegmentedAdvancedLoadWriteStore provide all the various operations that deal with segments.

A cache store can be segmented if it does one of the following:

- Implements the SegmentedAdvancedLoadWriteStore interface. In this case only a single store instance is used per cache.

- Has a configuration that extends the AbstractSegmentedConfiguration abstract class. Doing this requires you to implement the `newConfigurationFrom` method where it is expected that a new `StoreConfiguration` instance is created per invocation. This creates a store instance per segment to which a node can write. Stores might start and stop as data is moved between nodes.

A provider might choose to only implement a subset of these interfaces:

- Not implementing the AdvancedCacheWriter makes the given writer not usable for purging expired entries or clear

- If a loader does not implement the AdvancedCacheLoader inteface, then it will not participate in preloading nor in cache iteration (required also for stream operations).

If you’re looking at migrating your existing store to the new API or to write a new store implementation, the SingleFileStore might be a good starting point/example.

#### 2.16.1. More implementations

Many more cache loader and cache store implementations exist. Visit this website for more details.