[[Distributable_Jakarta_Enterprise_Beans_Applications]]
= Distributable Jakarta Enterprise Beans Applications

Just as with standard web applications, session state of stateful session beans (SFSB) contained in a standard EJB application
is not guaranteed to survive beyond the lifespan of the Jakarta Enterprise Beans container. And, as with standard web applications, there *is* a way to
allow session state of SFSBs to survive beyond the lifespan of a single server, either through persistence or by replicating state to
other nodes in the cluster.

A _distributable_ SFSB is one whose state is made available on multiple nodes in a cluster and which supports failover of invocation attempts: if the node
on which the SFSB was created fails, the invocation will be retried on another node in the cluster where the SFSB state is present.

In the case of Jakarta Enterprise Beans applications, whether or not a bean is distributable is determined globally or on a per-bean basis, rather than on
an application-wide basis as in the case of distributed HttpSessions.

A stateful session bean within an Jakarta Enterprise Beans application indicates its intention to be distributable by using a passivation-capable cache
to store its session state. Cache factories were discussed in the link:Admin_Guide{outfilesuffix}#caches[Jakarta Enterprise Beans section of the Wildfly Admin Guide].
Additionally, the EJB application needs to be deployed into a server which uses an High Availability (HA) server profile, such as standalone-ha.xml
or standalone-full-ha.xml.

Since Jakarta Enterprise Beans are passivation-capable by default, generally, users already using an HA profile will not need to make any configuration changes
for their beans to be distributable and, consequently, to support failover. More fine-grained control over whether a bean is distrubutable can be
achieved using the _passivationCapable_ attribute of the @Stateful annotation (or the equivalent deployment descriptor override). A bean which is marked as
@Stateful(passivationCapable=false) will not exhibit distributable behavior (i.e. failover), even when the application containing it is deployed in a cluster.

NOTE: More information on passivation-capable beans can be found in Section 4.6.5 of the Jakarta Enterprise Beans specification.

In the sections that follow, we discuss some aspects of configuring distributable Jakarta Enterprise Beans applications in Wildfly.

[[distributable-ejb-subsystem]]
== Distributable EJB Subsystem

The purpose of the distributable-ejb subsystem is to permit configuration of clustering abstractions
required to support those resources of the ejb3 subsystem which support clustered operation. The key resources
of the ejb3 subsystem which require clustering abstractions are:

* `cache factories`
Passivating cache factories depend on a bean management provider to provide passivation and persistence of SFSB
session states in a local or distributed environment.

* `client mappings registries`
Supporting remote invocation on SFSB deployed in a cluster require storing client mappings information in a
 client mappings registry. The registry may be tailored for a local or a distributed environment.

These clustering abstractions are made available to the ejb3 subsystem via the specification and configuration of
clustering abstraction 'providers'. We describe the available providers below.

[[bean-management-providers]]
=== Bean management providers

A bean management provider provides access to a given implementation of a bean manager,
used by passivation-capable cache factories defined in the ejb3 subsystem to manage passivation and persistence.

Bean management provider elements are named, and represent different implementation and configuration choices for bean management.
At least one named bean management provider must be defined in the distributable-ejb subsystem and of those, one
instance must be identified as the default bean management provider, using the `default-bean-management` attribute of
the distributable-ejb subsystem.

The available bean management provider is:

[[infinispan-bean-management]]
==== infinispan-bean-management

The infinispan-bean-management provider element represents a bean manager implementation based on an Infinispan cache. The
attributes for the infinispan-bean-manager element are:

* `cache-container` specifies a cache container defined in the Infinispan subsystem used to support the session state cache
* `cache` specifies the session state cache and its configured properties
* `max-active-beans` specifies the maximum number of non-passivated session state entries allowed in the cache


[[client-mappings-registries]]
=== Client mappings registries

A client mappings registry provider provides access to a given implementation of a client mappings registry, used by
the EJB client invocation mechanism to store information about client mappings for each node in the cluster. Client mappings
are defined in the socket bindings configuration of a server and required to allow an EJB client application to connect
to servers which are multi-homed (i.e. clients may access the same server from different networks using a different IP address
ad port for each interface on the multi-homed server).

The available client mappings registry providers are:

[[infinispan-client-mappings-registry]]
==== infinispan-client-mappings-registry

The infinispan-client-mappings-registry provider is a provider based on an Infinispan cache and suitable for a clustered server.

* `cache-container` specifies a cache container defined in the Infinispan subsystem used to support the client mappings registry
* `cache` specifies the cache and its configured properties used to support the client mappings registry

[[local-client-mappings-registry]]
==== local-client-mappings-registry

The client mappings registry provider suitable for a local, non-clustered server.


[[timer-management]]
=== Timer management

The distributable-ejb subsystem defines a set of timer management resources that define behavior for persistent or non-persistent EJB timers.

To use distributable timer management for EJB timers, one must first disable the existing in-memory mechanisms in the ejb3 subsystem.
See link:Developer_Guide{outfilesuffix}#Jakarta_Enterprise_Beans_Distributed_Persistent_Timers[Jakarta Enterprise Beans Distributed Timer documentation] for details.

==== infinispan-timer-management

This provider stores timer metadata within an embedded Infinispan cache, and utilizes consistent hashing to distribute timer execution between cluster members.

* `cache-container` specifies a cache container defined in the Infinispan subsystem
* `cache` specifies the a cache configuration within the specified cache-container
* `max-active-timers` specifies the maximum number active timers to retain in memory at a time, after which the least recently used will passivate

To ensure proper functioning, the associated cache configuration, regardless of type, should use:

* BATCH transaction mode
* REPEATABLE_READ lock isolation

Generally, persistent timers will leverage a distributed or replicated cache configuration if in a cluster, or a local, persistent cache configuration if on a single server;
while transient timers will leverage a local, passivating cache configuration.

By default, all cluster members will be eligible for timer execution.
A given cluster member exclude itself from timer execution by using a cache `capacity-factor` of 0.

[[deploying-clustered-ejbs]]
== Deploying clustered EJBs

Clustering support is available in the HA profiles of WildFly. In this
chapter we'll be using the standalone server for explaining the details.
However, the same applies to servers in a domain mode. Starting the
standalone server with HA capabilities enabled, involves starting it
with the standalone-ha.xml (or even standalone-full-ha.xml):

[source, sh]
----
./standalone.sh -server-config=standalone-ha.xml
----

This will start a single instance of the server with HA capabilities.
Deploying the EJBs to this instance _doesn't_ involve anything special
and is the same as explained in the link:Admin_Guide{outfilesuffix}#Application_deployment[application
deployment chapter].

Obviously, to be able to see the benefits of clustering, you'll need
more than one instance of the server. So let's start another server with
HA capabilities. That another instance of the server can either be on
the same machine or on some other machine. If it's on the same machine,
the two things you have to make sure is that you pass the port offset
for the second instance and also make sure that each of the server
instances have a unique `jboss.node.name` system property. You can do
that by passing the following two system properties to the startup
command:

[source, sh]
----
./standalone.sh -server-config=standalone-ha.xml -Djboss.socket.binding.port-offset=<offset of your choice> -Djboss.node.name=<unique node name>
----

Follow whichever approach you feel comfortable with for deploying the
EJB deployment to this instance too.

[IMPORTANT]

Deploying the application on just one node of a standalone instance of a
clustered server does *not* mean that it will be automatically deployed
to the other clustered instance. You will have to do deploy it
explicitly on the other standalone clustered instance too. Or you can
start the servers in domain mode so that the deployment can be deployed
to all the server within a server group. See the
link:Admin_Guide{outfilesuffix}[admin guide] for
more details on domain setup.

Now that you have deployed an application with clustered EJBs on both
the instances, the EJBs are now capable of making use of the clustering
features.

[[failover-for-clustered-ejbs]]
=== Failover for clustered EJBs

Clustered EJBs have failover capability. The state of the @Stateful
@Clustered EJBs is replicated across the cluster nodes so that if one of
the nodes in the cluster goes down, some other node will be able to take
over the invocations. Let's see how it's implemented in WildFly. In
the next few sections we'll see how it works for remote (standalone)
clients and for clients in another remote WildFly server instance.
Although, there isn't a difference in how it works in both these cases,
we'll still explain it separately so as to make sure there aren't any
unanswered questions.

[[remote-standalone-clients]]
=== Remote standalone clients

In this section we'll consider a remote standalone client (i.e. a client
which runs in a separate JVM and _isn't_ running within another WildFly
8 instance). Let's consider that we have 2 servers, server X and server
Y which we started earlier. Each of these servers has the clustered EJB
deployment. A standalone remote client can use either the
link:Developer_Guide{outfilesuffix}#EJB_invocations_from_a_remote_client_using_JNDI[JNDI approach] or native JBoss EJB client APIs to
communicate with the servers. The important thing to note is that when
you are invoking clustered EJB deployments, you do *not* have to list
all the servers within the cluster (which obviously wouldn't have been
feasible due the dynamic nature of cluster node additions within a
cluster).

The remote client just has to list only one of the servers with the
clustering capability. In this case, we can either list server X (in
`jboss-ejb-client.properties`) _or_ server Y. This server will act as the
starting point for cluster topology communication between the client and
the clustered nodes.

Note that you have to configure the _ejb_ cluster in the
jboss-ejb-client.properties configuration file, like so:

[source,options="nowrap"]
----
remote.clusters=ejb
remote.cluster.ejb.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS=false
remote.cluster.ejb.connect.options.org.xnio.Options.SSL_ENABLED=false
----

[[cluster-topology-communication]]
=== Cluster topology communication

When a client connects to a server, the JBoss EJB client implementation
(internally) communicates with the server for cluster topology
information, if the server had clustering capability. In our example
above, let's assume we listed server X as the initial server to connect
to. When the client connects to server X, the server will send back an
(asynchronous) cluster topology message to the client. This topology
message consists of the cluster name(s) and the information of the nodes
that belong to the cluster. The node information includes the node
address and port number to connect to (whenever necessary). So in this
example, the server X will send back the cluster topology consisting of
the other server Y which belongs to the cluster.

In case of stateful (clustered) EJBs, a typical invocation flow involves
creating of a session for the stateful bean, which happens when you do a
JNDI lookup for that bean, and then invoking on the returned proxy. The
lookup for stateful bean, internally, triggers a (synchronous) session
creation request from the client to the server. In this case, the
session creation request goes to server X since that's the initial
connection that we have configured in our jboss-ejb-client.properties.
Since server X is clustered, it will return back a session id and along
with send back an _"affinity"_ of that session. In case of clustered
servers, the affinity equals to the name of the cluster to which the
stateful bean belongs on the server side. For non-clustered beans, the
affinity is just the node name on which the session was created. This
_affinity_ will later help the EJB client to route the invocations on
the proxy, appropriately to either a node within a cluster (for
clustered beans) or to a specific node (for non-clustered beans). While
this session creation request is going on, the server X will also send
back an asynchronous message which contains the cluster topology. The
JBoss EJB client implementation will take note of this topology
information and will later use it for connection creation to nodes
within the cluster and routing invocations to those nodes, whenever
necessary.

Now that we know how the cluster topology information is communicated
from the server to the client, let see how failover works. Let's
continue with the example of server X being our starting point and a
client application looking up a stateful bean and invoking on it. During
these invocations, the client side will have collected the cluster
topology information from the server. Now let's assume for some reason,
server X goes down and the client application subsequent invokes on the
proxy. The JBoss EJB client implementation, at this stage will be aware
of the affinity and in this case it's a cluster affinity. Because of the
cluster topology information it has, it knows that the cluster has two
nodes server X and server Y. When the invocation now arrives, it sees
that the server X is down. So it uses a selector to fetch a suitable
node from among the cluster nodes. The selector itself is configurable,
but we'll leave it from discussion for now. When the selector returns a
node from among the cluster, the JBoss EJB client implementation creates
a connection to that node (if not already created earlier) and creates a
EJB receiver out of it. Since in our example, the only other node in the
cluster is server Y, the selector will return that node and the JBoss
EJB client implementation will use it to create a EJB receiver out of it
and use that receiver to pass on the invocation on the proxy.
Effectively, the invocation has now failed over to a different node
within the cluster.

[[remote-clients-on-another-instance]]
=== Remote clients on another instance of WildFly

So far we discussed remote standalone clients which typically use either
the EJB client API or the jboss-ejb-client.properties based approach to
configure and communicate with the servers where the clustered beans are
deployed. Now let's consider the case where the client is an application
deployed another AS7 instance and it wants to invoke on a clustered
stateful bean which is deployed on another instance of WildFly. In
this example let's consider a case where we have 3 servers involved.
Server X and Server Y both belong to a cluster and have clustered EJB
deployed on them. Let's consider another server instance Server C (which
may or may _not_ have clustering capability) which acts as a client on
which there's a deployment which wants to invoke on the clustered beans
deployed on server X and Y and achieve failover.

The configurations required to achieve this are explained in
link:Developer_Guide{outfilesuffix}#EJB_invocations_from_a_remote_server_instance[this chapter]. As you can see the configurations are
done in a jboss-ejb-client.xml which points to a remote outbound
connection to the other server. This jboss-ejb-client.xml goes in the
deployment of server C (since that's our client). As explained earlier,
the client configuration need *not* point to all clustered nodes.
Instead it just has to point to one of them which will act as a start
point for communication. So in this case, we can create a remote
outbound connection on server C to server X and use server X as our
starting point for communication. Just like in the case of remote
standalone clients, when the application on server C (client) looks up a
stateful bean, a session creation request will be sent to server X which
will send back a session id and the cluster affinity for it.
Furthermore, server X asynchronously send back a message to server C
(client) containing the cluster topology. This topology information will
include the node information of server Y (since that belongs to the
cluster along with server X). Subsequent invocations on the proxy will
be routed appropriately to the nodes in the cluster. If server X goes
down, as explained earlier, a different node from the cluster will be
selected and the invocation will be forwarded to that node.

As can be seen both remote standalone client and remote clients on
another WildFly instance act similar in terms of failover.

NOTE: References in this document to Enterprise JavaBeans(EJB) refer to the Jakarta Enterprise Beans unless otherwise noted.
