Knowledgebase Partitioning

Warning

This is an experimental feature, subject to changes in the future.

The classic Rete algorithm is usually executed using a single thread. Although, as confirmed in several opportunities by Dr. Forgy, the algorithm itself is parallelizable. Drools implementation of the ReteOO algorithm supports coarse grained parallelization through rulebase partitioning.

When this option is enabled, the rulebase will be partitioned in several independent partitions and a pool of worker threads will be used to propagate facts through the partitions. The implementation guarantees that at most one worker thread will be executing tasks for a given partition, but multiple partitions may be "active" at a single point in time.

Everything should be transparent to the user, except that all working memory actions (insert/retract/modify) are executed assynchronously.

Important

This feature enables parallel LHS evaluation, but does not change the behavior of rule firing. I.e., rules will continue to fire sequentially, according to the conflict resolution strategy.

When partitioning is useful

Knowledge base partitioning is a very powerful feature for specific scenarios, but it is not a general case solution. To understand if this feature would be useful for a given scenario, the user may follow the checklist bellow:

  1. Does your hardware contains multiple processors?

  2. Does your knowledge session process a high volume of facts?

  3. Are the LHS of your rules expensive to evaluate? (ex: use expensive "from" expressions)

  4. Does your knowledge base contains hundreds or more rules?

If the answer to all the questions above is "yes", then this feature will probably increase the overall performance of your rulebase evaluation.

How to configure partitioning

To enable knowledge base partitioning, set the following option:

Example 2.16. enabling multithread evaluation (partitioning)

KnowledgeBaseConfiguration config = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
config.setOption( MultithreadEvaluationOption.YES );

The equivalent property is:

drools.multithreadEvaluation = <true|false>

The default value for this option is "false" (disabled).

Multithreading management

Drools offers a simple configuration option for users to control the size of the worker thread's pool.

To define the maximum size for the thread pool, the user may use the following configuration option:

Example 2.17. setting the maximum number of threads for rule evaluation to 5

KnowledgeBaseConfiguration config = KnowledgeBaseFactory.newKnowledgeBaseConfiguration();
config.setOption( MaxThreadsOption.get(5) );


The equivalent property is:

drools.maxThreads = <-1|1..n>

The default value for this configuration is 3 and a negative number means the engine will try to spawn as many threads as there are partitions in the rulebase.

Warning

It is usually dangerous to set this option with a negative number. Always set it with a sensible positive number of threads.