JBoss.org Community Documentation

- Prev
- Next

## Chapter 6. Benchmarking and tweaking

## 6.1. Finding the best configuration

Drools Planner supports several solver types, but you're probably wondering which is the best one? Although some solver types generally perform better then others, it really depends on your problem domain. Most solver types also have settings which can be tweaked. Those settings can influence the results of a solver a lot, although most settings perform pretty good out-of-the-box.

Luckily, Drools Planner includes a benchmarker, which allows you to play out different solver types and different settings against each other, so you can pick the best configuration for your problem domain.

## 6.2. Building a Benchmarker

You can build a `Benchmarker` instance with the`XmlSolverBenchmarker`. Configure it with a benchmarker configuration xml file:

        XmlSolverBenchmarker benchmarker = new XmlSolverBenchmarker();

        benchmarker.configure("/org/drools/planner/examples/nqueens/benchmark/nqueensSolverBenchmarkConfig.xml");

        benchmarker.benchmark();

        benchmarker.writeResults(resultFile);

A basic benchmarker configuration file looks something like this:

    <?xml version="1.0" encoding="UTF-8"?>

    <solverBenchmarkSuite>

        <benchmarkDirectory>local/data/nqueens</benchmarkDirectory>

        <inheritedSolverBenchmark>

            <unsolvedSolutionFile>data/nqueens/unsolved/unsolvedNQueens32.xml</unsolvedSolutionFile>

            <unsolvedSolutionFile>data/nqueens/unsolved/unsolvedNQueens64.xml</unsolvedSolutionFile>

            <localSearchSolver>

                <scoreDrl>/org/drools/planner/examples/nqueens/solver/nQueensScoreRules.drl</scoreDrl>

                <scoreDefinition>

                    <scoreDefinitionType>SIMPLE</scoreDefinitionType>

                </scoreDefinition>

                <termination>

                    <maximumSecondsSpend>20</maximumSecondsSpend>

                </termination>

                <selector>

                    <moveFactoryClass>org.drools.planner.examples.nqueens.solver.move.factory.NQueensMoveFactory</moveFactoryClass>

                </selector>

                <forager>

                    <pickEarlyType>NEVER</pickEarlyType>

                </forager>

            </localSearchSolver>

        </inheritedSolverBenchmark>

        <solverBenchmark>

            <name>Solution tabu</name>

            <localSearchSolver>

                <acceptor>

                <completeSolutionTabuSize>1000</completeSolutionTabuSize>

                </acceptor>

            </localSearchSolver>

        </solverBenchmark>

        <solverBenchmark>

            <name>Move tabu 5</name>

            <localSearchSolver>

                <acceptor>

                <completeMoveTabuSize>5</completeMoveTabuSize>

                </acceptor>

            </localSearchSolver>

        </solverBenchmark>

        <solverBenchmark>

            <name>Move tabu 7</name>

            <localSearchSolver>

                <acceptor>

                <completeMoveTabuSize>7</completeMoveTabuSize>

                </acceptor>

            </localSearchSolver>

        </solverBenchmark>

        <solverBenchmark>

            <name>Solution tabu and move tabu 7</name>

            <localSearchSolver>

                <acceptor>

                <completeSolutionTabuSize>1000</completeSolutionTabuSize>

                <completeMoveTabuSize>7</completeMoveTabuSize>

                </acceptor>

            </localSearchSolver>

        </solverBenchmark>

    </solverBenchmarkSuite>

This benchmarker will try 4 configurations (1 solution tabu, 2 move tabu's and 1 solution-move tabu) on 2 data sets (32 and 64 queens), so it will run 8 solvers.

Every `solverBenchmark` entity contains a solver configuration (for example a local search solver) and one or more `unsolvedSolutionFile` entities. It will run the solver configuration on each of those unsolved solution files. A `name` is optional and generated if absent. The common part of multiple `solverBenchmark` entities can be extracted to the `inheritedSolverBenchmark` entity, but they can still be overwritten per `solverBenchmark` entity.

You need to specify a `benchmarkDirectory` (relative to the working directory). A summary statistic and the best solution of each solver run will be written in that directory.

## 6.3. Best score over time statistic (graph and CSV)

The benchmarker supports outputting the best score over time statistic as a graph and a CSV (comma separated values) file. Here's an example of a graph:

Figure 6.1. Best score over time statistic

To configure graph and CSV output of the best score over time, just add a `solverStatisticType` line:

    <solverBenchmarkSuite>

        <benchmarkDirectory>local/data/nqueens/solved</benchmarkDirectory>

        <solverStatisticType>BEST_SOLUTION_CHANGED</solverStatisticType>

        ...

    </solverBenchmarkSuite>

It will output all graphs and CSV files in the `benchmarkDirectory`.

- PrevChapter 5. Local search solver
- Top of page
- Front page
- NextIndex