Skip to main content

Drools Performance Limits

Posted by schaefa on October 13, 2005 at 3:40 PM PDT

I had the chance to test the Rules Engine Drools and even
tough I like it I concluded that Drools did not perform as we had
expected. The main reason to drop Drools where that the number of
elements participating in the rules where not confined to a small
number. My tests showed that Drools is fine when either the number of
elements in the test is small or when the many of the possible
combinations can be removed because of failing conditions. Again I did
not conclude that Drools is not a good tool but rather that I tried to
find the limitations of the tool.

Here I want to show a very, very simplistic test of drools
performance just to show my point. I have three classes which does
return the same object: a string called "Test".:

package com.madplanet.drools.test;

public class A {

    public Object getObject() {

       
return "TEST";

    }

}

Then I created a Drools Rule Script which just tests each
Class' getObject() value against one of the other type:

<?xml version="1.0"?>



<rule-set name="zip.code.rule"

    xmlns="http://drools.org/rules"

   
xmlns:java="http://drools.org/semantics/java"

   
xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"

   
xs:schemaLocation="http://drools.org/rules rules.xsd

                      
http://drools.org/semantics/java java.xsd">



   
<import>com.madplanet.drools.test.A</import>

   
<import>com.madplanet.drools.test.B</import>

   
<import>com.madplanet.drools.test.C</import>



    <rule name="Test Transitiveness"
salience="10">

       
<parameter identifier="a">

           
<class>com.madplanet.drools.test.A</class>

       
</parameter>

       
<parameter identifier="b">

           
<class>com.madplanet.drools.test.B</class>

       
</parameter>

       
<parameter identifier="c">

           
<class>com.madplanet.drools.test.C</class>

       
</parameter>

       
<java:condition>

           
a.getObject().equals( b.getObject() )

       
</java:condition>

       
<java:condition>

           
b.getObject().equals( c.getObject() )

       
</java:condition>

       
<java:condition>

           
c.getObject().equals( a.getObject() )

       
</java:condition>

       
<java:consequence>

       
</java:consequence>

    </rule>

</rule-set>

Finally I created a jUnit Test Case that creates the same
number of A, B and C instances, assign it to the working memory of
Drools and finally fires the rule:

    public void
testZipCodeExcludeWithinInclude()

       
    throws Exception {

       
for( int i = 50; i < 150; i += 5 ) {

           
WorkingMemory lWorkingMemory = mRuleBase.newWorkingMemory();

           
long lStartTime = System.currentTimeMillis();

           
System.out.println( "--- Start: " + i + ". Loop ---" );

           
for( int j = 0; j < i; j++ ) {

               
lWorkingMemory.assertObject( new A() );

           
}

           
printLog( "   ", "Added all As", i, lStartTime );

           
for( int j = 0; j < i; j++ ) {

               
lWorkingMemory.assertObject( new B() );

           
}

           
printLog( "   ", "Added all Bs", i, lStartTime );

           
for( int j = 0; j < i; j++ ) {

               
lWorkingMemory.assertObject( new C() );

           
}

           
printLog( "   ", "Added all Cs", i, lStartTime );

           
lWorkingMemory.fireAllRules();

           
printLog( "", "Finished (fired rules)", i, lStartTime );

        }

    }

Below is the output of the test above and as you can see the
cross product of all the possible combinations defines the duration of
the execution. Please keep in mind that in that test all conditions
succeed and so it is the worst case scenario. If all condition would
fail the test would go through in a blink of the eye but this is the
other extreme. Also this test is so simple that it does not represent
any actual business case.

--- Start: 50. Loop ---

   Added all As, loop: 50, duration: 0s

   Added all Bs, loop: 50, duration: 0s

   Added all Cs, loop: 50, duration: 0s

Finished (fired rules), loop: 50, duration: 1s

--- Start: 55. Loop ---

   Added all As, loop: 55, duration: 0s

   Added all Bs, loop: 55, duration: 0s

   Added all Cs, loop: 55, duration: 0s

Finished (fired rules), loop: 55, duration: 1s

--- Start: 60. Loop ---

   Added all As, loop: 60, duration: 0s

   Added all Bs, loop: 60, duration: 0s

   Added all Cs, loop: 60, duration: 1s

Finished (fired rules), loop: 60, duration: 2s

--- Start: 65. Loop ---

   Added all As, loop: 65, duration: 0s

   Added all Bs, loop: 65, duration: 0s

   Added all Cs, loop: 65, duration: 1s

Finished (fired rules), loop: 65, duration: 2s

--- Start: 70. Loop ---

   Added all As, loop: 70, duration: 0s

   Added all Bs, loop: 70, duration: 0s

   Added all Cs, loop: 70, duration: 1s

Finished (fired rules), loop: 70, duration: 3s

--- Start: 75. Loop ---

   Added all As, loop: 75, duration: 0s

   Added all Bs, loop: 75, duration: 0s

   Added all Cs, loop: 75, duration: 2s

Finished (fired rules), loop: 75, duration: 4s

--- Start: 80. Loop ---

   Added all As, loop: 80, duration: 0s

   Added all Bs, loop: 80, duration: 0s

   Added all Cs, loop: 80, duration: 2s

Finished (fired rules), loop: 80, duration: 5s

--- Start: 85. Loop ---

   Added all As, loop: 85, duration: 0s

   Added all Bs, loop: 85, duration: 0s

   Added all Cs, loop: 85, duration: 3s

Finished (fired rules), loop: 85, duration: 7s

--- Start: 90. Loop ---

   Added all As, loop: 90, duration: 0s

   Added all Bs, loop: 90, duration: 0s

   Added all Cs, loop: 90, duration: 4s

Finished (fired rules), loop: 90, duration: 8s

--- Start: 95. Loop ---

   Added all As, loop: 95, duration: 0s

   Added all Bs, loop: 95, duration: 0s

   Added all Cs, loop: 95, duration: 4s

Finished (fired rules), loop: 95, duration: 9s

--- Start: 100. Loop ---

   Added all As, loop: 100, duration: 0s

   Added all Bs, loop: 100, duration: 0s

   Added all Cs, loop: 100, duration: 6s

Finished (fired rules), loop: 100, duration: 11s

--- Start: 105. Loop ---

   Added all As, loop: 105, duration: 0s

   Added all Bs, loop: 105, duration: 0s

   Added all Cs, loop: 105, duration: 7s

Finished (fired rules), loop: 105, duration: 14s

--- Start: 110. Loop ---

   Added all As, loop: 110, duration: 0s

   Added all Bs, loop: 110, duration: 0s

   Added all Cs, loop: 110, duration: 7s

Finished (fired rules), loop: 110, duration: 15s

--- Start: 115. Loop ---

   Added all As, loop: 115, duration: 0s

   Added all Bs, loop: 115, duration: 0s

   Added all Cs, loop: 115, duration: 8s

Finished (fired rules), loop: 115, duration: 17s

--- Start: 120. Loop ---

   Added all As, loop: 120, duration: 0s

   Added all Bs, loop: 120, duration: 0s

   Added all Cs, loop: 120, duration: 11s

Finished (fired rules), loop: 120, duration: 21s

--- Start: 125. Loop ---

   Added all As, loop: 125, duration: 0s

   Added all Bs, loop: 125, duration: 0s

   Added all Cs, loop: 125, duration: 13s

Finished (fired rules), loop: 125, duration: 26s

--- Start: 130. Loop ---

   Added all As, loop: 130, duration: 0s

   Added all Bs, loop: 130, duration: 0s

    [junit] Tests run: 1, Failures: 0,
Errors: 1, Time elapsed: 180.885 sec

    [junit] [ERROR] TEST
com.madplanet.drools.test.ObjectEqualsTest FAILED

The loop in the output means how many instances of A, B and C
are created. The 'Added' line is printed out when all of the respective
object were added to the Drools working memory. Finally he
test case fails because the test runs out of memory (I used the Ant's
default settings) and can be easily fixed by increasing the heap size.

My conclusion
is that Drools works fine when the number of participating elements is
small but Drools may run into a performance problem when the cross
product of the participating objects is huge. In my test it was getting
slow when the cross product run over style="font-weight: bold;">half a million.

Comments

Nice article. Can you

Nice article. Can you recommend another business rule engine that doesn't have this problem? We are using it and we are experiencing some OutOfMemoryErrors :( As you said, we can increase the heap size but this will be too heavy sooner or later.