Clustering tests

Submitted by kobit (not verified) on Tue, 2008-10-21 10:02
::

The whole Tigase server development is oriented around automatic tests we run using Tigase Test Suite. Our aim is to not release new stuff before it is tested using automated tests. Only this way we can be sure all the server features work the same way for every release. The drawback is that we first need to create tests, then we can run them to test the server. It is a bit like double development effort.

The suite evolves together with the server. At the beginning I was running only functional tests executing test cases sequentially. Then I added performance tests which required running many tests concurrently on a separate connections and stability tests which in turn required lower resources consumption and..... stability. Every new addition required slightly new features set.

A few months back when I was running load tests that was another challenge to run the test suite from multiple machines and again I had to lower resources consumption to be able to open more than 60k user connections from each test suite instance.

We extensively use Bosh in a few Tigase installations so to provide decent support for web clients I have added Bosh tests which required a different way to maintain connections to the server because the Bosh client can send each request on a separate TCP/IP connection.

Now the clustering in Tigase. We have working clustering mode in the server for a few months already but we decided not to release a new version before it is well tested using automatic tests. New challenge - open multiple connections from the test suite and each connection must end up on a different cluster node.

Today I am happy to announce all the cluster tests have passed. This is a big step for us in our preparation for publishing Tigase 4.0. Now we can focus on those countless tiny bits required to prepare binary packages for the new version.

Below is a copy of the clustering tests report, for those who are interested in more details.

The tests have been run on the cluster installation with 3 nodes in domain: xmpp-test.tigase.org. If you look carefully you may notice uneven times for some tests depending on the node used. This is because we don't have 3 identical machines to do our tests. One of the node is QuadCore quite recent box while others are an old IBM laptop and 7 years old P4. Therefore these tests results can not be used as the performance indication. They are functional tests for the Tigase server working in the cluster mode.

 

Server version info:

 

Name:     Tigase
Version:     4.0.0-b1182
OS:     Linux-amd64-2.6.23-gentoo-r8, Java HotSpot(TM) 64-Bit Server VM-10.0-b22-Sun Microsystems Inc.
       
Local IP:     81.143.149.100
Remote IP:     81.143.149.98

 

Server basic configuration parameters:

 

sess-man: admins   kobit@xmpp-test.tigase.org
sess-man: anonymous-domains   xmpp-test.tigase.org, test-domain
sess-man: auth-repo-class   tigase.db.jdbc.DrupalAuth
sess-man: auth-repo-url   jdbc:mysql://localhost/xmpp_test?user=xxxx&password=xxxxxx&online_status=true
sess-man: component-id   sess-man@veloci.tigase.org
sess-man: def-hostname   veloci.tigase.org
sess-man: hostnames   xmpp-test.tigase.org, test-domain
sess-man: max-queue-size   10000
sess-man: plugins   jabber:iq:auth, urn:ietf:params:xml:ns:xmpp-sasl, urn:ietf:params:xml:ns:xmpp-bind, urn:ietf:params:xml:ns:xmpp-session, roster-presence, jabber:iq:privacy, msgoffline, jabber:iq:version, http://jabber.org/protocol/stats, starttls, vcard-temp, http://jabber.org/protocol/commands, jabber:iq:private, urn:xmpp:ping, http://jabber.org/protocol/pubsub
sess-man: trusted   admin@xmpp-test.tigase.org, admin@test-domain
sess-man: user-repo-class   tigase.db.jdbc.JDBCRepository
sess-man: user-repo-url   jdbc:mysql://localhost/xmpp_test?user=drupal&password=drupal12&autoCreateUser=true

 

Tests results:

 

Test name Result Total time OK Average Description History
Version success 0 sec 1 22 ms Get server version Version
Configuration success 0 sec 1 56 ms Server configuration Configuration
Statistics success 0 sec 1 48 ms Server statistics Statistics
Test 1 success 0 sec 3 7 ms Socket connect to all cluster nodes. Test 1
Test 2 success 0 sec 3 17 ms Stream open to all cluster nodes. Test 2
Test 3 success 1 sec 3 392 ms TLS connection open to all cluster nodes. Test 3
Test 4 success 0 sec 3 146 ms SASL authorization to all cluster nodes to different accounts. Test 4
Test 5 success 0 sec 3 121 ms SASL authorization to all cluster nodes to the same account 'tester'. Test 5
Test 6 success 0 sec 3 66 ms Message send within one node to self user. Test 6
Test 7 success 1 sec 3 215 ms Message send within 'veloci.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response. Test 7
Test 8 success 1 sec 3 259 ms Message send within 'shell2.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response. Test 8
Test 9 success 1 sec 3 226 ms Message send within 'test-i.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response. Test 9
Test 10 success 1 sec 3 262 ms Message send beetwen 3 nodes: from 'shell2.tigase.org', user 'admin' and from 'test-i.tigase.org', user 'all-xmpp-test_1' to user 'tester-1' on 'veloci.tigase.org' and wait for a response. Test 10
Test 11 success 1 sec 3 232 ms Message send beetwen 3 nodes: from 'test-i.tigase.org', user 'admin' and from 'veloci.tigase.org', user 'all-xmpp-test_1' to user 'tester-2' on 'shell2.tigase.org' and wait for a response. Test 11
Test 12 success 1 sec 3 264 ms Message send beetwen 3 nodes: from 'veloci.tigase.org', user 'admin' and from 'shell2.tigase.org', user 'all-xmpp-test_1' to user 'tester-3' on 'test-i.tigase.org' and wait for a response. Test 12
Test 13 success 0 sec 2 169 ms 2 user connections, each using a different resource on 'veloci.tigase.org' node. Test 13
Test 14 success 0 sec 2 213 ms 2 user connections, each using a different resource on 'shell2.tigase.org' node. Test 14
Test 15 success 0 sec 2 180 ms 2 user connections, each using a different resource on 'test-i.tigase.org' node. Test 15
Test 16 success 1 sec 2 274 ms 2 user connections, on node 'veloci.tigase.org' and 'shell2.tigase.org'. Test 16
Test 17 success 0 sec 2 241 ms 2 user connections, on node 'shell2.tigase.org' and 'test-i.tigase.org'. Test 17
Test 18 success 0 sec 2 192 ms 2 user connections, on node 'veloci.tigase.org' and 'test-i.tigase.org'. Test 18
Test 19 success 0 sec 3 19 ms XEP-0092 get-version - 3 user connections, each using a different node. Test 19
Test 20 success 1 sec 3 92 ms XEP-0039 Statistics Gathering - 3 user connections, each using a different node. Test 20
Test 21 success 1 sec 3 79 ms Test for feature-not-implemented error - 3 user connections, each using a different node. Test 21
Test 22 success 0 sec 3 113 ms XEP-0199: XMPP Ping - 3 user connections, each using a different node. Test 22
Test 23 success 0 sec 3 81 ms XEP-0030: XMPP Service Discovery - 3 user connections, each using a different node. Test 23
Bosh 1 success 0 sec 3 4 ms Socket connect to all cluster nodes. Bosh 1
Bosh 2 success 0 sec 3 31 ms Stream open to all cluster nodes. Bosh 2
Bosh 3 success 0 sec 3 73 ms SASL authorization to all cluster nodes to different accounts. Bosh 3
Bosh 4 success 0 sec 3 74 ms SASL authorization to all cluster nodes to the same account 'tester'. Bosh 4
Bosh 5 success 0 sec 3 117 ms Message send within one node to self user. Bosh 5
Bosh 6 success 1 sec 3 236 ms Message send within one node from 'admin' to 'tester' and wait for a response. Bosh 6
Bosh 7 success 2 sec 3 468 ms Message send within 'shell2.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response. Bosh 7
Bosh 8 success 2 sec 3 417 ms Message send within 'test-i.tigase.org' node from 'admin' and from 'all-xmpp-test_1'to 'tester' and wait for a response. Bosh 8
Bosh 9 success 2 sec 3 443 ms Message send beetwen 3 nodes: from 'shell2.tigase.org', user 'admin' and from 'test-i.tigase.org', user 'all-xmpp-test_1' to user 'tester' on 'veloci.tigase.org' and wait for a response. Bosh 9
Bosh 10 success 1 sec 3 264 ms Message send beetwen 3 nodes: from 'test-i.tigase.org', user 'admin' and from 'veloci.tigase.org', user 'all-xmpp-test_1' to user 'tester' on 'shell2.tigase.org' and wait for a response. Bosh 10
Bosh 11 success 1 sec 3 289 ms Message send beetwen 3 nodes: from 'veloci.tigase.org', user 'admin' and from 'shell2.tigase.org', user 'all-xmpp-test_1' to user 'tester' on 'test-i.tigase.org' and wait for a response. Bosh 11
Bosh 12 success 0 sec 2 186 ms 2 user connections, each using a different resource on 'veloci.tigase.org' node. Bosh 12
Bosh 13 success 0 sec 2 240 ms 2 user connections, each using a different resource on 'shell2.tigase.org' node. Bosh 13
Bosh 14 success 0 sec 2 209 ms 2 user connections, each using a different resource on 'test-i.tigase.org' node. Bosh 14
Bosh 15 success 1 sec 2 286 ms 2 user connections, on node 'veloci.tigase.org' and 'shell2.tigase.org'. Bosh 15
Bosh 16 success 1 sec 2 294 ms 2 user connections, on node 'shell2.tigase.org' and 'test-i.tigase.org'. Bosh 16
Bosh 17 success 1 sec 2 282 ms 2 user connections, on node 'veloci.tigase.org' and 'test-i.tigase.org'. Bosh 17
Bosh 18 success 0 sec 3 43 ms XEP-0092 get-version - 3 user connections, each using a different node. Bosh 18
Bosh 19 success 0 sec 3 163 ms XEP-0039 Statistics Gathering - 3 user connections, each using a different node. Bosh 19
Bosh 20 success 0 sec 3 124 ms Test for feature-not-implemented error - 3 user connections, each using a different node. Bosh 20
Bosh 21 success 0 sec 3 136 ms XEP-0199: XMPP Ping - 3 user connections, each using a different node. Bosh 21
Bosh 22 success 0 sec 3 137 ms XEP-0030: XMPP Service Discovery - 3 user connections, each using a different node. Bosh 22
Prep 1 success 0 sec 1 0 ms Setup a user daemon wating for messages Prep 1
Multi 1 success 0 sec 30 23 ms Multi-thread test of server Version info on node 'veloci.tigase.org' Multi 1
Multi 2 success 3 sec 30 131 ms Multi-thread test of server Version info on node 'shell2.tigase.org' Multi 2
Multi 3 success 1 sec 30 42 ms Multi-thread test of server Version info on node 'test-i.tigase.org' Multi 3
Multi 4 success 0 sec 30 30 ms Multi-thread test of server Version info on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 4
Multi 5 success 2 sec 30 84 ms Multi-thread test of server Statistics info on node 'veloci.tigase.org' Multi 5
Multi 6 success 17 sec 30 576 ms Multi-thread test of server Statistics info on node 'shell2.tigase.org' Multi 6
Multi 7 success 7 sec 30 240 ms Multi-thread test of server Statistics info on node 'test-i.tigase.org' Multi 7
Multi 8 success 3 sec 30 128 ms Multi-thread test of server Statistics info on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 8
Multi 9 success 37 sec 90 415 ms Multi-thread test of server Statistics info. Each user makes 3 connection, each on a different node and different resource: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 9
Multi 10 success 4 sec 30 163 ms Multi-thread test Roster management functions on node 'veloci.tigase.org' Multi 10
Multi 11 success 26 sec 30 886 ms Multi-thread test Roster management functions on node 'shell2.tigase.org' Multi 11
Multi 12 success 12 sec 30 412 ms Multi-thread test Roster management functions on node 'test-i.tigase.org' Multi 12
Multi 13 success 6 sec 30 228 ms Multi-thread test Roster management functions on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 13
Multi 14 success 9 sec 30 331 ms Multi-thread test privacy lists management on node 'veloci.tigase.org' Multi 14
Multi 15 success 41 sec 30 1391 ms Multi-thread test privacy lists management on node 'shell2.tigase.org' Multi 15
Multi 16 success 21 sec 30 709 ms Multi-thread test privacy lists management on node 'test-i.tigase.org' Multi 16
Multi 17 success 10 sec 30 334 ms Multi-thread test privacy lists management on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 17
Multi 18 success 2 sec 30 77 ms Multi-thread test of message send to daemon user on node 'veloci.tigase.org' Multi 18
Multi 19 success 12 sec 30 414 ms Multi-thread test of message send to daemon user on node 'shell2.tigase.org' Multi 19
Multi 20 success 29 sec 30 973 ms Multi-thread test of message send to daemon user on node 'test-i.tigase.org' Multi 20
Multi 21 success 4 sec 30 141 ms Multi-thread test of message send to daemon user on all nodes: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 21
Multi 22 success 26 sec 90 274 ms Multi-thread test of message send to daemon user. Each user makes 3 connections to all cluster nodes using different resources: 'veloci.tigase.org', 'shell2.tigase.org' and 'test-i.tigase.org' Multi 22

 

Test end time: Oct 21, 2008 12:46:40 AM