New tests page

Submitted by kobit on Sat, 2008-11-15 12:34
::

This is a page with reports for the automatic tests we run for each public release of the Tigase server. In fact we run these tests much more often. They are part of our development.

This is the new page with redesigned layout for test reports. The old page for all older versions is available here. We are now running more and more tests for each release to limit number of bugs to minimum thus the new page to present results in more convenient way.

What tests do we run? There are 4 major automatic tests scenarios we run:

  1. Functional tests - published for each release for all databases we support: MySQL, PostgreSQL and Derby. There are also 3 variants for each database the server can be configured in: default authentication, stored procedures based (SP) authentication and authentication based on custom SQL queries (SQL). Functional tests are run for all of them.
  2. Performance tests - published for each major release for all databases we support: MySQL, PostgreSQL and Derby.
  3. Load tests - we run them occasionally when we have an access to proper equipment as we don't own hardware good enough to run load tests. The last load tests results are published on a separate page. Click on the 'Load tests' link in the menu on the left hand side of the page.
  4. Clustering tests - we run them during our development process to make sure the server works correctly in the cluster mode. The results are not directly related to any public release yet and they are available on a separate page. To find the report please click 'Clustering tests' link in the menu on the left hand side of the page.

Functional tests

Basic tests to check all the functionality of the server

 

Derby database

Tests for Apache Derby database

Derby database
Version Derby default Derby SP Auth  Derby Custom SQL
5.0.0-b2135 00:01:10 00:01:09 00:01:09
5.0.0-b2103 00:01:19 00:01:12 00:01:14
4.3.1-b1858 00:01:29 00:01:18 00:01:11
4.2.0-b1720 00:00:52 00:00:53 00:00:50
4.1.0-b1315 00:00:45 00:00:45 00:00:43
4.0.1-b1229 00:00:39 00:00:40 00:00:36
4.0.0-b1212 00:00:35 00:00:35 00:00:34

 

MySQL database

Tests for MySQL database

MySQL database
Version MySQL Default MySQL SP Auth MySQL Custom SQL
5.0.0-b2135 00:01:07 00:01:07 00:01:06
5.0.0-b2103 00:01:08 00:01:08 00:01:06
4.3.1-b1858 00:01:15 00:01:17 00:01:11
4.2.0-b1720 00:00:49 00:00:47 00:00:48
4.1.0-b1315 00:00:39 00:00:39 00:00:40
4.0.1-b1229 00:00:29 00:00:31 00:00:34
4.0.0-b1212 00:00:30 00:00:29 00:00:30

 

PostgreSQL database

Tests for PostgreSQL database

PostgreSQL database
Version PostgreSQL Default PostgreSQL SP Auth PostgreSQL Custom SQL
5.0.0-b2135 00:01:05 00:01:05 00:01:05
5.0.0-b2103 00:01:06 00:01:04 00:01:06
4.3.1-b1858 00:01:21 00:01:16 00:01:08
4.2.0-b1720 00:00:48 00:00:48 00:00:46
4.1.0-b1315 00:00:40 00:00:38 00:00:38
4.0.1-b1229 00:00:34 00:00:33 00:00:33
4.0.0-b1212 00:00:30 00:00:30 00:00:30

 

Clustering tests

Checking the server functionality in the cluster mode.

Clustering tests results
Version Test results
4.3.1-b1858 00:01:45