blogs

It has happened recently that we have tried very hard to fix a few annoying problems on one of the Tigase installations. Whatever we did, however the problems still existed after uploading a new version and the server restart. It worked fine in our development environment and it just didn't on the target system.

It turned out that due to a specific environment settings on the target system an old version of the Tigase server was always started regardless updates we were uploading. When I finally started looking at the installation the first indication that something is wrong was lack of any log files in place where I expected them.

The best way to check all the environment settings used to start the Tigase server is to use..... check command line parameter:

./scripts/tigase.sh check etc/tigase.conf
Checking arguments to Tigase
TIGASE_HOME = .
TIGASE_JAR = ./jars/tigase-server.jar
TIGASE_PARAMS = etc/tigase.conf
TIGASE_CONFIG =  etc/tigase.xml
TIGASE_RUN = tigase.server.XMPPServer -c etc/tigase.xml --property-file etc/init.properties
TIGASE_PID = ./logs/tigase.pid
TIGASE_OPTIONS = --property-file etc/init.properties
JAVA_OPTIONS = -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 \
    -Djdbc.drivers=com.mysql.jdbc.Driver:org.postgresql.Driver \
    -server -Xms100M -Xmx200M -XX:PermSize=32m -XX:MaxPermSize=256m
JAVA =  /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin/java
JAVA_CMD = 
CLASSPATH = ./jars/tigase-server.jar:./libs/jdbc-mysql.jar:./libs/jdbc-postgresql.jar:\
    ./libs/tigase-extras.jar:./libs/tigase-muc.jar:./libs/tigase-pubsub.jar:\
    ./libs/tigase-utils.jar:./libs/tigase-xmltools.jar
TIGASE_CMD = /System/Library/Frameworks/JavaVM.framework/Versions/1.6/Home/bin/java \
    -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 \
    -Djdbc.drivers=com.mysql.jdbc.Driver:org.postgresql.Driver \
    -server -Xms100M -Xmx200M -XX:PermSize=32m -XX:MaxPermSize=256m \
    -cp ./jars/tigase-server.jar:./libs/jdbc-mysql.jar:./libs/jdbc-postgresql.jar:\
    ./libs/tigase-extras.jar:./libs/tigase-muc.jar:./libs/tigase-pubsub.jar:\
    ./libs/tigase-utils.jar:./libs/tigase-xmltools.jar tigase.server.XMPPServer \
    -c etc/tigase.xml --property-file etc/init.properties
TIGASE_CONSOLE_LOG = ./logs/tigase-console.log

In our case TIGASE_HOME was set to a fixed location pointing to an old version of the server files. The quick check command may be a real time saver.

The Tigase server is very extensible through the well defined API. You can easily attach your code to any part of the server and load it at the startup time. It is quite simple to connect the Tigase to any database, add own components, plugins, packets filters, resource monitors and so on.

There are, however at least two disadvantages of this kind of extensibility. You have to write Java code and the code can not be reloaded at run-time. So basically you have to restart the server to apply your changes.

Installing Groovy script

What about a code in a scripting language which could be reloaded/added/removed at runtime without affecting normal work of the server? Which scripting language? Ideally any, ideally your preferred language. How? Of course using ad-hoc commands.

This is possible now. You can create your scripts in almost any language and load the script at runtime to the server to do some work for you. The Tigase server supports scripting through JSR-223 API in Java6. Maybe it doesn't make sense to use some of the supported languages like AWK for example in the Tigase server. Nevertheless I have learned already that users' use cases go far beyond my expectations. Therefore there are virtually no restrictions on the language you choose. Just put all required JAR files in the Tigase libs/ directory and the language is available to you.

The main part of the Tigase server development is testing. Actually I spend much more time on testing the server in different scenarios than on writing the actual code.

Most of the tests can be run locally on the development machine. Performance tests and especially load tests require more hardware but usually the development machine as a client and one mor as the server is enough.

One category of tests however require more hardware to be involved and in the best case scenario that hardware should be always on to see how the software behaves over the longer period of time. Clustering tests. They require ideally 3 machines for the server deployment. I run automated tests on the Tigase server very often and also this clustered installation is available to the public at the address: xmpp-test.tigase.org. By running it for a long time and exposing it to the public I can catch many exceptional cases and fix problems which don't appear on short automatic test runs.

At the moment I use 3 machines which are always turned on. Unfortunately 2 of them are very old hardware which have started to refuse working more and more often. Therefore I am looking at replacing them with something different. Ideally it would be low power consumption machine with at least 512MB RAM. There is even no hard disk needed as the system could be started from USB stick or DVD disk. It could be also a laptop with at least single core Pentium M 1.5GHz and 512MB RAM.

If you have such a hardware and can offer it to the project I would very much appreciate your help. Please send me a message using the contact form, or via email/xmpp at the address: kobit AT tigase.org to discuss details. 

This is our the most recent addition to the Tigase server implementation: automatic and self-monitoring tools inside the server. This stuff is exceptionally useful for advanced users and those who manage Tigase installations working under a high load.

There have always been statistics in the server accessible via ad-hoc commands which could provide you with lots of details about the server internal states, the load on the server, performance and so on. The disadvantage of using them was that you need to periodically check the statistics and actively monitor the server in order to detect that something might go wrong.

Alternatively you could write a bot which could pull statistics periodically, analyze them and send a notification if something goes wrong. This solved the problem with manual checking statistics but still monitoring of the server is somehow limited.

To overcome all the limitations and even expand possibilities to monitor the state of the Tigase installation we have added automatic monitoring tools to the server. They allow to early detect if something goes wrong or if the server is under high load and can't keep up with processing all the data.

It consists of 2 parts:

1. Monitor Framework and API - which creates a basement for the monitoring system and allows interaction between a user and the monitoring system.
2. Resource Monitors - which are small plugins responsible for monitoring certain resource. It's kind of Unix like approach to have small and simple applications responsible for just one thing.

This is a simple presentation showing how to manage virtual hosts on the Tigase server from the XMPP client.

Java is still known as the environment with high resources consumption and programs written in Java as being very slow. When I talk to some people or developers I hear that you need at least 10MB of memory just to start Java Virtual Machine and if you want to run non-trivial application you need hundreds of megs.

This is something which seems to be verifiable quite easily. Let's write a few lines of identical code in C, C++ and Java. Compile it, run and measure time and memory usage. Java stays way behind even if you measure just code execution and omit application startup time.

In my Tigase server development I run lots of tests to ensure not only that all functions work correctly but also to ensure good performance, to check the server limits on certain hardware and... to test resources consumption.

Below is a table with tests reports I've run for the Tigase server with only 10MB of memory assigned to JVM. This is a full server installation with all typical components: SM, c2s, Bosh, MUC, PubSub, StanzaReceiver, VHostManager and even one external protocol component (XEP-0114) for connecting transport or Yate server. All tests passed fine for all supported databases.

Obviously you can not expect high performance from such installation and you can not connect too many users but it is perfectly usable and functional. It seems to be an ideal solution for small installations if there are just a few people using it for local communication. It can be put on old machine laying around or can be used as an embedded XMPP server for routers or other small devices.

An interesting conclusion from this test is that while Java seems to be slow and use lot's of resources for trivial applications it might be quite opposite for a big and complex programs. I think it is actually easier to implement high performance complex software in Java than it is in C. And I say this as a developer with 8 years of experience at programming in C and ASM on Linux. You have just to count every single bit while programming in Java as you would do this while programming in C or Assembler.

Our tests reports page has been changed and is now available at the new address. You can still access test reports for old version at the old tests reports address.

The reason for this change is that we run much more tests now than we used to do before and the page needed some redesign to make it easier to access all reports. We not only increase number of individual test cases but also we run increase number of test scenarios.

We have plans for the future to add more operating systems, more databases and more complex scenarios to the test suite to make sure we release new versions with as little bugs as possible.

If you have any suggestions to the test report page or any ideas for test scenarios use cases I would be happy to hear them.

We are not moving project tracker or Tigase server development to launchpad.net however.

The purpose of the project created over there is to host prebuilt Ubuntu packages in a PPA (personal package archive).

A few weeks ago I announced that Gentoo ebuilds are available. I have also received information about RPM packages being maintained by someone. That would be the third packaging system for the Tigase server I know of. This makes live easier for all those who want to install Tigase server on different Linux distributions.

While I can not maintain packages for all systems myself I am always happy to help and assist those who decide to take this responsibility. If anybody is interested in helping with those please contact me. The Gentoo Linux ebuilds will be hosted in the main Tigase server SVN repository. Ubuntu packages on launchpad.net, I am not sure where RPM packages will be available but there is always a lot of space in our SVN.

The last release of the Tigase server included 2 our components: PubSub and MUC.

Unfortunately both components were not working with Apache Derby database. Due to a small bug. On top of that there was name change for the main class of the MUC component in the mean time I was not aware of. This causes configuration to be incorrectly generated by the installer.

We are going to release a big fix version today or at latest tomorrow for the Tigase server. In mean time for all those keen to get both components working ASAP I am giving both files to download below this article. If you want to install them, remove tigase-pubsub.jar and tigase-muc.jar from your libs/ directory and put there files downloaded from here.

For the MUC component you have to also edit init.properties file and replace MUCService with MUCComponent. Don't forget to remove the existing XML config file after before restarting the server if you edited the init.properties.

We are sorry for all the inconvenience that could caused.

The whole Tigase server development is oriented around automatic tests we run using Tigase Test Suite. Our aim is to not release new stuff before it is tested using automated tests. Only this way we can be sure all the server features work the same way for every release. The drawback is that we first need to create tests, then we can run them to test the server. It is a bit like double development effort.

The suite evolves together with the server. At the beginning I was running only functional tests executing test cases sequentially. Then I added performance tests which required running many tests concurrently on a separate connections and stability tests which in turn required lower resources consumption and..... stability. Every new addition required slightly new features set.

A few months back when I was running load tests that was another challenge to run the test suite from multiple machines and again I had to lower resources consumption to be able to open more than 60k user connections from each test suite instance.

We extensively use Bosh in a few Tigase installations so to provide decent support for web clients I have added Bosh tests which required a different way to maintain connections to the server because the Bosh client can send each request on a separate TCP/IP connection.

Now the clustering in Tigase. We have working clustering mode in the server for a few months already but we decided not to release a new version before it is well tested using automatic tests. New challenge - open multiple connections from the test suite and each connection must end up on a different cluster node.

Today I am happy to announce all the cluster tests have passed. This is a big step for us in our preparation for publishing Tigase 4.0. Now we can focus on those countless tiny bits required to prepare binary packages for the new version.

Below is a copy of the clustering tests report, for those who are interested in more details.

Thanks to mazzachre user we have now Gentoo Linux ebuilds for the Tigase server. The euilds are not yet available in the Gentoo main portage tree. Instead they are in the Gentoo Sunrise overaly.

You can browse for the ebuilds in the source repository looking for the im/tigase-server entry. The ebuilds install the Tigase server from sources.

Please let us know if you spot anything unusual or incorrect with them. And also, if you have any suggestions or notices don't hesitate to contact us.

Initially the Tigase project was about the XMPP/Jabber server development. After a while the project received a support from other developers more interested in client side development. So now we host quite a few projects related to the IM and XMPP in particular.

The main focus is put to the server development and to the web clients development. By web client I mean not just the Minichat application used by many visitors of our website but also our AJAX libraries for XMPP/Bosh communication which are used in a few bespoke projects and also our child project - Tigase Messenger.

We have been working on the Tigase Messenger for some time already and it is still not yet finished product but we think it can be demonstrated to the public. Please go to the messenger.tigase.org page, login, play with it and let us know what you think.

For number of reasons the database schema had to be changed for Tigase server version 4.0. The most important are:

• Compliance with the XMPP RFC which says that each part of JID may have up to 1023 characters. We store in the database user JIDs without resource name thus the maximum possible size of the user id is 2047. There aren't really JIDs that long yet but we experienced quite long JIDs in a few installations already so we decided to prepare Tigase to accept any JID allowed by RFC.
• Performance and flexibility - the Tigase server now accesses database using stored procedures. This allows for any database storage format and it doesn't really matter for the Tigase server what is the database schema how data is organized inside. What it needs is just bunch of stored procedures to access the data. This allows for much more flexibility in storing user data as well as much easier integration with third-party systems and also organize data in more efficient way.

Therefore when you run the Tigase server now it may (depending on what exact SVN revision you use) refuse to start if it detects that the database schema is not updated. If it happens just follow steps below to update the database schema and start the server again. Updating of the database schema is very easy and almost fully automated process. Just follow the steps below and you should be able to run new version of the Tigase server in a few minutes or even seconds depending on your database size. It takes around 7 minutes to update database with 200k user accounts on an average machine. Note. Do not update the database schema before the Tigase server tells you to do so. And do a database backup before starting the schema update.

The dependency for Tigase Utils Package has changed. This is important for everybody who builds the Tigase server manually from sources using Ant tool. The Maven handles all the dependencies automatically and scripts have been updated.

Please keep reading for more details how to compile the server from sources in current SVN repositories.

We have just launched the Tigase XMPP testing zone. This is the Tigase server clustered installation with 2 nodes. The installation is integrated with the Drupal CMS.

At the moment it offers c2s, s2s and Bosh protocols. We are going to enable more services for testing soon: MUC, PubSub, MSN transport and possibly more.

Please feel free to create an account on the Drupal website and use it for testing of your software.

Note. The installation is available through rather average broadband connection and it is not suitable for any kind of load or high traffic tests.

As usually all comments and suggestions are very welcomed. If you have ideas to improve the installation and make it easier for your tests please let me know.