bosh speed tests

Now with the newly announced node-xmpp-bosh (NXB) my first tests left me with the impression that this thing is pretty speedy to what I am used to. So I decided to do some quick benchmarks on different implementations of BOSH. I did these tests on my desktop (some intel core 2 duo with 2,4 GHz and 4GB of RAM).

Login-Times (in ms):

Native NXB
Ejabberd 2.1.6 593 (process_delay: 100ms)
417 (process_delay 10ms)
204
Tigase 4.3.1 145 222
Prosody 0.7.0 111 170

So what we can see here is that NXB might be a feasible improvement over ejabberd’s native BOSH interface while tigase and prosody do fine in both cases.
Would be nice of course to also test with some decent load and see how the configurations behave with different loads put on them. But that’s out of scope of my quick tests for now, sorry.

Update: Ejabberd honors a ‘process_delay’ parameter which makes the BOSH CM wait for this amount of 100ms for incoming packets from the server in order to reduce the overall number of roundtrips between client and BOSH CM. This results in an overall improved client experience. Although it might be a bit slower at login times.
By default this value is set to 100ms so I did my tests again with ‘process_delay’ set to 10ms. A value of 1ms or 0ms made ejabberd fail to work properly unfortunately so I can’t provide any further numbers. But as the overall number of request roundtrips for a login is 4, the maximum number of time saved for the login would be 40ms in best case when only dealing with the ‘process_delay’ parameter.

9 Responses to bosh speed tests


Comments

  1. Comment by Phil | 2011/03/30 at 03:56:50

    Any chance that you can share you’re test code? I’d be interested to retest with ejabberd 3.0 as they are apparently changing out their admittedly slow http capabilities with mochiweb.

    Also, any explanation/theories as to why NXB is slower proxy’ing tigase and prosody … but faster than ejabberd native?

  2. Comment by Steve | 2011/03/30 at 08:37:15

    Phil, my test code was just some simple jsjac based client that starts a timer before login and stops it once authentication has been successful. The code can be found at https://github.com/e-hulp/xmpptk.

    As to your question about NXB being slower than native implementations: An external component will always be slower than a native BOSH interface (if its logic itself is comparably fast) because it has some overhead involved. It needs to establish another tcp/ip connection between the CM and the server and it needs to parse the BOSH request and then the request must be parsed again at the server. While a native interface only needs to parse the whole request once.

    In case of ejabberd the BOSH interface itself is so much slower than the external component that those advantages just don’t have much effect.

    My guess is that ejabberd’s BOSH might be so slow because of the involvement of mnesia. Which also makes ejabberd’s BOSH interface quite different from the others as it is clusterable without any further requirements. So maybe the pure speed comparisons are not totally fair.

  3. Comment by Steve | 2011/03/30 at 09:00:40

    Stupid me!

    I forgot about ejabberd’s ‘process_delay’ feature. At every request ejabberd’s BOSH interface waits per default 100ms for incoming data from the server in order to reduce the overall number of requests. If one would test a real client login with loading roster and handling all incoming presence (which is a bit hard to determine automatically) I’m pretty sure the results would look quite different. But anyway. I did my tests again with process_delay set to 10ms and got a value of 417ms for ejabberd. Which is significantly lower then the first value but still significantly higher than all others.
    I’ve also tried setting process_delay to a value lower than 10ms but then ejabberd doesn’t seem to work at all anymore. Probably there’s a bug. Anyway, as there are exactly 4 roundtrips involved with a single login the maximum time that could be safed by lowering process_delay would be another 40ms.

    But I think a process_delay anywhere between 10ms and 100ms is a good thing for the overall client experience.

  4. Comment by Phil | 2011/03/30 at 11:31:59

    Thanks for the update.

    And again for the explanation about the process delay and the explanation about mnesia and its role in the auth process. Sometime in the near future I’ll be circling back on this and investigating what the cost would be under load for auth only, auth+presence, auth+presence+roster with the various built-in and one custom auth model for ejabberd.

    I suspect that odbc based auth (which is what you want for semi-elastic scaling via clustering for a domain) will be even slower. But, as you pointed out, you get other benefits that may outweigh the overhead.

  5. Comment by Steve | 2011/03/30 at 11:36:26

    Hi again. Just a remark to avoid confusion: When I’ve been talking about Mnesia being involved this on the one hand means yes, authentication. But Mnesia is also needed for runtime information shared across the cluster. Even if you’re having odbc as your storage engine those Mnesia based parts are still required.

  6. Comment by Dhruv | 2011/04/14 at 17:23:02

    Hello,
    I am curious to know when you started the timer and when did you stopped it? The reason I ask is because NXB will currently send a positive session creation response as soon as it receives a request (without even contacting the XMPP server). It then tries a bunch of strategies to look up the server (based on the ‘to’ attribute) and then connects to it. Later on when the packet is available, it is sent back to the client, which is when the connection to the backend XMPP server was actually established. This is done mainly to optimize the case of multiple streams.

  7. Comment by Steve | 2011/04/14 at 17:43:32

    Dhruv, the clocked was stopped after a full and successfull xmpp login (without fetching rosters).

  8. Comment by Dhruv | 2011/04/14 at 20:13:00

    Steve, Thanks!!
    I am assuming the clock was started when the session creation request was sent right? If that’s the case then it’s perfect!!

  9. Comment by Steve | 2011/04/15 at 12:49:25

    Yes, exactly, the clock was started right before the session creation request has been sent.


Comments are closed