Webservice and (threading?) performance issues with json output - Part 2

[Bruce]

this is a continuation of
http://www.nettalkcentral.com/index.php?option=com_smf&Itemid=36&topic=6673.msg27186#new

>> I've looked at the jFiles class a little trying to improve performance and can't really tell what's making it slower

the key to improving performance is to use a Profiler to show where it is going slow. I'm using this;
http://www.capesoft.com/accessories/Profilersp.htm  -  Trying to do it by "eye" is a waste of time.

>> In itself the json is roughly a factor 1.2 to 1.5 slower than xml (for the tests I'v run) that's fine/acceptable. but when the requests come in randomly serving json becomes really troublesome. My production server will have to server even larger result sets and I can't hope that there will be no "collisions".

Clearly collisions will be happening so it needs to cope with that. I haven't managed to get back to testing yet, but there are a variety of approaches that can be taken. I've also got some suggestions for reducing memory usage, so that's good.

>> So I'm still looking for a way to produce and serve json fast Maybe a really stupid question: but is there a chance of producing JSON with the xFiles class (in the end it goes through the same structures and I can see "some" resemblence between xml and json <g>)?

some years back I looked into that - indeed there are some remnants of that in the xfiles.clw file, but it becomes somewhat limited quite quickly. It's possible that for edge cases there are approaches like that, but I would likely extend jFiles to handle edge cases rather than tweak xFiles at this point.

But there are a number of other options to try first before going down that road.

Cheers
Bruce

[AtoB]

Hi Bruce,

actually I agree with you (that it's best solved in the json class ...), although that doesn't help much of course <g>.

If you need any additional info/testing etc. just let me know

thanks so far
and regards
Ton

[Bruce]

Hi Ton,

progress update: I was playing with the memory allocator from Dave Nichols, and also fiddling with jFiles a bit.

At the moment I'm running the client and server on the same machine for testing (which obviously adds a bit of extra work to the server.)

I'm doing 109 requests across 20 client threads. The total time for the test is 2 minutes 30 seconds. Processing time for each request is around 1.5 seconds. The server only has 4 cores though so they queue up a bit. But they clear pretty quickly - as I say, all 109 are done inside 2.5 minutes.

I need to do some more testing before releasing updates etc, but it's certainly looking quite positive.

(Remember this is processing a really large record set, so some time is inevitable.)

Ram consumed is stable at around 90 megs for the server.

cheers
Bruce

[AtoB]

Hi Bruce,

that's good news, if you need a tester for this, let me know!


thanks so far

regards,
Ton

[Bruce]

hi Ton,

I've made a "charityware" product out of it ($20).
http://www.capesoft.com/accessories/fastmemsp.htm

I'll also be doing an update for jFiles (free ) which should reduce the memory used there a bit.

Let me know how it goes for you.

cheers
Bruce

[AtoB]

Hi Bruce,

I've implemented Dave's memory management model and allthough it hughly improves the speed for the jFiles class, I'm not convinced on other parts of my app yet. I see some minor improvements here and there, but also speed degradation for some procedures. Also I'm not sure about any side effects (not initializing object properties ...) it might have. In short:for me (!) it's not worth the risk ...

so I decided to write my little "json" class that mimics the output I'm after (just a bunch of queues in a collection mainly ...), but than as fast as possible. That resulted in the producing of the return packet from taking 1.39 seconds to 0.18/0.19 seconds for each run (so the the total server side time take dropped from roughly 3.3 to 2.1 seconds for a single run) and as this code only creates a single (stringtheory) object it serves multiple request fast too!

I whish we could tie this memory improvement only to the creation of objects, then we all could have best of both worlds. But that's probably only up to softvelocity ...

Regards,
Ton

[Bruce]

Hi Ton,

>> I've implemented Dave's memory management model and allthough it hughly improves the speed for the jFiles class, I'm not convinced on other parts of my app yet.

>>  I see some minor improvements here and there, but also speed degradation for some procedures.

we're doing various benchmarks and also investigating various situations to make sure that speed is optimal. I think there are some cases at the moment where speed is slightly slower (for some very small value of slightly) but we're seeing what can be done about that. It's still early days.

But I'm glad you've found an alternative approach that suits you.

>>  Also I'm not sure about any side effects (not initializing object properties ...) it might have. In short:for me (!) it's not worth the risk ...

I understand. I'm not sure there are side-effects with the properties, but I'll check that to be sure.

update 1: I think you have misunderstood something along the way. In my testing here it definitely does clear the object properties to 0 just as the Clarion one does. Failure to do this would indeed break a lot of code, but in my tests here it's definitely clearing the properties.

update 2: the performance benefit is definitely the most significant when multiple threads are in play. At the moment there seems to be a slight (emphasis on slight) performance penalty with single-threaded situations - something Dave is working to eradicate.

cheers
Bruce

[oggy]

Just one question about it: is it working with Nettalk web server + SOAP.
I used it a lot to get and return some xml files? I noticed that under some time all slow down to the point where service must be closed and re-opened...

[Bruce]

Hi Oggy,

I don't think it will have any impact on SOAP, since that is using XML, not Json.

>>  I noticed that under some time all slow down to the point where service must be closed and re-opened...

What XML engine are you using?
Have you investigated the slow-down in any way? (ie ram usage, cpu usage etc?)

cheers
Bruce

[oggy]

 Let me explain it.
My SOAP service do not create any of XML "file" in this process. Just simply get it via SOAP, reroute it to the another service, outside my domain, wait for response from that second service, and back it to the client. Right after sending back response to client I write to MS SQL database.. All works well, some time...
There is random lockup, I think this is random, because sometimes all works well for days, and sometimes I must restart service couple times in one day.
I noticed only that when traffic is heavy, about 5-10 simultaneous request/respons in one seconds, thats occured... And in some time, this traffic will be even heavyier....
Just yesterday, i bought Fastmem, and for now, there is no lockup or slowing down, that is why I ask this question. I am interested in some information, how well, or how fast is NETTALK webserver is, particulary with that SOAP request/response? How much these requests NETTALK can handle in one second?
Regards, Oggy

[Bruce]

Hi Oggy,

You mentioned a slow-down earlier - now a lockup? I'm a little confused as to what exactly is going on in your app, but no matter.
If the fastmem has helped then that's a good thing.

>> I am interested in some information, how well, or how fast is NETTALK webserver is, particulary with that SOAP request/response? How much these requests NETTALK can handle in one second?

I have a car. With a 3 liter engine. From that can you tell how fast it goes? Not really.
The speed of the car depends on all the other parts (tires, gearbox and so on) as well as the state of the road, amount of traffic and so on.

It's the same with a web server. The speed of the server is completely dependent on the machine you are running it on, the code you have written, and the network environment it is running in. For example, you mention communicating with another service, so presumably you need to factor the speed of the other service into the speed of your service.

Even relatively simple tasks are going to be bound to the hardware - the faster the machine the faster the server. the faster the disk and database the faster the server.

The NetTalk part of the equation is "fast enough" so that it's not usually the bottleneck in a situation. On my machine I can serve 200 to 300 requests per second. But of course that number is meaningless when comparing to your situation, your server, your network and your code.

If you feel there is a performance issue in play, then the first step is to determine what the constraining factors are. Are you running out of CPU? Disk speed? network speed? your code speed? and so on.

Cheers
Bruce

[oggy]

Lockup and slowdown is problems that I encounteres from time to time. First I noticed slowdown and after some time the service is "dead", no response. The service then must be terminated.
This all stuff is on one IBM x3650 M edition of server, 2 Xeon 6 cores each, RAID 1+0 15k hard disks, for OS and backups, and one SSD disk where SQL database is.
32 GB of RAM is on the server. OS is Win 2008 R2. We are on optic-fiber internet, 200 Mb download and 100 Mb upload speed, but soon this will be changed to 500/250 Mb system. I ask this question primarily because I am not sure if I go to contract where customers will pay for each response/request cycle about a 1 cent (EURO). If I sign that contract I must be sure to have about 99% of time without lockups... 24/7 cycle, 365 days in year...
Also, one problem, that I mentioned some times ago, in other post(s) is that NETTALK 7 did not use async posts. NETTALK 9 is in financial plans for some times, if async works well in NT9.
EDIT: for some times I suspect that MS SQL is my problem, and for two days I just disabled database, and again, slowdowns and lockups occured.
Two days ago I enabled Fastmem template and for now there is no problems. Maybe, just maybe, the memory allocation system (raw Clarion) is bottleneck for this... Now I have about 5, roughly, per seconds, and my system works well But, my wishes to get about 20 per seconds....

[Bruce]

Hi Oggy,

Machine sounds fine - the key is that when there is a slowdown you inspect the server to see what is happening. Too many threads running? CPU busy? disk busy? and so on.

>> Also, one problem, that I mentioned some times ago, in other post(s) is that NETTALK 7 did not use async posts. NETTALK 9 is in financial plans for some times, if async works well in NT9.

I'm not sure I understand what you mean here. Perhaps start another thread though - if this one gets too long I get locked out.
Everything in NetTalk is asynchronous, and always has been, so in your new thread please elaborate on what you mean by "async post" - and more importantly what you feel NT7 is currently not doing.

From a performance point of view, NT9 is much faster than NT 8, which was much faster than NT7. Each release leads to more optimizations which make things faster. But it doesn't sound like raw speed is your problem here. I think you have some other issue in play.

Cheers
Bruce

[oggy]

My server do nothing except what I mentioned, SOAP response and request. The CPU workload is so low I cannot believe that CPU is problem. Nor memory... In peak hours I watched memory and never, ever, the memory load of my service does not go over 100 MB. Never.
The disks on machine is OK. SSD disk is also ok.
In my code I do not any sort of line that will break down entire service. I think so...
Just plain catching incoming xlm into stringtheory, and reroute it to other service, waiting for response from another service, and returning back. End of story, My server is just serving like postman between my client and other service.
For two days, after installing Fastmem template, all works like i want it, fingers crossed.
In monday I will start with another instance of the same program, on the other machine, solely for testing. if I ask some help in testing, I will wrote some basic test program to send simple xml
and catch it back.
Best regards. Oggy
BTW, NT9 will be soon in our "garage"