Crash - PLEASE HELP!

[ralonso2001]

I have a WebServer with about 50 users working on line. The app crashes two or three times a day.
I checked my code but dont find anything weird. I use Clarion 10.0.11384 and NetTalk 8.51. My database is MSSQL, also use InMemory driver for my father - son webforms.
All this is working on a virtual server.
Some times the app works fine two or three days and then some days it crashes 2,3 or even more times.
I close and open de webserver and everything goes fine againt till next hang...

What kind of debbugin tool can I use to detect the problem? I dont know if its a windows problem (running in win server 2012 standard), an MS SQL problem (SQL Server 2012) or a Clarion, NetTAlk, or my app programming Bug.

My client is tired of this situation (me too!) but I really cant find what is going on....

I made all the performance recomendation I have seen in NT Central, but still hanging...

Any help will be great to help me (if need the app or DCT or an access to the page to take a look I will be pleased to do it!!)

Thank you!!!

Rodrigo Alonso

[terryd]

Hi Rodrigo
What sort of crash
Do you get an error screen, or does the application just stop?

[ralonso2001]

Hi Terry, the app doesnt respond via web. If I try to close the web server window then I get the GPF error message with some memory address...

[Keith]

Hi Rodrigo

The app is crashing with a GPF.  If this was because of a commonly executed component then you would be down all the time but in that the problem is intermittent and you can run for days then this might indicate that it is related to an activity that is not frequently requested.

Can you narrow the area of concern and find out what is being processed just before the crash - by interrogating a log of the web server activity?  Once you know the area you will be in a much better position to debug.

Cheers

Keith

[Bruce]

Hi Rodrigo,

>> If I try to close the web server window then I get the GPF error message with some memory address...

This is common (and not a problem) if you close the server window while threads are still running. The worker threads reference back to the main thread - if you close the main thread then the worker threads will GPF. This is easy to see because the GPf is usually on Thread 3 or higher.

So let's go back to your earlier observation (and likely the root of the problem.)

>> the app doesnt respond via web.

So presumably it _is_ responding to events on the server window? You can move the window around, change tabs and so on?
If you go to the performance tab what do you see there for number of connections, number of threads, number of sessions etc? Perhaps post a screen-shot of that...

cheers
Bruce

[kevin plummer]

You should also check SQL to see if there are any process's there that are hung due to SQL Locking. This can give the effect of an app not responding.

[Rob Kolanko]

My experience with server applications, it is very hard to know what is causing program to lockup or crash without the right tools.
1)  Make sure you have Message and Halt hooks to store any internal clarion messages to a file. Clarion can throw messages for array index errors, etc and unless you are running the app on the console you will not see the message.
2)  Add Capesoft GPF Reporter to trap all the source lines of the procedures that called the procedure that caused the GPF. This way you can see the last line in your code which caused the error. I understand the Clarion 10 has a feature to read the Exception Log File from a GPF to trace the program stack,but I have not tried it.
3) Remember that there a lot of parallel processing going on in different threads a Net Talk web app. Your expected order of operations may not be what is happening. 

With step 2, at least you have a starting point for investigation. Your problem will be something that you least expected.   For example I was using the javascript timer to log off the user as well as the Net Talk server was set to terminate the user after a period of inactivity. What I forgot was that when a user timed out, my session cleanup procedure was run twice at the same time and caused a GPF when memory was freed.

I hope this helps.
Rob

[ralonso2001]

Hi, first of all, thank you all for responding!

Bruce, I didnt have the performance controls populated in my web server window, Im adding them right now to see what the say. The window is responding but if you login to the web page it never logs...

Kevin, how can I check my SQL to see if there are locks going arround? Theres a tool in SQL Management Console?

Rob, Im using Capesoft Message Box so any message window is closed in 5 seconds.

I will work on this tips and tell you what happened.

Again, thank you very much!

[kevin plummer]

Kevin, how can I check my SQL to see if there are locks going arround? Theres a tool in SQL Management Console?

> there is a query you can run - I normally just google it. SQL2005 you could see them via management options

Rob, Im using Capesoft Message Box so any message window is closed in 5 seconds.

> 100th sec would be better.

[ralonso2001]

Im looking for the SQL locking querys.
I will change message box to 100th second!
Thank you!

[ralonso2001]

Is this driver string for FM3 ok for my web server?

/MULTIPLEACTIVERESULTSETS=TRUE /BUSYHANDLING=2 /BUSYRETRIES=5 /BUSYMESSAGE=La conexion esta ocupada. /LOGONSCREEN=FALSE /VERIFYVIASELECT=TRUE

In my app in global properties - File Control - "Enclose RI code in transaccion frame" is unchecked... is this a problem?

[ralonso2001]

Bruce, here are 2 print screen of performance. The web server was responding ok but the web page was very, very slow, imposible to the users to work. The print screen are of that moment.
The CPU procesos was at 98% total with my exe consuming most of the procesor.
I close the web server and open it again and everything start workin ok...
How can I change the sql lockin time, I think it could be that my sql locks and that causes de web server to get so slow...


[attachment deleted by admin]

[kevin plummer]

If it is SQL locking then it is a bit of a process to fix...

You will need to use p_web.setSQLTimeout(Table,net:on) before all file access and p_web.setSQLTimeout(Table,net:off) after all file access.

If you can identify the sql queries\tables that are locking that is a good starting point to just put the above in place on that block of code.

In my experience it is tables that are heavily read\written that cause the problem. Having Prop:SQL as opposed to ABC file access meant I was able to see the SQL statement that was locked and zero in on fixing that block of code.

I also added code on my indexpage to detect if tables were locked and to restart the web app but this was before the above solution. This might be helpful to just get an app running. Make sure you switch on the xfiles save\restore session tables so users won't really notice that you app restarted.

Also do a search on the NG as it is something that comes up every now and then.

[bruce2]

Hi Rodrigo,

>> How can I change the sql lockin time, I think it could be that my sql locks and that causes de web server to get so slow...

Do you have any evidence for this, or are you just guessing? (guessing is ok, but it's useful to know if it's a stronger feeling than that.)

From the performance stats you can see that currently 11 threads a re running. By looking a the Get/Post log you will be able to see what the last request for each of those threads are, and when they occurred. Perhaps you have an endless loop in your code, or an especially expensive process or report or something, that swamps the server.

In other words, let's say you have a report which takes 5 minutes to generate. A large number of request for that report would slow down the system a lot, and inhibit other requests.

However I notice you have 11 threads, but only 3 connections, which implies the others maybe abandoned the request. Which may indicate that it's a never-ending process which is happening. But not just never-ending quiet (which is what I would expect from a SQL lock) but never-ending-busy which I would expect from a simple code bug nor breaking out of a loop correctly.

You'll also see from the stats that you have a fair number of requests (274) which have a high response time. This might be a report (which makes sense) or it might be for something else you are not expecting to take long. So more investigation there is useful.

I'd start with the 11 threads though - that's where the biggest clue lies.

cheers
Bruce

[ralonso2001]

Hi Bruce, hi Kevin.

Thank you both, I feel im getting closer to the solution with your tips.

Bruce, about the locking, its just a guess, dont really know if thats the problem.
I dont undestand much how the threads work but I going to look at the code (and the get/post log) to detect some bug.
I have some users that generate large reports all the time, so Im thinking of separating them in other instance on a "offline" database so they can work on reports on that database and not in the working one.
I will investigate the 11 threads!
Thank you very much!!!!