GPF when sending emails Pt2

[useless]

Reposting as there seems to be a problem with the forum.

The C6.3 RTL is mixing up the threads running.

Attached is a screenshot of the threads of the threads getting mixed up and a screeshot of the program code in question

[attachment deleted by admin]

[useless]

Raw debugview output log.

Off to try this in C8, but I've also identified a problem not connected with NT where IF statements fail the logic test when a window is subclassed as well, so its anyone guess if these two problems cause other problems like data corruption in files for example.

[attachment deleted by admin]

[Bruce]

the RTL losing track of threads seems unlikely, possible of course, but unlikely... - any chance you can post an example so I can see if we can duplicate the problem here?

[useless]

Here you go, this is a C6.3.9054 example app & dct. Look at the Thread 10 & 11 windows, you'll notice when the window opens it displays in the window title the correct thread number and the corresponding Loc:ThreadNo value but later on the thread changes to Thread 1 which you can see in the Debugview output.

Basically the ProcessEmails window on a 1 second timer starting processing emails in the outbox ie EMA:Status = 3, looks up the global array for a zero value and fills it with the EMA:EmailID that needs to be sent. The SendScheduledEmails window on its 1sec timer looks up the corresponding global array and passes it to a local var which is just a mirror of the global var, all wrapped in critical sections I might add to avoid any issues on that front.
Either way it crashes repeatedly at the same place, and although this is using TPS files, its the same behaviour with MS SQL in my apps.

All you need to do is compile this and run debugview at the same time, I have not tested this in C8 at the moment, been a busy day with meetings.

Let us know what you think.

[attachment deleted by admin]

[Bruce]

hi Richard,

The first thing I notice is that there are two "thread numbers" in play here.

the first is the THREAD() value, which is the "RTL Thread number". This is the value that the debugclass is displaying as   ' Thread('& THREAD()  in the debuger.clw class.

The second is your internal variable Loc:EmailSendThreadsRunning, which is passed as a parameter to each started worker thread (pThreadNo). There's nothing wrong obviously with passing that number, but since Loc:EmailSendThreadsRunning is a sequential integer starting with 1, there's no guarantee that pThreadno will match THREAD().

While your program often starts on Thread 1, there are cases where it won't (like when it's running as a service). Equally there are other threads in play, like the NetTalk thread itself, and thread numbers are "reused" after a thread ends. In short you should never try and predict the value of THREAD() in any situation.

So perhaps that's a bit of a red-herring.

As a complete aside, I also notice you favor the original method for prototyping procedures;
Prototype: (String)
Parameters: (pThreadNo)
A better way to prototype is to include both, in both places. ie
(String pThreadNo)

I don't think that's affecting you here, but there are cases where link errors can occur if you use the former method.

I'll continue to check the code to see if I can see the source of your GPF.

Cheers
Bruce

[Bruce]

Hi Richard,

ok, here's some more. You have a few global variables which are being used as the "shared memory" between the controlling thread, and the emailing threads. That's fine.

In ProcessEmails, Main and SendScheduledEmails, you're declaring Critical Sections to access this data. But this approach is unfortunately incorrect.

In order to work, a CriticalSection must have the same scope as the data it is protecting. In other words, the GlobalsLock object needs to be declared globally, not locally in the 2 procedures.

As it happens, because the global variables are LONGs it's unlikely that this is the root of the problem though. (The current critical sections are ineffective, but probably not dangerous).  The investigation continues.

Cheers
Bruce

[Bruce]

Hi Richard,

I think I've found the source of your GPF.
So, from what I can see, all the email threads start slowly, but then all kick in to try and send an email all more or less at the same time. This is, I think, causing the NetTalk thread some strain, which in turn (on C8) leads to a "Window already open" error.

If you add a nettalk object to the Main Service Window (I added a NetAutoServer object) then NetTalk initializes when the main window opens. Thus by the time the email threads all spring to life, the engine is ready to go.

Cheers
Bruce

[useless]

I cant see NetAutoServer in the list of objects for the NT extension template, do you mean NetAutoCloseServer instead?
I've added NetAutoCloseServer to the main window and these are the current problems:
CPU cycles at 100% (new).
App no longer closes down. (new)
Duplicated emails sometimes as many as 9 duplicates of an email (new - see screenshot).
Thread() is still returning the wrong thread.

I've adjusted the calling of the GlobalLocks CriticalSection so its now called from the global embeds once because that was my mistake.

I've also wrapped the Glo:ShutDownService in a critical section on all windows to avoid any problems there as well although it doesnt matter if this var is wrapped or not, the app still doesnt shutdown properly now since adding NetAutoCloseServer to the main window.

Duplicated emails, originally I used to only get two duplicates but now I get as many as 9 duplicates but this varies from 4-9 emails for each of the 20 test emails sent, did you not see the duplicates coming through or being sent through Ethereal?

With regard to the function Thread() not returning the right thread number is it possible the Clarion 6 runtime is actually messing up like it did in C55 and the underlying problem is the runtime, becuase it returns the correct/expected thread number when the window opens and it sets the window title properly but later on when Thread() is called its returns thread=1 later on.

Is this nettalk messing up the thread() function or is the Clarion runtime messing up its own threads?

Any other suggestions?



[attachment deleted by admin]

[useless]

Just an update, I'm upto 109 112 emails received for one of the test emails (test email 17) as you can see from the screenshot and I have already deleted quite a few before hand.


[attachment deleted by admin]

[Bruce]

Hi Richard,

>> Is this nettalk messing up the thread() function or is the Clarion runtime messing up its own threads?

It's probably worth following through on this comment, because as far as I can see, it's not messing up the Thread function. Thread() is always returning the same value on a thread, unless I am missing something?

Your number, the one you pass into a new thread (as pThreadNumber) is not the THREAD() number, and never will be.

Regarding the duplicates - as far as I can see it's because your algorithm is resending the same mails over an over. If you like I can look at that a bit closer, but your "Process" procedure never seems to end.

Cheers
Bruce

[Bruce]

specifically, your ProcessEmails procedure never checks ema:staus.

cheers
bruce

[useless]

I think there is a problem with the forum as my previous post & attachment has disappeared but your points in the two posts have been addressed.

Attached is the updated program
The problems I still have is I cant find the netautoserver object/class you refer to unless you mean the netautocloseserver?

In the attached sample app, I'm using netautocloseserver in the main window which is hopefully right because I've tried other nettalk objects and it just crashes the app in different places, but when using the netautocloseserver on the main window I still get duplicate emails and it wont shutdown properly unless all emails have been sent despite the code being there to handle shutting down the other threads even when emails are queued up.

If I take the netautocloseserver off the main window it shutdown fine.

So this problem would hang the servers and they shut down and reboot at 6am on Saturdays when ever there are MS updates to download and I've used group policy to do this automatically for the servers and workstations so I could get rid of WSUS as it was hogging and using up a lot of resource.

So if NetAutoCloseServer is the right object to use on the main window, anyway I can get rid of the random duplicate emails that are now appearing and how can I get it to shutdown properly when NetAutoCloseServer is on the main window?

[attachment deleted by admin]

[Bruce]

Hi Richard,

>> unless you mean the netautocloseserver?

yes.

However the point of this class was just to "fire up" the NetTalk engine. You could just as easily do it by sending yourself an email. What you want to prevent here is that all the email threads jump in together, and hence all try and initialize the engine at the same time. So any nettalk interaction (by just one of the threads, or the main thread, or whatever) is what you're after.

I wouldn't have though that the AutoCloseServer would have interfered with the shut-down process. But I'll take your word for it.

I'll see if I can spot the problem causing your duplicates later this morning.

cheers
Bruce

[useless]

I've removed the NetAutoCloseServer from the main app window and I've added

    IF Loc:FirstTimeRun = 0
        Loc:FirstTimeRun = 1
        Loc:RandomSleepPeriod = Random(GLO:EmailThreads * 10,GLO:EmailThreads * 1000)
        dbg.DebugOut(Thread() &' '& GetCurrentThreadID() & ' Timer1 Loc:RandomSleepPeriod='&Loc:RandomSleepPeriod)
        Sleep(Loc:RandomSleepPeriod)
        dbg.DebugOut(Thread() &' '& GetCurrentThreadID() & ' Timer1 In the words of Murray Walker... He''s Off, He''s Off, He''s Off!')
    END

to the SendScheduleEmail window immediately before the routine call Do SendEmail in the TimerEvent to make sure each thread doesnt start at the same time.

So the 1st random delayed start on which ever thread is first in theory fires up the NT send email engine which should do the equivalent job of your suggestion to place NetAutoCloseServer on the main app window?

However despite the above workaround the NT send email engine is duplicating emails and I still get some GPF's but not as often, its random.  

Does the NT send email engine need a minimum period of time between each 1st send on each thread because the risk with the random delay workaround is I might still get two threads starting for the first time very close together in terms of term, maybe just milliseconds apart?

Is the NT SendEmail code threaded or is it just one thread handling the asynchronous communication despite being called multiple threads?

I wonder why you havent seen this in your apps which are running multi threaded that you mentioned in a previous post?