GPF When Sending Emails Pt3

[Bruce]

Richard,

I don't know why the threads seems to go blank in these forums - I suspect somewhere it has something to do with the length of the actual page. But it's hard to say.

Here's your last post to recap;

------------------------------

I've removed the NetAutoCloseServer from the main app window and I've added

    IF Loc:FirstTimeRun = 0
        Loc:FirstTimeRun = 1
        Loc:RandomSleepPeriod = Random(GLO:EmailThreads * 10,GLO:EmailThreads * 1000)
        dbg.DebugOut(Thread() &' '& GetCurrentThreadID() & ' Timer1 Loc:RandomSleepPeriod='&Loc:RandomSleepPeriod)
        Sleep(Loc:RandomSleepPeriod)
        dbg.DebugOut(Thread() &' '& GetCurrentThreadID() & ' Timer1 In the words of Murray Walker... He''s Off, He''s Off, He''s Off!')
    END

to the SendScheduleEmail window immediately before the routine call Do SendEmail in the TimerEvent to make sure each thread doesnt start at the same time.

So the 1st random delayed start on which ever thread is first in theory fires up the NT send email engine which should do the equivalent job of your suggestion to place NetAutoCloseServer on the main app window?

However despite the above workaround the NT send email engine is duplicating emails and I still get some GPF's but not as often, its random.   

Does the NT send email engine need a minimum period of time between each 1st send on each thread because the risk with the random delay workaround is I might still get two threads starting for the first time very close together in terms of term, maybe just milliseconds apart?

Is the NT SendEmail code threaded or is it just one thread handling the asynchronous communication despite being called multiple threads?

I wonder why you havent seen this in your apps which are running multi threaded that you mentioned in a previous post?

[Bruce]

>> Does the NT send email engine

it's probably worth noting that it's not the "send email engine" but rather the whole "comms layer" than initializes TCP/IP and so on that has to be fired up. Which is why _any_ networking (including the AutoCloseServer) was sufficient.

>> need a minimum period of time between each 1st send on each thread because the risk with the random delay workaround is I might still get two threads starting for the first time very close together in terms of term, maybe just milliseconds apart?

Regardless of the email engine, it's not a good idea to start threads overly quickly in Clarion, unless you have a fairly good understanding of how the threading system actually works. I usually like to get at least one event through the ACCEPT loop between thread starts. Very often I'll sync the thread being started (using NOTIFY) so that the parent doesn't start the next thread until the first one has completely started.

>> Is the NT SendEmail code threaded or is it just one thread handling the asynchronous communication despite being called multiple threads?

To answer this I first need to mention the idea of "layers". TCP itself is build on the "layered" idea, where each layer interacts only with the layer below it, and the layer above it. NetTalk extends this idea.

At the end of the day there's only one Winsock interface. So the NetTalk DLL is a "layer" that wraps this up and acts as the "bridge" between your application and Winsock. This happens on a single thread. (The "NetTalk window" is a hidden window, running on a thread in the background.) Indeed this "extra" thread is where you got misled by the whole thread numbering in the RTL.

Each Send-Email object is running on a separate thread. It passes information across to the NetTalk thread, and gets back responses from the Nettalk thread.

>> I wonder why you haven't seen this in your apps which are running multi threaded that you mentioned in a previous post?

I think one of the primary differences in our approaches is that you are using timers, and "time delays" to manage your thread handling. I prefer a system where there are no timers, but where events are used to trigger the next action. For example when an email thread has finished initializing then it posts an event back to the caller. When the processor has a new email to process it would send an event to the appropriate email thread and so on. It's quite a different architecture to the one you have built.

cheers
Bruce

[useless]

Which form of event messaging would you go with then before I get started?
1. NOTIFY and NOTIFICATION.
2. Posting User Events (to different threads).
3. Subclassing and using PostThread Message. http://msdn.microsoft.com/en-us/library/windows/desktop/ms644946%28v=vs.85%29.aspx
4. Build my own Accept loop to process all the WM_messages from the message queue.

I have identified a problem with the Clarion RTL where IF statements randomly fail the logic test on thread1. I only noticed this in a different app when I started programming more defensively to trap errors and started to log the variable values, so I dont subclass as much now.

[Bruce]

>> I have identified a problem with the Clarion RTL where IF statements randomly fail the logic test on thread1.

If you have an example of this I'd love to see it.

As to your question, I'd definitly not do 3 or 4. There's simply no reason to make it complicated when the Event system already exists.

Either of 1 or 2 would work, personally I'd tend towards Notify, because it can take a parameter. For example if the email processer decides to allocate an email to a thread, it can pass the id number of the email to the worker thread using the Notify command.

You will need a Queue though to manage the threads. ie Instead of a "fixed number" of threads, I'd just make a queue. As each thread starts it adds itself to the queue, when it ends, it removes itself. (Obviously the queue is managed by a critical section.) Also in the queue could be the "status" - as in whether it is busy sending an email, and so on.

The nice part about this approach is that it scales very well. Threads can be started, or stopped, "on the fly" (while the processor is busy dishing out work etc.)

Cheers
Bruce

[useless]

>>If you have an example of this I'd love to see it.
I'll build an example into this example app for you to see.

>>You will need a Queue though to manage the threads. ie Instead of a "fixed number" of threads, I'd just make a queue. As >>each thread starts it adds itself to the queue, when it ends, it removes itself. (Obviously the queue is managed by a critical >>section.) Also in the queue could be the "status" - as in whether it is busy sending an email, and so on.

>>The nice part about this approach is that it scales very well.

There shouldnt really be any difference between the datatype used beit a Long Array or a Queue, but I'll do a queue for this exercise.

[useless]

Attached is the latest example which uses no timers, it just posts events and notification but it still crashes and the GPF report still cites the Clarion runtime and NT as the problem.

Sleep(1000) is used in the ProcessEmail procedure which is the procedure which adds all the emailID's to the global queue that holds what threads are running.

Some observations, despite putting a "Do Nothing" button on the sendscheduledemail window to make sure at "least one event through the ACCEPT loop" had occurred as per your previous post, the use of notifying events to threads is much quicker than my previous example which used random periods of time to put a thread to sleep, if you run it through your capesoft profiler you'll see just how much quicker this is running which is increasing the risk of the threads/windows being started too quickly, something we know should be avoided and my 1st example dealt with!

Anyway, it still crashes when the emails start sending, so I could put a random sleep in place for each thread but then I am back to one of my previous questions which is how long does the SendEmail class need before sending the next thread? If you remember, my 1st example also put each thread to sleep for a random period of time to ensure at least the "send email engine" had started up.

As there is no timers involved in this example and you will see it works as per your suggestions, what next do you suggest to stop this from crashing when sending an email?


[attachment deleted by admin]

[Bruce]

Hi Richard,

I've taken your example, and modified it a bit. The attached app is in Clarion 8 format, I hope that's ok. It should be easy enough to duplicate the procedures in C6 if you need to.

a) I added a key to the email file.

b) I dropped the "Process" window (which was opening and closing a lot) because having a window open and close over and over like that is gonna cause you focus problems and so on. Rather I have a separate Window procedure that just sits there, passing emails through to the threads as needed.

c) I made a new procedure to start all the email threads, really just so that the code is all in one place, and nice and clean. As each thread starts it notifies this procedure which then starts the next one. When all are opened, it closes. You can call this procedure multiple times though if you wanted to increase the value in GLO:EmailThreads

d) The only use of the critical section is around the Queue. The rest of the global variables are longs, and are of the "write once, read multiple times" variety, so don't need a critical section.

e) If there are no emails to send, the ProcessEmails procedure starts a 15 second timer, at which point it checks the Emails file again. You could obviously make this timer much longer.

I hope the code is reasonably straight-forward to follow - it seems to be working ok here, no duplicates or crashes.

Cheers
Bruce


[attachment deleted by admin]

[useless]

I dont have C8 here so could you attach the Clw files for me to look at the code instead thanks?

[Bruce]

Here's the C6 version


[attachment deleted by admin]

[useless]

Thanks, I'll check it out.

[useless]

Just compiled and it GPF's when opening the sendscheduledemail windows still citing the clarion RTL and NT dll in the GPF report.

Any other suggestions?

[Bruce]

I saw that in C6 on the first compile as well. But not on any subsequent compiles.
didn't happen on C8 at all.

What you can try is just recompiling the exe again - perhaps with different debug settings to force it to regen all the OBJ's.

cheers
Bruce

[useless]

Did that and more changes yesterday so the OBJ's have been regenerated but still getting the same gpf.

[useless]

I switched on the logging in Nettalk so I could trace the nettalk debugview calls and have tracked the GPF down to the 2nd time the "[192] [NetDLL] [19] CallBackWindowSrc() : Being called - will open (NetMainWindow)" is called in quick succession and seems to tie in with the GPF reports. I've done this a few times and every time the Nettalk CallBackWindowSrc() pops up in the debugview in quick succession the GPF strikes.

Is this CallBackWindowSrc() used elsewhere in Nettalk?

[useless]

Any progress with this?

I'm also having problems with your other forum, my posts are not appearing if I make one to a thread, but if I make two then then both appear, then they dissappear again.

I've trying to progress these problems as well as I have customers chewing me off a strip and whilst I am no longer experiencing the problem where my userid and password was no longer recognised, my subsequent posts are still dissappearing when I try to follow these up.

http://clarion.capesoft.com/Entries?Topic=NWOKSLKL
http://clarion.capesoft.com/Entries?Topic=SLUPBUSK
http://clarion.capesoft.com/Entries?Topic=NYOSNCNL