Memory issue - Solved?

[Graham]

Hi All

NT 11.35, StringTheory 3.21

Books Example Web71 - In the browser, app works fine...

Select WebServer App window:

 - click on a GET item in log - no problem
 - click on a POST item a few times, memory use goes thru the roof

See video: https://youtu.be/oPaN4feq3v8

Anyone else see this...

Thanks
Graham

[Bruce]

yes, I've seen this, but I don't think it's a NetTalk thing.
I think it's something to do with you clicking on the WebServer window itself.
Perhaps do the test again, but _don't_ click on the webserver window.

If that narrows down the scope of the issue, we can move to the next step...

Bruce

[Graham]

Hi Bruce

Correct, I am clicking on the WebServer window - I would like to see the POST data

Specifically, it only happens when I click on a POST log entry where the _thread_ is 0 - believe this is just the POST Header.

In Netweb.clw, around line 686

            self.AddLog(self._ConnectionDataQueue.Data[1:x],self.packet.FromIP)

Problem goes away if I change this line with this:  "[1:x-1]"

            self.AddLog(self._ConnectionDataQueue.Data[1:x-1],self.packet.FromIP)

Not sure if that will break something further along - doesn't appear to...?

Have no idea what is at the end of the string to cause it to lose it's marbles like that

Cheers
Graham

[Bruce]

thanks Graham - I've made the tweak.
It'll exclude the <13> which was in position x, which should do no harm.
(It should do no harm being there either, but hey, if removing it helps...)

cheers
Bruce

[jslarve]

Good find, Graham. I wonder if that <13> was confusing the ?Web:LastPost TEXT control on the WebServer window.

Will check that out tomorrow.

I was dealing with this for several hours today, but didn't discover the actual cause.

https://www.screencast.com/t/0j7Xm8Pb2R

When I commented out the one line, it stopped locking up. But I didn't take note of the memory usage.

If Field() = ?LogQueue and Event() = Event:NewSelection
      Get(LogQueue,Choice(?LogQueue))
      If ErrorCode() = 0
        Case Upper(Sub(LogQueue.Desc,1,3))
        Of 'POS'
          web:LastPost = LogQueue.Desc !This line locks up.
        Of 'GET'
          web:LastGet = LogQueue.Desc
        Else
          web:LastGet = LogQueue.Desc
        End
        Display()
      End
    End

thanks Graham - I've made the tweak.
It'll exclude the <13> which was in position x, which should do no harm.
(It should do no harm being there either, but hey, if removing it helps...)

cheers
Bruce

[Graham]

Hi Jeff

Yip, that's what I was running into and it only happens if you touch the POST with thread number 0 which is the POST Header

App appears to lock up but if you leave it long enough, it comes back (sort of) but Ram used has gone thru the roof and impossible to use after that.

Think its actually a <10> - this is the code:

        x = instring ('<13,10><13,10>', self._ConnectionDataQueue.Data[1 : self._ConnectionDataQueue.DataLen], 1, pos)  ! Look for request terminating character
        if x > 0
          ! Found Header End Marker
          self._ConnectionDataQueue.HeaderEndPosition = x + 3

Cheers
Graham

[Jane]

Thanks for figuring this out, guys!

My non-analytical mind just noticed that sometimes clicking in that list box invoked the spinning cursor symbol of death, so I've just steered away completely from doing that for the past year or more.  Never realized it was specific to POSTs.

Good catch!

Cheers,

Jane

[jslarve]

Hi Graham -

On my machine it's a lone <13> (0dh) without the <10>.

I tried saving the offending string to a file and duplicating the error by assigning to a TEXT control in a test app, but it didn't exhibit the offending behavior. So I guess there's more to the circumstance than just the <13>. But I am glad you figured out how to get around it.

Hi Jeff

Yip, that's what I was running into and it only happens if you touch the POST with thread number 0 which is the POST Header

App appears to lock up but if you leave it long enough, it comes back (sort of) but Ram used has gone thru the roof and impossible to use after that.

Think its actually a <10> - this is the code:

        x = instring ('<13,10><13,10>', self._ConnectionDataQueue.Data[1 : self._ConnectionDataQueue.DataLen], 1, pos)  ! Look for request terminating character
        if x > 0
          ! Found Header End Marker
          self._ConnectionDataQueue.HeaderEndPosition = x + 3

Cheers
Graham

[jslarve]

Until the next NT build I went with this, just in case x might equal 1 some time.

        IF x > 1
                self.AddLog(self._ConnectionDataQueue.Data[1:x-1],self.packet.FromIP)
        ELSE
                self.AddLog(self._ConnectionDataQueue.Data[1:x],self.packet.FromIP)
        END

Hi Bruce

Correct, I am clicking on the WebServer window - I would like to see the POST data

Specifically, it only happens when I click on a POST log entry where the _thread_ is 0 - believe this is just the POST Header.

In Netweb.clw, around line 686

            self.AddLog(self._ConnectionDataQueue.Data[1:x],self.packet.FromIP)

Problem goes away if I change this line with this:  "[1:x-1]"

            self.AddLog(self._ConnectionDataQueue.Data[1:x-1],self.packet.FromIP)

Not sure if that will break something further along - doesn't appear to...?

Have no idea what is at the end of the string to cause it to lose it's marbles like that

Cheers
Graham

[DonRidley]

Some feedback on this topic...

I had been observing some weird memory use issues for quite a while.  My multi-site host apps would steadily use more and more memory until I would have to restart the apps.  I usually had to do this every 2 or 3 days.   Went through the app and optimized everything I thought could contribute to the issue.

I added Jeff's code and the memory issue has disappeared.  I tested the app with the web strain app and the memory was actually LESS than before the strain test!! 

After several days of continuous use, my apps have maintained a consistent and expected memory use.

Thank you Graham for finding this!!  It has been a thorn in my side for quite a while!

Don

[Graham]

Hi Don

Good to hear your issue resolved - just happy to have found a solution for my own sanity <g>

A few months ago, I posted re: Connections growing issue I was seeing - http://www.nettalkcentral.com/forum/index.php?topic=8434.msg34352#msg34352

Updated this app with this change and so far, so good.

Graham