transfer-encoding: chunked

[Djordje Radovanovic]

I made application as WebServer to receive some xml data in aes-256 encrypted mode. This application decrypt this data and store it in database. Everything works well when I am sending data from my application. WebServer receive this data and do just it meant to do, but (there is always some but) when I receive same data from HW which pack XML in a same way (aes-256) I have problem.

transfer-encoding is chunked.

This mean that I receive data in small packets and after last packet I have whole message that I could decrypt.

My server receive only the first part of message with header. Header has all elements I need to recognize it but only difference is that I have attribute transfer-encoding: chunked.

My question is:
How can I receive the rest of message when I recognize this attribute in a header?

Best regards,

Djordje Radovanovic

[Bruce]

Hi Djordje,

which build are you on? All "recent" builds (certainly from the NT7 days) support incoming "chunked" transfer-encoding.
So there should be nothing specific for you to do, the "complete request" will be passed to WebHandler as normal.

Cheers
Bruce

[Djordje Radovanovic]

Hi Bruce, I am on 8.10 NT. As I learn from your post that chunked is supported I dig into my code to understand why I am not receiving message.
Well, webserver detect header and post message details into log queue but request never reach my web page.

This is from log queue:
POST /report HTTP/1.1
Host: 188.2.94.2:82
Connection: keep-alive
Content-Type: text/xml
transfer-encoding: chunked
TSS: 1.00
TSN: SDC001000670
TMM: 2301
TSW: 2.1302.2.8
Content-Encoding: aes-256

and that is just what I need. But report page has never been hit.
I walk through debugger and found that in a NetWeb.clw Process method WholeRequestFound never change state from false to true and I believe that this is the reason why I am not receiving this request.

Is there something I can do to solve this problem?

Best regards,

Djordje Radovanovic

[Bruce]

Hi Djordje,

ok, you'll need to do a bit of debugging to see what's going on. The key will be in the NetWeb.clw file, .Process method, ParseIncoming routine.

You should inspect the last 9 chars of self._ConnectionDataQueue.Data there to see what the end of the request looks like. it should end with the ascii values;
13 10 13 10 48 13 10 13 10

If you add a trace statement there - something like;
x# = self._ConnectionDataQueue.DataLen
if x# > 9
  self.trace( val(self._ConnectionDataQueue.Data[x#-8]) & '  ' & self._ConnectionDataQueue.Data[x#-7]) & '  ' &self._ConnectionDataQueue.Data[x#-6]) & '  ' &self._ConnectionDataQueue.Data[x#-5]) & '  ' &self._ConnectionDataQueue.Data[x#-4]) & '  ' &self._ConnectionDataQueue.Data[x#-3]) & '  ' &self._ConnectionDataQueue.Data[x#-2]) & '  ' &self._ConnectionDataQueue.Data[x#-1]) & '  ' &self._ConnectionDataQueue.Data[x#-0]) )
end

An incoming "chunked" request should go through this routine a few times (one for each incoming packet) but once it is "finished" it will be interesting to see what the final 9 characters are set to.

background:
A POST appears in the log twice - once when the header has arrived, and a second time once the data has arrived. If it only appears once then it "never got to the end of the data".

background:
A normal POST sends a "content-length" so we know when we've got the end of the packet. Chunked sends in multiple packets, with each packet containing the length of that packet. It ends with a packet of 0 length.

Cheers
Bruce

[Djordje Radovanovic]

Hi Bruce,

One <13,10> is too much.

  if instring ('<13,10>0<13,10><13,10>', self._ConnectionDataQueue.Data[1 : self._ConnectionDataQueue.DataLen],1,1)
    WholeRequestFound = TRUE
  end

best regards,

Djole

[Djordje Radovanovic]

Hi Bruce,

just to mention it is not tweaked in this 8.11 so it stays for tweaking in a new version.

Best regards,

Djordje Radovanovic

[Djordje Radovanovic]

I downloaded yesterday and installed your new version  .

Obviously, when I compare last download and this new one there is no difference but this is my screenshot from this morning.

So, as you often say

Cheers

Djordje Radovanovic

[attachment deleted by admin]

[Bruce]

That's cause I updated the NetTalk Lite build yesterday, and accidentally told it to upload the docs - which are in an Interim state....

[Djordje Radovanovic]

Bruce this is not fer. You deleted my last message which was my message to promote me from Jr. member to Full member.

Please do not delete this one 


Best regards,

Djole

[Bruce]

Hi Djole,

  - I don't delete them to keep your posts down, but if the thread gets too long this forum software seems to "lock the participants out" of the thread. I can only assume it's some sort of "flame war" defense, but it's a real pain. So if I see a thread getting long, I'll remove some of the "non important" posts in order to keep it alive.

So, don't worry, it's not personal 

(I deleted one of my own from this thread to create this one )

cheers
Bruce