Simple-Client/Server using WholePacket

[Wolfgang Orth]

Hello Bruce and all!

Using .SEND() with a Simple-Client/Server combo limits the packet size to 16K.

In this case I have to split larger amounts of data into those 16K pieces and send one after the after during a LOOP.

According to the manual, using WholePacket permits sending packets of any size in one rush.

This is at least so far my understanding from what I read.

My question now: can I use this WholePacket stuff only when I write both Clinet and Server? When one of them is not under my control, then I have to use .SEND()?

Or can I use WholePacket with foreign apps also, _except_ I use the .WholePacketUseLengthField=1 Property?

Obviously I misunderstand the manual at this point.

I tend to believe that this WholePacket thingie is unique to Nettalk, and .WholePacketUseLengthField=1 is to make it even easier to use (well, once you understood it...).

My intention is to send binary files. Stuffing a PDF into a StringTheory-object, do a .Base64Encode(), then send it as a whole packet and .Base64Decode() it again seems cool to me. Just let the user pick a file and hit the Send-Button.

Is such a Simple-C/S combo reliable for this task, or would a Web-C/S combo be the better choice?

Thanks for enlightning,
Wolfgang

[Bruce]

Hi Wolfgang,

<< My question now: can I use this WholePacket stuff only when I write both Clinet and Server? When one of them is not under my control, then I have to use .SEND()?

correct, (in almost all cases).

>> I tend to believe that this WholePacket thingie is unique to Nettalk, and .WholePacketUseLengthField=1 is to make it even easier to use.

It is unique in the sense that it's not based on any internet protocol. However it's not unique in the sense that we're not the only ones who do it. What the WholePacket does is _prepend_ your stuff with a single 4 byte long. This long is, if you like, a "header". In this way the receiver can know how many bytes to expect.

It turns out that at least some other folk are doing the same thing, because recently I added support for changing the "endianess" of the long - ie the byte-order of the long. So remote systems that send stuff to you, where the first 4 bytes are the length, and systems that receive from you, expecting the first 4 bytes to be the length, are essentially compatible with our WholePacket approach. Chances are though you _won't_ be talking to a system like this, in which case you go back to a simple SEND.

<< ...do a .Base64Encode()

of course, it all depends on the system you are talking to, but there's no real reason to do a Base64encode. The internet does binary just fine.

>> Is such a Simple-C/S combo reliable for this task, or would a Web-C/S combo be the better choice?

For sending and receiving files, I'd personally use a WebServer / WebClient combination. This includes a HTTP header with all the information, and is a standard internet protocol. Plus it supports Binary, so will be smaller than base64encoded (by about a third.)

Cheers
Bruce

[Wolfgang Orth]

Thanks for your reply, Bruce, but I am still not...... what is the opposite of confused? Effused? Suffused?

WholePacket always prepends that LONG when .WholePacketUseLengthField=1.

Will a packet look like this:


123456789<CRLF>
Here is the contents of the packet, whatever that packet contains.



Or like this:


123456789Here is the contents of the packet, whatever that packet contains.



I have to cut off either 6 (the 4-byte LONG plus <13><10>) or 4 byte (just the LONG) in front.

With .WholePacketUseLengthField=0 there is nothing to cut.....

... wouldn't that be easier to handle?



  << ...do a .Base64Encode()

  > of course, it all depends on the system you are talking to, but there's no real reason
  > to do a Base64encode. The internet does binary just fine.
 
hmmm, in my previous tests the binary files seem to not arrive completely. On the other hand, the files weren't the same after I tried using Encode/Decode. I sent JPGs for testing and image-viewer complaint about "invalid file"<headscratch>

  >> Is such a Simple-C/S combo reliable for this task, or would a Web-C/S combo be the better choice?

  > For sending and receiving files, I'd personally use a WebServer / WebClient combination. 
  > This includes a HTTP header with all the information, and is a standard internet protocol.
  > Plus it supports Binary, so will be smaller than base64encoded (by about a third.)

Again I am confused.... Beside the fact that I was not able to send unaltered binaries, but having a prepended HTTP-header inside my packet would require to remove that header, before this packet can be stored as a binary (PDF) on the server-side.


POST / HTTP/1.0
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, */*
Accept-Language: en
User-Agent: Mozilla/5.0 .....
Host: 127.0.0.1:88
Content-Length: 52
Connection: Keep-Alive

§$%&/&%$§$$&% .. more binary code....


My basic idea was this:

 - Client says: Server, wake up, here comes 2012-04-15_blabla.PDF

 - Server says: okay, I listen now

 - Client says: <BIG_PACKET_SENDING>

 - Server: stringtheory.SaveFile('2012-04-15_blabla.PDF', Append)

and at the end I have a PDF (or whatever) more on the server, but did not work that way I thought.


I will do some more testing....... and let you know about my results.

[Bruce]

Hi Wolfgang,

Ok, let's cover the web server side first.

With the HTTP approach, because you are using WebServer and WebClient classes, not NetSimple, the code you have to write becomes a lot, lot, simpler.

On the sending side you have something like;

client.SetValue('fieldname',loc:filename,true)
client.post('someurl')

On the server side you need, well, _nothing_. Any server with just a simple bog standard WebServer and WebHandler procedure would store the incoming file away for you - in the uploads folder. You could literally run the "ErrorPage" example and it would work.

So, as I say, the WebServer route makes for _much_ less code for you. Plus, it has some advantages, like being multi-threaded and so on.

For the client to get a file from the server, it's just as easy;

client.Fetch('someurl/uploads/filename')
and in .PageReceived method;
client.SavePage('somename')

HTTP is exactly _made_ for file transfer, and does all the hard work for you.

Back to NetSimple;
If you are using WholePacket then you should jsut set the property at both ends, and forget about it. The number will get added, and removed for you. It takes 4 bytes, not 6 - but chances are you're reaking it by removing they byts, as they've already been removed for you.

Cheers
Bruce

[Wolfgang Orth]

client.SetValue('fieldname',loc:filename,true)
client.post('someurl')
...
client.Fetch('someurl/uploads/filename')
and in .PageReceived method;
client.SavePage('somename')

Two lines each - I guess it can't be done with less!

I added an empty string as a second parameter to client.Post() and it works like a charm!

Thanks for pointing me.

Due to the asynchronous character of the web, there is no feedback when the file has arrived completely from the client to the server, right? For this particular task I recently work on, sending 100 KB large PDF, its okay to guestimate when I can close the client automatically (will be an invisible background job).

It even works with 100+ MB huge files, albeit it would be difficult then to predict when to close the program. I also tried to send 600 MB large files, In that case I just saw a spike in the network traffic curve, but nothing thereafter.

If you are using WholePacket then you should jsut set the property at both ends, and forget about it. The number will get added, and removed for you. It takes 4 bytes, not 6 - but chances are you're reaking it by removing they byts, as they've already been removed for you.

Good to know that this long will be not part of the saved files. I will also write a program that way, because I think I can send a feedback then back and forth.

Anyway, Nettalk has again proven to be a tool that makes work so much easier!

[Bruce]

>> its okay to guestimate when I can close the client automatically (will be an invisible background job).

absolutly not! guessing is never a good idea.

When the page has been sent the .PageReceived method will be called. You should definitly not close the window before then. (*)

Indeed, if the window was gonna be completely automatic, I'd add post(event:closewindow) to the .PageReceived method, and the .ErrorTrap method.

(*) A web client is always "asking for something", even if it is doing a POST. You may not care about the result that the server sends you, but it's still the right way to know the request has completed. If, for example, you post to the ErrorPage example, chances are your URL won't resolve to anything, so you get a 404 response. But the key is, you get a response.

Of course, if you use a valid url, like the name of a netWebPage on the serve, then the server can send back any response you like, and you may choose to take action on that.

Cheers
Bruce