Limit on HTTP response length

[Thys]

Bruce,

In my integration tool the processors can return HTTP/XML as a response. It looks as if the limit on the response size is 4000H. Can this be changed to return larger strings?

Thys

[Thys]

Bruce,

Looks like it's the value of NET:MaxBinData that is 16384 (4000h). Is it safe to change this to be much larger?

Thys

[Bruce]

Hi Thys,

>> Looks like it's the value of NET:MaxBinData that is 16384 (4000h). Is it safe to change this to be much larger?

no, you should not just change the size of this equate.

>> In my integration tool the processors can return HTTP/XML as a response. It looks as if the limit on the response size is 4000H. Can this be changed to return larger strings?

you don't mention the context of what you are doing. ie are you using a NetSimple object? A webClient object? a WebServer object?
You can send strings much larger than NET:MaxBinData but I need to know more about what you are doing to be more specific.

cheers
Bruce

[Thys]

Bruce,

The app is a WebServer app serving XML. It receives XML and returns XML. The limitations I found are:

Request - parameter value sizes are limited to 256 characters (Net:ValueSize)
Response - total response size limited to 4000H in the PACKET variable (NET:MaxBinData)

Changing these values in NetAll.inc won't help because the NetTalk dll's won't know about the changed sizes. I'm trying to find workarounds in my app (like splitting request parameters and limiting response sizes), but even if I find a solution now the size issue will hit again in future. We did solve the response problem by returning blocks of data. But the request parameter is a bigger issue that I can't get a workaround for now.

Thanks
Thys

[Bruce]

I think you want to post an example Thys. Neither of these limits are correct, so if you are seeing a limit it's not in the place you are looking. A simple example of what you are doing would be helpful in pointing you in the correct direction.

cheers
Bruce

[Thys]

Bruce,

It's difficult to post an example from the app that I have. Here is more detail of each:

Request: I use p_web._LocalDataQueue.Value to read the parameters, which is defined to have the size of Net:ValueSize, set to 256 in NetAll.inc.
Response: I use the packet variable which is created as String(NET:MaxBinData) in the template where NET:MaxBinData has the length of 16384 (4000H) in NetAll.inc.

Thys

[Bruce]

Hi Thys,

>> p_web._LocalDataQueue.Value to read the parameters, which is defined to have the size of Net:ValueSize

short answer: It's not a good idea to try and read the value from the queue directly. If I were you I'd loop through the queue to get the parameter names (assuming you don't know what they are) then use the GetValue method to get the (unlimited) string value.

long answer: the NetWebServerLocalDataQueueType contains a number of fields, including the Value field (which is used for short values) and the ExtValue field, which is used for long values. Most values are short, so for performance reasons it's not necessary to use a variable length string there. However long ones are parsed into ExtValue. The GetValue method shows you the way to read a value in either case, which is why it's easiest for you just to use that method when reading a value.

>> packet variable which is created as String(NET:MaxBinData) in the template

you don't mention the context, but I'm assuming it's a NetWebPage. You'll notice in that procedure is a routine called SendPacket.

You can call this routine multiple times. So if you have a large string that needs sending you can just do a
   packet = 'something'
  do sendPacket
  packet = 'something more'
  do SendPacket
and so on.

Incidentally now that StingTheory is a given, the goal is to move these "packet" variables to being StringTheory objects, which in turn simplifies this sort of thing.

Cheers
Bruce

[Thys]

Thanks Bruce. I'll try it.

[Thys]

Bruce,

I made the changes and they are working. Because of our workarounds, I haven't yet tested a large response yet. But I believe it will work. I now call SendPacket for every 1000 characters. Is it more effective to call the routine for shorter or for longer PACKET strings - or does it matter?

Thys

[Bruce]

Hi Thys,

It doesn't matter. Assuming you have on the NT7 switch to compress dynamic content (which makes a big performance difference if the thing at the other end can accept compressed text) it all gets sent together anyway.

cheers
Bruce
 

[Thys]

We're not on NT7 yet - we have to finish this project first before I change the tooling.

I like the idea of compression. On a similar topic - what I also do need to do is strong text encryption/decryption on the server and the client. This is to ensure that the comms is done using dynamic keys and that server/clients are who they say they are. I am looking to use an algorithm called Alpha Crypt - purely because it is written in JavaScript. Do you have any other suggestions?

Thys

[Bruce]

Is your client a browser? If that's the case then the _only_ strong encryption is SSL. It's also the easiest. It's also the standard. If you like you can issue a private key to the browser, to identify the user.

Encryption in JavaScript is simply not, because everything (including the key) is exposed.

[Thys]

Bruce,

No, the client will rarely be a browser in this solution. Although our apps are all written in Clarion, many of the dynamic functions we have are written in JavaScript. So we call the NetTalk app (using HTTP) from our Clarion front-end app via the JavaScript-enabled procedures. These are dynamic procedures - in short, they don't exist when we compile the Clarion application but the framework in the Clarion code exists for developers to add custom procedures and reports.

I chose HTTP instead of native TCP/IP because of the wide possible use of HTTP. But I know TCP/IP is much faster and I might consider providing a TCP/IP interface for a next version of our integration tool.

Thys