cURL - FileUpload to a NT-(SOAP)Server

[Wolfgang Orth]

Hello Bruce and all others,

the plan is to automatically upload files via cURL to a Nettalk-Webserver. NT v10.25 with StringTheory 3

From one of the previous webinars I understood, that a SOAP-Server would suit best for this. However, I do not get it going. :-(

I see the incoming request, but the incoming data are completly empty. In the .PrimeParameter-Method I placed
dbgView(p_web.GetValue('IncomingFile'))
to get shown the contents of the incoming data. In my test I send a TXT.

First I tried
curl -T "g:\cURL\manual.txt" http://192.168.178.113:88/FileUpload
which appears in the logging like


PUT /FileUpload HTTP/1.1
Host: 192.168.178.113:88
User-Agent: curl/7.55.1
Accept: */*
Content-Length: 12206
Expect: 100-continue

This is the Manual bla bla bla...
.... and here comes the text of the sent file.

Then I tried (beside many variations)
curl -F "manual=@g:\cURL\manual.txt;type=multipart/form-data" http://192.168.178.113:88/FileUpload
is in the logging of the server like

POST /FileUpload HTTP/1.1
Host: 192.168.178.113:88
User-Agent: curl/7.55.1
Accept: */*
Content-Length: 12455
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------cb39bf6890b52e75

--------------------------cb39bf6890b52e75
Content-Disposition: form-data; name="manual"; filename="manual.txt"
Content-Type: multipart/form-data

This is the Manual bla bla bla...
etc etc etc

So, its in there, but not in p_web.GetValue('IncomingFile') and therefore my attempt to save the file in the .ServiceMethod
IncomingFile.SaveFile(ThisProgram.EXEPath() & 'web\uploads\mimimi.txt')
results in an empty file "mimimi.txt"

DebugView returns this:

*** excerpt copied from DebugView ****

[st] [netTalk][thread=3] FileUpload START Event=
(in this line contents ought to appear from dbgView() in .PrimeParameters, but that string is empty)
[st] [netTalk][thread=3] Incoming Parameters Format: xml=0 ; json=0
[st] [netTalk][thread=3] IncomingFile =  .....

*** /excerpt copied from DebugView ****

What am I missing?

Or should I better tinker a FORM for such upload? Maybe because POST expects a FORM? Which then would lead to the question, how a FORM has to look like. But one step after the other. And there seem to exist some FORMs in the examples.

Thanks in advance!
Wolfgang

[Matthew51]

Nether of those are SOAP Packets. One would look something like:

Code: [Select]

POST /FileUpload HTTP/1.1
Host: 192.168.178.113:88
User-Agent: curl/7.55.1
Accept: */*
Content-Length: 12455
Content-Type: text/xml
SOAPAction: http://192.168.178.113:88/FileUpload

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Body>
    <FileUpload xmlns="FileService">
      <Manual>Bla Bla Bla</Manual>
    </FileUpload>
  </soap:Body>
</soap:Envelope>
Since you are only sending one piece of data Making a whole web service is a bit overkill. The simplest method would be to pair your first attemp with a NetWebSource procedure. You should be able to access the set packet in the ProcessedCode embed stored in p_web.RequestData.DataString. All that's left is to remove the http header. This can be done by looking for the first <13,10,13,10> and putting everything after that into a new string. NetTalk may have just the contents stored somewhere, but I can't find it.

You're second attempt looks like it would work with a NetWebForm. It may be a better option as a "proper" way to upload a file. But that would take some experimenting, and working through NetTalk build in Security.

[Bruce]

Hi Matthew,

>>  The simplest method would be to pair your first attemp with a NetWebSource procedure.

you can't call NetWebSources directly, so this won't work.

Using a NetWebServiceMethod procedure is actually trivial, and the easiest approach.
Wolfgang must just use a FILE parameter, and post it with the correct fieldname.
(See UserGroup webinar #250)

You are right though - it's not SOAP - it's just a simple WebServiceMethod accepting www-form-encoding (which is what CURL will send).

the best part is that if you use a NetWebServiceMethod it'll come with a CURL example already in the docs for you. !

cheers
Bruce

[Wolfgang Orth]

Matthew,

using a NetServiceMethod is actually dead simple. If you do not the mistakes I did. ;-)

Once Bruce pointed me to the missing bits, it worked instantly!

Yet I ran into another problem, see my other post.

Thanks for your input!

[Wolfgang Orth]

the best part is that if you use a NetWebServiceMethod it'll come with a CURL example already in the docs for you. !

Hmmm, where is that?

I did a grep and all I found was the Multisite- and PHP-example. One fined was calcUrl, the other was php_curl.dll.

Anyway, I got it working instantly, once I followed you advice, to set the type to FILE and use the proper fieldname. Works like a charm! Everything from TXT to ZIP or JPG got send to and stored on the server.

However....

I expanded the cURL-commandline with a second parameter, a Description-field about the file, that gets uploaded.

curl -F "IncomingFile=@g:\cURL\yaddayadda.txt;type=multipart/form-data" -F "Description=Yadda Yadda" http://192.168.178.113:88/FileUpload

Still all perfect! Except the fact, that if the contents of "Description" or the name of the "IncomingFile" contains special characters like german umlauts as äöü or ÄÖÜ or ß, which is quite probably to happen. Well, I am a bit confused about what happens when, because it all looks so weird to me. I took some screenshots to explain, what happens when.

The first image is the cURL command itself. I created a TXT with the name rübärö.txt and take just that for the "Description" as well, to make things obious.

The second image shows the incoming POST in the server. The filename has been received correctly (while the contents is displayed wrong, ought to be "kurz und bündig!", but finally gets stored correct, both filename and contents, so no problem here). The contents of "Description", which is just as the filename itself, is displayed correct.

When the file is stored successfully (or not), I return the result to the client. For this I extract the filename, using StringTheory .FileNameOnly() and take this + the Description, stuff it into a variable and send this variable back. As seen in image three. I use the output into DebugView to display this. Cunningly the name of the file is correct here, while assigning to the variable. The description is already broken. All umluats are turned to ?. But it gets weirder! The next line, which starts with [st], comes directly from StringTheory. (I have checked debunggin on in the Template.) These four lines are actually what gets sent, and the umlauts are changed to some higgeldy-piggeldy, but very different to the ? in the Description.

But it still gets better. Image four shows the incoming on the client-side, where it has turned into gobbledygoo.

Something is going on here, but I do not what and where. And because I do not believe in sorcery, there mut be a reason. A very well hidden reason. I have some guesses, but those guesses I tried were all proven wrong.

If it would be helpful I could pass this APP in for the next webinar.

Thanks for all your patience
Wolfgang

[Jane]

the best part is that if you use a NetWebServiceMethod it'll come with a CURL example already in the docs for you. !

Hmmm, where is that?

I did a grep and all I found was the Multisite- and PHP-example. One fined was calcUrl, the other was php_curl.dll.

In the generated documentation for the service ??

[Wolfgang Orth]

In the generated documentation for the service ??

Ahhh, yes, maybe this is what Bruce meant!

However, I am locked in Nettalk 9 right now, so I did not notice that. I will inspect this an report later.

Thanks, Jane!