Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
1
Web Server - Ask For Help / Re: Netwebmethod/JSON and international characters
« on: May 08, 2023, 09:22:07 AM »
Hi Bruce,
number 1 is het good answer
(I looked in jfiles.clw to find something related to codepages or so, but I didn't realize it was all stringtheory behind the scenes). The prop:charset is a nice way to get all my stringtheory objects in the right codepage.
Thanks!
number 1 is het good answer
(I looked in jfiles.clw to find something related to codepages or so, but I didn't realize it was all stringtheory behind the scenes). The prop:charset is a nice way to get all my stringtheory objects in the right codepage.
Thanks!
2
Web Server - Ask For Help / Re: Netwebmethod/JSON and international characters
« on: May 04, 2023, 03:12:49 AM »
Hi Bruce,
two additional questions (both performance related):
1 - jFiles apparently does the translations of the "/u...." stuff somewhere when loading/parsing the data (but assumes another codepage?). Is this configurable? So can I set the target codepage prior to loading and all will be processed in one go?
2 - is the SELF.rP_web.Getvalue('_postdata_') somewhere accessable directly as a stringtheory object?
TIA,
Ton
two additional questions (both performance related):
1 - jFiles apparently does the translations of the "/u...." stuff somewhere when loading/parsing the data (but assumes another codepage?). Is this configurable? So can I set the target codepage prior to loading and all will be processed in one go?
2 - is the SELF.rP_web.Getvalue('_postdata_') somewhere accessable directly as a stringtheory object?
TIA,
Ton
3
Web Server - Ask For Help / Re: Netwebmethod/JSON and international characters
« on: May 03, 2023, 05:39:09 AM »
Hi Bruce,
thanks, that worked and the codepage is correct too :-)
thanks, that worked and the codepage is correct too :-)
4
Web Server - Ask For Help / Netwebmethod/JSON and international characters
« on: May 02, 2023, 08:54:55 AM »
Hi all,
I'm receiving utf postdata via nettalk and probably need to convert somewhere in the process as I don't save the international characters correctly, escpecially the euro-sign (although special characters like
etc. work fine).
I receive the following (as a json string within quotes), which seem to me like correctly encoded:
"\u00e4\u00e0\u00e2\u00e9\u20ac"
The last one displays as "?" in my screens and should be the euro sign (but the first four get "translated" somehow correctly)
Currently I use following code to load the json (gtSt is effectively a stringtheory object) into my queues
gtSt.Setvalue(SELF.rP_web.Getvalue('_postdata_'))
json.start()
json.SetTagCase(jf:CaseAsIs)
json.Load(element, gtSt) ! Load From a StringTheory object
Where (and with what method) should I modify this string.
TIA,
Ton
I'm receiving utf postdata via nettalk and probably need to convert somewhere in the process as I don't save the international characters correctly, escpecially the euro-sign (although special characters like
etc. work fine). I receive the following (as a json string within quotes), which seem to me like correctly encoded:
"\u00e4\u00e0\u00e2\u00e9\u20ac"
The last one displays as "?" in my screens and should be the euro sign (but the first four get "translated" somehow correctly)
Currently I use following code to load the json (gtSt is effectively a stringtheory object) into my queues
gtSt.Setvalue(SELF.rP_web.Getvalue('_postdata_'))
json.start()
json.SetTagCase(jf:CaseAsIs)
json.Load(element, gtSt) ! Load From a StringTheory object
Where (and with what method) should I modify this string.
TIA,
Ton
5
Web Server - Ask For Help / NT 12.45 : regular crashes on (apparently) on the same code
« on: March 20, 2023, 09:03:40 AM »
I'm currently on NT 12.45 on several sites (used as an API-server), but my most havely used one is crashing a lot the last week. The crashing has been present since I upgraded to 12.45 (came from 9.15) last summer/fall, but not more than once every 6 weeks or so.
Currently traffic is exploding on this site (this always happens this period of the year, sometimes 5-10K requests per hour) and now it crashes 3-5 times a day and I notice that it's always the very same crash report that's generated (see attached GPFReport.txt)
I also attached a screenshot (see attached code.png) of the code (from version 12.45) it seems to point to line 1301 (but I'm not an expert in reading GPF logs ...)
I haven't upgraded to 12.55 as I could not find any fix related to this in de history docs (so to me it looks like more risks only ...)
Also problematic is that the "connectionchecker" (shipped with nettalk) I use doesn't seem to kill the exe most of the times (although it restarts a new instance ... but this new instance conflicts with the old instance).
Are there things I can do to make the connectionchecker work?
If you need additional info, let me know
Currently traffic is exploding on this site (this always happens this period of the year, sometimes 5-10K requests per hour) and now it crashes 3-5 times a day and I notice that it's always the very same crash report that's generated (see attached GPFReport.txt)
I also attached a screenshot (see attached code.png) of the code (from version 12.45) it seems to point to line 1301 (but I'm not an expert in reading GPF logs ...)
I haven't upgraded to 12.55 as I could not find any fix related to this in de history docs (so to me it looks like more risks only ...)
Also problematic is that the "connectionchecker" (shipped with nettalk) I use doesn't seem to kill the exe most of the times (although it restarts a new instance ... but this new instance conflicts with the old instance).
Are there things I can do to make the connectionchecker work?
If you need additional info, let me know
6
Web Server - Ask For Help / Re: (Nettalk) Webserver security?
« on: January 11, 2023, 12:48:36 PM »
Hi all,
thanks for your input and insights.
Today I dived into the security stuff a little deeper and concluded that I really know next to nothing about this and I probably need to keep it this way :-)
My client asked me for advice because they know even less than I do and they think they will be spending too much (I feel the same). But I'd better stay out of this.
Apart from DOS/DDOS attacks, I think the nettalk server is "pretty" safe on it's own and told the client this, if they want to feel more secure they can add wathever hardware/software they want or investigate further.
Thanks again!
thanks for your input and insights.
Today I dived into the security stuff a little deeper and concluded that I really know next to nothing about this and I probably need to keep it this way :-)
My client asked me for advice because they know even less than I do and they think they will be spending too much (I feel the same). But I'd better stay out of this.
Apart from DOS/DDOS attacks, I think the nettalk server is "pretty" safe on it's own and told the client this, if they want to feel more secure they can add wathever hardware/software they want or investigate further.
Thanks again!
7
Web Server - Ask For Help / (Nettalk) Webserver security?
« on: January 10, 2023, 02:35:54 PM »
Hi all,
I'm running the latest and greatest Nettalk for an api server (not a webserver, so only json in and out). Site runs secure (port 443 only) and tests as safe in for example ssllabs.com. I've read the nettalk docs on security and apart from Denial of Service attacks I can't see any vulnerabilities. SQL injection is covered (all clarion statements).
Now network folks want to install a fysical firewall (fortigate) and software called MalwareBytes to filter out "bad requests" and limit the requests only to certain ip-addresses (which is max. 25 in this case, but some don't have guaranteed fixed addresses).
Personally I think they are introducing a lot of costs for the client (fortigate) and quite a lot of hassle to keep the ip-addresses up to date (I know that I'm the one getting the first support calls when an ip-number has changed and traffic blocks ...)
But I've no real good arguments against these features. Problem is they don't trust the unknown Nettalk server ...
Does anybody have any insights on this matter or some source of info I can delve into to get me in a more knowledgeable state with regard to webservers and security.
Will a fortigate firewall for example be able to counter DoS attacks (and so be a valuable addition)?
TIA
I'm running the latest and greatest Nettalk for an api server (not a webserver, so only json in and out). Site runs secure (port 443 only) and tests as safe in for example ssllabs.com. I've read the nettalk docs on security and apart from Denial of Service attacks I can't see any vulnerabilities. SQL injection is covered (all clarion statements).
Now network folks want to install a fysical firewall (fortigate) and software called MalwareBytes to filter out "bad requests" and limit the requests only to certain ip-addresses (which is max. 25 in this case, but some don't have guaranteed fixed addresses).
Personally I think they are introducing a lot of costs for the client (fortigate) and quite a lot of hassle to keep the ip-addresses up to date (I know that I'm the one getting the first support calls when an ip-number has changed and traffic blocks ...)
But I've no real good arguments against these features. Problem is they don't trust the unknown Nettalk server ...
Does anybody have any insights on this matter or some source of info I can delve into to get me in a more knowledgeable state with regard to webservers and security.
Will a fortigate firewall for example be able to counter DoS attacks (and so be a valuable addition)?
TIA
8
Web Server - Ask For Help / NetWebClient : handling of general HTTP errors (like 402, 404, etc)
« on: October 24, 2022, 11:39:31 AM »
Hi All,
in het NetWebClient.PageReceived method I can parse the HTTP error from the header, but is there a property for this?
in het NetWebClient.PageReceived method I can parse the HTTP error from the header, but is there a property for this?
9
Web Server - Ask For Help / jFiles : "pretty" JSON?
« on: October 24, 2022, 11:24:24 AM »
Hi all,
I've this json that I receive and I can nicely parse this with jFiles. But sometimes I need the json in "pretty" format (for debugging etc.). Is there an option in JFiles to convert the json to pretty (and back to compact)?
Currently I move the raw data to an external editor and format it there, but maybe jFiles has this feature somewhere in it ...
TIA
I've this json that I receive and I can nicely parse this with jFiles. But sometimes I need the json in "pretty" format (for debugging etc.). Is there an option in JFiles to convert the json to pretty (and back to compact)?
Currently I move the raw data to an external editor and format it there, but maybe jFiles has this feature somewhere in it ...
TIA
10
Web Server - Ask For Help / Re: NetWebClient : sending "HTTPS" to an insecure server
« on: October 14, 2022, 03:36:19 AM »
Hi Bruce & Bill,
the woocommerce site is "secure", but the certificate is probably "rubbish".
Now I found out (with the demo), that I simply need to activate "DontVerifyRemoteCertificateWithCARoot" and I'm good to go
Thanks for pointing me to this option
regards,
Ton
the woocommerce site is "secure", but the certificate is probably "rubbish".
Now I found out (with the demo), that I simply need to activate "DontVerifyRemoteCertificateWithCARoot" and I'm good to go
Thanks for pointing me to this option
regards,
Ton
11
Web Server - Ask For Help / Re: NetWebClient : sending "HTTPS" to an insecure server
« on: October 12, 2022, 11:52:01 PM »
Hi Bruce,
(story of my life: me asking questions leading to even more questions :-) still learning to ask better questions )
This indeed is a locally installed wordpress/woocommerce site ONLY for development/testing purposes. There is a production site (secure!), but that one is filled with production data and I can access that environment nicely and without problems, but I tend to make lots of errors during development so I'd rather stay away from the production environment :-).
For some reason the woocommerce api requires the word "https" in the requests and I think it serves a (self created?) certificate now, but that gives me problems in all environments (clarion/nettalk : netsimple errorTrap : -69) because of "wrong" certificate.
PostMan gives me the option to send "https" and ignore the (validation of the) certificate/SSL part. I was looking for such a thing in Nettalk (NetWebClient) ...
I've tried installing a valid certificate with several wordpress plugins ("Really Simple SSL", "WP Encryption"), but they (for some reason) all fail ...
And I just want to program in Clarion/Nettalk :-)
regards,
Ton
(story of my life: me asking questions leading to even more questions :-) still learning to ask better questions )
This indeed is a locally installed wordpress/woocommerce site ONLY for development/testing purposes. There is a production site (secure!), but that one is filled with production data and I can access that environment nicely and without problems, but I tend to make lots of errors during development so I'd rather stay away from the production environment :-).
For some reason the woocommerce api requires the word "https" in the requests and I think it serves a (self created?) certificate now, but that gives me problems in all environments (clarion/nettalk : netsimple errorTrap : -69) because of "wrong" certificate.
PostMan gives me the option to send "https" and ignore the (validation of the) certificate/SSL part. I was looking for such a thing in Nettalk (NetWebClient) ...
I've tried installing a valid certificate with several wordpress plugins ("Really Simple SSL", "WP Encryption"), but they (for some reason) all fail ...
And I just want to program in Clarion/Nettalk :-)
regards,
Ton
12
Web Server - Ask For Help / NetWebClient : sending "HTTPS" to an insecure server
« on: October 12, 2022, 02:27:41 PM »
I've been strugling to get a locally installed wordpress/woocommerce api working. The site is local and I've not been able to get a certificate on the site working (tried several plugins at no avail).
I found out (using Postman) that the woocommerce api needs "https" in the GET in order to work, although the wordpress site is actually insecure. In Postman the GET with https will by default activate ssl, but you can optionally ignore the invalid certificate and so effictively work insecurely.
In Nettalk (NetWebClient, see "netwww.clw") I found out that commenting out the following line (no. 660) in the .fetch method
if lower(self._CommandURLSt.UrlProtocolOnly()) = 'https' then self.SSL = true.
will do the "trick" for me. Now I the api (at least for GET works).
So can I (via Nettalk) send the text "https" without actually making traffic secure in a clean manner somehow?
Or is there another way to tell netwebclient to ignore the ssl stuff (so effectively disabling SSL certificate verification)?
B.t.w.
If someone knows how to get the local site secure easily, that suggestion is welcome too
TIA
I found out (using Postman) that the woocommerce api needs "https" in the GET in order to work, although the wordpress site is actually insecure. In Postman the GET with https will by default activate ssl, but you can optionally ignore the invalid certificate and so effictively work insecurely.
In Nettalk (NetWebClient, see "netwww.clw") I found out that commenting out the following line (no. 660) in the .fetch method
if lower(self._CommandURLSt.UrlProtocolOnly()) = 'https' then self.SSL = true.
will do the "trick" for me. Now I the api (at least for GET works).
So can I (via Nettalk) send the text "https" without actually making traffic secure in a clean manner somehow?
Or is there another way to tell netwebclient to ignore the ssl stuff (so effectively disabling SSL certificate verification)?
B.t.w.
If someone knows how to get the local site secure easily, that suggestion is welcome too
TIA
13
Web Server - Ask For Help / Multiple instances of an API-server?
« on: April 07, 2021, 04:02:08 AM »
Hi All,
I'm running into several issues with a heavily used api-server (Clarion runtimes/driver not being totally thread safe and I actually need multiple database connections to the database server, while Clarion only allows one per exe). I sometimes use critical sections around my method/endpoints to accomplish what I need and the circumvent the threadsafety issues, but when there is a heavy workload, things are being served sequentially instead of concurrently.
So my plan was to start several instances of the exe/api-server (listening on different ports?) to accomplish this, but I probably need some "router" software to manage traffic.
Does anybody have any experience on this?
Is this a good plan?
Is there any documentation for this in the Nettalk docs?
Where to start looking :-)?
TIA
regards,
Ton
I'm running into several issues with a heavily used api-server (Clarion runtimes/driver not being totally thread safe and I actually need multiple database connections to the database server, while Clarion only allows one per exe). I sometimes use critical sections around my method/endpoints to accomplish what I need and the circumvent the threadsafety issues, but when there is a heavy workload, things are being served sequentially instead of concurrently.
So my plan was to start several instances of the exe/api-server (listening on different ports?) to accomplish this, but I probably need some "router" software to manage traffic.
Does anybody have any experience on this?
Is this a good plan?
Is there any documentation for this in the Nettalk docs?
Where to start looking :-)?
TIA
regards,
Ton
14
Web Server - Ask For Help / Re: StringTheory : MakeGuid not so unique?
« on: April 03, 2021, 08:02:21 AM »
The randomness indeed probably is dependant on when the server is actually started.
Out of curiousity:
- this "seed" is the thing that needs to be rather random (as each seed as a "fixed" array of numbers in it)?
- the s1 += chr(random(1,100)) is only to get a more random start in the array?
- effectively this could also be s1 = random(1,100), as the s1 is going nowhere?
Out of curiousity:
- this "seed" is the thing that needs to be rather random (as each seed as a "fixed" array of numbers in it)?
- the s1 += chr(random(1,100)) is only to get a more random start in the array?
- effectively this could also be s1 = random(1,100), as the s1 is going nowhere?
15
Web Server - Ask For Help / StringTheory : MakeGuid not so unique?
« on: April 02, 2021, 05:24:15 AM »
Hi All,
I'm posting this here is I ran into this when working with services in NetTalk. I created a webservice endpoint that needs a GUID for a record-ID and I ran some stress tests on this method. I noticed the backend sometimes gave errors "primary key not unique". Actually restarting the nettalk server several times in a row creates quite a lot of not unique guids.
I've looked at the StringTheory (latest version 3.36) procedure "SeedRandom" (see below) and I think (!) this is maybe where things go wrong. I'm not sure but several things seem strange (but I'm not an expert here ....):
- the "stSeedRandom" calll apparently always takes 0 as an argument (see the self.trace line I added), the clock() and ProcesId seem of no influence ...
- the MouseX/MouseY always return 0 (which maybe is not strange for a nettalk server)
- the seed is "only" determined once (via the static variable), is there a reason for doing this once?
At least the 16 char GUID is not unique enough ...
StringTheory.SeedRandom Procedure()
s1 long,auto
code
stSeedRandom(band(clock(),bshift(stGetCurrentProcessId(),24))) ! process Id's are 16 bit, so we only get the low part of the process ID here
self.trace('mouseex : ' & mousex() & ' stGetCurrentProcessId() : ' & stGetCurrentProcessId() & ' argument :' & band(clock(),bshift(stGetCurrentProcessId(),24)))
loop mousex() TIMES ! take some random calls off the random sequence just to add some more variability
s1 += Chr(Random(1,100))
end
loop mousey() TIMES
s1 += Chr(Random(1,100))
end
I'm posting this here is I ran into this when working with services in NetTalk. I created a webservice endpoint that needs a GUID for a record-ID and I ran some stress tests on this method. I noticed the backend sometimes gave errors "primary key not unique". Actually restarting the nettalk server several times in a row creates quite a lot of not unique guids.
I've looked at the StringTheory (latest version 3.36) procedure "SeedRandom" (see below) and I think (!) this is maybe where things go wrong. I'm not sure but several things seem strange (but I'm not an expert here ....):
- the "stSeedRandom" calll apparently always takes 0 as an argument (see the self.trace line I added), the clock() and ProcesId seem of no influence ...
- the MouseX/MouseY always return 0 (which maybe is not strange for a nettalk server)
- the seed is "only" determined once (via the static variable), is there a reason for doing this once?
At least the 16 char GUID is not unique enough ...
StringTheory.SeedRandom Procedure()
s1 long,auto
code
stSeedRandom(band(clock(),bshift(stGetCurrentProcessId(),24))) ! process Id's are 16 bit, so we only get the low part of the process ID here
self.trace('mouseex : ' & mousex() & ' stGetCurrentProcessId() : ' & stGetCurrentProcessId() & ' argument :' & band(clock(),bshift(stGetCurrentProcessId(),24)))
loop mousex() TIMES ! take some random calls off the random sequence just to add some more variability
s1 += Chr(Random(1,100))
end
loop mousey() TIMES
s1 += Chr(Random(1,100))
end