Load balancing

[Alberto]

Hi Bruce, Do you plain to add load balancing to NT14 ?

[Bruce]

Load Balancing is something that happens outside the server, not inside the server.

A Load Balancer accepts traffic, then passes it on to another server (the NetTalk Server) for processing.

Lots and lots of Load Balancer programs already exist, and are perfectly good at balancing. So re-writing one of those in NetTalk seems like it adds a lot of work, for zero gain in actual functionality.

So, to answer your question, no I'm not planning on implementing a load balancer in NT14.

Cheers
Bruce

[osquiabro]

but how would it be done with one of the existing tools? do you have example, documentation or webminar?

[Jane]

but how would it be done with one of the existing tools? do you have example, documentation or webminar?

Gordon Holfelder did a webinar on this for the CIDC 2019 conference, which is still online if you purchased that conference:
https://www.cidc2019.com/Presentations#LoadBalancingaWebserverAcrossMultipleInstances

As I recall, one of his challenges was sharing session values across multiple web servers. 

The Net Sessions interface that Bruce introduced in NT 12 should make sharing session values easier.

[Bruce]

>> but how would it be done with one of the existing tools?

the web is full of information on load balancing. Google is your friend. I recommend searching on
"http session based load balancer"

>> do you have example, documentation or webinar?

personally, no.

>> As I recall, one of his challenges was sharing session values across multiple web servers. 

not really [1].
Session based load balancers are a thing, and there are plenty of them, including things like nginx.

What Gordon wanted was to share the session itself between multiple servers, because this makes redundancy better (a single machine can be removed at will).

>> The Net Sessions interface that Bruce introduced in NT 12 should make sharing session values easier.

This is correct. However Gordon uses a different framework for his apps, one which basically does very (very) few reads and writes to session data. So storing the session data off-server makes sense. For a regular NetTalk web app, I don't recommend off-server session storage.

Since session-based load balancing is a thing, I would recommend this route first.

cheers
Bruce

[DonRidley]

Hello Alberto,

I use Nginx Core to perform load balancing between 3 multi-site host web servers.

Nginx Core is their free version.  Nginx Plus is the commercial version.

The setup works well.  It does not address sessions. 

Getting the Nginx settings right is a little tedious but not incredibly difficult.

I could post a copy of my settings here if you're interested.

[Jane]

It does not address sessions. 

So how do you deal with that, Don?

Does Nginx just always direct session xyz to the same instance that session used for the previous call(s) ?

Cheers,

Jane

[DonRidley]

So how do you deal with that, Don?

Does Nginx just always direct session xyz to the same instance that session used for the previous call(s) ?

Cheers,

Jane

With the IP Hash of Nginx Core, once a request is made from an IP address, subsequent requests from that IP address are directed to the same upstream server.  So the session stays alive on that server.

If I stop that server, that IP will be directed to one of the other servers.  However, the session is lost and the user will have to login again.  This is the holy grail for me.  Getting session data moved to the new server.  I haven't solved that yet.  Now, if users do not require a login it would work perfectly.

I hope that answers your question.

[Jane]

Thanks, Don.

Coincidentally, Gordon did a ClarionLive webinar last Friday where he showed an overview of his current architecture.

As Bruce indicated, Gordon has moved a lot of stuff into the javascript app in the browser (which is something like 300K lines of code) to minimize use of the session queue.

What session queue he IS using is on a separate database.  https://www.youtube.com/live/6x1wvtstK74?t=2806

But he also has a large-ish team of programmers and many years of coding underlying the app.

[DonRidley]

I watched that webinar.  Gordon and Flint have done some really cool things with NetTalk.

At the 2019 CIDC they demonstrated a way to save session data and load it into another server.  But, I never could get it to work on my servers.  That has to do with how they are using their server where I'm using it more "out of the box" so to speak.

It's still on my to do list.

[Matthew51]

Greetings everyone

I'm a little late to this party. Christmas is a busy time for me. I had also not planned on announcing this as I'm still working out a few kinks as well as adding features.

I do have a working sessions sharing, load balanced set up. It's currently designed to work on Amazon's AWS. You can see it running at https://cloud.posal.com. The server group sees over 100,000 requests per day, with pdf reports being in the 100s. It is split between 2-3 servers depending on the time of day.
Login:  TryPosal
Password:  posal123

As I said I'm still working out some kinks. Individual servers go unhealthy, and need to be restarted more often then I would like. Approximately one a month the whole server group will become unhealthy. I fell like the later is because one of my users is unintentionally DoS me by spamming large requests.

Things I'm still working on, or would like to add:
Better diagnostics and reporting on server health
Automatically changing the group size by more then 1
Changing group size based on load (currently only by schedule)
Creating a new AWS server from an image (currently uses cold spares)
Server requesting that they are temperately removed from the group when handling intensive requests (like multi page reports)

As I'm a contractor and this has taken a lot of work to even get this far I will need to charge for my work. I hadn't decided how much, though anyone helping with the testing phase will get a discount.

If you're interested contact me at https://www.bisware.ca/contact

[DonRidley]

Hello Matthew,

I like your demo site.  Nice and clean interface.

I would be interested in learning more about your approach to handling sessions.

[Matthew51]

The theory is quite simple. Capture every change to session data and broadcast that to every other server so they can make the same changes. A separate monitoring program manages the health of each server and collects diagnostics data. The web folder is just a windows mapped drive for now. I want to move the shared folder to S3 storage as I need to use a hack to get the share working with a windows service.

In practice it's a few thousand lines of code and several month of work to get this far.

While the code is very efficient it does have an n² problem if you get too many servers.

[WillieB82]

Hello Alberto,

I use Nginx Core to perform load balancing between 3 multi-site host web servers.

Nginx Core is their free version.  Nginx Plus is the commercial version.

The setup works well.  It does not address sessions. 

Getting the Nginx settings right is a little tedious but not incredibly difficult.

I could post a copy of my settings here if you're interested.

Hi Don,

I was trying to find some info on setting up nginx and came upon your port.  Do you mind sending me a copy of how your nginx conf file looks like?

I could get nginx to open the login page but having trouble changing the "working folder" to the folder where the webserver is in.

Thank you

[DonRidley]

Sure I?ll send it but it?ll probably be tomorrow.