Server Specs

[kevin plummer]

Hi All,

I'm looking to co-host a server and wanted to see if anyone had any advice on the following setup.

I plan to run vmware 5.0 on the server and have around 8 servers virtualized running win2008r2. I will be running sqlserver 2008,nt web apps, windows apps and windows apps served up using go global and active directory on different VM's

I'm looking at a dell R720 server with the following specs:

128gb Ram
2 x 8 core cpu's Xeon E5-2640v2 2.0 ghz
2 x 2GB SD Card
4 x 2TB 7.2k near line drives
2 x 480gb SSD SATA Read intensive 3GBPS
Redundant Power Supply.

Thanks,

Kevin

[Bruce]

That's a fair bit of grunt!

[Keith]

Kevin

One point to consider is how many virtual CPUs you will need to allocate to your 'around 8 servers'.  This 720 has 16 cores so you can only run a max of 16 x 1 vcpu machines without the possibility of scheduling clashes (but of course if each server was only 10% busy then you would be fine).

If your servers could all perform well with 2 vcpus then you would be ok but (for example) if your DB server and some other functions needed 4 vcpu then you may struggle if their loading was high.

As always, you will need to work out the expected workload on each of your v servers and map them onto your hardware to see if you have enough cores.

However, I would be surprised if you actually needed more  - functions running on Windows servers don't typically use a significant percentage of the CPU.

Keith

[kevin plummer]

Thanks Keith. I'm only looking at running 8vm's but did read a post recently about the problem you mentioned of over allocating CPU's to VM's which actually slows down the server because it needs to wait for the total# of available CPU's allocated before it can process. In this case, the user allocated 4cpu's to SQLServer and had detected lots of pauses in processing. Dropping back to 2 fixed the problem but I guess it was dependent on his hardware and the rest of his VM setup.

[bshields]

Hi Kevin,

I recently moved all our servers from our own environment into co-location (Global Switch in Ultimo Sydney).

I am running 2 x Dell R620s and 3 x Dell R610s. 2 x Fortigate 200a Routers (old but good), You don't need to run your own routers but we host lots of websites so I want it all under my control so I don't have to ask the hosting people for anything.

We use VMWare 5.5.

#1 Dell R620, 2 x 8 Core E5-2650 @ 2Ghz,128 GB RAM, 4 x SAS, 4 x SSD, 4 x 1G Ethernet aggregated

#2 Dell R620, 2 x 6 Core E5-2640 @ 2.5Ghz, 72GB RAM, 4 x SAS, 4 x SSD, 4 x 1G Ethernet aggregated

#3 Dell R610, 2 x 6 Core X5650 @ 2.66Ghz, 48 GB RAM, 6 X SAS, 4 x 1G Ethernet aggregated

#4 Dell R610, 2 x 6 Core X5660 @ 2.8Ghz, 72GB RAM, 6 x SAS, 4 x 1G Ethernet aggregated

#5 Dell R610 is using OpenFiler as a SAN (since VMWare loves SANs). Can't rememebr its config, but its a 2 x 6C machine like the ones above.

I run 26 VMs with Windows Server 2003, 2008 or 2012 (mixed bag depending upon the licences I owned) spread over 3 of those Hosts. Host #4 is doing nothing at the moment.

CPU utilisation on the three main Hosts sits a below 10% and mostly below 5%.

These servers hammer.

Oh and we choose the 6xx series over the 7xx series as we pay for our hosting based on RUs used, so slimmer the better.

A friend of mine (who knows an awful lot more about VMWare than me) strongly recommended that you only allocate just enough CPUs to the VMs, to avoid scheduling problems. Ie, if your VM wants 8 CPUs it cannot have any until all 8 are available.

Regards
Bill

[bshields]

Hi Kevin,

I gave your config a little more thought, since I went thought the same experience spec'ing my machines only last October.

I went away from NL-SAS Drives due to their lower reliability (although only one order of magnitude compared with SAS drives) and also their slower rotational speed. However, it did mean my maximum SAS Drive size was only 1.2TB, since all my drives are 2.5" (and they were expensive - when you need 8 ).

However, you can't beat NL-SAS for price/capacity. Especially, as I expect you are using your SSD drives for your SQL databases (as i do). SQL loves SSDs. I did a lot of reading on SSD and the best raid, I ended up just going raid 5, as my SQL is probably 100 reads for every write. But, to be honest, I just gave up one trying to digest all the stats, and just went with raid 5 for reliability, performance and capacity.

I Built the SAN because it makes management, especially moving VMs from one host to another and also using VMWares Data Recovery Appliance, much easier. But if you are running one host i guess this isn't a problem.

Most of my apps are NT web servers and MySQL databases.

So your config looks pretty much exactly the same as what i did.

Regards
Bill

[kevin plummer]

Hi Bill,

I know you are a bit of a hardware junky so I appreciate your comments.

Here are some questions back...

what capacity are you using on your SSD and SAS drives say on server#1?
I thought raid5 didn't work on a vmware server. It was a comment made on a clarion live webinar about setting up a dedicated server. Maybe I misinterpreted?
Another comment on this webinar was using the SSD drives for VMWare swap files which greatly enhanced performance.

I'll end up running 3 hosts (AU, UK, Canada) but my current budget could only afford vm5 essentials. Not sure if I can leverage any benefits of a SAN with that version.

My basic plan is to replicate the data between the 3 and provide IP failover.

Cheers,

Kev

[bshields]

Hi Kevin,

My SAS are either 1.2TB or 600GB.

The SAN has mirrored 146GBs for boot and 4x1.2TB in Raid 5 for Data. But this was predicated by the SAN being stubborn rather than my choice. I don't store VMs on the SAN, its just for backups, ISOs and when using the free version of VMWare it allows me to copy VMs from host to host.

The SSDs are 120GB or 240GB (basically because it was cheaper and i didn't need the storage). The VMs that require fast storage get 2 drives a C drive from the SAS and a D drive from the SSD only the SQL goes onto the D drive.

You are correct VMWare can detect SSD drives and can utilise them for swap space, but this is more important on systems where the VMs are stored on a SAN and the bottleneck is say a 10GBe or 4Gb fibre connection, you don't wanna swap over that. As you are running your VMs on the host drives, which are way faster than a 1, 4 or 10Gb SAN, those SSDs are best setup as drives for SQL. With your RAM you wont or shouldn't be swapping anyway, and if you are swapping buy more RAM.

You have to remember much of the talk about VMWare config is geared toward very large VM counts and a large amount of shared resources, 100 of VMs on a Host, all stored on a SAN. These are very different to config compared to what you and I are doing. We have relatively few VMs and we want serious performance from each of them. I've done some work with Sydney University and my "VMWare" friend is the manager of NSW University Infrastructure, and they have hundreds of lazy, forgotten servers, doing not much, with  the VMs stored on the SAN.

VMWare doesn't care what raid you run, as the raid is setup via the PERC before vmware is installed anyway. It works, but maybe the comments you heard where more inline with perhaps a recommended setup, Raid 5 can be slower to write than some other configs, so perhaps someone had a view on that.

I have VMWare Essentials Plus 3 Host Version (retail is about $7k) I got mine from a HP Reseller in Melbourne for $2.25k, much more reasonable!

But, after while with a little VMWare practice you can get a lot done with their free version. My 4th Host will be running the free version.

If you want to place your hosts into a cluster and have VMWare manage fallover for you, you'll need full licences. But some VMs like my Domain Controllers, Backup Devices, General Admin stuff, can sit on the free one anyway. So you can get some serious value out of VMWare without paying a fortune for it.

I almost went down the path of Microsoft HyperV due to VMware's bizarre licencing, but i really love VMware now, makes working with my servers a joy. I don't even need a technician anymore (saving of $70k pa), I do it myself and actually like it.

Regards
Bill

[bshields]

Oh i forgot (its late), Even the free version of VMWare will talk to a SAN. In fact a SAN makes the free version much more flexible, as you have shared storage all your hosts can see.

Normally a SAN is very expensive. I didn't entertain it, but on more stuffing around with VMWare I realised just how good it could be.

I found OpenFiler which is free and if placed on good hardware (you can run it within a VM, but that seemed crazy to me) its really quite good.

Its a complete pig to setup, or I was unlucky, or both. But, after some practice I can now set it up pretty quick. It is however, impossible without reading the manual, and quite a bit in addition.

If you have the hardware laying around (originally I setup my SAN on a Dell 1950, but with only 4 x 2.5" Drive slots) and the SAN having issues unless I gave it its own boot drive and placing my data on another drive, I only ended up with 2 drives for data.

Anyway, maybe some of my misadventure will help you out.

[kevin plummer]

Thanks Bill - your comments are great and all make sense. Who was the HP reseller? At $2.25k it might be worth upgrading.

[bshields]

SystemMax Technologies
03 9357 6414