VMware Virtual SAN 6 – Setup and Configuration [Part 2]

In this post I will go through enabling and configuring VMware Virtual SAN (VSAN) 6 cluster within vSphere. Virtual SAN is embedded directly in the hypervisor and does not require any additional software to be installed.

VSAN Architecture


vBoring VSAN Series:

Configure VMkernel for VMware VSAN 6:

First let’s setup a VMkernel so VSAN can talk to the other hosts in the VSAN cluster. In a testing/lab environment it can piggy back off your Management VMkernel adapter but in my case I am going to dedicate a uplink for VSAN traffic.

I created a port group on my distributed switch.

1 VSAN - Create VSAN port group

I gave the port group it’s own dedicated uplink for VSAN.

1-0 VSAN - Create VSAN port group and assign uplink

Next I created the VMkernel for VSAN. Go to Hosts and Clusters, click on a host, go to the Manage tab, Networking, Virtual Switches or VMkernel adapters page (the button you need is on them both), then click on Add Host Networking.

2 VSAN - Add VMkernel Network Adapter

Select VMkernel Network Adapter, click Next:

3 VSAN - Select VMkernel Network Adapter

If you created a port group you can select it now. If you are using a standard switch use the second option. Click Next:

4 VSAN - Select Network

Leave the TCP/IP stack on Default, make sure you check the box beside Virtual SAN traffic. Click Next:

5 VSAN - Enable VSAN VMkernel Service

Assigned a IP address to the VMkernel. Click Next:

6 VSAN - Enter IP Address for VSAN

Ready to Complete! Ensure Virtual SAN traffic says Enabled and a IP address is assigned. Click Finish:

7 VSAN - Ready to Complete

Enable VSAN on the Cluster:

Now we have our disks setup (Part 1), and a VMkernel created for virtual SAN traffic we are ready to enable VSAN on the cluster! VSAN options are only accessible via the web client and will not find them on the thick client.

Before VSAN can be enabled, High Availability (HA) needs to be disabled on the cluster. Click on the cluster, go to Manage, Settings, vSphere HA, then click Edit to turn it off.

8 VSAN - Ensure HA is turned off

Now we are finally ready to enable VSAN. Click on General, then Edit:

9 VSAN - Turn VSAN on

Put a check in the Turn ON Virtual SAN. You have a choice for Add disks to storage. In Manual Mode you have to manually add disks into VSAN. With Automatic it will grab any available disks and use them. I like Manual Mode so you can pick the disks. Once you make your choice click Ok.

10 VSAN - Enable VSAN

Once the tasks completed go back to the General page. It should say Virtual SAN is Turned ON and the Network status as Normal.

11 VSAN - VSAN enabled

Now that Virtual SAN is turned on we are ready to create a Disk Group! Click on Disk Management, then the Claim Disks button.

12 VSAN - Claim Disks

This screen will let us pick the disks on each host that will make up our first Disk Group. A Disk Group is basically a logical container that groups your SSD cache to your HDDs. If you want to read more check out Duncan Epping post about multiple Disk Groups.

For each host select at least 1 Flash disk and the HDD that will contribute storage. Click Ok:

13 VSAN - Claim Disks for multiple hosts

Once the task finishes you will see each host now has a Disk group. When you select a disk group it will show what disks are assigned.

14 VSAN - Confirm Disk Group

If you go to the General page you will see the total capacity of the VSAN with the current disk groups. Adding more disks or another disk group will expand the total capacity.

15 VSAN - VSAN General Summary

VSAN will present a new datastore called vsanDatastore.

16 VSAN - vsanDatastore

You can now go and re enabled High Availability (HA) on your cluster.

Note: VSAN will only present a single datastore no matter how many disk groups you add. I would love to see the ability to create multiple datastores in a single VSAN cluster. For example I have 10k SAS drives for my Tier 1 datastore and my 7200K SATA drives for Tier 2 (backups and such). Though in a production environment you probably wouldn’t have tiered storage but would be nice if that ability was there! 🙂

Hope this post helps in your setup of VMware Virtual SAN 6!! Post comment below with any issues or help advise in your VSAN deployments.

Additional VSAN Reading:

VSAN 6 Maximums

Duncan Eppings VSAN FAQ

How VSAN handles disk/host failures

How to configure the Virtual SAN observer for monitoring/troubleshooting

10 thoughts on “VMware Virtual SAN 6 – Setup and Configuration [Part 2]”

  1. Great article! Do you by any chance have some guide like this of how to configure network with 3 ESXI hosts + SAN. with mgnt, storage, vMotion subnets…
    Thank you

    Reply
  2. Rishat: you seem to be describing this post, which I consider the definitive guide to setting up ESXi on 3 hosts with VSAN. I think that setting up SAN is the hard part, because there is a lot of research to do to pick the right hardware, and there are a lot of ways you can get it subtly wrong. Can you rephrase your question in terms of exactly what’s missing from this guide?

    Reply
  3. I just upgraded our environment from VMWare 6.0 to 6.0 Update 2, so that VSAN will support FT. Do we need manually upgrade VSAN or was it automatically done when we updated ESXi and VMWare?

    Reply
  4. I have a DELL R710. I have 4 drives in RAID5 for 5.7TB and 1 2TB drive in RAID0. I converted the 2TB to SSD via web console. I enable VSAN and it sees both these drives. I add both and then I get a msg:
    hosts cannot communicate with other nodes in the virtual san enabled cluster.
    I do have 3 ESXi hosts (1 is nested) Does each host have to supply disks?

    Reply
  5. Dave, sounds like you didn’t Enable the VSAN on the dedicated vsan network. Once you do that and make sure they have an IP you should be good to go.

    Reply
  6. Without actually seeing verify and test the basics. Can each VSAN vmk vmkping the other VSAN vmks jumbo vmkping. Does each one have a valid IP. Are the network ports properly configured. If VLAN, is it being tagged with the right VLAN id. Disable HA, DRS, and VSAN. Re-Enable VSAN again with Manual and retest then re-enable HA/DRS. Hope it helps if not report results we will figure it out. Remeber you need at minimum one SSD for cache Tier per host and one SSD for capacity per host (Not formatted with VMFS otherwise wipe the disks. Also don’t forget you must install esxi to usb, sd, dedicated disk(s), or use Auto Deploy that is not on a VSAN disk.

    I believe in order for the nested esxi to function/communicate correctly with the physical vsan hosts you have to change or modify something(s). Not sure of the rules here so if you Google it you’ll see where it is discuss and assess if it applies to you

    Reply

Leave a Reply