vSphere Storage Introduction
The vSphere 5.0 storage architecture consists of layers of abstraction that hide and manage the complexity and differences among physical storage subsystems. The VMkernel contains drivers for the physical storage and works with the Virtual Machine Monitor (VMM) to provide virtual disks to the virtual machines. The VMM provides virtual SCSI disks to the applications and guest operating systems inside each virtual machine. A guest operating system sees only a virtual disk file, which is presented to the guest through a virtual Bus Logic or LSI Logic disk controller and is not exposed to FC SAN, iSCSI SAN, direct attached storage, and NAS.
From outside the virtual machine, virtual disks are simply large files identified with the extension .vmdk. They can be copied, archived and backed up as easily as any other files. Virtual disk files are also hardware independent, so for example, you can move one from a local attached SCSI disk used by one virtual machine to a SAN where the exact same virtual disk can be used by a second virtual machine.
The Virtual Machine File System (VMFS) is a clustered file system that leverages shared storage to allow multiple physical hosts to read and write to the same storage simultaneously. VMFS provides on-disk locking to ensure that the same virtual machine is not powered on by multiple servers at the same time. If a physical host fails, the on-disk lock for each virtual machine is released so that virtual machines can be restarted on other physical hosts.
VMware vSphere® Virtual Machine File System or VMFS is one of two file system types that are supported by vSphere on shared storage. VMFS is a high-performance cluster file system designed specifically for virtual machines.
Apart from VMFS, vStorage also works with Network File System or NFS shared storage to host a virtual machine.
VMFS also supports Raw Device Mapping (RDM). The RDM provides a mechanism for a virtual machine to have direct access to a LUN on the physical storage subsystem (FC or iSCSI only).
vSphere 5.0 introduces a new version of VMFS i.e. VMFS5 which has many improvements to its predecessors, VMFS3 and VMFS2, like 2TB plus LUN support, increased resource limits, scalability along with online in-place upgrades.
The virtual disk Thin Provisioning (TP) makes it possible to have thin virtual machine disks to start with. The VMFS does not reserve disk space until needed. Advantages are Improved disk utilization, reduced costs.
vStorage vMotion technology enables the live migration of virtual machine disk files across storage arrays with no disruption in service.
vStorage thin provisioning reduces the storage requirement for virtual environments by allocating storage only when required. It also enables report and alert capabilities required to track the actual storage usage.
Storage Distributed Resource Scheduler or Storage DRS provides the virtual disk placement and load balancing recommendations to datastores in a Storage DRS-enabled datastore cluster. Storage DRS is a mechanism that initiates a Storage vMotion when the datastore exceeds a user-specified I/O latency or space utilization threshold. It manages storage resources across ESXi hosts and load balances for space and I/O latency. It also enables automated initial placement of virtual machines disks when installing new virtual machines.
Virtual Machine Storage: A virtual machine monitor in the VMkernel is the interface between the guest operating system and applications in a virtual machine and the physical storage subsystem in the ESXi host.
A guest operating system sees only a virtual disk that is presented to the guest through a virtual SCSI controller. Depending on the virtual machine configuration, available virtual SCSI controllers are BusLogic Parallel, LSI Logic Parallel, LSI Logic SAS, or VMware Paravirtual SCSI controller.
Each virtual machine can be configured with up to four virtual SCSI controllers, each with up to 15 virtual SCSI disks. Each virtual disk accessed through a virtual SCSI controller is mapped to a physical datastore available to the ESXi host. This datastore can be formatted with either a VMFS or NFS file system.
The datastore can be located on either a local SCSI disk or on a FC, iSCSI, or NAS array.
LUN , Volume and Datastore : A LUN is a Logical Unit Number. It simply means that a logical space has been carved from the storage array by the storage administrator. To ease the identification of this space, the storage administrator assigns a number to each logical volume.
The term LUN can refer to an entire physical disk, a subset of a larger physical disk, or a disk volume. A single LUN can be created from the entire space on the storage disk or array or from a part of the space called a partition. If a virtual machine needs to see a LUN directly, this can be achieved via Raw Disk Mapping or RDM. Although the word LUN is universally accepted, some storage vendors follow the concept of LUNs and MetaLUNs.
When a LUN is mapped to an ESXi, it is referred to as a volume. The volume size can be less than or more than the size of a physical disk drive. When LUN uses disk space on more than one physical disk or partition, it still presents itself as a single volume to an ESXi.
When a volume is formatted with either VMFS or NFS file system, it is referred to as a datastore. Datastores are logical containers, analogous to the file systems that hide specifics of each storage device and provide a uniform model for storing the virtual machine files. Therefore, a datastore is a partition of the volume that is formatted with a file system.
For best performance, a LUN should not be configured with mutliple partitions and multiple VMFS datastores. Each LUN should have only a single VMFS datastore.
Datastore: A datastore serves as a storage space for the virtual disks that stores the virtual machine content. A virtual machine is stored as a set of files in its own directory in a datastore. A datastore is formatted either as a VMFS or NFS volume depending on the type of physical storage in datacenter. It can be manipulated, such as backed-up, just like a file.
VMFS5 allows up to 256 VMFS volumes per system with the minimum volume size of 1.3GB and maximum size of 64TB. By default, up to 8 NFS datastores per system can be supported and can be increased to 64 NFS datastores per system. In addition to virtual machine files, a datastore can also be used to store ISO images, virtual machine templates, and floppy disk images.
Virtual Machine Contents: A virtual machine usually resides in a folder or subdirectory that is created by an ESXi host. When a user creates a new virtual machine, virtual machine files are automatically created on a datastore.
First is a .vmx file. This is the virtual machine configuration file and is the primary configuration file that stores settings chosen in the New Virtual Machine Wizard or virtual machine settings editor.
Second is a .vmxf file that is additional configuration file for virtual machine.
Third is a .vmdk file. This is an ASCII text file that stores information about the virtual machine's hard disk drive. There could be one or more virtual disk files.
Fourth is a -flat.vmdk file. This is a single pre-allocated disk file containing the virtual disk data.
Fifth is a .nvram file. This is a non-volatile RAM that stores virtual machine BIOS information.
Sixth is a .vmss file which is the virtual machine suspending state file that stores the state of a suspended virtual machine.
Seventh is a .vmsd file. This is a centralized file for storing information and metadata about snapshots.
Eighth is a .vmsn file. This is the snapshot state file that stores the running state of a virtual machine at the time you take the snapshot.
Ninth is a .vswp file that is virtual machine swap file for memory allocation.
And at last there is a .log file, which is virtual machine log file and can be useful in troubleshooting when you encounter problems. This file is stored in the directory that holds the configuration (.vmx) file of the virtual machine.
Types of Datastores: The type of datastore to be used for storage depends upon the type of physical storage devices in the datacenter. The physical storage devices include local SCSI disks and networked storage, such as FC SAN disk arrays, iSCSI SAN disk arrays, and NAS arrays.
Local SCSI disks store virtual machine files on internal or external storage devices attached to the ESXi host through a direct bus connection.
Networked storage stores virtual machine files on external shared storage devices or arrays located outside the ESXi host. The ESXi host communicates with the networked devices through a high‐speed network.
Please note that you should format local disk, FC SANs and iSCSI SANs to the VMFS file type for the ESXi host to access them. It is important to format NAS arrays to the NFS file type for the ESXi host to access them.
VMFS Volume: A VMFS volume is a clustered file system that allows multiple hosts read and write access to the same storage device simultaneously.
The cluster file system enables key vSphere features, such as live migration of running virtual machines from one host to another. It also enables an automatic restart of a failed virtual machine on a separate host and the clustering of virtual machines across different hosts.
VMFS provides an on-disk distributed locking system to ensure that the same virtual machine is not powered-on by multiple hosts at the same time. If an ESXi host fails, the on-disk lock for each virtual machine can be released so that virtual machines can be restarted on other ESXi hosts.
Besides locking functionality, virtual machines operate safely in SAN environment with even multiple ESXi host sharing the same VMFS datastore. Please note that you can connect up to 128 hosts to a single VMFS5 volume.
The hard disk drive of a virtual machine is actually a file on the VMFS volume. These files are the .vmdk and .flat-vmdk files. The virtual machine accesses the hard disk through a virtual SCSI controller in the virtual machine. It has no knowledge of the underlying storage architecture.
VMFS can be deployed on a variety of SCSI-based storage devices, such as the FC and iSCSI SAN equipment. A virtual disk stored on a VMFS always appears as a mounted SCSI device to a virtual machine. Virtual disk hides the physical storage layer from the virtual machine’s operating system. This feature allows running an operating system that is not certified for SAN inside a virtual machine. For the operating system inside a virtual machine, VMFS preserves the internal file system semantics ensuring correct application behavior and data integrity for applications running in the virtual machines.
You can create or store multiple virtual machines on same VMFS volume where each virtual machine is defined by a set of files in a single directory.
NFS Volumes: NFS is a file-sharing protocol and is used to establish a client-server relationship between the ESXi hosts and the NAS device. As opposed to block storage, the NAS system itself is responsible for managing the layout and the structure of the files and directories on a physical storage. The ESXi host mounts the NFS volume, and creates one directory for each virtual machine. NFS volume provides shared storage capabilities to support ESXi, such as vMotion, DRS, VMware vSphere High Availability, ISO images, and virtual machine snapshot.
NFS allows volumes to be accessed simultaneously by multiple ESXi hosts that run multiple virtual machines.
The strength of NFS is similar to those of VMFS datastores. After the storage is provisioned to the ESXi hosts, the vCenter administrator is free to use the storage as needed. Additional benefits of NFS datastores include high performance and storage savings provided by thin provisioning. Thin provisioning is the default format for VMDKs created on NFS. The NFS client built into ESXi uses NFS protocol version 3 for communicating with the NAS or NFS servers. By default, NFS uses thin disk provisioning for virtual disks. The NFS datastore with VAAI hardware acceleration supports flat disk, thick provision, and thin provision.
Please note that NFS datastores are popular for deploying storage in VMware infrastructure.
Datastore Clusters and Storage DRS : Like host clusters, you can also create datastore clusters that support resource allocation policies. You can set a threshold on a datastore for space utilization. When the usage exceeds the threshold, Storage DRS recommends or performs Storage vMotion to balance space utilization across the datastores in the cluster. You can also set an I/O threshold for bottlenecks. When I/O latency exceeds the set threshold, Storage DRS either recommends or performs a Storage vMotion to relieve the I/O congestion.
NFS and VMFS datastores cannot be combined in the same Storage DRS-enabled datastore cluster. The datastores can be of different sizes and I/O capacities. The datastores can also be of different arrays with different vendors that configure a datastore cluster.
Please note that any host that connects to datastores in a datastore cluster, must be ESXi 5.0 or later. Earlier version of ESX or ESXi is not supported in a datastore cluster.
Apart from regular hard disk drives, ESXi also supports SSDs that are resilient and provide faster access to data.
Solid State Drives : SSDs use semiconductors for the storage data and have no spindles or disks that rotate and move like in the traditional hard disk drives. An ESXi host can automatically distinguish SSDs from regular hard drives. SSDs provide several advantages. For improved performance, you can use SSDs for per-virtual machine swap areas. It provides high I/O throughput that helps increase the virtual machine consolidation ratio.
Please note that a guest operating system can identify an SSD as a
virtual SSD. A virtual SSD allows a user to create a virtual disk on the SSD device and allows the guest OS to see that it is an SSD.
You can use PSA SATP claim rules for tagging SSD devices that are not detected automatically. A virtual SSD is supported on virtual hardware version 8, ESXi 5.0 hosts, or VMFS 5 file type or later.
Raw Device Mapping (RDM): RDM provides a mechanism for a virtual machine to have direct access to LUN on the physical storage subsystem. RDM is available only on block-based storage arrays. RDM is a mapping file in a separate VMFS volume that acts as a proxy for a raw physical storage device. It allows a virtual machine to directly access and use the storage device and contains metadata for managing and redirecting the disk access to the physical device.
The mapping file gives some of the advantages of direct access to a physical device while keeping some advantages of a virtual disk in VMFS. As a result, it merges the VMFS manageability with the raw device access. There are various terms describing RDM, such as mapping a raw device into a datastore, mapping a system LUN, or a disk file to a physical disk volume.
You can use the vSphere Client to add raw LUNs to virtual machines. You can also use vMotion to migrate virtual machines with RDMs as long as both the source and target hosts have access to the raw LUN. Additional benefits of RDM include distributed file locking, permissions, and naming functionalities.
VMware recommends using VMFS datastores for most virtual disk storage.
RDM Compatibility Modes: There are two compatibility modes available for RDMs, virtual and physical.
The virtual compatibility mode appears exactly as a VMFS virtual disk to a virtual machine. It provides the benefits of VMFS, such as advanced file locking system for data protection and snapshots. However, the real hardware characteristics of the storage disk are hidden from the virtual machine.
In the physical compatibility mode, the VMkernel passes all SCSI commands directly to the device except for the Report LUNs command. Because of this, all characteristics of the underlying storage are exposed to the virtual machine. However, blocking Report LUNs prevents the virtual machine from discovering any other SCSI devices except for the device mapped by the RDM file. This SCSI command capability is useful when the virtual machine is running SAN management agents or SCSI target-based software.
Please note that for RDMs in physical compatibility mode, you can neither convert RDMs to virtual disks nor can you perform operations such as Storage vMotion, migration, or cloning. Also, you cannot relocate RDMs except to the VMFS5 datastore. VMFS 5 supports RDMs in physical compatibility mode that is more than 2TB disk size.
Uses for RDMs: You might need to use raw LUNs with RDMs in situations when SAN snapshot or other layered applications run in a virtual machine. The RDM enables the scalable backup off-loading systems by using features inherent to SAN.
You may also need to use RDM in any Microsoft Cluster Services or MSCS clustering scenario that spans physical hosts in virtual-to-virtual clusters and physical-to-virtual clusters. In this case, the cluster data and quorum disks should be configured as RDM rather than as files on a shared VMFS.
A new LUN is required for each virtual machine with RDM.
FC SAN components: The components of FC SAN can be divided into three categories, the host, the fabric, and the storage component.
The host components of SAN consist of hosts themselves. The host components also contain Host Bus Adapter or HBA components that enable hosts to be physically connected to SAN. HBAs are located in individual host servers. Each host connects to the fabric ports through its HBAs. HBA drivers running on hosts enable the server’s operating systems to communicate with the HBA.
In an FC SAN environment, the ESXi hosts access the disk array through a dedicated network called fabric components. All hosts connect to the storage devices on SAN through the SAN fabric. The network portion of a SAN consists of the fabric components.
SAN switches can connect to hosts, storage devices, and other switches. Therefore, they provide the connection points for the SAN fabric. The type of SAN switch, its design features, and port capacity contributes to its overall capacity, performance, and fault tolerance. The number of switches, types of switches, and manner in which the switches are connected define the fabric topology.
SAN cables are usually special fiber optic cables that connect all the fabric components. The type of SAN cable, the fiber optic signal, and switch licensing determines the maximum distances among SAN components and contribute to the total bandwidth rating of SAN.
The fabric components communicate using the FC communications protocol. FC is the storage interface protocol used for most SANs. FC was developed as a protocol for transferring data between two ports on a serial I/O bus cable at high speed. It supports point-to-point, arbitrated loop, and switched
fabric topologies.
The storage components of a SAN are the storage arrays. Storage arrays include the storage processors or SPs that provide the front-end of the storage array. SPs communicate with the disk array which includes all the disks in the storage array and provide the Redundant Array of Independent Drives or RAID and volume functionality.
SPs provide front-side host attachments to the storage devices from the servers, either directly or through a switch. SPs provide internal access to the drives, which can use either a switch or a bus architecture. In high-end storage systems, drives are normally connected in loops. The back-end loop technology employed by the SP provides several benefits, such as high-speed access to the drives, ability to add more drives to the loop, and redundant access to a single drive from multiple loops when drives are dual-ported and attached to two loops.
Data is stored on disk arrays or tape devices (or both).
Disk arrays are groups of multiple disk devices and are the typical SAN disk storage devices. They can vary greatly in design, capacity, performance, and other features. The distance between the server and the disk array can also be greater than that permitted in a directly attached SCSI environment. Disk arrays are managed by OEM vendor proprietary operating system with built-in intelligence to manage
the arrays.
Please note that switched fabric topology is the basis for most current SANs. The iSCSI is also considered as SAN.
iSCSI Storage Components: The iSCSI allows block-level data to be transported over the IP networks. iSCSI builds on the SCSI protocol by encapsulating the SCSI commands in IP datagrams. It allows these encapsulated data blocks to be transported to an unlimited distance through the TCP/IP packets over traditional Ethernet networks or the Internet.
iSCSI uses client-server architecture. With an iSCSI connection, the ESXi host system and the initiator communicates with a remote storage device and the target in the same manner as it communicates with a local hard disk.
An initiator is typically a host, hosting an application that makes periodic requests for data to a related storage device. Initiators are also referred to as host computers. The iSCSI device driver that resides on a host may also be called an initiator.
The initiators begin iSCSI data transport transactions by sending an application request to send or receive data. The application request is immediately converted into SCSI commands. It is then encapsulated into iSCSI where a packet and a header are added for transportation through the TCP/IP over the Internet or traditional Ethernet networks.
There are two types of SCSI initiators. Both store data are on remote iSCSI storage devices. First is the hardware initiator that uses hardware-based iSCSI HBAs to access data, and second is the software initiator that uses software-based iSCSI code program in the VMkernel to access the data. This type of SCSI initiator requires a standard network adapter for network connectivity.
Targets are the storage devices that reside on a network. Targets receive iSCSI commands from various initiators or hosts on a network. On the target side, these commands are then broken into their original SCSI format to allow the block data to be transported between the initiator and the storage device. The target responds to a host data request by sending SCSI commands back to that host. These commands are again encapsulated through iSCSI for transportation over the Ethernet or the Internet. The targets can be of any type of storage device, such as a storage array that is part of a larger IP SAN.
NAS Components: The NAS device attached to an existing network provides a standalone storage solution that can be used for data backup or additional storage capabilities for the virtual network clients. The primary difference between NAS and SAN depends on how they process communications. NAS communicates over the network using a network share, while SAN primarily uses the FC communication channel.
The NAS devices transfer data from a storage device to a host in the form of files. It uses files systems that are managed independently. They also manage user authentication.
Network vs Local Storage : The SAN storage provides multiple hosts with access to the same storage space. This capability means that all virtual machine templates and ISO images are located on shared storage and also helps with vMotion because the virtual machine data is located on shared storage. It allows clusters of virtual machines across different ESXi hosts. The SAN storage helps perform backups of machines and run these machines quickly after the host failure. This storage also ensures that important data is not lost by minimizing or preventing downtime. The SAN Storage allows moving virtual machines from one ESXi host to another for regular maintenance or other issues. In addition, this storage provides data replication technologies that need to be used for disaster recovery from primary to secondary sites. The SAN storage improves datastore load balancing and performance, by moving virtual disks from one datastore to another along with the Storage DRS technology. The SAN Storage also provides back up solutions by mounting the virtual disks with snapshot technology. Finally, the SAN storage provides great redundancy to the virtual machines with VMware clustering features, such as DRS, vSphere HA, and VMware Fault Tolerance or FT.
The local storage offers extremely high-speed access to data depending on the type of SCSI controller used. A local storage is certainly more economical than the SAN infrastructure. The local storage is also best suited for smaller environments with one or two hosts. Though SAN provides significant benefits over locally attached storage, sometimes these benefits do not outweigh the costs.
Shared or Local Storage: Shared storage is more expensive than local storage, but it supports a larger number of vSphere features. However, local storage might be more practical in a small environment with only a few ESXi hosts.
Shared VMFS partitions offer a number of benefits over local storage. The simple use of vMotion is a huge benefit to any environment, such as the ability to have a fast and central repository for virtual machine templates, to recover virtual machines on another host if a host fails, and the ability to allocate large amounts of storage (terabytes) to the ESXi hosts, and many more. The real idea here is that a shared implementation offers a truly scalable and recoverable ESXi solution.
You can carry out ESXi maintenance without any disruption to virtual machines or users, if shared storage is SAN. Once you have decided on local or shared storage, the next important decision to make is whether the storage is isolated or consolidated.
Storage Considerations: For VMFS volumes, make sure you have one VMFS volume per LUN and carve up the VMFS volume into many VMDKs.
Use spanning to add more capacity. When virtual machines running on this datastore require more space, you can dynamically increase the capacity of a VMFS datastore by adding a new extent. An extent is a partition on a storage device, or LUN. You can add up to 32 new extents of the same storage type to an existing VMFS datastore. The spanned VMFS datastore can use any of all its extents at any time. It does not need to fill up a particular extent before using the next one.
Keep the test and production environment on separate VMFS volumes and use RDMs with virtual machines that use physical-to-virtual clusters or cluster across boxes.
You must keep iSCSI on a separate and isolated IP network for best performance, and keep NAS on a separate and isolated IP network for best performance.
Please note that you have no more than eight NFS mounts per ESXi host because this is the default number while the maximum number of NFS mounts is sixty four.
Another point to remember is that ESXi 5.0 does not support VMFS 2 file systems. Therefore, you have to first upgrade VMFS 2 to VMFS 3 and then to VMFS 5.
vSphere Thin provisioning: vSphere thin provisioning can be done at the array level and the virtual disk level. Thin provisioning at the array level is done by the storage administrator.
Virtual Disk provisioning: When you create a virtual machine, a certain amount of storage space
on a datastore is provisioned, or allocated, to the virtual disk files.
By default, ESXi offers a traditional storage provisioning method. In this method, the amount of storage the virtual machine will need for its entire lifecycle is estimated, a fixed amount of storage space is provisioned to its virtual disk, and the entire provisioned space is committed to the virtual disk during its creation. This type of virtual disk that occupies the entire provisioned space is called a thick disk format.
A virtual disk in the thick format does not change its size. From the beginning, it occupies its entire space on the datastore to which it is assigned.
Thin provisioning: To avoid over-allocating storage space and minimize stranded storage, vSphere supports storage over-commitment in the form of thin provisioning. When a disk is thin provisioned, the virtual machine thinks it has access to a large amount of storage. However, the actual physical footprint is much smaller.
Disks in thin format look just like disks in thick format in terms of logical size. However, the VMware vSphere® Virtual Machine File System or VMFS drivers manage the disks differently in terms of physical size. The VMFS drivers allocate physical space for the thin-provisioned disks on first write and expand the disk on demand, if and when the guest operating system needs it. This capability enables the vCenter Server administrator to allocate the total provisioned space for disks on a datastore at a greater amount than the actual capacity.
If the VMFS volume is full and a thin disk needs to allocate more space for itself, the virtual machine prompts the vCenter Server administrator to provide more space on the underlying VMFS datastore.
vSphere also provides alarms and reports that specifically track allocation versus current usage of storage capacity so that the vCenter Server administrator can optimize allocation of storage for the virtual environment.
Thin provisioning is supported only on VMFS 3 version and later.
Add Extent method: When virtual machines running on a VMFS datastore require more space, you can dynamically increase the capacity of the datastore by using the add extent method. This method enables you to expand a VMFS datastore by attaching an available hard disk space as an extent. The datastore can span 32 physical storage extents and up to 64TB.
Volume Grow method: Certain storage arrays enable you to dynamically increase the size of a LUN within the array. After the LUN size is increased, the VMFS volume grow method can be used to increase the size of the VMFS datastore up to the 64TB limit.
Another use of volume grow is that if the original LUN was larger than the VMFS volume created, then you can use the additional capacity of the LUN by growing the VMFS volume.
Note: volume grow for RDM is not supported.
For both Add Extent and Volume Grow methods, there is no need to power off virtual machines. Both methods can be used on an existing array that has expanded LUN.
Additionally, you can grow a volume any number of times, up to a limit of 64TB. A datastore can have a maximum of 32 extents, but the maximum datastore size is still 64TB. No new partition is added for volume grow, but a new partition is added when performing an extent grow. This new partition has a dependency on the first extent. Therefore, if the first extent fails, virtual machines lose access to the entire volume. With volume grow, as long as the datastore has only one extent, virtual machine availability is never affected.
Monitoring Space Utilization: vCenter Server administrators can monitor space utilization by setting up alarms that send a notification when a certain threshold is reached. They can also analyze reports and charts that graphically represent statistical data for various devices and entities and give real-time data on the utilization.
The vSphere administrator can set alarms on all managed objects in the inventory. When an alarm is set on a parent entity, such as a cluster, all child entities inherit the alarm. Alarms cannot be changed or overridden at the child level.
Alarms have a trigger and an action. A trigger is a set of conditions that must be met for an alarm to register. An action is the operation that occurs in response to the trigger.
Multipathing: To maintain a constant connection between an ESXi host and its storage, ESXi supports multipathing. To support path switching with Fibre Channel or FC SAN, an ESXi host typically has two or more HBAs available, from which the storage array can be reached using one or more switches.
The multipathing capability of ESXi supports both HBA and SP failover.
With Internet Small Computer System Interface or iSCSI storage, ESXi takes advantage of the multipathing support built into the IP network. This support enables the network to perform routing.
Through the Dynamic Discovery process, iSCSI initiators obtain a list of target addresses that the initiators can use as multiple paths to iSCSI LUNs for fail over purposes. In addition, with the software‐initiated iSCSI, the vSphere administrator can use Network Interface Card or NIC teaming, so that multipathing is performed through the networking layer in the VMkernel.
Pluggable Storage Architecture or PSA : To manage storage multipathing, ESXi uses a special VMkernel layer called the Pluggable Storage Architecture or PSA. PSA is an open, modular framework that coordinates the simultaneous operation of multiple multipathing plug-ins or MPPs.
By default, ESXi provides the VMware Native Multipathing Plug-in or NMP. NMP is an extensible module that manages subplug-ins. There are two types of NMP subplug-ins, Storage Array Type Plug-ins or SATPs, and Path Selection Plug-ins or PSPs. SATPs and PSPs can be built-in and are provided by VMware. They can also be provided by a third-party vendor.
Storage I/O Control - SIOC is supported on FC, iSCSI, and NFS storage. However, it does not support RDM or datastores with multiple extents.
In vSphere 5.0, SIOC is enabled by default on the Storage DRS-enabled datastore clusters.
vStorage APIs for Array Integration (VAAI) : VAAI is a set of protocol interfaces between ESXi, storage arrays, and new application programming interfaces in the VMkernel. VAAI helps storage vendors provide hardware assistance to speed up VMware I/O operations that are more efficiently accomplished in the storage hardware. VAAI plug‐ins improve the performance of data transfer and are transparent to the end-user.
VAAI plug-ins are used by ESXi to issue a small set of primitives or fundamental operations to storage arrays. These operations are used to perform storage functions such as cloning and snapshots, which the storage arrays perform more efficiently than the host. In this way, ESXi uses VAAI to improve its
storage services.
vStorage APIs for Storage Awareness or VASA is used by the VASA providers to provide information about their storage arrays to vCenter Server. The vCenter Server instance gets information from storage arrays by using plug-ins called VASA provider. The storage array informs the VASA providers of its configuration, capabilities, storage health, and events. The VASA provider, in turn, informs vCenter Server. This information can then be displayed in vSphere Client.
When VASA provider components are used, vCenter Server can integrate with external storage, both block storage and NFS. This helps you obtain comprehensive and meaningful information about resources and storage data. This also helps you choose the right storage in terms of space, performance, and Service-Level Agreement or SLA requirements.
Profile Driven Storage: Profile-Driven Storage enables you to have a greater control over your storage resources. It also enables virtual machine storage provisioning to become independent of the specific storage available in the environment. You can define virtual machine placement rules in terms of the storage characteristics and monitor a virtual machine's storage placement based on user-defined rules.
Profile-Driven Storage uses VASA to deliver the storage characterization supplied by the storage vendors. VASA improves visibility into the physical storage infrastructure through vCenter Server. It also enables the vSphere administrator to tag storage based on customer-specific descriptions.
Storage vMotion: Using Storage vMotion, you can migrate a virtual machine and its files from one datastore to another, while the virtual machine is running. The virtual machine stays on the same host and the virtual machine files are individually moved to a different datastore location. You can choose to place the virtual machine and all its files in a single location or select separate locations for the virtual machine configuration file and each virtual disk.
You can migrate a virtual machine from one physical storage type like FC to different storage type like iSCSI. Storage vMotion supports FC, iSCSI, and NAS network storage.
The Storage vMotion migration process does not disturb the virtual machine. There is no downtime and the migration is transparent to the guest operating system and the application running on the virtual machine.
Storage vMotion is enhanced in vSphere 5.0 to support migration of virtual machine disks with snapshots.
Storage vMotion has a number of uses in virtual datacenter administration. For example, during an upgrade of VMFS datastore from one version to the next, the vCenter Server administrator can migrate the virtual machines that are running on a VMFS3 datastore to a VMFS5 datastore and then upgrade the VMFS3 datastore without any impact on virtual machines.
Another use is for redistributing storage load. The vCenter Server administrator can use Storage vMotion to redistribute virtual machines or virtual disks to different storage volumes in order to balance the capacity and improve performance.
How Storage vMotion Works: In vSphere 5.0, Storage vMotion uses a new mirroring architecture that guarantees migration success even when facing a slower destination. It also guarantees a more predictable and shorter migration time.
Mirror mode works in a following manner.
The virtual machine directory is copied from the source datastore to the destination datastore. The mirror mode driver takes a single pass and copies the virtual disk files from the source to the destination. The mirror mode driver keeps track of which blocks have been copied to the destination disk. If a write occurs to a disk block on the source that has already been copied to the destination, the mirror mode driver copies the modified block to the destination. The virtual machine on the destination datastore is started using the copied files. The destination virtual machine waits for all virtual machine disk files to finish being copied from the source datastore to the destination datastore. After the single-pass copy is complete, Storage vMotion transfers control to the virtual machine on the destination datastore. Finally, the virtual machine directory and the virtual machine’s disk files are deleted from the source datastore.
Storage DRS: vSphere 5.0 introduces a new storage feature called Storage DRS. This feature helps you to manage multiple datastores as a single resource, called a datastore cluster. A datastore cluster is a group of datastores grouped together but functioning separately. It serves as a container or folder where users can store their datastores.
Storage DRS collects resource usage information for the datastore cluster it has enabled. It then makes recommendations about the initial virtual machine or VMDK placement and migration to avoid I/O and space utilization bottlenecks on the datastores in the cluster.
Storage DRS can be configured to work in either manual mode or fully automated mode.
References and Further reading:
No comments:
Post a Comment