Abstract:
The Container Storage Interface (CSI) is a standard for exposing arbitrary block and file storage systems to containerized workloads on Container Orchestration Systems (COs) like Kubernetes. Using CSI third-party storage providers can write and deploy plugins exposing new storage systems in Kubernetes without ever having to touch the core Kubernetes code.
There are many solutions developed as CSI driver plugins. However, solutions that can be easily integrated with On-prem environments are limited. One must perform some tests to choose between these solutions and see their pros/cons.
This article proposes a benchmark test to discover reliable Container Storage Interface (CSI) drivers for Kubernetes clusters in on-premises environments. The test assesses the performance of Longhorn, NFS, and vSphere CSI drivers. The experiment entailed deploying virtual machines in a VMware architecture and configuring a Kubernetes cluster (v1.24.9). Using best practices, Longhorn, CSI Driver NFS, and vSphere CSI Driver were installed on these Kubernetes clusters.
Methodology:
- Cluster Configuration:
- Virtual machines were established using the VMware architecture
- The virtual machines were configured to run a Kubernetes cluster (v1.24.9).
- Driver Installations:
-
By recommended standards, Longhorn , NFS CSI Driver , and vSphere CSI Driver were deployed on the Kubernetes cluster.
-
- Storageclass Configuration:
- Each storage solution was assigned a unique "storageclass" on the cluster.
- The "storageclass" setups were used during the tests.
- Testing Environment:
- Statefulset deployments were established.
- Testing was carried out in pods within the Statefulsets.
- Persistent Volume Claims (PVCs) were produced in Read-Write-Once (RWO) access mode.
- Test Parameters:
- Test parameters included a "random rw" pattern.
- To obtain accurate results, the disc cache was disabled.
- Tests were run concurrently for all CSI kinds on several hosts.
- Tool Usage:
- fio (Flexible IO Tester):
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --bs=4k --iodepth=64 --readwrite=randrw --rwmixread=75 --size=4G --filename=/var/data/testfile- Ioping
ioping -c 200
Test Outputs:
Benchmark testing yielded throughput, IO, and latency values, displayed in three separate graphs.
Throughput:
Throughput is a metric that describes the amount of data able to flow through a point in the data path over a given time. Throughput is typically the best storage metric when measuring data that needs to be streamed rapidly.
Input/Output (I/O):
IOPS, or Input/Output operations per second, measures the number of storage transactions processed through a system each second. This metric is a great way to measure smaller data objects like web traffic logs.
I/O Latency:
Latency describes the time required for a sub-system to process a single data request or transaction. Latency also includes the time it takes to find the required data blocks and prepare to transfer data.
Conclusion:
The benchmark test results show how Longhorn, NFS, and vSphere CSI drivers perform in a Kubernetes cluster on VMware infrastructure. These findings can help you choose the best CSI driver for your on-premises Kubernetes infrastructure. Each CSI driver has different pros/cons besides performance. It is necessary to consider these values as well as performance when making a choice.