Direct data movement with less CPU overhead
RDMA can reduce copying, context switching and data-path latency between systems. The benefit is real—but only when applications, adapters, switches, firmware and operational processes are engineered as one qualified fabric.
Registered memory
Queue and protection state
Direct placement
Application or storage service
The operating system still controls resources; the fast path moves data more directly
RDMA does not mean unrestricted access to another computer’s memory. Applications or protocol implementations create protected resources and register specific buffers before data transfer begins.
Registered memory
The RDMA stack pins and protects defined memory regions. Keys and access rights determine which remote operations are permitted.
Queues and completions
Work requests are posted to queue pairs. Completion queues report finished operations without requiring the normal socket data path for each transfer.
Direct placement
An RDMA-capable adapter can place payloads into the intended buffer, reducing intermediate copies and CPU involvement in high-rate transfers.
RDMA is not an automatic application accelerator. The application or storage protocol must use an RDMA-aware interface, and the workload must be limited by data movement or CPU overhead before the faster path can create material value.
RoCE, RoCEv2, InfiniBand and Soft-RoCE are not interchangeable labels
They expose related RDMA semantics but differ in routing, hardware, network operation and qualification effort.
| Transport | Network model | Routability | Operational profile | Best fit |
|---|---|---|---|---|
| RoCE v1 | RDMA over Ethernet at Layer 2 | Same broadcast domain | Ethernet QoS and loss design must be consistent | Contained legacy or specialist fabrics |
| RoCEv2 | RDMA over UDP/IP | Layer 3 routable | ECN, PFC where required, QoS, buffers and endpoints validated together | Converged enterprise Ethernet |
| InfiniBand | Native InfiniBand link and transport | Fabric routing under InfiniBand management | Dedicated switches, HCAs and Subnet Manager | HPC, AI and specialised low-latency environments |
| Soft-RoCE | RoCE semantics implemented in software over an Ethernet interface | Follows the underlying IP/Ethernet design | Consumes host CPU; behaviour depends on OS implementation | Functional testing, labs and selected CPU-tolerant workloads |
RDMA is an architecture—not a NIC checkbox
The transport can remove work from the host data path, but it also makes consistency across endpoints and the network more important. A single mismatched MTU, queue policy, driver or firmware combination can erase the expected benefit or create intermittent failures.
Modern platforms can support both lossless designs using PFC plus ECN and validated ECN-led designs without PFC. Follow the endpoint and switch vendor’s supported architecture rather than copying one universal configuration.
Workloads benefit only when the data path is the real constraint
NVMe over RDMA
Preserves the parallel NVMe command model across RoCEv2 or InfiniBand and can lower protocol overhead for latency-sensitive shared storage.
NFS over RDMA
Moves NFS payloads over an RDMA transport. It is useful for Linux-based shared-file workloads where client, server and network support have been validated.
VMware datastores
ESXi can consume NVMe/RDMA storage through supported RoCEv2 adapters. Exact NIC, driver, firmware, PFC and target behaviour must be qualified.
eEVOS
The integrated NVMe-oF initiator can consume compatible remote NVMe/RDMA storage; eEVOS itself remains a complete virtualisation, backup and storage alternative.
Databases
Lower transport latency and CPU work can help high-concurrency I/O, provided consistency, queue depth and failover are tested with the actual database.
HPC, AI and analytics
Large message rates and direct data placement can reduce communication overhead when compute, memory and storage pipelines are balanced.
NVMe/RDMA and NFS over RDMA solve different storage problems
RDMA is the transport mechanism. It does not change whether the host receives raw blocks or a shared file system.
NVMe over RDMA
Block
The client receives an NVMe namespace and owns the file system or application data structure placed on it.
- VMFS and virtualisation datastores
- Database volumes
- Clustered applications with block-aware coordination
- Multipath NVMe host connectivity
NFS over RDMA
File
The server owns the file system and presents shared directories, file names, permissions and locking semantics to multiple clients.
- Linux shared files
- HPC and technical data sets
- Application content and project directories
- Central file permissions and namespace
NFS over ordinary TCP and NVMe/TCP remain strong choices where operational simplicity, compatibility or routability matter more than the lowest possible CPU overhead.
Low latency is valuable only when the recovery path is understood
A second cable is not a complete availability architecture. Test the entire chain under representative application I/O.
Failure tests belong in acceptance testing.
Validate cable loss, switch loss, congestion, target restart, controller or node failover, path recovery and application response. Benchmark steady-state performance only after recovery behaviour is proven.
The same RDMA transport serves different deployment models
Choose the availability and scale model before choosing a product. euroNAS is used here as a practical example because it is the implementation we know and can document directly.
| Platform | NVMe/RDMA role | Relevant transports | Multipathing | Deployment model |
|---|---|---|---|---|
| euroNAS Premium | Target and initiator | RoCE/RoCEv2 and InfiniBand where qualified | Supported | Standalone storage server or virtual storage appliance |
| euroNAS HA Cluster | Target and initiator | RoCE/RoCEv2 and InfiniBand where qualified | Supported | Two-node storage continuity with automatic failover |
| eEKAS | Target and initiator | RDMA-capable gateway paths where qualified | Supported | Ceph-backed scale-out storage and gateways |
| eEVOS | Built-in initiator; no NVMe target | Connects to compatible NVMe/RDMA storage | Supported | Virtualisation host or cluster consuming shared storage |
Soft-RoCE and hardware choice: the euroNAS RDMA implementation can also use Soft-RoCE where qualified. For hardware-offloaded production designs, typical enterprise building blocks include NVIDIA/Mellanox ConnectX or other supported RDMA adapters in HPE, Supermicro and comparable server platforms. The exact adapter, firmware, driver and switch combination matters more than the logo.
euroNAS Premium
Present compatible NVMe/RDMA storage from standard enterprise server hardware.
HA Cluster
Coordinate protected data, portal addressing and target service failover across two nodes.
eEKAS
Combine distributed Ceph data protection with qualified high-performance gateway connectivity.
eEVOS
Use the integrated initiator to consume compatible shared NVMe/RDMA storage.
Select and inspect RDMA storage without routine command-line work
These euroNAS screens illustrate transport selection and target visibility. Network qualification remains an infrastructure task even when storage configuration is GUI-driven.
Select the required NVMe-oF transportChoose TCP or RDMA explicitly during the target workflow.
Verify target and connection stateReview namespace, NQN, transport, portal and connected-controller information.Standards and platform guidance
RDMA architecture overview
Red Hat explains memory placement, kernel bypass and the supported RDMA network models.
RoCEv2 specification context
The InfiniBand Trade Association describes the Layer 3 routing capability added by RoCEv2.
RoCE network design
NVIDIA documents lossless and ECN-led RoCE modes, traffic classes, PFC and congestion handling.
Soft-RoCE implementation
The Linux RDMA project documents the RXE software implementation and its configuration.
NFS over RDMA
Red Hat documents server and client requirements for NFSoRDMA on current enterprise Linux.
VMware NVMe/RDMA
Broadcom documents connecting ESXi to NVMe/RDMA storage through RoCEv2 adapters.
Qualify the complete path before selecting RDMA
Tell us which hosts, protocols, adapters, switches, workloads and availability requirements must work together. We can review whether RoCEv2, InfiniBand, Soft-RoCE or a TCP-based alternative is the appropriate design.
RoCE, InfiniBand, NVMe, VMware and related marks belong to their respective owners. VMware and hardware manufacturers are referenced as technical interoperability examples; this page does not imply a partnership or certification. Supported configurations depend on the current euroNAS release, licence, qualified hardware, drivers, firmware, switching platform and validated client software.