Keep compute supplied without making storage the hidden bottleneck
AI training, inference, simulation and analytics move data in very different ways. The right architecture balances sustained throughput, metadata responsiveness, checkpoint writes, capacity growth and recovery—rather than selecting a fabric from one benchmark.
Design every lane for its own concurrency, protection and growth behaviour.
Start with how the application moves data
Capacity and headline bandwidth do not reveal file count, reuse, concurrency, checkpoint frequency or the cost of a stalled compute node.
Feed many workers repeatedly
- Parallel readers revisit large datasets over many epochs.
- Millions of small samples can become a namespace and metadata workload.
- Checkpoint writes compete with foreground reads unless planned separately.
Mix models, indexes and live data
- Model loading is only one part of the service path.
- Vector databases, document collections and object data may use different storage access.
- Tail latency can matter more than peak transfer speed.
Protect long-running computation
- Shared files, temporary results and checkpoint/restart traffic may arrive in bursts.
- Directory operations and small files can limit workflows that do not resemble a bandwidth test.
- Recovery time must include data validation and job restart.
Scan, transform and retain
- Large scans and concurrent pipelines consume sustained bandwidth.
- S3 and shared file access suit different application ecosystems.
- Ingest, transformation and retention policies influence usable capacity.
Separate the demands before choosing the platform
Keep workers supplied
Measure concurrent reads, reuse, sample size and the effect of preprocessing—not only a single sequential stream.
Protect metadata responsiveness
File creation, lookup, directory traversal and permissions can dominate workflows with many small files.
Write without stalling the job
Define checkpoint size, interval, retention and whether several workers write at the same time.
Do not overlook databases and logs
Schedulers, catalogues, vector databases and telemetry often need predictable small-I/O latency alongside bulk data.
A useful test reproduces the whole pipeline: data loading, preprocessing, worker concurrency, checkpointing, background protection and degraded operation should be measured together.
Match the data path to the application interface
No transport is automatically best for AI or HPC. The application, operating system and data layout must be able to use the path effectively.
| Data path | Often fits | Potential advantage | Validate before deployment |
|---|---|---|---|
| Local NVMe | Node-local cache, scratch space and staging | Very short I/O path and high local bandwidth | Dataset distribution, node failure, rehydration, consistency and capacity duplication |
| NVMe over RDMA | Latency-sensitive shared block access and qualified high-performance fabrics | Low transport overhead and efficient remote NVMe access | RoCE/RoCEv2 or InfiniBand design, adapters, switches, multipathing, topology and application support |
| NVMe over TCP | Shared block storage on familiar routed IP networks | NVMe semantics with broadly understood Ethernet operations | CPU demand, congestion, queue design, host support and real workload latency |
| NFS or SMB | Shared datasets, model repositories and collaborative workflows | Common file interface and centrally managed namespace | Client concurrency, metadata rate, locking, caching and application-specific support |
| S3-compatible object | Data lakes, training corpora, model artefacts, archive and pipeline exchange | API-based scale, independent clients and flexible capacity tiers | Application S3 support, object size, request rate, consistency expectations and local caching |
Block, file and object are complementary: a common pattern uses S3 for durable datasets, shared file access for active collaboration and block storage for databases or latency-sensitive application volumes.
RDMA does not imply direct access to GPU memory
NVMe over RDMA provides efficient remote storage access across an RDMA fabric. Technologies that transfer directly between storage and GPU memory require a separate, end-to-end qualified software and hardware stack.
Clear scope: the euroNAS NVMe-oF and RDMA capabilities described on this site must not be interpreted as validated NVIDIA GPUDirect Storage support. Direct transfers into GPU memory are not presented as a euroNAS feature.
Qualified RDMA projects may use NVIDIA/Mellanox adapters and enterprise servers from vendors such as Supermicro or HPE. The required firmware, driver, NIC, switch and PCIe combinations must be verified for every deployment.
Large datasets can still be limited by very small operations
AI and HPC workflows frequently combine bulk data with file lookups, directory scans, scheduler databases and object metadata.
File-system metadata
A separate CephFS metadata pool holds namespace information such as file names, directories and permissions. Placing this latency-sensitive pool on fast SSD or NVMe media can improve metadata-heavy workflows while bulk file data remains on capacity media.
Most relevant for- Many small training samples
- Large directory trees
- Frequent create, delete and lookup operations
OSD DB/WAL placement
With HDD-backed Ceph OSDs, internal database and write-ahead-log activity may benefit from substantially faster media. This accelerates internal object metadata work and is a separate decision from the CephFS metadata pool.
Most relevant for- Capacity-oriented HDD nodes
- Mixed object and file activity
- Designs requiring responsive metadata handling
Keep the distinction clear: DB/WAL accelerates internal OSD work; the CephFS metadata pool serves file-system namespace operations. eEKAS presents both choices through guided graphical workflows without requiring Ceph command-line knowledge.
Manage the distributed layer without a wall of commands
eEKAS provides graphical workflows for cluster creation, drive selection, protection profiles, DB/WAL placement, file-system metadata and service endpoints. Administrators do not need previous Ceph or command-line experience for routine provisioning and monitoring.
Use euroNAS platforms as implementation examples
The appropriate model depends on scale, failure boundaries, application interfaces and operational ownership.
Focused high-performance storage
A dedicated storage system for a compact environment using the required block or file access. Availability and complete-server recovery must be handled by the surrounding design.
- NVMe-oF, iSCSI, Fibre Channel or file services according to host support
- Can also be deployed as a Virtual Storage Appliance
- Suitable when scale-out storage is not required
Two-node storage availability
A local two-node design for shared block or file services that require automatic storage failover, using either a synchronous Mirror or dual controllers with shared storage.
- Multipathed access for supported block protocols
- Synchronous local continuity and separate asynchronous recovery options
- Best for a defined local scale rather than distributed expansion
Independent scale-out data platform
Distributed storage for large shared datasets and several consumer groups, exposing block, file and S3 services while capacity and performance grow across qualified nodes.
- Ceph protection across defined failure domains
- Replication or erasure coding according to workload
- Graphical DB/WAL and metadata placement
Virtual services around the data pipeline
eEVOS can host the supporting virtual services around an AI or analytics environment and connect to internal, external shared or Ceph storage. Accelerator requirements must be validated separately.
- VM high availability and live migration
- Integrated Backup & Disaster Recovery
- Instant Backup & Recovery for protected virtual workloads
Questions that reveal the real bottleneck
How many files or objects exist, how large are they and how often are they reused?
How many workers read and write simultaneously, and do they access the same data?
What is the checkpoint size, interval, retention and acceptable restart time?
Which drivers, libraries, filesystems, block devices and S3 interfaces are supported by the application?
What throughput remains acceptable during a node loss, rebuild or data rebalancing event?
Will capacity, compute nodes and client count grow together—or must they scale independently?
Independent technical guidance
Validate the final design against current application, transport and storage documentation.
NVM Express: NVMe over RDMA
Official description of the RDMA transport for moving data between computer and storage memory across a fabric.
Open NVM Express guidance →CephFS architecture
Ceph documentation covering separate file data and metadata pools and their role in shared filesystem operation.
Open Ceph documentation →Bring the data pipeline—not only the capacity figure
We can review dataset shape, concurrency, network options, metadata behaviour, checkpointing, resilience and growth for a business AI, HPC or analytics project.