Resource Consumption Metrics For YDB Distributed Storage

The YDB distributed storage has a specific throughput capacity, limited by the resources of physical devices within the cluster, and cannot provide low latency for requests if the workload exceeds this limit. Throughout the operation of YDB clusters, there periodically arises an issue with increased response times of the distributed storage, which may be related both to exceeding this throughput capacity and to other problems within the system. To facilitate diagnostics in such cases and provide users with information about the level of resource usage of the distributed storage, we have developed new metrics that can assess whether the load volume exceeds the physical devices’ capacity. Using these metrics, we have formulated the necessary conditions to ensure a guarantee of low latency for BlobStorage operations. The presence of such a guarantee will allow users of the distributed storage to more accurately balance their load on storage groups to avoid overloading which may cause severe latency degradation.