Red Hat Ceph Storage: Object storage performance and sizing guide

Red Hat Ceph Storage is a proven, petabyte-scale, object storage solution designed to meet the scalability, cost, performance, and reliability challenges of large-scale, media-serving, savvy organizations. Designed for web-scale object storage and cloud infrastructures, Red Hat Ceph Storage delivers the scalable performance necessary for rich media and content-distribution workloads.

While most of us are familiar with deploying block or file storage, object storage expertise is less common. Object storage is an effective way to provision flexible and massively scalable data storage without the arbitrary limitations of traditional proprietary or scale-up storage solutions. Before building object storage infrastructure at scale, organizations need to understand how to best configure and deploy software, hardware, and network components to serve a range of diverse workloads. They also need to understand the performance and scalability they can expect from given hardware, software, and network configurations.

This reference architecture/performance and sizing guide describes Red Hat Ceph Storage coupled with QCT (Quanta Cloud Technology) storage servers and networking as object storage infrastructure. Testing, tuning, and performance are described for both large-object and small-object workloads. This guide also presents the results of the tests conducted to evaluate the ability of configurations to scale to host hundreds of millions of objects.

After hundreds of hours of [Test ⇒ Tune ⇒ Repeat] exercises, this reference architecture provides empirical answers to a range of performance questions surrounding Ceph object storage, such as (but not limited to):

  • What are the architectural considerations before designing object storage?
  • What networking is most performant for Ceph object storage?
  • What does performance look like with dedicated vs. co-located Ceph RGWs?
  • How many Ceph RGW nodes do I need?
  • How do I tune object storage performance?
  • What are the recommendations for small/large object workloads?
  • What should I do? I’ve got millions of objects to store.

And the list of questions goes on. You can unlock the performance secrets of Ceph object storage for your organization with the help of the Red Hat Ceph Storage/QCT performance and sizing guide.

Library of Ceph and Gluster reference architectures – Simplicity on the other side of complexity

The Storage Solution Architectures team at Red Hat develops reference architectures, performance and sizing guides, and test drives for Gluster- and Ceph-based solutions. We’re a group of architects who perform lab validation, tuning, and interoperability development for composable storage services with target workloads on optimized server and network configurations. We seek simplicity on the other side of complexity.

At the end of this blog entry is a full library of our current publications and test drives.

In our modern era, a top company asset is pivotability. Pivotability based on external market changes. Pivotability after unknowns become known. Pivotability after golden ideas become dark alleys. For most enterprises, pivotability requires a composable technology infrastructure for shifting resources to meet changing needs. Composable storage services, such as those provided by Ceph and Gluster, are part of many companies’ composable infrastructures.

Composable technology infrastructures are most frequently described by the following attributes:

  • Open source v. closed development.
  • On-demand architectures v. fixed architectures.
  • Commodity hardware v. proprietary appliances.
  • Cross-industry collaboration v. isolated single-vendor silos.

As noted in the following figure, a few companies with large staffs of in-house experts can create composable infrastructures from raw technologies. Their large investments in in-house expertise allows them to convert raw technologies into solutions with limited pre-integration by technology suppliers. AWS, Google, and Azure are all examples of DIY businesses. A larger number of other companies, also needing composable infrastructures, rely on technology suppliers and the community for solution pre-integration and guidance to reduce their in-house expertise costs. We’ll label them “Assisted DIY.” Finally, the majority of global enterprises lack the in-house expertise for deploying these composable infrastructures. They rely on public cloud providers and pre-packaged solutions for their infrastructure needs. We’ll call them “Pre-packaged.”

Brent_Slide

The reference architectures, performance and sizing guides, and test drives produced by our team are primarily focused on the “Assisted DIY” segment of companies. Additionally, we strive to make Gluster and Ceph composable storage services available to the “Pre-packaged” segment of companies by using what we learn to produce pre-packaged combinations of Red Hat software with partner hardware targeting specific workload use cases.

We enjoy our roles at Red Hat because of the many of you with whom we collaborate to produce value.  We hope you find these guides useful.

Team-produced with partner collaboration:

Partner-produced with team collaboration:

Pre-packaged solutions:

Hands-on test drives:

Summit Spotlight: Don’t Miss These Storage Tracks and Sessions

Summit Spotlight: Don’t Miss These Storage Tracks and Sessions

Red Hat Summit kicks off this year from April 14-17th in San Francisco, CA. We’ve organized more than 150 breakout sessions, each with a unique solution-focus, but attendees of all experience levels will see a variety of products, demos, customer success stories, and more.

Continue reading “Summit Spotlight: Don’t Miss These Storage Tracks and Sessions”

Manageability Becoming A Key Component of Open, Software Defined Storage (Red Hat Storage Console Now Available!)

By Steve Bohac, Red Hat Storage Product and Solution Marketing

Open software-defined storage is transforming the way organizations tackle their data management challenges. We are seeing that more and more customers are realizing that an open software-based approach can create opportunities to significantly reduce costs and efficiently contend with their exploding data landscape. Additionally, open software-defined storage solutions can help discover new roles and value for enterprise storage.

Continue reading “Manageability Becoming A Key Component of Open, Software Defined Storage (Red Hat Storage Console Now Available!)”

Red Hat’s Approach with Open, Software-Defined Storage (A Four Part Series)

Today’s Post: Red Hat Storage Server In Action

By Steve Bohac, Red Hat Storage Product Marketing

Several weeks ago, we posted the blog “Open Software Defined Storage – Don’t Get Fooled By The False ‘Open’ and Get Locked-In Again”. Today’s entry is the conclusion of this four part mini-series.

We understand how difficult it is to optimize your storage for innovation and growth, and our goal is to help enterprises on their journey to convert their data centers from cost centers into revenue-generators. Red Hat Storage Server has helped businesses of all varieties achieve their objectives. Here’s how open, software-defined storage has helped a few organizations get to the next level:

Continue reading “Red Hat’s Approach with Open, Software-Defined Storage (A Four Part Series)”

Red Hat Storage Predictions for 2013

Looking through a storage crystal ball, here are our top predictions for storage in 2013. Should these predictions come true, not only will the landscape of the IT industry change, but more importantly, innovation will happen at the pace customers need and not at the pace vendors dictate.

Storage Silos Come Crashing Down!

We predict there will be an emergence of storage solutions that provide a unified approach to procuring, provisioning, and managing enterprise data – solutions that are agnostic to the type of data, such as files, objects, blocks, and semi-structured or unstructured data.

Continue reading “Red Hat Storage Predictions for 2013”

Interview from Structure Conference

Tom Trainer was interviewed recently by Data Center Knowledge at GigaOm’s Structure Conference. The topic was “Gluster: Open Source Storage for the Cloud.” If you’re looking for a good introduction to what Gluster does, this is an excellent place to start.

What Will Dell Do Now?

For a company that prides itself on listening to its customers and delivering more than just what meets requirements, EMC received the message loud and clear from Dell about what the company is looking for in an enterprise class storage product: like a 3Par InServ. Can EMC deliver an enterprise storage array for Dell? Will Dell select a new acquisition target? Now that HP has won the bidding war, what will Dell do now?