Red Hat Ceph Storage: Object storage performance and sizing guide

Red Hat Ceph Storage is a proven, petabyte-scale, object storage solution designed to meet the scalability, cost, performance, and reliability challenges of large-scale, media-serving, savvy organizations. Designed for web-scale object storage and cloud infrastructures, Red Hat Ceph Storage delivers the scalable performance necessary for rich media and content-distribution workloads.

While most of us are familiar with deploying block or file storage, object storage expertise is less common. Object storage is an effective way to provision flexible and massively scalable data storage without the arbitrary limitations of traditional proprietary or scale-up storage solutions. Before building object storage infrastructure at scale, organizations need to understand how to best configure and deploy software, hardware, and network components to serve a range of diverse workloads. They also need to understand the performance and scalability they can expect from given hardware, software, and network configurations.

This reference architecture/performance and sizing guide describes Red Hat Ceph Storage coupled with QCT (Quanta Cloud Technology) storage servers and networking as object storage infrastructure. Testing, tuning, and performance are described for both large-object and small-object workloads. This guide also presents the results of the tests conducted to evaluate the ability of configurations to scale to host hundreds of millions of objects.

After hundreds of hours of [Test ⇒ Tune ⇒ Repeat] exercises, this reference architecture provides empirical answers to a range of performance questions surrounding Ceph object storage, such as (but not limited to):

  • What are the architectural considerations before designing object storage?
  • What networking is most performant for Ceph object storage?
  • What does performance look like with dedicated vs. co-located Ceph RGWs?
  • How many Ceph RGW nodes do I need?
  • How do I tune object storage performance?
  • What are the recommendations for small/large object workloads?
  • What should I do? I’ve got millions of objects to store.

And the list of questions goes on. You can unlock the performance secrets of Ceph object storage for your organization with the help of the Red Hat Ceph Storage/QCT performance and sizing guide.

Library of Ceph and Gluster reference architectures – Simplicity on the other side of complexity

The Storage Solution Architectures team at Red Hat develops reference architectures, performance and sizing guides, and test drives for Gluster- and Ceph-based solutions. We’re a group of architects who perform lab validation, tuning, and interoperability development for composable storage services with target workloads on optimized server and network configurations. We seek simplicity on the other side of complexity.

At the end of this blog entry is a full library of our current publications and test drives.

In our modern era, a top company asset is pivotability. Pivotability based on external market changes. Pivotability after unknowns become known. Pivotability after golden ideas become dark alleys. For most enterprises, pivotability requires a composable technology infrastructure for shifting resources to meet changing needs. Composable storage services, such as those provided by Ceph and Gluster, are part of many companies’ composable infrastructures.

Composable technology infrastructures are most frequently described by the following attributes:

  • Open source v. closed development.
  • On-demand architectures v. fixed architectures.
  • Commodity hardware v. proprietary appliances.
  • Cross-industry collaboration v. isolated single-vendor silos.

As noted in the following figure, a few companies with large staffs of in-house experts can create composable infrastructures from raw technologies. Their large investments in in-house expertise allows them to convert raw technologies into solutions with limited pre-integration by technology suppliers. AWS, Google, and Azure are all examples of DIY businesses. A larger number of other companies, also needing composable infrastructures, rely on technology suppliers and the community for solution pre-integration and guidance to reduce their in-house expertise costs. We’ll label them “Assisted DIY.” Finally, the majority of global enterprises lack the in-house expertise for deploying these composable infrastructures. They rely on public cloud providers and pre-packaged solutions for their infrastructure needs. We’ll call them “Pre-packaged.”

Brent_Slide

The reference architectures, performance and sizing guides, and test drives produced by our team are primarily focused on the “Assisted DIY” segment of companies. Additionally, we strive to make Gluster and Ceph composable storage services available to the “Pre-packaged” segment of companies by using what we learn to produce pre-packaged combinations of Red Hat software with partner hardware targeting specific workload use cases.

We enjoy our roles at Red Hat because of the many of you with whom we collaborate to produce value.  We hope you find these guides useful.

Team-produced with partner collaboration:

Partner-produced with team collaboration:

Pre-packaged solutions:

Hands-on test drives:

Jack of all trades: New Cisco UCS S-Series and Red Hat Storage

imagesToday, Cisco announced its new UCS S-Series storage-optimized server with the introduction of the UCS S3260, marking its entry into the emerging server market for data intensive workloads.

Red Hat and Cisco have worked together for a long time, including our collaboration on Red Hat OpenStack Platform.

Out with the old…

By jumping into the high-density storage-optimized server market, Cisco validates what we see as the continued movement to emerging software-defined, scale-out architectures for solutions like OpenStack and container-native storage and hyper-converged infrastructure.

With the ability to spread data across multiple servers, both Red Hat Ceph Storage and Red Hat Gluster Storage are helping to drive this trend. Open, software-defined storage enables enterprises to build an elastic cloud infrastructure for newer, data intensive workloads.

Ceph provides unified storage over a distributed object store (RADOS) as its core by providing unified block, object and file interfaces, while Gluster provides an elastic, scale out NAS file storage system.

As more organizations move to open source SDS from appliances / traditional SAN arrays, they often miss the recipes for a best practice deployment. Red Hat has worked with Cisco to produce reference design architectures to take the guess work out of configuring throughput-optimized, cost / capacity-optimized and emerging high IOPs performing clusters, including whitepapers for both Red Hat Ceph Storage and Red Hat Gluster Storage with Cisco’s previous generation of the S-Series, the C3160 high density rack server.

Open source drives storage innovation

Both Ceph and Gluster use community-powered innovation to accelerate their core feature sets faster than what is possible via a single proprietary vendor. Red Hat is a top contributor to both Ceph and Gluster upstream development, but several hardware, software and cloud service providers, including eBay, Yahoo!, CERN (Ceph) and Facebook (Gluster), all contribute to the code base. Cisco itself is a top-50 contributor to Ceph in terms of code commits.

Versatility

The Cisco UCS S-Series builds on the x86 storage-optimized server trend – but seemingly shuffles the deck with more of an enterprise spin via features such as dual-node servers, quadruple fans and power supplies, connected to Cisco UCS Fabric Interconnects.

One aspect of the new UCS S-Series design we are excited about is “versatility”. UCS offers common, consistent architecture for variety of IT needs that we expect may enable it to become a standard hardware building block for enterprise environments. S-Series includes features such as a modular chassis design, facilitating upgrades to new Intel chipsets including its disk expander module, providing the ability to swap out a server node for an additional 4 drives (increasing the raw capacity from 560 to 600 TB).

Cisco has also integrated networking fabric into its storage-optimized servers, making it easier to extend your interconnect as your cluster scales out. The S3260 offers dual 40GbE ports for each server node. As one moves to denser servers (with more than 24 drives) in Ceph configurations, the need for 40Gb Ethernet becomes greater. Enterprises can benefit from tightly-integrated fabric interconnect which translates to less latency, which is important for applications like video streaming.

A key piece is the UCS Manager configuration and handling tool which can simplify deployment. UCS Manager enables the creation of an initial configuration profile for storage, network, compute, etc. for the S3260, helping customers to more easily grow their Ceph environments by pushing out the profile to additional S3260s as they expand.

Combined with the Red Hat Storage ability to handle block, object and file access along with being flexible enough to handle throughput optimized, cost / capacity and high IOPS workloads, Cisco’s UCS S-Series may not just be a jack of all trades, but also a master of many.

Stay tuned for more upcoming joint solution papers from the Cisco UCS S3260 and Red Hat Ceph Storage teams. In the interim, learn more about the UCS S-Series at cisco.com/go/storage.

Latest OpenStack user survey shows Ceph continues to dominate

According to the OpenStack Foundation’s sixth and most recent user survey released just prior to this week’s OpenStack Summit in Tokyo, 62% of users selected Ceph RBD block storage for their OpenStack use cases, nearly three and more than four times the two closest alternatives, LVM (default) and NetApp, respectively.  In production, a full 38% of respondents indicated that their OpenStack deployments depended on Ceph as their Cinder driver, with the same comparisons. A survey of the largest production clouds, those exceeding 1,000 cores, showed similar results, with 37% of users selecting Ceph RBD followed by NetApp at 12%.  Interestingly enough, with 9% of respondents using GlusterFS in production, development & quality assurance, or proof of concept across all OpenStack deployments, more than 70% of OpenStack users are relying on block storage championed by Red Hat Storage.

Continue reading “Latest OpenStack user survey shows Ceph continues to dominate”

Ceph Deployment at Target: Best Practices and Lessons Learned

In October 2014, the first Ceph environment at Target, a U.S. based international chain of department stores, went live. In this insightful slide show (embedded at the end of this post), Will Boege, Sr. Technical Architect at Target, talks about the process, highlighting challenges faced and lessons learned in Target’s first ‘official’ Openstack release.

Continue reading “Ceph Deployment at Target: Best Practices and Lessons Learned”

Ceph’s on the move… and coming to a city near you!

What’s that saying: Ya can’t keep a good person down? Well, ya can’t keep a good technology contained—and that’s why Ceph’s been appearing at venues across the globe.

Ceph Day just hit Chicago

Most recently—this past August—Ceph made its way to Chicago, home of Chicago-style pizza and hot dogs, a place known worldwide for its Prohibition-era ruckus as well as its present-day spirits and brews. There, at Ceph Day Chicago, Bloomberg’s Chris Jones, senior cloud infrastructure, architecture/DevOps, explained how Ceph helps power storage at the financial giant.

Continue reading “Ceph’s on the move… and coming to a city near you!”

The Top 5 Q&As From Sage Weil’s Recent Reddit AMA

Picture1

Sage Weil, Red Hat’s chief architect of Ceph and co-creator of Cephamong many other credentials – recently held an “ask me anything” session on Reddit. Though you can read the whole thing for yourself, here, we’ve collected the top questions and answers for your edification. Read on!

Continue reading “The Top 5 Q&As From Sage Weil’s Recent Reddit AMA”

Check out our rich media deep-dive webinar – on demand!

Want to know more about rich media? In a recent webinar, Red Hat Senior Solutions Architect Kyle Bader took a deep dive into rich media and the unique demands it places on storage systems. We recap some highlights from the webinar here, but please register for the on-demand version here to get the full experience.

Continue reading “Check out our rich media deep-dive webinar – on demand!”

Test drive Red Hat Ceph Storage — for free

Want to try Red Hat Ceph Storage for yourself? Then it’s your lucky day. You can test drive it now. For free!

Available on Amazon Web Services (AWS) in a number of short, easily digestible labs, the Red Hat Ceph Storage test drive gives you a hands-on opportunity to explore the product’s features and see for yourself—in real time—how Red Hat Ceph Storage works. In addition to letting you explore the management features and fundamentals of Red Hat Ceph Storage, test drive labs walk you through such things as how to deploy a Ceph cluster and how to set up erasure-coded and replication storage pools.

Continue reading “Test drive Red Hat Ceph Storage — for free”

The Red Hat Ceph Storage/Supermicro reference architecture has arrived

If you’re building public or private clouds—or simply need massively scalable, flexible storage designed for the cloud—you’ve probably heard about Ceph. But what you may not have heard is that there’s now a reference architecture covering configurations and benchmarks results for Red Hat Ceph Storage on Supermicro storage servers. For real!

supermicro

Continue reading “The Red Hat Ceph Storage/Supermicro reference architecture has arrived”

Visit our Slideshare page for presentations from Red Hat Summit 2015 and everything you’ve ever wanted to know about storage

For easy access to a wealth of Red Hat Storage information, from product updates and use case insights to industry best practices, check out the Red Hat Storage Slideshare page. With presentations going back a few years you’ll be able to find just about anything – including the presentations used during select sessions at Red Hat Summit 2015. Below is just a sampling of some of the presentations you’ll find.

Continue reading “Visit our Slideshare page for presentations from Red Hat Summit 2015 and everything you’ve ever wanted to know about storage”

Learn how Ansible can reduce the time to roll out Ceph to new clusters from hours to minutes

When you put a time consuming challenge with a lot of steps in front of an engineer, their mind invariably strays to “How can I make this faster, easier, better?” We recently spoke with Sébastien Han, senior cloud architect at Red Hat in France, who, like any good engineer, thinks along those lines.

Continue reading “Learn how Ansible can reduce the time to roll out Ceph to new clusters from hours to minutes”