The third one’s a charm

By Federico Lucifredi, Red Hat Storage

 

 

Red Hat Ceph Storage 3 is our annual major release of Red Hat Ceph Storage, and it brings great new features to customers in the areas of containers, usability, and raw technology horsepower. It includes support for CephFS, giving us a comprehensive, all-in-one storage solution in Ceph spanning block, object, and file alike. It introduces iSCSI support to provide storage to platforms like VMWare ESX and Windows Server that currently lack native Ceph drivers. And we are introducing support for client-side caching with dm-cache.

On the usability front, we’re introducing new automation to manage the cluster with less user intervention (dynamic bucket sharding), a troubleshooting tool to analyze and flag invalid cluster configurations (Ceph Medic), and a new monitoring dashboard (Ceph Metrics) that brings enhanced insight into the state of the cluster.

Last, but definitely not least, containerized storage daemons (CSDs) drive a significant improvement in TCO through better hardware utilization.

Containers, containers, never enough containers!

We graduated to fully supporting our Ceph distribution running containerized in Docker application containers earlier in June 2017 with the 2.3 release, after more than a year of open testing of tech preview images.

Red Hat Ceph Storage 3 raises the bar by introducing colocated CSDs as a supported configuration. CSDs drive a significant TCO improvement through better hardware utilization; the baseline object store cluster we recommend to new users spans 10 OSD storage nodes, 3 MON controllers, and 3 RGW S3 gateways. By allowing colocation, the smaller MON and RGW nodes can now run colocated on the OSDs’ hardware, allowing users to avoid not only the capital expense of those servers but the ongoing operational cost of managing those servers. Pricing that configuration using a popular hardware vendor, we estimate that users could experience a 24% hardware cost reduction or, in alternative, add 30% more raw storage for the same initial hardware invoice.

“All nodes are storage nodes now!”

We are accomplishing this improvement by colocating any of the Ceph scale-out daemons on the storage servers, one per host. Containers reserve RAM and CPU allocations that protect both the OSD and the co-located daemon from resource starvation during rebalancing or recovery load spikes. We can currently colocate all the scale-out daemons except the new iSCSI gateway, but we expect that in the short term MON, MGR, RGW, and the newly supported MDS should take the lion’s share of these configurations.

As my marketing manager is found of saying, all nodes are storage nodes now! Just as important, we can field a containerized deployment using the very same ceph-ansible playbooks our customers are familiar with and have come to love. Users can conveniently learn how to operate with containerized storage while still relying on the same tools—and we continue to support RPM-based deployments. So if you would rather see others cross the chasm first, that is totally okay as well—You can continue operating with RPMs and Ansible as you are accustomed to.

CephFS: now fully supportawesome

The Ceph filesystem, CephFS for friends, is the Ceph interface providing the abstraction of a POSIX-compliant filesystem backed by the storage of a RADOS object storage cluster. CephFS achieved reliability and stability already last year, but with this new version, the MDS directory metadata service is fully scale-out, eliminating our last remaining concern to its production use. In Sage Weil’s own words, it is now fully awesome!

“CephFS is now fully awesome!” —Sage Weil

With this new version, CephFS is now fully supported by Red Hat. For details about CephFS, see the Ceph File System Guide for Red Hat Ceph Storage 3. While I am on the subject, I’d like to give a shout-out to the unsung heroes in our awesome storage documentation team: They have steadily introduced high-quality guides with every release, and our customers are surely taking notice.

iSCSI and NFS: compatibility trifecta

Earlier this year, we introduced the first version of our NFS gateway, allowing a user to mount an S3 bucket as if it was an NFS folder, for quick bulk import and export of data from the cluster, as literally every device out there speaks NFS natively. In this release, we’re enhancing the NFS gateway with support for NFS v.3 alongside the existing NFS v.4 support.

The remaining leg of our legacy compatibility plan is iSCSI. While iSCSI is not ideally suited to a scale-out system like Ceph, the use of multipathing for failover makes the fit smoother than one would expect, as no explicit HA is needed to manage failover.

With Red Hat Ceph Storage 3, we’re bringing to GA the iSCSI gateway service that we have been previewing during the past year. While we continue to favor the LibRBD interface as it is more featureful and delivers better performance, iSCSI makes perfect sense as a fall-back to connect VMWare and Windows servers to Ceph storage, and generally anywhere a native Ceph block driver is not yet available. With this initial release, we are supporting VMWare ESX 6.5, Windows Server 2016, and RHV 4.x over an iSCSI interface, and you can expect to see us adding more platforms to the list of supported clients next year as we plan to increase the reach of our automated testing infrastructure.

¡Arriba, arriba! ¡Ándale, ándale!

Red Hat’s famous Performance and Scale team has revisited client-side caching tuning with the new codebase and blessed an optimized configuration for dm-cache that can now be easily configured with Ceph-volume, the new up-and-coming tool that is slated by the Community to eventually give the aging ceph-disk a well-deserved retirement.

Making things faster is important, but equally important is insight into performance metrics. The new dashboard is well deserving of a blog on its own right, but let’s just say that it plainly makes available a significant leap in performance monitoring to Ceph environments, starting with the cluster as a whole and drilling into individual metrics or individual nodes as needed to track down performance issues. Select users have been patiently testing our early builds with Luminous this summer, and their consistently positive feedback makes us confident you will love the results.

Linux monitoring has many flavors, and while we supply tools as part of the product, customers often want to integrate their existing infrastructure, whether it is Nagios alerting in very binary tones that something seems to be wrong, or another tool. For this reason, we joined forces with our partners at Datadog to introduce a joint configuration for SAAS monitoring of Ceph using Datadog’s impressive tools.

Get the stats

More than 30 features are landing in this release alongside our rebasing of the enterprise product to the Luminous codebase. These map to almost 500 bugs in our downstream tracking system and hundreds more upstream in the Luminous 12.2.1 release we started from. I’d like to briefly call attention to about 20 of them that our very dedicated global support team prioritized for us as the most impactful way to further smooth out the experience of new users and move forward on our march toward making Ceph evermore enterprise-ready and easy to use. This is our biggest release yet, and its timely delivery 3 months after the upstream freeze is an impressive achievement for our Development and Quality Assurance engineering teams.

As always, those of you with an insatiable thirst for detail should read the release notes next—and feel free to ping me on Twitter if you have any questions!

The third one’s a charm

Better economics through improved hardware utilization, great value-add for customers in the form of new access modes in file, iSCSI, and NFS compatibility, joined by improved usability and across-the-board technological advancement are the themes we tried to hit with this release. I think we delivered… but we aren’t done yet. We plan to send more stuff your way this year! Stay tuned.

But if you can’t wait to hear more about the new object storage features in Red Hat Ceph Storage 3, read this blog post by Uday Boppana.

Red Hat Ceph Storage: Object storage performance and sizing guide

Red Hat Ceph Storage is a proven, petabyte-scale, object storage solution designed to meet the scalability, cost, performance, and reliability challenges of large-scale, media-serving, savvy organizations. Designed for web-scale object storage and cloud infrastructures, Red Hat Ceph Storage delivers the scalable performance necessary for rich media and content-distribution workloads.

While most of us are familiar with deploying block or file storage, object storage expertise is less common. Object storage is an effective way to provision flexible and massively scalable data storage without the arbitrary limitations of traditional proprietary or scale-up storage solutions. Before building object storage infrastructure at scale, organizations need to understand how to best configure and deploy software, hardware, and network components to serve a range of diverse workloads. They also need to understand the performance and scalability they can expect from given hardware, software, and network configurations.

This reference architecture/performance and sizing guide describes Red Hat Ceph Storage coupled with QCT (Quanta Cloud Technology) storage servers and networking as object storage infrastructure. Testing, tuning, and performance are described for both large-object and small-object workloads. This guide also presents the results of the tests conducted to evaluate the ability of configurations to scale to host hundreds of millions of objects.

After hundreds of hours of [Test ⇒ Tune ⇒ Repeat] exercises, this reference architecture provides empirical answers to a range of performance questions surrounding Ceph object storage, such as (but not limited to):

  • What are the architectural considerations before designing object storage?
  • What networking is most performant for Ceph object storage?
  • What does performance look like with dedicated vs. co-located Ceph RGWs?
  • How many Ceph RGW nodes do I need?
  • How do I tune object storage performance?
  • What are the recommendations for small/large object workloads?
  • What should I do? I’ve got millions of objects to store.

And the list of questions goes on. You can unlock the performance secrets of Ceph object storage for your organization with the help of the Red Hat Ceph Storage/QCT performance and sizing guide.

Library of Ceph and Gluster reference architectures – Simplicity on the other side of complexity

The Storage Solution Architectures team at Red Hat develops reference architectures, performance and sizing guides, and test drives for Gluster- and Ceph-based solutions. We’re a group of architects who perform lab validation, tuning, and interoperability development for composable storage services with target workloads on optimized server and network configurations. We seek simplicity on the other side of complexity.

At the end of this blog entry is a full library of our current publications and test drives.

In our modern era, a top company asset is pivotability. Pivotability based on external market changes. Pivotability after unknowns become known. Pivotability after golden ideas become dark alleys. For most enterprises, pivotability requires a composable technology infrastructure for shifting resources to meet changing needs. Composable storage services, such as those provided by Ceph and Gluster, are part of many companies’ composable infrastructures.

Composable technology infrastructures are most frequently described by the following attributes:

  • Open source v. closed development.
  • On-demand architectures v. fixed architectures.
  • Commodity hardware v. proprietary appliances.
  • Cross-industry collaboration v. isolated single-vendor silos.

As noted in the following figure, a few companies with large staffs of in-house experts can create composable infrastructures from raw technologies. Their large investments in in-house expertise allows them to convert raw technologies into solutions with limited pre-integration by technology suppliers. AWS, Google, and Azure are all examples of DIY businesses. A larger number of other companies, also needing composable infrastructures, rely on technology suppliers and the community for solution pre-integration and guidance to reduce their in-house expertise costs. We’ll label them “Assisted DIY.” Finally, the majority of global enterprises lack the in-house expertise for deploying these composable infrastructures. They rely on public cloud providers and pre-packaged solutions for their infrastructure needs. We’ll call them “Pre-packaged.”

Brent_Slide

The reference architectures, performance and sizing guides, and test drives produced by our team are primarily focused on the “Assisted DIY” segment of companies. Additionally, we strive to make Gluster and Ceph composable storage services available to the “Pre-packaged” segment of companies by using what we learn to produce pre-packaged combinations of Red Hat software with partner hardware targeting specific workload use cases.

We enjoy our roles at Red Hat because of the many of you with whom we collaborate to produce value.  We hope you find these guides useful.

Team-produced with partner collaboration:

Partner-produced with team collaboration:

Pre-packaged solutions:

Hands-on test drives:

Jack of all trades: New Cisco UCS S-Series and Red Hat Storage

imagesToday, Cisco announced its new UCS S-Series storage-optimized server with the introduction of the UCS S3260, marking its entry into the emerging server market for data intensive workloads.

Red Hat and Cisco have worked together for a long time, including our collaboration on Red Hat OpenStack Platform.

Out with the old…

By jumping into the high-density storage-optimized server market, Cisco validates what we see as the continued movement to emerging software-defined, scale-out architectures for solutions like OpenStack and container-native storage and hyper-converged infrastructure.

With the ability to spread data across multiple servers, both Red Hat Ceph Storage and Red Hat Gluster Storage are helping to drive this trend. Open, software-defined storage enables enterprises to build an elastic cloud infrastructure for newer, data intensive workloads.

Ceph provides unified storage over a distributed object store (RADOS) as its core by providing unified block, object and file interfaces, while Gluster provides an elastic, scale out NAS file storage system.

As more organizations move to open source SDS from appliances / traditional SAN arrays, they often miss the recipes for a best practice deployment. Red Hat has worked with Cisco to produce reference design architectures to take the guess work out of configuring throughput-optimized, cost / capacity-optimized and emerging high IOPs performing clusters, including whitepapers for both Red Hat Ceph Storage and Red Hat Gluster Storage with Cisco’s previous generation of the S-Series, the C3160 high density rack server.

Open source drives storage innovation

Both Ceph and Gluster use community-powered innovation to accelerate their core feature sets faster than what is possible via a single proprietary vendor. Red Hat is a top contributor to both Ceph and Gluster upstream development, but several hardware, software and cloud service providers, including eBay, Yahoo!, CERN (Ceph) and Facebook (Gluster), all contribute to the code base. Cisco itself is a top-50 contributor to Ceph in terms of code commits.

Versatility

The Cisco UCS S-Series builds on the x86 storage-optimized server trend – but seemingly shuffles the deck with more of an enterprise spin via features such as dual-node servers, quadruple fans and power supplies, connected to Cisco UCS Fabric Interconnects.

One aspect of the new UCS S-Series design we are excited about is “versatility”. UCS offers common, consistent architecture for variety of IT needs that we expect may enable it to become a standard hardware building block for enterprise environments. S-Series includes features such as a modular chassis design, facilitating upgrades to new Intel chipsets including its disk expander module, providing the ability to swap out a server node for an additional 4 drives (increasing the raw capacity from 560 to 600 TB).

Cisco has also integrated networking fabric into its storage-optimized servers, making it easier to extend your interconnect as your cluster scales out. The S3260 offers dual 40GbE ports for each server node. As one moves to denser servers (with more than 24 drives) in Ceph configurations, the need for 40Gb Ethernet becomes greater. Enterprises can benefit from tightly-integrated fabric interconnect which translates to less latency, which is important for applications like video streaming.

A key piece is the UCS Manager configuration and handling tool which can simplify deployment. UCS Manager enables the creation of an initial configuration profile for storage, network, compute, etc. for the S3260, helping customers to more easily grow their Ceph environments by pushing out the profile to additional S3260s as they expand.

Combined with the Red Hat Storage ability to handle block, object and file access along with being flexible enough to handle throughput optimized, cost / capacity and high IOPS workloads, Cisco’s UCS S-Series may not just be a jack of all trades, but also a master of many.

Stay tuned for more upcoming joint solution papers from the Cisco UCS S3260 and Red Hat Ceph Storage teams. In the interim, learn more about the UCS S-Series at cisco.com/go/storage.

Latest OpenStack user survey shows Ceph continues to dominate

According to the OpenStack Foundation’s sixth and most recent user survey released just prior to this week’s OpenStack Summit in Tokyo, 62% of users selected Ceph RBD block storage for their OpenStack use cases, nearly three and more than four times the two closest alternatives, LVM (default) and NetApp, respectively.  In production, a full 38% of respondents indicated that their OpenStack deployments depended on Ceph as their Cinder driver, with the same comparisons. A survey of the largest production clouds, those exceeding 1,000 cores, showed similar results, with 37% of users selecting Ceph RBD followed by NetApp at 12%.  Interestingly enough, with 9% of respondents using GlusterFS in production, development & quality assurance, or proof of concept across all OpenStack deployments, more than 70% of OpenStack users are relying on block storage championed by Red Hat Storage.

Continue reading “Latest OpenStack user survey shows Ceph continues to dominate”

Ceph Deployment at Target: Best Practices and Lessons Learned

In October 2014, the first Ceph environment at Target, a U.S. based international chain of department stores, went live. In this insightful slide show (embedded at the end of this post), Will Boege, Sr. Technical Architect at Target, talks about the process, highlighting challenges faced and lessons learned in Target’s first ‘official’ Openstack release.

Continue reading “Ceph Deployment at Target: Best Practices and Lessons Learned”

Ceph’s on the move… and coming to a city near you!

What’s that saying: Ya can’t keep a good person down? Well, ya can’t keep a good technology contained—and that’s why Ceph’s been appearing at venues across the globe.

Ceph Day just hit Chicago

Most recently—this past August—Ceph made its way to Chicago, home of Chicago-style pizza and hot dogs, a place known worldwide for its Prohibition-era ruckus as well as its present-day spirits and brews. There, at Ceph Day Chicago, Bloomberg’s Chris Jones, senior cloud infrastructure, architecture/DevOps, explained how Ceph helps power storage at the financial giant.

Continue reading “Ceph’s on the move… and coming to a city near you!”

The Top 5 Q&As From Sage Weil’s Recent Reddit AMA

Picture1

Sage Weil, Red Hat’s chief architect of Ceph and co-creator of Cephamong many other credentials – recently held an “ask me anything” session on Reddit. Though you can read the whole thing for yourself, here, we’ve collected the top questions and answers for your edification. Read on!

Continue reading “The Top 5 Q&As From Sage Weil’s Recent Reddit AMA”

Check out our rich media deep-dive webinar – on demand!

Want to know more about rich media? In a recent webinar, Red Hat Senior Solutions Architect Kyle Bader took a deep dive into rich media and the unique demands it places on storage systems. We recap some highlights from the webinar here, but please register for the on-demand version here to get the full experience.

Continue reading “Check out our rich media deep-dive webinar – on demand!”

Test drive Red Hat Ceph Storage — for free

Want to try Red Hat Ceph Storage for yourself? Then it’s your lucky day. You can test drive it now. For free!

Available on Amazon Web Services (AWS) in a number of short, easily digestible labs, the Red Hat Ceph Storage test drive gives you a hands-on opportunity to explore the product’s features and see for yourself—in real time—how Red Hat Ceph Storage works. In addition to letting you explore the management features and fundamentals of Red Hat Ceph Storage, test drive labs walk you through such things as how to deploy a Ceph cluster and how to set up erasure-coded and replication storage pools.

Continue reading “Test drive Red Hat Ceph Storage — for free”

The Red Hat Ceph Storage/Supermicro reference architecture has arrived

If you’re building public or private clouds—or simply need massively scalable, flexible storage designed for the cloud—you’ve probably heard about Ceph. But what you may not have heard is that there’s now a reference architecture covering configurations and benchmarks results for Red Hat Ceph Storage on Supermicro storage servers. For real!

supermicro

Continue reading “The Red Hat Ceph Storage/Supermicro reference architecture has arrived”

Visit our Slideshare page for presentations from Red Hat Summit 2015 and everything you’ve ever wanted to know about storage

For easy access to a wealth of Red Hat Storage information, from product updates and use case insights to industry best practices, check out the Red Hat Storage Slideshare page. With presentations going back a few years you’ll be able to find just about anything – including the presentations used during select sessions at Red Hat Summit 2015. Below is just a sampling of some of the presentations you’ll find.

Continue reading “Visit our Slideshare page for presentations from Red Hat Summit 2015 and everything you’ve ever wanted to know about storage”