Comparing GlusterFS performance as a persistent storage layer for containerized applications on bare metal vs. PaaS deployments

By Amrik Jalif, Head of Storage UK&I, Red Hat

Container technology offers significant improvements in application density and deployment time. Containers have the versatility and power to do to virtualized environments what virtualization did to “a single server for every app.” Density goes up from 20 virtual machines per node to the equivalent of 50. Managing storage in a containerized world should be easy and automated. By containerizing the storage layer, developers have much more control over their environments.

In previous posts, we’ve outlined how Red Hat Storage can offer significant benefits to enterprises looking to deploy applications in containerized and PaaS environments, such as OpenShift Enterprise. We now have benchmark studies that compare performance between accessing GlusterFS from applications running on bare-metal x86 servers vs. from application containers in a PaaS environment, as in in OpenShift Enterprise.

We used the Flexible IO (FIO) tool in distributed mode as the workload generator for random and sequential workloads. We scaled OpenShift pods starting from 5 to 500, varying the file size and number of jobs in each pod while the dataset size was kept constant at 400GB. For a true “apples to apples” comparison, we ran distributed FIO on 6 OpenShift pods and on 6 bare-metal clients and scaled the pods from 5 to 1000 to push the limits on performance.

Thinking persistent storage for containers? Five reasons you need Red Hat Gluster Storage

By Irshad Raihan, Red Hat Storage

In a recent post, we announced that Red Hat Gluster Storage is now available as a Docker image in the Red Hat container registry. This represents the second important step toward our vision of storage as a service, delivered and managed alongside applications deployed in containers. The first step was its availability as a fully supported storage back-end for Red Hat OpenShift Enterprise 3.1 in a dedicated storage cluster, leveraging kubernetes GlusterFS volume driver.

As we get closer to a public announcement in the summer, please continue to watch this space for new developments and offer feedback on interesting use cases or trends around container adoption. Watch the following video on efforts underway at Red Hat to containerize the storage platform, from Luis Pabón, our lead engineer on the emerging technologies team.

Here are the top five reasons you’d consider Red Hat Gluster Storage over any other software-defined or traditional storage provider as a persistent storage layer for containers:

Welcome to Red Hat Storage Nation, SanDisk!

By Daniel Gilfix, Red Hat Storage

Wine Glasses

Those of you familiar with Red Hat Storage might remember that our mission is to offer, with our partners, a unified, open, software-defined storage portfolio for next-generation workloads that helps accelerate the transition to modern IT infrastructures. The recent news with SanDisk affirms our commitment to this mission. Our newly found friendship with one of the global leaders in flash storage is predicated upon extending Red Hat Ceph Storage to even further heights, and it demonstrates how open source software benefits not only from the innovation of those in the community but also from an open commercial ecosystem that traditional solutions are hard-pressed to duplicate.

Joining forces with another product giant

Kelpies

The Kelpies, at the Helix, Scotland, downloaded from https://en.wikipedia.org/wiki/The_Kelpies

On the heels of the Red Hat Ceph Storage 1.3.2 point release–one stealthily promoted yet proudly delivered with management, security, and performance upgrades–we’ve formalized a strategic alliance with SanDisk aimed at supporting the scalability and performance needs of enterprise customers. The combined solution, which incorporates Red Hat Ceph Storage and Red Hat Enterprise Linux with the SanDisk InfiniFlash™ system, has been optimized and tested to offer high density and peak performance at scale.

Taking aim at joint customer pain points

The motivation between such an alliance stems from a common understanding of what our customers need and how satisfying these needs is essential for next-generation storage in a software-defined datacenter. We’re talking about applications requiring high performance and low latency, like demanding OpenStack cloud platforms, large-scale Splunk log analytics, high-speed messaging servers, and read-intensive apps that process large amounts of data. We’re talking about customers deploying multiple storage solutions for different classes of applications at the risk of over-provisioning hard disk drives (HDDs) to meet SLAs and exceeding their capacity, space, power, and cooling capabilities. We’re talking about increasingly scrutinized budgets, cost containment, and the pressure to achieve economies of scale in datacenter operations.

One and one equals three!

Red Hat and SanDisk knew that our respective solutions delivered bona fide business benefits for storage customers, but together, we had the best of both worlds. We simply needed to document the configurations, certify compatibility, and provide basic training on our respective products to SanDisk Global Customer Care and Red Hat Global Support Services.

The result is nothing short of “performance that scales” in an integrated, single-platform solution. Red Hat Ceph Storage and SanDisk’s InfiniFlash System offer petabyte-scale capacity, high density, and performance, which can yield powerful cost-economies for our joint customers with big data storage requirements. The combination delivers the performance of an all-flash array with the economics of an HDD-based system and the scalability and versatility of a Red Hat Ceph Storage software-defined storage platform. Our customers can apply it not only to integrated OpenStack environments but a wide array of workloads in medium to large deployments.

We encourage customers to seek more information about this offering from their Red Hat or SanDisk sales reps and to consider how the combined benefits of Ceph and flash storage might help their organizations. At the same time, let’s give a shout out to our friends at SanDisk for joining us as part of the Red Hat Storage Nation and setting the bar even higher for next-generation, scale-out storage.

Seagull

Screaming Seagull by Vera Kratochvil, available on PublicDomainPictures.net
(http://www.publicdomainpictures.net/view-image.php?large=1&image=8345)

Enterprising businesses get to Red Hat Storage Days!

By E.G. Nadhan, Chief Technology Strategist, Red Hat

As businesses are increasingly being asked to cope with burgeoning amounts of valuable data, there’s a data explosion in the enterprise. This explosion fuels the need for a new approach to storage, which is the underlying theme of Mark Thiele’s article, 400 million new servers needed by 2020 — Think different or lose. “You aren’t ready to scale the way you’re going to need to scale,” he contends, noting that data sources are projected to reach 30 to 50 million by 2020. In his article, Are 18-month org charts and constant training the new reality of IT?, Curt Carver agrees, saying, “To plan for an IT future, you’ve got to be agile….”

So the drumbeat is growing louder: Digital enterprises need storage solutions that are agile and continuously evolving to accommodate dynamically changing requirements.

What’s an enterprising business to do?

To meet contemporary storage challenges head on, digital enterprises of the future should embrace open source and all it enables. To that end, Red Hat GM for Cloud Product Strategy, Bryan Che, penned an article titled, Open source is programming the digital future, noting that innovative companies like Uber and Facebook have embraced a digital approach to help ensure their long-term viability. Che’s article highlights the role of open source in this evolution as it contributes to the creation of new categories of functionality across enabling technologies.

Get to a Red Hat Storage Day!

So how do you get in on the action–and help position your storage solutions to meet the ever-changing, demanding needs of today’s IT infrastructures? Get to a Red Hat Storage Day!

Bloomberg Storage Day NYC 01.16

Bloomberg shared their insights at Red Hat Storage Day New York City in January 2016.

Designed as educational events in the full Red Hat Storage portfolio, Red Hat Storage Days give you an opportunity to both learn about innovative open source storage solutions firsthand and interface with some of the industry’s storage experts, as well as some of Red Hat’s partner companies.

Red Hat Storage solutions offer a variety of features to help you develop the storage solution you need. Following are a few:

  • Red Hat Storage embraces the fundamentals of software-defined storage.
  • Our solutions are highly scalable and provisioned in a self-healing, self-managing platform with no single point of failure.
  • We design our storage solutions to make it easier for you to migrate to the cloud by deploying across traditional, physical hardware or its virtualized simulation in the public or private cloud. Our goal is for your migration to be transparent to internal and external customers.
  • Red Hat Storage allows you to more efficiently store and manage incredibly large volumes of big, unstructured, and semistructured data in the cloud so you can focus on the timely availability of your data.

Register today!

So if your business–and its data–would benefit from this kind of storage insight, sign up today for our next Red Hat Storage Day: Red Hat Storage Day Minneapolis on Tuesday, April 12, 2016. Register here.

And if you can’t make that? No worries! We’re planning more Red Hat Storage Days all the time. You can also check out what’s coming to Yahoo headquarters on Wednesday, March 30: the next Ceph Day! Register here for Ceph Day Sunnyvale, the latest in a string of events designed to share Ceph’s transformative power and help foster the vibrant Ceph community.

Looking forward to seeing you at a Red Hat Storage event soon!

Latest release of Red Hat Gluster Storage enhanced with container support, file-based auto tiering, writable snapshots, and more

By Alok Srivastava, Senior Product Manager, Red Hat Gluster Storage and Data, Red Hat

Red Hat Gluster Storage 3.1.2 became generally available today, adding such key functionality as containerization and file-based auto tiering to its already feature-rich base. For detailed information on all the latest Red Hat Gluster Storage features and enhancements, see the documentation and read on here. For my interview on the topic, watch our video below.

 

 

Key feature enhancements

 

Red Hat Gluster Storage image on Red Hat’s container registry for persistent file storage for containerized applications

Persistent file-based storage from containerized Red Hat Gluster Storage can be consumed by containerized applications over the network (see following figure). You can quickly bring up a Red Hat Gluster Storage container in a Red Hat Atomic Host or on Red Hat Enterprise Linux by simply pulling a Red Hat Gluster Storage image from Red Hat’s container registry.

 

RHGS 3.1.2 (1)

 

File-based auto tiering to leverage innovation in flash and HDD design

The file-based auto tiering feature of Red Hat Gluster Storage 3.1.2, initially available only in tech preview and supported on Red Hat Enterprise Linux 7 based on Red Hat Gluster Storage, places frequently accessed or “hot” data on fast media (e.g., flash, SSDs) and seldom-accessed or “cold” data on higher capacity yet slower-spinning media (see following figure).

 

RHGS 3.1.2 (2)

 

Tiering combines SSDs and HDDs to enable intelligent data placement and thereby boost performance and cost efficiencies. For example, a new or an existing volume may be distributed (erasure coded) on HDDs and the hot tier could be distributed on SSDs. The “attach” operation of tiering, transparent to applications, converts existing volumes into a cold tier. Together, the combination is one tiered volume.

 

Writable snapshots with shared backend LVM store

This release of Red Hat Gluster Storage supports writable snapshots–clones of snapshotted volumes with read/write permissions. Until today, only thin, LVM-based, read-only snapshots were available in the product. The writable clones, which are space efficient and instantly created, share the backend LVM store with their parent snapshots (see following figure).

 

RHGS 3.1.2 (3)

 

Other feature enhancements

 

Bit rot detection status

The bit rot detection functionality in Red Hat Gluster Storage periodically scans all data bricks in the product, compares checksums, and identifies and sometimes proactively fixes corrupted files. This release augments this feature so that you can identify corrupted files easily and take any necessary corrective action.

Faster SMB access

SMB shares back-ended by Red Hat Gluster Storage now perform better for streaming workloads. With this release, asynchronous IO from Samba to Gluster is supported and the relevant option is enabled by default in smb.conf. We have seen up to 100% performance increase from the previous release in our performance labs!

Offline console installation

Offline installation of the Red Hat Gluster Storage console is now enabled by offering an OVA image. You can now import RHGS-C OVA image to Red Hat Enterprise Virtualization Manager and install the console s/w on virtual machines.

Red Hat Gluster Storage now available in Google Cloud Platform

By Sayan Saha, Head of Product Management, Red Hat Gluster Storage and Data,
Red Hat

Today we announced the availability of Red Hat Gluster Storage in Google Cloud Platform as a fully supported offering. Red Hat Gluster Storage will give Google Cloud Platform users the ability to use a scale-out, POSIX-compatible, massively scalable, elastic file storage solution with a global namespace.

This offering will bring existing users of Red Hat Gluster Storage another supported public cloud environment in which they can run their POSIX-compatible file storage workloads. For their part, Google Cloud Platform users will have access to Red Hat Gluster Storage, which they can use for several cloud storage use cases, including active archives, rich media streaming, video rendering, web serving, data analytics, and content management. POSIX compatibility will give users the ability to move their existing on-premise applications to Google Cloud Platform without having to rewrite applications to a different interface.

 

RHGS_Google

Enterprises can also migrate their data from an on-premise environment to the Google Cloud Platform, easily leveraging the geo-replication capabilities of Red Hat Gluster Storage.

A Red Hat Gluster Storage node in Google Cloud Platform is created by attaching Google standard persistent disks (HDD) or solid-state persistent disks (SSD) to a Google Compute Engine (GCE) instance. Two or more such nodes make up the trusted storage pool of storage nodes. To help protect against unexpected failures, the Red Hat Gluster Storage nodes that constitute the trusted storage pool should be instantiated across Google Cloud Platform’s zones (within the same region), up to and including a single zone.

Gluster volumes are created by aggregating available capacity from Red Hat Gluster Storage instances. Capacity can be dynamically expanded or shrunk to meet your changing business demands. Additionally, Red Hat Gluster Storage provides geo-replication capabilities that enable data to be asynchronously replicated from one Google Cloud Platform region to another, thereby enabling disaster recovery for usage scenarios that need it in a master-slave configuration.

Anticipated roadmap features like file-based tiering in Red Hat Gluster Storage include providing the capability to create volumes with a mix of SSD- and HDD-based persistent disks providing storage tiering (hierarchical storage management) in the cloud in a transparent manner.

Red Hat Gluster Storage in Google Cloud Platform will be accessed using the highly performant Gluster native (Fuse-based) client from Red Hat Enterprise Linux 6, Red Hat Enterprise Linux 7, and other Linux-based clients. Users may also use NFS or SMB.

We are excited that users will be able to take advantage of all the Red Hat Gluster Storage features in Google Cloud Platform, including replication, snapshots, directory quotas, erasure coding, bit-rot scrubbing and geo-replication, because they now have a compelling option for their scale-out file storage use cases in Google’s cloud.

Red Hat Ceph Storage captures throne

For the second straight year, Red Hat Storage has received the prestigious Brand Leader award for Scale-out Object Storage Software by IT Brand Pulse. The 2015 winners were selected by IT professionals in an independent, non-vendor-sponsored survey of IT Brand Pulse’s million members and 100+ online IT groups.

Red Hat Storage Award

The importance of partners

As we approach the holiday season, our latest partner announcement is appropriate. Why? Because, partner-wise, Red Hat Storage is setting up for a veritable feast.

Storage is an ingredient, not a meal

Let’s face it. Nobody ever deploys storage technologies by themselves. They’re always deployed alongside hardware platforms and workloads–they have to be. Without hardware platforms to run on top of and applications needing their services, storage technologies don’t really do anything.

Availability of Red Hat Gluster Storage in Microsoft Azure

Sayan Saha, head of Product Management, Red Hat Gluster Storage and Big Data, Red Hat

Today, we announced our plans to make several Red Hat offerings, including Red Hat Gluster Storage, available in Microsoft Azure as fully supported offerings. Red Hat Gluster Storage offers Azure users a scale-out, POSIX compatible, massively scalable, elastic file storage solution with a global namespace.

Why Software Defined Storage is set to disrupt the world of containers. And why you should care.

Containers have the potential to be hugely disruptive – they are about to impact almost every process and person within the data center. Container technology will also impact how we think about storage for applications and microservices. In turn, software defined storage will impact how storage is dynamically provisioned and managed for containerized applications.

Latest OpenStack user survey shows Ceph continues to dominate

According to the OpenStack Foundation’s sixth and most recent user survey released just prior to this week’s OpenStack Summit in Tokyo, 62% of users selected Ceph RBD block storage for their OpenStack use cases, nearly three and more than four times the two closest alternatives, LVM (default) and NetApp, respectively.  In production, a full 38% of respondents indicated that their OpenStack deployments depended on Ceph as their Cinder driver, with the same comparisons. A survey of the largest production clouds, those exceeding 1,000 cores, showed similar results, with 37% of users selecting Ceph RBD followed by NetApp at 12%.  Interestingly enough, with 9% of respondents using GlusterFS in production, development & quality assurance, or proof of concept across all OpenStack deployments, more than 70% of OpenStack users are relying on block storage championed by Red Hat Storage.

Address security challenges with software-defined storage

Hand-in-hand with the explosive growth of data and iCloud services comes a range of security threats, all of which can be addressed with software-defined storage. To learn how, register for the following webinars – the link is at the bottom of the post!