HOWTO: GlusterFS + ZFS on Linux

Continuing with the theme of unearthing useful tidbits on the internet, I came across a post from Giovanni Toraldo about using GlusterFS with ZFS on Debian/Ubuntu Linux. Why would someone want to do this? With OpenSolaris’ future essentially over, ZFS’s future is on Linux, and there has been significant headway on the ZFS on Linux project. Disk filesystems like ZFS go hand-in-hand with GlusterFS, because they each bring different features to the table. See last week’s post on [intlink id=”4807″ type=”post”]building a failover SAN with deduplication[/intlink].

Read the full article at gionn.net.

HOWTO: Create a Fail-Over SAN Cluster With Thin Provisioning, Dedup, and/or Compression

Ever wondered how to roll your own fail-over SAN cluster with off-the-shelf open source software? I found this handy HOWTO from El Di Pablo who blogs at Bauer-Power.net. What motivated him to do this?

I was getting discouraged with setting up a redundant failover SAN cluster because I found that DRBD was just too flaky in the setup I wanted. The problem though was that almost all home-grown cluster solution tutorials on the web use DRBD with either Heartbeat or Corosync. It is almost impossible to find a different, or even better solution… Until now of course.

So since DRBD doesn’t work… I decided to look elsewhere. Enter GlusterFS!

I like it way better than DRBD because it doesn’t require meta-data to sync before it will work. It just works! In fact, once it is setup it writes like a true RAID 1. If you put a file in your sync’d directory it will automagically show up on the other node in the cluster in real time!

Ok, so I figured out how to make clustering work with GlusterFS and Heartbeat. What’s this about deduplication and thin provisioning? Yes! I got that working as well. In fact, not only can we do deduplication, we can do compression if we want. How? It’s all thanks to the miracle that is ZFS!

So there you have it – read the full HOWTO to see how he stretches GlusterFS over ZFS for a full-featured fail-over SAN. This is exactly the type of ingenuity and innovation made possible by open source software.

Gluster for Geeks Webinar: Performance Tuning Tips & Tricks

It’s that time again! Time for another Gluster for Geeks webinar.

Webinar will be on Thursday, August 25, 10am PDT / 1pm EDT / 17:00 UTC

In this Gluster for Geeks technical webinar, Jacob Shucart, Senior Systems Engineer, will provide useful tips and tricks to make your Gluster cluster meet your performance requirements.  He will review considerations for all different phases including planning, configuration, implementation, tuning, benchmarking, and trouble shooting.

Register here.

Big data SMAQ-down

The term "big data," is getting thrown around a lot these days, and in certain circles it is threatening to overtake "cloud" as the most overused and misused term in IT.

Interestingly, some of the large, traditional storage vendors are embracing the term big data, using it as an umbrella term for all large collections of data and hence an umbrella term for all of their offerings. A more nuanced understanding of big data actually shows it to be antithesis of both the technology and the business models of the traditional storage vendors.

read more

A Crucial Piece of the Hadoop Ecosystem: Scale-out Storage

By now you may have heard our announcement that GlusterFS 3.3 will be able to serve as a scale-out data store for Apache Hadoop. This is one area where a solution has been sorely needed, and we’re happy to provide one. Here’s the relevant piece from the press release:

Compatible with existing MapReduce-based applications and with the ability to co-exist with HDFS, this new functionality opens up data within Hadoop deployments to any file- or object-based application, providing a broad new range of options for big data deployments.

Some of the benefits for Hadoop users include:

  • Fast access to data – with no centralized metadata or name server, GlusterFS delivers better performance and easier manageability, with no centralized point of failure;
  • N-way replication – synchronous and asynchronous replication enables users to replicate data, changes are tracked and queued to ensure data stays synchronized regardless of latency or potential network interruptions;
  • High-availability – GlusterFS includes built-in failover, self-healing and no single point of failure;
  • Increased flexibility in sizing – with Gluster, Hadoop deployments are no longer limited to large block sizes or 16 TB volume sizes;
  • POSIX-compliant NAS (NFS, CIFS) and object access to files within Hadoop deployments – with Gluster, big data deployments can now enable operating systems to directly mount the storage, making it much easier to get data into and out of the system;
  • Easy access – to both file and object storage which simplifies management and controls storage costs; and,
  • Open source – available under a GPLv3 open source license.

Want to learn more?

Gluster will host a meetup on the topic of GlusterFS 3.3 at Gluster HQ in Sunnyvale on August 30, beginning at 6:00 pm PDT. Topics will include storage for Hadoop, as well as Unified File and Object storage.

We will also host a webinar, “What’s new in GlusterFS 3.3” on Tuesday, August 30 at 2 p.m. ET. To register for the event visit https://www3.gotomeeting.com/register/155968278 – this is a good way to interact with Gluster experts and learn more about our recent announcements.

Availability
The Hadoop GlusterFS 3.3 beta 2 is available now. To participate in the public beta simply visit our 3.3 beta resource page: http://bit.ly/nUtssL. To sign up as an official beta site, contact glusterfs3.3beta@gluster.com or info@gluster.com.

Gluster Announces Apache Hadoop Storage Compatibility in Latest GlusterFS Release

Expands data access options with addition of open source, scale-out NAS and object-based file access for Hadoop deployments

Sunnyvale, Calif. – August 23, 2011 – Gluster, the leading provider of scale-out, open source storage solutions, today announced the release of GlusterFS 3.3 beta 2 which includes compatibility for Apache Hadoop. This latest enhancement to GlusterFS uses the standard file system APIs available in Hadoop to provide a new storage option for Hadoop deployments. With this new feature, GlusterFS is the first open source, POSIX-compliant file and object storage solution that works with Hadoop. Compatible with existing MapReduce-based applications and with the ability to co-exist with HDFS, this new functionality opens up data within Hadoop deployments to any file- or object-based application, providing a broad new range of options for big data deployments.

“As an open source, distributed storage solution designed to work on huge numbers of heterogeneous, commodity devices—Gluster is a perfect complement to the Hadoop ecosystem,” said Ben Golub, CEO of Gluster. “With this release organizations, can ensure their long term storage infrastructure is aligned with their long term, “big data” infrastructure.”

Apache Hadoop is an open source software framework that supports data-intensive distributed applications, such as search analytics, financial modeling, atmospheric science, genomics research and more. The Hadoop approach distributes both data and processing to large numbers of devices, enabling huge datasets to be efficiently managed and analyzed in parallel. Given this design, a petascale, open source, software only, distributed approach to storage is a far better fit for Hadoop deployments than proprietary, monolithic, big box approaches.

Eric Baldeschwieler, CEO at Hortonworks, a company formed by the key architects and core Hadoop committers from the Yahoo! Hadoop software engineering team in order to accelerate the development and adoption of Apache Hadoop said, “Hortonworks is dedicated to ensuring Apache Hadoop becomes the standard platform for storing, processing, managing and analyzing big data. We are excited that Gluster is embracing Apache Hadoop and delivering incremental value to enterprises deploying Hadoop solutions by unifying their storage environments into a single scale-out pool.”

GlusterFS provides Hadoop deployments with a proven scale-out NAS solution in production at petabyte scale. Features and benefits include:

• Fast access to data – with no centralized metadata or name server, GlusterFS delivers better performance and easier manageability, with no centralized point of failure;
• N-way replication – synchronous and asynchronous replication enables users to replicate data, changes are tracked and queued to ensure data stays synchronized regardless of latency or potential network interruptions;
• High-availability – GlusterFS includes built-in failover, self-healing and no single point of failure;
• Increased flexibility in sizing – with Gluster, Hadoop deployments are no longer limited to large block sizes or 16 TB volume sizes;
• POSIX-compliant NAS (NFS, CIFS) and object access to files within Hadoop deployments – with Gluster, big data deployments can now enable operating systems to directly mount the storage, making it much easier to get data into and out of the system;
• Easy access – to both file and object storage which simplifies management and controls storage costs; and,
• Open source – available under a GPLv3 open source license.

“GlusterFS is a natural fit for the Hadoop platform because it was designed as a petabyte scale file system for commodity hardware,” said AB Periasamy, co-founder and CTO of Gluster. “For Hadoop deployments, GlusterFS delivers on our overall theme of unifying storage environments into one single scale-out pool. We are delivering Hadoop users with a NAS storage alternative that seamlessly integrates and opens up the data to more users and applications while providing reliability and high performance.”

Gluster will be hosting a webinar, “What’s new in GlusterFS 3.3” on Tuesday, August 30 at 2 p.m. ET. To register for the event visit https://www3.gotomeeting.com/register/155968278

Product Availability
GlusterFS 3.3 beta 2 is available now. To participate in the public beta simply visit our 3.3 beta resource page: http://bit.ly/nUtssL. To sign up as an official beta site, contact glusterfs3.3beta@gluster.com or info@gluster.com.

Supporting Resources
Follow Gluster on the web for the latest news and information at:

About Gluster
Gluster is the leading provider of open source storage solutions for public, private and hybrid clouds. Over 150 enterprises worldwide have used Gluster in commercial deployments ranging from a few terabytes to multiple petabytes, across the most demanding applications in digital media delivery, healthcare, Internet, energy and biotech. Gluster is privately-held and headquartered in Sunnyvale, California. Visit us at www.gluster.com.

# # #
Media Contact:
Danielle Tarp
Mindshare PR for Gluster
650-947-7405
Danielle@mindsharepr.com

Gluster Named as One of the Top 25 Coolest Emerging Vendors in 2011 by CRN

Leading storage provider chosen for its open source, software-only, scale-out NAS storage solution for on-premise, virtual, public and private cloud environments

Sunnyvale, Calif. – August 22, 2011 – Gluster, the leading provider of scale-out, open source storage solutions, today announced the company has been selected by CRN as one of the “25 Coolest Emerging Vendors” in 2011. Gluster is one of the hot startups handpicked from 170 companies to be part of CRN’s Emerging Vendors 2011 list, an annual report that names the most innovative emerging vendors in the technology industry. To be included on CRN’s Emerging Vendors list, companies must have a strong strategy to leverage the channel.

Over the past year, Gluster has built and launched a channel strategy from the ground up, signing more than 40 new partners. The Gluster channel team works hard to define the right partner types, routes to market, contribution mix, and partner profiles and to ensure readiness tools such as sales, marketing and services are provided to Gluster partners.

Gluster’s software-only, scale-out NAS solutions let enterprises deploy storage the same way they deploy computing today–as a scale-on-demand pool running on commodity hardware, radically improving storage economics. Combined with the customer’s choice of physical or virtual resources, Gluster can scale-out to petabytes of capacity and GB/s of throughput at a fraction of the cost of proprietary systems. Gluster is the primary author and maintainer of the open-source GlusterFS software, which has been downloaded over 200,000 times.

“Gluster is honored to be named one of CRN’s 25 coolest emerging vendors for 2011. We are among an amazing group of innovative companies all changing the face of technology,” said Lori Budin, VP of channels and partnerships at Gluster. “Recognition like this validates our commitment to providing our partners with the next-generation of storage solution that allow them to deliver customers cost-effective, highly scalable storage for their private and public cloud deployments.”

To view the full list of CRN’s 2011 Emerging Vendors winners visit here, and the complete list of the 25 Coolest Emerging Vendors for 2011 can be viewed here.

Supporting Resources
Follow Gluster on the web for the latest news and information at:

About Gluster
Gluster is the leading provider of open source storage solutions for public, private and hybrid clouds. Over 150 enterprises worldwide have used Gluster in commercial deployments ranging from a few terabytes to multiple petabytes, across the most demanding applications in digital media delivery, healthcare, Internet, energy and biotech. Gluster is privately-held and headquartered in Sunnyvale, California. Visit us at www.gluster.com.

# # #
Media Contact:
Danielle Tarp
Mindshare PR for Gluster
650-947-7405
Danielle@mindsharepr.com

Holy Webinars, Batman! Gluster for Geeks and More…

Never let it be said that we don’t provide avenues of education for Gluster customers. We have some very interesting webinars coming up that are worth checking out. From cloud storage with Nimbula and intro to GlusterFS, to performance tuning tips and tricks, there’s something for everyone.

Here is our webinar schedule for the rest of August:

August 17 – Scalable Cloud Storage Solutions with Nimbula & Gluster

5pm PDT Registration – 8pm EDT / 00:00 UTC (today!)
August 25 – Gluster for Geeks: Performance Tuning Tips & Tricks
Registration Web Link: https://www3.gotomeeting.com/register/939735854
August 26 – Introduction to GlusterFS
Registration Web Link: https://www3.gotomeeting.com/register/974216486

Introducing the Brand-new Gluster.org

If you regularly visit gluster.org – and really, why *wouldn’t* you? – you may have noticed something. We changed the design and layout. Here’s a list of amazing things that are different from the old site:

  • It looks nicer. Ok, so “nicer” is not an objective statement, but let’s be honest here, the old gluster.org wasn’t exactly easy on the eyes
  • Brings our front and center. From our twitter feed, to blog posts and community Q&A, the front page of gluster.org gives you a birds-eye view of what’s taking place in the GlusterFS community
  • Prominent search interface. Right now, the search bar at the top of the page pulls content from our community Q&A site. We will expand that in future iterations to include all documents.
  • Easier navigation. The primary avenues of interaction in the community are 1 click away.

This is just the first of a general community web site overhaul. There’s a lot more where this came from, such as a guide to using the site, community contests, a forge area for your projects and much, much more. Feel free to add a comment below and tell us what you would like to see.

Visit the new Gluster.org

LinuxCon: Vancouver, here we come!

LinuxCon Logo
LinuxCon North America 2011

If you’re going to Vancouver for LinuxCon, you’ll have not one but *two* opportunities to see us live and in the flesh. On Thursday, August 18, at 5:15pm, we will be hosting a birds-of-a-feather session to discuss all sorts of fun things: GlusterFS roadmap, integrations with OpenStack and Hadoop, and whatever else you want to know about as it pertains to GlusterFS. AB Periasamy, Gluster co-founder and CTO, will be in attendance, which gives attendees a great chance to get the straight dope directly from the source.

Next, AB will present at 10:15am on bringing intelligent storage to OpenStack. This will focus on our ability to be the storage layer for compute VM’s, as well as a way to use GlusterFS as the unified file and object storage for OpenStack-based applications.

This comes on the heels of our joining the Linux Foundation, so look for us at more events in the future.

Update on GlusterFS 3.3 Beta – Unified File and Object Storage

On July 20th we announced and delivered the first beta of  GlusterFS 3.3, which featured unified file and object storage. I am excited to give you an update on how things are going with the beta and how we see our customers deploying this innovative file and object storage technology.

At a glance – use cases for GlusterFS 3.3:

  • Can eliminate the need for gateway devices bridging the data center to the cloud.
  • Back up email for Exchange environments
  • Global storage for photo processing and archiving
  • Easier onramp to cloud storage
  • On-premise, S3-like object storage for private cloud

Both our public beta testing and testing at select enterprise customer sites continues on schedule, and unique deployments of this innovative technology are being crafted which leverage the unified file and object storage capability. There is still space available in the private beta and if you are interested in participating then email us at glusterfs3.3beta@gluster.com.

Email Backups On-premise and in the Cloud

I want to share with you some of the ways in which our unified file and object storage can be used. One way is for email back-up and archive. Exchange environments can back up to a GlusterFS NAS target and then move the backup files out as objects to S3 – in many cases this is eliminating the need for gateway devices bridging the data center to the cloud.

Data Processing and Archiving – Photos

Another example is digital photo processing, editing and archiving where photos are stored in files as well as objects simultaneously within GlusterFS. The uniqueness of GlusterFS is that the object can be accessed from any application that accesses objects for other business purposes, from any cloud. In some cases this includes other photo processing applications (editing items such as photo attributes and color corrections). This is an example where Gluster provides file and object storage as well as an object archive.

Easier Onramp to Public Cloud Storage

Also, we see customers who have deployed our NAS in the cloud via Amazon Web Services, or GoGrid, developing creative solutions that export objects from the data center to the cloud via HTTP (lower protocol overhead as compared to NFS and CIFS) and directly into GlusterFS unified file and object in the cloud. By moving information quickly, and having it immediately available for processing, customers save both time and money and will improve their overall operational efficiency. For some customers this means faster time to-market for their products and for others it means faster report generation and speedier, more well informed business decisions. We’re pleased to be able to further accelerate our customers’ ability to move to the cloud.

On-premise S3-like Object Storage

Additionally, for those customers who have been looking for an on-premise, cost effective, object storage system, they now have the best of both worlds – highly available, high performance, unified file and object storage deployable in a cost effective manner. On-premise customers can now deploy an S3-like private cloud storage capability on commodity hardware.

Contact us to learn more about leveraging unified file and object storage in your environment.

All the best,

Tom Trainer

Gluster CTO to Discuss How to Bring Intelligent Storage to OpenStack at LinuxCon North America

Open source expert to present code examples and use cases of storage in OpenStack deployments; BoF on innovative file system GlusterFS

VANCOUVER — LinuxCon — August 15, 2011 – Gluster, the leading provider of scale-out, open source storage solutions, today announced its co-founder and chief technology officer, Anand Babu (AB) Periasamy, will be among the industry experts presenting at LinuxCon, taking place August 17 – 19 at the Hyatt Regency in Vancouver. On Friday, August 19 from 10:15 a.m. to 11:00 a.m., Periasamy will present “Bringing Intelligent Storage to OpenStack.” Additionally, John Mark Walker, Gluster’s community director, will host a GlusterFS Birds-of-a-Feather (BoF) on Thursday, August 18 at 5:15 p.m.

During Periasamy’s presentation he will discuss the pain points of storage in cloud computing environments and specifically address the benefits of using OpenStack and Gluster. Additionally, the discussion will feature code examples and use cases. Prior to Gluster, Periasamy served as CTO at California Digital Corporation and, in 2004, led the development of the world’s second-fastest supercomputer, “Thunder,” for Lawrence Livermore National Laboratory. Periasamy also serves on the board of “Free Software Foundation – India.”

“OpenStack has experienced blinding success over the past year, with many developers, system administrators and companies participating in its ecosystem,” said Periasamy. “We have been collaborating with the OpenStack community to determine the largest pain points and storage is one of them. My discussion will outline how Gluster provides highly-scalable and highly available VM storage functionality for OpenStack.”

The GlusterFS BoF will be an opportunity for LinuxCon attendees to discuss and learn more about the GlusterFS roadmap, recent integration work and recipes for deploying at scale. Periasamy will be on hand to share his expertise and answer audience questions.
For more information and to register for LinuxCon, visit: http://events.linuxfoundation.org/events/linuxcon

Supporting Resources
Follow Gluster on the web for the latest news and information at:

About Gluster Technology
Gluster’s software-only solutions let enterprises deploy storage the same way they deploy computing today–as a virtualized, commoditized, and scale-on-demand pool, radically improving storage economics. Combined with the customer’s choice of commodity computing and storage resources, Gluster can scale-out to petabytes of capacity and GB/s of throughput at a fraction of the cost of proprietary systems. Gluster ensures high availability with n-way replication both within and between public and private data centers. Gluster is deployable both on-premise (as a virtual appliance or bare-metal software appliance) and in public clouds such as Amazon Web Services. Gluster is the primary author and maintainer of the open-source GlusterFS software, which has been downloaded over 200,000 times.

About Gluster
Gluster is the leading provider of open source storage solutions for public, private and hybrid clouds. Over 150 enterprises worldwide have used Gluster in commercial deployments ranging from a few terabytes to multiple petabytes, across the most demanding applications in digital media delivery, healthcare, Internet, energy and biotech. Gluster is privately-held and headquartered in Sunnyvale, California. Visit us at www.gluster.com.

# # #
Media Contact:
Danielle Tarp
Mindshare PR for Gluster
650-947-7405
Danielle@mindsharepr.com

  • Page 1 of 2
  • 1
  • 2
  • >