IDC Report: Storage Software to Hit $13.8B in 2011

It’s a good day to be in the storage software business, it seems. IDC just released a new report showing that the storage software business will grow by 7.8% in 2011, with  $13.8 billion in revenue. This is due to “pent up demand” and IT budgets finally getting approved this year, after a couple of years of stagnation or decline:

IDC’s research director storage software, Eric Sheppard, said the storage software market spent 2010 recovering from a difficult downturn, but “fortunately, there was plenty of pent-up demand and new product innovations to help create momentum in the market.”

“Looking forward, demand for storage software appears poised to remain strong around the world as organizations continue to address inefficiencies related to storing, protecting, and managing corporate data,” Sheppard added.

Promising stuff. For Gluster, this is good for 2 reasons:

  1. There’s more $$$$ being thrown at storage software solutions, and
  2. This market is still ripe for disruption

One need only look at the outlandish fees charged by established players for inflexible, complicated storage software to know that there’s room for companies, like Gluster, willing to deliver more value for a lower price.

Linus Torvalds doesn't understand user-space filesystems

I was directed to a recent mailing list post by Linus Torvalds on linux-fsdevel in which he derided the concept of user-space filesystems. Not a particular implementation, mind you, but the very concept of it.

Jeff Darcy, of Red Hat and CloudFS fame, wrote a wonderful response, which you should read first before continuing further.

From my perspective, as the creator of GlusterFS, Linus is rather blinkered on this issue. The fact is, user space advantages far outweigh kernel space advantages. You’ll notice that Linus pointed to no benchmarks or studies confirming his opinion, he merely presented his bias as if it were fact. It is not.

Hypervisors are the modern micro kernels. Microkernel is not about size, but about what should be in kernel mode. Linus’s ideas about filesystems are rather old. He thinks that it is a bad idea to push the filesystems to user space, leaving the memory manager to run in kernel mode. The bulk of the memory buffers are filesystem contents, and you need both of them to work together.  This is true for root filesystems with relatively small amounts of data but not true when it comes to scalable storage systems. Don’t let the kernel manage the memory for you. In my opinion, Kernel-space does a poor job of handling large amounts of memory with 4k pages. If you see the bigger picture, disks and memory have grown much larger, and user requirements have grown 1000-fold. To handle today’s scalable, highly available storage needs, filesystems need to scale across multiple commodity systems, which is much easier to do in user space. Real bottlenecks come from the network/disk latencies, buffer-copying and chatty IPC/RPC communications. Kernel-user context switches are hardly visible in the broader picture, thus whatever performance improvements it offers are irrelevant. Better, then, to use the simpler, easier methods offered in user-space to satisfy modern storage needs. Operating systems run in user-space in virtualized and cloud environments, and kernel developers should over come this mental barrier.

Once upon a time, Linus eschewed microkernels for a monolithic architecture for sake of simplicity. One would hope that he would be able to grasp the reasons why simplicity wins in this case, too. Unfortunately, he seems to have learned the wrong lesson from the microkernel vs. monolithic kernel debates: instead of the lesson being that all important stuff gets thrown into the kernel, it should have been that simplicity outweighs insignificant improvements elsewhere. We have seen this in the growth of virtualization and cloud computing, where the tradeoff between new features and performance loss has proved to be irrelevant.

There are bigger issues to address. Simplicity is *the* key to scalability. Features like online self-healing, online upgrade, online node addition/removal, HTTP based object protocol support, compression/encryption support, HDFS APIs, and certificate based security are complex in their own right. Necessitating that they be in kernel space only adds to the complexity, thus hampering progress and development. Kernel mode programming is too complex, too restrictive and unsustainable in many ways. It is hard to find kernel hackers, hard to write code and debug in kernel mode, and it is hard to handle hardware reliability when you scale out due to multiple points of failure.

GlusterFS got its inspiration from the GNU Hurd kernel. Many years before, GNU Hurd was able to mount tar balls as a filesystem, FTP as a filesystem, and POP3 as an mbox file. Users could extend the operating system in clever ways. A FUSE-like user space architecture was an inherent part of the Hurd operating system design. Instead of treating filesystems as a module of the operating system, Hurd treated Filesystems as the operating system. All parts of the operating system were developed as stackable modules, and Hurd handled hardware abstraction. Didn’t we see the benefits of converging the volume manager, software RAID and filesystem in ZFS? GNU Hurd took it a step further, and GlusterFS brought it to the next level with Linux and other Unix kernels. It treats the Linux kernel as a microkernel that handles hardware abstraction and broaches the subject that everyone is thinking, if not stating outloud: the cloud is the operating system. In this brave new world, stuffing filesystems into kernel space is counter-productive and hinders development. GlusterFS has inverted the stack, with many traditional kernel space jobs now handled in user space.

In fact, when you begin to see the cloud and distributed computing as the future (and present), you realize that the entire nomenclature of user space vs. kernel space is anachronistic. In a world where entire operating systems sit inside virtualized containers in user space, what does it even mean to be kernel space any more? Looking at the broader trends, arguing against user space filesystems is like arguing against rising and falling tides. To suggest that nothing significant is accomplished in user space is to ignore all major computing advances of the last decade.

To solve 21st-century distributed computing problems, we needed 21st-century tools for the job, and we wrote them into GlusterFS. GlusterFS manages most of the operating system functionality within its own user space, from memory management, IO scheduling, volume management, NFS, and RDMA to RAID-like distribution. For memory management, it allocates large blocks for large files, resulting in far fewer page table entries, and it is easier to garbage collect in user space. Similarly with IO scheduling, GlusterFS uses elastic hashing across nodes and IO-threads within the nodes. It can scale threads on demand and group blocks belonging to the same inodes together, eliminating disk contention. GlusterFS does a better job of managing its memory or scheduling, and the Linux kernel doesn’t have an integrated approach. It is user-space storage implementations that have scaled GNU/Linux OS beyond petabytes seamlessly. That’s not my opinion, it’s a fact: the largest deployments in the world are all user-space. Whats wrong with FUSE simplying filesystem development to the level of toy making? 🙂

Some toys are beautiful and work better than others.

Gluster and the Open Virtualization Alliance

Open Virtualization Alliance

You may have noticed our press release that we joined the Open Virtualization Alliance. So what is the Open Virtualization Alliance (OVA)? According to their web site:

The Open Virtualization Alliance’s goal is to help:

  • Increase overall awareness and understanding of Kernel-based Virtual Machine (KVM).
  • Foster the adoption of KVM as an open virtualization alternative to proprietary solutions.
  • Accelerate the emergence of an ecosystem of third-party solutions around KVM.
  • Encourage interoperability, promote best practices, and highlight examples of customer successes.

Oh, and did we mention KVM? At Gluster, we love free and open. It’s what we do. So when we saw the OVA was forming, we jumped at the chance.

Of course, we will not change our support for other virtualization platforms in any way. This is a way for us to publicly support a good, open initiative, and I strongly suspect we’ll announce more of these in the future. We can’t share what they are yet, unfortunately, but rest assured there are some exciting things afoot here at Gluster headquarters.

Brightcove announced as customer + case study

Brightcove Case Study

This week, we announced that Brightcove is using its GlusterFS-based storage solution to power video for tens of thousands of websites. Brightcove’s Video Cloud online video platform is the platform of choice for media companies, businesses, and organizations in over 50 different countries, offering everything from mobile video solutions to third-party player plug-ins, innovative ad formats, and pre-built integrations with CMS systems, ad servers and ad networks.

Brightcove now joins Pandora and one other large media service (to be named later) as Gluster customers. GlusterFS is quickly becoming the storage platform of choice for content delivery networks.

Read the Brightcove case study here.

Webinar: Introduction to GlusterFS

Looking for a high performance, scale-out NAS file system? Or are you a new user of GlusterFS and want to learn more? Join us for a webinar on Thursday June 30, for an introduction and review of the GlusterFS architecture and key functionalities. Learn how GlusterFS is deployed in the datacenter, in the cloud, or between the two. We’ll also cover a brief history of how GlusterFS was created and where it’s headed.

On the agenda:

  • Brief intro to Gluster’s History
  • Gluster Architecture Design Goals
  • Key Technical Differentiators
  • Gluster Elastic Hashing Algorithm
  • Deployment scenarios
  • Use Cases

Date: Thursday, June 30, 2011
Time: 11 AM PT / 2 PM ET / 7 PM UK (London)
RSVP: Webinar Registration

Looking forward to the webinar!

GlusterFS 3.1.5 now available

For customers and community members on the 3.1.x series, we recommend that you upgrade to the latest, GlusterFS 3.1.5, available today.

Here are some issues fixed in this release:

  • Bug 2294: Fixed the issue occurred during creating and sharing of volumes with both RDMA and TCP/IP transport type.
  • Bug 2522: Fixed the issue of invalid argument while removing a file. This was caused by gfid mismatch, which happens during distribute rename.
  • Bug 2553: Fixed the issue occurred during NFS file creation with Mac client.
  • Bug 2904: Fixed the issues occurred during upgrading from 3.1.x version to 3.2.x.
  • Bug 2949: Fixed the issue of glusterfs hanging when self-heal command is issued on two glusterfs clients.
  • Bug 2941: Fixed the issue of glusterd breaking down when starting hundreds of volumes.
  • Bug 2870: Fixed the issue of inconsistent xattr values when creating bricks.
  • Bug 2994: Fixed the issue of glusterfs hanging when untar and rm commands are run in parallel.

The yum repository has been updated to reflect this new release.

IDC: Public cloud market to hit $72.9B in 2015

From the Between the Lines blog over at ZD, came this prediction on the public cloud market:

Public cloud computing services will be a $72.9 billion market in 2015, up from $21.5 billion in 2010, according to IDC.

That growth equates to a compound annual growth rate of 27.6 percent, but IDC estimated that public cloud computing will fuel the information technology industry for the next 25 years.

This feels like an appropriate time to bring up this (often overused) Gandhi quote:

First they ignore you, then they laugh at you, then they fight you, then you win.

For some time, some people have laughed off the idea of cloud computing, saying it was too far-fetched and didn’t solve the needs of customers, as well as unnecessarily placing data at risk. Part of it was justified, and part of it was just plain FUD by entrenched players afraid of losing market share. No one’s laughing now – cloud computing is here to stay. The only question is how much of in impact will it make on the IT industry, and how will the losers and winners be sorted.

I have to be honest and say that I, too, laughed at the term cloud computing, thinking it was marketing speak. While it is unfortunate that some companies continue to use “Cloud” in this way, there can be no doubt that the technologies powering cloud computing are indeed transformational.

Empire State of Mind: A Walk in the Clouds in New York City

If you’re like me and wondered just how much “cloud computing” resonates with an enterprise IT audience, you would have found an answer in New York City last week. The answer is a resounding confirmation that cloud computing is no longer this fanciful idea the future of computing. In fact, it lives and breathes in the here and now. I found leaders of industry in full attendance last week, including those from banking and financial services, major airlines, media and entertainment conglomerates, publishing, pharmaceuticals, and even major church organizations. They all flocked to the Jacob K. Javitz Convention Center to find their place in the cloud computing ecosystem. This week was a clear departure from years past; few and far between could be heard expressing confusion and wondering aloud – as did one kind gentleman from Poland, “Vat iss diss clooouuuud???” While he may have been confused, he was certainly in the right place at the right time to get his question answered.

Whereas mature, moribund markets tend to produce trade shows with bizarre marketing spiels and models in the booths, one of the hallmarks of nascent, dynamic markets is the sheer number of people looking for – and providing – meaty information. Between CloudCamp, the Cloud Expo conference sessions, the RightScale User Conference and Partner Connection, and the Amazon Web Services (AWS) Summit, one could get a well-rounded view of all facets of cloud computing in one location, in one week. Cloud Expo itself was spectacular – add in the co-located events, the multitude of educational tracks and factor in the approximated collective energy and IQ of the audience and you have a culmination of brain power that put this week well over the top from a cloud intelligence perspective.

If you haven’t previously attended a CloudCamp session I highly encourage you to get on this cloud, attend and participate. A lot of information is shared at camp track sessions, and in the evenings there are five minute lightning talks given by some of the vendors who help sponsor the camps. The lightning talks are a good way to hear about new technology and ask questions directly to cloud product providers. Additionally, impromptu panels are assembled based on attendee subject matter interest. This panel consists of you – the attendee – and other subject matter experts in attendance. I can tell you from personal experience that CloudCamp is worth the time and investment.

Cloud Expo gave attendees a great overview of the cloud ecosystem by virtue of the diversity of vendors on the expo floor. Walking through the exhibits, there were cloud services, software, hardware, and cloud management vendors giving their own particular cloud-y points-of-view. Our own booth was mobbed by hundreds of IT professionals looking for an elastic storage solution that could handle additional capacity as needed, regardless of infrastructure platform. We were happy to be able to tell folks that we run on any infrastructure – public cloud, private cloud and even hybrid clouds.

In the photo above, I explain the concept of “a NAS file server in the cloud” (AWS) to a Cloud Expo Conference attendee. That’s me in one of my colorful shirts I’m known for wearing at industry conferences and trade show.

In this photo, area sales Manager Rich Nave and Product Marketing Manager Heather Wellington explain Gluster deployments within virtual machine environments and in the data center on bare metal.

The RightScale User Conference provided attendees with a wealth of information on more effective and efficient cloud infrastructure deployment and management. RightScale’s Partner Connection meeting at the ACE hotel resulted in a collective of RightScale partners who had previously not had the opportunity to meet. The results of this kind of meet-up are new and creative solutions for users – and I can tell there are a few new solutions being discussed even as I write this blog. Additionally, RightScale hosted a cocktail party for everyone – user conference attendees and partners alike. The rooftop party at the Gansevoort New York City hotel provided the perfect venue for a mix of cloud conversations and cocktails. If you attend the RightScale User Conference this fall, or next spring in New York, I highly encourage you to attend the cocktail party and dive into conversation with some of the industry’s most talented cloud technologists.

Capping off the week was the AWS Summit at the midtown Hilton hotel. This was our first AWS Summit, and we were glad to spend time speaking with well over 1,000 attendees. AWS did a great job providing updates on their infrastructure and methodologies to better leverage their offerings. It was great to see so many AWS solutions partners there with us explaining how they fit within the AWS cloud. In simple terms, Gluster provides a NAS file server in the AWS cloud which can be deployed in minutes, scaled to petabytes and support synchronous and asynchronous data replication. We are also participating in the AWS Summit in San Francisco on June 21st at the Westin St. Francis hotel. Here’s a link to the summit:

http://aws.amazon.com/about-aws/aws-summit-2011/ We hope to see you there!

But back to NYC. We were happy to see that the Jacob K. Javitz Conference Center is undergoing remodeling and the whole area around the conference center is under redevelopment. New hotels, restaurants and even a new subway station leading right to the foot of the conference center are planned for opening in 2012.

I’ll leave you with one final thought. When companies and individuals set out to innovate, anything is possible. AWS has proven this by deploying what is arguably the standard in emerging cloud infrastructure. RightScale has proven that innovation can lead to a better way to manage the emerging cloud. Now, with innovation in cloud storage from Gluster, there is a better way to do storage – a better way to deploy, scale, manage and protect your businesses life blood…its information. We are proud to be a part of the new wave of innovation in cloud computing. Hope to see you soon and talk to you about all the great things happening with Gluster.

Tom

Our first GlusterFS Meetup! 6/30 @ 6:30pm

One of the things we’ve been really excited about is the prospect of hosting user groups at our office and seeding many more of them around the globe. We’re thrilled to be hosting our very first GlusterFS meetup on June 30, here at Gluster HQ in Sunnyvale. This will be the very first of many, many meetups.

In this meetup, our beloved CTO and co-founder A. B. Periasamy will be on hand to take attendees on a deep dive of GlusterFS’ architecture. If you’ve always had questions about GlusterFS’ magic hashing and lack of a metadata server, this is the perfect forum to ask.

Be sure and RSVP soon, because this is filling up very quickly. We will also look into live video streaming and posting the video here afterwards – more on that later.

Gluster Joins the Open Virtualization Alliance

Leading provider of scale-out cloud storage joins alliance to provide businesses with knowledge and advice about today’s virtualized storage options

Sunnyvale, Calif. – June 23, 2011 – – Gluster, the leading provider of scale-out, open source storage for public and private clouds, today announced it has joined the Open Virtualization Alliance, an alliance working to help provide businesses with education and technical advice to better understand and evaluate virtualization options. Existing open source communities are joining together to provide best practices on managing the development of the KVM hypervisor and its associated capabilities.

“Over the last few years, we’ve witnessed rapid growth in the KVM community as organizations look to virtualize their compute infrastructure. Gluster enables them to virtualize their storage layer providing greater flexibility at a lower cost,” said Ben Golub, president and CEO of Gluster. “Joining the Open Virtualization Alliance is a great opportunity to work alongside other leading software vendors to help organizations evaluate and better understand their virtualized storage choices.”

The Gluster Virtual Storage Appliance for KVM enables organizations to accelerate their move to the cloud and leverage their current infrastructure assets, delivering scale-out linear performance that meets their most demanding storage demands. Gluster provides enterprises with the ability to combine commodity compute and storage into a virtualized, standardized, and centrally managed storage pool, in both public and private clouds. With GlusterFS, the software platform at the core of the Gluster Virtual Storage Appliance, organizations can deploy high-performance NAS storage at a fraction of the price of traditional storage options.

“We are very pleased to welcome Gluster into the Open Virtualization Alliance (OVA),” said Scott Crenshaw, vice president and general manager, Cloud Business at Red Hat and a founding member of the Open Virtualization Alliance. “This groundswell of support for the Open Virtualization Alliance from industry leaders and innovators shows the overwhelming demand for KVM as the open virtualization alternative. KVM delivers leading performance, scalability and security, making it the ideal alternative to proprietary virtualization technologies.”

Supporting Resources

Follow Gluster on the web for the latest news and information at:

About Gluster Technology
Gluster’s software-only solutions let enterprises deploy storage the same way they deploy computing today–as a virtualized, commoditized, and scale-on-demand pool, radically improving storage economics. Combined with the customer’s choice of commodity computing and storage resources, Gluster can scale-out to petabytes of capacity and GB/s of throughput at a fraction of the cost of proprietary systems. Gluster ensures high availability with n-way replication both within and between public and private data centers. Gluster is deployable both on-premise (as a virtual appliance or bare-metal software appliance) and in public clouds such as Amazon Web Services. Gluster is the primary author and maintainer of the open-source GlusterFS software, which has been downloaded over 200,000 times.

About Gluster
Gluster is the leading provider of open source storage solutions for public, private and hybrid clouds. Over 150 enterprises worldwide have used Gluster in commercial deployments ranging from a few terabytes to multiple petabytes, across the most demanding applications in digital media delivery, healthcare, Internet, energy and biotech. Gluster is privately-held and headquartered in Sunnyvale, California. Visit us at www.gluster.com.

# # #
Media Contact:
Danielle Tarp
Mindshare PR for Gluster
650-947-7405
Danielle@mindsharepr.com

Gluster Announces Brightcove as Cloud Storage Customer

Innovative scale-out cloud storage solution delivers cost-effective, highly scalable and highly available storage to leading cloud content services company

Sunnyvale, Calif. – June 21, 2011 – – Gluster, the leading provider of scale-out, open source storage for public and private clouds, today announced that Brightcove, the leading cloud content services company, is using its storage solution to power video for tens of thousands of websites. Brightcove’s Video Cloud online video platform is the platform of choice for media companies, businesses, and organizations in over 50 different countries, offering everything from mobile video solutions to third-party player plug-ins, innovative ad formats, and pre-built integrations with CMS systems, ad servers and ad networks.

“Our mission at Brightcove is to publish and distribute the world’s professional digital media, so efficient storage of that media is mission-critical for us,” said Rich Paret, Director of Systems Engineering at Brightcove. “Gluster provides us with an effective solution for our storage infrastructure to ensure we constantly meet business demands. It delivers the scalability, performance and cost savings we need to continue offering cost-effective solutions without having to sacrifice performance and consistency.”

With Gluster, Brightcove is able to add storage transparently without affecting the end user. Gluster provides Brightcove with mission-critical features at a low cost with limitless scalability and high availability. Gluster is also an open source storage solution that is able to meet Brightcove’s storage demands, providing the technology necessary to address hyper growth without breaking the budget.

“Brightcove’s Video Cloud is considered an innovative, cloud-based video platform by its customers, the company selected an innovative storage platform designed for cloud environments,” said AB Periasamy, Co-Founder and CTO of Gluster. “Gluster is delivering the limitless scalability and high availability Brightcove needs to meet its extensive customer demands at a price that is appealing to the company’s business model.”

Supporting Resources
Follow Gluster on the web for the latest news and information at:

About Gluster Technology
Gluster’s software-only solutions let enterprises deploy storage the same way they deploy computing today–as a virtualized, commoditized, and scale-on-demand pool, radically improving storage economics. Combined with the customer’s choice of commodity computing and storage resources, Gluster can scale-out to petabytes of capacity and GB/s of throughput at a fraction of the cost of proprietary systems. Gluster ensures high availability with n-way replication both within and between public and private data centers. Gluster is deployable both on-premise (as a virtual appliance or bare-metal software appliance) and in public clouds such as Amazon Web Services. Gluster is the primary author and maintainer of the open-source GlusterFS software, which has been downloaded over 200,000 times.

About Gluster
Gluster is the leading provider of open source storage solutions for public, private and hybrid clouds. Over 150 enterprises worldwide have used Gluster in commercial deployments ranging from a few terabytes to multiple petabytes, across the most demanding applications in digital media delivery, healthcare, Internet, energy and biotech. Gluster is privately-held and headquartered in Sunnyvale, California. Visit us at www.gluster.com.

# # #
Media Contact:
Danielle Tarp
Mindshare PR for Gluster
650-947-7405
Danielle@mindsharepr.com

Webinar Recording: What's New in GlusterFS 3.2

If you weren’t able to join 200 of our closes friends and attend the super-popular webinar on “What’s New in GlusterFS 3.2” here’s your chance to view the recording. Craig Carl, our uber-SE, does a great job walking through the headline features: geo-replication, elasticity, global namespace, et al.

Just go to the webinar page (registration req’d).

  • Page 1 of 2
  • 1
  • 2
  • >