History of Gluster

Where We Come From

When we started the company, we actually didn’t set out to build a file system. However, our background in clustered computing was instrumental in taking a new approach to scale out storage.

The founding team came together at California Digital Corporation (CDC) in 2003 to build a supercomputer for Lawrence Livermore National Laboratory: ‘Thunder’. Put into production in 2004, Thunder was built on the Intel Itanium platform with an open source software stack (CDC had acquired VA Linux in 2002). At the time it was the second-fastest supercomputer in the world, achieving 22.9 Teraflops. The success and efficiency of Thunder was the latest proof that scale out computing had come a long way and would continue to evolve as the de facto model.

Looking for the next challenge, the team founded Z Research with the goal of simplifying clustered computing on a foundation of open source software and commodity hardware and bringing it to the enterprise. Where did Gluster come from? Glad you asked: it’s a combination of GNU plus cluster. We formally changed the company name to Gluster based on the awareness of the product, the community, and as a nod to our history. The product plan originally called for us to begin with Gluster compute, and then add Gluster provisioning followed by Gluster storage.

The first customer engagement was with Petróleos de Venezuela S.A (PDVSA), the Venezuelan national oil company. PDVSA needed a prototype compute and storage solution to support energy exploration and production. Most problems on the compute side were well understood and had solutions, plus the Z Research team was able to leverage much of the experience from Thunder. The need to solve the storage problem was the catalyst that led to Gluster.

The team first looked to existing file system products but the investigation and their experience quickly led to the conclusion that current products would not meet the needs of their customers. The options were to undertake major modifications on existing products or build from scratch. Given the complex architecture of available file systems at the time, building from scratch looked to be no more difficult and allowed the team to work without existing constraints. Knowing what they wanted to avoid, the team put together a set of architectural principals for the product they would build:

  • Build in userspace for speed, flexibility, and ease of use
  • Eliminate a centralized metadata server for performance and reliability
  • Make the product modular to simplify configuration and minimize dependencies
  • Make use of the underlying disk file system and don’t reinvent the wheel
  • Store data as standard files and don’t use proprietary formats

The prototype was delivered in early 2007 and Gluster was released to the community. Version 2 of the product was released in May of 2009 and the original architectural foundation is still in place today. The rest, as they say, is history.  Stay tuned as we continue to write this story.