In today’s Storage Hangout, Brian catches up with Erin Boyd, principle software engineer of Red Hat Big Data to learn about the Hadoop Plug-in Refresh Release. You can see the release information in detail here.
Q: Tell us more about the Hadoop Plug-in Refresh release. What can the big data community expect from this new release?
A: For the latest release of the plugin we have joined forces with our partner, Hortonworks, to release their 2.1 version of Hortonworks Data Platform for Hadoop using Ambari.
Q: What is Ambari and why should organizations use it?
A: Ambari is the only open source Hadoop management tool in the ASF and the preferred tool used by Hortonworks to release their Data Platform. Ambari is used to provision, manage and monitor Hadoop clusters. Ambari provides a wizard to easily deploy clusters across many different hosts, allowing the user to select and configure their Hadoop services taking a lot of the guesswork out of the equation. By providing the services in a ‘stack’ configuration, the user can be confident the services they select will work properly when deployed together.
It also provides a rich REST API that developers can use to integrate into their own management and monitoring of applications.
For this release, my team was able to take the base Hortonworks Data Platform stack, or HDP stack, and expand it to use an alternative storage to the traditional HDFS storage. By offering to our customers the ability to run analytics on their data in place on Red Hat Storage, customers will not longer have to have their data ingested into Hadoop. It’s a huge win.
In addition, the Hadoop ecosystem is in a constant state of flux. New services are being introduced often to address different big data use cases. Where once Hadoop was synonymous with MapReduce, this isn’t the only way to analyze big data sets.
Q: You mentioned the Hadoop landscape is changing…how is Ambari embracing these changes
New services in HDP 2.1.GlusterFS
- New services from tech preview
- Identity Manager integration for the application of LDAP and Kerberos
- User experience
- Simplified installation experience
- Improved presence of GlusterFS on UI for better customization
The Ambari community has been a leader expanding the traditional Hadoop stack to encompass lots of new technologies. As in the 2.1 version we are releasing today with the Hadoop plug-in refresh release, we have services like Tez, Hive and HBase to name a few along with the traditional MapReduce services to expand the options to our big data customers. We have expanded the stack from our tech preview release a few months ago by 5 new services.
We have also been leaders in implementing a Hadoop Compatible File System adoption by the Hadoop community. Another Apache project, Bigtop, of whom our team has a committer, Jay Vyas, has created a standard for non-HDFS file systems. Bigtop is helping push the envelope on standardizing the services and continuous integration and testing of them. It’s an exciting project and we are glad to be part of it.
In the next release of Ambari, it will have the ability to deploy a Bigtop stack as well.
Q: How much does Red Hat participate in the Ambari project?
A: Currently I lead a team of 3 other developers all of whom are Ambari committers. Red Hat has the second most committers behind Hortonworks for the Ambari project.
Q: What are some new features we can expect to see in Ambari in the coming months?
A: There are some huge changes going on in Ambari we can expect to release in the first quarter of 2015. One of the features I think is really going to be a win for Storage customers is the release of Ambari views. Views create a more user centric feature to Ambari that was previously missing. It allows us to create plug-able interfaces within Ambari focused on what our customers want and need to see to better understand their data and issues in their cluster. It will allow us to help our Storage customers get the most out of Hadoop. In addition, the next release will have 7 new services to continue to expand analytics capability and create more secure clusters. It is a true testament to the agility of the Ambari community to embrace and implement new technologies right on the cutting edge. We are excited to continue partnering with Hortonworks to bring these technologies to our Red Hat customers.