Think you need Hadoop? Think again

January 25, 2014 by
Filed under: Uncategorized 

If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!

Today’s post is written by Rick DelGado. His guest post is about the need of Hadoop. Thanks Rick for your input.

In the big data buzz, Hadoop has been the big data solution of choice leaving many feeling like Hadoop is their only option for harnessing big data. However, there are many other big data options out there that offer different features than Hadoop, and may actually fit your business needs better. Flash array storage, in particular, has made it easier to create fast, affordable storage options, so check out these other big data solutions before settling on Hadoop.

HPCC Systems

This system developed by LexisNexis is similar to Hadoop in that it is used to build clusters of servers for the purpose of analysing large data sets. HPCC uses Enterprise Control Language to make the process of writing parallel-processing workflows easier, and, also like Hadoop, has an ecosystem of tools built around it including the Roxy Rapid Data Delivery Cluster, a data warehouse similar to HBase, and the Thor Data Refinery Cluster, a data processor.

While still in its preliminary stages, HPCC’s similarities to Hadoop make it a running alternative as a big data solution.

Storm

Storm, first developed by Backtype before being purchased by Twitter, was dubbed the “Hadoop of real time processing” in a blog post by Nathan Marz. This distinction was made because Hadoop is a batch processor that works with fixed data. Storm, on the other hand, can process data as it streams. Of course, the real-time issue is being addressed in Hadoop providing some competition for Storm.

Disco Project

Developed by Nokia Research, Disco Project has been around for a while without a lot of attention. With Disco Project, data is distributed and replicated in a similar manner to Hadoop, and Disco has job scheduling features. However Disco Project doesn’t use its own file system. The advantage of Disco is that it’s backend is written in Erlang, a language with support for fault tolerance, distribution and concurrency.

Spark

Spark was specifically created by UC Berkeley to make it faster to write and run data analytics and is one of the newest options on the market. A key difference from other MapReduce systems is that Spark permits in-memory querying of data instead of disk I/O. Spark also performs better than Hadoop on several iterative algorithms and is written in Scala—an object-oriented language that permits users to make queries directly from the Scala interpreter.

GraphLab

GraphLab was created to make designing and implementing parallel machine learning algorithms easier. It varies from MapReduce in that it has an update phase that can read and modify data sets that overlap while MapReduce requires that all data sets be separated. GraphLab also offers its own version of the reduce stage called sync operation in which the output is global rather than local.

Microsoft Alternatives

Microsoft offers three Hadoop Alternatives.

Azure Table Storage is offered in Microsoft’s cloud and is meant to serve as an alternative data store to the one provided by Hadoop. It is not an analytics system.

LINQ to HPC allows you to build clusters of servers in Microsoft’s programming language and complete data analytics with unstructured data just like Hadoop does.

Azure Project Daytona is a research project that is based on MapReduce. It runs as a service that provides ready-to-use algorithms and can be delivered through Excel.

As you can see, there is actually plenty to choose from when looking for a big data solution, and an added benefit is that many of these tools can work together, including with Hadoop, to create a customized solution for your individual needs.

Short bio

I’ve been blessed to have a successful career and have recently taken a step back to pursue my passion of writing. I’ve started doing freelance writing and I love to write about new technologies and how it can help us and our planet.” – Rick DelGado

Share

Comments

36 Comments on Think you need Hadoop? Think again

  1. George on Mon, 27th Jan 2014 6:55 am
  2. You forgot stratosphere.

  3. Durai on Fri, 31st Jan 2014 8:49 am
  4. With HPCC, the Roxy is comparable to Hive rather HBase.

  5. Mohamed Goneam on Sun, 2nd Feb 2014 9:13 pm
  6. Thanks for sharing, its help me alot.
    below site maybe assist you

    Computek International

    Regards

  7. logesh on Mon, 3rd Feb 2014 7:35 am
  8. Great Blog with lot of tips on data mining. It will be more helpful blog for beginners to know more about it. Once again thanks for sharing this blog!

  9. Nick Wilson on Fri, 18th Apr 2014 2:58 pm
  10. SciDB also serves as a viable alternative to Hadoop, particulary as an analytical tool. It is a horizontally-scalable open-source technology specifically designed for answering data-intensive questions.

    Unlike a traditional DBMS which stores data in two-dimensional tables of rows and columns, SciDB holds data in multidimensional arrays. This architecture naturally facilitates the linear algebra computation which undergirds many statistical and machine learning techniques, such as linear regression, PCA, and SVD. Furthermore, SciDB offers interfaces familiar to analysts: R and Python. See http://www.scidb.org for details.

  11. vignesh on Thu, 26th Jun 2014 10:57 am
  12. I got a job by saying this answer in my last interview. thanks for awesome help. I got more idea about Hadoop from Besant Technologies.
    I learned a lot from Besant Technologies in my college days. They are the Best Hadoop Training Institute in Chennai

    http://www.hadooptrainingchennai.co.in

  13. Big Data Training on Mon, 26th Jan 2015 8:49 am
  14. Thank you for your sharing this informative blog. I have completed hadoop certification course at a leading academy. If you are looking for best Hadoop Training in Chennai visit FITA IT training and placement academy. To know more about hadoop training please refer this link.
    http://www.fita.in/big-data-hadoop-training-in-chennai/

  15. Andrpoid Training in Chennai on Mon, 2nd Feb 2015 5:11 am
  16. Hi this is Kathiresan i am having 3 years of experience as a dot net developer and i am certified. i have knowledge on OOPS concepts in .NET but dont know indepth. After learning android will be enough to get a good career in IT with good package? and i crossed Android Training in Chennai website where someone please help me to identity the syllabus covers everything or not??

    Thanks,
    kathiresan

  17. Android Training in Chennai on Mon, 2nd Feb 2015 5:11 am
  18. Hi this is Kathiresan i am having 3 years of experience as a dot net developer and i am certified. i have knowledge on OOPS concepts in .NET but dont know indepth. After learning android will be enough to get a good career in IT with good package? and i crossed Android Training in Chennai website where someone please help me to identity the syllabus covers everything or not??

    Thanks,
    kathiresan

  19. hadoop training in chennai on Mon, 2nd Feb 2015 8:37 am
  20. hi,you have to sharing for to helpful information..you will should be sharing effective information….http://www.besanttechnologies.com/training-courses/data-warehousing-training/hadoop-training-institute-in-chennai

  21. HTML5 Training on Mon, 16th Feb 2015 10:50 am
  22. Thanks for sharing this informative blog.. If anyone want to get HTML5 Training in Chennai please visit FITA academy located at Chennai, Velachery. Rated as No.1 training and placement academy in Chennai.

  23. SAP Training on Thu, 19th Feb 2015 8:37 am
  24. Thanks for sharing this informative blog. FITA provides SAP Training in Chennai with years of experienced professionals and fully hands-on classes. SAP is one of the CRM. Today’s most of the IT industry use this software for customer relationship management. To know more details about sap reach FITA Academy. Rated as No.1 SAP Institutes in Chennai.

  25. Java Training on Thu, 26th Feb 2015 12:16 pm
  26. Java is one of the popular technologies with improved job opportunity for hopeful professionals. Java Training in Chennai helps you to study this technology in details.If you are looking for best Java Training Institutes in Chennai reach Fita academy.

  27. parim on Mon, 2nd Mar 2015 5:52 am
  28. Best Oracle Training Tambaram Best Oracle Training in Chennai

  29. Oracle Training on Mon, 2nd Mar 2015 8:49 am
  30. Your blog is really useful for me. Thanks for sharing this informative blog. If anyone wants to get real time Oracle Training in Chennai reach FITA located at Chennai. They give professional and job oriented training for all students.

  31. Salesforce Training on Fri, 6th Mar 2015 10:54 am
  32. Thanks for sharing this informative blog..If anyone want to get Salesforce Training in Chennai reach FITA academy.

  33. emi on Sat, 7th Mar 2015 1:53 pm
  34. I get a lot of great information from this blog. Thank you for your sharing this informative blog. I have bookmarked this page for my future reference. Just now I have completed hadoop certification course at a leading academy. If you are looking for best Hadoop Training Chennai visit FITA IT training and placement academy which offer Big Data Training Chennai.

  35. Digital Marketing Training on Wed, 11th Mar 2015 11:06 am
  36. Thanks for sharing……….

  37. Salesforce Training in Chennai on Wed, 11th Mar 2015 11:08 am
  38. I have read your blog and i got a very useful and knowledgeable information from your blog.its really a very nice article.You have done a great job . If anyone want to get Salesforce Developer Training in Chennai, Please visit FITA academy located at Chennai Velachery.

  39. Big Data Training Chennai on Thu, 12th Mar 2015 7:20 am
  40. Hi, I am Victoria lives in Chennai. I am technology freak. I did Hadoop Course in Chennai at FITA which offers best Big Data Course in Chennai. This is useful for me to make a bright career in IT field.

  41. steve jobs on Fri, 13th Mar 2015 12:18 pm
  42. It’s a nice blog with lot of information,Thanks for sharing….

    java training in chennai | android training in chennai

  43. ricardo on Fri, 13th Mar 2015 12:20 pm
  44. It’s a nice blog with lot of information,Thanks for sharing….

    java training in chennai | android training in chennai

  45. salesforce training in chennai on Tue, 17th Mar 2015 10:38 am
  46. The Salesforce customer portal provides customers the ability to track their own cases, includes a social networking plug-in that enables the user to join the conversation about their company on social networking websites, provides analytical tools and other services including email, chat, Google search, and access to customers’ entitlement and contracts.The Sales Cloud includes a real-time sales collaborative tool called Chatter.

    salesforce training in chennai

  47. Digital Marketing Chennai on Wed, 18th Mar 2015 8:46 am
  48. Thanks for sharing this informative blog. Recently I did Digital Marketing Training in Chennai at a leading digital marketing company. It’s really useful for me to make a bright career. To know more details about this course please visit FITA.

  49. Nancy Rani on Fri, 20th Mar 2015 12:18 pm
  50. I got a job by saying this answer in my last interview. thanks for awesome help. I got more idea about SAS from Besant Technologies. If anyone wants to get SAS Training in Chennai visit Besant Technologies.

  51. Jeni on Fri, 20th Mar 2015 12:20 pm
  52. Its really helpful for me to understand where we i lost in my previous interview. Thanks. If anyone wants to Learn SAS Training go to the No.1 Training Institute in Chennai.

  53. Big Data Training in Chennai on Sat, 21st Mar 2015 8:50 am
  54. Thank you so much sharing this incredible content. Your unique article helps me to update myself with the current technology. Keep up this by sharing more articles. This might help many like me to seek the best Big Data Course in Chennai

  55. Oracle DBA Training in Chennai on Fri, 27th Mar 2015 12:15 pm
  56. Oracle rapidly converging all IT required solutions like middleware, storage and OS by acquiring leading core pioneers and technology players in market. It is providing very strong and unique supports to customers.
    Still now oracle has acquired around 40 companies and expands its wings around 90% on RDBMS and ERP tools and services and related technologies.

  57. Linux Training in Chennai on Fri, 27th Mar 2015 12:17 pm
  58. Linux Foundation survey Linux skills are in high demand among IT employers. Linux System administrators are always having high demand and good pay in job market.
    Visit Us, Linux Training in Chennai

  59. SAS Training in Chennai on Fri, 27th Mar 2015 12:19 pm
  60. The process involves designing studies, collecting good data, describing the data with numbers and graphs, analyzing the data, and then making Decisions / conclusions. Visit Us, SAS Training in Chennai

  61. Oracle Training in Chennai on Sat, 28th Mar 2015 1:04 pm
  62. Oracle Training Institutes in Chennai

    I get a lot of great information from this blog. Recently I did oracle certification course at a leading academy. If anyone interested to learn best Oracle Training in Chennai visit FITA academy which offer PL SQL Training in Chennai.

    Regards…

    Oracle Training

  63. Jenny on Sun, 29th Mar 2015 6:40 am
  64. Hadoop dealing with different type of data structure.In hadoop HIVE topics is very interesting.
    Software Testing Training in Chennai

  65. IOS training in Chennai on Mon, 30th Mar 2015 1:42 pm
  66. Dealing with hadoop is quite interesting. But what about data mining.

    Java training in Chennai | Android training in chennai

  67. Android Training in Chennai on Tue, 31st Mar 2015 8:17 am
  68. Thanks for your article on Android technology. Android is an open source platform that allows developers to create stunning website loaded with various advanced features and functionalities. Android Training Institute in Chennai | Best Android Training institute in Chennai

  69. IOS Training in Chennai on Tue, 31st Mar 2015 8:19 am
  70. Thanks for your informative article on ios mobile application development. Your article helped me to explore the future of mobile apps developers. Having sound knowledge on mobile application development will help you to float in mobile application development. iOS Training | iOS Training Institutes in Chennai

  71. Hadoop training chennai on Tue, 31st Mar 2015 12:03 pm
  72. I have read your blog it was nice to follow even I am looking for your future updates. Hadoop is a highly growing & scoopful technology in IT market it’s an open-source software framework for managing big data in a distributed fashion on large commodity computing hardware. FITA provides Hadoop training in Chennai get in to fita and out with your career.

Tell me what you're thinking...





  • Swiss Association for Analytics

  • Most Popular Posts

  • T-shirts, Mugs & Mousepads


    All benefits given to a charity association
  • Data Mining Search Engine

    Supported by AnalyticBridge

  • Archives

  • Reading Recommandations