Top 10 challenging problems in data mining

March 27, 2008 by Sandro Saitta
Filed under: ICDM, KDD, data mining article, top 10 data mining problems 

In a previous post, I wrote about the top 10 data mining algorithms, a paper that was published in Knowledge and Information Systems. The “selective” process is the same as the one that has been used to identify the most important (according to answers of the survey) data mining problems. The paper by Yang and Wu has been published (in 2006) in the International Journal of Information Technology & Decision Making. The paper contains the following problems (in no specific order):

  • Developing a unifying theory of data mining
  • Scaling up for high dimensional data and high speed data streams
  • Mining sequence data and time series data
  • Mining complex knowledge from complex data
  • Data mining in a network setting
  • Distributed data mining and mining multi-agent data
  • Data mining for biological and environmental problems
  • Data Mining process-related problems
  • Security, privacy and data integrity
  • Dealing with non-static, unbalanced and cost-sensitive data

I sometimes receive emails from master student or practitioners interested in data mining. The usual question is “What can I do as research in data mining?”. Of course, the answer depends on what you like and the opportunities of the moment. However, this paper can maybe give some hints on possible directions for research.

As usual, the “data mining automation process” issue is mentioned. It is worth noting that researchers argue that they need to find a way to automate data mining, while practitioners say that they can do it (for example KXEN). Finally, I think that one of the most important issue is pointed out by the following sentence in the paper:

“[...] they’re [data mining systems] unable to relate the results of mining to the real-world decisions they affect [...]“

In my opinion, it is more subjective to rank top problems than top algorithms. Most people will certainly agree on the selected data mining algorithms. The question is more subjective regarding data mining problems since some of them may only be relevant to certain fields of research.

No TweetBacks yet. (Be the first to Tweet this post)
  • Share/Bookmark

Comments

21 Comments on Top 10 challenging problems in data mining

  1. M.Rajeswari on Fri, 9th Oct 2009 11:03 am

    2. I am a full-time Ph.D.Research Scholar in computer science departmentin India (Bharathiar University,Coimbatore,TamilNadu,India).I have carried out my research under data mining using customer relationship management. i want some relevant problems that could be useful for carriying out further actions.

  2. Sandro Saitta on Sat, 10th Oct 2009 5:03 pm

    3. Hi, I think you should ask your question on http://www.kdnugget.com forums. Hope this helps.

  3. sowjanya on Mon, 2nd Nov 2009 9:58 am

    4. i would like to know whether applying combination of data mining algorithms will be applicable on any data set and improve the result

  4. Sandro Saitta on Fri, 6th Nov 2009 6:17 pm

    5. @sowjanya: it depends on your goal. For example, a feature selection technique such as genetic algorithm can be combined with support vector machines (SVM) to improve classification accuracy.

  5. Abhik Ray on Sat, 5th Dec 2009 2:04 pm

    6. Could you tell me what kinds of opportunities exist for research in Data Mining in mobile computing environments? Also what work could be done by combining Data Mining, Mobile Computing and Information Retrieval?
    Thanks.

  6. Sandro Saitta on Tue, 8th Dec 2009 12:19 pm

    7. @Abhik: I don’t have experience in data mining in the mobile world. I think you should ask your question on the KDnuggets forum.

  7. Jameel on Tue, 15th Dec 2009 6:12 am

    8. Hi
    I wish to do a research in data mining related to semantics under Retail domain.
    but im not able to corner the problem for research. Please suggest me some open research problems in this area

  8. Sandro Saitta on Thu, 17th Dec 2009 10:50 am

    9. @Jameel: please try the KDnuggets forums.

  9. ramesh on Fri, 18th Dec 2009 10:38 am

    10. Hi, Can you suggest suitable research area in datamining application in agriculture. if any one comeacross this topic pl. let me know the details. thank u

  10. Sathyan Munirathinam on Wed, 17th Mar 2010 5:35 pm

    11. Hi,
    This is Sathyan (sat_hyan@hotmail.com) working for IBM in Data Warehousing Domain. Parallely, I am doing Part-time Phd in Bharathiar University in the area of “Distributed Data Mining in Grid Environments” under the guidance of Dr.Ramadoss. Please send an email, if any one wants to discuss on the topics of Data Mining.

    Thanks
    M.Sathyan

  11. Savita on Fri, 19th Mar 2010 1:52 pm

    12. Hi,
    I am working as a faculty in engineering college. My interested area is Datamining.I have finished my Mtech with Datamining Project that is on Simulating Outlier detection algorithm on research paper. Now i want to do my Phd. How can i proceed my work. Do you have any suggestion or any new ideas on this.
    thanks
    Savita

  12. Sandro Saitta on Fri, 2nd Apr 2010 1:00 pm

    13. @Savita: You can try to apply at any university. It’s hard to give any advice, but you can try EPFL (www.epfl.ch)

  13. venkat on Wed, 21st Apr 2010 9:08 am

    14. Hi,
    I am working as a Associate Professor in engineering college. My interested area is Datamining.i am doing ph.d in andhra university visakhapatnam.I am interested in data mining .my guide suggested to do the data mining techniques using neural net works. he suggested one related ieee paper also.Now i want to continue my work. How can i proceed my work. Do you have any suggestion or any new ideas on this.please help me.
    thanks
    venkat

  14. mathi on Wed, 21st Apr 2010 11:45 am

    15. I want to do ph.d in the field of datamining. give me problem in association rule mining to carry out ph.d word

  15. mathi on Wed, 21st Apr 2010 11:49 am

    16. what can i do as research in datamining

  16. shweta kharya on Sat, 24th Apr 2010 8:05 am

    17. list me some recent work which is going on Data mining…….

  17. kalbania on Thu, 20th May 2010 7:27 am

    18. Hi

    I want to do my PHD in the data mining. Can you please suggest me some research problems in this area and some research paper that would help me to write PHD Thesis proposal.

  18. Sandro on Thu, 20th May 2010 2:05 pm

    19. @Kalbania: I would suggest that you start with a book shown in the top right of this blog. Or you can also find books by looking for “advance data mining”

  19. Kumaran on Fri, 2nd Jul 2010 6:20 pm

    20. Hi,
    I am working as a Associate Professor in engineering college. My interested area is Datamining.i am going to do ph.d .My guide suggested to do the ieee related one so now i want to continue my work. How can i proceed my work. How can i get problems in datamining. Do you have any suggestion or any new ideas on this.please help me.
    thanks
    Kumaran

  20. Sandro Saitta on Mon, 5th Jul 2010 8:48 am

    21. @Kumaran: I guess the easiest is to post your question on the KDnugget forum

  21. Ramesh M on Mon, 12th Jul 2010 11:12 pm

    22. kindly send the current open problem in “security issues in datamining” for research.

Tell me what you're thinking...





  • Data Mining Search Engine

    Supported by AnalyticBridge

    Participate to this search engine!

  • Reading Recommandations

  • T-shirts, Mugs & Mousepads

  • Archives

  • Pages

  • Disclaimer

    The opinions discussed on Data Mining Research are my own and do not reflect the position of my current employer, FinScore. The views and opinions expressed by visitors to this blog are theirs and do not necessarily reflect mine.

  • Meta