Outlier Distance n

dennisdennis Member Posts: 2 Contributor I
edited November 2018 in Help

Hey,

I am new to Rapid Miner. I was wondering how one determines the number of outliners Rapid Miner should look for in a dataset of for example 100 datapoints.

 

Thanks in advance.

 

Den

Best Answer

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn
    Solution Accepted

    Hi Dennis, unfortunately there isn't a simple one-size-fits-all answer to the question of outlier detection.  It is very contextual to your dataset.  So here are a few ideas:

    1.  In the standard outlier detection operator, you can try setting the threshold for outliers at 5, and then visually examine the outliers that it selects.  Ask yourself, are these examples really outliers in the context of my dataset?  Then try changing the threshold again (either up or down) and seeing what examples are added or subtracted.  Sometimes this will help get a sense of the appropriate threshold.
    2. You should download the free anomaly detection extension from the marketplace.  This contains operators that do not require you to specify in advance how many outliers there should be.  You can then look at multiple methods of identifying outliers based on similarity measures.  If you don't have too many attributes, then the histogram-based outlier score (HBOS) is a very useful one.  If you have too many attributes, you will be better off with a nearest-neighbor approach such as the local outlier factor (LOF) approach.

    I hope this is helpful.

    Regards,

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts

Answers

  • dennisdennis Member Posts: 2 Contributor I

    Hey Telecontar120,

     

    thanks for your great suggestions! I will test them!

     

    Regards,

     

    dennis

Sign In or Register to comment.