Do we need to normalize the Data in outlier detection?

AnkiAnki Member Posts: 39 Contributor II
Hello All,

For finding the outliers, Do we need to normalize the Data?
Can you please tell me impact without normalizing?
Which methods are best for Normalized data for Outlier detection?
And Which methods are better if I don't want to normalize?

Thank you in advance

Yours
Anki

Answers

  • IngoRMIngoRM Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, Community Manager, RMResearcher, Member, University Professor Posts: 1,751 RM Founder
    Hi,

    For finding the outliers, Do we need to normalize the Data?
    this is certainly recommended, yes. All outlier detection methods depend on distances and / or densities and distance calculations can be really skewed on non-normalized data.

    Can you please tell me impact without normalizing?
    If you have a single dimension with a much larger scale than the other dimensions, this single dimension might overrule the others.

    Which methods are best for Normalized data for Outlier detection?
    Totally depends, there should probably also be a No-Free-Lunch-Theorem for outlier detection  ;)
    In general, I have good experiences with the local outlier factor method.

    And Which methods are better if I don't want to normalize?
    Don't do this (see above).

    Cheers,
    Ingo
  • AnkiAnki Member Posts: 39 Contributor II
    Hi Ingo,

    Thank you very much.

    Yours
    Anki
  • wesselwessel Member Posts: 537 Maven
    Agreed with all above.

    I would like to add:
    Visualization. Finding a way to nicely visualize your data can be time consuming but should be worth it.
    You can often spot outliers yourself.

    You can also look at misclassifications. The ada-boost algorithm adds weight to instances that are misclassified.
    Outliers are often given a very high weight after only a few iterations.

    Best regards,

    Wessel
Sign In or Register to comment.