Outlier Removal Problem

  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by AndersI View Post
    JEFMAN - my first reaction when reading that was "will there ever be more than three points left?" As sigma goes down on each iteration (we removed at least one 'outlier') new points will be found to be 'outliers'...
    Yes, it's the problem.
    I tried with 2000 random values, Excel removes 3 points at k=3, PC-DMIS also.
    With k=2, Excel removes 70 values, PC-DMIS 85.
    With k=1, Excel removes about 150 values, PC-DMIS 1993 !!!!!!!!!

    (For K=2 and 3, I just try to remember the number, just from my memory, but the order is here - for k=1, pc-dmis keeps only 7 points !!!!!!!)


    • #17
      Thanks for the investigation!

      Well, that shows that filtering outliers with factor 1 x sigma is never OK. I think PC-DMIS should issue an error, or at least a warning "I'm about to remove almost all your measuring points - is that OK?" Actually, I'd say that an outlier factor <= sigma is always an error.
      SW support - Hexagon Metrology Nordic AB


      • #18
        Oke, so I found out that the algorithm for outlier removal is wayy more sophisticated! My idea for outlier removal was just the basic concept how it works in PCDMIS. There are many forms for outliers, like point outliers, contextual outliers and collective outliers. It is far more complex than I thought For more info check: https://towardsdatascience.com/a-brief-overview-of-outlier-detection-techniques-1e0b2c19e561.

        Last edited by MihaelR; 01-12-2019, 11:23 AM.


        Related Topics