Options

Robustness metrics

teioteio Member Posts: 2 Newbie
in Help

Hello,

Is there any metric provided to assess robustness / stability of a ML module? As far as I understand the ones provided are more related with the functionality.
I understand reliability as the ability to perform the intended function in the presence of abnormal or unknown inputs, related to ‘reliability’.

Any comment is welcome
Thanks in advance

Answers

  • Options
    MartinLiebigMartinLiebig Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 3,508 RM Data Scientist
    Hi there,
    anything in particular you search for? You could for example use SHAPly and see how much the output changes?

    Best,
    Martin
    - Sr. Director Data Solutions, Altair RapidMiner -
    Dortmund, Germany
  • Options
    teioteio Member Posts: 2 Newbie
    Hello,

    Thanks for your answer. I have to analyse it but it could work. I am Quality Assurance people and we are searching for metrics provided by the commercial tools to assess the quality of a ML/AI model. In our research, the metrics provided by the mostly used tools in the market, are related to functionality and not to others characteristics mentioned in ISO/IEC standards for AI systems as Robustness, Transparency, User Controllability.

    Our first option is always to use a tool to asssess quality than manual.

    Thanks for your support, Regards Teresa
Sign In or Register to comment.