Options
Automatically Selecting Single Model from Multiple Models
keith_drake
Member Posts: 11 
Contributor II
I want to employ a strategy that builds 24 separate, independent models by segmenting the input feature space by two variables: one into three segments and the other into eight segments: 3 x 8 = 24.
I know how to use the Filter Examples operator to do the incoming splits and then how to use X-Validation, Backward Elimination, etc., to do the modeling. But what operators can I use to select from among the resulting 24 models, so that I can run my hold-out data through them/it?
My approach will result in 24 different models, each derived from a different segment of the original data. When I process new (hold-out) data, only one of the 24 models will be appropriate to use since it was trained using the same segment of data (out of 24) as the new example. The other 23 models should be ignored.
My challenge is determining overall performance statistics for the 24 models automatically--without running each model separately and manually putting together the individual results to manually calculate RMSE, AE, etc.
If you can point me in the right direction by suggesting the operator(s) I should look at, that is all I need.
Thanks!
I know how to use the Filter Examples operator to do the incoming splits and then how to use X-Validation, Backward Elimination, etc., to do the modeling. But what operators can I use to select from among the resulting 24 models, so that I can run my hold-out data through them/it?
My approach will result in 24 different models, each derived from a different segment of the original data. When I process new (hold-out) data, only one of the 24 models will be appropriate to use since it was trained using the same segment of data (out of 24) as the new example. The other 23 models should be ignored.
My challenge is determining overall performance statistics for the 24 models automatically--without running each model separately and manually putting together the individual results to manually calculate RMSE, AE, etc.
If you can point me in the right direction by suggesting the operator(s) I should look at, that is all I need.
Thanks!
0
