[SOLVED] How Big ist Big

michaelhechtmichaelhecht Member Posts: 89  Guru
edited November 2018 in Help
Hello,

I've some problems with a "big" data set. The set contains about 150000 lines and (due to nominal to binominal conversion) 56 attributes. If I send this to a neural network (RM version, not Weka) I get an error " PM SEVERE: Process failed: Cannot reset network to a smaller learning rate"

With only one percent of the data all works fine. RM 5 64bit uses about 8GB of memory and I have 12 available, so no problem of memory.

What is the problem? Is the data set too big?

Thanks in advance,

Michael

Answers

  • michaelhechtmichaelhecht Member Posts: 89  Guru
    It seems that this problem is known but not yet solved, isn't it?

    http://rapid-i.com/rapidforum/index.php/topic,2324.0.html
  • michaelhechtmichaelhecht Member Posts: 89  Guru
    Ok, it is obviously a problem of the data. Since I did sampling of al little amount I "missed" the three data sets with missing labels. Increasing the fraction of sampled data increased also the probability of getting one data set with missing label.

    So, it would be nice, to get a better fitting error message, i.e. not "Cannot reset network to a smaller learning rate" but "Cannot handle missing labels".

Sign In or Register to comment.