RapidMiner

RapidMiner

SOM Reduction - Preprocessing Save doesn't work

SOM Reduction - Preprocessing Save doesn't work

When I attempt to create and save a preprocessing model with 205 features ( a mix of nominal and numeric features ) with about 10,000 training rows, after processing for about a hour, the modelwrite writes out about a 12 meg model, then hangs and freezes. Any attempts to use this model results in errors. Is this a size or complexity limitation?
7 REPLIES
Moderator

Re: SOM Reduction - Preprocessing Save doesn't work

Hi,

which preprocessing model did you create and save? Could you please post your process XML?! Given your description, I have no idea what could be the problem here.

Regards,
Tobias
Elite

Re: SOM Reduction - Preprocessing Save doesn't work

Hi,
since the SOMReduction should save the generated SOM, there should be no complexity problem with the number of examples. (Ignoring time needed to compute the SOM)
How long did it freeze, before it terminated the process? It might be, that saving might last a few minutes.

A quoting of the error messages would be usefull, too.


Greetings,
  Sebastian
Old World Computing - Establishing the Future

Check out the Jackhammer Extension for RapidMiner! Crunch more data easier and with up to 700% speed up! Available only here

Re: SOM Reduction - Preprocessing Save doesn't work

Here's the error message when trying to load the model created from the SOM preprocessing

Exception: com.rapidminer.operator.UserError
Message: Could not read file 'F:\ModelDevelopment\preprocess_som.mod': Cannot read from XML stream, wrong format: Could not call java.security.CodeSource.readObject()
---- Debugging information ----
message            : Could not call java.security.CodeSource.readObject()
cause-exception    : java.lang.RuntimeException
cause-message      : null
class              : com.rapidminer.operator.ContainerModel
required-type      : java.security.CodeSource
line number        : 2224478
-------------------------------.
Stack trace:

  com.rapidminer.operator.io.ModelLoader.apply(ModelLoader.java:97)
  com.rapidminer.operator.Operator.apply(Operator.java:656)
  com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
  com.rapidminer.operator.Operator.apply(Operator.java:656)
  com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
  com.rapidminer.operator.Operator.apply(Operator.java:656)
  com.rapidminer.Process.run(Process.java:571)
  com.rapidminer.Process.run(Process.java:541)
  com.rapidminer.Process.run(Process.java:531)
  com.rapidminer.gui.ProcessThread.run(ProcessThread.java:61)

Cause
Exception: java.io.StreamCorruptedException
Message: invalid stream header: 1F8B0800
Stack trace:

  java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:783)
  java.io.ObjectInputStream.<init>(ObjectInputStream.java:280)
  com.rapidminer.operator.io.ModelLoader.apply(ModelLoader.java:76)
  com.rapidminer.operator.Operator.apply(Operator.java:656)
  com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
  com.rapidminer.operator.Operator.apply(Operator.java:656)
  com.rapidminer.operator.OperatorChain.apply(OperatorChain.java:377)
  com.rapidminer.operator.Operator.apply(Operator.java:656)
  com.rapidminer.Process.run(Process.java:571)
  com.rapidminer.Process.run(Process.java:541)
  com.rapidminer.Process.run(Process.java:531)
  com.rapidminer.gui.ProcessThread.run(ProcessThread.java:61)
Elite

Re: SOM Reduction - Preprocessing Save doesn't work

Hi,
I just found your bug. Funny one Smiley Happy Since the SOM was a thread itself, the IOObjectWriter tried to save nearly everything. Complete rapidMiner, libraries, everything, since the thread knew its threadpool, which probably knew all threads including the gui...And when the IOObjectWriter started to save itself, it hang up. What was probably a good idea after all Smiley Happy

In the newest repository version it now works.

Greetings,
  Sebastian
Old World Computing - Establishing the Future

Check out the Jackhammer Extension for RapidMiner! Crunch more data easier and with up to 700% speed up! Available only here

Re: SOM Reduction - Preprocessing Save doesn't work

Great. How or where do I access the repository version? I assume it's CVS somewhere, and is the repository built into binaries or do I have to compile ... ?
Moderator

Re: SOM Reduction - Preprocessing Save doesn't work

Hi,

http://rapid-i.com/content/view/25/48/lang,de/

explains the anonymous CVS access to our developer version via Eclipse. The developer CVS repository naturally only contains the source code. The compilation or running of RapidMiner, respectively, is pretty easy using Eclipse. Just follow the instructions which are explained on the aforementioned website.

Regards,
Tobias

Re: SOM Reduction - Preprocessing Save doesn't work

Checked it out with eclipse. Compile worked great, including the fix on the SOM preprocessing saving/loading issue. Thanks.