[SOLVED] Number of Examples in Set

zis

Hi,

I have data in ExampleSet like :

ID	NAME	VALUE
1	Bob	123.0
1	Bob	456.4
1	Bob	4786.4
1	Alice	333.4
1	Brad	88.4
2	Jacob	353.4
2	Jacob	663.4
2	Jacob	633.4

And now, I need to cut data, that every same name in specific  ID is occurs max twice.
The wanted result is:

ID	NAME	VALUE
1	Bob	123.0
1	Bob	456.4
1	Alice	333.4
1	Brad	88.4
2	Jacob	353.4
2	Jacob	663.4

Is that possible in RapidMiner? Any regular expression or something.

Thanks,
Regards,
John

zis

I tried a Loop Examples, but is exists something like condition if/else? Data value I extracted by Filter Examples Macro(data value) and if I said that the Example with name Bob is twice or more, then continue. Exists something like Continue operator and Counter operator?

Marco_Boeck

Hi,

yes it is possible, but currently I can't think of a way of doing it without the "Execute Script" operator. Basically iterate over the sorted example set, count the number of same names, then flag every example which should be removed and afterwards remove all flagged examples.
Though you might want and try to come up with something clever using macros (as counter variables) and the "Branch" operator, it might very well be possible

Regards,
Marco

zis

Wow,

Execution Script seems like a powerful tool. Thanks a lot. I wrote something like that:

ExampleSet exampleSet = operator.getInput(ExampleSet.class);

int instanceCounter = 0;
String lastValue;
int counter = 0;
for(Example example : exampleSet)
{	
	String value = example["InstanceID"];

	if(lastValue == value)
		instanceCounter++;
	else
		instanceCounter = 0;

	if(instanceCounter >= 5)
		exampleSet.remove(counter);		

	counter++;
	lastValue = value;
}

operator.setOutput(exampleSet);

But it throws Exceptions, that something is wrong with method Remove (seems like a parser doesnt know her). Do you know, how can I remove an Example and at the end returns this new ExampleSet to output port? You were talking about flags. What do you mean?

Thanks,
Regards,
John

zis

Ok,

I have changed a method according to IOObject.remove(T class) implementation to exampleSet.remove(example) (from example above), and Exceptions are gone. But the Operator doesnt return the ExampleSet.

MODIFIED:

ok now, i can return the ExampleSet, but the method remove is wrong. Without remove method the process proceed well, but I need this method or similar to remove specific Example.

zis

I have read that you cannot remove Example from ExampleSet while you iterating. So I set names (>2) to value -1. Is any operator, that I can use to remove all examples with NAME attribute and value -1, after Execute Script operator?

Thanks,
Regards,
John

Marco_Boeck

Hi,

you can use the "Filter Examples" operator. Set condition class parameter to "attribute_value_filter" and then simply put in "NAME!=-1" (without the quotes)

Regards,
Marco

zis

Nice one, thanks a lot. Working fine.

Regards,
John