"RM 508 Simple Operators (Data Transformation using PCA)"

Answers

• Options

  jingram Member Posts: 8 Contributor II

  July 2010

  Oh, and if any one would like to correct a typo, the message:

  No data was deliverd at port PCA.example set input. 

  should read:

  No data was delivered at port PCA.example set input. 

  "delivered" was spelled incorrectly
  

  0
• Options

  haddock Member Posts: 849 Maven

  July 2010

  G'Day!
  
  And welcome to the house of shame  In your position I'd go for the white paper, unless loot is extra scarce. There was a big change between 4 and 5, namely, among a lot of other stuff, those ports. An alternative might be to look at the source of the extensions ( series, text, pmml etc..). Code for version 4 for laying down and avoiding I'd say.
  
  Wish I had better newz!
  
  

  0
• Options

  jingram Member Posts: 8 Contributor II

  July 2010

  Hi haddock,
  
  Thanks for your reply.
  
  From reading the topic on the white paper, it suggests that it is mainly for extending RM - at this stage I don't need to extend it but just want to convert my XML file into the equivalent code. If the white paper does indeed contain such code for basic operations I would consider purchasing it, but not if it is mainly directed towards writing extensions.
  
  I understand that this is only partly an open-source community, but I thought there might be some basic unit tests written in RM5 containing simple operations (similar to the wiki) which would eventually be used to update the wiki anyway.
  
  If the source of the extensions (e.g. text) contains such code to connect operators, what would be a good package to look in?
  
  Thanks again
  
  P.S. this is the 2nd piece of software that I have used written in Germany and I must say the quality of both has amazed me!

  0
• Options

  haddock Member Posts: 849 Maven

  July 2010

  G'day Jon!
  
  I wasn't really promoting the white paper as a guide for writing an application ( am I right in thinking Seb may produce one ? ), more as a quick way of understanding the classes, how they connect, and their required inputs. This latter point is a crucial difference between 4 and 5 - for example in 4 learner classes could be interrogated for their capabilities, now they use port conditions ( I had a Prolog parser which examined the classes in 4 for parallelism, and found this out the hard way! ).
  
  If time is pressing and loot is not, I still think that you could figure how to make an application out of your XML by understanding the implications of its XML operators. However, if time is pressing and so is loot, then dive into the source of the extensions and see how you get on. Finally, if neither press, urge Seb to burst into print!
  
  Happy coding!
  
  

  0
• Options

  jingram Member Posts: 8 Contributor II

  July 2010

  Howdy, Thanks again for your reply! I was lucky to find a small snippet in the forums http://rapid-i.com/rapidforum/index.php/topic,1641.msg6389.html#msg6389 that helped me to work out operater connectivity.
  
  I am in the process of doing this in my code and very soon I'll post the code here so that it can be seen by others, but perhaps if it's good enough and a simple-enough example, it can add to the article on the wiki page.
  
  Jon

  0
• Options

  haddock Member Posts: 849 Maven

  July 2010

  Nice one Jon, well done  ;D

  0
• Options

  land RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 2,531 Unicorn

  July 2010

  Hi all,
  yes, we are going to publish something for integrating RapidMiner. Might be I will start next week with writing it. But I would like to finish my other projects first...
  One of this projects is to improve operator documentation and since we decided to use the wiki (and hopefully the help of some users) for this, it's useless to add any new content. The totally outdated content will be removed completely.
  
  By the way: This software is totally open source and hence it is an open source community. The fact that we are working for a company and trying to earn our living costs from support and training does not mean that we are doing something unmoral like closed source products
  
  Thank you for the kind words about the code quality. We are always trying to improve it...
  
  Greetings,
  Sebastian
  

  0
• Options

  jingram Member Posts: 8 Contributor II

  July 2010

  Hi,
  
  As I mentioned, I have the operation connections working (well, it appears to be working but there's possibly a better or "best practices" way to do it). I have posted the entire Java source here in case it helps anyone else. One thing that I am struggling with is connecting my data to the first operator (or, perhaps to the root operator) - can anyone suggest how to do this? Currently, a dummy operator is generating useless data.
  
  After tracing through source, I can show exactly where I think the problem is:

  PerformPCA.java:
  
  process.run(input);
  
  Process.java:
  
  if (input != null) {
  	rootOperator.deliverInput(Arrays.asList(input.getIOObjects()));
  }
  
  ProcessRootOperator.java:
  
  public void deliverInput(List<IOObject> inputs) {
  	processInputExtender.deliver(inputs);		
  }
  
  OutputPortExtender.java:
  
  public void deliver(List<IOObject> inputs) {
  	int i = 0;
  	for (OutputPort port : getManagedPorts()) {
  	        if (port.isConnected()) {                                                          // I believe this line is the problem - but how do I connect this port so the input is delivered?
  			if (i >= inputs.size()) {
  				getPorts().getOwner().getOperator().getLogger().warning("Insufficient input for "+port.getSpec());
  			} else {
  				IOObject input = inputs.get(i);
  				port.deliver(input);
  				String name;
  				if (input instanceof ResultObject) {
  					name = ((ResultObject)input).getName();
  				} else {
  					name = input.getClass().getName();
  				}
  				name += " ("+input.getSource()+")";
  				getPorts().getOwner().getOperator().getLogger().fine("Delivering " + name + " to " + port.getSpec());
  			}
  			i++;
  		}
  	}		
  }
  

  And here is my PerformPCA.java:

  package thesis.rapidminer;
  
  import com.rapidminer.operator.OperatorCreationException;
  import com.rapidminer.tools.OperatorService;
  import com.rapidminer.Process;
  import com.rapidminer.RapidMiner;
  import com.rapidminer.example.Attribute;
  import com.rapidminer.example.ExampleSet;
  import com.rapidminer.example.table.AttributeFactory;
  import com.rapidminer.example.table.DoubleArrayDataRow;
  import com.rapidminer.example.table.MemoryExampleTable;
  import com.rapidminer.operator.IOContainer;
  import com.rapidminer.operator.IOObject;
  import com.rapidminer.operator.ModelApplier;
  import com.rapidminer.operator.Operator;
  import com.rapidminer.operator.OperatorException;
  import com.rapidminer.operator.features.transformation.PCA;
  import com.rapidminer.operator.generator.ExampleSetGenerator;
  import com.rapidminer.operator.ports.InputPort;
  import com.rapidminer.operator.ports.OutputPort;
  import com.rapidminer.operator.preprocessing.normalization.Normalization;
  import com.rapidminer.parameter.Parameters;
  import com.rapidminer.tools.Ontology;
  import java.util.LinkedList;
  import java.util.List;
  import java.util.logging.Level;
  import java.util.logging.Logger;
  
  public class PerformPCA {
  
  	public static void main(String[] args) {
  		RapidMiner.init();
  
  		// Create  the process
  		Process process = createProcess();
  
  		// Print process setup
  		Logger.getLogger(PerformPCA.class.getName()).log(Level.INFO, process.getRootOperator().createProcessTree(0));
  
  		// Create some input
  		//IOContainer input = createInput();
  
  		try {
  			// Run the process
  			process.run();
  			//process.run(input);
  		} catch (OperatorException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  	}
  
  	/**
  	 * Creates input from the PCA tutorial example set.
  	 *
  	 * @return The example set.
  	 */
  	private static IOContainer createInput() {
  		ExampleSet exampleSet = createPCATutorialExampleSet();
  
  		IOContainer container = new IOContainer(new IOObject[]{exampleSet});
  
  		return container;
  	}
  
  	/**
  	 * Generates an example set of data taken from Lindsay Smith's paper
  	 * "A tutorial on Principal Components Analysis".
  	 *
  	 * @return An example set containing the data to reduce.
  	 */
  	private static ExampleSet createPCATutorialExampleSet() {
  		List<Attribute> attributes = new LinkedList<Attribute>();
  
  		attributes.add(AttributeFactory.createAttribute("First", Ontology.REAL));
  		attributes.add(AttributeFactory.createAttribute("Second", Ontology.REAL));
  
  		MemoryExampleTable table = new MemoryExampleTable(attributes);
  
  		double[][] data = {
  			{2.5, 2.4},
  			{0.5, 0.7},
  			{2.2, 2.9},
  			{1.9, 2.2},
  			{3.1, 3.0},
  			{2.3, 2.7},
  			{2, 1.6},
  			{1, 1.1},
  			{1.5, 1.6},
  			{1.1, 0.9}
  		};
  
  		for (int i = 0; i < 10; ++i) {
  			table.addDataRow(new DoubleArrayDataRow(data));
  		}
  
  		ExampleSet exampleSet = table.createExampleSet();
  
  		return exampleSet;
  	}
  
  	/**
  	 * Creates a process that performs PCA.
  	 *
  	 * The process performs the following sequence of operations:
  	 *
  	 * Generate Data (Generate Data)
  	 * Normalize (Normalize)
  	 * PCA (Principal Component Analysis)
  	 * Apply Model (Apply Model)
  	 *
  	 * @return The created process.
  	 */
  	public static Process createProcess() {
  		// Create a process
  		Process process = new Process();
  
  		// Create operator to create some example data
  		Operator inputOperator = createInputOperator();
  
  		// Create the normalise operator
  		Operator normalisationOperator = createNormalisationOperator();
  
  		OutputPort outputOutputPort = inputOperator.getOutputPorts().getPortByName("output");
  		InputPort exampleSetInputPort = normalisationOperator.getInputPorts().getPortByName("example set input");
  
  		// Connect "output" to "example set input"
  		outputOutputPort.connectTo(exampleSetInputPort);
  
  		// Create the PCA operator
  		Operator pcaOperator = createPCAOperator();
  
  		OutputPort exampleSetOutputOutputPort = normalisationOperator.getOutputPorts().getPortByName("example set output");
  		exampleSetInputPort = pcaOperator.getInputPorts().getPortByName("example set input");
  
  		// Connect "output" to "example set input"
  		exampleSetOutputOutputPort.connectTo(exampleSetInputPort);
  
  		// Create the model applier operator
  		Operator modelApplierOperator = createModelApplierOperator();
  
  		OutputPort originalOutputPort = pcaOperator.getOutputPorts().getPortByName("original");
  		OutputPort preprocessingModelOutputPort = pcaOperator.getOutputPorts().getPortByName("preprocessing model");
  
  		InputPort unlabelledDataInputPort = modelApplierOperator.getInputPorts().getPortByName("unlabelled data");
  		InputPort modelInputPort = modelApplierOperator.getInputPorts().getPortByName("model");
  
  		// Connect "original" to "unlabelled data"
  		originalOutputPort.connectTo(unlabelledDataInputPort);
  		// Connect "preprocessing model" to "model"
  		preprocessingModelOutputPort.connectTo(modelInputPort);
  
  		// Add the operators to the process
  		process.getRootOperator().getSubprocess(0).addOperator(inputOperator);
  		process.getRootOperator().getSubprocess(0).addOperator(normalisationOperator);
  		process.getRootOperator().getSubprocess(0).addOperator(pcaOperator);
  		process.getRootOperator().getSubprocess(0).addOperator(modelApplierOperator);
  
  		return process;
  	}
  
  	/**
  	 * Creates the example set generator.
  	 *
  	 * The example set is generated using the "sum classification" function.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createInputOperator() {
  		try {
  			Operator inputOperator = OperatorService.createOperator(ExampleSetGenerator.class);
  
  			// Set parameters
  			inputOperator.setParameter(ExampleSetGenerator.PARAMETER_TARGET_FUNCTION, "sum classification");
  
  			return inputOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  
  	/**
  	 * Creates the normalisation operator.
  	 *
  	 * The normalisation operator normalises the data between 0 and 1.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createNormalisationOperator() {
  		try {
  			Operator normalisationOperator = OperatorService.createOperator(Normalization.class);
  
  			// Set parameters
  			// Normalize the data between 0 and 1
  			normalisationOperator.setParameter(Normalization.PARAMETER_NORMALIZATION_METHOD, String.valueOf(Normalization.METHOD_RANGE_TRANSFORMATION));
  			normalisationOperator.setParameter(Normalization.PARAMETER_MIN, "0");
  			normalisationOperator.setParameter(Normalization.PARAMETER_MAX, "1");
  
  			return normalisationOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  
  	/**
  	 * Creates the PCA operator.
  	 *
  	 * The PCA operator uses a variance threshold of 0.95.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createPCAOperator() {
  		try {
  			Operator pcaOperator = OperatorService.createOperator(com.rapidminer.operator.features.transformation.PCA.class);
  
  			// Set parameters
  			// Maintain enough PCs to account for 95% of the variance in the data
  			pcaOperator.setParameter(PCA.PARAMETER_VARIANCE_THRESHOLD, "0.95");
  
  			return pcaOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  
  	/**
  	 * Creates the model applier operator.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createModelApplierOperator() {
  		try {
  			Operator modelApplierOperator = OperatorService.createOperator(ModelApplier.class);
  
  			// Set parameters
  			modelApplierOperator.setParameter(ModelApplier.PARAMETER_APPLICATION_PARAMETERS, PCA.PARAMETER_VARIANCE_THRESHOLD + Parameters.PAIR_SEPARATOR + "0.95");
  
  			return modelApplierOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  }
  

  Can anyone help? Thanks
  
  EDIT: this isn't really tested as yet and I also believe I might need to connect my last operator (model applier) to a result (e.g. result 1, result 2) as in the XML file....hmmm?

  0
• Options

  jingram Member Posts: 8 Contributor II

  July 2010

  Firstly, sorry if it appears that I am double posting , but I believe I have pretty much finished creating my PCA transformation and I'd like to share it for others if they need, but also if some of the source writers want to suggest that I am doing something the wrong way. In case anyone wants to know the method I used to solve my problems, I basically created the process I wanted in RM, then I created a program that loads a process from XML file and I debugged that program to see each case that the XMLBuilder used to build connections, etc.
  
  The code prints out the results into XML format to System.out and when compared to the results in RM GUI, they match up.
  
  Here is the code that successfully (no guarantee ) gives the same results as the PCA tutorial I used:

  package thesis.rapidminer;
  
  import com.rapidminer.operator.OperatorCreationException;
  import com.rapidminer.tools.OperatorService;
  import com.rapidminer.Process;
  import com.rapidminer.RapidMiner;
  import com.rapidminer.example.Attribute;
  import com.rapidminer.example.ExampleSet;
  import com.rapidminer.example.table.AttributeFactory;
  import com.rapidminer.example.table.DoubleArrayDataRow;
  import com.rapidminer.example.table.MemoryExampleTable;
  import com.rapidminer.operator.ExecutionUnit;
  import com.rapidminer.operator.IOContainer;
  import com.rapidminer.operator.IOObject;
  import com.rapidminer.operator.ModelApplier;
  import com.rapidminer.operator.Operator;
  import com.rapidminer.operator.OperatorException;
  import com.rapidminer.operator.UserError;
  import com.rapidminer.operator.features.transformation.PCA;
  import com.rapidminer.operator.generator.ExampleSetGenerator;
  import com.rapidminer.operator.ports.InputPort;
  import com.rapidminer.operator.ports.OutputPort;
  import com.rapidminer.operator.preprocessing.normalization.Normalization;
  import com.rapidminer.parameter.Parameters;
  import com.rapidminer.tools.Ontology;
  import java.io.File;
  import java.io.IOException;
  import java.util.LinkedList;
  import java.util.List;
  import java.util.logging.Level;
  import java.util.logging.Logger;
  
  public class PerformPCA {
  
  	/**
  	 * Determines whether the data should be normalised before applying PCA.
  	 */
  	public static boolean normalise = false;
  	/**
  	 * The location and name to save the process XML file.
  	 */
  	public static String processFilePathname = "PCATutorial-code/processes/process.xml";
  
  	/**
  	 * Performs PCA on the hard-coded data based on Lindsay Smith's paper.
  	 * 
  	 * @param args The command line arguments.
  	 */
  	public static void main(String[] args) {
  		RapidMiner.init();
  
  		// Create  the process
  		Process process = createProcess();
  
  		// Print process setup
  		Logger.getLogger(PerformPCA.class.getName()).log(Level.INFO, process.getRootOperator().createProcessTree(0));
  
  		// Create some input
  		IOContainer input = createInput();
  
  		try {
  			// Run the process
  			process.run(input);
  
  			// Save the process to XML file
  			try {
  				process.save(new File(processFilePathname));
  			} catch (IOException ex) {
  				Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  			}
  
  			// Print the data at the output port "result 1"
  			try {
  				InputPort result1InputPort = process.getRootOperator().getSubprocess(0).getInnerSinks().getPortByName("result 1");
  
  				try {
  					IOObject data = result1InputPort.getData();
  
  					try {
  						data.write(System.out);
  					} catch (IOException ex) {
  						Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  					}
  				} catch (UserError ex) {
  					Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  				}
  			} catch (OperatorException ex) {
  				Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  			}
  
  			// Print the data at the output port "result 2"
  			try {
  				InputPort result2InputPort = process.getRootOperator().getSubprocess(0).getInnerSinks().getPortByName("result 2");
  
  				try {
  					IOObject data = result2InputPort.getData();
  
  					try {
  						data.write(System.out);
  					} catch (IOException ex) {
  						Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  					}
  				} catch (UserError ex) {
  					Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  				}
  			} catch (OperatorException ex) {
  				Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  			}
  		} catch (OperatorException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  	}
  
  	/**
  	 * Creates input from the PCA tutorial example set.
  	 *
  	 * @return The example set.
  	 */
  	private static IOContainer createInput() {
  		ExampleSet exampleSet = createPCATutorialExampleSet();
  
  		IOContainer container = new IOContainer(new IOObject[]{exampleSet});
  
  		return container;
  	}
  
  	/**
  	 * Generates an example set of data taken from Lindsay Smith's paper
  	 * "A tutorial on Principal Components Analysis".
  	 *
  	 * @return An example set containing the data to reduce.
  	 */
  	private static ExampleSet createPCATutorialExampleSet() {
  		List<Attribute> attributes = new LinkedList<Attribute>();
  
  		attributes.add(AttributeFactory.createAttribute("First", Ontology.REAL));
  		attributes.add(AttributeFactory.createAttribute("Second", Ontology.REAL));
  
  		MemoryExampleTable table = new MemoryExampleTable(attributes);
  
  		double[][] data = {
  			{2.5, 2.4},
  			{0.5, 0.7},
  			{2.2, 2.9},
  			{1.9, 2.2},
  			{3.1, 3.0},
  			{2.3, 2.7},
  			{2, 1.6},
  			{1, 1.1},
  			{1.5, 1.6},
  			{1.1, 0.9}
  		};
  
  		for (int i = 0; i < 10; ++i) {
  			table.addDataRow(new DoubleArrayDataRow(data));
  		}
  
  		ExampleSet exampleSet = table.createExampleSet();
  
  		return exampleSet;
  	}
  
  	/**
  	 * Creates a process that performs PCA.
  	 *
  	 * The process performs the following sequence of operations:
  	 *
  	 * Generate Data (Generate Data)
  	 * ? Normalize (Normalize) - only if the normalise variable is true
  	 * PCA (Principal Component Analysis)
  	 * Apply Model (Apply Model)
  	 *
  	 * @return The created process.
  	 */
  	public static Process createProcess() {
  		// Create a process
  		Process process = new Process();
  
  		ExecutionUnit executionUnit = process.getRootOperator().getSubprocess(0);
  
  		OutputPort input1OutputPort = executionUnit.getInnerSources().getPortByName("input 1");
  
  		Operator normalisationOperator = null;
  
  		if (normalise) {
  			// Create the normalise operator
  			normalisationOperator = createNormalisationOperator();
  
  			// Add the operator to the process
  			executionUnit.addOperator(normalisationOperator);
  
  			InputPort exampleSetInputPort = normalisationOperator.getInputPorts().getPortByName("example set input");
  
  			// Connect "input 1" to "example set input"
  			input1OutputPort.connectTo(exampleSetInputPort);
  		}
  
  		// Create the PCA operator
  		Operator pcaOperator = createPCAOperator();
  
  		// Add the operator to the process
  		executionUnit.addOperator(pcaOperator);
  
  		InputPort exampleSetInputPort = pcaOperator.getInputPorts().getPortByName("example set input");
  
  		if (normalise) {
  			OutputPort exampleSetOutputOutputPort = normalisationOperator.getOutputPorts().getPortByName("example set output");
  
  			// Connect "output" to "example set input"
  			exampleSetOutputOutputPort.connectTo(exampleSetInputPort);
  		} else {
  			// Connect "input 1" to "example set input"
  			input1OutputPort.connectTo(exampleSetInputPort);
  		}
  
  		// Create the model applier operator
  		Operator modelApplierOperator = createModelApplierOperator();
  
  		// Add the operator to the process
  		executionUnit.addOperator(modelApplierOperator);
  
  		OutputPort originalOutputPort = pcaOperator.getOutputPorts().getPortByName("original");
  		OutputPort preprocessingModelOutputPort = pcaOperator.getOutputPorts().getPortByName("preprocessing model");
  
  		InputPort unlabelledDataInputPort = modelApplierOperator.getInputPorts().getPortByName("unlabelled data");
  		InputPort modelInputPort = modelApplierOperator.getInputPorts().getPortByName("model");
  
  		// Connect "original" to "unlabelled data"
  		originalOutputPort.connectTo(unlabelledDataInputPort);
  		// Connect "preprocessing model" to "model"
  		preprocessingModelOutputPort.connectTo(modelInputPort);
  
  		OutputPort labelledDataOutputPort = modelApplierOperator.getOutputPorts().getPortByName("labelled data");
  		OutputPort modelOutputPort = modelApplierOperator.getOutputPorts().getPortByName("model");
  
  		InputPort result1InputPort = executionUnit.getInnerSinks().getPortByName("result 1");
  		InputPort result2InputPort = executionUnit.getInnerSinks().getPortByName("result 2");
  
  		// Connect "labelled data" to "result 1"
  		labelledDataOutputPort.connectTo(result1InputPort);
  		// Connect "model" to "result 2"
  		modelOutputPort.connectTo(result2InputPort);
  
  		return process;
  	}
  
  	/**
  	 * Creates the example set generator.
  	 *
  	 * The example set is generated using the "polynomial" function.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createInputOperator() {
  		try {
  			Operator inputOperator = OperatorService.createOperator(ExampleSetGenerator.class);
  
  			// Set parameters
  			inputOperator.setParameter(ExampleSetGenerator.PARAMETER_TARGET_FUNCTION, "polynomial");
  
  			return inputOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  
  	/**
  	 * Creates the normalisation operator.
  	 *
  	 * The normalisation operator normalises the data between 0 and 1.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createNormalisationOperator() {
  		try {
  			Operator normalisationOperator = OperatorService.createOperator(Normalization.class);
  
  			// Set parameters
  			// Normalize the data between 0 and 1
  			normalisationOperator.setParameter(Normalization.PARAMETER_NORMALIZATION_METHOD, String.valueOf(Normalization.METHOD_RANGE_TRANSFORMATION));
  			normalisationOperator.setParameter(Normalization.PARAMETER_MIN, "0");
  			normalisationOperator.setParameter(Normalization.PARAMETER_MAX, "1");
  
  			return normalisationOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  
  	/**
  	 * Creates the PCA operator.
  	 *
  	 * The PCA operator uses a variance threshold of 0.95.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createPCAOperator() {
  		try {
  			Operator pcaOperator = OperatorService.createOperator(com.rapidminer.operator.features.transformation.PCA.class);
  
  			// Set parameters
  			// Maintain enough PCs to account for 95% of the variance in the data
  			pcaOperator.setParameter(PCA.PARAMETER_VARIANCE_THRESHOLD, "0.95");
  
  			return pcaOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  
  	/**
  	 * Creates the model applier operator.
  	 *
  	 * @return Null if the operator could not be created.
  	 */
  	private static Operator createModelApplierOperator() {
  		try {
  			Operator modelApplierOperator = OperatorService.createOperator(ModelApplier.class);
  
  			// Set parameters
  			modelApplierOperator.setParameter(ModelApplier.PARAMETER_APPLICATION_PARAMETERS, PCA.PARAMETER_VARIANCE_THRESHOLD + Parameters.PAIR_SEPARATOR + "0.95");
  
  			return modelApplierOperator;
  		} catch (OperatorCreationException ex) {
  			Logger.getLogger(PerformPCA.class.getName()).log(Level.SEVERE, null, ex);
  		}
  
  		return null;
  	}
  }
  

  Now onto adding in more operators (outlier detection) and adding some logic for my application to use PCA or ICA or SPCA (a special form of PCA called Simple PCA) etc. I hope I don't have too much trouble writing a custom Operator for SPCA :P
  
  EDIT: it won't print "result 2" because after doing "write" the first time, it actually closes System.out (because "write" closes the OutputStream) :S - I am not sure if this is the intended behaviour.

  0