How to use Polynomial Regression in rapidminer correctly

rookie · March 2020

Hello, everyone. This is my first forum post asking questions about polynomial regression in rapidminer.

The original data is：x:4194.06 3466.45 2070.08 874.98 corresponding to y：91540.07 109460.36 120338.64 102182.19

As shown in the first flow, the first result expression is obtained by using the polynomial regression operator.

<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">

</context>

</list>

</operator>

</list>

</operator>

</operator>

</process>

</operator>

</process>

The second flow, based on the original data, creates a new list of attributes as x^2=z, and uses the linear regression operator to make the second result expression.

<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">

</context>

</list>

</operator>

</list>

</operator>

</operator>

</list>

</operator>

</operator>

</operator>

</process>

</operator>

</process>

I want to ask why the results of the two processes are not the same, the original data presents a quadratic nonlinear relationship, and why the quadratic expression cannot be made by polynomial regression.

Thanks you very much!

yyhuang · March 2020

Hi @rookie,

Thanks for sharing the data and process.

If we have got four (4) example and train a polynomial regression, we may fail for the model. So I filled up the gap with interpolation to add more data here. Also this Polynomial regression will not perform well without the normalization...

Process attached here for your reference.

<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="false" class="read_excel" compatibility="9.6.000" expanded="true" height="68" name="Read Excel" width="90" x="45" y="136">
    <parameter key="excel_file" value="C:\Users\1\Desktop\question data.xlsx"/>
    <parameter key="sheet_selection" value="sheet number"/>
    <parameter key="sheet_number" value="1"/>
    <parameter key="imported_cell_range" value="A1"/>
    <parameter key="encoding" value="SYSTEM"/>
    <parameter key="first_row_as_names" value="true"/>
    <list key="annotations"/>
    <parameter key="date_format" value=""/>
    <parameter key="time_zone" value="SYSTEM"/>
    <parameter key="locale" value="English (United States)"/>
    <parameter key="read_all_values_as_polynominal" value="false"/>
    <list key="data_set_meta_data_information">
      <parameter key="0" value="x.true.real.attribute"/>
      <parameter key="1" value="y.true.real.attribute"/>
    </list>
    <parameter key="read_not_matching_values_as_missings" value="false"/>
    <parameter key="datamanagement" value="double_array"/>
    <parameter key="data_management" value="auto"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="utility:create_exampleset" compatibility="9.6.000" expanded="true" height="68" name="Create ExampleSet" width="90" x="45" y="34">
    <parameter key="generator_type" value="comma separated text"/>
    <parameter key="number_of_examples" value="100"/>
    <parameter key="use_stepsize" value="false"/>
    <list key="function_descriptions"/>
    <parameter key="add_id_attribute" value="false"/>
    <list key="numeric_series_configuration"/>
    <list key="date_series_configuration"/>
    <list key="date_series_configuration (interval)"/>
    <parameter key="date_format" value="yyyy-MM-dd HH:mm:ss"/>
    <parameter key="time_zone" value="SYSTEM"/>
    <parameter key="input_csv_text" value="x,y&#10;4194.06, 91540.07&#10;3466.45, 109460.36&#10;2070.08,  120338.64&#10;874.98,  102182.19      "/>
    <parameter key="column_separator" value=","/>
    <parameter key="parse_all_as_nominal" value="false"/>
    <parameter key="decimal_point_character" value="."/>
    <parameter key="trim_attribute_names" value="true"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="parse_numbers" compatibility="9.6.000" expanded="true" height="82" name="Parse Numbers" width="90" x="179" y="34">
    <parameter key="attribute_filter_type" value="all"/>
    <parameter key="attribute" value=""/>
    <parameter key="attributes" value=""/>
    <parameter key="use_except_expression" value="false"/>
    <parameter key="value_type" value="nominal"/>
    <parameter key="use_value_type_exception" value="false"/>
    <parameter key="except_value_type" value="file_path"/>
    <parameter key="block_type" value="single_value"/>
    <parameter key="use_block_type_exception" value="false"/>
    <parameter key="except_block_type" value="single_value"/>
    <parameter key="invert_selection" value="false"/>
    <parameter key="include_special_attributes" value="false"/>
    <parameter key="decimal_character" value="."/>
    <parameter key="grouped_digits" value="false"/>
    <parameter key="grouping_character" value=","/>
    <parameter key="infinity_representation" value=""/>
    <parameter key="unparsable_value_handling" value="fail"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="set_role" compatibility="9.6.000" expanded="true" height="82" name="Set Role" width="90" x="313" y="34">
    <parameter key="attribute_name" value="y"/>
    <parameter key="target_role" value="label"/>
    <list key="set_additional_roles">
      <parameter key="x" value="regular"/>
    </list>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="normalize" compatibility="9.6.000" expanded="true" height="103" name="Normalize" width="90" x="447" y="34">
    <parameter key="return_preprocessing_model" value="false"/>
    <parameter key="create_view" value="false"/>
    <parameter key="attribute_filter_type" value="all"/>
    <parameter key="attribute" value=""/>
    <parameter key="attributes" value=""/>
    <parameter key="use_except_expression" value="false"/>
    <parameter key="value_type" value="numeric"/>
    <parameter key="use_value_type_exception" value="false"/>
    <parameter key="except_value_type" value="real"/>
    <parameter key="block_type" value="value_series"/>
    <parameter key="use_block_type_exception" value="false"/>
    <parameter key="except_block_type" value="value_series_end"/>
    <parameter key="invert_selection" value="false"/>
    <parameter key="include_special_attributes" value="true"/>
    <parameter key="method" value="Z-transformation"/>
    <parameter key="min" value="0.0"/>
    <parameter key="max" value="1.0"/>
    <parameter key="allow_negative_values" value="false"/>
    <description align="center" color="transparent" colored="false" width="126">rescale the range</description>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="subprocess" compatibility="9.6.000" expanded="true" height="82" name="Subprocess" width="90" x="581" y="34">
    <process expanded="true">
      <operator activated="true" class="sort" compatibility="9.6.000" expanded="true" height="82" name="Sort" width="90" x="112" y="34">
        <parameter key="attribute_name" value="x"/>
        <parameter key="sorting_direction" value="increasing"/>
      </operator>
      <operator activated="true" class="generate_id" compatibility="9.6.000" expanded="true" height="82" name="Generate ID" width="90" x="246" y="34">
        <parameter key="create_nominal_ids" value="false"/>
        <parameter key="offset" value="0"/>
      </operator>
      <operator activated="true" class="generate_attributes" compatibility="9.6.000" expanded="true" height="82" name="Generate Attributes" width="90" x="380" y="34">
        <list key="function_descriptions">
          <parameter key="INDEX" value="50*(id-1)+1"/>
        </list>
        <parameter key="keep_all" value="true"/>
      </operator>
      <operator activated="true" class="time_series:equalize_numerical_indices" compatibility="9.6.000" expanded="true" height="82" name="Equalize Numerical Indices" width="90" x="514" y="34">
        <parameter key="indices_attribute" value="INDEX"/>
        <parameter key="equalize_method" value="number of examples and range(start,stop)"/>
        <parameter key="number_of_examples" value="custom"/>
        <parameter key="custom_number_of_examples" value="151"/>
        <parameter key="start_value" value="same as original data"/>
        <parameter key="custom_start_value" value="0.0"/>
        <parameter key="stop_value" value="same as original data"/>
        <parameter key="custom_stop_value" value="100.0"/>
        <parameter key="step_size_(numerical)" value="1.0"/>
        <parameter key="replace_type_numerical" value="linear interpolation"/>
        <parameter key="replace_type_nominal" value="previous value"/>
        <parameter key="replace_type_date_time" value="previous value"/>
        <parameter key="replace_value_numerical" value="0.0"/>
        <parameter key="replace_value_nominal" value="unknown"/>
        <parameter key="skip_other_missings" value="true"/>
        <parameter key="replace_infinity" value="true"/>
        <parameter key="replace_empty_strings" value="true"/>
        <parameter key="ensure_finite_values" value="true"/>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="9.6.000" expanded="true" height="82" name="Select Attributes" width="90" x="715" y="34">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attribute" value=""/>
        <parameter key="attributes" value="x|y"/>
        <parameter key="use_except_expression" value="false"/>
        <parameter key="value_type" value="attribute_value"/>
        <parameter key="use_value_type_exception" value="false"/>
        <parameter key="except_value_type" value="time"/>
        <parameter key="block_type" value="attribute_block"/>
        <parameter key="use_block_type_exception" value="false"/>
        <parameter key="except_block_type" value="value_matrix_row_start"/>
        <parameter key="invert_selection" value="false"/>
        <parameter key="include_special_attributes" value="false"/>
      </operator>
      <connect from_port="in 1" to_op="Sort" to_port="example set input"/>
      <connect from_op="Sort" from_port="example set output" to_op="Generate ID" to_port="example set input"/>
      <connect from_op="Generate ID" from_port="example set output" to_op="Generate Attributes" to_port="example set input"/>
      <connect from_op="Generate Attributes" from_port="example set output" to_op="Equalize Numerical Indices" to_port="example set"/>
      <connect from_op="Equalize Numerical Indices" from_port="equalized example set" to_op="Select Attributes" to_port="example set input"/>
      <connect from_op="Select Attributes" from_port="example set output" to_port="out 1"/>
      <portSpacing port="source_in 1" spacing="0"/>
      <portSpacing port="source_in 2" spacing="0"/>
      <portSpacing port="sink_out 1" spacing="0"/>
      <portSpacing port="sink_out 2" spacing="0"/>
    </process>
    <description align="center" color="transparent" colored="false" width="126">add more points to the training set</description>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="multiply" compatibility="9.6.000" expanded="true" height="103" name="Multiply" width="90" x="715" y="34"/>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="polynomial_regression" compatibility="9.6.000" expanded="true" height="82" name="Polynomial Regression (2)" width="90" x="849" y="238">
    <parameter key="max_iterations" value="280"/>
    <parameter key="replication_factor" value="2"/>
    <parameter key="max_degree" value="2"/>
    <parameter key="min_coefficient" value="-10.0"/>
    <parameter key="max_coefficient" value="10.0"/>
    <parameter key="use_local_random_seed" value="false"/>
    <parameter key="local_random_seed" value="23"/>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="generate_attributes" compatibility="9.6.000" expanded="true" height="82" name="Generate Attributes (2)" width="90" x="849" y="34">
    <list key="function_descriptions">
      <parameter key="z" value="x^2"/>
    </list>
    <parameter key="keep_all" value="true"/>
    <description align="center" color="transparent" colored="false" width="126">get quadratic term x^2</description>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="false" class="concurrency:optimize_parameters_grid" compatibility="9.6.000" expanded="true" height="145" name="Optimize Parameters (Grid)" width="90" x="648" y="289">
    <list key="parameters">
      <parameter key="Polynomial Regression.max_iterations" value="[1.0;300;30;linear]"/>
      <parameter key="Polynomial Regression.max_degree" value="[1.0;2;3;linear]"/>
      <parameter key="Polynomial Regression.replication_factor" value="[1.0;2;3;linear]"/>
    </list>
    <parameter key="error_handling" value="fail on error"/>
    <parameter key="log_performance" value="true"/>
    <parameter key="log_all_criteria" value="false"/>
    <parameter key="synchronize" value="false"/>
    <parameter key="enable_parallel_execution" value="true"/>
    <process expanded="true">
      <operator activated="true" class="concurrency:cross_validation" compatibility="9.6.000" expanded="true" height="145" name="Cross Validation" width="90" x="246" y="34">
        <parameter key="split_on_batch_attribute" value="false"/>
        <parameter key="leave_one_out" value="false"/>
        <parameter key="number_of_folds" value="5"/>
        <parameter key="sampling_type" value="automatic"/>
        <parameter key="use_local_random_seed" value="false"/>
        <parameter key="local_random_seed" value="1992"/>
        <parameter key="enable_parallel_execution" value="true"/>
        <process expanded="true">
          <operator activated="true" class="polynomial_regression" compatibility="9.6.000" expanded="true" height="82" name="Polynomial Regression" width="90" x="112" y="34">
            <parameter key="max_iterations" value="50"/>
            <parameter key="replication_factor" value="5"/>
            <parameter key="max_degree" value="5"/>
            <parameter key="min_coefficient" value="-10.0"/>
            <parameter key="max_coefficient" value="10.0"/>
            <parameter key="use_local_random_seed" value="false"/>
            <parameter key="local_random_seed" value="23"/>
          </operator>
          <connect from_port="training set" to_op="Polynomial Regression" to_port="training set"/>
          <connect from_op="Polynomial Regression" from_port="model" to_port="model"/>
          <portSpacing port="source_training set" spacing="0"/>
          <portSpacing port="sink_model" spacing="0"/>
          <portSpacing port="sink_through 1" spacing="0"/>
        </process>
        <process expanded="true">
          <operator activated="true" class="apply_model" compatibility="9.6.000" expanded="true" height="82" name="Apply Model" width="90" x="45" y="34">
            <list key="application_parameters"/>
            <parameter key="create_view" value="false"/>
          </operator>
          <operator activated="true" class="performance_regression" compatibility="9.6.000" expanded="true" height="82" name="Performance" width="90" x="179" y="34">
            <parameter key="main_criterion" value="first"/>
            <parameter key="root_mean_squared_error" value="true"/>
            <parameter key="absolute_error" value="false"/>
            <parameter key="relative_error" value="false"/>
            <parameter key="relative_error_lenient" value="false"/>
            <parameter key="relative_error_strict" value="false"/>
            <parameter key="normalized_absolute_error" value="false"/>
            <parameter key="root_relative_squared_error" value="false"/>
            <parameter key="squared_error" value="false"/>
            <parameter key="correlation" value="false"/>
            <parameter key="squared_correlation" value="false"/>
            <parameter key="prediction_average" value="false"/>
            <parameter key="spearman_rho" value="false"/>
            <parameter key="kendall_tau" value="false"/>
            <parameter key="skip_undefined_labels" value="true"/>
            <parameter key="use_example_weights" value="true"/>
          </operator>
          <connect from_port="model" to_op="Apply Model" to_port="model"/>
          <connect from_port="test set" to_op="Apply Model" to_port="unlabelled data"/>
          <connect from_op="Apply Model" from_port="labelled data" to_op="Performance" to_port="labelled data"/>
          <connect from_op="Performance" from_port="performance" to_port="performance 1"/>
          <portSpacing port="source_model" spacing="0"/>
          <portSpacing port="source_test set" spacing="0"/>
          <portSpacing port="source_through 1" spacing="0"/>
          <portSpacing port="sink_test set results" spacing="0"/>
          <portSpacing port="sink_performance 1" spacing="0"/>
          <portSpacing port="sink_performance 2" spacing="0"/>
        </process>
      </operator>
      <connect from_port="input 1" to_op="Cross Validation" to_port="example set"/>
      <connect from_op="Cross Validation" from_port="model" to_port="model"/>
      <connect from_op="Cross Validation" from_port="example set" to_port="output 1"/>
      <connect from_op="Cross Validation" from_port="performance 1" to_port="performance"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="source_input 2" spacing="0"/>
      <portSpacing port="sink_performance" spacing="0"/>
      <portSpacing port="sink_model" spacing="0"/>
      <portSpacing port="sink_output 1" spacing="0"/>
      <portSpacing port="sink_output 2" spacing="0"/>
    </process>
  </operator>
</process>
<?xml version="1.0" encoding="UTF-8"?><process version="9.6.000">
  <operator activated="true" class="h2o:generalized_linear_model" compatibility="9.3.001" expanded="true" height="124" name="Generalized Linear Model" width="90" x="983" y="34">
    <parameter key="family" value="AUTO"/>
    <parameter key="link" value="family_default"/>
    <parameter key="solver" value="AUTO"/>
    <parameter key="reproducible" value="false"/>
    <parameter key="maximum_number_of_threads" value="4"/>
    <parameter key="use_regularization" value="false"/>
    <parameter key="lambda_search" value="false"/>
    <parameter key="number_of_lambdas" value="0"/>
    <parameter key="lambda_min_ratio" value="0.0"/>
    <parameter key="early_stopping" value="true"/>
    <parameter key="stopping_rounds" value="3"/>
    <parameter key="stopping_tolerance" value="0.001"/>
    <parameter key="standardize" value="false"/>
    <parameter key="non-negative_coefficients" value="false"/>
    <parameter key="add_intercept" value="true"/>
    <parameter key="compute_p-values" value="false"/>
    <parameter key="remove_collinear_columns" value="false"/>
    <parameter key="missing_values_handling" value="MeanImputation"/>
    <parameter key="max_iterations" value="0"/>
    <parameter key="specify_beta_constraints" value="false"/>
    <list key="beta_constraints"/>
    <parameter key="max_runtime_seconds" value="0"/>
    <list key="expert_parameters"/>
  </operator>
</process>

Image: https://us.v-cdn.net/6030995/uploads/editor/1m/8abk398m7fq8.png

Image: https://us.v-cdn.net/6030995/uploads/editor/at/yh356g3hq9rs.png

These two models are close but I can not guarantee the polynomial will output similar coefficient without normalization

I would strongly suggest to use GLM with new attribute manually created or attributes from Auto Feature Engineer.

Happy Rapid-Mining and Stay Healthy!

YY

rookie · March 2020

First of all, thank you for your answer

. According to your description, I am as the data is too little, and not standardized, to lead to the results out? But these four samples are real data , need the four data to construct a yuan quadratic polynomial, Because nonlinear equations can be converted to linear equations , so I use z instead of x2, I have the linear regression equation. But why do with polynomial regression is not to come out, how do you explain that please？Polynomial regression is there any limit to this operator ？

rookie · April 2020

hi @yyhuang，
Sorry in advance, I don't know how to use the function of this forum.That's why it took so long to reply
First of all, thank you for your answer

. According to your description, I am as the data is too little, and not standardized, to lead to the results out? But these four samples are real data , need the four data to construct a yuan quadratic polynomial, Because nonlinear equations can be converted to linear equations , so I use z instead of x2, I have the linear regression equation. But why do with polynomial regression is not to come out, how do you explain that please？Polynomial regression is there any limit to this operator ？

Howdy, Stranger!

Quick Links

Categories

Altair RapidMiner Community

GET HELP. LEARN BEST PRACTICES. NETWORK WITH YOUR PEERS.

How to use Polynomial Regression in rapidminer correctly

Answers