speeding up Numerical To Binomial?

lansuminc
lansuminc New Altair Community Member
edited November 2024 in Community Q&A
After some pre-processing I ended up with a big table of 0's and 1's. These have type real. But I want them to become binomials.
The operator Numerical To Binomial does the trick... slowly. For my table of (test!) data (1.5mm rows of 6k columns) it is now taking 2.5 hours and no sign of it being complete.

Does anyone have a trick to speed things up?

One thing I thought of is writing the table to a temp CSV file and then use the smart guessing of the CSV import to make them binomial.
I've tried the GuesTypes operator, but that I killed after having run for an hour or so...
Writing to a tmp database table does not work as my postgresql database does not like more than 1600 columns.

Answers

  • Andrew2
    Andrew2 New Altair Community Member
    Try the Materialize Data operator immediately before the slow operator.

    Regards

    Andrew

Welcome!

It looks like you're new here. Sign in or register to get started.

Welcome!

It looks like you're new here. Sign in or register to get started.