Deal with Extract Macro datetime issue

phivuphivu Member Posts: 34 Guru
edited November 2018 in Help

Hi RapidMiner,

 

I got an issue when using Extract Macro with datetime values. The datetime value i want to extract is "Dec 03, 2016 9:04:51 AM", but after using Extract Macro, the value extracted is "03/12/2016 09:04:51 AM" (I used Log operator to see this value), which mean "Mar 12, 2016 9:04:51 AM" (day becomes month and month becomes day). And then when I used Filter Examples to keep only records after "Dec 03, 2016 9:04:51 AM", i got all records after "Mar 12, 2016 9:04:51 AM", which is not what i want. Do you have any idea to solve this? Thank you very much.

 

I attached my process here.

 

Best,

phivu

Tagged:

Best Answers

  • Edin_KlapicEdin_Klapic Moderator, Employee, RMResearcher, Member Posts: 299 RM Data Scientist
    Solution Accepted

    Hi phivu,

     

    I recommend using Generate Attributes to create a numerical Attribute representing the date value in milliseconds since Jan, 1st 1970 (see screenshot).

    You can then apply Extract Macro and Filter Examples on this attribute.

     

    Best,

    Edinimage.png

     

     

     

     

    phivu
  • phivuphivu Member Posts: 34 Guru
    Solution Accepted

    Thanks Edin, this should work.

     

    Best

    phivu

Answers

  • Thomas_OttThomas_Ott RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,761 Unicorn

    Thanks for the process but I can't see your data. Can you verify that the date-time you actually use in the process is set as date-time format by RapidMiner?

    phivu
  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    I'm not saying this isn't a problem, but it sounds like a date format issue--the difference described is between the American vs the European way of writing dates (3/12/16 is March 12th to an American and Dec 3rd to a European).  The date format could be modified in RapidMiner using Date to Nominal and then Nominal to Date using a different format.  There may be a more efficient way of solving this though.

     

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
    Thomas_Ottphivu
  • phivuphivu Member Posts: 34 Guru

    Hi Thomas,

     

    The datetime i used is under RapidMiner format. I attached the input data in this message.

    The problem here i think is the inconsistence between "Extract Macro" (EM) and "Filter Examples" (FE). FE uses the datetime value from EM but it reads the value in a different format (as what Brian explained above).

     

    Thanks.

     

    Best,

    phivu

    ACV1.csv 942.8K
  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,635 Unicorn

    You can do a similar extraction directly with the "Date to Numeric" operator, and choose units other than milliseconds as well (if you'd rather think about this in a unit that is not as large in absolute terms or that is more meaningful to you like days or weeks).

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
    phivu
Sign In or Register to comment.