RapidMiner 9.7 is Now Available

Lots of amazing new improvements including true version control! Learn more about what's new here.

CLICK HERE TO DOWNLOAD

Unix timestamp converter

MBMMBM Member Posts: 23 Contributor I
edited November 2018 in Help

Hello, 

 

is there a unix timestamp converter in RM? For example I have the value 1473415924 and I want YMDMS. Or do I have to do it with excel first?

 

edit: I found a forumla for excel. Can I somehow use it in RM maybe "Generate Attribute"?

 "("+dateupload+"/86400)+25569+(n/-5/24)" can I 

 

Greets

Marcel

Answers

  • mschmitzmschmitz Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert, University Professor Posts: 2,534  RM Data Scientist

    Dear MBM,

     

    sure. RapidMiner internally maps all dates to Unix timestamps as well. Numerical to Date should solve your problem. There are also some parsers for dates in GenerateAttributes.

     

    ~Martin

    - Head of Data Science Services at RapidMiner -
    Dortmund, Germany
  • bhupendra_patilbhupendra_patil Administrator, Employee, Member Posts: 168  RM Data Scientist

    You can convert the number to date first using the "generate attributes" functions, if the number represents seconds from epoch (jan 1, 1970, 12:00am) the expression will look something like below

    date_add(date_parse("01/01/1970"),Date,DATE_UNIT_SECOND)

     

    Here date is my numeric column

    Basically we are generating a date value by adding ot to offset

     

    If it is some other base date or unit, you will need to modify the expression to suit that.

    see attached example

     

    You can also explore numerical to date operator that works on milliseconds

     

     You can then use the  date_str***  functions to convert to necessary strings as you decide

  • MBMMBM Member Posts: 23 Contributor I

    Thank you guys. It works fine! But the problem is that the expression needs an integer. And I just tryed "Write Excel" and after that I imported the data again in RM and set the data types. Without excel I don't know how to make polynomial to integer. When I try "Numerical to Real" and then "Real to Integer" it does not really work. I unfortunatelly can't remember the certain operator which allowes me to set the data types after I imported them (looks like the import wizzard if i remember correctly). 

     

    edit: it worked fine with "Gues Types". It automatically changed it into integer! 

     

    One more question to the expression date_add(date_parse("01/01/1970"),dateupload,DATE_UNIT_SECOND) how do I adjust the correct time zone? 

  • richardsona4richardsona4 Member, University Professor Posts: 11  University Professor

    Hello,

     

    I am having an issue with the Numerical to Date TimeStamp Converter. I have (13) character length timestamp, in milliseconds. When I convert it, for some reason, I am not able to observe the milliseconds, and the output is wrong for unique epoch records. See images below.

     

    Any guidance is appreciated, as I am a brand-new Rapid Miner user, thanks.

     

    before.PNGBefore ConversionAfter.PNGAfter Conversion

     

    -Anthony

  • Telcontar120Telcontar120 Moderator, RapidMiner Certified Analyst, RapidMiner Certified Expert, Member Posts: 1,497   Unicorn

    Is this really wrong, or is it simply that the ordinary display of date/time fields doesn't include milliseconds?  If you need precision to the millisecond level, why not simply leave the field unconverted in ms format?  Also, you can always copy the original field and use Date to Numeric operator to extract the year, the month, the day, etc, if you want those for aggregations or for reporting or for other purposes.

    Brian T.
    Lindon Ventures 
    Data Science Consulting from Certified RapidMiner Experts
    richardsona4
  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,952  Community Manager

    agreed. I cannot see anything wrong. The converter is working correctly. If you want a custom date format that shows milliseconds, I'd recommend using Generate Attributes and use one of the "Date Converter" functions.

     

    Scott

     

    richardsona4
  • richardsona4richardsona4 Member, University Professor Posts: 11  University Professor

    Thank you for the quick follow-up. Yes, I will need to aggregate my data at the macro-level, as I have millions of rows gathered over a year for 15K users, so I am looking at seasonality. I will need to do some micro-level millisecond MS-related testing items as well, so I could leave it as is, but since I am new to RM, I was expecting to see something else. 

  • richardsona4richardsona4 Member, University Professor Posts: 11  University Professor

    Thank you Scott.

     

    Would this be the function I need to use?

     

    date_millis()

     

    Thanks,

     

    Anthony

  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,952  Community Manager

    no worries about being a newbie, @richardsona4 - we were all there once ourselves. :)

     

    So if you use date_millis(), you're just going back to epoch. That's not going to help.

     

    In RapidMiner you have to understand that there is a difference between what it displays and what it actually stores when it comes to dates and times. All RapidMiner dates are displayed the same way - to the nearest second. But the millis are there in the background. So if you want to do calculations, don't worry - your date/time calcs will use those milliseconds and off you go.

     

    If you want to display the millisecond part of the date, you can always do that in a separate column for whatever reason you want. I've built an example in the Community Samples repo that you can explore at your leisure:

     

    DATES, TIMES AND MILLISECONDS EXPLANATION DEMO

     

    Screen Shot 2018-08-22 at 8.42.42 PM.png

     

    Scott

     

     

     

     

     

    tftemmeswang123
Sign In or Register to comment.