Problem with Date Time creation
Contributor IIHi everyone,
I am dealing with a problem regarding date-times in Rapidminer.
I read a CSV with a nominal variable "date" with format "dd-MM-yyyy hh:mm:ss". I am using the Nominal to Date operator, converting this nominal variable to a date_time variable.
The same way, I have a loop where I increase another date attribute by one hour, using the Generate attributes with the function expression: "date_add(myDate, 1, DATE_UNIT_HOUR").
Problem comes when, inside the loop, I try to Join the two datasets. These are the values for these variables in both datasets:
Dataset1 Dataset2
| Tue Jan 09 08:00:00 CET 2018 | Tue Jan 09 08:00:00 CET 2018 |
| Tue Jan 09 09:00:00 CET 2018 | Tue Jan 09 09:00:00 CET 2018 |
| Tue Jan 09 10:00:00 CET 2018 | Tue Jan 09 10:00:00 CET 2018 |
| Tue Jan 09 11:00:00 CET 2018 | Tue Jan 09 11:00:00 CET 2018 |
| Tue Jan 09 12:00:00 CET 2018 | Tue Jan 09 00:00:00 CET 2018 |
| Tue Jan 09 13:00:00 CET 2018 | Tue Jan 09 13:00:00 CET 2018 |
| Tue Jan 09 14:00:00 CET 2018 | Tue Jan 09 14:00:00 CET 2018 |
| Tue Jan 09 15:00:00 CET 2018 | Tue Jan 09 15:00:00 CET 2018 |
| Tue Jan 09 16:00:00 CET 2018 | Tue Jan 09 16:00:00 CET 2018 |
Problem is that the date on red, is written as "Jan 9, 2018 12:00:00 AM CET" in one dataset, and "Jan 9, 2018 12:00:00 PM CET" in the other one... I don't know why this happens, neither how I can solve it.
Any ideas on this matter?
Thanks in advance,
Answers
Huh, is there a Tue Jan 09 00:00:00 CET 2018 in Dataset 1? 00:00:00 is midnight, which we all know is differened from 12:00:00 (noon). The Join is throwing an error because the date-times are different there.
Just a sanity check, but does RapidMiner recognize both datasets as date-times?
hi...joining timestamps can be tricky. If it were me, I would convert all timestamps to epoch using the date_millis() function in Generate Attributes. This way you know exactly what you're dealing with.
Scott