The Altair Community is migrating to a new platform to provide a better experience for you. The RapidMiner Community will merge with the Altair Community at the same time. In preparation for the migration, both communities are on read-only mode from July 15th - July 24th, 2024. Technical support via cases will continue to work as is. For any urgent requests from Students/Faculty members, please submit the form linked here.

Change unix time stamp on loop

websiteguywebsiteguy Member Posts: 24 Maven
edited August 2020 in Help
Hi - i have spent a lot of time searching the forum, but cant work it out.

Wondering if anyone can help me please. I am trying to loop using a macro and changing a value that is inserted in to the URL (a different UNIX timestamp) on each iteration,.

I want to do this in order to loop sucessive calls to the API to get more history data as per call it is limited to 2000 records (hours in this case).

So I would like to get a longer history in order to try some predictions on it.  

I have worked out how to get the last value of the UNIX time stamp from the first run, and then use that in a loop as the starting point for a second run and merge both sets of data to one file to get 4000 hours. 

The problem is I cant work out how to loop gain and change the value for a more sucessive times each loop. to get more history.

 -  The data is hour based ticker price, from cryptocompare limited to 2000 hours per call,

It is posible to insert the value from the earliest UNIX time stamp date from the first run to complete sucessive runs to get earlier data.

&limit=2000&toTs={the earliest timestamp received}{fsym}&tsym=%{tsym}&limit=%{limit}&toTs=%{toTs}&api_key={api_key}

Process attached

(i have removed my API) you can get one free to test it.  from here

Any help appreciated - thanks, Lee

Best Answer

  • Options
    SGolbertSGolbert RapidMiner Certified Analyst, Member Posts: 344 Unicorn
    Solution Accepted
    Hi @websiteguy

    You should use the Generate Macro operator. With this you can update the macro's value by evaluating the previous value.



  • Options
    hbajpaihbajpai Member Posts: 102 Unicorn
    How about you generate the macro "toTS" inside the loop again by subtracting the time as a function of iteration index. 
  • Options
    websiteguywebsiteguy Member Posts: 24 Maven
    edited August 2020
    Hi @hbajpai, thanks for your reply. Im a novice, been struggling with this for 2 days :)

    Would it be possible to send me an example or screenshot of how to set up subtracting the time as function of iteration index, or modifing the processs file attached to my question as described? it would be greatly appreciated. Regards Lee
  • Options
    websiteguywebsiteguy Member Posts: 24 Maven
    Hi @SGolbert - thanks for reply :) pointed me in the right direction Generate macro was the Key. After some more exsperementing got it working where it loops through and collects data.

    Had to change the name of the second macro generated after the first loop to myMacro1 and use that value in toTs in the loop.

    Very happy to have sorted it out - thanks for your advice. happy to have spent two days trying to work it out and then sucess :) BOOM
  • Options
    itsm6679itsm6679 Member Posts: 1 Newbie
    Hello, I am pulling in submission through a form on our website and it is all formatted in UNIX/UTC date time - that's what I found out by googling this issue, but I haven't found a workable solution to convert UNIX/UTC time code to user readable date and time stamp (01/01/2016 01:00:00 PM) etc. Please help thank you
  • Options
    BalazsBaranyBalazsBarany Administrator, Moderator, Employee, RapidMiner Certified Analyst, RapidMiner Certified Expert Posts: 955 Unicorn

    It is possible to convert into both directions.

    Unix timestamp (large number) to date: Use the Numerical to Date operator. It operates on milliseconds, classic Unix timestamp is in seconds. So if you get values around 1970, just multiply the number with 1000 to get the actual datetime.

    Date/time to Unix (Java) timestamp: Use Generate Attributes with the date_millis() function, or Date to Numerical with the "relative to" setting set to the epoch.

Sign In or Register to comment.