Options
2 Database mySQL - query depend each other
Contributor I
Hello,
i have 2 mySQL Databases. First one have event entries with a date. Second database have many entries
25.567.157 rows with Value and Date.
I'd like classify the second Database depend on event entries of first database.
For example: event date is 12.08.2009 14:00:00 (DD.MM.YYYY HH:MM:SS) in first database
then i would analyse database one for this one day
at the moment i do this manually entry in first database query
SELECT `date`, `value`
FROM `myvaluelist`
WHERE `date` between 20090812000000 and 20090812235959
i won't read the whole database one with 1GB data - i only would query entries which are as event in database 2
my thought is to get from database 2 all dates from events and export this to a file - it ist possibly to transform this
automatically in a WHERE clausel and write to file - so i can set for database one the query by file
or exists an other method? loading complete database one with select * from myvaluelist occurs this
"Out of Memory - Java Heap size error"
database two have 5.215 entries with events
Thank you for help!
i have 2 mySQL Databases. First one have event entries with a date. Second database have many entries
25.567.157 rows with Value and Date.
I'd like classify the second Database depend on event entries of first database.
For example: event date is 12.08.2009 14:00:00 (DD.MM.YYYY HH:MM:SS) in first database
then i would analyse database one for this one day
at the moment i do this manually entry in first database query
SELECT `date`, `value`
FROM `myvaluelist`
WHERE `date` between 20090812000000 and 20090812235959
i won't read the whole database one with 1GB data - i only would query entries which are as event in database 2
my thought is to get from database 2 all dates from events and export this to a file - it ist possibly to transform this
automatically in a WHERE clausel and write to file - so i can set for database one the query by file
or exists an other method? loading complete database one with select * from myvaluelist occurs this
"Out of Memory - Java Heap size error"
database two have 5.215 entries with events
Thank you for help!
Tagged:
0
Answers
this is a very common preprocessing task we are facing and it is completely solvable using RapidMiner.
The problem is, that I cannot give all the knowledge in detail, it would simply take far too long. This usually is taught in a two day seminar.
The bottom line is, that you can load the complete first data base and transform it's values, so that you can use them later on for including them into the query. After you did this, you can loop over each single example and extract the values for the query into a macro. After doing this, you can make a query and include the macros into the query string. During execution they will be replaced by the values stored into the macros. This way you can load your data in chunks for each single event data.
You will need the Loop Examples and Extract Data operator for this.
I hope this helps. If you need more information or doesn't have the time to find it out yourself, you might book a seminar or webinar from our shop.
Greetings,
Ā Sebastian
Thank you for the information. I hope i get running it.
Descripes the Whitpaper "How to Extend RapidMiner 5.0" Preis: ā¬40.00Ā this step, how to set up the preprocessing task with sql databases?
Notify i'm student and i have not the pocket money for booking a webinar or seminar.
unfortunately it does not. It describes how to write own code to extend RapidMiner's functionality. But what you need is already covered by RapidMiner. If you search the forum for macros, you will find several examples how to use them. I bet you can comprehend from them how it works out for you.
Greetings,
Ā Sebastian