"Reuse of already opened sql connection"

RLynx

I Loop over big number of values of specific attribute. On each iteration I get additional data from particular (always the same database) and produce calculation results. If it goes too fast Rapid Miner eventually can't open new connection to that database, all ports are in use... so, I have to introduce some kind of delay in order to grab new data from database.. I think, it would be more practical to save and re-use already opened connection... Question: how we can do it?

I do have additional workaround for this problem: Use Remember operator and do filtering on each iteration step... but this involves more memory and time consumption - size of overall data is huge.

Thank you for you comments and suggestions!

MariusHelf

Hi,

the problem why RapidMiner can't reconnect to the server is, as you stated above, that the connection limit on the server has been reached. That happens because of a bug which did not close the connections the server after use. This issue is fixed in current svn and will be released with the next update. Eventually we will also implement reusing connections, but currently we don't support that.

As a workaround you might find a possiblity to split up your many small queries into a larger one and do additional filtering in RapidMiner, as you are currently doing. How big the "big" query should be depends on the process - you have a trade-off between rather slow client side filtering and the overhead of querying the sql server many times.

Regards,
Marius

RLynx

Thank you for good news about an update and good workaround suggestion.

best regards,
RLynx