Why is my fetch taking so long?

DU_caDU_ca Member Posts: 2 Newbie
I am able to connect to my Oracle database system, but the "Fetching Database Tables" process takes 45 minutes to complete.  Why is this happening?

Answers

  • rfuentealbarfuentealba Moderator, RapidMiner Certified Analyst, Member, University Professor Posts: 361   Unicorn
    Hello @DU_ca:

    It appears to be something not fully compliant in the JDBC driver for Oracle that requires an extra step. Let's call @mschmitz and ask him if he knows someone who can help us, because I tried to clone the RapidMiner Studio source code and found out that the open source version of it doesn't include the JDBC connectors, so I just used Navicat and copy/pasted :-(

    All the best,

    Rodrigo.
  • DU_caDU_ca Member Posts: 2 Newbie
    Thanks for trying, Rodrigo.  Unfortunately, I'm still having this problem and so I guess RapidMiner is not a good option for me.
  • jczogallajczogalla Employee, Member Posts: 122   RM Engineering
    Hi @DU_ca!
    Can you maybe share your process and/or explain what exactly you want to do?
    If you are trying to load all your data from your database, this might be quite some effort to do if you have a huge database. RapidMiner has to convert each table into its own memory format, which might take a lot of RAM and time.
    If you want to manipulate a lot of your database data, especially preprocessing, it maybe would help you to use the in-DB extension from the marketplace, as discussesd here: https://community.rapidminer.com/discussion/55642/challenge-with-rm-server-running-out-of-memory#latest
    I hope this helps!
    Jan

    sgenzer
  • sgenzersgenzer 12Administrator, Moderator, Employee, RapidMiner Certified Analyst, Community Manager, Member, University Professor, PM Moderator Posts: 2,320  Community Manager
    using the caching tools in the Jackhammer extension can also be helpful here...

    https://oldworldcomputing.com/en/tutorial-introduction-to-caching-functions-of-the-jackhammer-extension-by-old-world-computing/

    Scott
    jczogalla
  • rfuentealbarfuentealba Moderator, RapidMiner Certified Analyst, Member, University Professor Posts: 361   Unicorn
    Hey @jczogalla and @sgenzer,

    Since I've found this problem too, I can provide a few details:

    This is not about the amount of data. In fact, the error comes from trying to retrieve the list of tables from a database in the Parameters list. However, this was solved in RapidMiner 9.3. It's still a little slow (40 seconds on my MacBook i9-9900) but manageable.



    @DU_ca, you may want to upgrade to 9.3 to see if this works.

    All the best,

    Rodrigo.

    sgenzerjczogalla
Sign In or Register to comment.