08-15-2017 12:28 AM
I was able to connect to oracle database and view list of tables in rapidminer studio. However when I am trying to view the content of one table which has around 5M rows (by double clicking on table name) nothing happens. I have also tried using Read Database operator and it takes ages, nothing happens. Even after two hours the Read Database operator is still running.
I also have small tables in oracle database which I was able to read and view one without any trouble.
I even have that same table with 5M rows in a csv format stored locally and rapidminer studio was able to read it successfully, although it took around 20 seconds to open.
For those who guessed it right - yes I have studio license.
Can someone please tell me is this a bug or is there any better way to read from oracle?
08-15-2017 02:38 AM
What column formats are the rows of this table in when stored in Oracle?
If you are selecting all columns from the table & it has some BLOB fields, then it's going to take a while. Try running it as a query with
SELECT <id column of table> FROM tablename;
and seeing how quickly it brings back 1 row.
Next thing to look at are the Indexes on that Oracle table, are they as optimized as they could be?
Also check the packet size on the database is set to the maximum and that the driver matches that packet size. That should reduce the number of total packets sent between client & server when fetching larger result sets.
Other options to try are:
08-16-2017 12:02 PM
A colleague and I tried to replicate the issue with an instance of Oracle DB and RapidMiner Studio and the load took under a minute to read 5m rows and 10 attributes (columns), however, it took up more than 25GB of RAM; I doubt that is the issue if you can load the CSV.
- Is there any restriction or timeout on your network or VPN? We hear of data access issues a lot from people behind VPN.
- How strict is your corporate firewall? is it a problem accessing the DB from other tools like DB front ends and extracting the same table?
- Are there any restrictions in the amount of data you can query out of the Database?
- How many columns does your table have?
I look forward to hearing from you.