Options
"How to process thousands of CSV files (data files) in batch mode?"
Contributor I
There are about 3000 CSV files which are source data files. Each file has 7 attributes but different instances.
E.G.
Date Open-Price High-Price Low-Price Close-price Amount Volume-money
07/02/2007 0.83 0.85 0.82 0.82 1624001 13294947
07/03/2007 0.81 0.83 0.79 0.81 2837039 22465612
07/04/2007 0.81 0.84 0.8 0.82 2587505 21029224
07/05/2007 0.81 0.85 0.79 0.81 2178333 17773198
07/06/2007 0.79 0.84 0.79 0.84 2496353 20309706
07/09/2007 0.83 0.87 0.83 0.86 2556154 21822494
07/10/2007 0.86 0.86 0.82 0.82 2288000 18957010
07/11/2007 0.81 0.85 0.79 0.84 1541856 12688349
07/12/2007 0.84 0.88 0.83 0.88 3545073 30796872
07/13/2007 0.88 0.9 0.85 0.86 2222693 19366442
07/16/2007 0.86 0.88 0.84 0.85 1050127 8936699
I put all CSV files under one file folder.The files are named XXXXXXYY.csv E.G.:
000010SZ.csv
600020SH.csv
000035SZ.csv
...
1. I want to create a process to select every two CSV files to join the atrributes into one Example Set.
2. Calculate correlaction of open-price, open-price_from_ES
3. Write result to a text file.
How to make a batch to make a selection of two data files from thousands source data files and loop all the files to calculate the correlation?
Thanks in advance.
Gerald
E.G.
Date Open-Price High-Price Low-Price Close-price Amount Volume-money
07/02/2007 0.83 0.85 0.82 0.82 1624001 13294947
07/03/2007 0.81 0.83 0.79 0.81 2837039 22465612
07/04/2007 0.81 0.84 0.8 0.82 2587505 21029224
07/05/2007 0.81 0.85 0.79 0.81 2178333 17773198
07/06/2007 0.79 0.84 0.79 0.84 2496353 20309706
07/09/2007 0.83 0.87 0.83 0.86 2556154 21822494
07/10/2007 0.86 0.86 0.82 0.82 2288000 18957010
07/11/2007 0.81 0.85 0.79 0.84 1541856 12688349
07/12/2007 0.84 0.88 0.83 0.88 3545073 30796872
07/13/2007 0.88 0.9 0.85 0.86 2222693 19366442
07/16/2007 0.86 0.88 0.84 0.85 1050127 8936699
I put all CSV files under one file folder.The files are named XXXXXXYY.csv E.G.:
000010SZ.csv
600020SH.csv
000035SZ.csv
...
1. I want to create a process to select every two CSV files to join the atrributes into one Example Set.
2. Calculate correlaction of open-price, open-price_from_ES
3. Write result to a text file.
How to make a batch to make a selection of two data files from thousands source data files and loop all the files to calculate the correlation?
Thanks in advance.
Gerald
Tagged:
0
