Options
"[SOLVED] Aggregation: generating collections grouped by attrbutes"
Contributor II
Hello,
I have a data set with three attributes: first, second and rank.
I have sorted the data-set so that it is ordered first by "first" and then rank.
I need to:
1. generate subsets of the data grouped by "first"
2. for each group data-set select the first k elements
In the end I would like to select any group data-set via the "first" attribute.
Â
Anyone know if this is possible?
If so how?
TIA,
Hugo F
P.S: I have looked at Aggregation and Collections to no avail
I have a data set with three attributes: first, second and rank.
I have sorted the data-set so that it is ordered first by "first" and then rank.
I need to:
1. generate subsets of the data grouped by "first"
2. for each group data-set select the first k elements
In the end I would like to select any group data-set via the "first" attribute.
Â
Anyone know if this is possible?
If so how?
TIA,
Hugo F
P.S: I have looked at Aggregation and Collections to no avail
Tagged:
0
Answers
try to use the Loop Values operator to loop over the values of "first". In the inner process, use "Filter Examples" such that you have only examples with the current value of first (you need the attribute_value_filter with an expression like "first=%{loop_value}". Then Sort by rank and use Filter Example Range to keep only the first k elements. Outside of the loop, you can use Append to recombine the results of the loop operator.
Best regards,
Marius
Thanks once again for the input. Worked as expected.Â
Took some time though (36.34 min) because I am dealing with
roughly 8.36 m records.
Which brings me to my next question. Yesterday I considered
using a DB which I expect is useful due to the indexing. So
my question is: do these "Filter" operators also use indexes or
would I be better off using SQL for the "filtering"?
TIA,
Hugo F.
Ok. Pretty quick for 8M records though.
Thanks.
Hugo F.