Bigquery Rank() Function - Resources Exceeded
I have a table with 7 columns and ~8.5 mil rows. I'm attempting to select into a destination table with 'Allow large results' checked. Note I've already had to decompose a large
Solution 1:
Due to the current implementation of the window functions, these errors are kind of expected when trying to run window functions over big datasets (as in this case, that requires the large results flag).
While these limitations are in place, I would suggest running the query in multiple steps, as in:
SELECT col1, col2, col3, col4, RANK() OVER (PARTITIONBY col1 ORDERBY col4 DESC) rank FROM [dataset.table]
WHEREABS(HASH(col1)) %4=0(replace 0 with 1, 2, and 3 to complete the whole process - or 4 with a bigger number if resources still exceeded)
Post a Comment for "Bigquery Rank() Function - Resources Exceeded"