I want to create a hive table in PARQUET format but also compressed with SNAPPY. Can't find the way with the Store in Hive operator. Any help would be appreciated
@cesar_ortiz Store in Hive has a parameter 'custom storage' If selected, a new parameter called 'impala file format' there you could select 'PARQUET '.
@Pavithra_Rao The parameter 'hive file format' also has the option for PARQUET, ORC, AVRO,… but I wanted to change the compression algorithm.
After experimenting with hive CLI, I realized the Store in Hive operator creates the table in Parquet with the default compression algorithm, in this case, Snappy. Good enough for the moment but would be nice to have the option to change to gzip or lzo.
Answers
Store in Hive has a parameter 'custom storage' If selected, a new parameter called 'impala file format' there you could select 'PARQUET '.
Hope this helps.
The parameter 'hive file format' also has the option for PARQUET, ORC, AVRO,… but I wanted to change the compression algorithm.
After experimenting with hive CLI, I realized the Store in Hive operator creates the table in Parquet with the default compression algorithm, in this case, Snappy. Good enough for the moment but would be nice to have the option to change to gzip or lzo.
Thanks for your help