Hi all, I am analyzing a dataset (hyper extract) at my workplace using historical data. It can easily amount to a few hundred million data points in total. This results in an insanely long query time when designing the dashboard.
Thus, I would like to understand what are some of the methods (that works for you) to optimize the query speed to handle big data. The method I tried is streamlining data (ie. forgo columns that I don't need). But the processing time is still lengthy.
Separately, I would also like to ask what happens at the back-end engine (ie. Tableau logic) between using LIVE connection vs Extract, with regards to processing speed.
Thanks gurus!
Hi @Adrian Zinovei , apologies I mean data rows, not data points. My dataset is stored in my workplace, so a rough guide would be like sales transactions (eg. Datetime, Region, State, Store, Sales, Quantity, etc), similar to Superstore dataset but with more historical data.
Okay, I understand extract would be the way to go.
Side track a bit, I am also using Tableau Server at my premises, which is tapping on MS SQL. It has since accumulated quite a fair bit of storage on the TempDB. Want to check whether there is any impact if I wipe/shrink the TempDB data? It is currently set to unlimited because I understand if there is a limit, it will affect the performance of the dashboards. But by setting unlimited means I need to housekeep from time to time. Not sure if there is a better way to handle this.
