I have to create a hyper file with single table by joining multiple data frames. I have attached the code and the error message that I'm receiving. My approach is:
1- Read input data and convert to pandas data frame
2- Define temporary tables using Table definition:
3-Create Schema and add the data through iteration
4- Using SQL to join the data
Code:
- Step 1:
- table_one_csv_path = "table_one.csv"
- table_two_csv_path = "table_two.csv"
- table_one = pd.read_csv(table_one_csv_path)
- table_two = pd.read_csv(table_two_csv_path)
- Step 2:
with HyperProcess(telemetry=Telemetry.DO_NOT_SEND_USAGE_DATA_TO_TABLEAU) as hyper:
- # Creates new Hyper file ".
- with Connection(endpoint=hyper.endpoint, database=hyper_name,
- create_mode=CreateMode.CREATE_AND_REPLACE) as connection:
- #Create Schema
- connection.catalog.create_schema("Extract")
- # Creates multiple tables.
- sales = TableDefinition(
- table_name=TableName("Extract", "sales"),
- columns=[
- TableDefinition.Column("Product Key", SqlType.int()),
- TableDefinition.Column("Sales", SqlType.int())
- ],
- persistence=Persistence.TEMPORARY
- )
- products = TableDefinition(
- table_name=TableName("Extract", "products"),
- columns=[
- TableDefinition.Column("Product Key", SqlType.int()),
- TableDefinition.Column("Product Name", SqlType.text())
- ],
- persistence=Persistence.TEMPORARY
- )
- Step 3:
# I think I should not create table as in Table definition I set the persistence to Temporary, But I still receiving error --> "sales" table not found
- connection.catalog.create_table(sales)
- connection.catalog.create_table(products)
- with Inserter(connection, sales) as inserter_1:
- for index, row in table_one.iterrows():
- inserter_1.add_row(row)
- inserter_1.execute()
- with Inserter(connection, products) as inserter_2:
- for index, row in table_two.iterrows():
- inserter_2.add_row(row)
- inserter_2.execute()
Step 4:
- table_names = connection.catalog.get_table_names("Extract")
- path_to_database = Path(hyper_name)
- print(f"Tables available in {path_to_database} are: {table_names}")
- joined_connections = connection.execute_command(f'CREATE TABLE "Extract"."Extract" AS SELECT * FROM {sales.table_name} as A LEFT OUTER JOIN {products.table_name} as B ON A.{escape_name("Product Key")} = B.{escape_name("Product Key")}')
Error Message, while Im creating table using connection.catalog.create_table()
"tableauhyperapi.hyperexception.HyperException: cannot create temporary object in non-temporary schema "
Error Message, while Im not creating the table, but keeping the persistence to Temporary:
"tableauhyperapi.hyperexception.HyperException: table "joined_hyper"."Extract"."sales" does not exist: "
Hello!
The problem is, that temporary tables are only allowed to be in a temporary schema. However, your code tries to place them in the non-temporary "Extract" schema.
I would suggest you still use temporary tables but instead of:
TableName("Extract", "sales")
you just use:
TableName("sales")
Then, the temporary table is automatically placed in the temp-schema.
Also, you might want to take a look at pantab, which is a python library written by tableau users for easy and fast pandas <-> hyper conversions.
It uses a bunch of tricks under the hood which should make it quite a bit faster than the usual approach you took in your program.
Let me know if you have further questions.
Cheers,
Jonas