I have a Prep flow where I am passing the output of a union to a Python script.
I've ordered the union result fields as follows:
and when I put an output to xlsx step directly after the union to test it, the fields are indeed in this order:
I then insert my Python script step between the union and the output step. One of the first lines within the script is:
print(input.info())
to peek at what's being passed in by Prep. And I see:
The columns are in a different order, which is causing my script to fail.
Now of course I could edit my Python script to handle the fields in this order, but I envisage having to add / change fields a number of times in the future and its much less work I can just reorder them in the union in Prep and have confidence they will be passed to the script in that order. So I'd really like to understand why the order here differs from that being passed to Excel if I remove the script step and output straight from the union step.
Any suggestions please?
Update:
I've got everything working by adding a line at the start of my script to force the order of the columns:
input = input[['Title','change_start_date_and_time','change_end_date_and_time','change_reference','Table Names','status','Category']]
but I'd like to understand if I'll always need this at the start of scripts used within Prep. i.e. is there possibly a bug within prep causing the columns to be passed in the wrong order - or is it something I'm doing wrong when defining the union (the previous step)
