How to transform large CSV files into Json response. I'm assuming memory issue.

Hi, I'm new to Mule but have a task. We have endpoint that gets a call which then picks up a csv file with over 25,000 lines. This file is then sent through a transform which does a bit of string manipulation to extract first name, last name, MI, start date, end date and convert the dates to UTC.

I run this on my local studio and it goes out to the server, downloads the csv and after about 30 seconds is able to map the entire message. I run this on our RTF and it gets past the sftp download, starts to process the message, then redownloads the file, and tries to process the message... does that a few times and times out. I cut the file to 100 lines and it works as expected.

We have increased the vcore and memory a bit but it doesn't help. Without my transform mapping the message it works with a simple json.

I can't do a batch can I? I need the message as one reply, not a series of messages sent somewhere. The vendor doing this call will take the return and import it. If I do a batch it only gives the results but not the actual file.

Heres the code i'm using. Is there a way to simplify the code so it's less processor intensive? Other ideas?

%dw 2.0
import * from dw::core::Strings
output application/json
 
---
payload map {
    careAssignmentID:$."UDF_5",
    System: "API",
    eventDateTime: now() >>'UTC',
    staff:{
        employeeID: $."Provider/Employee_id",
        firstName: substringBefore(substringAfter($."Name"," ")," "),
        lastName: substringBefore($."Name",","),
        middleName: trim(substringAfter((substringAfter($."Name"," "))," ") remove "MD")
    }, 
    role:"",
    startDate:($." Start Date" ++ " " ++ first($."Start Time",2) ++ ":" ++ last($."Start Time",2) ++ ":00 CST") as DateTime {format:"MM/dd/yyyy HH:mm:ss z"} >> 'UTC' as String {format:"yyyy-MM-dd'T'HH:mm:ss"},
    endDate: ($."End Date" ++ " " ++ first($."End Time",2) ++ ":" ++ last($."End Time",2) ++ ":00 CDT") as DateTime {format:"MM/dd/yyyy HH:mm:ss z"} >> 'UTC' as String {format:"yyyy-MM-dd'T'HH:mm:ss"},
    location: {facility: $."UDF_6"}
}