Automate due date on task

Hi all,

A client requested the Due Date on a Task record type be set automatically based on a few criteria of the Account the task is related to and the Task type field. Depending on the criteria, the due date can be same day, + 1 day, + 3 days, + 7 days, +14 days, etc. Due Date is also read-only for this specific record type.

I used process builder and set up an automation, where anytime a task record is created or edited, the due date is automatically populated based on the criteria of the account and the task type.

This was working fine until recently when the client noticed that anytime a user edits the comments on an existing task on the due date that was set prior, or after that due date, the process builder fires and changes the date. 

I'm not sure how to solve this. I thought about creating a process builder that fires only when a record is created, but I'm thinking if 

#Automation any of the criteria changes, we would need to update the due date.

Any thoughts on a better approach or a correction would be appreciated.

Thanks,