I discovered we have thousands of these. Can they and their related Duplicate Record Item be deleted? I'm pretty sure it's because one of our managed packages has a custom dupe management process, and when that is used the single Duplicate Record Item in the Set is not auto-deleted.
I'm a little late to the party, but I solved this by creating a Triggered Flow on the DuplicateRecordSet object to fire "On Update". The Entry Condition is set to "Record Count Less Than or Equal 1" with the "Only when a record is updated to meet the condition requirements" selected. I added a Delete element configured to delete the "Triggering DuplicateRecordSet"
This works because the Record Count field is a Roll-Up Summary and RUS do fire update triggers when their value changes.