I may be missing it, but a combinatio of Profiles and Permission Sets don't seem to be able to achieve this. So I tried a validation rule to restrict edit capabilities to specific users, but that prevents other users from changing record ownership (because the validation fails).
