I've written an Apex class with methods that parse data out of a CSV file, create account and contact lists with custom fields used as external IDs, and upsert that data using the custom fields as the comparison for duplicates. However, Apex thinks the data in the contact list contains duplicates. I have confirmed in the file itself and via debug code (comparing against a set) that it does not, and I can see when I write debug code to only add the contact to the list if it is not already in the list that Apex thinks every record is a duplicate.
I've written code to ensure and confirm that the list is empty before I start adding to it, I've edited the match rules so they only match on the external ID, and I've uninstalled Dupe Eliminator and Duplicate Check (installed by my predecessor) to be sure they aren't creating restrictions I can't access. Also, I've tried to check Apex triggers, but they are managed and can't be viewed/edited. We do have quite a few other packages/apps installed. Could these be the issue? Could it be a scope of the variable issue? Does anyone have any suggestions on what else might be the issue?
I am including my code below with the caveat that it is pretty messy right now with all my debug code, comments, and variables for other methods in place. Below that code is a screenshot of the debug log starting at the beginning where you can see the first test for the contact existing in the list results in TRUE, before any other contact has been added to the list, and that there is no debug output for the code outputting
any contacts already in the list before I start adding to it. Finally, I am also including a screenshot of our installed packages.
public class mkfa_ImportLegacyData {
//Create an affiliation, contact, and account list for those being manipulated by methods.
public static List<npe5__Affiliation__c> affiliationList = new List<npe5__Affiliation__c>();
public static List<Contact> contactList = new List<Contact>();
public static List<Account> accountList = new List<Account>();
//Create string lists for holding all data & 1 row from document
public static List<String> dataRecs = new List<String>();
public static List<String> fieldValues = new List<String>();
public static List<List<String>> parsedData = new List<List<String>>();
public static Boolean proceedFlag = true;
//Create a contact,account, affiliation, and IDs
public static Contact tempContact = new Contact();
public static VOID parseFileRecs(string docName){
//Get the file
final ContentVersion doc = [SELECT VersionData FROM ContentVersion WHERE Title = :docName AND IsLatest = true];
//******CSV file must be in UTF-8 format******
String bodyString = doc.VersionData.toString();
//Split the content into records by new line character
dataRecs = bodyString.split('\n');
//Debug code
//system.debug('Body of the document to import:'+bodyString);
//Iterate through the records in the file to create a list of recs, sans the header, each of which is a list of fields
for(Integer i=1;i<dataRecs.size();i++){
fieldValues = dataRecs[i].split(',');
parsedData.add(fieldValues);
}
}
public static VOID parseContactFields(String fileName){
//Make sure the contact list to upsert is empty.
contactList.clear();
/*Debug code to be sure contactList is empty. */
for (Contact thisContact : contactList){
System.debug(thisContact);
}
//Parse the data from the file
parseFileRecs(fileName);
proceedFlag = true;
List<Account> contactAcctList = new List<Account>();
List<String> AccountNumList = new List<String>();
String tempVal;
//Set to test for duplicates
//Set<Contact> externalIDs = new Set<Contact>();
//Retrieve all the account numbers from the CSV file
for(Integer j=0;j<parsedData.size();j++){
AccountNumList.add(parsedData[j][1]);
//Debug code
//system.debug('Acct # added: '+AccountNumList[j]);
}
//Query the db for all accounts matching the list of account numbers
contactAcctList = [SELECT Id, mkfa_AccountNum__c FROM Account WHERE mkfa_AccountNum__c IN :AccountNumList];
//Iterate through the records in the file
for(Integer i=0;i<parsedData.size();i++){
//Create contact from this row of data
tempContact.mkfa_External_ID__c = parsedData[i][2];
//Checking for duplicates
/*
if(externalIDs.contains(tempContact)){
system.debug('This ID is a duplicate: ' + tempContact);
}
else{
externalIDs.add(tempContact);
}
*/
//debug Code
//system.debug(i+': Contact External ID: '+ tempContact.mkfa_External_ID__c);
tempContact.FirstName = parsedData[i][4];
//debug code
//system.debug('Contact First Name: '+ tempContact.FirstName);
tempContact.LastName = parsedData[i][5];
//debug code
//system.debug('Contact Last Name: '+ tempContact.LastName);
//Get the account # from the file, test it against the list of accounts we queried earlier in SoQL
tempVal = parsedData[i][1];
for(Integer k=0;k<contactAcctList.size() && proceedFlag==true;k++){
if(contactAcctList[k].mkfa_AccountNum__c==tempVal){
tempContact.AccountId = contactAcctList[k].Id;
proceedFlag = False;
}
}
//Add the contact to the contact list
if(!contactList.contains(tempContact)){
contactList.add(tempContact);
system.debug('Contact added: ' + tempContact);
}
else{
system.debug('Contact is dup: ' + tempContact);
}
//debug code
//system.debug('Added contact: '+tempContact);
proceedFlag = true;
}
//Testing for duplicates. Makes sure the set used to look for dups is the same size as the list of contacts.
/* if(contactList.size()==externalIDs.size()){
system.debug('No problems with dups. Proceed.');
}
else{
system.debug('Hold up. There are dups the SS did not find.');
}
*/
//Try to upsert the contacts *****Use external ID
Database.UpsertResult[] upsertResults = Database.upsert(contactList,Contact.mkfa_External_ID__c,true);
/* try{
upsert contactList Contact.fields.mkfa_External_ID__c;
} catch(DmlException e) {
System.debug('The upsert failed. That is OK. Return to the maze and take this message to guide you: ' +
e.getMessage());
}
*/
}
}
Thanks for the offer! This endeavor has moved to low priority, but if you do find it piques your interest and you want to spend any time helping, I'd appreciate the help to getting it to run correctly.
I realize I can clean up some of the code with options such as break instead of using flags, and using local variables where I don't need continuity between methods, and I will take care of that when I swing back around to this. Also, I've removed code I had been using for debugging but was not using on the last run. Finally, I've only included the methods pertinent to this issue, but I left all the public variables in place, so there may be some that seem unnecessary.
public class mkfa_ImportLegacyData {
//Create an affiliation, contact, and account list for those being manipulated by methods.
public List<npe5__Affiliation__c> affiliationList = new List<npe5__Affiliation__c>();
public List<Contact> contactList = new List<Contact>();
public List<Account> accountList = new List<Account>();
//Create string lists for holding all data & 1 row from document
public List<String> dataRecs = new List<String>();
public List<String> fieldValues = new List<String>();
public List<List<String>> parsedData = new List<List<String>>();
public Boolean proceedFlag = true;
//Create a contact,account, affiliation, and IDs
public Account tempAcct = new Account();
public npe5__Affiliation__c tempAffiliation = new npe5__Affiliation__c();
public ID affiliateAcctID;
//Method to get data from a file object
//***Clean quotation marks out of names in CSV file first.****
public VOID parseFileRecs(string docName){
//Get the file
final ContentVersion doc = [SELECT VersionData FROM ContentVersion WHERE Title = :docName AND IsLatest = true];
//******CSV file must be in UTF-8 format******
String bodyString = doc.VersionData.toString();
//Split the content into records by new line character
dataRecs = bodyString.split('\n');
//Debug code
//system.debug('Body of the document to import:'+bodyString);
//Iterate through the records in the file to create a list of recs, sans the header, each of which is a list of fields
for(Integer i=1;i<dataRecs.size();i++){
fieldValues = dataRecs[i].split(',');
parsedData.add(fieldValues);
}
}
/*Method to parse the contact info from the CSV file, search SQL for accounts matching eT's Account number,
* create a list of those accounts, search that list for each contact to find that contact's corresponding account,
* and populate the contact list accordingly*/
public VOID upsertContacts(String fileName){
//Make sure the contact list to upsert is empty.
contactList.clear();
/*Debug code to be sure contactList is empty. */
for (Contact thisContact : contactList){
System.debug(thisContact);
}
//Parse the data from the file
parseFileRecs(fileName);
proceedFlag = true;
List<Account> contactAcctList = new List<Account>();
List<String> AccountNumList = new List<String>();
String tempVal;
//Retrieve all the account numbers from the CSV file
for(Integer j=0;j<parsedData.size();j++){
AccountNumList.add(parsedData[j][1]);
}
//Query the db for all accounts matching the list of account numbers
contactAcctList = [SELECT Id, mkfa_AccountNum__c FROM Account
WHERE mkfa_AccountNum__c IN :AccountNumList
ORDER BY mkfa_AccountNum__c ASC];
//Iterate through the records in the file
for(Integer i=0;i<parsedData.size();i++){
Contact tempContact = new Contact();
//Create contact from this row of data
tempContact.mkfa_External_ID__c = parsedData[i][2];
tempContact.FirstName = parsedData[i][4];
tempContact.LastName = parsedData[i][5];
//Get the account # from the file, test it against the list of accounts we queried earlier in SoQL
tempVal = parsedData[i][1];
for(Integer k=0;k<contactAcctList.size() && proceedFlag==true;k++){
if(contactAcctList[k].mkfa_AccountNum__c==tempVal){
tempContact.AccountId = contactAcctList[k].Id;
proceedFlag = False;
}
}
//Add the contact to the contact list
if(!contactList.contains(tempContact)){
contactList.add(tempContact);
system.debug('Contact added: ' + tempContact + 'ID: ' + tempContact.Id);
}
else{
system.debug('Contact is dup: ' + tempContact + 'ID: ' + tempContact.Id);
}
proceedFlag = true;
}
//Try to upsert the contacts *****Use external ID
Database.UpsertResult[] upsertResults = Database.upsert(contactList,Contact.mkfa_External_ID__c,true);
}
}
