So we updated to Lightning last week.  This week I updated the app for lightning in outlook.  It works find but the old side panel for classic outlook is still there.  I only have one salesforce app in my manage app file, so removing it removes both the new lightning panel and the old classic panel.  How do I get ride of the classic panel on the far right side of my outlook page.  I want to make sure I have this right before doing this on all our sales team's emails. 

Thanks