Start tracking your progress
Trailhead Home
Trailhead Home

Package Your App and Automate CI/CD

Now that you have the app source code hosted on GitLab.com and you’ve added CI/CD environment variables, it’s time to automate. 

Create Your DreamHouse Unlocked Package

In this step, you create an unlocked package of your DreamHouse project and install it into your Trailhead Playground.

Note

Note

If you are new to Salesforce CLI and packages, you may benefit from taking the Package Development Model module and Unlocked Packages for Customers module before continuing with this project.

  1. From the command line, navigate to your DreamHouse-sfdx directory, if you’re not there already.
  2. Create an unlocked package named DreamHouse.
    sfdx force:package:create --path force-app --name "DreamHouse" --description "GitLab CI Package Example" --packagetype Unlocked
Note

Note

Note that you give this package an explicit name of DreamHouse, which is the same as the PACKAGE_NAME environment variable that you previously set in the GitLab CI/CD configuration page. The package name is case-sensitive, and must match exactly the value of the environment variable PACKAGE_NAME. If you spelled the name differently here than what you entered into GitLab, this is a good time to update your GitLab CI/CD variable so that the values match.

You should see something like the following:

sfdx-project.json has been updated.
Successfully created a package. 0Ho1U000000fxczSAA
=== Ids
NAME        VALUE
──────────  ──────────────────
Package Id  0Ho1U000000fxd4SAA

You have successfully created an unlocked package of your DreamHouse project. The package name (also referred to as the package alias) is now associated with the Package Id (0Ho). Salesforce CLI commands that accept Package Ids as arguments also accept these user-friendly aliases. How convenient!

The package alias to package id mappings were automatically added to your sfdx-project.json file of the project when the package was created. Now you need to commit these changes to GitLab.

  1. Add the sfdx-project.json file to a git commit with a message describing this change.
    git add sfdx-project.json
    git commit -m "Add DreamHouse package id and alias"
  2. Push the changes to GitLab.
    git push -u origin master

So far you've created the definition of the DreamHouse unlocked package. The package starts empty with no metadata in it just like a new change set starts empty. In the next steps, you automate the creation of new package versions. A package version represents a snapshot of your project's metadata that you then can install into orgs, such as scratch orgs, sandboxes, and production.

Create and Run a GitLab Pipeline

Create a .gitlab-ci.yml file in your repository. This will define stages of testing, integration, and deployment to automate delivery of your Salesforce application. The pipeline automatically uses scratch orgs for these steps (for example, unit testing) and deploys the final version of your application post-approval to a persistent environment, such as your Trailhead Playground. 

This is essentially your blueprint for automation. 

  1. Log in to https://gitlab.com if you’re not already there.
  2. Click Projects dropdown and search for DreamHouse to find your cloned repository, and select DreamHouse-sfdx.
    DreamHouse-sfdx selected from the Projects dropdown, with DreamHouse appearing in the search box.
  3. Click Repository then Files.
    Repository menu in the left side bar.
  4. Click Web IDE.
    Web IDE button on the repository files page.
  5. Click the New File icon New File iconjust above the directory.
    New File icon in Web IDE
  6. Click .gitlab-ci.yml to automatically create a new file with the same name.
    Creating a new file in Web IDE
  7. In the .gitlab-ci.yml text editor that opens, enter the following contents. We added comments indicated by the hashtag (#) so you can inspect what’s happening. You can also check out GitLab Docs if you want to learn more about yml files.
    #
    # GitLab CI/CD Pipeline for deploying DreamHouse App using Salesforce DX
    #
    #
    # Run these commands before executing any build jobs,
    # such as to install dependencies and set environment variables
    #
    before_script:
        # Decrypt server key
        - openssl enc -aes-256-cbc -md sha256 -salt -d -in assets/server.key.enc -out assets/server.key -k $SERVER_KEY_PASSWORD -pbkdf2
        # Install jq, a json parsing library
        - apt update && apt -y install jq
        # Setup SFDX environment variables
        # https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_cli_env_variables.htm
        - export SALESFORCE_CLI_URL=https://developer.salesforce.com/media/salesforce-cli/sfdx-linux-amd64.tar.xz
        - export SFDX_AUTOUPDATE_DISABLE=false
        - export SFDX_USE_GENERIC_UNIX_KEYCHAIN=true
        - export SFDX_DOMAIN_RETRY=600
        - export SFDX_LOG_LEVEL=DEBUG
        # Install Salesforce CLI
        - mkdir sfdx
        - wget -qO- $SALESFORCE_CLI_URL | tar xJ -C sfdx --strip-components 1
        - './sfdx/install'
        - export PATH=./sfdx/$(pwd):$PATH
        # Output CLI version and plug-in information
        - sfdx update
        - sfdx --version
        - sfdx plugins --core
    #
    # Define the stages of our pipeline
    #
    stages:
        - code-testing
        - integration-testing
        - app-deploy
    #
    # Stage 1 -- Create a scratch org for code testing
    #
    code-testing:
        stage: code-testing
        script:
            # Authenticate to the Dev Hub using the server key
            - sfdx force:auth:jwt:grant --setdefaultdevhubusername --clientid $SF_CONSUMER_KEY --jwtkeyfile assets/server.key --username $SF_USERNAME
            # Create scratch org
            - sfdx force:org:create --setdefaultusername --definitionfile config/project-scratch-def.json --wait 10 --durationdays 7
            - sfdx force:org:display
            # Push source to scratch org (this is with source code, all files, etc)
            - sfdx force:source:push
            # Assign DreamHouse permission set to scratch org default user
            - sfdx force:user:permset:assign --permsetname DreamHouse
            # Add sample data into app
            - sfdx force:data:tree:import --plan data/sample-data-plan.json
            # Unit Testing
            - sfdx force:apex:test:run --wait 10 --resultformat human --codecoverage --testlevel RunLocalTests
            # Delete Scratch Org
            - sfdx force:org:delete --noprompt
    #
    # Stage 2 -- Create a scratch org, create a package version, and push into org for testing
    #
    integration-testing:
        # Specify file paths that we want available
        # in downstream build stages for this pipeline execution.
        # This is a way to pass dynamic values from one stage to another.
        artifacts:
            paths:
                - PACKAGE_VERSION_ID.TXT
                - SCRATCH_ORG_USERNAME.TXT
        stage: integration-testing
        script:
            # Authenticate to the Dev Hub using the server key
            - sfdx force:auth:jwt:grant --setdefaultdevhubusername --clientid $SF_CONSUMER_KEY --jwtkeyfile assets/server.key --username $SF_USERNAME
            # Create scratch org
            - sfdx force:org:create --setdefaultusername --definitionfile config/project-scratch-def.json --wait 10 --durationdays 7
            - sfdx force:org:display
            # Increment package version number
            - echo $PACKAGE_NAME
            - PACKAGE_VERSION_JSON="$(eval sfdx force:package:version:list --concise --released --packages $PACKAGE_NAME --json | jq '.result | sort_by(-.MajorVersion, -.MinorVersion, -.PatchVersion, -.BuildNumber) | .[0] // ""')"
            - echo $PACKAGE_VERSION_JSON
            - IS_RELEASED=$(jq -r '.IsReleased?' <<< $PACKAGE_VERSION_JSON)
            - MAJOR_VERSION=$(jq -r '.MajorVersion?' <<< $PACKAGE_VERSION_JSON)
            - MINOR_VERSION=$(jq -r '.MinorVersion?' <<< $PACKAGE_VERSION_JSON)
            - PATCH_VERSION=$(jq -r '.PatchVersion?' <<< $PACKAGE_VERSION_JSON)
            - BUILD_VERSION="NEXT"
            - if [ -z $MAJOR_VERSION ]; then MAJOR_VERSION=1; fi;
            - if [ -z $MINOR_VERSION ]; then MINOR_VERSION=0; fi;
            - if [ -z $PATCH_VERSION ]; then PATCH_VERSION=0; fi;
            - if [ "$IS_RELEASED" == "true" ]; then MINOR_VERSION=$(($MINOR_VERSION+1)); fi;
            - VERSION_NUMBER="$MAJOR_VERSION.$MINOR_VERSION.$PATCH_VERSION.$BUILD_VERSION"
            - echo $VERSION_NUMBER
            # Create packaged version
            - export PACKAGE_VERSION_ID="$(eval sfdx force:package:version:create --package $PACKAGE_NAME --versionnumber $VERSION_NUMBER --installationkeybypass --wait 10 --json | jq -r '.result.SubscriberPackageVersionId')"
            # Save your PACKAGE_VERSION_ID to a file for later use during deploy so you know what version to deploy
            - echo "$PACKAGE_VERSION_ID" > PACKAGE_VERSION_ID.TXT
            - echo $PACKAGE_VERSION_ID
            # Install package in DevHub org (this is a compiled library of the app)
            - sfdx force:package:list
            - sfdx force:package:install --package $PACKAGE_VERSION_ID --wait 10 --publishwait 10 --noprompt
            # Assign DreamHouse permission set to scratch org default user
            - sfdx force:user:permset:assign --permsetname DreamHouse
            # Add sample data into app
            - sfdx force:data:tree:import --plan data/sample-data-plan.json
            # Run unit tests in scratch org
            - sfdx force:apex:test:run --wait 10 --resultformat human --codecoverage --testlevel RunLocalTests
            # Get the username for the scratch org
            - export SCRATCH_ORG_USERNAME="$(eval sfdx force:user:display --json | jq -r '.result.username')"
            - echo "$SCRATCH_ORG_USERNAME" > ./SCRATCH_ORG_USERNAME.TXT
            # Generate a new password for the scrach org
            - sfdx force:user:password:generate
            - echo -e "\n\n\n\n"
            # Display username, password, and instance URL for login
            # Be careful not to do this in a publicly accessible pipeline as it exposes the credentials of your scratch org
            - sfdx force:user:display
    #
    # Stage 3 -- Promote the package to downstream environment for UAT for example
    #
    app-deploy:
        stage: app-deploy
        # This stage must be started manually as an example of
        # conditional stages that need to wait for an approval,
        # such as waiting for QA signoff from the previous stage.
        when: manual
        script:
            # Read the scratch org username from file created in prior stage
            - export SCRATCH_ORG_USERNAME=`cat ./SCRATCH_ORG_USERNAME.TXT`
            - echo $SCRATCH_ORG_USERNAME
            # Authenticate with your playground or sandbox environment
            - sfdx force:auth:jwt:grant --setdefaultdevhubusername --clientid $SF_CONSUMER_KEY --jwtkeyfile assets/server.key --username $SF_USERNAME
            - sfdx force:config:set defaultusername=$SF_USERNAME
            # Delete Scratch Org that you were inspecting from your browser
            - sfdx force:data:record:delete --sobjecttype ScratchOrgInfo --where "SignupUsername='$SCRATCH_ORG_USERNAME'"
            # Read the package version id from file created in prior stage
            - export PACKAGE_VERSION_ID=`cat ./PACKAGE_VERSION_ID.TXT`
            - echo $PACKAGE_VERSION_ID
            # Promote the package version
            - sfdx force:package:version:promote --package $PACKAGE_VERSION_ID --noprompt
            # Install the package version
            - sfdx force:package:install --package $PACKAGE_VERSION_ID --wait 10 --publishwait 10 --noprompt
  8. Click Commit... to stage the change in Web IDE.
    Committing file in Web IDE
  9. Click Commit... again to open the commit form in Web IDE.
    Committing staged change in Web IDE
  10. If the master branch is not selected, select the master branch and then click Stage & Commit to save the change to your GitLab repository. And now that your repository has a GitLab Pipelines configuration file, GitLab will automatically start your first CI/CD pipeline run.
    Stage and commit change to repository in Web IDE

You just set up some powerful automation that involves the creation of multiple scratch orgs, the deployment and testing of your Apex code, and creating and installing new unlocked package versions into your orgs.

In this yml file, we specify that scratch orgs will be automatically deleted after 7 days. In practice, we recommend setting the duration to 1 day. To change the duration, update the --durationdays parameter when you run sfdx force:org:create to the desired length of time. 

Note

Note

Fun Fact: With a Trailhead Playground org, you can create up to 6 scratch orgs a day. In this project, you will create 5.

Review the Pipeline

Let’s see the CI/CD pipeline you just created.

  1. Click Projects dropdown and search for DreamHouse to find your cloned repository, and select DreamHouse-sfdx.
    DreamHouse-sfdx selected from the Projects dropdown, with DreamHouse appearing in the search box.
  2. Click CI / CD then Pipelines.
    CI/CD menu in the left side bar.
  3. You should see your pipeline running, indicated by “running” under the Status column. Click the pipeline's status to see more.
    Pipeline jobs running
  4. Drill down even further by clicking the code-testing or integration-testing jobs to see their console outputs.
    Pipeline detail page showing pending jobs

Let the pipeline run, it can take several minutes. It will stop after completing the integration-testing stage because the third stage, app-deploy, is configured to start manually rather than automatically.

Why require app deployment to be manual?

That's a great question! GitLab Pipelines are a great tool for automating continuous integration and deployment. In this project, the pipeline you set up assumes there is an approval process before installing the new package version. You definitely want to have a review before updating your app.

If your pipeline is still running, maybe have another drink of water, and appreciate the fact that you have a CI/CD pipeline running, you savvy developer, you.

Note

Note

If your pipeline fails with error "You must have My Domain deployed", unfortunately, this means the Salesforce CLI didn't detect that the org's My Domain was not fully ready before attempting to deploy the metadata. You can retry the pipeline, which will retry with a new scratch org. If you continue to encounter this error, please contact Salesforce support.

What’s Happening With Salesforce?

While you’re waiting, let’s talk about what’s happening on the Salesforce platform right now. With the GitLab pipeline running, Salesforce scratch orgs are being created to execute your Apex tests. Then they’re deleted once the tests are complete. It’s almost like the org never existed, as if a passing dream, one of the reasons scratch orgs are also known as ephemeral orgs.  

In this case, two scratch orgs are created and then taken down—one for the code-testing stage, which validates that the unpackaged metadata is deployable and passes your tests, and one for the integration-testing stage, which validates that the metadata is packageable.

Note

Note

You’re operating on GitLab.com’s Free tier, executing CI code on relatively lower-powered, shared resources. In addition, the DreamHouse application is fairly large with sample data that occupies time to package, hence why the pipeline can take 15+ minutes overall to run.

Resources