Skip to main content

Use Bulk API 2.0

Learning Objectives

After completing this unit, you’ll be able to:

  • Describe how an asynchronous request differs from a synchronous request.
  • Create a bulk job using the Postman web app.
  • Import data to your Salesforce org by adding data to a job.
  • Monitor a job’s progress.
  • Get a job’s results.

Bulk API and Asynchronous Requests

Bulk API is based on REST principles and is optimized for working with large sets of data. You can use it to insert, update, upsert, or delete many records asynchronously, meaning that you submit a request and come back for the results later. Salesforce processes the request in the background.

In contrast, SOAP and REST API use synchronous requests and are optimized for real-time client applications that update a few records at a time. You can use both of these APIs for processing many records, but when the data sets contain hundreds of thousands of records, they’re less practical. Bulk API’s asynchronous framework is designed to make it simple and efficient to process data from a few thousand to millions of records.

The easiest way to use Bulk API is to enable it for processing records in Data Loader using CSV files. With Data Loader, you don’t have to write your own client app. Sometimes, though, unique requirements necessitate writing a custom app. Bulk API lets you take the ship’s wheel into your own hands and steer the course toward a solution that works for you.

For this unit, you use a newer version of Bulk API called Bulk API 2.0. If you want to apply the things you learn in this unit to the previous, still supported version of Bulk API, you need to use different resource URIs, and create and manage batches as well as jobs. For more information on the previous version of Bulk API see Bulk API Developer Guide.

Set Up Your Playground and Postman

To explore Bulk API, you’ll use Postman to create some account records.

  1. Log in to your Trailhead Playground.
  2. Log in to the Postman web app.
  3. Connect your Playground to Postman by getting a new token.
  4. Test that your connection is working with the REST GET Limits resource.

You learned how to do this in Quick Start: Connect Postman to Salesforce, so revisit that project if you aren’t sure about any of the steps.

Create a Bulk Job

The first step is to create a job in your fork of the Salesforce API Collection. A job specifies the type of operation and data object you’re working with. It functions as a bucket into which you add data for processing.

  1. In Collections, open the Bulk v2 folder.
  2. Click POST Create job.

Postman Collection window showing the Bulk v2 folder open and the POST Create Job resource selected.

Because Bulk API is REST-based, the request takes the familiar form of a REST request with four components: URI, HTTP method, headers, and body. The HTTP method is POST.

Notice the URI built in the main window when you clicked the resource in the collection: /services/data/v{{version}}/jobs/ingest. Let’s note a few things about this URI.

  • You’re using /services/data, which is the same endpoint used for the REST API. Bulk API uses the same framework that the REST API uses, which means that Bulk API supports many of the same features, such as OAuth authentication.
  • /jobs/ingest indicates that you’re accessing the resource for creating Bulk API jobs.

Postman request window showing the POST Create Job URI.

Let’s create the request body.

To create a Bulk API 2.0 job, use the lineEnding request field to specify the line ending used to create the CSV formatted text . Bulk API 2.0 supports two line-ending formats: linefeed (LF), and carriage-return plus linefeed (CRLF). The default lineEnding value, if not specified, is LF . Different operating systems use different characters to mark the end of a line:

  • Unix / Linux / OS X uses LF (line feed, '\n', 0x0A)
  • Windows / DOS uses CRLF (carriage return followed by line feed, '\r\n', 0x0D0A)

Note that text editors used to create a CSV file might be configured for a specific line-ending format that supersedes the default operating system format.

  1. Copy and paste the sample CSV body from your browser into a text editor to clear the formatting. Indicate the appropriate line ending with the lineEnding request field based on the OS and text editor you are using. In the example, we're using a Windows machine with "CRLF" as the value for the lineEnding parameter.
{
"operation" : "insert",
"object" : "Account",
"contentType" : "CSV",
"lineEnding" : "CRLF"
}

   

     2. Click Save.

3. Click Send and check out the response.

Postman response window showing the job id returned, the state of the job "Open" and additional details about the job.

The response includes all sorts of properties about the job, most of which aren’t useful to us right now because you haven’t added data yet. However, let’s take a look at a few things. 

  1. Look at the "id"row. This shows the returned job ID for this job.
    1. Now, in the Request section, click the Tests tab.
    2. This script automatically adds the job ID to the __jobID variable. This injects the jobID into future requests so you don’t need to copy and paste it.
  1. Next look at the "state"property.
    1. When you create a job, it’s immediately set to the Open state. That means it’s ready to start receiving data.
  1. Finally, look at the "contentUrl"property.
    1. This property shows the URL that you use to load data for the job.

Add Data to the Job

Now you can insert account data to our job. Data for a job is sent to the server as a set of records in a PUT request. The server processes the set of records, determining the optimal way to load the data into Salesforce. All you have to do is upload the data.

Create a new request in Postman. In your fork of the Salesforce APIs Collection, in the Bulk v2 folder, click PUT Upload Job Data. Note that the HTTP method is PUT.

For this example, you’re adding a set of records with only four accounts. Usually, you use Bulk API to add thousands or millions of records, but the principle is the same. You can upload a CSV file by selecting the binary radio button and uploading your .csv file, or you can paste in a list. In this example, we'll paste in a list.

  1. Click the Body tab and select Raw from the dropdown.
  2. Copy the following text into a text editor to clear any extra formatting, then copy it from the text file into the request body field.
"Name"
"Global Treasures & Mapping Company"
"Ahab’s Mighty Masts"
"Planks R Us"
"Cap’n Cook’s Kitchen Supplies"

Postman request window showing PUT Upload Job Data URI and, in the Body field the details of the data being uploaded.

3. Click Headers. Notice that Content Type says text/csv. That’s because you specified the content type in your first request.

Postman request window showing the Headers value field set to text/csv.

     4. Click Save.

5. Click Send.

The response contains just a status code of 201 Created, which indicates that Salesforce successfully received the job data.

Postman response window showing the 201 job status code indicating the successful upload of the job data.

Close the Job

Now that you’ve submitted your data, you need to let Salesforce know that it’s time to process the data. 

  1. In your Salesforce API fork, Bulk v2, Query folder click PATCH Close or Abort a Job.
  2. Click the Body tab and notice that "state": is already populated with “UploadComplete”.

Postman request window showing the PATCH Close or Abort a Job URI and the Body window showing a status of Upload Complete.

     3. Click the Headers tab and notice that the Content-Type is set to application/json.

Postman request window showing the Headers value field set to application/json.

4. Click Send.

Postman response window showing the job details and a status of Upload Complete.

The response contains job status information. The state property indicates that the job state is UploadComplete. At this point, Salesforce starts processing the job.

Check the Status of the Job

You’ve submitted your data and you’ve let Salesforce know you’re done uploading data. Now it’s up to the server to process the request. You can monitor the server’s progress by checking the status of the job through the Salesforce UI, or through the API. Let’s take a look at each method.

Here’s how to check the status of the job in your Trailhead playground. 

  1. From Setup, enter Bulk Data Load Jobs in the Quick Find box
  2. Select Bulk Data Load Jobs.

You can check the status of a job on this page. Or, you can click a job ID to check the status and get detailed results for that job.

Salesforce Setup window showing Bulk Data Load Jobs completed in the last seven days.

Here’s how you check the status of the job in Postman from the Bulk v2 folder.

  1. Select GET Job Info. Notice that the http method used for this type of request is GET.
  2. Click Send.

You see something like this.

Postman response window showing details of the job and the status of the job as Job Complete.

If your state is still UploadComplete instead of JobComplete, Salesforce is still processing the job. Don’t worry, it’ll be processed in a few minutes. In the meantime, go treat yourself to a mug of coconut milk and try the same request again when you get back. If you’re lucky and your job is already processed, continue on to retrieve the job results.

Get the Job Results

Once a job is in the JobComplete state (or Failed state) you can get results information in the form of successfully and unsuccessfully processed records.

Let’s look at the successfully processed records in the Bulk v2 folder.

  1. Click the GET Get Job Successful Record Results resource. Notice that the HTTP method is GET.
  2. Click Send. You see something like this.

Postman response window showing the names of the accounts returned.

Salesforce returns a list of all the records in the job that were successfully processed. In this module, you created several Account records. Line 1 shows the value types of the responses returned below it. The list data contains the record IDs of the records created, a value of true for the sf__Created columns, and the names of the accounts created. Great job! 

Occasionally some records can’t be processed. Maybe the job tried to create Account records that already existed. Maybe the job data was missing some required fields. In these scenarios, you can ask Salesforce for a list of records that had an error during processing, along with more information on what went wrong. Let’s take a look at failed records in the Bulk v2 folder.

  1. Select the GET Get Job Failed Record Results resource. Note that the HTTP method is, again, GET.
  2. Click Send. The results look something like this.

Postman response window showing an error because no jobs in our example failed.

Postman provides a list that contains a list of records that encountered errors during processing, along with the record ID and the error message. In this case, all your records were successfully inserted, so the list of records is empty. Tidy work, Captain!

Resources

Keep learning for
free!
Sign up for an account to continue.
What’s in it for you?
  • Get personalized recommendations for your career goals
  • Practice your skills with hands-on challenges and quizzes
  • Track and share your progress with employers
  • Connect to mentorship and career opportunities