Create Custom Job Steps

Learning Objectives

After completing this unit, you’ll be able to:

List the steps you need to take to create a custom job.
Describe the difference between a task-oriented and chunk-oriented script module.
Explain what the steptypes.json file is.

Introduction

Linda Rosenberg, the Cloud Kicks' administrator, is creating a job and can’t find a system step that does exactly what she wants. She wants to import data directly from an external product information (PIM) system into the production instance. She needs custom code to do this because she wants to import the catalog data, process it, and then save it in the Salesforce B2C Commerce database. It’s not a simple import. To create this custom job, she asks a Cloud Kicks developer, Vijay Lahiri, to do these steps.

Create a cartridge. A cartridge is a mechanism for packaging and deploying B2C Commerce storefront program code and data.
Write a task-oriented or chunk-oriented script to run the process steps, for example to read a product record, process it, and then write it to the database.
Create a steptypes.json file to describe the custom steps, and place the file in the root of the cartridge.
Upload the cartridge and include it on the cartridge path.

Vijay is happy to help.

Vijay Lahiri, Developer

When he's finished, Linda:

Replicates the code to production and activates the code version that includes the new cartridge.
Creates a job using the custom step in Business Manager.

Task-Oriented Script Module

This is going to get a little technical, but Linda needs to understand the following developer-focused concepts and processes so she can collaborate with Vijay, monitor job success, and plan for future requirements.

A task-oriented CommonJS script module exposes a function to be called as the main function for the job step. When Linda creates jobs using Business Manager, she sets parameters that are available as scriptable objects for the module's function and for the dw.job.JobStepExecution object. The dw.job.JobStepExecution object allows read-only access to information about the current step execution and job execution.

To control the exit status, the function can return a dw.system.Status API object. If the script finishes with an unhandled exception, the exit status code is ERROR and the error status flag is true by default. If no status object is returned and no exception occurs, the status code is OK by default.

See the Infocenter for an example of a task-oriented script module that connects via FTP and then downloads data. Vijay uses this example to create the script module.

Chunk-Oriented Script Module

A chunk-oriented CommonJS script module reads and processes items in chunks of a specified size. If the list contains more items than B2C Commerce can process in one chunk, it starts a new chunk. A chunk-oriented script can include any series of processing steps, not just database transactions.

A job step that uses a chunk-oriented script allows fine-grained progress monitoring because B2C Commerce updates the number of elements that are written each time a chunk is finished.

Vijay and Linda take a look at a chunk-processing example.

Assume you export a list of eight orders to a file, and the size of one chunk is four orders. The script processes the chunks in this sequence.

Read Order1, Process Order1, Read Order2, Process Order2, Read Order3, Process Order3, Read Order4, Process Order4
Write Order1, Write Order2, Write Order3, Write Order4
Read Order5, Process Order5, Read Order6, Process Order6, Read Order7, Process Order7, Read Order8, Process Order8
Write Order5, Write Order6, Write Order7, Write Order8

Example showing read, process, and write in chunks of 4 orders

Chunk Script Functions

Linda noticed three functions in the example. A chunk-oriented script module exposes functions for read, process, and write.

read-function: Returns one item or nothing.
process-function: Lets you transform items and apply business logic to them.
write-function: Receives a list of items.

You can also use these optional functions in a chunk-oriented script module.

total-count-function: Returns the total number of items that are available before chunk processing begins.
before-step-function: Executes before a chunk step begins.
before-chunk-function: Executes before a chunk begins.
after-chunk-function: Executes after a chunk finishes.
after-step-function: Executes after a chunk step finishes successfully.

See the Infocenter for an example of a chunk-oriented script module.

Steptypes File

When Vijay creates a custom step, he must create a steptypes.json file to describe the metadata of the step. This file specifies the script module that implements the step, the parameters the step requires, and the exit statuses the step returns. The steptypes.json file requires a very specific syntax. For example, a portion of steptypes.json file uses these name/value pairs:

"@name":"MyParameter1",
"@type":"boolean",
"@required":"true",
"description":"A required boolean parameter."

See the Infocenter for details about the syntax and an example of a steptypes.json file.

Troubleshooting Invalid steptypes.json Files

B2C Commerce parses and loads the steptypes.json file:

At server startup.
When the active code version is changed.
On sandboxes, each time a step is run.

If a steptypes.json file contains errors because of missing attributes or other problems, B2C Commerce logs the errors in an error file and does not register the custom step. B2C Commerce then loads steps from the steptypes.json files of other cartridges.

Invalid steptypes.json files cause messages like this to appear in Business Manager:

Invalid step [Step1]! Type with id [custom.MyCustomStep1] is unknown!

If Linda imports invalid jobs, she gets a warning in the import/export log and an error message in Business Manager. If Vijay changes steptypes.json so that it creates an invalid job, it no longer executes, and nothing happens except an entry in the job log.

Use steptypes.xml

Vijay can use an XML definition instead of JSON to define the custom steps. He must call the file steptypes.xml and place it in a custom cartridge in the root folder. B2C Commerce parses and reloads it whenever the active code version is changed. There can be only one steptypes.json or steptypes.xml file.

Next Steps

In this module, Linda learned how to create a custom job step with the help of Vijay, a Cloud Kicks developer. Next, she learns how to create and configure jobs.

¿Necesita ayuda?

Descubrir más