What is a batch job?

A batch job is a scheduled program that is assigned to run on a computer without further user interaction. Batch jobs are often queued up during working hours, then executed during the evening or weekend when the computer is idle.

Once the batch job is submitted, the job enters into a queue where it waits until the system is ready to process the job. If the job queue contains many jobs waiting to be processed, the system processes the jobs either in chronological order or by priority.

Batch jobs are frequently used to automate tasks that need to be performed on a regular basis, like payroll, but don’t necessarily need to occur during the day or have an employee interacted with the system. Jobs that happen on a regular basis are incorporated into batch schedules.

A job runs one or more instances of your executable code. Unlike applications, which handle HTTP requests, jobs are designed to run one time and exit. When you create a job, you can specify workload configuration information that is used each time that the job is run.

The most valuable benefits of batch jobs include:

Large programs can utilize more dedicated servers when the work processes are in night mode.
With fewer users or employees online during off hours, the performance will be faster and more efficient at night. During the day it could be restricted to fewer servers and numbers needed.
Employees are freed up to focus on less repeatable, more creative tasks.

The following guide will walk you through creating a Code Engine project and application using sample code, and then you will create batch jobs and run them. The following steps can also be followed to deploy an application to Code Engine from source code after following the setup instructions in Next Steps.

Happy Hacking!

Let's Build Something

Steps

Sign Up for a Free IBM Cloud Account
Install the IBM Cloud Developer Tools CLI (command line interface, the commands you type into your Terminal)
Install the IBM Cloud Code Engine Plugin
Login to Your IBM Cloud Account via the CLI
Create a Code Engine Application
Create a Job from a Public Repo
Run a Batch Job
NEXT STEPS! Build a Container Image from Source Code to Deploy to Code Engine and Run Batch Jobs

Setup & Installation

2. Install the IBM Cloud Developer Tools CLI

For Mac and Linux, run the following command in Terminal:

curl -sL https://raw.githubusercontent.com/IBM-Cloud/ibm-cloud-developer-tools/master/linux-installer/idt-installer | bash

Verify the IBM Cloud Developer Tools CLI is installed

ibmcloud dev help

For Windows 10 Pro, run the following command as an administrator in Powershell:

[Net.ServicePointManager]::SecurityProtocol = "Tls12, Tls11, Tls, Ssl3"; iex(New-Object Net.WebClient).DownloadString('https://raw.githubusercontent.com/IBM-Cloud/ibm-cloud-developer-tools/master/windows-installer/idt-win-installer.ps1')

Note For Windows Users: If you encounter a Git error similar to the one below, you will need to install Git in the correct path.

bash: git: command not found

Follow the Windows Guide HERE

3. Install the IBM Cloud Code Engine Plugin

For Mac, Linux, and Windows 10 Pro, run the following command:

ibmcloud plugin install code-engine

Verify the IBM Cloud Code Engine Plugin is installed

ibmcloud ce help

4. Login to Your IBM Cloud Account via the CLI

For Mac, Linux, and Windows 10 Pro, run the following command:

ibmcloud login

Enter email and password
View available resource groups

ibmcloud resource groups

Assign a target resource group (default to your "Default")

ibmcloud target -g Default

Deploy to Code Engine & Run Batch Jobs

5. Create a Code Engine Application

Create a new Code Engine project and give it a name

ibmcloud ce project create --name PROJECT_NAME

ibmcloud ce project create --name sandbox

Create a new app from a sample Container Image

ibmcloud ce application create --name APP_NAME --image IMAGE

ibmcloud ce application create --name myapp --image docker.io/ibmcom/hello

Check the application status

ibmcloud ce application get -n APP_NAME

ibmcloud ce application get -n myapp

Get the live application URL

ibmcloud ce application get -n APP_NAME -output url

ibmcloud ce application get -n myapp -output url

View the live application at the URL in your browser

6. Create a Job from a Public Repo

Create a job configuration that is named myjob and uses the container image docker.io/ibmcom/firstjob

ibmcloud ce job create --name JOB_NAME  --image IMAGE

ibmcloud ce job create --name myjob  --image ibmcom/firstjob

Note: the format for creating a new job from an image is ibmcom/firstjob from the image docker.io/ibmcom/firstjob.

7. Run a Batch Job

Run a job

ibmcloud ce jobrun submit --name testjobrun --job myjob --array-indices "1 - 5"

Note: the following jobrun submit command creates five new instances to run the container image that is specified in the myjob job. the resource limits and requests are applied per instance, so each instance gets 4 G memory and 1 vCPU. This job allocates 5 * 4 G = 20 G memory and 5 * 1 vCPU = 5 vCPUs.

Resubmit a job run based the configuration of a previous job run, use the jobrun resubmit command

ibmcloud ce jobrun resubmit --jobrun testjobrun

Access the Job Details, including the status of your instances, configuration details, and environment variables of your job

ibmcloud ce job get --name myjob

Options for Creating and Running Jobs

CONGRATULATIONS!

If you've made it this far, then Kudos to you! The example commands given can be followed to create an application from a container image and deploy it to Code Engine with your own images. You can also follow the example commands given to create and run batch jobs with your own public images and repositories. For more information to get started with your own container images and/or source code, check out the Tutorial Tuesday Code Engine Guide along with the Code Engine Documentation.