Implementing batch queue on Linux

Many older operating systems had a batch (or job) queue, which allowed running a certain number of simultaneous jobs in backgroud. The purpose of this queue was to fully utilize the computing resources when they are not being used. Launched tasks were waiting for the resources to be available and only then running.

Today, this function is less in demand when most systems are interactive and assume that they will respond within a reasonable time. However, even today there still room for such use. Usually the need is resolved by scheduling jobs through crontab or similar, but this solution does not take into account the number of parallel jobs that can cause the system be overloaded.

Another reason for the reincarnation of this function may be a license restriction. I've seen software that allows you to run only X threads in parallel in accordance with the purchased license.

There are number of queue implementation exist for Linux, but one exist from the beginning - printing queue. In this memo we will implement batch queue via cups functionality on RedHat 6 example.

As bonus for simple implementation, you can get job management via lp, lpq, lpadmin and cups* commands.

Writing batch CUPS backend

CUPS backends are executables that really writes data to printer. It is obvious that in our case the printer will be /bin/bash that "printed" script runs on it.

The backend should return a list of attached printers when called without arguments. It also should be able print STDIN or supplied file to printer.

Put the following file as /usr/lib/cups/backend/batch and make it executable:

#!/bin/bash
# /usr/lib/cups/backend/batch
# The backend should return connected printers if called without arguments:
[ $# -lt 1 ] && { echo "direct batch \"RAW0\" \"Batch BASH\"" ; exit 0 ; }

# Prepare batch file:
# $6 is a printing source or STDIN:
INPUTFILE="-"
test -n "$6" && INPUTFILE=$6
BATCHFILE=/var/tmp/batch.$$.sh
cat $INPUTFILE > $BATCHFILE
chmod 555 $BATCHFILE

# $2 is a user submitted print. Run script as this user. STDERR will help debug problems.
if /usr/bin/sudo -u $2 $BATCHFILE ; then
        echo 'INFO:' 1>&2
else
        echo "ERROR: $0 : sudo or script errors" 1>&2
fi

[ -f $BATCHFILE ] && rm -f $BATCHFILE

# Exit HAVE to be 0, otherwise FAULTY SCRIPT WILL RETRY TO RUN.
exit 0

Once backend script installed, you can see it by command lpinfo -l -v

A script will run as user lp if you will not take care about. As you can see from script body, a $2 is the name of user that submit the job (print) itslef. It is better from security point of view, that submitted script will run as this user. Thus, the backend file will create job file as /var/tmp/batch.$$.sh. You can decide on your own convention. Then add the following permissions to sudo:

lp      ALL=(ALL)       NOPASSWD: /var/tmp/batch*

Security hole here, but you need own "lp" first.

Selinux will block run sudo by "lp" user, you can tune that or disable selinux.

Creating printers and queue

Be sure you have installed CUPS, enabled it and have it running:

# /etc/init.d/cups restart
# chkconfig cups on

CUPS, when taking care of printing, passes the printed file to printer's driver (filter in CUPS terms), that translates printed file format to printer's language (PCL, for example). We do not need translate our jobfiles to no language, our driver have to be RAW.

Our example job queue will run 5 simultanious jobs for some reason (probably we have 4 CPU, may be license allow to run only 5 threads and so on)

Create 5 "printers" named RAW0x and enable it in one command:

# for p in RAW0{1..5} ; do lpadmin -p $p -v "batch:" -m raw -E ; done
# lpstat -a
RAW01 accepting requests since Tue Apr 18 16:58:06 2017
RAW02 accepting requests since Tue Apr 18 16:58:06 2017
RAW03 accepting requests since Tue Apr 18 16:58:06 2017
RAW04 accepting requests since Tue Apr 18 16:58:06 2017
RAW05 accepting requests since Tue Apr 18 16:58:06 2017

Now, group them to one queue (it is "class" in term of CUPS):

# for p in RAW0{1..5} ; do lpadmin -p $p -c BATCH5 ; done
# cupsaccept BATCH5 ; cupsenable BATCH5
# lpstat -a
BATCH5 accepting requests since Tue Apr 18 17:05:43 2017
RAW01 accepting requests since Tue Apr 18 16:58:06 2017
RAW02 accepting requests since Tue Apr 18 16:58:06 2017
RAW03 accepting requests since Tue Apr 18 16:58:06 2017
RAW04 accepting requests since Tue Apr 18 16:58:06 2017
RAW05 accepting requests since Tue Apr 18 16:58:06 2017

You can remove printer by:

# lpadmin -x NAME

Submitting job

Here is an example of job file test-job-file:

#!/bin/bash
#
# Take care about output, otherwise it will go to CUPS log files
exec >/tmp/myout.$$.log 2>&1

# sudo usually sanitize the environment, set it:
source /etc/profile

whoami
hostname

Submit it:

$ lp -d BATCH5 test-job-file
request id is BATCH5-7 (1 file(s))

Troubleshooting

Enable debug messages for CUPS (do not forget turn them off later):

# grep LogLevel /etc/cups/cupsd.conf
#LogLevel warn
LogLevel debug
# /etc/init.d/cups restart
# tail -f /var/log/cups/*

If you have selinux enabled, sudo command for user lp will not work even defined in /etc/sudoers. You should fix selinux policies.