LMI Job Manager
===============

Is a glib-based library managing jobs created as a consequence of invoking
asynchronous method of c provider. It takes care of their scheduling depending
on priority and their time of creation, their removal after being completed,
their persistency by (de)serialization to file and finally generating
indications for them.

### Job States
Represents a phase of life of job. Job starts as a `NEW`, passes `RUNNING`
phase and ends up in one of final states. The complete list of states follows:

 * `NEW`
 * `RUNNING`
 * `COMPLETED*`
 * `TERMINATED*`
 * `EXCEPTION*`

Symbol `*` denotes a final state.

Job is created with a call to `jobmgr_new_job()`. `NEW` is its initial state.
It means that it has not been started yet. Once it is created, several important
attributes need to be set. `method-name` property and input parameters at
minimum. Once it's done, `jobmgr_run_job()` needs to be called to put the job
in a priority queue where jobs wait for being processed. `LMI_InstCreation`
indication instance is generated when job is created. Possible transitions from
this state are: `RUNNING`, `TERMINATED`.

`RUNNING` is a next state of a job. It's set by job manager when the job is
picked up from priority queue of pending jobs. Job processing callback provided
by a developer is invoked to process it. The callback runs in its own thread and
may be cancellable. It shall update the job directly using `lmi_job_set_*`
family of functions. During this state many indications of
`LMI_InstModification` will be generated by job manager which listens for job's
changes. Next possible transitions are: `COMPLETED`, `TERMINATED` and
`EXCEPTION`.

`COMPLETED` is a final state representing successful completion of job. This
transition is done with a call to `lmi_job_finish_ok()` in processing callback.
This generates several indications of `LMI_InstModification` class. Job stays
in job manager until it is deleted.

`TERMINATED` is a final state representing cancelled job. Job may be cancelled
with a call to
`CIM_ConcreteJob.RequestStateChange(RequestedState="Terminate")`. This is
always successful for jobs waiting in a queue of pending jobs (the job is
`NEW`). For `RUNNING` jobs it depends on processing callback whether it allows
to cancel them or not. Job in any final state can't be terminated. Job stays
in job manager until it is deleted.

`EXCEPTION` is a final state representing failed job. It is usually set by
processing callback with a meaningful error message. Job stays in job manager
until it is deleted.

### Job Scheduling
Applies to `NEW` jobs if `concurrent_processing` is disabled. Jobs wait in
priority queue sorted by `(job_priority, job_number)`. The lower the numbers,
the higher the priority, the closer to queue's exit. Priority property of job
can be changed dynamically by user with `ModifyInstance()` method invoked on
corresponding `LMI_ConcreteJob` instance. Its default value is `128`.

#### Job's Removal
Once the job arrives to any final state, it may be deleted. Deletion happens
automatically after certain amount of time unless user decides to keep it. This
is controlled with `CIM_Job.DeleteOnCompletion`. If this property is `True`, job
will be deleted after `CIM_ConcreteJob.TimeBeforeRemoval` seconds after the
job's completion. Both properties may be modified any time with
`ModifyInstance()` method. User may also delete the job manually with
`DeleteInstance()` method.

### Job's Persistency
Unless disabled, each job is serialized to its own file in job persistent
storage whenever it is modified. This storage is usually located in
`/var/lib/openlmi-providers/jobs/${profile_name}/`. If enabled, jobs will
survive provider's cleanup and even broker's restart. When a job is deleted,
its file will be removed as well. Note that when all jobs are deleted and
broker is restarted, the job numbering will start from 1 again.

### Indication sending
There are sever static indication filters prepared for various events that may
happen to job during its life. The complete list with corresponding *CQL*
queries follows:

 * Changed

        "SELECT * FROM LMI_${profile_name}InstModification WHERE
            SourceInstance ISA ${job_class_name} AND
            SourceInstance.CIM_ConcreteJob::JobState <>
            PreviousInstance.CIM_ConcreteJob::JobState"

 * Created

        "SELECT * FROM LMI_${profile_name}InstCreation WHERE
            SourceInstance ISA ${job_class_name}"

 * Deleted

        "SELECT * FROM LMI_${profile_name}InstDeletion WHERE
            SourceInstance ISA ${job_class_name}"

 * Failed

        "SELECT * FROM LMI_${profile_name}InstModification WHERE
            SourceInstance ISA ${job_class_name} AND
            SourceInstance.CIM_ConcreteJob::JobState = 10"

 * PercentUpdated

        "SELECT * FROM LMI_${profile_name}InstModification WHERE
            SourceInstance ISA ${job_class_name} AND
            SourceInstance.CIM_ConcreteJob::PercentComplete <>
            PreviousInstance.CIM_ConcreteJob::PercentComplete"

 * Succeeded

        "SELECT * FROM LMI_${profile_name}InstModification WHERE
            SourceInstance ISA ${job_class_name} AND
            SourceInstance.CIM_ConcreteJob::JobState = 7"

Only these filters are supported, user can't subscribe to his custom ones.

Use `Changed` filter id to subscribe for any job state change.

Subscribe with `PercentUpdated` filter id to be notified whenever job's
`percent-complete` property  gets updated.

The other filter ids are self-explanatory.

Using Job Manager
-----------------
Now let's focus on implementing C provider with a help of job manager. This
document assumes you use [KonretCMPI](http://konkretcmpi.org/) to generate
headers and skeleton C modules for your providers.

### Overview
Job management can be utilized in a profile having at least one service with a
method that allows for asynchronous invocation. Such a method looks typically
like this:

    [  ValueMap { "0", "1", "2", "3", "4", "5", "6", "..", "4096",
          "4097", "4098..32767", "32768..65535" },
       Values { "Job Completed with No Error", "Not Supported",
          "Unspecified Error", "Timeout", "Failed",
          "Invalid Parameter", "Target In Use", "DMTF Reserved",
          "Method Parameters Checked - Job Started",
          "Asynchronous Job already in progress", "Method Reserved",
          "Vendor Specific" }]
    uint32 MyAsyncMethod(
        /* input parameters */
          [IN, Description("Foo rules them all.")]
        uint16 Foo,
          [IN]
        string Bar,
        /* output parameters */
          [IN (false), OUT, Description (
           "Reference to the job (may be null if job completed).")]
        CIM_ConcreteJob REF Job, /* this is required */
          [OUT, IN (false)]
        uint16 Baz[]
    );

When invoked it checks its input arguments and returns immediately with `4096`
code. It means that a job was created as a side effect and will be processed in
a background thread. Reference to it is passed in output parameter `Job`. Other
output parameters are not set because they won't be known until job is
completed. User can now subscribe to indications for this job to monitor its
state (see above for complete list). Or he may choose simple polling.

There are several CIM classes that need to be implemented in order for
indications and job introspection to work.

#### CIM classes related to job management
Following classes are related to job management and are mandatory for
implementation:

 * `LMI_${profile_name}Job`
 * `LMI_${profile_name}MethodResult`
 * `LMI_Associated${profile_name}JobMethodResult`
 * `LMI_Affected${profile_name}JobElement`
 * `LMI_Owning${profile_name}JobElement`

##### indications

 * `LMI_${profile_name}InstCreation`
 * `LMI_${profile_name}InstDeletion`
 * `LMI_${profile_name}InstModification`

##### special classes

 * `__MethodParameters_${method_name}`
 * `__MethodParameters_${method_name}_Result`

It's important to stick to naming scheme presented. For example if you
implement software profile for package management with
`InstallFromSoftwareIdentity()` asynchronous method, your CIM classes will be
named:

 * `LMI_SoftwareJob`
 * ...
 * `__MethodParameters_InstallFromSoftwareIdentity_Result`

Last two CIM classes beginning with `__MethodParameters` do not have any
instrumenting provider (not even registration file). They serve as a container
of method parameter values in properties like
`CIM_ConcreteJob.JobInParameters`, `CIM_ConcreteJob.JobOutParameters` and
`CIM_InstMethodCall.MethodParameters`. They need to be provided as a workaround
of Pegasus' shortcoming: instances of unknown classes can't be assigned to
instance properties.

Following example illustrates definition of such classes for asynchronous
method shown above:

    [Association]
    class __MethodParameters_MyAsyncMethod
    {
          [Description("Foo rules them all.")]
        uint16 Foo;
        string Bar;
    };

    [Association]
    class __MethodParameters_MyAsyncMethod_Result :
            __MethodParameters_MyAsyncMethod
    {
        uint16 Baz[];
        uint32 __ReturnValue;
    };

The former contains input parameters as it is used to fill
`CIM_ConcreteJob.JobInParameters`. The latter shall inherit from the former and
add any output parameters except for `Job`. All but `IN`/`OUT` qualifiers shall
be preserved.

`__ReturnValue` is a return value of asynchronous method that would be returned
if invoked as a synchronous one.

Store these classes into separate mof file e.g.
`60_LMI_${profile_name}_MethodParameters.mof` because they can be registered
with pegasus only - unlike regular mof files. Once the Pegasus properly support
instances of unknown classes, these can be safely removed.

#### LmiJob
Is a `GObject` class representing a job. Job manager operates upon objects of
this class and its descendants. Each such class needs to be registered with job
manager (see below). It has quite a few properties accessible with standard
glib's getters and setters. Convenience accessing functions shall be preferred
though. When any property is changed, signal `modified()` will be emitted to
anybody subscribed. Signals defined by `LmiJob` are used by job manager for
monitoring states of jobs. They do not need to be used in a provider.

### Implementation
Although implementing above CIM classes is mandatory to conform with Job
Control profile, it's not necessary to implement everything before testing can
be done. Incremental development may be used with a success:

 1. Define and register all above mentioned classes (you may skip
    `__MethodParameters*` classes for now).
 2. Implement job proof-of-concept processing callback that calls
    `lmi_job_finish_ok()` on given job at its end.
 3. Add job manager initialization and cleanup functions to your provider's
    init and cleanup (see below).
 4. In your asynchronous method, create new job with `jobmgr_new_job()`, set its
    method name (`lmi_job_set_method_name()`), set its input parameters
    (`lmi_job_set_in_param()`) and schedule it for processing
    (`jobmgr_run_job()`). Finally return `Method Parameters Checked - Job Started
    (4096)` as a method result.
 5. Implement `LMI_${profile_name}Job` provider. With KonkretCMPI you get
    Provider module for free. Just `EnumInstances()` needs to be implemented to
    get started (see below).
 6. Now it's possible to test `GetInstance()` and `EnumInstances()` on
    `LMI_${profile_name}Job` after the job is spawned by invoking particular
    asynchronous method.
 7. Add other missing functionality to `LMI_${profile_name}JobProvider.c` like
    `DeleteInstance()`, `ModifyInstance()`, `RequestStateChange()`.
 8. Implement `LMI_${profile_name}MethodResult` provider and
    `LMI_Associated${profile_name}JobMethodResult`.
 9. Define and register `__MethodParameters*` classes for your asynchronous
    method if you haven't already done so.
 10. Implement `make_job_params_func()` callback that fills properties of
     instances of `__MethodParameters_${method_name}*` classes and pass it to
     `jobmgr_init()`.
 11. Now it's possible to obtain the result of a completed job from
     `CIM_ConcreteJob.JobOutParameters.__ReturnValue`. Note that it's not yet
     possible to subscribe to indications.
 12. Implement `LMI_Affected${profile_name}JobElement` provider.
 13. Implement `LMI_${profile_name}InstModification` class.
 14. Now it's possible to subscribe to indications for (not)successful
     completion of job.
 15. `lmishell` now needs to be told how to recognize your new CIM job class.
     Add your class name to `LMIMethod._INDICATION_JOB_CLASSNAMES` and send a
     patch to openlmi-tools upstream. With that it's possible to use lmishell
     to call your method synchronously e.g: `rval, oparms, _ =
     my_tiny_service.SyncMyAsyncMethod(Foo=10, Bar="/tmp/bar")`. Note the
     `Sync` prefix before our asynchronous method. This invocation will listen
     for incoming indication implemented step 13.
 16. Implement `LMI_${profile_name}InstCreation` and
     `LMI_${profile_name}InstDeletion` in similar fashion.

Subsections below describe several above points in detail.

#### Subclassing LmiJob
*Note* that **subclassing** here means deriving from `GObject` or one of its
children defined by `GObject` library that provides object-oriented C-based
API.

It does not need to be done. You may safely use `LmiJob` directly. Here are
few cases where you might find this necessary:

* Your profile has more than one service and thus you need more CIM job classes
  to represent them.
* Your service has more than one asynchronous method used for unrelated
  purposes and thus their executions need to be represented by different CIM
  job classes.
* Your service has more than one asynchronous method and you need to pass
  register job types with different flags to job manager. For example job
  spawned with the first method is cancellable while the other is not.

##### Example
Subclassing is trivial and is well described in GObject library reference. Let's
take for example GObject representing `LMI_SoftwareInstallationJob`. Let's name
the subclass `LmiSwInstallationJob`. Then it's header file will have
following contents:

    #include "lmi_job.h"

    #define LMI_TYPE_SW_INSTALLATION_JOB \
        (lmi_sw_installation_job_get_type ())
    #define LMI_SW_INSTALLATION_JOB(obj) \
        (G_TYPE_CHECK_INSTANCE_CAST ((obj), LMI_TYPE_SW_INSTALLATION_JOB, \
                                    LmiSwInstallationJob))
    #define LMI_IS_SW_INSTALLATION_JOB(obj) \
        (G_TYPE_CHECK_INSTANCE_TYPE ((obj), LMI_TYPE_SW_INSTALLATION_JOB))
    #define LMI_SW_INSTALLATION_JOB_CLASS(klass) \
        (G_TYPE_CHECK_CLASS_CAST ((klass), LMI_TYPE_SW_INSTALLATION_JOB, \
                                    LmiSwInstallationJobClass))
    #define LMI_IS_SW_INSTALLATION_JOB_CLASS(klass) \
        (G_TYPE_CHECK_CLASS_TYPE ((klass), LMI_TYPE_SW_INSTALLATION_JOB))
    #define LMI_SW_INSTALLATION_JOB_GET_CLASS(obj) \
        (G_TYPE_INSTANCE_GET_CLASS ((obj), LMI_TYPE_SW_INSTALLATION_JOB, \
                                    LmiSwInstallationJobClass))

    typedef struct _LmiSwInstallationJob        LmiSwInstallationJob;
    typedef struct _LmiSwInstallationJobClass   LmiSwInstallationJobClass;

    struct _LmiSwInstallationJob {
        LmiJob parent;
    };

    struct _LmiSwInstallationJobClass {
        LmiJobClass parent_class;
    };

    GType lmi_sw_installation_job_get_type();

    LmiSwInstallationJob *lmi_sw_installation_job_new();

The C module on the other hand:

    G_DEFINE_TYPE(LmiSwInstallationJob, lmi_sw_installation_job, LMI_TYPE_JOB)

    static void lmi_sw_installation_job_class_init(LmiSwInstallationJobClass *klass)
    {
        G_OBJECT_CLASS(klass);
    }

    static void lmi_sw_installation_job_init(LmiSwInstallationJob *self) {}

That's it. Other descendants will have just another name. Then they just need
to be registered with job manager separately. Each with a different CIM class
name assigned.

#### Initialization and cleanup
Initialization comprises of:

 1. job registration
 2. job manager initialization

These can also be done with a single call.

It's recommended to create one initialization function and call it from
initialization function of each provider of your profile. If you don't need job
persistency, you may safely put this initialization and cleanup code only to
provider modules of job-related CIM classes mentioned above.

##### Job registration
Must be done before job manager is initialized. It's done like this:

    jobmgr_register_job_type(
        LmiJob,         // if you don't subclass
        "LMI_MyJob",    // CIM class name of LMI_ConcreteJob descendant
        /* A callback that should set additional CIM instance's
         * properties. Invoked right before an indication is sent. */
        cim_convert_func,
        /* A callback that should fill input and output
         * parameters of job's asynchronous method to prepaired
         * method. May be `NULL`. */
        make_job_params_func,
        /* Type of value that can be set to result property of
         * job. This value is used in CIM_InstMethodCall.ReturnValueType
         * property of associated LMI_MethodResult instance. */
        METHOD_RESULT_VALUE_TYPE_UINT32,
        /* A callback that should execute given job and finally
         * set it to final state. */
        process_func,
        /* Does a job support cancellation? If yes,
         * *process_func* will be passed *cancellable* object that
         * needs to be checked during job's processing for
         * cancelled state. It enables the use of
         * `jobmgr_terminate_job()` function. */
        TRUE,
        /* Whether to write job's state to file under
         * persistent storage directory whenever it gets modified
         * or created. Persistent storage is read upon job manager's
         * initialization to populate internal structures with
         * saved jobs. */
        TRUE);
    /* repeat for any other job type */
    /* and continue with job manager initialization */

##### Job manager initialization

    status = jobmgr_init(cb, ctx,
                    "My",     /* profile_name */
                    provider_name,  /* e.g. LMI_MyTinyService */
                    /* Whether this provider implements CIM
                     * indication class. Such classes derive from
                     * `CIM_InstIndication` and typically
                     * `InstIndication` is a suffix of their names. */
                    is_indication_provider,
                    /* Says whether multiple jobs can be run in
                     * parallel. If `false`, it will be ensured that
                     * at most one job can be in *RUNNING* state.
                     * The others will stay *NEW* until currently
                     * running job completes. */
                    FALSE,
                    /* Optional job type information. If NULL or empty,
                     *it's expected that job types were already
                     * registered with `jobmgr_register_job_type()`. */
                    NULL,
                    /* Number of entries in `job_type_entries` array. */
                    0);
    if (status.rc) {
        lmi_error("Failed to initialize provider %s: %s",
                provider_name, status.msg ?
                     CMGetCharsPtr(status.msg, NULL) : "unknown error");
    } else {
        lmi_debug("Software provider %s initialized.", provider_name);
    }

Or you may join above two steps into one:

    /* This approach is especially useful for multiple job types. */
    static JobTypeEntry _my_job_type_entries[] = \
        { { 0 // needs to be set later with LMI_TYPE_SW_INSTALLATION_JOB
          , LMI_MyJob_ClassName
          , cim_convert_func
          , make_job_params_fun
          , METHOD_RESULT_VALUE_TYPE_UINT32
          , process_func
          , TRUE, TRUE }
    };
    #define JOB_TYPE_ENTRIES_COUNT 1

    void my_provider_init(const gchar *provider_name,
                   const CMPIBroker *cb,
                   const CMPIContext *ctx,
                   gboolean is_indication_provider,
                   const ConfigEntry *provider_config_defaults)
    {
        CMPIStatus status;
        lmi_init(profile_name, cb, ctx, provider_config_defaults);
        /* LMI_TYPE_MY_JOB is unknown during compilation */
        _sw_job_type_entries[0].job_type = LMI_TYPE_MY_JOB;
        status = jobmgr_init(cb, ctx,
                        "My",     /* profile_name */
                        provider_name,
                        is_indication_provider,
                        FALSE,    /* concurrent_processing */
                        _my_job_type_entries, JOB_TYPE_ENTRIES_COUNT);
        /* log errors */
    }

##### Cleanup function
Is trivial:

    CMPIStatus my_provider_cleanup(const gchar *provider_name)
    {
        lmi_debug("My provider %s cleanup started.", provider_name);
        CMPIStatus status = {CMPI_RC_OK, NULL};
        status = jobmgr_cleanup(provider_name);
        lmi_debug("My provider %s cleanup finished.", provider_name);
        return status;
    }

#### Implementing providers
Because it is planned to provide a generator for provider modules, it won't be
documented. Please refer to [upstream
project](https://fedorahosted.org/openlmi/ "OpenLMI wiki") and clone the
`openlmi-provider` git repository. In there look for:

    src/software-dbus/lmi_sw_job.c
    src/software-dbus/LMI_SoftwareInstallationJobProvider.c
    src/software-dbus/LMI_SoftwareVerificationProvider.c
    src/software-dbus/LMI_AffectedSoftwareJobElementProvider.c
    src/software-dbus/LMI_SoftwareMethodResultProvider.c
    src/software-dbus/LMI_AssociatedSoftwareJobMethodResultProvider.c
    src/software-dubs/LMI_SoftwareInstCreationProvider.c
    src/software-dubs/LMI_SoftwareInstDeletionProvider.c
    src/software-dubs/LMI_SoftwareInstModificationProvider.c
