WorkQueue mechanism in Linux

1.Work Queue

The Linux Workqueue mechanism is to simplify the creation of kernel threads. By calling workqueue interface you can create kernel threads. And it may be based on the current number of system CPU to create the number of threads so that the transaction can be parallelized thread. work queue is the kernel simple and effective mechanism, he apparently simplifies the creation of the kernel daemon facilitate the programming of the user.

Work Queue (work queue) is in the form of work after another push executed after the work queue can push the work, referred to a kernel thread to perform, that is, the lower half can be executed in the process context. The most important thing is the work queue is allowed to reschedule or even sleep.


2.Data Structure
We pushed back the implementation of the task called work , describing its data structure work_struct,

struct work_struct {
    atomic_long_t data;      / * Work handler func parameter * /
#define WORK_STRUCT_PENDING 0        /* T if work item pending execution */
#define WORK_STRUCT_STATIC 1        /* static initializer (debugobjects) */
#define WORK_STRUCT_FLAG_MASK (3UL)
#define WORK_STRUCT_WQ_DATA_MASK (~WORK_STRUCT_FLAG_MASK)
    struct list_head entry;        /*Connection work pointer*/
    work_func_t func;              /*work handler*/
#ifdef CONFIG_LOCKDEP
    struct lockdep_map lockdep_map;
#endif
};
The work queue structure to organize a work queue (workqueue), its data structure

struct workqueue_struct {
 struct cpu_workqueue_struct *cpu_wq;
 struct list_head list;
 const char *name;   /*workqueue name*/
 int singlethread;   / * Is not a single thread - threaded our preferred first CPU -0 indicates the default worker thread event*/
 int freezeable;  /* Freeze threads during suspend */
 int rt;
}; 
If it is multi-threaded, Linux CPU number of the current system to create its structure is based on cpu_workqueue_struct,
struct cpu_workqueue_struct {
 spinlock_t lock;
 struct list_head worklist;
 wait_queue_head_t more_work;
 struct work_struct *current_work; 
 struct workqueue_struct *wq;  
 struct task_struct *thread; 
} ____cacheline_aligned;

In the structure of the main maintains a job queue, and kernel threads need to sleep waiting queue, and also maintains a task context, the task_struct.

The relationship between the three is as follows:

Creating work
3.1 Creating job queue
a. create_singlethread_workqueue (name)
Implementation mechanism of the function shown below, the function returns a pointer to a variable of type struct workqueue_struct of the pointer variable points to the memory address inside the function call kzalloc dynamically generated. So the driver call void destroy_workqueue (struct workqueue_struct * wq) In the case of the work queue is no longer used to release the memory address here.

Cwq figure is a per-CPU type of address space. For create_singlethread_workqueue terms, even for multi-CPU system, the kernel is responsible for creating a worker_thread only kernel process. After the kernel process is created, it will first define a graph in wait node, and then check cwq worklist in a body of the loop, if the queue is empty, then the node will be added to the wait cwq the more_work then Sleep in the waiting queue.

Driver calls queue_work (struct workqueue_struct * wq, struct work_struct * work) was added to the wq work node. work will in turn increase the cwq-> worklist points list. queue_work to cwq-> worklist added a work node, while calls to awaken dormant wake_up on cwq-> more_work of worker_thread process. wake_up will first wait on the call autoremove_wake_function function node, then the node is removed from the wait cwq-> more_work in.

worker_thread again is scheduled to begin processing cwq-> worklist for all the nodes work ... when all work is completed node processing, worker_thread again will wait node to cwq-> more_work, then dormant in the waiting queue until Driver call queue_work again ...

b. create_workqueue

With respect create_singlethread_workqueue, create_workqueue wq will also be assigned a job queue, but differs in that for the purposes of multi-CPU system, for each CPU, cwq structure will be created for a per-CPU, which corresponds every cwq, will generate a new worker_thread process. However, when a node to submit work on cwq with queue_work, which CPU calling the function, then the CPU Bianxiang worklist cwq corresponding increase in work on the node.

c. Summary
When the user invokes workqueue initialization interfaces create_workqueue or create_singlethread_workqueue queue for workqueue initialized, the kernel starts the user is assigned a workqueue object and its chain to a global workqueue queue. Then Linux CPU based on the current situation, in order to workqueue object is assigned the same number of CPU cpu_workqueue_struct objects, each cpu_workqueue_struct objects will exist a task queue. Then, Linux cpu_workqueue_struct for each object is assigned a kernel thread, namely the kernel daemon to process each queue tasks. At this point, the user invokes the initialization interface workqueue initialization is complete, return workqueue pointer. After Workqueue initialization is complete, the task run context to build up, but not yet perform specific tasks, therefore, need to define specific work_struct object. Then work_struct added to the job queue, Linux daemon wakes up to handle the task.

 Workqueue kernel implementation of the principles described above can be described as follows:

3.2 Creating work
To use the job queue, the first thing to do is to create some post-needed push to complete the work. The structure can be built statically compiled by DECLARE_WORK:
DECLARE_WORK (name, void (* func) (void *), void * data);
This will create a file called statically name, pending function func, parameter data of work_struct structure.
Similarly, you can create a pointer at runtime by a work:
INIT_WORK (structwork_struct * work, woid (* func) (void *), void * data);


4. Scheduling
a. schedule_work

In most cases, you do not need to create their own job queue, but only defines the work, the work attached to the core structure predefined event scheduler work queue defines a static global amount in kernel / workqueue.c in Work Queue static struct workqueue_struct * keventd_wq; default worker thread is called events / n, where n is the number of processors, each processor corresponds to a thread. For example, single-processor systems only events / 0 of such a thread. The dual-processor system will be more of an events / 1 thread.
Scheduling work structure, work structure will add to the overall work of the event queue keventd_wq, called queue_work common module. Foreign shielded keventd_wq interface, users do not know this parameter is equivalent to using the default parameters. keventd_wq by the kernel to maintain their own, create, destroy. Such work will soon be scheduled once the worker thread in the processor on which it is awakened, it will be executed.

. b schedule_delayed_work (& work, delay);
Sometimes I do not want to work will be executed immediately, but want it to execute again after some delay. In this case, and you can also use the timer to delay the scheduling, after the expiration of a default timer callback registration work. After delay delay, by the timer wake up, will add to the work queue wq work in.

Work is not the priority queue, essentially as a FIFO manner.

Example

#include <linux/module.h>
#include <linux/init.h>
#include <linux/workqueue.h>

static struct workqueue_struct *queue=NULL;
static struct work_struct   work;

staticvoid work_handler(struct work_struct *data)
{
       printk(KERN_ALERT"work handler function.\n");
}

static int __init test_init(void)
{
      queue=create_singlethread_workqueue("hello world");
      if (!queue)
            goto err;

       INIT_WORK(&work,work_handler);
       schedule_work(&work);

      return0;
err:
      return-1;
}

static   void __exit test_exit(void)
{
       destroy_workqueue(queue);
}
MODULE_LICENSE("GPL");
module_init(test_init);
module_exit(test_exit);

The longstanding task queue interface was removed in 2.5.41; in its place is a new "workqueue" mechanism. Workqueues are very similar to task queues, but there are some important differences. Among other things, each workqueue has one or more dedicated worker threads (one per CPU, by default) associated with it. So all tasks running out of workqueues have a process context, and can thus sleep. Note that access to user space is not possible from code running out of a workqueue; there simply is no user space to access. Drivers can create their own work queues - with their own worker threads - but there is a default queue (for each processor) provided by the kernel that will work in most situations.

Workqueues are created with one of:

    struct workqueue_struct *create_workqueue(const char *name);
    struct workqueue_struct *create_singlethread_workqueue(const char *name);

A workqueue created with create_workqueue() will have one worker thread for each CPU on the system;create_singlethread_workqueue(), instead, creates a workqueue with a single worker process. The name of the queue is limited to ten characters; it is only used for generating the "command" for the kernel thread(s) (which can be seen in ps or top).

Tasks to be run out of a workqueue need to be packaged in a struct work_struct structure. This structure may be declared and initialized at compile time as follows:

    DECLARE_WORK(name, void (*function)(void *), void *data);

Here, name is the name of the resulting work_struct structure, function is the function to call to execute the work, and data is a pointer to pass to that function.

To set up a work_struct structure at run time, instead, use the following two macros:

    INIT_WORK(struct work_struct *work, 
              void (*function)(void *), void *data);
    PREPARE_WORK(struct work_struct *work, 
                 void (*function)(void *), void *data);

The difference between the two is that INIT_WORK initializes the linked list pointers within the work_struct structure, while PREPARE_WORKchanges only the function and data pointers. INIT_WORK must be used at least once before queueing the work_struct structure, but shouldnot be used if the work_struct might already be in a workqueue.

Actually queueing a job to be executed is simple:

    int queue_work(struct workqueue_struct *queue, 
                   struct work_struct *work);
    int queue_delayed_work(struct workqueue_struct *queue, 
                    struct work_struct *work,
                           unsigned long delay);

The second form of the call ensures that a minimum delay (in jiffies) passes before the work is actually executed. The return value from both functions is nonzero if the work_struct was actually added to queue (otherwise, it may have already been there and will not be added a second time).

Entries in workqueues are executed at some undefined time in the future, when the associated worker thread is scheduled to run (and after the delay period, if any, has passed). If it is necessary to cancel a delayed task, you can do so with:

    int cancel_delayed_work(struct work_struct *work);

Note that this workqueue entry could actually be executing when cancel_delayed_work() returns; all this function will do is keep it from starting after the call.

To ensure that none of your workqueue entries are running, call:

    void flush_workqueue(struct workqueue_struct *queue);

This would be a good thing to do, for example, in a device driver shutdown routine. Note that if the queue contains work with long delays this call could take a long time to complete. This function will not (as of 2.5.68) wait for any work entries submitted after the call was first made; you should ensure that, for example, any outstanding work queue entries will not resubmit themselves. You should also cancel any delayed entries (with cancel_delayed_work()) first if need be.

Work queues can be destroyed with:

    void destroy_workqueue(struct workqueue_struct *queue);

This operation will flush the queue, then delete it.

Finally, for tasks that do not justify their own workqueue, a "default" work queue (called "events") is defined. work_struct structures can be added to this queue with:

    int schedule_work(struct work_struct *work);
    int schedule_delayed_work(struct work_struct *work, unsigned long delay);

Most users of workqueues can probably use the predefined queue, but one should bear in mind that it is a shared resource. Long delays in the worker function will slow down other users of the queue, and should be avoided. There is a flush_scheduled_work() function which will wait for everything on this queue to be executed. If your module uses the default queue, it should almost certainly callflush_scheduled_work() before allowing itself to be unloaded.

With first call, the work is scheduled immediately and is run as soon as the events worker thread on the current processor wakes up.

While with second call, the work_struct represented by &work will not execute for at least delay timer ticks into the future.

One final note: schedule_work(), schedule_delayed_work() and flush_scheduled_work() are exported to any modules which wish to use them. The other functions (for working with separate workqueues) are exported to GPL-licensed modules only.

Reference
https://lwn.net/Articles/23634/