Sample code for handling _IO

To handle _IO_NOTIFY messages, we can add the code samples shown below to either of the examples provided in the “Simple device resource manager examples” section of the Bones of a Resource Manager chapter. Both of those examples provided the device name /dev/sample. With the changes indicated below, clients can use writes to send data to this device, which the resource manager will store as discrete messages. Other clients can use either ionotify(), poll(), or select() to request notification when that data arrives. When these clients receive notification, they can issue reads to get the data.

Define the device-specific data storage

First, we need to replace this code that's located above the main() function:

#include <sys/iofunc.h>
#include <sys/dispatch.h>

static resmgr_connect_funcs_t    connect_funcs;
static resmgr_io_funcs_t         io_funcs;
static iofunc_attr_t             attr;

with the following:

struct device_attr_s;
#define IOFUNC_ATTR_T   struct device_attr_s

#include <sys/iofunc.h>
#include <sys/dispatch.h>

/*
 * Define a structure and variables for storing the data that
 * is received. When clients write data to us, we store it here.
 * When clients do reads, we get the data from here.
 * The result is a simple message queue.
*/
typedef struct item_s {
    struct item_s   *next;
    char            *data;
} item_t;

/* Extended attributes structure */
typedef struct device_attr_s {
    iofunc_attr_t   attr;
    iofunc_notify_t notify[3];  /* notification list used by
                                   iofunc_notify*() */
    item_t          *firstitem; /* the queue of items */
    int             nitems;     /* number of items in the queue */
} device_attr_t;

/* We only have one device; device_attr is its attribute structure */
static device_attr_t    device_attr;

int io_read( resmgr_context_t *ctp, io_read_t  *msg,
             RESMGR_OCB_T *ocb );
int io_write( resmgr_context_t *ctp, io_write_t *msg,
              RESMGR_OCB_T *ocb );
int io_notify( resmgr_context_t *ctp, io_notify_t *msg,
               RESMGR_OCB_T *ocb );
int io_close_dup( resmgr_context_t *ctp, io_close_t* msg,
                  RESMGR_OCB_T *ocb );

static resmgr_connect_funcs_t  connect_funcs;
static resmgr_io_funcs_t       io_funcs;

We need a place to keep data that's specific to our device. A good place for this is in an attribute structure that we can associate with the /dev/sample device name that we registered. So, in the code above, we defined device_attr_t and IOFUNC_ATTR_T for this purpose. We talk more about this type of device-specific attribute structure in the POSIX-Layer Data Structures chapter.

We need two types of device-specific data:

an array of three notification lists, one for each possible condition that a client can ask to be notified about. In device_attr_t, we called this notify.
a queue to keep the data that gets written to us (the server) and that we use to reply to a client. For this, we defined item_t; it's a type that contains data for a single item, as well as a pointer to the next item_t. In device_attr_t we use firstitem (which points to the first item in the queue) and nitems (which is the number of items).

Note that we removed the definition of attr, since we use device_attr instead.

Create and fill in the I/O functions table

Of course, we have to give the resource manager library the address of our handlers so that it'll know to call them. In the code for main() where we called iofunc_func_init(), we'll add the following code to register our handlers:

/* initialize functions for handling messages */
iofunc_func_init(_RESMGR_CONNECT_NFUNCS, &connect_funcs,
                 _RESMGR_IO_NFUNCS, &io_funcs);

/* for handling _IO_NOTIFY, sent as a result of client
   calls to ionotify(), poll(), and select() */
io_funcs.notify = io_notify;

io_funcs.write = io_write;
io_funcs.read = io_read;
io_funcs.close_dup = io_close_dup;

Since we're using device_attr instead of attr, we need to replace this code:

/* initialize attribute structure used by the device */
iofunc_attr_init(&attr, S_IFNAM | 0666, 0, 0);

/* attach our device name */
id = resmgr_attach(dpp,            /* dispatch handle        */
                   &resmgr_attr,   /* resource manager attrs */
                   "/dev/sample",  /* device name            */
                   _FTYPE_ANY,     /* open type              */
                   0,              /* flags                  */
                   &connect_funcs, /* connect routines       */
                   &io_funcs,      /* I/O routines           */
                   &attr);         /* handle                 */

with the following:

/* initialize attribute structure used by the device */
iofunc_attr_init(&device_attr.attr, S_IFNAM | 0666, 0, 0);
IOFUNC_NOTIFY_INIT(device_attr.notify);
device_attr.firstitem = NULL;
device_attr.nitems = 0;

/* attach our device name */
id = resmgr_attach(dpp,            /* dispatch handle        */
                   &resmgr_attr,   /* resource manager attrs */
                   "/dev/sample",  /* device name            */
                   _FTYPE_ANY,     /* open type              */
                   0,              /* flags                  */
                   &connect_funcs, /* connect routines       */
                   &io_funcs,      /* I/O routines           */
                   &device_attr);  /* handle                 */

Note that we set up our device-specific data in device_attr, and, in the call to resmgr_attach(), we passed &device_attr for the handle parameter.

Implement the notify handler

Now, we need to include the new function to handle the _IO_NOTIFY message:

int io_notify(resmgr_context_t *ctp, io_notify_t *msg,
              RESMGR_OCB_T *ocb)
{
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;
    int             trig;
    
    /* 
     * 'trig' will tell iofunc_notify() which conditions are
     * currently satisfied.  'dattr->nitems' is the number of
     * messages in our list of stored messages.
    */
    trig = _NOTIFY_COND_OUTPUT; /* clients can always give us data */
    if (dattr->nitems > 0)
        trig |= _NOTIFY_COND_INPUT; /* we have some data available */
    
    /*
     * iofunc_notify() will do any necessary handling, including
     * adding the client to the notification list if need be.
    */
    return (iofunc_notify( ctp, msg, dattr->notify, trig,
                           NULL, NULL));
}

As stated above, our io_notify handler will be called when a client calls ionotify(), poll, or select(). In our handler, we're expected to remember who those clients are and what conditions they want to be notified about. We should be able to respond immediately with conditions that are already true. The iofunc_notify() helper function makes this easy.

The first thing we do is determine which of the conditions we handle have currently been met. In this example, we're always able to accept writes, so in the code above we set the _NOTIFY_COND_OUTPUT bit in trig. We also check nitems to see if we have data and set the _NOTIFY_COND_INPUT if we do.

We then call iofunc_notify(), passing it the message that was received (msg), the notification lists (notify), and which conditions have been met (trig). If one of the conditions that the client is asking about has been met, and the client wants us to poll for the condition before arming, then iofunc_notify() will return with a value that indicates what condition has been met and the condition will not be armed. Otherwise, the condition will be armed. In either case, we'll return from the handler with the return value from iofunc_notify().

Earlier, when we talked about the three possible conditions, we mentioned that if you specify _NOTIFY_COND_INPUT, the client is notified when there's one or more units of input data available, and that the number of units is up to you. We said a similar thing about _NOTIFY_COND_OUTPUT and _NOTIFY_COND_OBAND. In the code above, we let the number of units for all these conditions default to 1 by specifying NULL as the second last parameter for iofunc_notify(). If we wanted to use something different, then we would declare an array such as:

int notifycounts[3] =  { 10, 2, 1 };

This sets the units for: _NOTIFY_COND_INPUT to 10; _NOTIFY_COND_OUTPUT to 2; and _NOTIFY_COND_OBAND to 1. We then would pass notifycounts to as the second last parameter.

Implement the write handler

When data arrives, we notify whichever clients have asked for notification. In our sample program, data arrives through clients sending us _IO_WRITE messages, and we handle it using an io_write handler:

int io_write(resmgr_context_t *ctp, io_write_t *msg, RESMGR_OCB_T *ocb)
{
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;
    int             status;
    item_t          *newitem;

    if ((status = iofunc_write_verify(ctp, msg, ocb, NULL)) != EOK)
        return (status);

    if ((msg->i.xtype & _IO_XTYPE_MASK) != _IO_XTYPE_NONE)
        return (ENOSYS);

    if (msg->i.nbytes > 0) {
        
        /* get and store the data */
        if ((newitem = malloc(sizeof(item_t))) == NULL)
            return (errno);
        if ((newitem->data = malloc(msg->i.nbytes+1)) == NULL) {
            free(newitem);
            return (errno);
        }
        /* read the data from the sender's message */
        resmgr_msgget(ctp, newitem->data, msg->i.nbytes,
                       sizeof(msg->i));
        newitem->data[msg->i.nbytes] = '\0';

        if (dattr->firstitem)
            newitem->next = dattr->firstitem;
        else
            newitem->next = NULL;
        dattr->firstitem = newitem;
        dattr->nitems++;

        /* notify clients who may have asked to be notified when
           there is data */
        if (IOFUNC_NOTIFY_INPUT_CHECK(dattr->notify,
                                      dattr->nitems, 0))
            iofunc_notify_trigger(dattr->notify, dattr->nitems,
                                  IOFUNC_NOTIFY_INPUT);
    }
   
    /* set up the number of bytes (returned by client's write()) */
    _IO_SET_WRITE_NBYTES(ctp, msg->i.nbytes);

    if (msg->i.nbytes > 0)
        ocb->attr->attr.flags |= IOFUNC_ATTR_MTIME |
                                 IOFUNC_ATTR_CTIME;

    return (_RESMGR_NPARTS(0));
}

The important part of the above io_write handler is the code within the following section:

if (msg->i.nbytes > 0) {
    ....
}

Here we first allocate space for the incoming data, then use resmgr_msgget() to copy the data from the client message into the allocated space, and then add the data to our queue.

Next, we pass the number of input units that are available to IOFUNC_NOTIFY_INPUT_CHECK() to see if there are enough units to notify clients about. This is checked against the notifycounts that we mentioned above when talking about the io_notify handler. If there are enough units available, then we call iofunc_notify_trigger() telling it that nitems of data are available (IOFUNC_NOTIFY_INPUT means input is available). The iofunc_notify_trigger() function checks the lists of clients asking for notification (notify) and notifies any that asked about input being available.

Implement the read handler

Any client that gets notified will then perform a read to get the data. In our sample program, we handle this with the following io_read handler:

int io_read(resmgr_context_t *ctp, io_read_t *msg, RESMGR_OCB_T *ocb)
{
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;
    int             status;
    
    if ((status = iofunc_read_verify(ctp, msg, ocb, NULL)) != EOK)
        return (status);

    if ((msg->i.xtype & _IO_XTYPE_MASK) != _IO_XTYPE_NONE)
        return (ENOSYS);

    if (dattr->firstitem) {
        size_t  nbytes;
        item_t  *item, *prev;
        
        /* get last item */
        item = dattr->firstitem;
        prev = NULL;
        while (item->next != NULL) {
            prev = item;
            item = item->next;
        }

        /* 
         * figure out number of bytes to give, write the data to the 
         * client's reply buffer, even if we have more bytes than they
         * are asking for, we remove the item from our list
        */
        nbytes = min (strlen (item->data), msg->i.nbytes);

        /* set up the number of bytes (returned by client's read()) */
        _IO_SET_READ_NBYTES (ctp, nbytes);

        /* 
         * write the bytes to the client's reply buffer now since we
         * are about to free the data
        */
        resmgr_msgwrite (ctp, item->data, nbytes, 0);

        /* remove the data from the queue */
        if (prev)
            prev->next = item->next;
        else
            dattr->firstitem = NULL;
        free(item->data);
        free(item);
        dattr->nitems--;
    } else {
        /* the read() will return with 0 bytes */
        _IO_SET_READ_NBYTES (ctp, 0);
    }   

    /* mark the access time as invalid (we just accessed it) */
    if (msg->i.nbytes > 0)
        ocb->attr->attr.flags |= IOFUNC_ATTR_ATIME;

    return (EOK);
}

The important part of the above io_read handler is the code within this section:

if (firstitem) {
    ....
}

We first walk through the queue looking for the oldest item. Then we use resmgr_msgwrite() to write the data to the client's reply buffer. We do this now because the next step is to free the memory that we're using to store that data. We also remove the item from our queue.

Implement the close file handler

Lastly, if a client closes its file descriptor, we must remove the client from our list. This is done using an io_close_dup handler:

int io_close_dup(resmgr_context_t *ctp, io_close_t* msg, 
                 RESMGR_OCB_T *ocb)
{
    device_attr_t   *dattr = (device_attr_t *) ocb->attr;

    /*
     * A client has closed its file descriptor or has terminated.
     * Unblock any threads waiting for notification, then
     * remove the client from the notification list.
     */
    iofunc_notify_trigger_strict( ctp, dattr->notify, INT_MAX, 
                                  IOFUNC_NOTIFY_INPUT );
    iofunc_notify_trigger_strict( ctp, dattr->notify, INT_MAX, 
                                  IOFUNC_NOTIFY_OUTPUT );
    iofunc_notify_trigger_strict( ctp, dattr->notify, INT_MAX, 
                                  IOFUNC_NOTIFY_OBAND );
 
    iofunc_notify_remove(ctp, dattr->notify);

    return (iofunc_close_dup_default(ctp, msg, ocb));
}

In the io_close_dup handler, we called iofunc_notify_remove() and passed it ctp, which contains the information that identifies the client, and notify, which contains the list of clients, to remove the client from the lists.