Synchronizing Memory Objects

At various points when the memory object is accessed (including the time of removal of the DMA resources), the driver might need to synchronize the memory object with respect to various caches. This section gives guidelines on when and how to synchronize memory objects.

Cache

Cache is a very high-speed memory that sits between the CPU and the system's main memory (CPU cache), or between a device and the system's main memory (I/O cache), as shown in Figure 8-1.

Figure 8-1 CPU and System I/O Caches

When an attempt is made to read data from main memory, if the associated cache determines whether it contains the requested data, it quickly satisfies the request. If the cache does not have the data, it retrieves the data from main memory, passes the data on to the requestor, and saves the data in case that data is requested again.

Similarly, on a write cycle, the data is stored in the cache quickly and the CPU or device is allowed to continue executing (transferring). This takes much less time than if the CPU or device had to wait for the data to be written to memory.

With this model, after a device transfer has been completed, the data can still be in the I/O cache but not yet in main memory. If the CPU accesses the memory, it might read the wrong data from the CPU cache. To ensure a consistent view of the memory for the CPU, the driver must call a synchronization routine to flush the data from the I/O cache and update the CPU cache with the new data. Similarly, a synchronization step is required if data modified by the CPU is to be accessed by a device.

There might also be additional caches and buffers between the device and memory, such as caches associated with bus extenders or bridges. Use ddi_dma_sync(9F) to synchronize all applicable caches.

ddi_dma_sync(9F)

If a memory object has multiple mappings--such as for a device (through the DMA handle) and for the CPU--and one mapping is used to modify the memory object, the driver needs to call ddi_dma_sync(9F) to ensure that the modification of the memory object is complete before accessing the object through another mapping. ddi_dma_sync(9F) can also inform other mappings of the object that any cached references to the object are now stale. Additionally, ddi_dma_sync(9F) flushes or invalidates stale cache references as necessary.

Generally, the driver has to call ddi_dma_sync(9F) when a DMA transfer completes. The exception to this is that deallocating the DMA resources with ddi_dma_unbind_handle(9F), does an implicit ddi_dma_sync(9F) on behalf of the driver.

int ddi_dma_sync(ddi_dma_handle_t handle, off_t off,
size_t length, uint_t type);

If the object is going to be read by the DMA engine of the device, the device's view of the object must be synchronized by setting type to DDI_DMA_SYNC_FORDEV. If the DMA engine of the device has written to the memory object, and the object is going to be read by the CPU, the CPU's view of the object must be synchronized by setting type to DDI_DMA_SYNC_FORCPU.

Here is an example of synchronizing a DMA object for the CPU:

if (ddi_dma_sync(xsp->handle, 0, length, DDI_DMA_SYNC_FORCPU)
    == DDI_SUCCESS) {
    /* the CPU can now access the transferred data */
    ...        
} else {
         error handling
}

If the only mapping that concerns the driver is one for the kernel (such as memory allocated by ddi_dma_mem_alloc(9F)), the flag DDI_DMA_SYNC_FORKERNEL can be used. If the system can synchronize the kernel's view faster than the CPU's view, it will do so; otherwise, it acts the same as DDI_DMA_SYNC_FORCPU.

DMA Windows

The system might be unable to allocate resources for a large object. If this occurs, the transfer must be broken into a series of smaller transfers. The driver can either do this itself, or it can let the system allocate resources for only part of the object, thereby creating a series of DMA windows. Allowing the system to allocate resources is the preferred solution, as the system can manage the resources more effectively than the driver.

A DMA window has attributes offset (from the beginning of the object) and length. After a partial allocation, only a range of length bytes starting at offset has resources allocated for it.

A DMA window is requested by specifying the DDI_DMA_PARTIAL flag as a parameter to ddi_dma_buf_bind_handle(9F) or ddi_dma_addr_bind_handle(9F). Both functions return DDI_DMA_PARTIAL_MAP if a window can be established. However, the system might allocate resources for the entire object (less overhead), in which case DDI_DMA_MAPPED is returned. The driver should check the return value (see Example 8-7) to determine whether DMA windows are in use.

Example 8-7 Setting Up DMA Windows

static int
xxstart (caddr_t arg)
{
        struct xxstate *xsp = (struct xxstate *)arg;
        struct device_reg *regp = xsp->reg;
        ddi_dma_cookie_t cookie;
        int    status;
        mutex_enter(&xsp->mu);
        if (xsp->busy) {
            /* transfer in progress */
            mutex_exit(&xsp->mu);
            return (DDI_DMA_CALLBACK_RUNOUT);
        }
        xsp->busy = 1;
        mutex_exit(&xsp->mu);
        if (transfer is a read) {
            flags = DDI_DMA_READ;
        } else {
            flags = DDI_DMA_WRITE;
        }
        flags |= DDI_DMA_PARTIAL;
        status = ddi_dma_buf_bind_handle(xsp->handle, xsp->bp,
            flags, xxstart, (caddr_t)xsp, &cookie, &ccount);
        if (status != DDI_DMA_MAPPED &&
            status != DDI_DMA_PARTIAL_MAP)
                return (DDI_DMA_CALLBACK_RUNOUT);
        if (status == DDI_DMA_PARTIAL_MAP) {
            ddi_dma_numwin(xsp->handle, &xsp->nwin);
            xsp->partial = 1;
            xsp->windex = 0;
        } else {
            xsp->partial = 0;
        }
        ...
        program the DMA engine
        ...
        return (DDI_DMA_CALLBACK_DONE);
}

Two functions operate with DMA windows. The first, ddi_dma_numwin(9F), returns the number of DMA windows for a particular DMA object. The other function, ddi_dma_getwin(9F), allows repositioning (reallocation of system resources) within the object. It shifts the current window to a new window within the object. Because ddi_dma_getwin(9F) reallocates system resources to the new window, the previous window becomes invalid.

Caution - Do not move the DMA windows with a call to ddi_dma_getwin(9F) before transfers into the current window are complete. Wait until the transfer to the current window is complete (when the interrupt arrives) then call ddi_dma_getwin(9F) or data will be corrupted.

ddi_dma_getwin(9F) is normally called from an interrupt routine (see Example 8-8). The first DMA transfer is initiated as a result of a call to the driver. Subsequent transfers are started from the interrupt routine.

The interrupt routine examines the status of the device to determine if the device completed the transfer successfully. If not, normal error recovery occurs. If the transfer was successful, the routine must determine if the logical transfer is complete (the entire transfer specified by the buf(9S) structure) or if only one DMA window was moved. If it was only one window, the interrupt routine moves the window with ddi_dma_getwin(9F), retrieves a new cookie, and starts another DMA transfer.

If the logical request has been completed, the interrupt routine checks for pending requests and starts a transfer, if necessary. Otherwise, it returns without invoking another DMA transfer. Example 8-8 illustrates the usual flow control.

Example 8-8 Interrupt Handler Using DMA Windows

static uint_t
xxintr(caddr_t arg)
{
        struct xxstate *xsp = (struct xxstate *)arg;
        uint8_t    status;
        volatile   uint8_t   temp;
        mutex_enter(&xsp->mu);
        /* read status */
        status = ddi_get8(xsp->access_hdl, &xsp->regp->csr);
        if (!(status & INTERRUPTING)) {
            mutex_exit(&xsp->mu);
            return (DDI_INTR_UNCLAIMED);
        }
        ddi_put8(xsp->access_hdl,&xsp->regp->csr, CLEAR_INTERRUPT);
        /* for store buffers */
        temp = ddi_get8(xsp->access_hdl, &xsp->regp->csr);
        if (an error occurred during transfer) {
            bioerror(xsp->bp, EIO);
            xsp->partial = 0;
        } else {
            xsp->bp->b_resid -= amount transferred;
        }

        if (xsp->partial && (++xsp->windex < xsp->nwin)) {
            /* device still marked busy to protect state */
            mutex_exit(&xsp->mu);
            (void) ddi_dma_getwin(xsp->handle, xsp->windex,
                    &offset, &len, &cookie, &ccount);
         program the DMA engine with the new cookie(s)
            ...
            return (DDI_INTR_CLAIMED);
        }
        ddi_dma_unbind_handle(xsp->handle);
        biodone(xsp->bp);
        xsp->busy = 0;
        xsp->partial = 0;
        mutex_exit(&xsp->mu);
        if (pending transfers) {
            (void) xxstart((caddr_t)xsp);
        }
        return (DDI_INTR_CLAIMED);
}