Writing Device Drivers

Callback Functions

Certain DDI mechanisms provide a callback mechanism. DDI functions provide a mechanism for scheduling a callback when a condition is met. Conditions for which callback functions are used include:

When a transfer has completed
When a resource might have become available
When a time-out period has expired

In some sense, callback functions are similar to entry points--interrupt handlers, for example. DDI functions that allow callbacks expect the callback function to perform certain tasks. In the case of DMA routines, a callback function must return a value indicating whether the callback function needs to be rescheduled in case of a failure.

Callback functions execute as a separate interrupt thread and must handle all the usual multithreading issues.

Note - A driver must cancel all scheduled callback functions before detaching a device.

Software State Management

To assist device driver writers in allocating state structures, the Solaris 9 DDI/DKI provides a set of memory management routines called the software state management routines (also known as the soft state routines). These routines dynamically allocate, retrieve, and destroy memory items of a specified size, and hide the details of list management. An instance number is used to identify the desired memory item; this number can be (and usually is) the instance number assigned by the system.

Routines are provided to:

Initialize a driver's soft state list
Allocate space for an instance of a driver's soft state
Retrieve a pointer to an instance of a driver's soft state
Free the memory for an instance of a driver's soft state
Finish using a driver's soft state list

See "Loadable Driver Interfaces" for an example of how to use these routines.

Programmed I/O Device Access

Programmed I/O device access is the act of reading and writing of device registers or device memory by the host CPU. The Solaris DDI provides interfaces for mapping a device's registers or memory by the kernel as well as interfaces for reading and writing to device memory from the driver. These interfaces are designed to enable drivers to be developed that are platform and bus independent, by automatically managing any difference in device and host endianness as well as enforcing any memory-store ordering requirements imposed by the device.

Direct Memory Access (DMA)

Solaris defines a high-level, architecture-independent model for supporting DMA-capable devices. The Solaris DDI is designed to shield drivers from platform-specific details, which enables a common driver to be developed that runs across multiple platforms and architectures.

Driver and Device Statistics

Solaris provides a rich set of interfaces for maintaining and exporting kernel-level statistics, or kstats. Drivers are free to use these interfaces to export driver and device statistics that can be used by user applications to observe the internal state of the driver. See the kstat_create(9F) and kstat(3KSTAT) man pages for additional information.

Driver Context

The driver context determines which kernel routines the driver is permitted to call. There are four contexts in which driver code executes:

User context - A driver entry point has user context if it was directly invoked because of a user thread. For example, the read(9E) entry point of the driver, invoked by a read(2) system call, has user context.
Kernel context - A driver function has kernel context if it was invoked by some other part of the kernel. In a block device driver, the strategy(9E) entry point can be called by the pageout daemon to write pages to the device. Because the page daemon has no relation to the current user thread, strategy(9E) has kernel context in this case.
Interrupt context - Interrupt context is a more restrictive form of kernel context. Driver interrupt routines operate in interrupt context and have an interrupt level associated with them. Callback routines also operate in an interrupt context. See Chapter 7, Interrupt Handlers for more information.
High-level interrupt context - High-level interrupt context is a more restricted form of interrupt context. If ddi_intr_hilevel(9F) indicates that an interrupt is high level, the driver interrupt handler will run in high-level interrupt context. See Chapter 7, Interrupt Handlers for more information.

The manual pages in section 9F document the allowable contexts for each function. For example, in kernel context the driver must not call copyin(9F).

Returning Errors

Device drivers do not usually print messages, except for unexpected errors such as data corruption. Instead, the driver entry points should return error codes so that the application can determine how to handle the error. If the driver must print a message, it should use cmn_err(9F) to do so. This is similar to the C function printf(3C), which prints to the console, to the message buffer, or both.

The format string specifier interpreted by cmn_err(9F) is similar to the printf(3C) format string, with the addition of the format %b, which prints bit fields. Callers to cmn_err(9F) also specify the level, which indicates the label to be printed. The first character of the format string is treated specially. See the cmn_err(9F) man page for more details.

The level CE_PANIC has the side effect of crashing the system. This level should be used only if the system is in such an unstable state that to continue would cause more problems. It can also be used to get a system core dump when debugging. It should not be used in production device drivers.

Dynamic Memory Allocation

Device drivers must be prepared to simultaneously handle all attached devices that they claim to drive. There should be no driver limit on the number of devices that the driver handles, and all per-device information must be dynamically allocated.

void *kmem_alloc(size_t size, int flag);

The standard kernel memory allocation routine is kmem_alloc(9F). It is similar to the C library routine malloc(3C), with the addition of the flag argument. The flag argument can be either KM_SLEEP or KM_NOSLEEP, indicating whether the caller is willing to block if the requested size is not available. If KM_NOSLEEP is set, and memory is not available, kmem_alloc(9F) returns NULL.

kmem_zalloc(9F) is similar to kmem_alloc(9F), but also clears the contents of the allocated memory.

Note - Kernel memory is a limited resource, not pageable, and competes with user applications and the rest of the kernel for physical memory. Drivers that allocate a large amount of kernel memory can cause system performance to degrade.

void kmem_free(void *cp, size_t size);

Memory allocated by kmem_alloc(9F) or by kmem_zalloc(9F) is returned to the system with kmem_free(9F). This is similar to the C library routine free(3C), with the addition of the size argument. Drivers must keep track of the size of each object they allocate in order to call kmem_free(9F) later.

Hotplugging

In general, this manual does not highlight hotplugging information; following the rules and suggestions for writing device drivers given in this book should enable any driver to handle hotplugging. In particular, you should ensure that autoconfiguration works (see Chapter 5, Driver Autoconfiguration) and always include a working detach(9E) routine in any driver. Writers of drivers with power management issues should also follow the information given in Chapter 9, Power Management. SCSI HBA drivers may need to add a cb_ops structure to their dev_ops structure (see Chapter 15, SCSI Host Bus Adapter Drivers) to take advantage of hotplugging capabilities.

Previous versions of the Solaris operating system required hotpluggable drivers to include a DT_HOTPLUG property, but such a property is no longer required. (Driver writers are free, however, to include it and have routines make use of it as they see fit.)

For more information, visit http://soldc.sun.com/developer/support/driver/notes/scsi-hotplug.html, which contains links to hotplugging whitepapers.

Driver Layout

Driver code is usually divided into the following files:

Headers (.h files)
Source files (.c files)
Optional configuration files (driver.conf file)


2. Overview of Solaris Device Drivers Device Driver Design Considerations DDI/DKI Facilities Interrupt Handling


	Callback Functions Certain DDI mechanisms provide a callback mechanism. DDI functions provide a mechanism for scheduling a callback when a condition is met. Conditions for which callback functions are used include: When a transfer has completed When a resource might have become available When a time-out period has expired In some sense, callback functions are similar to entry points--interrupt handlers, for example. DDI functions that allow callbacks expect the callback function to perform certain tasks. In the case of DMA routines, a callback function must return a value indicating whether the callback function needs to be rescheduled in case of a failure. Callback functions execute as a separate interrupt thread and must handle all the usual multithreading issues. Note - A driver must cancel all scheduled callback functions before detaching a device. Software State Management To assist device driver writers in allocating state structures, the Solaris 9 DDI/DKI provides a set of memory management routines called the software state management routines (also known as the soft state routines). These routines dynamically allocate, retrieve, and destroy memory items of a specified size, and hide the details of list management. An instance number is used to identify the desired memory item; this number can be (and usually is) the instance number assigned by the system. Routines are provided to: Initialize a driver's soft state list Allocate space for an instance of a driver's soft state Retrieve a pointer to an instance of a driver's soft state Free the memory for an instance of a driver's soft state Finish using a driver's soft state list See "Loadable Driver Interfaces" for an example of how to use these routines. Programmed I/O Device Access Programmed I/O device access is the act of reading and writing of device registers or device memory by the host CPU. The Solaris DDI provides interfaces for mapping a device's registers or memory by the kernel as well as interfaces for reading and writing to device memory from the driver. These interfaces are designed to enable drivers to be developed that are platform and bus independent, by automatically managing any difference in device and host endianness as well as enforcing any memory-store ordering requirements imposed by the device. Direct Memory Access (DMA) Solaris defines a high-level, architecture-independent model for supporting DMA-capable devices. The Solaris DDI is designed to shield drivers from platform-specific details, which enables a common driver to be developed that runs across multiple platforms and architectures. Driver and Device Statistics Solaris provides a rich set of interfaces for maintaining and exporting kernel-level statistics, or kstats. Drivers are free to use these interfaces to export driver and device statistics that can be used by user applications to observe the internal state of the driver. See the `kstat_create`(9F) and `kstat`(3KSTAT) man pages for additional information. Driver Context The driver context determines which kernel routines the driver is permitted to call. There are four contexts in which driver code executes: User context - A driver entry point has user context if it was directly invoked because of a user thread. For example, the `read`(9E) entry point of the driver, invoked by a `read`(2) system call, has user context. Kernel context - A driver function has kernel context if it was invoked by some other part of the kernel. In a block device driver, the `strategy`(9E) entry point can be called by the pageout daemon to write pages to the device. Because the page daemon has no relation to the current user thread, `strategy`(9E) has kernel context in this case. Interrupt context - Interrupt context is a more restrictive form of kernel context. Driver interrupt routines operate in interrupt context and have an interrupt level associated with them. Callback routines also operate in an interrupt context. See Chapter 7, Interrupt Handlers for more information. High-level interrupt context - High-level interrupt context is a more restricted form of interrupt context. If `ddi_intr_hilevel`(9F) indicates that an interrupt is high level, the driver interrupt handler will run in high-level interrupt context. See Chapter 7, Interrupt Handlers for more information. The manual pages in section 9F document the allowable contexts for each function. For example, in kernel context the driver must not call `copyin`(9F). Returning Errors Device drivers do not usually print messages, except for unexpected errors such as data corruption. Instead, the driver entry points should return error codes so that the application can determine how to handle the error. If the driver must print a message, it should use `cmn_err`(9F) to do so. This is similar to the C function `printf`(3C), which prints to the console, to the message buffer, or both. The `format` string specifier interpreted by `cmn_err`(9F) is similar to the `printf`(3C) format string, with the addition of the format %b, which prints bit fields. Callers to `cmn_err`(9F) also specify the `level`, which indicates the label to be printed. The first character of the format string is treated specially. See the `cmn_err`(9F) man page for more details. The level `CE_PANIC` has the side effect of crashing the system. This level should be used only if the system is in such an unstable state that to continue would cause more problems. It can also be used to get a system core dump when debugging. It should not be used in production device drivers. Dynamic Memory Allocation Device drivers must be prepared to simultaneously handle all attached devices that they claim to drive. There should be no driver limit on the number of devices that the driver handles, and all per-device information must be dynamically allocated. `void kmem_alloc(size_t size, int flag);` The standard kernel memory allocation routine is `kmem_alloc`(9F). It is similar to the C library routine `malloc`(3C), with the addition of the `flag` argument. The `flag` argument can be either `KM_SLEEP` or `KM_NOSLEEP`, indicating whether the caller is willing to block if the requested size is not available. If `KM_NOSLEEP` is set, and memory is not available, `kmem_alloc`(9F) returns `NULL`. `kmem_zalloc`(9F) is similar to `kmem_alloc`(9F), but also clears the contents of the allocated memory. Note -* Kernel memory is a limited resource, not pageable, and competes with user applications and the rest of the kernel for physical memory. Drivers that allocate a large amount of kernel memory can cause system performance to degrade. `void kmem_free(void cp, size_t size);` Memory allocated by `kmem_alloc`(9F) or by `kmem_zalloc`(9F) is returned to the system with `kmem_free`(9F). This is similar to the C library routine `free`(3C), with the addition of the `size` argument. Drivers must* keep track of the size of each object they allocate in order to call `kmem_free`(9F) later. Hotplugging In general, this manual does not highlight hotplugging information; following the rules and suggestions for writing device drivers given in this book should enable any driver to handle hotplugging. In particular, you should ensure that autoconfiguration works (see Chapter 5, Driver Autoconfiguration) and always include a working `detach`(9E) routine in any driver. Writers of drivers with power management issues should also follow the information given in Chapter 9, Power Management. SCSI HBA drivers may need to add a `cb_ops` structure to their `dev_ops` structure (see Chapter 15, SCSI Host Bus Adapter Drivers) to take advantage of hotplugging capabilities. Previous versions of the Solaris operating system required hotpluggable drivers to include a `DT_HOTPLUG` property, but such a property is no longer required. (Driver writers are free, however, to include it and have routines make use of it as they see fit.) For more information, visit http://soldc.sun.com/developer/support/driver/notes/scsi-hotplug.html, which contains links to hotplugging whitepapers. Driver Layout Driver code is usually divided into the following files: Headers (`.h` files) Source files (`.c` files) Optional configuration files (`driver.conf` file)