| RSMERR_BAD_SEG_HNDL | Invalid segment handle. |
Event Operations
Event operations enable processes synchronization on memory access events. If a process cannot use the rsm_intr_signal_wait() function, it may multiplex event waiting by obtaining a poll descriptor with rsm_memseg_get_pollfd() and using the poll system call.
Note - Using the rsm_intr_signal_post() and rsm_intr_signal_wait() operations incurs the processing overhead of ioctl calls to the kernel.
Post Signal
int rsm_intr_signal_post(void *memseg, uint_t flags);
The void pointer *memseg can be type cast to either an import segment handle or an export segment handle. If *memseg refers to an import handle, this function signals the exporting process. If *memseg refers to an export handle, this function signals all importers of that segment. Setting the flags argument to RSM_SIGPOST_NO_ACCUMULATE discards this event if an event is already pending for the target segment.
Return Values: Returns 0 if successful; returns an error value otherwise.
| RSMERR_BAD_SEG_HNDL | Invalid segment handle |
| RSMERR_REMOTE_NODE_UNREACHABLE | Remote node not reachable |
Wait for Signal
int rsm_intr_signal_wait(void * memseg, int timeout);
The void pointer *memseg can be type cast to either an import segment handle or an export segment handle. The process blocks for up to timeout milliseconds or until an event occurs. If the value is -1, the process blocks until an event occurs or until interrupted.
Return Values: Returns 0 if successful; returns an error value otherwise.
| RSMERR_BAD_SEG_HNDL | Invalid segment handle |
| RSMERR_TIMEOUT | Timer expired |
| RSMERR_INTERRUPTED | Wait interrupted |
Get pollfd
int rsm_memseg_get_pollfd(void *memseg, struct pollfd *pollfd);
This function initializes the specified pollfd structure with a descriptor for the specified segment and the singular fixed event generated by rsm_intr_signal_post(). Use the pollfd structure with the poll system call to wait for the event signalled by rsm_intr_signal_post. If the memory segment is not currently published, this will not return a valid pollfd. Each successful call increments a pollfd reference count for the specified segment.
Return Values: Returns 0 if successful; returns an error value otherwise.
| RSMERR_BAD_SEG_HNDL | Invalid segment handle |
Release pollfd
int rsm_memseg_release_pollfd(oid *memseg);
This call decrements the pollfd reference count for the specified segment. If the reference count is nonzero, segment unpublish, destroy, or unmap operations will fail.
Return Values: Returns 0 if successful; returns an error value otherwise.
| RSMERR_BAD_SEG_HNDL | Invalid segment handle |
RSMAPI General Usage Notes
These usage notes describe general considerations for the export and import sides of a shared-memory operation, in addition to general information regarding segments, file descriptors, and RSM configurable parameters.
Segment Allocation and File Descriptor Usage
The system allocates a file descriptor (inaccessible to the application importing or exporting memory) for each export/import operation. The default per-process file descriptor allocation limit is 256. The importing or exporting application must adjust the allocation limit appropriately (for example, by using getrlimit or setrlimit system calls). If the application increases the file descriptor limit beyond 256, values of file descriptors allocated for export and import segments will start at 256 to avoid interfering with normal file descriptor allocation by the application. This behavior accommodates the use of certain libc functions in 32-bit applications that only work with file descriptor values below 256.
Export-Side Considerations
The application must prevent access to segment data until the rebind operation is complete. Segment data access during rebind will not cause a system failure, but data content results are undefined. The virtual address space must be currently mapped and valid.
Import-Side Considerations
The controller specified for a segment import must have a physical connection with the controller used in the export of the segment.
RSM Configurable Parameters
The SUNWrsm software package includes an rsm.conf file, located in /usr/kernel/drv. This file is a configuration file for RSM and can be used to specify values for certain configurable RSM properties. The configurable properties currently defined in rsm.conf include max-exported-memory and enable-dynamic-reconfiguration.
| max-exported-memory | This property specifies an upper limit on the amount of exportable memory, expressed as a percentage of total available memory. Giving this property a value of 0 indicates that there is no limit to the amount of exportable memory. |
| enable-dynamic-reconfiguration | The value of this property indicates whether dynamic reconfiguration is enabled. A value of 0 indicates dynamic reconfiguration is disabled; a value of 1 enables dynamic reconfiguration support. The default value for this property is 1. |
RSMAPI Usage Example
This section provides a simple program to illustrate the usage of the RSMAPI. The program runs on two nodes: an exporter node and an importer node. The exporter node creates and publishes a memory segment, then waits for a message to be written in the segment. The importer node connects to the exported segment, writes a message, and then signals the exporter.
/*
* Copyright (c) 1998 by Sun Microsystems, Inc.
* All rights reserved.
*/
#include <stdio.h>
#include <rsm/rsmpai.h>
#include <errno.h>
/*
To run this program do the following:
First node(assuming node id = 1):
rsmtest -e -n 2
Second node(assuming node id = 2):
rsmtest -i -n 1
The program will prompt the importer for a message at the
console. Enter any message and hit return. The message will
be displayed on the export console.
*/
typedef struct {
char out;
char in;
char data[1];
}msg_t;
#define SEG_ID 0x400000
#define EXPORT 0
#define IMPORT 1
#define BUFSIZE (1024 * 8)
#define DEFAULT_SEGSZ BUFSIZE
#define RSM_PERM_READ 0400
#define RSM_PERM_WRITE 0200
#define RSM_PERM_RDWR (RSM_PERM_READ|RSM_PERM_WRITE)
#define
RSM_ACCESS_TRUSTED
0666
rsm_topology_t *tp;
int iterations = 10;
int mode = EXPORT;
int test = 0;
char *buf;
int buflen = BUFSIZE;
int offset = 0;
volatile char *iva;
int status;
rsm_memseg_id_t segid;
rsmapi_controller_handle_t ctrl;
rsmapi_controller_attr_t attr;
rsm_memseg_export_handle_t seg;
rsm_memseg_import_handle_t imseg;
rsm_access_entry_t list[2];
rsm_node_id_t dest;
extern void *valloc(size_t);
extern void exit();
extern void sleep();
extern int atoi(const char *);
/* The following function exports a segment and publishes it.
*/
static int
export()
{
int i;
/* allocate and clear memory */
buf = (char *)valloc(buflen);
if (!buf) {
(void) fprintf(stderr, "Unable to allocate memory\n");
exit (1);
}
for (i = 0; i < buflen; i++)
buf[i] = 0;
/* Create an export memory segment */
status = rsm_memseg_export_create(ctrl, &seg, (void *)buf, buflen);
if (status != 0) {
(void) fprintf(stderr,
"unable to create an exported segment %d\n", status);
exit(1);
}
/* Set up access list for publishing to nodes 1 and 2 */
list[0].ae_node = tp->topology_hdr.local_nodeid ;
/* Allow read and write permissions */
list[0].ae_permission = RSM_ACCESS_TRUSTED;
list[1].ae_node = tp->topology_hdr.local_nodeid + 1;
/* Allow read and write permissions */
list[1].ae_permission = RSM_ACCESS_TRUSTED;
/* Publish the created export segment */
status = rsm_memseg_export_publish(seg, &segid, list, 0);
if (status != 0) {
(void) fprintf(stderr, "unable to pub segment %d\n", status);
exit(1);
}
return (0);
}
/* The following function is used to connect to an exported memory segment.
*/
static void
import()
{
/* Connect to exported segment and set up mapping for
* access through local virtual addresses.
*/
again:
status = rsm_memseg_import_connect(ctrl, dest, segid, RSM_PERM_RDWR,
&imseg);
if (status != 0) {
(void) fprintf(stderr,
"unable to conect to segment %x err %x\n",
segid, status);
sleep(1);
goto again;
}
iva = NULL;
status = rsm_memseg_import_map(imseg, (void **)&iva,
RSM_MAP_NONE, RSM_PERM_RDWR, 0, buflen);
if (status != 0) {
(void) fprintf(stderr, "unable to mmap segment %d\n", status);
exit(1);
}
}
/* Unpublish and destroy the export segment */
static void
export_close()
{
again:
status = rsm_memseg_export_unpublish(seg);
if (status != 0) {
(void) fprintf(stderr,
"unable to create an unpub segment %d\n", status);
sleep(10);
goto again;
}
status = rsm_memseg_export_destroy(seg);
if (status != 0) {
(void) fprintf(stderr, "unable to destroy segment %d\n",
status);
exit(1);
}
}
/* Unmap the virtual address mapping and disconnect the segment */
static void
import_close()
{
status = rsm_memseg_import_unmap(imseg);
if (status != 0) {
(void) fprintf(stderr, "unable to unmap segment %d\n", status);
exit(1);
}
status = rsm_memseg_import_disconnect(imseg);
if (status != 0) {
(void) fprintf(stderr,
"unable to disconnect segment %d\n", status);
exit(1);
}
}
static void
test0()
{
volatile msg_t *mbuf;
/* Barrier to report error */
rsmapi_barrier_t bar;
int i;
if (mode == EXPORT) {
(void) export();
mbuf = (msg_t *)(buf + offset);
mbuf->in = mbuf->out = 0;
} else {
import();
mbuf = (msg_t *)(iva + offset);
rsm_memseg_import_init_barrier(imseg, RSM_BARRIER_NODE, &bar);
}
(void) printf("Mbuf is %x\n", (uint_t)mbuf);
while (iterations-- > 0) {
int e;
switch (mode) {
case EXPORT:
while (mbuf->out == mbuf->in) {
(void) rsm_intr_signal_wait(seg, 1000);
}
(void) printf("msg [0x%x %d %d] ",
(uint_t)mbuf, (int)mbuf->out, mbuf->in);
for (i = 0; mbuf->data[i] != '\0' && i < buflen; i++) {
(void) putchar(mbuf->data[i]);
mbuf->data[i] = '?';
}
(void) putchar('\n');
mbuf->out++;
break;
case IMPORT:
(void) printf("Enter msg [0x%x %d]: ",
(uint_t)mbuf, mbuf->out, mbuf->in);
retry:
e = rsm_memseg_import_open_barrier(&bar);
if (e != 0) {
(void) printf("Barrier open failed %x\n", e);
exit(1);
}
for (i = 0; (mbuf->data[i] = getchar()) != '\n'; i++)
;
mbuf->data[i] = '\0';
rsm_memseg_import_order_barrier(&bar);
mbuf->in++;
e = rsm_memseg_import_close_barrier(&bar);
if (e != 0) {
(void) printf("Barrier close failed, %d\n", e);
goto retry;
}
(void)rsm_intr_signal_post(imseg);
break;
}
}
if (mode == IMPORT) {
import_close();
} else {
export_close();
}
}
void
main(int argc, char *argv[])
{
int unit = 0;
char *device = "sci0";
int i;
segid = SEG_ID;
buflen = DEFAULT_SEGSZ;
while ((i = getopt(argc, argv, "OCGeid:b:sl:n:k:t:c:u:v")) != -1) {
switch (i) {
case 'e':
mode = EXPORT;
break;
case 'i':
mode = IMPORT;
break;
case 'n':
dest = atoi(optarg);
if ((int)dest < 0) dest = 0;
break;
default:
(void) fprintf(stderr, "Usage: %s -ei -n dest\n",
argv[0]);
exit(1);
}
}
status = rsm_get_controller(device, &ctrl);
if (status != 0) {
(void) fprintf(stderr, "Unable to get controller\n");
exit(1);
}
status = rsm_get_controller_attr(ctrl, &attr);
status = rsm_get_interconnect_topology(&tp);
if (status != 0) {
(void) fprintf(stderr, "Unable to get topology\n");
exit(1);
} else {
(void) printf("Local node id = %d\n",
tp->topology_hdr.local_nodeid);
}
if (dest == 0) {
dest = tp->topology_hdr.local_nodeid;
(void) printf("Dest is adjusted to %d\n", dest);
}
switch (test) {
case 0:
test0();
break;
default:
(void) printf("No test executed\n");
break;
}
}
|