Interprocess Communication

The following interprocess communication (IPC) primitives continue to work between 64-bit and 32-bit processes:

The System V IPC primitives, such as shmop(2), semop(2), msgsnd(2)
mmap(2) on shared files
pipe(2) between processes
door_call(3DOOR) between processes
rpc(3NSL) between processes on the same or different machines using the external data representation described in xdr(3NSL)

Although all these primitives allow interprocess communication between 32-bit and 64-bit processes, you might need to take explicit steps to ensure that data being exchanged between processes is correctly interpreted by all of them. For example, two processes sharing data described by a C data structure containing variables of type long cannot do so without understanding that a 32-bit process views this variable as a 4-byte quantity, while a 64-bit process views this variable as an 8-byte quantity.

One way to handle this difference is to ensure that the data has exactly the same size and meaning in both processes. Build the data structures using fixed-width types, such as int32_t and int64_t.

A family of derived types that mirrors the system derived types is available in <sys/types32.h>. These types possess the same sign and sizes as the fundamental types of the 32-bit system but are defined in such a way that the sizes are invariant between the ILP32 and LP64 compilation environments.

Sharing pointers between 32-bit and 64-bit processes is substantially more difficult. Obviously, pointer sizes are different, but more importantly, while there is a 64-bit integer quantity (long long) in existing C usage, a 64-bit pointer has no equivalent in a 32-bit environment. In order for a 64-bit process to share data with a 32-bit process, the 32-bit process can only "see" up to 4 gigabytes of that shared data at a time.

The XDR routine xdr_long(3NSL) might seem to be a problem; however, it is still handled as a 32-bit quantity over the wire to be compatible with existing protocols. If the 64-bit version of the routine is asked to encode a long value that does not fit into a 32-bit quantity, the encode operation fails.

ELF and System Generation Tools

64-bit binaries are stored in files in ELF64 format, which is a direct analog of the ELF32 format, except that most fields have grown to accommodate full 64-bit applications. ELF64 files can be read using elf(3ELF) APIs; for example, elf_getarhdr(3ELF).

Both 32-bit and 64-bit versions of the ELF library,elf(3ELF) , support both ELF32 and ELF64 formats and their corresponding APIs. This allows applications to build, read, or modify both file formats from either a 32-bit or a 64-bit system (though a 64-bit system is still required to execute a 64-bit program).

In addition, Solaris provides a set of GELF (Generic ELF) interfaces that allow the programmer to manipulate both formats using a single, common API. See elf(3ELF).

All of the system ELF utilities, including ar(1), nm(1), ld(1) and dump(1), have been updated to accept both ELF formats.

`/proc`

The /proc interfaces are available to both 32-bit and 64-bit applications. 32-bit applications can examine and control the state of other 32-bit applications. Thus, an existing 32-bit debugger can be used to debug a 32-bit application.

64-bit applications can examine and control 32-bit or 64-bit applications. However, 32-bit applications are unable to control 64-bit applications, because the 32-bit APIs do not allow the full state of 64-bit processes to be described. Thus, a 64-bit debugger is required to debug a 64-bit application.

`libkvm` and `/dev/ksyms`

The 64-bit version of the Solaris system is implemented using a 64-bit kernel. Applications that examine or modify the contents of the kernel directly must be converted to 64-bit applications and linked with the 64-bit version of sysinfo(2).

Before doing this conversion and cleanup work, you should examine why the application needs to look directly at kernel data structures in the first place. It is possible that in the time since the program was first ported or created, additional interfaces have been made available on the Solaris platform, to extract the needed data with system calls. See sysinfo(2), kstat(3KSTAT), sysconf(3C), and proc(4) as the most common alternative APIs. If these interfaces can be used instead of kvm_open(3KVM), use them and leave the application as 32-bit for maximum portability. As a further benefit, most of these APIs are probably faster and might not require the same security privileges needed to access kernel memory.

The 32-bit version of libkvm returns a failure from any attempt to use kvm_open(3KVM) on a 64-bit kernel or crash dump. Similarly, the 64-bit version of libkvm returns failure from any attempt to use kvm_open(3KVM) on a 32-bit kernel crash dump.

Because the kernel is a 64-bit program, applications that open /dev/ksyms to examine the kernel symbol table directly need to be enhanced to understand ELF64 format.

The ambiguity over whether the address argument to kvm_read() or kvm_write() is supposed to be a kernel address or a user address is even worse for 64-bit applications and kernel. All applications using libkvm that are still using kvm_read() and kvm_write() should transition to use the appropriate kvm_read(), kvm_write(), kvm_uread() and kvm_uwrite() routines. (These routines were first made available in Solaris 2.5.)

Applications that read /dev/kmem or /dev/mem directly can still run, though any attempt they make to interpret data they read from those devices might be wrong; data structure offsets and sizes are almost certainly different between 32-bit and 64-bit kernels.

`libkstat`

The sizes of many kernel statistics are completely independent of whether the kernel is a 64-bit or 32-bit program. The data types exported by named kstats (see kstat(3KSTAT)) are self-describing, and export signed or unsigned, 32-bit or 64-bit counter data, appropriately tagged. Thus, applications using libkstat need not be made into 64-bit applications to work successfully with the 64-bit kernel.

Note - If you are modifying a device driver that creates and maintains named kstats, you should try to keep the size of the statistics you export invariant between 32-bit and 64-bit kernels by using the fixed-width statistic types.

Changes to `stdio`

In the 64-bit environment, the stdio facility has been extended to allow more than 256 streams to be open simultaneously. The 32-bit stdio facility continues to have a 256 streams limit.

64-bit applications should not rely on having access to the members of the FILE data structure. Attempts to access private implementation-specific structure members directly can result in compilation errors. Existing 32-bit applications are unaffected by this change, but any direct usage of these structure members should be removed from all code.

The FILE structure has a long history, and a few applications have looked inside the structure to glean additional information about the state of the stream. Because the 64-bit version of the structure is now opaque, a new family of routines has been added to both 32-bit libc and 64-bit libc to allow the same state to be examined without depending on implementation internals. See, for example, __fbufsize(3C).

Performance Issues

The following sections discuss advantages and disadvantages of 64-bit performance.

64-bit Application Advantages

Arithmetic and logical operations on 64-bit quantities are more efficient.
Operations use full-register widths, the full-register set, and new instructions.
Parameter passing of 64-bit quantities is more efficient.
Parameter passing of small data structures and floating point quantities is more efficient.

64-bit Application Disadvantages

64-bit applications require more stack space to hold the larger registers.
Applications have a bigger cache footprint from larger pointers.
64-bit applications do not run on 32-bit platforms.

System Call Issues

The following sections discuss system call issues.

What Does `EOVERFLOW` Mean?

The EOVERFLOW return value is returned from a system call whenever one or more fields of the data structure used to pass information out of the kernel is too small to hold the value.

A number of 32-bit system calls now return EOVERFLOW when faced with large objects on the 64-bit kernel. While this was already true when dealing with large files, the fact that daddr_t, dev_t, time_t, and its derivative types struct timeval and timespec_t now contain 64-bit quantities might allow more EOVERFLOW return values to be observed by 32-bit applications.


	Interprocess Communication The following interprocess communication (IPC) primitives continue to work between 64-bit and 32-bit processes: The System V IPC primitives, such as `shmop`(2), `semop`(2), `msgsnd`(2) `mmap`(2) on shared files `pipe`(2) between processes `door_call`(3DOOR) between processes `rpc`(3NSL) between processes on the same or different machines using the external data representation described in `xdr`(3NSL) Although all these primitives allow interprocess communication between 32-bit and 64-bit processes, you might need to take explicit steps to ensure that data being exchanged between processes is correctly interpreted by all of them. For example, two processes sharing data described by a C data structure containing variables of type long cannot do so without understanding that a 32-bit process views this variable as a 4-byte quantity, while a 64-bit process views this variable as an 8-byte quantity. One way to handle this difference is to ensure that the data has exactly the same size and meaning in both processes. Build the data structures using fixed-width types, such as int32_t and int64_t. A family of derived types that mirrors the system derived types is available in `<sys/types32.h>`. These types possess the same sign and sizes as the fundamental types of the 32-bit system but are defined in such a way that the sizes are invariant between the ILP32 and LP64 compilation environments. Sharing pointers between 32-bit and 64-bit processes is substantially more difficult. Obviously, pointer sizes are different, but more importantly, while there is a 64-bit integer quantity (long long) in existing C usage, a 64-bit pointer has no equivalent in a 32-bit environment. In order for a 64-bit process to share data with a 32-bit process, the 32-bit process can only "see" up to 4 gigabytes of that shared data at a time. The XDR routine `xdr_long`(3NSL) might seem to be a problem; however, it is still handled as a 32-bit quantity over the wire to be compatible with existing protocols. If the 64-bit version of the routine is asked to encode a long value that does not fit into a 32-bit quantity, the encode operation fails. ELF and System Generation Tools 64-bit binaries are stored in files in ELF64 format, which is a direct analog of the ELF32 format, except that most fields have grown to accommodate full 64-bit applications. ELF64 files can be read using `elf`(3ELF) APIs; for example, `elf_getarhdr`(3ELF). Both 32-bit and 64-bit versions of the ELF library,`elf`(3ELF) , support both ELF32 and ELF64 formats and their corresponding APIs. This allows applications to build, read, or modify both file formats from either a 32-bit or a 64-bit system (though a 64-bit system is still required to execute a 64-bit program). In addition, Solaris provides a set of GELF (Generic ELF) interfaces that allow the programmer to manipulate both formats using a single, common API. See `elf`(3ELF). All of the system ELF utilities, including `ar`(1), `nm`(1), `ld`(1) and `dump`(1), have been updated to accept both ELF formats. `/proc` The `/proc` interfaces are available to both 32-bit and 64-bit applications. 32-bit applications can examine and control the state of other 32-bit applications. Thus, an existing 32-bit debugger can be used to debug a 32-bit application. 64-bit applications can examine and control 32-bit or 64-bit applications. However, 32-bit applications are unable to control 64-bit applications, because the 32-bit APIs do not allow the full state of 64-bit processes to be described. Thus, a 64-bit debugger is required to debug a 64-bit application. `libkvm` and `/dev/ksyms` The 64-bit version of the Solaris system is implemented using a 64-bit kernel. Applications that examine or modify the contents of the kernel directly must be converted to 64-bit applications and linked with the 64-bit version of `sysinfo`(2). Before doing this conversion and cleanup work, you should examine why the application needs to look directly at kernel data structures in the first place. It is possible that in the time since the program was first ported or created, additional interfaces have been made available on the Solaris platform, to extract the needed data with system calls. See `sysinfo`(2), `kstat`(3KSTAT), `sysconf`(3C), and `proc`(4) as the most common alternative APIs. If these interfaces can be used instead of `kvm_open`(3KVM), use them and leave the application as 32-bit for maximum portability. As a further benefit, most of these APIs are probably faster and might not require the same security privileges needed to access kernel memory. The 32-bit version of `libkvm` returns a failure from any attempt to use `kvm_open`(3KVM) on a 64-bit kernel or crash dump. Similarly, the 64-bit version of `libkvm` returns failure from any attempt to use `kvm_open`(3KVM) on a 32-bit kernel crash dump. Because the kernel is a 64-bit program, applications that open `/dev/ksyms` to examine the kernel symbol table directly need to be enhanced to understand ELF64 format. The ambiguity over whether the address argument to `kvm_read()` or `kvm_write()` is supposed to be a kernel address or a user address is even worse for 64-bit applications and kernel. All applications using `libkvm` that are still using `kvm_read()` and `kvm_write()` should transition to use the appropriate `kvm_read()`, `kvm_write()`, `kvm_uread()` and `kvm_uwrite()` routines. (These routines were first made available in Solaris 2.5.) Applications that read `/dev/kmem` or `/dev/mem` directly can still run, though any attempt they make to interpret data they read from those devices might be wrong; data structure offsets and sizes are almost certainly different between 32-bit and 64-bit kernels. `libkstat` The sizes of many kernel statistics are completely independent of whether the kernel is a 64-bit or 32-bit program. The data types exported by named `kstats` (see `kstat`(3KSTAT)) are self-describing, and export signed or unsigned, 32-bit or 64-bit counter data, appropriately tagged. Thus, applications using `libkstat` need not be made into 64-bit applications to work successfully with the 64-bit kernel. Note - If you are modifying a device driver that creates and maintains named `kstats`, you should try to keep the size of the statistics you export invariant between 32-bit and 64-bit kernels by using the fixed-width statistic types. Changes to `stdio` In the 64-bit environment, the `stdio` facility has been extended to allow more than 256 streams to be open simultaneously. The 32-bit `stdio` facility continues to have a 256 streams limit. 64-bit applications should not rely on having access to the members of the FILE data structure. Attempts to access private implementation-specific structure members directly can result in compilation errors. Existing 32-bit applications are unaffected by this change, but any direct usage of these structure members should be removed from all code. The FILE structure has a long history, and a few applications have looked inside the structure to glean additional information about the state of the stream. Because the 64-bit version of the structure is now opaque, a new family of routines has been added to both 32-bit `libc` and 64-bit `libc` to allow the same state to be examined without depending on implementation internals. See, for example, `__fbufsize`(3C). Performance Issues The following sections discuss advantages and disadvantages of 64-bit performance. 64-bit Application Advantages Arithmetic and logical operations on 64-bit quantities are more efficient. Operations use full-register widths, the full-register set, and new instructions. Parameter passing of 64-bit quantities is more efficient. Parameter passing of small data structures and floating point quantities is more efficient. 64-bit Application Disadvantages 64-bit applications require more stack space to hold the larger registers. Applications have a bigger cache footprint from larger pointers. 64-bit applications do not run on 32-bit platforms. System Call Issues The following sections discuss system call issues. What Does `EOVERFLOW` Mean? The `EOVERFLOW` return value is returned from a system call whenever one or more fields of the data structure used to pass information out of the kernel is too small to hold the value. A number of 32-bit system calls now return `EOVERFLOW` when faced with large objects on the 64-bit kernel. While this was already true when dealing with large files, the fact that daddr_t, dev_t, time_t, and its derivative types struct timeval and timespec_t now contain 64-bit quantities might allow more `EOVERFLOW` return values to be observed by 32-bit applications.

Interprocess Communication

ELF and System Generation Tools

/proc

libkvm and /dev/ksyms

libkstat

Changes to stdio