Chapter 3

Using Authentication Services (Tasks)

The first part of this chapter provides information about the Diffie-Hellman authentication mechanism that can be used with Secure RPC. The second part covers the Pluggable Authentication Module (PAM) framework. PAM provides a method to "plug-in" authentication services and provides support for multiple authentication services.

This is a list of the step-by-step instructions in this chapter.

Overview of Secure RPC

Secure RPC is an authentication method that authenticates both the host and the user who is making a request for a service. Secure RPC uses the Diffie-Hellman authentication mechanism. This authentication mechanism uses DES encryption. Applications that use Secure RPC include NFS and the NIS+ name service.

NFS Services and Secure RPC

NFS enables several hosts to share files over the network. Under the NFS service, a server holds the data and resources for several clients. The clients have access to the file systems that the server shares with the clients. Users who are logged in to the client machines can access the file systems by mounting them from the server. To the user on the client machine, it appears as if the files are local to the client. One of the most common uses of NFS is to allow systems to be installed in offices, while keeping all user files in a central location. Some features of the NFS service, such as the mount -nosuid option, can be used to prohibit the opening of devices, as well as file systems, by unauthorized users.

The NFS service uses Secure RPC to authenticate users who make requests over the network. This process is known as Secure NFS. The authentication mechanism, AUTH_DH, uses DES encryption with Diffie-Hellman authentication to ensure authorized access. The AUTH_DH mechanism has also been called AUTH_DES.

How to set up and administer Secure NFS in "Administering the Secure NFS System" in System Administration Guide: Resource Management and Network Services
How to set up the NIS+ tables and enter names in the cred table in System Administration Guide: Naming and Directory Services (DNS, NIS, and LDAP)
An outline of the steps that are involved in RPC authentication in "Implementation of Diffie-Hellman Authentication"

DES Encryption

The Data Encryption Standard (DES) encryption functions use a 56-bit key to encrypt data. If two credential users (or principals) know the same DES key, they can communicate in private, by using the key to encipher and decipher text. DES is a relatively fast encryption mechanism. A DES chip makes the encryption even faster. However, if the chip is not present, a software implementation is substituted.

The risk of using just the DES key is that an intruder can collect enough cipher-text messages that were encrypted with the same key to be able to discover the key and decipher the messages. For this reason, security systems such as Secure NFS change the keys frequently.

Kerberos Authentication

Kerberos is an authentication system that was developed at MIT. Encryption in Kerberos is based on DES. Kerberos V4 support is no longer supplied as part of Secure RPC. However, a client-side implementation of Kerberos V5, which uses RPCSEC_GSS, is included with this release. For more information see Chapter 6, Introduction to SEAM.

Diffie-Hellman Authentication

The Diffie-Hellman (DH) method of authenticating a user is nontrivial for an intruder to crack. The client and the server each have their own private key (sometimes called a secret key) which they use together with the public key to devise a common key. They use the common key to communicate with each other, by using an agreed-upon encryption/decryption function (such as DES). This method was identified as DES authentication in previous Solaris releases.

Authentication is based on the ability of the sending system to use the common key to encrypt the current time, which the receiving system can decrypt and check against its current time. Make sure that you synchronize the time on the client and the server.

The public keys and private keys are stored in an NIS or NIS+ database. NIS stores the keys in the publickey map. NIS+ stores the keys in the cred table. These files contain the public key and the private key for all potential users.

The system administrator is responsible for setting up NIS maps or NIS+ tables, and generating a public key and a private key for each user. The private key is stored in encrypted form with the user's password. This process makes the private key known only to the user.

Implementation of Diffie-Hellman Authentication

This section describes the series of transactions in a client-server session that use DH authorization (AUTH_DH).

Generating the Public Keys and Secret Keys

Sometime prior to a transaction, the administrator runs either the newkey or nisaddcred command to generate a public key and a secret key. Each user has a unique public key and secret key. The public key is stored in a public database. The secret key is stored in encrypted form in the same database. To change the key pair, use the chkey command.

Running the `keylogin` Command

Normally, the login password is identical to the secure RPC password. In this case, the keylogin command is not required. However, if the passwords are different, the users have to log in, and then run a keylogin command explicitly.

The keylogin command prompts the user for a secure RPC password and uses the password to decrypt the secret key. The keylogin command then passes the decrypted secret key to a program called the keyserver. The keyserver is an RPC service with a local instance on every computer. The keyserver saves the decrypted secret key and waits for the user to initiate a secure RPC transaction with a server.

If both the login password and the RPC password are the same, the login process passes the secret key to the keyserver. If the passwords are required to be different and the user must always run the keylogin command, then the keylogin command can be included in the user's environment configuration file, such as the ~/.login, ~/.cshrc, or ~/.profile file. Then, the keylogin command runs automatically whenever the user logs in.

Generating the Conversation Key

When the user initiates a transaction with a server, the following occurs:

The keyserver randomly generates a conversation key.
The kernel uses the conversation key to encrypt the client's time stamp (among other things).
The keyserver looks up the server's public key in the public key database (see the publickey(4) man page).
The keyserver uses the client's secret key and the server's public key to create a common key.
The keyserver encrypts the conversation key with the common key.

First Contact With the Server

The transmission, which includes the encrypted time stamp and the encrypted conversation key, is then sent to the server. The transmission includes a credential and a verifier. The credential contains three components:

The client's net name
The conversation key, which is encrypted with the common key
A "window," which is encrypted with the conversation key

The window is the difference in time that the client says should be allowed between the server's clock and the client's time stamp. If the difference between the server's clock and the time stamp is greater than the window, the server rejects the client's request. Under normal circumstances, this rejection will not happen, because the client first synchronizes with the server before starting the RPC session.

The client's verifier contains the following:

The encrypted time stamp
An encrypted verifier of the specified window, which is decremented by 1

The window verifier is needed in case somebody wants to impersonate a user and writes a program that, instead of filling in the encrypted fields of the credential and verifier, just stuffs in random bits. The server will decrypt the conversation key into some random key, and use it to try to decrypt the window and the time stamp. The result will be random numbers. After a few thousand trials, however, there is a good chance that the random window/time stamp pair will pass the authentication system. The window verifier makes guessing the right credential much more difficult.


	Chapter 3 Using Authentication Services (Tasks) The first part of this chapter provides information about the Diffie-Hellman authentication mechanism that can be used with Secure RPC. The second part covers the Pluggable Authentication Module (PAM) framework. PAM provides a method to "plug-in" authentication services and provides support for multiple authentication services. This is a list of the step-by-step instructions in this chapter. "Overview of Secure RPC" "Administering Diffie-Hellman Authentication" "PAM (Overview)" "PAM (Tasks)" "PAM (Reference)" Overview of Secure RPC Secure RPC is an authentication method that authenticates both the host and the user who is making a request for a service. Secure RPC uses the Diffie-Hellman authentication mechanism. This authentication mechanism uses DES encryption. Applications that use Secure RPC include NFS and the NIS+ name service. NFS Services and Secure RPC NFS enables several hosts to share files over the network. Under the NFS service, a server holds the data and resources for several clients. The clients have access to the file systems that the server shares with the clients. Users who are logged in to the client machines can access the file systems by mounting them from the server. To the user on the client machine, it appears as if the files are local to the client. One of the most common uses of NFS is to allow systems to be installed in offices, while keeping all user files in a central location. Some features of the NFS service, such as the `mount` `-nosuid` option, can be used to prohibit the opening of devices, as well as file systems, by unauthorized users. The NFS service uses Secure RPC to authenticate users who make requests over the network. This process is known as Secure NFS. The authentication mechanism, `AUTH_DH`, uses DES encryption with Diffie-Hellman authentication to ensure authorized access. The `AUTH_DH` mechanism has also been called `AUTH_DES`. How to set up and administer Secure NFS in "Administering the Secure NFS System" in System Administration Guide: Resource Management and Network Services How to set up the NIS+ tables and enter names in the `cred` table in System Administration Guide: Naming and Directory Services (DNS, NIS, and LDAP) An outline of the steps that are involved in RPC authentication in "Implementation of Diffie-Hellman Authentication" DES Encryption The Data Encryption Standard (DES) encryption functions use a 56-bit key to encrypt data. If two credential users (or principals) know the same DES key, they can communicate in private, by using the key to encipher and decipher text. DES is a relatively fast encryption mechanism. A DES chip makes the encryption even faster. However, if the chip is not present, a software implementation is substituted. The risk of using just the DES key is that an intruder can collect enough cipher-text messages that were encrypted with the same key to be able to discover the key and decipher the messages. For this reason, security systems such as Secure NFS change the keys frequently. Kerberos Authentication Kerberos is an authentication system that was developed at MIT. Encryption in Kerberos is based on DES. Kerberos V4 support is no longer supplied as part of Secure RPC. However, a client-side implementation of Kerberos V5, which uses RPCSEC_GSS, is included with this release. For more information see Chapter 6, Introduction to SEAM. Diffie-Hellman Authentication The Diffie-Hellman (DH) method of authenticating a user is nontrivial for an intruder to crack. The client and the server each have their own private key (sometimes called a secret key) which they use together with the public key to devise a common key. They use the common key to communicate with each other, by using an agreed-upon encryption/decryption function (such as DES). This method was identified as DES authentication in previous Solaris releases. Authentication is based on the ability of the sending system to use the common key to encrypt the current time, which the receiving system can decrypt and check against its current time. Make sure that you synchronize the time on the client and the server. The public keys and private keys are stored in an NIS or NIS+ database. NIS stores the keys in the `publickey` map. NIS+ stores the keys in the `cred` table. These files contain the public key and the private key for all potential users. The system administrator is responsible for setting up NIS maps or NIS+ tables, and generating a public key and a private key for each user. The private key is stored in encrypted form with the user's password. This process makes the private key known only to the user. Implementation of Diffie-Hellman Authentication This section describes the series of transactions in a client-server session that use DH authorization (`AUTH_DH`). Generating the Public Keys and Secret Keys Sometime prior to a transaction, the administrator runs either the `newkey` or `nisaddcred` command to generate a public key and a secret key. Each user has a unique public key and secret key. The public key is stored in a public database. The secret key is stored in encrypted form in the same database. To change the key pair, use the `chkey` command. Running the `keylogin` Command Normally, the login password is identical to the secure RPC password. In this case, the `keylogin` command is not required. However, if the passwords are different, the users have to log in, and then run a `keylogin` command explicitly. The `keylogin` command prompts the user for a secure RPC password and uses the password to decrypt the secret key. The `keylogin` command then passes the decrypted secret key to a program called the keyserver. The keyserver is an RPC service with a local instance on every computer. The keyserver saves the decrypted secret key and waits for the user to initiate a secure RPC transaction with a server. If both the login password and the RPC password are the same, the login process passes the secret key to the keyserver. If the passwords are required to be different and the user must always run the `keylogin` command, then the `keylogin` command can be included in the user's environment configuration file, such as the `~/.login`, `~/.cshrc`, or `~/.profile` file. Then, the `keylogin` command runs automatically whenever the user logs in. Generating the Conversation Key When the user initiates a transaction with a server, the following occurs: The keyserver randomly generates a conversation key. The kernel uses the conversation key to encrypt the client's time stamp (among other things). The keyserver looks up the server's public key in the public key database (see the `publickey`(4) man page). The keyserver uses the client's secret key and the server's public key to create a common key. The keyserver encrypts the conversation key with the common key. First Contact With the Server The transmission, which includes the encrypted time stamp and the encrypted conversation key, is then sent to the server. The transmission includes a credential and a verifier. The credential contains three components: The client's net name The conversation key, which is encrypted with the common key A "window," which is encrypted with the conversation key The window is the difference in time that the client says should be allowed between the server's clock and the client's time stamp. If the difference between the server's clock and the time stamp is greater than the window, the server rejects the client's request. Under normal circumstances, this rejection will not happen, because the client first synchronizes with the server before starting the RPC session. The client's verifier contains the following: The encrypted time stamp An encrypted verifier of the specified window, which is decremented by 1 The window verifier is needed in case somebody wants to impersonate a user and writes a program that, instead of filling in the encrypted fields of the credential and verifier, just stuffs in random bits. The server will decrypt the conversation key into some random key, and use it to try to decrypt the window and the time stamp. The result will be random numbers. After a few thousand trials, however, there is a good chance that the random window/time stamp pair will pass the authentication system. The window verifier makes guessing the right credential much more difficult.