Configuring Connections with setkey

To configure your IPsec connections, you can use the setkey tool. This tool contains several instructions for managing rules in the IPsec policy and security databases. You use the add instruction to add a security association to the security database (SAD), and the spdadd instruction to add a policy to the policy database (SPD). The ah term designates that the instruction is being applied to the authentication header (AH), and esp indicates the encryption to be implemented by the encryption security payload (ESP). To implement setkey operations, it is best to use a script invoking setkey with the -f option and listing the setkey instructions. The following example creates a simple script to add authentication and encryption instructions for a particular connection, as well as create a security policy for it:

#!/sbin/setkey -f
add 192.168.0.2 192.168.0.5 ah 15700 -A hmac-md5 "secret key";
add 192.168.0.2 192.168.0.5  esp 15701 -E 3des-cbc "secret key ";
spdadd 192.168.0.2 192.168.0.5  any -P out ipsec
   esp/transport//require
   ah/transport//require;

Security Associations: SA

You use security associations to indicate that you want the authentication header (AH) and encryption payload (ESP) encrypted. A particular connection, such as that between two hosts, can have its authentication headers encrypted using specified encryption methods and designated secret keys. The same can be done for the encryption payload, the main content of transmissions. A secret key can be determined manually or automatically using key exchanges. The following example specifies that for the connection between 192.168.0.2 and 192.168.0.5, the hmac-md5 authentication method and a secret key (here designated by the placeholder secret key) will be used for the authentication header ah.

add 192.168.0.2 192.168.0.5 ah 15700 -A hmac-md5 "secret key";

The security association for the encryption payload uses the 3des-cbc encryption method and a different secret key.

add 192.168.0.2 192.168.0.5 esp 15701 -E 3des-cbc "secret key";

Each instruction is identified with a security parameter index (SPI), in this case, 15700 and 15701. In fact, identical instructions with different SPIs are considered different instructions.

Bear in mind that the security associations only specify possible encryption procedures. They do not implement them. For that, you need to set security policies.

Security Policy: SP

A security policy will implement an IPsec security procedure for a connection. You can designate a host or port connection. Once a policy is set for a connection, the kernel will determine what security associations to apply, using the SAD database. A security policy is added with the spdadd instruction. Either encryption, authentication, or both can be required.

The following example will encrypt and authenticate transmissions between hosts 192.168.0.2 and 192.168.0.5. Any outgoing transmissions between these hosts will be both encrypted and authenticated.

spdadd 192.168.0.2 192.168.0.5 any -P out ipsec esp/transport//require
ah/transport/require;

In the spdadd instruction, you will need to specify the connection, such as one between two hosts or two networks. For two hosts, you would use their IP addresses, in this example, 192.168.0.2 and 192.168.0.5. You then specify the kind of packet and its direction, in this case any outgoing packet, any -P out. Then you can specify the ipsec directives for either the ESP or AH protocols, or both. For each entry, you specify the mode (transport or tunnel), the hosts involved (this can be different in tunnel mode), and the policy for the encryption, usually require. The following example shows that the ESP protocol will use the transport mode for connections between 192.168.02 and 192.168.0.5, and it will be required.

esp/transport/192.168.02-192.168.0.5/require

You can leave out the host information if it is the same, as in the prior example.

esp/transport//require

Receiving Hosts

For a host to receive an encrypted IPsec transmission, it must have corresponding security association instructions in its own SAD database that tell it how to authenticate and decrypt the received instructions. The security association instructions would mirror those of the sender's instructions, using the same encryption method, secret keys, and security indexes. A corresponding policy, though, is not required.

#!/sbin/setkey -f
add 192.168.0.2 192.168.0.5 ah 15700 -A hmac-md5 "secret key";
add 192.168.0.2 192.168.0.5 esp 15701 -E 3des-cbc "secret key";

Receiving hosts may want to set up policies to screen incoming packets on secure connections, discarding those not encrypted. The following policy will accept only incoming encrypted and authenticated IPsec transmissions from 192.168.0.2.

spdadd 192.168.0.2 192.168.0.5 any -P in ipsec esp/transport//require
ah/transport//require;

Two-way Transmissions

The previous example set up a secure connection between two hosts going only one way, from 192.168.0.2 to 192.168.0.5, not the other way, from 192.168.0.5 to 192.168.0.2. To implement two-way secure transmissions between two hosts, both need to be configured as the sender and the receiver, with corresponding security associations to match. The following scripts are based on common examples of a simple two-way IPsec connection between two hosts. They set up a secure two-way IPsec connection between hosts 192.168.0.2 and 192.168.0.5. Corresponding incoming policies are also included, but not required.

First is the configuration for host 192.168.0.2:

#!/sbin/setkey -f
add 192.168.0.2 192.168.0.5 ah 15700 -A hmac-md5 "secret key";
add 192.168.0.5 192.168.0.2 ah 24500 -A hmac-md5 "secret key";
   
add 192.168.0.2 192.168.0.5 esp 15701 -E 3des-cbc "secret key";
add 192.168.0.5 192.168.0.2 esp 24501 -E 3des-cbc "secret key";
   
spdadd 192.168.0.2 192.168.0.5 any -P out ipsec esp/transport//require
ah/transport//require;
spdadd 192.168.0.5 192.168.0.2 any -P in ipsec esp/transport//require
ah/transport//require;

The corresponding host, 192.168.0.5, uses the same instructions but with the IP connections reversed. Notice that the security indexes for instructions for the sender and receiver at each end correspond.

#!/sbin/setkey -f
add 192.168.0.5 192.168.0.2 ah 15700 -A hmac-md5 "secret key";
add 192.168.0.2 192.168.0.5 ah 24500 -A hmac-md5 "secret key";
   
add 192.168.0.5 192.168.0.2 esp 15701 -E 3des-cbc "secret key";
add 192.168.0.2 192.168.0.5 esp 24501 -E 3des-cbc "secret key";
   
spdadd 192.168.0.5 192.168.0.2 any -P out ipsec esp/transport//require
ah/transport//require;
spdadd 192.168.0.2 192.168.0.5 any -P in ipsec esp/transport//require
ah/transport//require;