IPsec: is it possible to encrypt transit traffic in transport mode?

[Lev Serebryakov]

Hello Freebsd-net,

I have two router like this:

[NET 10.1.0.0/24] <-> (10.1.0.1 HOST A 10.2.0.1)
<->
(10.2.0.2 HOST B 10.10.10.1) <-> [NET 10.10.10.0/24)

Both HOST A and HOST B tun FreeBSD, both are routers (forwrading is
enabled), host A has "route -net 10.10.10.0/24 10.2.0.2" and host B has
"route -net 10.1.0.0/24 10.2.0.1".

I could pass traffic from 10.1.0.0/24 to 10.10.10.0/24 and back without
problems.

Now, I want to encrypt this transit traffic between routers (!) but
without creation of tunnel.

Is it possible to encrypt this traffic with IPsec in *transport* mode?
I've tried to create SAs for 10.2.0.1 and 10.2.0.2 and SPDs for 10.1.0.0/24
and 10.10.10.0/24 on A and B (not on endpoint devices) but looks like it
doesn't work, traffic stops. It is not as encrypted traffic is sent but
dropped on other end, no, interfaces between Host A and Host B becomes
silent according to "tcpdump" and all forwarded/dropped/error counters in
"nestat -s" don't change anymore, only "input packets" in "netstat -s -p ip"
is still counting.

My SAs and SPDs looks like this (for UDP only, for tests):

Host A:

add 10.2.0.1 10.2.0.2 esp 0x10001 -m transport -E null "";
add 10.2.0.2 10.2.0.1 esp 0x10001 -m transport -E null "";

spdadd 10.1.0.0/24 10.10.10.0/24 udp -P out ipsec esp/transport//require;
spdadd 10.10.10.0/24 10.1.0.0/24 udp -P in ipsec esp/transport//require;

Host B:

add 10.2.0.1 10.2.0.2 esp 0x10001 -m transport -E null "";
add 10.2.0.2 10.2.0.1 esp 0x10001 -m transport -E null "";

spdadd 10.10.10.0/24 10.1.0.0/24 udp -P out ipsec esp/transport//require;
spdadd 10.1.0.0/24 10.10.10.0/24 udp -P in ipsec esp/transport//require;

[Eugene Grosbein]

Post by Lev Serebryakov
[NET 10.1.0.0/24] <-> (10.1.0.1 HOST A 10.2.0.1)
<->
(10.2.0.2 HOST B 10.10.10.1) <-> [NET 10.10.10.0/24)
Both HOST A and HOST B tun FreeBSD, both are routers (forwrading is
enabled), host A has "route -net 10.10.10.0/24 10.2.0.2" and host B has
"route -net 10.1.0.0/24 10.2.0.1".
I could pass traffic from 10.1.0.0/24 to 10.10.10.0/24 and back without
problems.
Now, I want to encrypt this transit traffic between routers (!) but
without creation of tunnel.
Is it possible to encrypt this traffic with IPsec in *transport* mode?
I've tried to create SAs for 10.2.0.1 and 10.2.0.2 and SPDs for 10.1.0.0/24
and 10.10.10.0/24 on A and B (not on endpoint devices) but looks like it
doesn't work, traffic stops. It is not as encrypted traffic is sent but
dropped on other end, no, interfaces between Host A and Host B becomes
silent according to "tcpdump" and all forwarded/dropped/error counters in
"nestat -s" don't change anymore, only "input packets" in "netstat -s -p ip"
is still counting.
add 10.2.0.1 10.2.0.2 esp 0x10001 -m transport -E null "";
add 10.2.0.2 10.2.0.1 esp 0x10001 -m transport -E null "";
spdadd 10.1.0.0/24 10.10.10.0/24 udp -P out ipsec esp/transport//require;
spdadd 10.10.10.0/24 10.1.0.0/24 udp -P in ipsec esp/transport//require;
add 10.2.0.1 10.2.0.2 esp 0x10001 -m transport -E null "";
add 10.2.0.2 10.2.0.1 esp 0x10001 -m transport -E null "";
spdadd 10.10.10.0/24 10.1.0.0/24 udp -P out ipsec esp/transport//require;
spdadd 10.1.0.0/24 10.10.10.0/24 udp -P in ipsec esp/transport//require;

It is possible and it is the way I use extensively for long time since very old
FreeBSD versions having KAME IPSEC and it works with 11.2-STABLE, too.

You need to read setkey(8) manual page, section ALGORITHMS and make sure
you use proper sized keys or it won't work, though.

And example of transport mode IPSEC with low-powered device having on-board
Geode LX Security Block crypto accelerator with AES-128-CBC support:

add 1.1.1.1 2.2.2.2 esp 1081 -m transport -E rijndael-cbc "1234567890123456" -A hmac-md5 "0123456789123456";
add 2.2.2.2 1.1.1.1 esp 2081 -m transport -E rijndael-cbc "9876543210987654" -A hmac-md5 "6543219876543210";

spdadd 1.1.1.1/32 2.2.2.2/32 any -P out ipsec esp/transport//require;
spdadd 2.2.2.2/32 1.1.1.1/32 any -P in ipsec esp/transport//require;

You have to use bigger keys if you use another -A algorithm like sha*, each character counts for 8 bits.

[Andrey V. Elsukov]

Post by Eugene Grosbein

Post by Lev Serebryakov
Is it possible to encrypt this traffic with IPsec in *transport* mode?
I've tried to create SAs for 10.2.0.1 and 10.2.0.2 and SPDs for 10.1.0.0/24
and 10.10.10.0/24 on A and B (not on endpoint devices) but looks like it
doesn't work, traffic stops. It is not as encrypted traffic is sent but
dropped on other end, no, interfaces between Host A and Host B becomes
silent according to "tcpdump" and all forwarded/dropped/error counters in
"nestat -s" don't change anymore, only "input packets" in "netstat -s -p ip"
is still counting.

It is possible and it is the way I use extensively for long time since very old
FreeBSD versions having KAME IPSEC and it works with 11.2-STABLE, too.
You need to read setkey(8) manual page, section ALGORITHMS and make sure
you use proper sized keys or it won't work, though.
And example of transport mode IPSEC with low-powered device having on-board
add 1.1.1.1 2.2.2.2 esp 1081 -m transport -E rijndael-cbc "1234567890123456" -A hmac-md5 "0123456789123456";
add 2.2.2.2 1.1.1.1 esp 2081 -m transport -E rijndael-cbc "9876543210987654" -A hmac-md5 "6543219876543210";
spdadd 1.1.1.1/32 2.2.2.2/32 any -P out ipsec esp/transport//require;
spdadd 2.2.2.2/32 1.1.1.1/32 any -P in ipsec esp/transport//require;
You have to use bigger keys if you use another -A algorithm like sha*, each character counts for 8 bits.

There is one problem. IPsec won't handle inbound packets, that are not
destined to your IP address. Inbound packets are handled based on the
destination address, protocol and SPI value, so if ip_input() doesn't
decide that ESP packet is for your host, it will not invoke
IPSEC_INPUT() and encrypted packet will be routed as is.

[Eugene Grosbein]

Post by Andrey V. Elsukov
There is one problem. IPsec won't handle inbound packets, that are not
destined to your IP address. Inbound packets are handled based on the
destination address, protocol and SPI value, so if ip_input() doesn't
decide that ESP packet is for your host, it will not invoke
IPSEC_INPUT() and encrypted packet will be routed as is.

That's why I use gif tunnels for such packets :-)

[Lev Serebryakov]

Hello Eugene,

Post by Eugene Grosbein

Post by Lev Serebryakov
add 10.2.0.1 10.2.0.2 esp 0x10001 -m transport -E null "";
add 10.2.0.2 10.2.0.1 esp 0x10001 -m transport -E null "";
spdadd 10.1.0.0/24 10.10.10.0/24 udp -P out ipsec esp/transport//require;
spdadd 10.10.10.0/24 10.1.0.0/24 udp -P in ipsec esp/transport//require;
add 10.2.0.1 10.2.0.2 esp 0x10001 -m transport -E null "";
add 10.2.0.2 10.2.0.1 esp 0x10001 -m transport -E null "";
spdadd 10.10.10.0/24 10.1.0.0/24 udp -P out ipsec esp/transport//require;
spdadd 10.1.0.0/24 10.10.10.0/24 udp -P in ipsec esp/transport//require;

It is possible and it is the way I use extensively for long time since very old
FreeBSD versions having KAME IPSEC and it works with 11.2-STABLE, too.

Eugeny, please note, that your example have SA and SPDs with same
addresses. It works for me too. It doesn't work for me if SAs have addresses
of routers and SPDs have addresses of routed networks. And if SPDs have
routers' addresses, then routed traffic is not encrypted, only host-to-host
(router-to-router) are.

Post by Eugene Grosbein
You need to read setkey(8) manual page, section ALGORITHMS and make sure
you use proper sized keys or it won't work, though.

Yes, I know that.

Post by Eugene Grosbein
And example of transport mode IPSEC with low-powered device having on-board
add 1.1.1.1 2.2.2.2 esp 1081 -m transport -E rijndael-cbc
"1234567890123456" -A hmac-md5 "0123456789123456";
add 2.2.2.2 1.1.1.1 esp 2081 -m transport -E rijndael-cbc
"9876543210987654" -A hmac-md5 "6543219876543210";
spdadd 1.1.1.1/32 2.2.2.2/32 any -P out ipsec esp/transport//require;
spdadd 2.2.2.2/32 1.1.1.1/32 any -P in ipsec esp/transport//require;
You have to use bigger keys if you use another -A algorithm like sha*, each character counts for 8 bits.

Unfortunately, this example shows not what I want to achieve.

[Eugene Grosbein]

Post by Lev Serebryakov

Post by Eugene Grosbein
It is possible and it is the way I use extensively for long time since very old
FreeBSD versions having KAME IPSEC and it works with 11.2-STABLE, too.

Eugeny, please note, that your example have SA and SPDs with same
addresses. It works for me too. It doesn't work for me if SAs have addresses
of routers and SPDs have addresses of routed networks. And if SPDs have
routers' addresses, then routed traffic is not encrypted, only host-to-host
(router-to-router) are.

Just add gif(4) to the picture.

[Lev Serebryakov]

Hello Eugene,

Post by Eugene Grosbein

Post by Lev Serebryakov

Post by Eugene Grosbein
It is possible and it is the way I use extensively for long time since very old
FreeBSD versions having KAME IPSEC and it works with 11.2-STABLE, too.

Eugeny, please note, that your example have SA and SPDs with same
addresses. It works for me too. It doesn't work for me if SAs have addresses
of routers and SPDs have addresses of routed networks. And if SPDs have
routers' addresses, then routed traffic is not encrypted, only host-to-host
(router-to-router) are.

Just add gif(4) to the picture.

I'm benchmarking different possible "native" VPN configurations and I have
gif(4) and gre(4) with and without IPsec in my battery. I have tunnel mode
IPsec too. Problem with gif(4) and gre(4) that hey are tremendously
expensive, and could be more expensive than IPsec itself on CPUs with AES-NI.

So, this configuration impossible, I understand. Nothing to benchmark :-)

[Olivier Cochard-Labbé]

Post by Lev Serebryakov
I'm benchmarking different possible "native" VPN configurations and I have
gif(4) and gre(4) with and without IPsec in my battery. I have tunnel mode
IPsec too. Problem with gif(4) and gre(4) that hey are tremendously
expensive, and could be more expensive than IPsec itself on CPUs with AES-NI.
So, this configuration impossible, I understand. Nothing to benchmark :-)

And what about using IPSec VTI (virtual tunneling interface) mode:
if_ipsec(4) ?

[Lev Serebryakov]

Hello Olivier,

 I'm benchmarking different possible "native" VPN configurations and I have
 gif(4) and gre(4) with and without IPsec in my battery. I have tunnel mode
 IPsec too. Problem with gif(4) and gre(4) that hey are tremendously
 expensive, and could be more expensive than IPsec itself on CPUs with AES-NI.
 So, this configuration impossible, I understand. Nothing to benchmark :-)

And what about using IPSec VTI (virtual tunneling interface) mode:  if_ipsec(4)

And this one too. It gives slightly more PPS than "setkey-based" tunnel
mode, which is surprise for me.

--
Best regards,
Lev mailto:***@FreeBSD.org

[Andrey V. Elsukov]

Post by Lev Serebryakov
Hello Olivier,

 I'm benchmarking different possible "native" VPN configurations and I have
 gif(4) and gre(4) with and without IPsec in my battery. I have tunnel mode
 IPsec too. Problem with gif(4) and gre(4) that hey are tremendously
 expensive, and could be more expensive than IPsec itself on CPUs with AES-NI.
 So, this configuration impossible, I understand. Nothing to benchmark :-)

And what about using IPSec VTI (virtual tunneling interface) mode:  if_ipsec(4)

And this one too. It gives slightly more PPS than "setkey-based" tunnel
mode, which is surprise for me.

If your goal is increasing of PPS throughput, there are several ways to
achieve it. For example, it is possible to make direct output from IPsec
code, I mean make a route lookup and call if_output() directly from
ipsec_process_done(). This removes many checks that does ip_output() and
also extra call to pfil(9).
Another idea is implementing some ipfw_ipsec(4) module, that can take
packets and do IPsec processing. Then this module can be attached to
Ethernet pfil hook and together with first idea, I think this can give a
measurable improvement of PPS rate.