Discussion:
IPSEC in GENERIC
Robert Elz
2006-02-20 08:49:23 UTC
Permalink
Back last November[*], Greg Troxel made a change (for i386, the
same probably should have been, and by now, might have been,
made for other architectures as well) along the lines of ...

| Add (empty) INSTALL.local, parallel to GENERIC.local.
| Change INSTALL kernels to include INSTALL.local instead of GENERIC.local.

| With this change, one can add IPSEC in GENERIC.local and still build a
| release, rather than having install kernels not fit on floppies.

This message isn't about that change, which is fine - not even really
asking why that change was needed (I understand that).

What I'm asking is why that change happened to be needed?

Or in other words, WTF isn't IPSEC enabled in GENERIC already???

GENERIC kernels have all kinds of other much less useful cruft
enabled in them (exactly what varies from arch to arch), but ISO
is pretty common, as is NETATALK, and NS and CCITT (X.25) are there in
more than just a couple or ports. What is it about IPSEC that would cause
it to get left disabled? It has to be more useful than any of that
other stuff these days (even NETATALK which is probably the most used
of the obscure protocols).

Note: I am not suggesting disabling any of those others for the ports
that have them enabled, so no-one needs to reply and defend their
favourite toy.

But I would suggest that on all ports for which they work, IPSEC,
IPSEC_ESP, and (if it works, without side effects -- I've never needed
to use it) IPSEC_NAT_T, be enabled, and certainly that this be done for
those ports (which includes i386) which have all of NS, ISO, CCITT and
NETATALK enabled in GENERIC.

If there is some good reason why this should not be done (ie: a good reason
why NetBSD doesn't believe that IPSEC should be enabled in normal kernels)
then it should be documented (most probably in ipsec(4)) so others can
take that reason into account before enabling it for themselves.

If the patent issue is enough to exclude NAT_T, that's fine, but that
should not affect any of the others. The reasoning can't be related to
excluding encryption, or anything like that - there's tons of that in
other parts of NetBSD already. So, why not?

kre

[*] Last November's e-mail is where I'm up to in my NetBSD mailing list
reading - I had been meaning to ask this question for some time, Greg's
change (which most likely might never have happened had IPSEC been in
GENERIC all the time) just provided the inspiration to actually ask it.
Garrett D'Amore
2006-02-20 09:02:31 UTC
Permalink
New guy here, but it strikes me that the only likely reason not to have
it (assuming you have IP networking) is the resource burden.

Actually, the same can be said to be true of a *lot* of things. Like
the 8 gazillion PCI, PCMCIA, etc. device drivers.

There seems to be some work towards LKM to assist in this. I think more
work needs to be done there. Then questions like this one can go away,
because they will become moot.

LKM needs, I think, to reasonably support:

1) pretty much any target driver should be an LKM

2) it should be possible to have a really really tiny kernel, with
only an MFS in it containing LKMs for other devices needed to mount /
and /usr.

3) there needs to be better linkage between probing a bus and the
module info. See for example Solaris' driver_aliases file. (Of course
that means that NetBSD needs some standard way of naming devices based
on info available without the driver loaded -- e.g. PCI vendor/device ids.)

3) then a lot of the various configs can consolidate, because you
don't need custom kernels for most cases. with the right LKMs you can
meet any need, and still be as lean as you need to be.

4) the few other kernel compile time options need to try hard to
become run-time options ala sysctl or somesuch. (Or in the case of
hard limits, become dynamic tunables.)

So, are folks progressing on this? Is help needed? How can I help?

I think the idea of the MFS based /lkm is *key* for certain kinds of
applications (like embedded systems). Maybe that is a place to start.

I also know that someone is working on a real kernel loader. How is
that work progressing? Any help wanted?

-- Garrett
Post by Robert Elz
Back last November[*], Greg Troxel made a change (for i386, the
same probably should have been, and by now, might have been,
made for other architectures as well) along the lines of ...
| Add (empty) INSTALL.local, parallel to GENERIC.local.
| Change INSTALL kernels to include INSTALL.local instead of GENERIC.local.
| With this change, one can add IPSEC in GENERIC.local and still build a
| release, rather than having install kernels not fit on floppies.
This message isn't about that change, which is fine - not even really
asking why that change was needed (I understand that).
What I'm asking is why that change happened to be needed?
Or in other words, WTF isn't IPSEC enabled in GENERIC already???
GENERIC kernels have all kinds of other much less useful cruft
enabled in them (exactly what varies from arch to arch), but ISO
is pretty common, as is NETATALK, and NS and CCITT (X.25) are there in
more than just a couple or ports. What is it about IPSEC that would cause
it to get left disabled? It has to be more useful than any of that
other stuff these days (even NETATALK which is probably the most used
of the obscure protocols).
Note: I am not suggesting disabling any of those others for the ports
that have them enabled, so no-one needs to reply and defend their
favourite toy.
But I would suggest that on all ports for which they work, IPSEC,
IPSEC_ESP, and (if it works, without side effects -- I've never needed
to use it) IPSEC_NAT_T, be enabled, and certainly that this be done for
those ports (which includes i386) which have all of NS, ISO, CCITT and
NETATALK enabled in GENERIC.
If there is some good reason why this should not be done (ie: a good reason
why NetBSD doesn't believe that IPSEC should be enabled in normal kernels)
then it should be documented (most probably in ipsec(4)) so others can
take that reason into account before enabling it for themselves.
If the patent issue is enough to exclude NAT_T, that's fine, but that
should not affect any of the others. The reasoning can't be related to
excluding encryption, or anything like that - there's tons of that in
other parts of NetBSD already. So, why not?
kre
[*] Last November's e-mail is where I'm up to in my NetBSD mailing list
reading - I had been meaning to ask this question for some time, Greg's
change (which most likely might never have happened had IPSEC been in
GENERIC all the time) just provided the inspiration to actually ask it.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
matthew green
2006-02-20 22:05:28 UTC
Permalink
I also know that someone is working on a real kernel loader. How is
that work progressing? Any help wanted?


got an in-kernel linker for LKM's handy..? :-)
Simon Burge
2006-02-20 09:32:01 UTC
Permalink
Post by Robert Elz
Or in other words, WTF isn't IPSEC enabled in GENERIC already???
In the past, I've heard the argument that there's a (significant?)
impact on network performance if IPSEC is enabled in the kernel config
file, even if ipsec itself isn't used.

I don't have any data to back that up, but it shouldn't be too hard to
prove or bust that myth. And yes, a Mythbusters episode just finished
here :-)

Simon.
--
Simon Burge <***@wasabisystems.com>
NetBSD Support and Service: http://www.wasabisystems.com/
Robert Elz
2006-02-20 09:49:49 UTC
Permalink
Date: Mon, 20 Feb 2006 01:02:31 -0800
From: "Garrett D'Amore" <***@tadpole.com>
Message-ID: <***@tadpole.com>

| New guy here, but it strikes me that the only likely reason not to have
| it (assuming you have IP networking) is the resource burden.

With the other noise that is in GENERIC (for several ports) I kind of
doubt that's really the issue (it might be for some architectures,
which typically have a rather lean GENERIC, and can remain that way).

| There seems to be some work towards LKM to assist in this.

Personally, I'd prefer to see support for LKMs simply dropped...

Not that I expect that to happen, but for an OS distributed in source
form, the real demand for LKMs should be pretty small (really only for
those occasions when you cannot predict when you boot the kernel
just what support you might need - so perhaps I might consider an
ext2fs LKM, just in case someone brought me a floppy, or thumb drive,
with an ext2fs file system on it and asked me to read it - but even
there I think I'd just prefer userland file system access tools - as
in an application that reads the drive and extracts files as requested,
kind of like mtools).

Aside from that, LKMs just add overhead in order to achieve laziness.

Rather than LKMs, I'd much prefer to see (someone else's...) effort
spent on better kernel config tools.

| 3) there needs to be better linkage between probing a bus and the
| module info. See for example Solaris' driver_aliases file.

Vendors who ship binary OS releases have lots of very good reasons to
put a lot of work into LKM support. So I absolutely understand that
kind of thing for Solaris. Similarly, anyone who wants to ship binary
versions of NetBSD most likely should be spending a bunch of effort for
making the LKM system work really well. But that does not need to a
be a TNF objective (IMO).

kre

ps: none of this was in any way relevant to the original question of course.
j***@britannica.bec.de
2006-02-20 13:34:50 UTC
Permalink
Post by Robert Elz
Aside from that, LKMs just add overhead in order to achieve laziness.
Sorry, this is just bull shit. LKMs add *zero* overhead to the kernel.
You have the symbol table in the kernel, as soon as e.g. debugging is
wanted too. Everything else is self-contained.

Proper kernel module support dramatically reduces the need to have
alternative kernels. Look at FreeBSD: they have two basic configurations
-- UP and SMP. No ACPI subkernels needed, since ACPI is loaded as module
or not, depending on need.

Joerg
Robert Elz
2006-02-20 13:56:04 UTC
Permalink
Date: Mon, 20 Feb 2006 14:34:50 +0100
From: ***@britannica.bec.de
Message-ID: <***@britannica.bec.de>

I'd prefer to keep this topic to the original question, but ...

| Sorry, this is just bull shit. LKMs add *zero* overhead to the kernel.

Huh? You mean they don't need any code sitting around occupying
memory waiting for an LKM to come along to load? And a call of a
function in an LKM (from outside, or a different LKM) goes exactly
as fast as a call of the same function if it is statically linked?

You might not consider the costs significant enough to care about,
or you might consider the benefits worth the cost, but don't start
claiming that the cost is zero.

| You have the symbol table in the kernel, as soon as e.g. debugging is
| wanted too.

But most people don't want debugging in the kernel. It crashes,
I reboot it. If it happens often, I might send a PR (but it doesn't).

| Proper kernel module support dramatically reduces the need to have
| alternative kernels.

Sure, and if you're doing binary distributions that matters. But
most of the use of LKMs (with NetBSD) is by people with the sources
who just don't want to be bothered compiling the kernel they need.
`
| No ACPI subkernels needed, since ACPI is loaded as module
| or not, depending on need.

How many times does one of your systems change from needing ACPI
to not needing ACPI dynamically, while it is running, without
rebooting it?

kre
Garrett D'Amore
2006-02-20 14:41:59 UTC
Permalink
Post by Robert Elz
Date: Mon, 20 Feb 2006 14:34:50 +0100
I'd prefer to keep this topic to the original question, but ...
| Sorry, this is just bull shit. LKMs add *zero* overhead to the kernel.
Huh? You mean they don't need any code sitting around occupying
memory waiting for an LKM to come along to load? And a call of a
function in an LKM (from outside, or a different LKM) goes exactly
as fast as a call of the same function if it is statically linked?
You might not consider the costs significant enough to care about,
or you might consider the benefits worth the cost, but don't start
claiming that the cost is zero.
Wow, I didn't think this would be such a hot issue. LKMs add a small
amount of data (typically detach/unload related logic for drivers that
don't already have it, and a structure with a few new ops), LKM
infrastructure adds a tiny bit (rather like a smallish driver). The
performance of binary linked vs. static will vary from platform to
platform, but yes, I think generally the benefits outweigh the costs. IMO.
Post by Robert Elz
| You have the symbol table in the kernel, as soon as e.g. debugging is
| wanted too.
But most people don't want debugging in the kernel. It crashes,
I reboot it. If it happens often, I might send a PR (but it doesn't).
Yes. And folks who do want it will just rebuild a kernel, because they
are develociraptors. :-)
Post by Robert Elz
| Proper kernel module support dramatically reduces the need to have
| alternative kernels.
Sure, and if you're doing binary distributions that matters. But
most of the use of LKMs (with NetBSD) is by people with the sources
who just don't want to be bothered compiling the kernel they need.
Frankly, configuring a kernel is one of the heartaches that other OS'
long ago realized is something *most* people don't want to do. Most
folks just want to be able to load a kernel that will do what it will
need to. And when they buy e.g. a new pcmcia or usb device, they don't
want to have to recompile.

Highly technical users might have fun compiling the kernel, but as *one*
such user, I can tell you that having worked with Solaris' LKM interface
it is *very* nice. It even has a binary DDI, so I can ship *one* binary
module that works across a Solaris versions going back 5 years or more.
And I do just that *EVEN FOR MY OPEN SOURCE DRIVERS*. Why? Because my
users want to pull down a binary and install it, they don't (usually)
want to recompile the source, even though they can do that from the same
location as the binaries.

There are other reasons that these are attractive for e.g. embedded
system use. I can have support for a bunch of different devices in my
kernel, but only the ones *in use* would actually be occupying kernel
memory. In small memory systems this is attractive.
Post by Robert Elz
`
| No ACPI subkernels needed, since ACPI is loaded as module
| or not, depending on need.
How many times does one of your systems change from needing ACPI
to not needing ACPI dynamically, while it is running, without
rebooting it?
kre
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Steven M. Bellovin
2006-02-20 15:33:44 UTC
Permalink
Post by Garrett D'Amore
Highly technical users might have fun compiling the kernel, but as *one*
such user, I can tell you that having worked with Solaris' LKM interface
it is *very* nice. It even has a binary DDI, so I can ship *one* binary
module that works across a Solaris versions going back 5 years or more.
And I do just that *EVEN FOR MY OPEN SOURCE DRIVERS*. Why? Because my
users want to pull down a binary and install it, they don't (usually)
want to recompile the source, even though they can do that from the same
location as the binaries.
I'm with you. I've been hacking kernels (or the equivalent) since 1967.
I'd much rather have LKMs. If I were king, I'd decree that *all*
device drivers must be loadable, and *all* device drivers should be
dynamically loaded except for those that are necessary to boot the
system and read in new device drivers.

--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Garrett D'Amore
2006-02-20 15:44:28 UTC
Permalink
Post by Steven M. Bellovin
Post by Garrett D'Amore
Highly technical users might have fun compiling the kernel, but as *one*
such user, I can tell you that having worked with Solaris' LKM interface
it is *very* nice. It even has a binary DDI, so I can ship *one* binary
module that works across a Solaris versions going back 5 years or more.
And I do just that *EVEN FOR MY OPEN SOURCE DRIVERS*. Why? Because my
users want to pull down a binary and install it, they don't (usually)
want to recompile the source, even though they can do that from the same
location as the binaries.
I'm with you. I've been hacking kernels (or the equivalent) since 1967.
I'd much rather have LKMs. If I were king, I'd decree that *all*
device drivers must be loadable, and *all* device drivers should be
dynamically loaded except for those that are necessary to boot the
system and read in new device drivers.
--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
This seems so obvious to me, that I was quite surprised to find the
point contentious. Anyway, as someone who is interested in it, I've
raised the question about what is needed to make it happen -- with a
willingness to provide some assistance if needed.

Of course, if there are technical concerns about whether such an idea is
good, then I think maybe core@ should set a direction.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Thomas E. Spanjaard
2006-02-20 16:43:44 UTC
Permalink
Post by Garrett D'Amore
Post by Steven M. Bellovin
If I were king, I'd decree that *all*
device drivers must be loadable, and *all* device drivers should be
dynamically loaded except for those that are necessary to boot the
system and read in new device drivers.
This seems so obvious to me, that I was quite surprised to find the
point contentious. Anyway, as someone who is interested in it, I've
raised the question about what is needed to make it happen -- with a
willingness to provide some assistance if needed.
If this happens, I'd like to go the whole nine yards and put all this in
separate VM spaces. In other words, go the microkernel way, as that's
where this is headed. Any objections regarding performance loss, well,
look at L4. If the kernel itself (and VMM) go in *that* direction,
performance will be of no issue (or even more academic, put everything
in the same VM space, but limit the memory mappings processes/modules
have, and not have to worry about many heavy context switches). Of
course, this is a very long way to go, and I doubt if many like it.
Post by Garrett D'Amore
Of course, if there are technical concerns about whether such an idea is
I concur.

Cheers,
--
Thomas E. Spanjaard
***@netphreax.net
Steven M. Bellovin
2006-02-20 16:50:40 UTC
Permalink
Post by Thomas E. Spanjaard
If this happens, I'd like to go the whole nine yards and put all this in
separate VM spaces. In other words, go the microkernel way, as that's
where this is headed. Any objections regarding performance loss, well,
look at L4. If the kernel itself (and VMM) go in *that* direction,
performance will be of no issue (or even more academic, put everything
Post by Thomas E. Spanjaard
in the same VM space, but limit the memory mappings processes/modules
have, and not have to worry about many heavy context switches). Of
course, this is a very long way to go, and I doubt if many like it.
That's a much larger arechitectural change, and I think there are
serious performance questions. It would be nice, but I don't think it
will work well enough. That said, there's only one way to find out...
(Personally, I don't think there's that much benefit to NetBSD from
doing that for device drivers. File systems, on the other hand...)

--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Garrett D'Amore
2006-02-20 17:06:03 UTC
Permalink
Post by Thomas E. Spanjaard
Post by Thomas E. Spanjaard
If this happens, I'd like to go the whole nine yards and put all this in
separate VM spaces. In other words, go the microkernel way, as that's
where this is headed. Any objections regarding performance loss, well,
look at L4. If the kernel itself (and VMM) go in *that* direction,
performance will be of no issue (or even more academic, put everything
Post by Thomas E. Spanjaard
in the same VM space, but limit the memory mappings processes/modules
have, and not have to worry about many heavy context switches). Of
course, this is a very long way to go, and I doubt if many like it.
That's a much larger arechitectural change, and I think there are
serious performance questions. It would be nice, but I don't think it
will work well enough. That said, there's only one way to find out...
(Personally, I don't think there's that much benefit to NetBSD from
doing that for device drivers. File systems, on the other hand...)
--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
I hear a master's thesis in there somewhere. :-) Too bad I don't need one.

Anyway, I generally concur with Steven, I'd rather get to basic LKMs now
and not get bogged down by over-engineering the project. The real
question is: does core@ agree, or are there still serious objections to
LKMs in principle. (It seems like the performance question is mostly
answered, though some actual benchmarking would be useful. Can't do
that without actually implementing more LKMs though.)
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Thor Lancelot Simon
2006-02-20 17:14:02 UTC
Permalink
Post by Garrett D'Amore
Anyway, I generally concur with Steven, I'd rather get to basic LKMs now
and not get bogged down by over-engineering the project. The real
LKMs in principle. (It seems like the performance question is mostly
answered, though some actual benchmarking would be useful. Can't do
that without actually implementing more LKMs though.)
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.

I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.

Thor
Peter Seebach
2006-02-20 17:22:03 UTC
Permalink
Post by Thor Lancelot Simon
I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.
Having lost entire days to Linux's insistence on initrds and loading
everything dynamically, I have to say, I very much agree. I would probably
use LKMs on some systems, but the ability to make a single kernel which I know
has the devices it needs is too useful.

-s
Matt Thomas
2006-02-20 17:46:11 UTC
Permalink
Post by Thor Lancelot Simon
Post by Garrett D'Amore
Anyway, I generally concur with Steven, I'd rather get to basic LKMs now
and not get bogged down by over-engineering the project. The real
objections to
LKMs in principle. (It seems like the performance question is mostly
answered, though some actual benchmarking would be useful. Can't do
that without actually implementing more LKMs though.)
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.
I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.
The one mistake I think we have is that LKM's are special. IMO, there
should be nothing special about a LKM. It should use the exact same
services to make itself to the kernel as a static module built into a
monolithic kernel.

A module should have one entry point. However, the use of link sets
complicates things. We need a common routine to do link set processing
for the various types of link sets.
j***@britannica.bec.de
2006-02-20 18:08:51 UTC
Permalink
Post by Matt Thomas
The one mistake I think we have is that LKM's are special. IMO, there
should be nothing special about a LKM. It should use the exact same
services to make itself to the kernel as a static module built into a
monolithic kernel.
LKMs *are* special. You want to reduce the impact of kernel options for
modules as much as possible. When an option triggers inlining of
certain functions, that's (often) fine for a statically compiled kernel.
For a module, it introduces hidden dependencies on options and that's
often very problematic. It also means a lot of decisions have to be made
more careful, since the kernel now has an ABI which might be worse to
preserver.

Joerg
Eric Haszlakiewicz
2006-02-21 20:56:33 UTC
Permalink
Post by Matt Thomas
Post by Thor Lancelot Simon
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.
I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.
The one mistake I think we have is that LKM's are special. IMO, there
should be nothing special about a LKM. It should use the exact same
services to make itself to the kernel as a static module built into a
monolithic kernel.
Including config. I actually started working on this, but it then fell
to the same fate as those in-kernel loaders. I have some pieces somewhat
working if anyone else is intersted in it. The basic idea was that
a lot of the glue code that is currently written by hand can be generated
by config by using a fragment of a kernel config file.

eric
Quentin Garnier
2006-02-21 21:58:39 UTC
Permalink
Post by Eric Haszlakiewicz
Post by Matt Thomas
Post by Thor Lancelot Simon
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.
I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.
The one mistake I think we have is that LKM's are special. IMO, there
should be nothing special about a LKM. It should use the exact same
services to make itself to the kernel as a static module built into a
monolithic kernel.
Including config. I actually started working on this, but it then fell
to the same fate as those in-kernel loaders. I have some pieces somewhat
working if anyone else is intersted in it. The basic idea was that
a lot of the glue code that is currently written by hand can be generated
by config by using a fragment of a kernel config file.
When I last worked on that, it's been a while, I changed config(1) to
merge the concepts of attributes and modules, i.e. a module is a
collection of attributes. Ideally, a module should only be one
attribute, but some source files depend on several attributes through
the logical operators of config(1). The module name was built from
the names of the included attributes, all of this conditioned by the
user specifying what was to be compiled as a module in the options
selection part of the configuration file.

The idea behind that was to have a GENERIC were every possible element
was modular; the user would just have to link the pieces together to
obtain a monolithic kernel like the usual.

The config(1) changes were complete, but the rest was barely touched.
I might still have the code somewhere...
--
Quentin Garnier - ***@cubidou.net - ***@NetBSD.org
"When I find the controls, I'll go where I like, I'll know where I want
to be, but maybe for now I'll stay right here on a silent sea."
KT Tunstall, Silent Sea, Eye to the Telescope, 2004.
Garrett D'Amore
2006-02-20 18:28:42 UTC
Permalink
Post by Thor Lancelot Simon
Post by Garrett D'Amore
Anyway, I generally concur with Steven, I'd rather get to basic LKMs now
and not get bogged down by over-engineering the project. The real
LKMs in principle. (It seems like the performance question is mostly
answered, though some actual benchmarking would be useful. Can't do
that without actually implementing more LKMs though.)
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.
Anyone working on it now?
Post by Thor Lancelot Simon
I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.
Yes, for some things a monolithic kernel is still going to be needed.

We do need versioning, and binary stable APIs as well... we have the
technology...

-- Garrett
Post by Thor Lancelot Simon
Thor
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Thor Lancelot Simon
2006-02-20 18:32:04 UTC
Permalink
Post by Garrett D'Amore
Post by Thor Lancelot Simon
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.
Anyone working on it now?
Not that I know of.
--
Thor Lancelot Simon ***@rek.tjls.com

"We cannot usually in social life pursue a single value or a single moral
aim, untroubled by the need to compromise with others." - H.L.A. Hart
Garrett D'Amore
2006-02-20 18:35:33 UTC
Permalink
Post by Thor Lancelot Simon
Post by Garrett D'Amore
Post by Thor Lancelot Simon
What's really needed is an in-kernel loader. At least two developers
have stepped up in the past and claimed to be in the midst of writing
one of those, but nothing's ever come of it.
Anyone working on it now?
Not that I know of.
Let me look at it and get back later then. My plate is pretty full, but
I think this functionality would be so useful that I'm willing to stare
hard at it for a while and *maybe* take ownership of it.

This isn't an ownership claim, yet.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Allen Briggs
2006-02-20 21:10:16 UTC
Permalink
Post by Garrett D'Amore
Let me look at it and get back later then. My plate is pretty full, but
I think this functionality would be so useful that I'm willing to stare
hard at it for a while and *maybe* take ownership of it.
Thanks for taking a look at this, Garrett. As long as it's done
well and remains optional (but not second-class), I would really
like to see improvements here.
Post by Garrett D'Amore
This isn't an ownership claim, yet.
Understood.

-allen
--
Use NetBSD! http://www.NetBSD.org/
Jason Thorpe
2006-02-20 23:40:57 UTC
Permalink
Post by Thor Lancelot Simon
I also question whether, without versioning of modules and without
inter-library dependencies, this will actually ever be particularly
useful. Finally, I think it's important to retain the ability to
build a monolithic kernel for applications where the entire blob
must be verified -- *without* running a chain of dependencies and
hoping you got it right.
Mac OS X 10.4 has a particularly good model to follow in this regard.

-- thorpej
der Mouse
2006-02-20 17:23:09 UTC
Permalink
Post by Steven M. Bellovin
I'm with you. I've been hacking kernels (or the equivalent) since
1967. I'd much rather have LKMs. If I were king, I'd decree that
*all* device drivers must be loadable, and *all* device drivers
should be dynamically loaded except for those that are necessary to
boot the system and read in new device drivers.
If I were trying to build hardened systems, I'd be really glad you're
not king. One of the first things I do when building a
security-critical system is remove LKM capability. Securing one file
(which may not even be a normal file) on boot media is a significantly
easier task than securing a few dozen files in the running system's
filesystem.

/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Steven M. Bellovin
2006-02-20 17:34:22 UTC
Permalink
Post by der Mouse
Post by Steven M. Bellovin
I'm with you. I've been hacking kernels (or the equivalent) since
1967. I'd much rather have LKMs. If I were king, I'd decree that
*all* device drivers must be loadable, and *all* device drivers
should be dynamically loaded except for those that are necessary to
boot the system and read in new device drivers.
If I were trying to build hardened systems, I'd be really glad you're
not king. One of the first things I do when building a
security-critical system is remove LKM capability. Securing one file
(which may not even be a normal file) on boot media is a significantly
easier task than securing a few dozen files in the running system's
filesystem.
You're right that it needs to be possible to build static kernels. I'd
love a framework where the identical .o could be used either way.


--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Thor Lancelot Simon
2006-02-20 17:38:04 UTC
Permalink
Post by Steven M. Bellovin
You're right that it needs to be possible to build static kernels. I'd
love a framework where the identical .o could be used either way.
As someone who used to rely pretty heavily on static Linux kernels to
preserve his sanity while debugging, I'd like to say that if we're going
to do it the Linux way, I *really* would not love such a framework.

Linux implements symbol versioning on all symbols in a kernel with loadable
module support by mangling their names in a way that makes it _extremely_
difficult to see what's going on with the debugger, or with normal tools
for looking at object files. It would be nice to not make that mistake,
which can also make it very difficult to see why the kernel fails to link
at build time, if it does.
--
Thor Lancelot Simon ***@rek.tjls.com

"We cannot usually in social life pursue a single value or a single moral
aim, untroubled by the need to compromise with others." - H.L.A. Hart
Steven M. Bellovin
2006-02-20 17:57:23 UTC
Permalink
Post by Thor Lancelot Simon
Post by Steven M. Bellovin
You're right that it needs to be possible to build static kernels. I'd
love a framework where the identical .o could be used either way.
As someone who used to rely pretty heavily on static Linux kernels to
preserve his sanity while debugging, I'd like to say that if we're going
to do it the Linux way, I *really* would not love such a framework.
Linux implements symbol versioning on all symbols in a kernel with loadable
module support by mangling their names in a way that makes it _extremely_
difficult to see what's going on with the debugger, or with normal tools
for looking at object files. It would be nice to not make that mistake,
which can also make it very difficult to see why the kernel fails to link
at build time, if it does.
So let's do it properly....

--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Garrett D'Amore
2006-02-20 18:33:01 UTC
Permalink
Post by Thor Lancelot Simon
Post by Steven M. Bellovin
You're right that it needs to be possible to build static kernels. I'd
love a framework where the identical .o could be used either way.
As someone who used to rely pretty heavily on static Linux kernels to
preserve his sanity while debugging, I'd like to say that if we're going
to do it the Linux way, I *really* would not love such a framework.
Linux implements symbol versioning on all symbols in a kernel with loadable
module support by mangling their names in a way that makes it _extremely_
difficult to see what's going on with the debugger, or with normal tools
for looking at object files. It would be nice to not make that mistake,
which can also make it very difficult to see why the kernel fails to link
at build time, if it does.
I propose we *not* do it the Linux way. Linux binary modules are
brain-dead. Solaris is a *vastly* superior example to follow.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Garrett D'Amore
2006-02-20 18:32:10 UTC
Permalink
Post by Steven M. Bellovin
Post by der Mouse
Post by Steven M. Bellovin
I'm with you. I've been hacking kernels (or the equivalent) since
1967. I'd much rather have LKMs. If I were king, I'd decree that
*all* device drivers must be loadable, and *all* device drivers
should be dynamically loaded except for those that are necessary to
boot the system and read in new device drivers.
If I were trying to build hardened systems, I'd be really glad you're
not king. One of the first things I do when building a
security-critical system is remove LKM capability. Securing one file
(which may not even be a normal file) on boot media is a significantly
easier task than securing a few dozen files in the running system's
filesystem.
You're right that it needs to be possible to build static kernels. I'd
love a framework where the identical .o could be used either way.
--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Yes, please.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Bill Studenmund
2006-02-21 19:31:07 UTC
Permalink
Post by der Mouse
Post by Steven M. Bellovin
I'm with you. I've been hacking kernels (or the equivalent) since
1967. I'd much rather have LKMs. If I were king, I'd decree that
*all* device drivers must be loadable, and *all* device drivers
should be dynamically loaded except for those that are necessary to
boot the system and read in new device drivers.
If I were trying to build hardened systems, I'd be really glad you're
not king. One of the first things I do when building a
security-critical system is remove LKM capability. Securing one file
(which may not even be a normal file) on boot media is a significantly
easier task than securing a few dozen files in the running system's
filesystem.
Uhm, you and a few other folks have talked about embedded systems in this
manner, and I really don't understand it. If you are doing embedded system
work, WTF is your root file system actually coming off of the boot media?

My experience is that it's far saner to do something like what NetBSD does
for install kernels. Put the root file system (with all the binaries) in
the kernel. Then mount the boot media somewhere and have symlinks from
/etc and such into that directory. Since everything you execute came with
the kernel, you do not have binary versioning issues (though you still
have configuration versioning issues).

That said, a static kernel is porbably still better. Files take up space
on the root file system (in memory), and loading an LKM would then need
that duplicated into KVM. So loaded modules would take up twice the space.

Take care,

Bill
Allen Briggs
2006-02-21 19:37:42 UTC
Permalink
Post by Bill Studenmund
Uhm, you and a few other folks have talked about embedded systems in this
manner, and I really don't understand it. If you are doing embedded system
work, WTF is your root file system actually coming off of the boot media?
There are multiple ways to do embedded systems. The method you
outline is handy for some systems, but is certainly not the only
reasonable way to do it.

-allen
--
Use NetBSD! http://www.NetBSD.org/
Bill Studenmund
2006-02-21 20:54:56 UTC
Permalink
Post by Allen Briggs
Post by Bill Studenmund
Uhm, you and a few other folks have talked about embedded systems in this
manner, and I really don't understand it. If you are doing embedded system
work, WTF is your root file system actually coming off of the boot media?
There are multiple ways to do embedded systems. The method you
outline is handy for some systems, but is certainly not the only
reasonable way to do it.
I agree there are other ways to do this and I'm not suggesting we make the
other methods stop working.

However other folks have used how embedded systems work (or don't work) to
influence design decisions within NetBSD. My concern is that these
discussions really involve a specific method of doing embedded systems yet
get painted with the broad brush of "embedded." So I think your comment
well applies to _everyone_ who has spoken about embedded systems. :-)

What I find vexing is that folks are often discussing pain points and
difficulties regarding "embedded" systems that really have to do with how
they are doing it. Do it differently and you don't have that pain point.
:-)

You may have other, more significant, pain points, and thus there may be
an excellent reason for WTF it's done that way. :-) But then anyone making
such a system should realize that you made a choice.

Take care,

Bill
j***@dsg.stanford.edu
2006-02-21 19:56:37 UTC
Permalink
Post by Bill Studenmund
Uhm, you and a few other folks have talked about embedded systems in this
manner, and I really don't understand it. If you are doing embedded system
work, WTF is your root file system actually coming off of the boot media?
My experience is that it's far saner to do something like what NetBSD does
for install kernels. Put the root file system (with all the binaries) in
the kernel. Then mount the boot media somewhere and have symlinks from
/etc and such into that directory. Since everything you execute came with
the kernel, you do not have binary versioning issues (though you still
have configuration versioning issues).
Yup.

I can't speak for der Mouse, but I do both the above and more: root
filesystem in MFS, much as for install kernels; *and* no LKM support,
*and* mounting filesystems write-xor-execute.

Plus whatever approximation of non-executable stack is feasible. And
static-linked executables (or crunched sets) for externally-visible
userland code, to limit return-into-libc vulnerability.
I suspect Thor takes a similar approach.
Post by Bill Studenmund
That said, a static kernel is porbably still better. Files take up space
on the root file system (in memory), and loading an LKM would then need
that duplicated into KVM. So loaded modules would take up twice the space.
Yes, quite. That said, there's almost as many way to set up embedded
systems as there are embedded systems. The root-in-mfs approach may
not be best for all.
der Mouse
2006-02-21 20:16:13 UTC
Permalink
Post by Bill Studenmund
Post by der Mouse
If I were king, I'd decree that *all* device drivers must be
loadable, and *all* device drivers should be dynamically loaded
except for those that are necessary to boot the system and read in
new device drivers.
If I were trying to build hardened systems, I'd be really glad
you're not king. [...]
Uhm, you and a few other folks have talked about embedded systems in
this manner, and I really don't understand it.
I actually wrote "hardened", not "embedded". They share some design
criteria, but are a long long way from being the same thing.
Post by Bill Studenmund
If you are doing embedded system work, WTF is your root file system
actually coming off of the boot media?
It might not be; boot media and root media do not necessarily bear any
particular relationship to one another. However, what does that have
to do with anything?
Post by Bill Studenmund
My experience is that it's far saner to do something like what NetBSD
does for install kernels. Put the root file system (with all the
binaries) in the kernel.
Yes, sometimes that's a sane choice. But sometimes it's not.
Post by Bill Studenmund
That said, a static kernel is porbably still better. Files take up
space on the root file system (in memory), and loading an LKM would
then need that duplicated into KVM. So loaded modules would take up
twice the space.
Yes. But, more important from a hardening point of view, if the kernel
doesn't have LKM support, it's *much* harder to load a rootkit-style
module into it at run time. (If it has LKM support but it's disabled
eg via securelevel, you are pinning all your security on the memory
word holding securelevel. If you don't even have LKM support in the
kernel, it's a great deal more difficult to do the equivalent of
loading an LKM, even if it may be just as theoretically possible.)

/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
j***@dsg.stanford.edu
2006-02-20 18:29:56 UTC
Permalink
Post by Garrett D'Amore
Post by Steven M. Bellovin
I'm with you. I've been hacking kernels (or the equivalent) since 1967.
I'd much rather have LKMs. If I were king, I'd decree that *all*
device drivers must be loadable, and *all* device drivers should be
dynamically loaded except for those that are necessary to boot the
system and read in new device drivers.
--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
This seems so obvious to me, that I was quite surprised to find the
point contentious. Anyway, as someone who is interested in it, I've
raised the question about what is needed to make it happen -- with a
willingness to provide some assistance if needed.
what's needed to make it happen?

1. the current LKM interface has to die: it's too primitive to support
a this-decade kernel composed of many dynamic modules.

2. We need an in-kernel linker for modules, so that modules can be
loaded early at boot.

3. Optionally, we need machinery in the machine-dependent bootloader
to load modules, and support in the kernel to link those modules into
the kernel "at boot". Examples: loading optional firmware for a boot
device which matches the kernel's driver for that boot device.
(Anyone familiar with Qlogic isp drivers and firmware changes/fixes
knows what [i'm getting at).

I understand FreeBSD has had the relevant machinery for what, nearly a
decade. Oh, and last, to keep us troglodyte builders of embedded
systems happy:

4. config machinery to support current-style fully-static kernels,
for those who decide they still want that.
Jason Thorpe
2006-02-20 23:39:00 UTC
Permalink
Post by Steven M. Bellovin
I'm with you. I've been hacking kernels (or the equivalent) since 1967.
I'd much rather have LKMs. If I were king, I'd decree that *all*
device drivers must be loadable, and *all* device drivers should be
dynamically loaded except for those that are necessary to boot the
system and read in new device drivers.
Yah, I'm pretty much in that camp, too.

-- thorpej
Valeriy E. Ushakov
2006-02-20 15:47:38 UTC
Permalink
Post by Garrett D'Amore
performance of binary linked vs. static will vary from platform to
platform, but yes, I think generally the benefits outweigh the
costs. IMO.
And given that to access driver or filesystem code the kernel already
uses function pointers anyway, I'm not even sure there is any
performance penalty after the LKM is loaded. The code is not PIC, it
needs to be relocated before being loaded (that's why we need ld(1)
for LKMs) into the kernel, so there's no PIC indirection price to
pay).

SY, Uwe
--
***@ptc.spbu.ru | Zu Grunde kommen
http://snark.ptc.spbu.ru/~uwe/ | Ist zu Grunde gehen
Garrett D'Amore
2006-02-20 15:51:06 UTC
Permalink
Post by Valeriy E. Ushakov
Post by Garrett D'Amore
performance of binary linked vs. static will vary from platform to
platform, but yes, I think generally the benefits outweigh the
costs. IMO.
And given that to access driver or filesystem code the kernel already
uses function pointers anyway, I'm not even sure there is any
performance penalty after the LKM is loaded. The code is not PIC, it
needs to be relocated before being loaded (that's why we need ld(1)
for LKMs) into the kernel, so there's no PIC indirection price to
pay).
ah. thanks for that clarification.

-- Garrett
Post by Valeriy E. Ushakov
SY, Uwe
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Simon Burge
2006-02-20 22:48:21 UTC
Permalink
Post by Valeriy E. Ushakov
Post by Garrett D'Amore
performance of binary linked vs. static will vary from platform to
platform, but yes, I think generally the benefits outweigh the
costs. IMO.
And given that to access driver or filesystem code the kernel already
uses function pointers anyway, I'm not even sure there is any
performance penalty after the LKM is loaded. The code is not PIC, it
needs to be relocated before being loaded (that's why we need ld(1)
for LKMs) into the kernel, so there's no PIC indirection price to
pay).
MIPS needs to use -mlong-calls to compile LKMs, and PowerPC and ARM use
trampolines to call functions external to the LKM, so there's at least
some performance hit for those architectures.

Simon.
--
Simon Burge <***@wasabisystems.com>
NetBSD Support and Service: http://www.wasabisystems.com/
j***@britannica.bec.de
2006-02-20 15:42:53 UTC
Permalink
Post by Robert Elz
| Sorry, this is just bull shit. LKMs add *zero* overhead to the kernel.
Huh? You mean they don't need any code sitting around occupying
memory waiting for an LKM to come along to load? And a call of a
function in an LKM (from outside, or a different LKM) goes exactly
as fast as a call of the same function if it is statically linked?
Less than 16kb on ia32. Most network drivers a bigger. Kernel modules are compiled
the way as normal code, unlike shared objects in userland. E.g. no
-fPIC. I don't think change of kernel image size counts as overhead and
otherwise doesn't alter any critical code paths.
Post by Robert Elz
| You have the symbol table in the kernel, as soon as e.g. debugging is
| wanted too.
But most people don't want debugging in the kernel. It crashes,
I reboot it. If it happens often, I might send a PR (but it doesn't).
Again, GENERIC has DDB by default. If you have your own kernel config,
it is completely up to you whether you enable it or not.
Post by Robert Elz
| Proper kernel module support dramatically reduces the need to have
| alternative kernels.
Sure, and if you're doing binary distributions that matters. But
most of the use of LKMs (with NetBSD) is by people with the sources
who just don't want to be bothered compiling the kernel they need.
In FreeBSD and DragonFly, PF is not part of the default kernel, but can
be loaded as module. You don't have to create a custom kernel if you
just want to try it -- meaning that a lot of users don't have to
download the source *at all*.
Post by Robert Elz
| No ACPI subkernels needed, since ACPI is loaded as module
| or not, depending on need.
How many times does one of your systems change from needing ACPI
to not needing ACPI dynamically, while it is running, without
rebooting it?
That's the wrong question. How many kernels does your live CD / bootable
USB stick have? Given that a GENERIC kernel is around 8 MB, not having
two full kernels is a real improvement. A lot of newer machines don't
boot properly without ACPI and some older machines have problems with
it, not speaking about APM.

But back to the original question -- this doesn't affect IPSec at all,
since it can't be made a module without a lot of efforts in any case.

Joerg
Garrett D'Amore
2006-02-20 15:50:22 UTC
Permalink
Post by j***@britannica.bec.de
Post by Robert Elz
| Sorry, this is just bull shit. LKMs add *zero* overhead to the kernel.
Huh? You mean they don't need any code sitting around occupying
memory waiting for an LKM to come along to load? And a call of a
function in an LKM (from outside, or a different LKM) goes exactly
as fast as a call of the same function if it is statically linked?
Less than 16kb on ia32. Most network drivers a bigger. Kernel modules are compiled
the way as normal code, unlike shared objects in userland. E.g. no
-fPIC. I don't think change of kernel image size counts as overhead and
otherwise doesn't alter any critical code paths.
making them modules *might* have an impact on cache performance.
however, since you'd only actually have modules loaded that were
actually being used, it isn't clear what exactly that impact would be.

if you don't have -fPIC (or at least -fpic), how do you get them to be
loadable? You have to be able to change the addresses, right? Or am I
missing something obvious?
Post by j***@britannica.bec.de
That's the wrong question. How many kernels does your live CD / bootable
USB stick have? Given that a GENERIC kernel is around 8 MB, not having
two full kernels is a real improvement. A lot of newer machines don't
boot properly without ACPI and some older machines have problems with
it, not speaking about APM.
this is the kind of thing I want to avoid -- having different configs
for different hardware that could easily be handled by just having
different drivers loaded seems wasteful to me.
Post by j***@britannica.bec.de
But back to the original question -- this doesn't affect IPSec at all,
since it can't be made a module without a lot of efforts in any case.
true, perhaps. but if so, then why? it seems a lot of ipsec at least
could be -- e.g. encryption and hash routines, etc.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Valeriy E. Ushakov
2006-02-20 15:54:47 UTC
Permalink
Post by Garrett D'Amore
if you don't have -fPIC (or at least -fpic), how do you get them to
be loadable? You have to be able to change the addresses, right?
Or am I missing something obvious?
You need to relocate them. LKMs (ab)use ld(1) for that. We need a
real in-kernel linker to avoid the LKM -> /usr dependency b/c of ld.

SY, Uwe
--
***@ptc.spbu.ru | Zu Grunde kommen
http://snark.ptc.spbu.ru/~uwe/ | Ist zu Grunde gehen
j***@britannica.bec.de
2006-02-20 15:58:51 UTC
Permalink
Post by Garrett D'Amore
Post by j***@britannica.bec.de
That's the wrong question. How many kernels does your live CD / bootable
USB stick have? Given that a GENERIC kernel is around 8 MB, not having
two full kernels is a real improvement. A lot of newer machines don't
boot properly without ACPI and some older machines have problems with
it, not speaking about APM.
this is the kind of thing I want to avoid -- having different configs
for different hardware that could easily be handled by just having
different drivers loaded seems wasteful to me.
Yes, I fully agree, but it is a sad reality.
Post by Garrett D'Amore
Post by j***@britannica.bec.de
But back to the original question -- this doesn't affect IPSec at all,
since it can't be made a module without a lot of efforts in any case.
true, perhaps. but if so, then why? it seems a lot of ipsec at least
could be -- e.g. encryption and hash routines, etc.
The encryption and hash algoritms could be done as crypto modules
without a bigger impact. The problem with ipsec itself is that it hooks
itself into quite a bunch of places. That's what can negatively affect
the system performance even if no ipsec is used at all.

Joerg
Thor Lancelot Simon
2006-02-20 16:03:05 UTC
Permalink
Post by Garrett D'Amore
Post by j***@britannica.bec.de
But back to the original question -- this doesn't affect IPSec at all,
since it can't be made a module without a lot of efforts in any case.
true, perhaps. but if so, then why? it seems a lot of ipsec at least
could be -- e.g. encryption and hash routines, etc.
Except that those routines are almost always in anyway.

IPsec hooks in all over the network code -- it is anything _but_ a "bump
in the stack" implementation. That makes it useful for more than toy
VPN applications (unlike many BITS implementations) but also means that
it is extremely difficult to cleanly separate out into a module, _and_
that just including it in the kernel causes a measurable decrease in
forwarding performance. Which is why it's not in the kernel by default.

The other outstanding issue is that the code selected by options
FAST_IPSEC needs to grow v6 support, and the code selected by options
IPSEC needs to die. I'd encourage anyone thinking of doing significant
work on our IPsec code to _not_ put it into something like modularizing
the KAME code, at this point!

Thor
j***@dsg.stanford.edu
2006-02-20 18:53:00 UTC
Permalink
Post by Thor Lancelot Simon
Post by Garrett D'Amore
Post by j***@britannica.bec.de
But back to the original question -- this doesn't affect IPSec at all,
since it can't be made a module without a lot of efforts in any case.
true, perhaps. but if so, then why? it seems a lot of ipsec at least
could be -- e.g. encryption and hash routines, etc.
Except that those routines are almost always in anyway.
IPsec hooks in all over the network code -- it is anything _but_ a "bump
in the stack" implementation. That makes it useful for more than toy
VPN applications (unlike many BITS implementations) but also means that
it is extremely difficult to cleanly separate out into a module, _and_
that just including it in the kernel causes a measurable decrease in
forwarding performance. Which is why it's not in the kernel by default.
We don't want to turn on IPsec for a simple reason: the KAME
code performs poorly, does not scale, and (worst of all for some uses)
the inline calls to crypto transforms impose head-of-line blocking on
all non-IPsec traffic.

But even if you don't use IPsec *at all*, turning on IPsec imposes a
significant packet-classification on all traffic (except for outbound
TCP traffic on connected sockets). Even inbound TCP incurs a
significant penalty, which is easily measurable on network benchmarks.

There's a simple test to gauge the impact of IPsec, which I think I've
described privately to Thor and others. Networking researchers have
used ttcp-over-UDP *receive* rates for decades as a quick,
rule-of-thumb estimate of the packet-processing ability of a given
interface/software/machine combination. One can use this
ttcp-over-UDP estimate as a quick measure of IPsec overhead:


0. Find two suitable machines, connected via a network link
with which the machines can, preferably, _just_ keep up.

1. Build two kernels, with and with IPsec enabled,
but otherwise identical.

2. Find a machine which can run ttcp -u -t fast enough to fill a wire
with UDP traffic.

3a. Boot the non-IPsec kernel on a receiving machine.
Run the ttcp -u sender at this machine.
Record the processed packet rate reported with ttcp -u -r.
(In a well-designed experiment, the receiver will not quit keep
up with the offered packet rate).

3b. Boot the IPsec kernel on the same receiving hardware.
Repeat the measurement in 3a. Compare and contrast.


For a more "real-world" but less simple test, one may configure
NFS-over-UDP traffic aas a test workload. I'd suggest setting up a
ramdisk (mfs, or tmpfs in -current) on a "server", export the
"ramdisk" via NFS, and use a second machine as a client. On the
client, measure peak write rate for doing dd's from a local file to
the NFS ramdisk. Repeat the measurements with and without IPsec on the
server (or better yet, with and without IPsec on both machines).

Either way, I find the results extremely discouraging.
Post by Thor Lancelot Simon
The other outstanding issue is that the code selected by options
FAST_IPSEC needs to grow v6 support, and the code selected by options
IPSEC needs to die. I'd encourage anyone thinking of doing significant
work on our IPsec code to _not_ put it into something like modularizing
the KAME code, at this point!
Quite. But speaking just personally, I'm more interested in adding
/etc/rc.d hooks to not enable utterly useless protocols[*] like
IPv6 on my machines, than I am in adding IPv6 support to FAST_IPSEC :-/.

[*] That is, IPv6 is utterly useless on *my* machines.
j***@dsg.stanford.edu
2006-02-20 19:16:01 UTC
Permalink
[..]
Post by j***@dsg.stanford.edu
There's a simple test to gauge the impact of IPsec, which I think I've
described privately to Thor and others. Networking researchers have
used ttcp-over-UDP *receive* rates for decades as a quick,
rule-of-thumb estimate of the packet-processing ability of a given
interface/software/machine combination. One can use this
0. Find two suitable machines, connected via a network link
with which the machines can, preferably, _just_ keep up.
1. Build two kernels, with and with IPsec enabled,
but otherwise identical.
2. Find a machine which can run ttcp -u -t fast enough to fill a wire
with UDP traffic.
3a. Boot the non-IPsec kernel on a receiving machine.
Run the ttcp -u sender at this machine.
Record the processed packet rate reported with ttcp -u -r.
(In a well-designed experiment, the receiver will not quit keep
up with the offered packet rate).
3b. Boot the IPsec kernel on the same receiving hardware.
Repeat the measurement in 3a. Compare and contrast.
Someone who cares could do a slighly more complicated experiment:
measuring packet rate for:

A. non-IPsec kernels
B. IPsec-aware (KAME or FAST_IPsec) kernels with no SAsa
C. IPsec-aware (KAME or FAST_IPsec) kernels with a modest number of SAs.

plus, for the masochistic:

D. IPsec-aware (KAME or FAST_IPsec) kernels with a modest number of SAs,
say a few hundred each SPD and SADB entries (if KAMe can actually
handle that many now; it couldn't, until fairly recently).

For most benchmarking purposes, it's A. vs. B. which is of interest.
In my experience, it's A. vs C which is painful, and A vs. D) which is
_really_ painful.

The A vs. B case _should_ be a wash. (If it isn't, it should be made so).

If configuring IPsec into the kernel but not configuring any IPsec at
runtime, adds no real penalty over the no-IPsec-configured case, why
not put IPsec into GENERIC?
Michael van Elst
2006-02-20 20:33:53 UTC
Permalink
Post by j***@dsg.stanford.edu
[*] That is, IPv6 is utterly useless on *my* machines.
Do you run GENERIC kernels on your machines?
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."
j***@dsg.stanford.edu
2006-02-20 20:52:07 UTC
Permalink
Post by Michael van Elst
Post by j***@dsg.stanford.edu
[*] That is, IPv6 is utterly useless on *my* machines.
Do you run GENERIC kernels on your machines?
Some yes, some no. What's that got to do with the fact that IPv6 *is*
utterly useless to me, or that i'd like to not enable IPv6 on machines
which are still running GENERIC kernels?
Michael van Elst
2006-02-20 21:19:16 UTC
Permalink
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Post by j***@dsg.stanford.edu
[*] That is, IPv6 is utterly useless on *my* machines.
Do you run GENERIC kernels on your machines?
Some yes, some no. What's that got to do with the fact that IPv6 *is*
utterly useless to me, or that i'd like to not enable IPv6 on machines
which are still running GENERIC kernels?
I wanted to stress that people like you (or me, or almost everybody reading)
often do not use GENERIC kernels and whatever is in GENERIC has little
impact on us.

You may disable IPv6 as you may disable^Wenable IPSEC if that's what you
need.

However, GENERIC is what gets installed initially, what might be the
only choice for some people and is necessarily the first choice for
newcomers. Having IPSEC there is worthwhile even when it spoils
benchmarks.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."
Jonathan Stone
2006-02-20 21:27:42 UTC
Permalink
Post by Michael van Elst
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Do you run GENERIC kernels on your machines?
Some yes, some no. What's that got to do with the fact that IPv6 *is*
utterly useless to me, or that i'd like to not enable IPv6 on machines
which are still running GENERIC kernels?
I wanted to stress that people like you (or me, or almost everybody reading)
often do not use GENERIC kernels and whatever is in GENERIC has little
impact on us.
You may disable IPv6 as you may disable^Wenable IPSEC if that's what you
need.
Michael, I *do* control machines which run GENERIC kernels, I want
to *keep* running GENERIC kernels, but I *do* want a way to turn off
IPv6 on those machines *without* having to build custom kernels.

Is any part of that hard to understand? I'm not understanding
why you don't understand it.
Post by Michael van Elst
However, GENERIC is what gets installed initially, what might be the
only choice for some people and is necessarily the first choice for
newcomers. Having IPSEC there is worthwhile even when it spoils
benchmarks.
Yes, those are precisely the sorts of reasons why detuning benchmark
performance of GENERIC is widely regarded (by several senior NetBSD
developers) as being a bad idea. I recall there was quite a strong
consensus on that, last time the issue came up.

In any case, building a kernel strikes me as quite a modest burden,
compared to configuring IPsec and IKE to acutally work.
(Especially if it's the first time one is setting up IPsec.)
Michael van Elst
2006-02-20 21:55:40 UTC
Permalink
Post by Jonathan Stone
Michael, I *do* control machines which run GENERIC kernels, I want
to *keep* running GENERIC kernels, but I *do* want a way to turn off
IPv6 on those machines *without* having to build custom kernels.
Is any part of that hard to understand? I'm not understanding
why you don't understand it.
Indeed, that is very difficult to understand, because for you it
is very easy to build a custom kernel.
Post by Jonathan Stone
Post by Michael van Elst
However, GENERIC is what gets installed initially, what might be the
only choice for some people and is necessarily the first choice for
newcomers. Having IPSEC there is worthwhile even when it spoils
benchmarks.
Yes, those are precisely the sorts of reasons why detuning benchmark
performance of GENERIC is widely regarded (by several senior NetBSD
developers) as being a bad idea. I recall there was quite a strong
consensus on that, last time the issue came up.
Let me rephrase this with the words in my argument to make sure that
I understand your words correctly.

Making a GENERIC kernel support IPSEC initially is bad.
Making a GENERIC kernel for people useful that rely on the initial install is bad.
Making a GENERIC kernel as versatile as possible for newcomers is bad.
Making a GENERIC kernel that spoils benchmarks is bad.
And that is supported by several senior NetBSD developers.

Yes, that makes your point crystal clear. But I do not agree.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."
j***@dsg.stanford.edu
2006-02-20 22:26:41 UTC
Permalink
Post by Michael van Elst
Post by Jonathan Stone
Michael, I *do* control machines which run GENERIC kernels, I want
to *keep* running GENERIC kernels, but I *do* want a way to turn off
IPv6 on those machines *without* having to build custom kernels.
Is any part of that hard to understand? I'm not understanding
why you don't understand it.
Indeed, that is very difficult to understand, because for you it
is very easy to build a custom kernel.
Michael, since you have a .de address, do you have difficulty
understanding simple English?

If I don't want to run IPv4, I don't have to build a special kernel;
GENERIC plus our boot-time /etc/rc.conf mechanisms can do that just
fine. So, why should it be any different if I want to run lPv4, but
not IPv6?

Whether or not *you* think it's trivial for me to build a custom
kernel, just to disable a feature which is of no use to me (and might
even be harmful), is besides the point.
Post by Michael van Elst
Post by Jonathan Stone
Post by Michael van Elst
However, GENERIC is what gets installed initially, what might be the
only choice for some people and is necessarily the first choice for
newcomers. Having IPSEC there is worthwhile even when it spoils
benchmarks.
Yes, those are precisely the sorts of reasons why detuning benchmark
performance of GENERIC is widely regarded (by several senior NetBSD
developers) as being a bad idea. I recall there was quite a strong
consensus on that, last time the issue came up.
Let me rephrase this with the words in my argument to make sure that
I understand your words correctly.
Let me go through those (apparently deliberate and inflammatory)
Post by Michael van Elst
Making a GENERIC kernel support IPSEC initially is bad.
No, I never said that, so please don't put words in my mouth like
that. There's nothing wrong with supporting IPsec, *provided* doing
so doesn't impair other uses. Unfortunately, as far as we know,
currently IPsec does impair other uses.
Post by Michael van Elst
Making a GENERIC kernel for people useful that rely on the initial install is bad.
Hah! Michael, do you realize you've just contradicted your own
argument? A kernel with IPv6 support is not useful for *me*, but you
claim that's not worth fixing because I can easily build my own
kernel. But so can anyone else. So by your own argument, what is (or
isn't, in UGENERIC) is beside the point, right?
Post by Michael van Elst
Making a GENERIC kernel as versatile as possible for newcomers is bad.
Indeed. But versatility is in the eye of the beholder.
Post by Michael van Elst
Making a GENERIC kernel that spoils benchmarks is bad.
Yes, that's the consensus of the developers who've been maintaining
portions of the stack. Not just me, but (if I recall correclty)
Thor, and Jason Thorpe, and others. I'm not sure if Matt Thomas
commented or not.
Post by Michael van Elst
And that is supported by several senior NetBSD developers.
Yes. In fact, more than that: there was a *consensus* not to turn on
IPsec in GENERIC kernels, on the understanding IPsec would cause a
noticeable performance hit.

As far as I can see, *you* want IPsec, you don't care about the impact
of IPsec on other users. That's not a very productive way to reach
your goal. OTOH, if you *do* want to do something productive, why
don't you try to quantify the actual, current impact of IPsec, via the
techniques I've outlined earlier?
Christos Zoulas
2006-02-20 22:56:48 UTC
Permalink
Post by j***@dsg.stanford.edu
Michael, since you have a .de address, do you have difficulty
understanding simple English?
This is completely inappropriate and uncalled for.

christos
Jonathan Stone
2006-02-21 02:05:53 UTC
Permalink
Post by Christos Zoulas
Post by j***@dsg.stanford.edu
Michael, since you have a .de address, do you have difficulty
understanding simple English?
This is completely inappropriate and uncalled for.
Christos, I truly, sincerely did not mean to cause any offense.
Michael and i seem to have difficulty understanding each other's
position. I'm wondering if that's due in part to language issues, as
(as I said in material you didn[t quote) I certainly do take issue
with Michael's paraprhase of what he thinks my position is.

What's so offensive about asking if language issues are part of that
mutual incomprehension?
Christos Zoulas
2006-02-21 02:32:40 UTC
Permalink
Post by Jonathan Stone
Post by Christos Zoulas
Post by j***@dsg.stanford.edu
Michael, since you have a .de address, do you have difficulty
understanding simple English?
This is completely inappropriate and uncalled for.
Christos, I truly, sincerely did not mean to cause any offense.
Michael and i seem to have difficulty understanding each other's
position. I'm wondering if that's due in part to language issues, as
(as I said in material you didn[t quote) I certainly do take issue
with Michael's paraprhase of what he thinks my position is.
What's so offensive about asking if language issues are part of that
mutual incomprehension?
It is the condenscending tone ("simple English") that bothered me.
You have to be very careful when saying things like this because
it is really easy for your audience to take it the wrong way (I did).
Specially if you are doing this in a public list instead on a one
to one exchange.

It is always better to err on the conservative side and blame
yourself: "maybe I am not explaining myself clearly" as opposed
to: "you don't seem to understand what I am saying". This shows
the hospitality of the "home court", since you are the native
speaker.

Or course this all depends on your goals. Do you want to educate/help
your audience or do you want to show off your knowledge and make
everyone else look/feel like an idiot?

christos
Jonathan Stone
2006-02-21 03:35:17 UTC
Permalink
Post by Christos Zoulas
It is always better to err on the conservative side and blame
yourself: "maybe I am not explaining myself clearly" as opposed
to: "you don't seem to understand what I am saying". This shows
the hospitality of the "home court", since you are the native
speaker.
Or course this all depends on your goals. Do you want to educate/help
your audience or do you want to show off your knowledge and make
everyone else look/feel like an idiot?
Christos, I think _that_ was uncalled for.

It's a historical fact that this is not the first time someone has
proposed turning on IPsec in GENERIC kernels. Last time this idea, to
turn on IPsec in GENERIC kernels came up, we decided that turning on
IPsec adversely impacts networking performance, that this negative
impact would be noticeable when people, magazines, etc. benchmarked
NetBSD against other BSDs or Linux. I recall concern that the typical
benchmarker might not bother to dig deep enough to see that NetBSD had
IPsec enabled: in other words, the reasons *why* NetBSD might have
lower performance than other open-source OSes wouldn't
register. Anyway, the consensus decision was that the negative impact
outweighed any benefits of having IPsec avialable in GENERIC kernels.

I've tried hard to explain that historical fact. I've suggested a
couple of ways by which the alleged overhead might be estimated. I've
suggested ways where, as far as i can see, any signifiant overhead
*should* be able to be eliminated.

Now doesn't that suggest to you that I'm trying to help find ways
whereeby we might turn on IPSEC in GENERIC kernels without the adverse
affects which outweighed IPsec last time the issue came up?

Yet my correspondent doesn't seem to care about any of that, he seems
to prefer to ignore it, and to repeat assertions that in *his*
opinion, IPSEC is more important and that we should overturn the prior
consensus, with no other justificaiton that *his* personal say-so.

I didn't think that was how NetBSD worked. Do you?
Christos Zoulas
2006-02-21 03:49:24 UTC
Permalink
On Feb 20, 7:35pm, ***@Pescadero.dsg.stanford.edu (Jonathan Stone) wrote:
-- Subject: Re: IPSEC in GENERIC

| >Or course this all depends on your goals. Do you want to educate/help
| >your audience or do you want to show off your knowledge and make
| >everyone else look/feel like an idiot?
|
| Christos, I think _that_ was uncalled for.

Fine, I am sorry for misjudging your intensions.

I see the summary of this thread as:

1. The reason we don't turn IPSEC on by default is performance.
2. There is FAST_IPSEC but it does not do IPV6.
3. The KAME code is too complicated/messy and hard to separate.
4. In order to do have the best of both worlds is to compile IPSEC
in and determine if we need to take the IPSEC complex path only
if the number of SPD's > 0 (which was proposed by you this round).

Now can you (or someone else) step up and see if doing (4) is feasible?

christos
j***@dsg.stanford.edu
2006-02-21 04:14:16 UTC
Permalink
Post by Christos Zoulas
-- Subject: Re: IPSEC in GENERIC
[...]
Post by Christos Zoulas
1. The reason we don't turn IPSEC on by default is performance.
2. There is FAST_IPSEC but it does not do IPV6.
3. The KAME code is too complicated/messy and hard to separate.
4. In order to do have the best of both worlds is to compile IPSEC
in and determine if we need to take the IPSEC complex path only
if the number of SPD's > 0 (which was proposed by you this round).
Now can you (or someone else) step up and see if doing (4) is feasible?
Someone else I think, since I have approximately zero in the old
KAME IPsec code. I can say that I beleive the approach I suggested is
workable, for locally-terminated traffic. I don't recall making any
measure of forwarding rates myself, which is the case Thor brought up.

As I tried to say earlier: right now I'm more interested in turning
off IPv6 on the machines I have with GENERIC kernels, so that those
machines don't automagically start using IPv6 for local-subnet
communication behind my back. That, I care about, whereas IPsec in
GENERIC I don't much care for.


In all honesty, in the local marketplace, I hear an awful lot of need
for "Cisco VPNs", "SSL VPNs", pptp, and almost no demand for IPsec.
The instructions I saw (Hubert's, I think, from Feb 2004?), for
configuring "Cisco VPN"s access say that one must first disable IPSEC
if it's configured, then add tun. These days, tun is on in most 3.0
GENERIC configs.

So, in all sincerity and thinking of just my local neck of the woods,
and assuming Hubert's caveat about IPSEC is still accurate, I'd tend
to think NetBSD might get more utility from excluding IPsec and
keeping tun, than from adding IPsec. Then again, maybe things are
very different where you are, or where Michael is.

Worth considering both ways, though.
Post by Christos Zoulas
christos
Garrett D'Amore
2006-02-21 04:34:35 UTC
Permalink
Post by j***@dsg.stanford.edu
Post by Christos Zoulas
-- Subject: Re: IPSEC in GENERIC
[...]
Post by Christos Zoulas
1. The reason we don't turn IPSEC on by default is performance.
2. There is FAST_IPSEC but it does not do IPV6.
3. The KAME code is too complicated/messy and hard to separate.
4. In order to do have the best of both worlds is to compile IPSEC
in and determine if we need to take the IPSEC complex path only
if the number of SPD's > 0 (which was proposed by you this round).
Now can you (or someone else) step up and see if doing (4) is feasible?
Someone else I think, since I have approximately zero in the old
KAME IPsec code. I can say that I beleive the approach I suggested is
workable, for locally-terminated traffic. I don't recall making any
measure of forwarding rates myself, which is the case Thor brought up.
As I tried to say earlier: right now I'm more interested in turning
off IPv6 on the machines I have with GENERIC kernels, so that those
machines don't automagically start using IPv6 for local-subnet
communication behind my back. That, I care about, whereas IPsec in
GENERIC I don't much care for.
I'm confused, is there some reason that just unconfiguring the interface
with ifconfig won't do this?

This doesn't sound anything like IPsec -- it sounds like you want a way
to force IPv4 at run-time. I think it should be possible to do that
regardless of whether or not IPv6 is *compiled* into the kernel.
Post by j***@dsg.stanford.edu
In all honesty, in the local marketplace, I hear an awful lot of need
for "Cisco VPNs", "SSL VPNs", pptp, and almost no demand for IPsec.
The instructions I saw (Hubert's, I think, from Feb 2004?), for
configuring "Cisco VPN"s access say that one must first disable IPSEC
if it's configured, then add tun. These days, tun is on in most 3.0
GENERIC configs.
Cisco VPN *is* IPsec, right? At least their "modern" VPN stuff is
pretty much just IPsec + IKE, as I understand it.

IPsec is critical for a lot of our deployments. We see demand for IPsec
increasing rapidly with some of our biggest customers.
--
Garrett D'Amore, Principal Software Engineer
Tadpole Computer / Computing Technologies Division,
General Dynamics C4 Systems
http://www.tadpolecomputer.com/
Phone: 951 325-2134 Fax: 951 325-2191
Christos Zoulas
2006-02-21 04:38:01 UTC
Permalink
On Feb 20, 8:14pm, ***@dsg.stanford.edu (***@dsg.stanford.edu) wrote:
-- Subject: Re: IPSEC in GENERIC

| Someone else I think, since I have approximately zero in the old
| KAME IPsec code. I can say that I beleive the approach I suggested is
| workable, for locally-terminated traffic. I don't recall making any
| measure of forwarding rates myself, which is the case Thor brought up.

I am almost done adding it. It seems pretty simple. I will post patches
once I have it all tested.

| As I tried to say earlier: right now I'm more interested in turning
| off IPv6 on the machines I have with GENERIC kernels, so that those
| machines don't automagically start using IPv6 for local-subnet
| communication behind my back. That, I care about, whereas IPsec in
| GENERIC I don't much care for.

That is a good point; I would send-pr :-)

| In all honesty, in the local marketplace, I hear an awful lot of need
| for "Cisco VPNs", "SSL VPNs", pptp, and almost no demand for IPsec.
| The instructions I saw (Hubert's, I think, from Feb 2004?), for
| configuring "Cisco VPN"s access say that one must first disable IPSEC
| if it's configured, then add tun. These days, tun is on in most 3.0
| GENERIC configs.
|
| So, in all sincerity and thinking of just my local neck of the woods,
| and assuming Hubert's caveat about IPSEC is still accurate, I'd tend
| to think NetBSD might get more utility from excluding IPsec and
| keeping tun, than from adding IPsec. Then again, maybe things are
| very different where you are, or where Michael is.
|
| Worth considering both ways, though.

It all depends if you use IPSEC or not. If you use IPSEC, then you want
it in the default kernel, if you don't then you don't care about it.

christos
Pavel Cahyna
2006-02-21 21:38:17 UTC
Permalink
Post by Jonathan Stone
It's a historical fact that this is not the first time someone has
proposed turning on IPsec in GENERIC kernels. (...)
Maybe because the performance impact is completely undocumented in
options(4)?

Pavel Cahyna
Thomas E. Spanjaard
2006-02-20 23:01:29 UTC
Permalink
<snip less-than-friendly arguments>
GENERIC is supposed to give a generic config that applies to most uses.
Seeing that the IPv6 deployment rate is steady and will continue to be
so in the future, I see no reason to leave it out. Given that in some
areas, IPv6 is close to mainstream (think Japan, mobile networks,
government support in a couple of other Asian countries as well,
provider backing in Europe), I see no reason to leave it out, being
pretty generic enough. IPsec however does not have this prospect (yet),
so it makes sense to leave it out. If IPsec turns out to be common in
the future, it should be added to GENERIC anyway, no matter what the
cost is (assuming it's fully working, ofcourse). Besides this, IPsec in
NetBSD is still somewhat shrouded in vagaries, at least it comes accross
to me like that. As for now, IPsec should not end up in GENERIC, for the
above reasons.

Hope this comment is of any help,
--
Thomas E. Spanjaard
***@netphreax.net
Michael van Elst
2006-02-20 23:52:40 UTC
Permalink
Post by j***@dsg.stanford.edu
If I don't want to run IPv4, I don't have to build a special kernel;
GENERIC plus our boot-time /etc/rc.conf mechanisms can do that just
fine. So, why should it be any different if I want to run lPv4, but
not IPv6?
There is no necessity to have it different. But it is different.
Post by j***@dsg.stanford.edu
Whether or not *you* think it's trivial for me to build a custom
kernel, just to disable a feature which is of no use to me (and might
even be harmful), is besides the point.
Maybe besides your point, but it is completely on topic unless
you can disable IPSEC with our boot-time /etc/rc.conf mechanism.

So far we need a special kernel to do IPSEC and we need a special
kernel to remove IPv6.
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Let me rephrase this with the words in my argument to make sure that
I understand your words correctly.
Let me go through those (apparently deliberate and inflammatory)
Interesting...
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel support IPSEC initially is bad.
No, I never said that, so please don't put words in my mouth like
that. There's nothing wrong with supporting IPsec, *provided* doing
so doesn't impair other uses. Unfortunately, as far as we know,
currently IPsec does impair other uses.
Now that language is not simple enough for me. But your conclusion
is that, as far as we know, putting IPSEC in GENERIC is bad.
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel for people useful that rely on the initial install is bad.
Hah! Michael, do you realize you've just contradicted your own
argument? A kernel with IPv6 support is not useful for *me*, but you
claim that's not worth fixing because I can easily build my own
kernel. But so can anyone else. So by your own argument, what is (or
isn't, in UGENERIC) is beside the point, right?
You can build your own kernel but "people that rely on the initial install"
obviously cannot. Your conclusion that anyone else has your abilities is wrong.
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel as versatile as possible for newcomers is bad.
Indeed. But versatility is in the eye of the beholder.
So you agree that I understood you correctly.
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel that spoils benchmarks is bad.
Yes, that's the consensus of the developers who've been maintaining
portions of the stack. Not just me, but (if I recall correclty)
Thor, and Jason Thorpe, and others. I'm not sure if Matt Thomas
commented or not.
Apparently having GENERIC tuned to certain benchmarks is not
on my priority list :)
Post by j***@dsg.stanford.edu
Post by Michael van Elst
And that is supported by several senior NetBSD developers.
Yes. In fact, more than that: there was a *consensus* not to turn on
IPsec in GENERIC kernels, on the understanding IPsec would cause a
noticeable performance hit.
Yes, I remember this dicussion. I wouldn't call it a *consensus*
but I guess that this is in the eye of the beholder. Back then
I might even have agreed, but now I think it is the wrong decision.
Post by j***@dsg.stanford.edu
As far as I can see, *you* want IPsec, you don't care about the impact
of IPsec on other users.
Putting words in other peoples mouth? You? :)

No, Jonathan. Otherwise I wouldn't care if building kernels is
easy for you. For me the impact of having IPSEC in GENERIC is small
and so it is for you.
Post by j***@dsg.stanford.edu
That's not a very productive way to reach
your goal.
I have many goals, forcing some decision on the contents of GENERIC
isn't among them. But I like to participate in the discussion, in
case other people without a focus on benchmarks agree with me. We
even might have a *consensus* on what GENERIC should be. But even
when there is no such *consensus* or even a decision, I wouldn't
just start to agree with the majority or the decision makers.
Post by j***@dsg.stanford.edu
OTOH, if you *do* want to do something productive, why
don't you try to quantify the actual, current impact of IPsec, via the
techniques I've outlined earlier?
I am currently more concerned about some weird TCP behaviour that makes
my backup crawl. Oh yes, and the abysmally slow NFS when used together
with null mounts. The read-before-write behaviour of our filesystems
is a problem too.

The impact of the IPSEC option on a system that doesn't use IPSEC
is difficult to measure. I only have a 100Mbps network and see nothing
and the bge driver probably has issues that impact performance much
more than the IPSEC option. What about FAST_IPSEC? It is of no use
to me because it doesn't support IPv6.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."
Jonathan Stone
2006-02-21 03:59:59 UTC
Permalink
Post by Michael van Elst
Post by j***@dsg.stanford.edu
If I don't want to run IPv4, I don't have to build a special kernel;
GENERIC plus our boot-time /etc/rc.conf mechanisms can do that just
fine. So, why should it be any different if I want to run lPv4, but
not IPv6?
There is no necessity to have it different. But it is different.
Michael, I'm trying hard, but for the life of me I cannot understand
that sentence. What are you trying to say?

[...snip quoted text]
Post by Michael van Elst
Maybe besides your point, but it is completely on topic unless
you can disable IPSEC with our boot-time /etc/rc.conf mechanism.
<forced patience> Yes, of *course* our rc.conf mechanisms can disable
the actual *use* of IPsec, sure. But the problem is that, last time
the idea to put IPsec in GENERIC came up here, even just *having*
IPsec configured into your kernel causes a noticeable performance hit.
Thor just repeated that claim in this thread.

And *that* is the reason for the prior consensus not to enable IPsec
in GENERIC kernels. If you want to overturn that consensus, I think
you need to present rational arguments why the prior consensus was
wrong. I've looked, and I can't find any real attempt to do that.
Post by Michael van Elst
So far we need a special kernel to do IPSEC and we need a special
kernel to remove IPv6.
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel support IPSEC initially is bad.
No, I never said that, so please don't put words in my mouth like
that. There's nothing wrong with supporting IPsec, *provided* doing
so doesn't impair other uses. Unfortunately, as far as we know,
currently IPsec does impair other uses.
Now that language is not simple enough for me.
How can I make it simpler? I personally can't imagine any technical
reason we'd not turn on IPsec in GENERIC if there was no downside,
since the upside (making in-kernel IPsec available in our default
installed kernel) is very clear. But there *is* a long-standing
observation that there *IS* a downside to adding IPsec to GENERIC
kernels: overall networking performance of GENERIC kernels would
suffer, and that the performance penalty (relative to other *BSD
kernels which don't turn on IPsec in their default kernel) would show
up in benchmarks, and be widely publicized, to the overall detriment
of NetBSD.
Post by Michael van Elst
But your conclusion
is that, as far as we know, putting IPSEC in GENERIC is bad.
No, not quite. That's not a conclusion, nor is it mine.
It's a widley-repeated statement. I'm just repeating it, as others
(Thor, for example) have already done in this thread.

There's also the consensus decision last time this idea came up, which
was that the known (or alleged) downside of slower networking
performance, outweighed the benefits of having IPsec in GENERIC kernels.

Once again, that's not *my* conclusion, but the consensus after
technical discussion.


[ .. discussion of adding IPsec to GENERIC vs. adding hooks to disable IPv6,
for those who want to disable it.]

Michael, I think we're talking at cross-purposes.

Adding hooks to our rc scripts, so as not to turn on IPv6. has no
signficiant impact on anyone. (It's one more shell variable in
/etc/defaults/rc.conf , and an extra test in already-existing "if"
clausees in 2 or 3 "if" statements that already exist in
/etc/rc.d/network. Now I don't know for sure that that'll do
everything I need, but I can't for the life of me see why anyone would
object to adding one more clause to those "if" statements.

And if more work is required -- adding runtime sysctl knobs to force
errno returns of EOPNOTSUPP or EPROTONOSUPPORT for attempts to create
a socket of an administratively-disabled family -- then *I'm* the one
saying I'll put time into that.
Post by Michael van Elst
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel as versatile as possible for newcomers is bad.
Indeed. But versatility is in the eye of the beholder.
So you agree that I understood you correctly.
No, I most certainly do not.
Post by Michael van Elst
Post by j***@dsg.stanford.edu
Post by Michael van Elst
Making a GENERIC kernel that spoils benchmarks is bad.
Yes, that's the consensus of the developers who've been maintaining
portions of the stack. Not just me, but (if I recall correclty)
Thor, and Jason Thorpe, and others. I'm not sure if Matt Thomas
commented or not.
Apparently having GENERIC tuned to certain benchmarks is not
on my priority list :)
Michael, can you explain why your personal priority list should
override the prior consensus on this issue? To me, that seems to be
what you've been suggesting repeatedly in This thread. But maybe I am
misunderstanding. If that is what your're suggesting, then I for one
find that repugnant.

On the other hand, if it's not what you are suggesting, then can you
try to rephrase your suggestion, because (if your decision-tree ends
up here) I'm clearly not understanding you at all.
Michael van Elst
2006-02-21 21:20:49 UTC
Permalink
Post by Jonathan Stone
Post by Michael van Elst
Post by j***@dsg.stanford.edu
So, why should it be any different if I want to run lPv4, but
not IPv6?
There is no necessity to have it different. But it is different.
Michael, I'm trying hard, but for the life of me I cannot understand
that sentence. What are you trying to say?
I don't know a reason why it should be any different and I wasn't
proposing it.
Post by Jonathan Stone
Post by Michael van Elst
Maybe besides your point, but it is completely on topic unless
you can disable IPSEC with our boot-time /etc/rc.conf mechanism.
<forced patience> Yes, of *course* our rc.conf mechanisms can disable
the actual *use* of IPsec, sure.
I said if you could disable IPSEC from rc.conf that would make a
difference. But we all know that this doesn't work but requires a kernel
change. I did not talk about the "use of IPsec".
Post by Jonathan Stone
And *that* is the reason for the prior consensus not to enable IPsec
in GENERIC kernels. If you want to overturn that consensus, I think
you need to present rational arguments why the prior consensus was
wrong. I've looked, and I can't find any real attempt to do that.
I have already overturned that *consensus* if there was ever one.
Apparently I disagree, so at least now there can't be a *consensus*
anymore. So that makes it a majority but not a *consensus*.

If you can't find real attempts then I believe that is your problem.
I gave a couple of reasons. Apparently the _conclusion_ depends on
your personal opinion. I don't agree with you, so if that is hard
to understand, then yes, you do have a problem.
Post by Jonathan Stone
How can I make it simpler? I personally can't imagine any technical
reason we'd not turn on IPsec in GENERIC if there was no downside,
Please remove the 'if' from the sentence. Apparently there are downsides
from either path. That's why you have to make a decision.
Post by Jonathan Stone
But there *is* a long-standing observation that there *IS* a downside
to adding IPsec to GENERIC kernels: overall networking performance of
GENERIC kernels would suffer, and that the performance penalty (relative
to other *BSD kernels which don't turn on IPsec in their default kernel)
would show up in benchmarks, and be widely publicized, to the overall
detriment of NetBSD.
There is a downside to not adding IPsec to GENERIC as well. However,
unlike you I do not fear "widely publicized benchmarks that show the
performance penalty against other *BSD kernels", so I see more the
advantages of having IPsec in GENERIC.
Post by Jonathan Stone
Post by Michael van Elst
But your conclusion
is that, as far as we know, putting IPSEC in GENERIC is bad.
No, not quite. That's not a conclusion, nor is it mine.
It's a widley-repeated statement. I'm just repeating it, as others
(Thor, for example) have already done in this thread.
If you believe in that statement then I suggest you make it yours.
There is no need to hide behind others.
Post by Jonathan Stone
Once again, that's not *my* conclusion, but the consensus after
technical discussion.
I wouldn't use the word "technical discussion" when the decision
made is more based on publicity.
Post by Jonathan Stone
Michael, can you explain why your personal priority list should
override the prior consensus on this issue?
You make the wrong statement that I want, that my personal priority
list overrides the prior *consensus*.
Post by Jonathan Stone
I'm clearly not understanding you at all.
Then you may want to end this discussion. It is fruitless if
you are not understanding.
--
--
Michael van Elst
Internet: ***@serpens.de
"A potential Snark may lurk in every tree."
Jonathan Stone
2006-02-21 21:44:54 UTC
Permalink
[...]
Post by Michael van Elst
Post by Jonathan Stone
And *that* is the reason for the prior consensus not to enable IPsec
in GENERIC kernels. If you want to overturn that consensus, I think
you need to present rational arguments why the prior consensus was
wrong. I've looked, and I can't find any real attempt to do that.
I have already overturned that *consensus* if there was ever one.
Apparently I disagree, so at least now there can't be a *consensus*
anymore. So that makes it a majority but not a *consensus*.
No, Michael. You are very mistaken about what 'consensus' means.
In English the word "consensus" means something like "general
agreement" or "group decision" or "collective decision". It does _not_
necessarily mean a unanimous descison. In other words, one dissenting
opinion does *not* overturn a conensus of the remainder of the group.

If you don't believe me, check Merriam-Webster Online or Oxford
(askoxford.com) or dictionary.com.

The *consensus* that was reached on this issue is that turning on
IPsec caues a bloody great huge hit in networking performance, and
that we cannot afford that hit. You can yell your disagreement with
that consensus all you wan't, but you arent' going to change or (in
your words) overturn the consensus until you change the minds of a
large majority of the group.


Meanwhile, what has happened instead is that we've attack the facts
which resulted in that prior consensus. By removing or attempting to
remove the signficant performance penalty imposed by IPsec, we remove
the technical justifaction behind that consensus.
Pavel Cahyna
2006-02-20 22:08:53 UTC
Permalink
Post by Jonathan Stone
Michael, I *do* control machines which run GENERIC kernels, I want
to *keep* running GENERIC kernels, but I *do* want a way to turn off
IPv6 on those machines *without* having to build custom kernels.
Does deleting all IPv6 link-local addresses qualify as "turning off IPv6"?

Pavel Cahyna
Rui Paulo
2006-02-20 22:25:35 UTC
Permalink
Post by Pavel Cahyna
Post by Jonathan Stone
Michael, I *do* control machines which run GENERIC kernels, I want
to *keep* running GENERIC kernels, but I *do* want a way to turn off
IPv6 on those machines *without* having to build custom kernels.
Does deleting all IPv6 link-local addresses qualify as "turning off IPv6"?
Some routes are still left even if you delete all those addresses.
--
Rui Paulo <rpaulo@{NetBSD{,-PT}.org,fnop.net}>
Robert Elz
2006-02-21 09:54:00 UTC
Permalink
Date: Mon, 20 Feb 2006 10:53:00 -0800
From: ***@dsg.stanford.edu
Message-ID: <E1FBG9e-0006HT-***@smeg.dsg.stanford.edu>

| But even if you don't use IPsec *at all*, turning on IPsec imposes a
| significant packet-classification on all traffic (except for outbound
| TCP traffic on connected sockets). Even inbound TCP incurs a
| significant penalty, which is easily measurable on network benchmarks.

I can easily believe this, and this is the kind of response I was
looking for in my original query (I was not looking for starting a
debate on LKMs - I really had not imagined LKMs for IPSEC purposes).

[Aside: I know you're not the only, or first, person to have made
this comment, I chose your message to reply to because of other
comments you made.]

What I'd prefer to know though is what "measureable" means, and how
that affects GENERIC. That is, I don't really care much if a GENERIC
kernel isn't suitable for your research - and I really doubt that
you do either. If you want IPSEC (or the KAME IPSEC) disabled, you
know how to make that happen...

I know you described how to measure the performance drop, I'd do it,
except that I don't think I have any systems with the necessary
characteristics (they're all so much faster than the network interfaces
that an extra 100% packet processing cost is barely going to be noticed,
I have nothing with 10G interfaces, and nothing connected to a 1G
interface that's on a switch capable of actually forwarding at Gb
rates...)

However, I can interpret the results before knowing what they are.
That is, for a system, that can just forward packets at N pps,
then for generic purposes, if enabling IPSEC (as it is now
implemented, the KAME version) reduces the handling rate to 0.8 * N
I wouldn't mind in the slightest. (Much) More than that as a reduction
and then I'd be more concerned about having IPSEC on by default,
but it would probably have to reduce to closer to 0.5*N before I'd
be convinced that IPSEC on by default was impossible..

The vast majority of systems never see packet loads anything like
their potential maximum, and a slight slowdown in processing time
wouldn't bother me in the default kernel. Those using the system
for applications where every last cycle is crucial can certainly
remove anything they don't need (and also quite likely, omptimise
for the particular processor they're running on).

Actually that raises a comparison question ... compiling kernels
that support all the versions of x86 processors, or sparc processors
(and probably others where one kernel supports a processor family)
has a bunch of costs, as the low level code needs to be constantly
checking which processor is currently being used to select the
correct code path. How does the IPSEC cost compare to that cost?

I know this is going to be not something easy to measure, as
the two are in largely unrelated parts of the kernel, but people
must have some feel (10% extra processing, 20%, ...)

Aside: and relating to comments in other messages on that
other topic - I don't run GENERIC kernels, I don't care what
options are in them for my own personal use, and I'm not
suggesting that IPSEC be enabled (or asking why it isn't)
in order to avoid needing to compile my own personal kernel.

I'm more concerned with the message it sends - options that
come commented out (disabled) in GENERIC is generally sending
a message that most probably that isn't something that people
really want to have enabled in their systems. On the other
hand, options that are enabled send the message that "this
is here and available, use it if you want, or disable it to
save space and time". I'd kind of prefer that IPSEC send
the 2nd kind of message.

That is, providing that NetBSD's implementation (currently
that means the KAME implementation) is really up to scratch
for the purpose. If it isn't, then that is the answer to
my original question.

| Either way, I find the results extremely discouraging.

How much does it take to discourage you? I tend to have IPSEC
enabled on my systems (used occasionally, though not often), and
have never been bothered by network performance. But nor am I
trying to get the max possible (I even tend to just keep on using
802.11b instead of plugging in 100baseTX fdx, even when I have the
cable sitting on my desk - just because converting between the
two is currently not seamless, and 802.11b performance is just
fine for me...)

| Quite. But speaking just personally, I'm more interested in adding
| /etc/rc.d hooks to not enable utterly useless protocols[*] like
| IPv6 on my machines, than I am in adding IPv6 support to FAST_IPSEC :-/.

For what it is worth, which is very little, I have some students
(undergrad project, so don't expect much) working on a way to really
dynamically remove protocols, while the system is running. After
the knob is turned, IPv6 (or whatever) would (almost) appear as if
it wasn't compiled into the kernel (except probably for connections
already using the protocol - and that would be TCP only). No new
sockets, no new incoming connections to listening sockets, and stuff like
examining the interface address lists, routing tables (from outside
the kernel) would simply not see the disabled protocol at all.

Or that's what I am hoping for ... it will be a year from now before
they've accomplished anything - assuming they ever get anything
that would be useful.

| [*] That is, IPv6 is utterly useless on *my* machines.

You can't just remove "options INET6" from your kernels?

That sounds as if it would solve your immediate problem more
easily than either a "don't bother configuring" or a "remove
this protocol" mechanism in a kernel that had INET6 included.

kre
Nathan J. Williams
2006-02-21 14:04:53 UTC
Permalink
Post by Robert Elz
Actually that raises a comparison question ... compiling kernels
that support all the versions of x86 processors, or sparc processors
(and probably others where one kernel supports a processor family)
has a bunch of costs, as the low level code needs to be constantly
checking which processor is currently being used to select the
correct code path.
Tim Rightnour did a few tests of i386 GENERIC vs
GENERIC-tuned-for-i686 recently. See the thread starting with this
message:

http://mail-index.netbsd.org/tech-perform/2006/02/02/0026.html

The improvement for using the i686-tuned code was in the 2-5% range.

- Nathan
Jonathan Stone
2006-02-21 17:34:43 UTC
Permalink
Post by Robert Elz
I can easily believe this, and this is the kind of response I was
looking for in my original query (I was not looking for starting a
debate on LKMs - I really had not imagined LKMs for IPSEC purposes).
[Aside: I know you're not the only, or first, person to have made
this comment, I chose your message to reply to because of other
comments you made.]
What I'd prefer to know though is what "measureable" means, and how
that affects GENERIC. That is, I don't really care much if a GENERIC
kernel isn't suitable for your research - and I really doubt that
you do either. If you want IPSEC (or the KAME IPSEC) disabled, you
know how to make that happen...
Late last year, I privately sent Thor and a couple of others the same
idea for measuring IPsec overhead. At the time I beleive I quoted
about a 50% penalty, on the machines and network I had, for a
configuration with IPsec enabled, and a handful of SAs active.

In other words, my recollection is that peak UDP receive rates, as
measured by this quick-and-dirty metric (but one passed on by
word-of-mouth in the research community) roughly halved.

I didn't keep careful records, I found the results too depressing.

Someone with more time to spend on this might come up with enough
datapoints to make an interesting paper, by tweaking various points in
design space: disable the PCB cache for IPsec state, then measure
oberhead (peak send rates, on a suitalbe machine/nic/switch combo),
without IPsec, then with and without the PCB cache.

(TCP mandates one ACK every 2 data segments packets, and the PCB cache
only helps for send processing, and only on connected TCP sockets, so
receive processing will incur IPsec overhead (if Ipsec is enabled)
whatever you do.)

Maybe I'll try to salvage a couple of P3 servers to run the tests:/.


Christos... I'm also wondering about Thor's comment about packet
fowarding. I'm assuming Thor's comment is independent of any of my
ad-hoc measurements. My, er, nasty suspicious mind is wondering if
Thor's results are from a low-end or embedded machine with a small
I-cache (say, 16k or less).

If so, then just calling into the IPsec codepath (even if the IPsec
code path checks for no active SADB/SDB entries where I recall it
does), might trigger enough additional I-cache capacity misses to
cause a noticeable penatly in peak packet-rate forwarding, for a
CPU-limited (or I-cache-limited) packet forwarder. OTOH, I'm doubtful
about how many of us would acutally run GENERIC kernels on such a
packet-forwarder.


Now I'm running late, I'll add a couple more comments this evening,
local time, once I get back home.
Christos Zoulas
2006-02-21 17:47:12 UTC
Permalink
On Feb 21, 9:34am, ***@Pescadero.dsg.stanford.edu (Jonathan Stone) wrote:
-- Subject: Re: IPSEC in GENERIC

| I didn't keep careful records, I found the results too depressing.

I agree here.

| Someone with more time to spend on this might come up with enough
| datapoints to make an interesting paper, by tweaking various points in
| design space: disable the PCB cache for IPsec state, then measure
| oberhead (peak send rates, on a suitalbe machine/nic/switch combo),
| without IPsec, then with and without the PCB cache.
|
| (TCP mandates one ACK every 2 data segments packets, and the PCB cache
| only helps for send processing, and only on connected TCP sockets, so
| receive processing will incur IPsec overhead (if Ipsec is enabled)
| whatever you do.)
|
| Maybe I'll try to salvage a couple of P3 servers to run the tests:/.
|
|
| Christos... I'm also wondering about Thor's comment about packet
| fowarding. I'm assuming Thor's comment is independent of any of my
| ad-hoc measurements. My, er, nasty suspicious mind is wondering if
| Thor's results are from a low-end or embedded machine with a small
| I-cache (say, 16k or less).
|
| If so, then just calling into the IPsec codepath (even if the IPsec
| code path checks for no active SADB/SDB entries where I recall it
| does), might trigger enough additional I-cache capacity misses to
| cause a noticeable penatly in peak packet-rate forwarding, for a
| CPU-limited (or I-cache-limited) packet forwarder. OTOH, I'm doubtful
| about how many of us would acutally run GENERIC kernels on such a
| packet-forwarder.

With my changes (untested so far) you don't call into the ipsec
code unless there are SPD's active. So I think that you get the
benefit of not being penalized of having IPSEC compiled into the
kernel if you are not using it. I also think that FAST_IPSEC is a
lot faster than the regular IPSEC. I wish I had the time to run
some benchmarks on all of this...

christos
j***@dsg.stanford.edu
2006-02-21 20:55:29 UTC
Permalink
Post by Christos Zoulas
With my changes (untested so far) you don't call into the ipsec
code unless there are SPD's active. So I think that you get the
benefit of not being penalized of having IPSEC compiled into the
kernel if you are not using it.
hi Christos,

What I was trying to say last time boils down to:

I think it's best to put the test for "no IPsec active, therefore none
needed" directly into ip_input() and ip6_input().

I'm not sure about forwarding, I'd have to look. If dimming memory
serves, the API from ipsec[46]_in_reject() *always* returns a non-NULL
policy, so what you'd want is to check for only the default policy
existing, and if so, skip all the ipsec*_in_reject goop completely.

That way, you might be able to still set the M_FASTFORWARD label in
the packet tag. (Which, reading ahead, turns out to be probably what
Thor wants.)

OTOH, once we start on that route, if it were up to me, I'd define a
tiny API with inline functions to test for "no IPsec or default-only
IPsec in effect".

That way FAST_IPSEC and ex-KAME IPsec could use the same API for the
test, and thus hopefully share one block of code. (The inline
functions isolate any differences in global-variable names, expected
values for "no IPsec active", etc).

I once intended to unify the ifdefs for IPsec in much the way
FreeBSD has done, so that what's now two sets of tests would (almost)
all become:

#if defined(IPSEC) || defined(FAST_IPSEC)
... code which calls common API into whichever IPsec
was configured at config-time
#endif

that approach becomes much more feasible, now that we have a sane,
unified struct in_pcbhdr. All the pointer-chasing from pcb to socket,
and then (three functions later) back again, in all those

switch (so->so_proto->pr_domain->dom_family) {

fragmets, to discriminate IPv4 PCBs from IPv6 6to4(?)[*] PCBs, really
should go away, and (now that pcbhdr lets us disambiguate IPv6 traffic
from IPv4), we should go back to passing PCBs around, instead of
passing sockets, and following pointers as in the line I quote above.

I'd welcome constructive comments on that idea.

[*] That is, the code with the somewhat-cryptic comment:

* NOTE: IPv6 mapped adddress concern is implemented here.

which is, I beleive, what broke last time I tried this particular code
reorg.
Christos Zoulas
2006-02-21 20:59:35 UTC
Permalink
On Feb 21, 12:55pm, ***@dsg.stanford.edu (***@dsg.stanford.edu) wrote:
-- Subject: Re: IPSEC in GENERIC

| hi Christos,
|
| What I was trying to say last time boils down to:
|
| I think it's best to put the test for "no IPsec active, therefore none
| needed" directly into ip_input() and ip6_input().
|
| I'm not sure about forwarding, I'd have to look. If dimming memory
| serves, the API from ipsec[46]_in_reject() *always* returns a non-NULL
| policy, so what you'd want is to check for only the default policy
| existing, and if so, skip all the ipsec*_in_reject goop completely.
|
| That way, you might be able to still set the M_FASTFORWARD label in
| the packet tag. (Which, reading ahead, turns out to be probably what
| Thor wants.)

Thanks, that is what I am doing.

| OTOH, once we start on that route, if it were up to me, I'd define a
| tiny API with inline functions to test for "no IPsec or default-only
| IPsec in effect".
|
| That way FAST_IPSEC and ex-KAME IPsec could use the same API for the
| test, and thus hopefully share one block of code. (The inline
| functions isolate any differences in global-variable names, expected
| values for "no IPsec active", etc).
|
| I once intended to unify the ifdefs for IPsec in much the way
| FreeBSD has done, so that what's now two sets of tests would (almost)
| all become:
|
| #if defined(IPSEC) || defined(FAST_IPSEC)
| ... code which calls common API into whichever IPsec
| was configured at config-time
| #endif
|
| that approach becomes much more feasible, now that we have a sane,
| unified struct in_pcbhdr. All the pointer-chasing from pcb to socket,
| and then (three functions later) back again, in all those
|
| switch (so->so_proto->pr_domain->dom_family) {

Yes, this is what rui is working on (merging inpcb/in6pcb).

| fragmets, to discriminate IPv4 PCBs from IPv6 6to4(?)[*] PCBs, really
| should go away, and (now that pcbhdr lets us disambiguate IPv6 traffic
| from IPv4), we should go back to passing PCBs around, instead of
| passing sockets, and following pointers as in the line I quote above.
|
| I'd welcome constructive comments on that idea.
|
| [*] That is, the code with the somewhat-cryptic comment:
|
| * NOTE: IPv6 mapped adddress concern is implemented here.
|
| which is, I beleive, what broke last time I tried this particular code
| reorg.

We all agree, so this change should not be controversial.

christos
j***@dsg.stanford.edu
2006-02-21 21:08:33 UTC
Permalink
In message <***@rebar.astron.com>,
Christos Zoulas writes:

[...]
Post by Christos Zoulas
| I once intended to unify the ifdefs for IPsec in much the way
| FreeBSD has done, so that what's now two sets of tests would (almost)
|
| #if defined(IPSEC) || defined(FAST_IPSEC)
| ... code which calls common API into whichever IPsec
| was configured at config-time
| #endif
|
| that approach becomes much more feasible, now that we have a sane,
| unified struct in_pcbhdr. All the pointer-chasing from pcb to socket,
| and then (three functions later) back again, in all those
|
| switch (so->so_proto->pr_domain->dom_family) {
Yes, this is what rui is working on (merging inpcb/in6pcb).
I'm not familiar with that.
Post by Christos Zoulas
| fragmets, to discriminate IPv4 PCBs from IPv6 6to4(?)[*] PCBs, really
| should go away, and (now that pcbhdr lets us disambiguate IPv6 traffic
| from IPv4), we should go back to passing PCBs around, instead of
| passing sockets, and following pointers as in the line I quote above.
|
| I'd welcome constructive comments on that idea.
|
|
| * NOTE: IPv6 mapped adddress concern is implemented here.
|
| which is, I beleive, what broke last time I tried this particular code
| reorg.
We all agree, so this change should not be controversial.
Exactly what is rui doing? If I rescuscitated and reapplied the patch
which (not grokking the full import of that cryptic comment) I
committed and (in hindsight) Itojun quite correctly backed out,
how far does that get us along the way?

Or to turn that around: what else is is in-progress that that'd break?
Christos Zoulas
2006-02-21 21:19:52 UTC
Permalink
On Feb 21, 1:08pm, ***@dsg.stanford.edu (***@dsg.stanford.edu) wrote:
-- Subject: Re: IPSEC in GENERIC

| Exactly what is rui doing? If I rescuscitated and reapplied the patch
| which (not grokking the full import of that cryptic comment) I
| committed and (in hindsight) Itojun quite correctly backed out,
| how far does that get us along the way?

He is merging inpcb and in6pcb, the same way FreeBSD and OpenBSD have done.

| Or to turn that around: what else is is in-progress that that'd break?

This will break his branch, and it will make us look very different than
FreeBSD and OpenBSD which have a merged inpcb/in6pbc.

christos
Jonathan Stone
2006-02-21 22:08:12 UTC
Permalink
Post by Christos Zoulas
He is merging inpcb and in6pcb, the same way FreeBSD and OpenBSD have done.
| Or to turn that around: what else is is in-progress that that'd break?
This will break his branch,
Sure. But if we finished clean-implmented code to using the struct
inpcbhdr, and removes all the pointer-jumping, then we don't *need*
Rui's branch, do we?

To turn my question around: after my first pass at this, we discussed
various approaches: merging pcbs, or introducing a common prefix.

What changed beetwen the introduction of struct inpbcbhdr and now, so
as to make a unified inpcb/in6pcb desirable now, when back then, that
wasn't a desirable approach?
Post by Christos Zoulas
and it will make us look very different than
FreeBSD and OpenBSD which have a merged inpcb/in6pbc.
FreeBSD's networking stack is already so different (SMP locking, ...)
that as I see it, this marginal compatibility with FreeBSD buys us
nothing, in practical terms. Is there something in OpenBSD which this
buys us?

I guess I'm partly confused becuase, from where I'm sitting, a common
header followed by IPv4 or IPv6-specific portions seems cleaner and
more in our spirit of "doing things right".
Jason Thorpe
2006-02-21 22:13:37 UTC
Permalink
Post by Jonathan Stone
I guess I'm partly confused becuase, from where I'm sitting, a common
header followed by IPv4 or IPv6-specific portions seems cleaner and
more in our spirit of "doing things right".
An even better way to do it would be one suggested by Matt Thomas --
IPv4 as a separate code base simply goes away, and instead takes the
mapped-address path through the IPv6 stack.

-- thorpej
Christos Zoulas
2006-02-21 22:18:25 UTC
Permalink
On Feb 21, 2:13pm, ***@shagadelic.org (Jason Thorpe) wrote:
-- Subject: Re: IPSEC in GENERIC

| On Feb 21, 2006, at 2:08 PM, Jonathan Stone wrote:
|
| > I guess I'm partly confused becuase, from where I'm sitting, a common
| > header followed by IPv4 or IPv6-specific portions seems cleaner and
| > more in our spirit of "doing things right".
|
| An even better way to do it would be one suggested by Matt Thomas --
| IPv4 as a separate code base simply goes away, and instead takes the
| mapped-address path through the IPv6 stack.

This would be ideal! Let's get mapped addresses working first please :-)

christos
Jonathan Stone
2006-02-21 23:24:03 UTC
Permalink
Post by Jason Thorpe
Post by Jonathan Stone
I guess I'm partly confused becuase, from where I'm sitting, a common
header followed by IPv4 or IPv6-specific portions seems cleaner and
more in our spirit of "doing things right".
An even better way to do it would be one suggested by Matt Thomas --
IPv4 as a separate code base simply goes away, and instead takes the
mapped-address path through the IPv6 stack.
Yeah, but only if you're prepared to use AF_INET6's ipv4-mapped
addresses. It's a non-starter for existing applications which create
PF_INET sockets and bind()/connect() them to PF_INET addresses. Or
for people who want to comment out INET6 from their kernels.

Besides that .... sure, I guess. But if we go that route, would we
have been better off going with the NRL stack in the first place?
Christos Zoulas
2006-02-21 22:17:37 UTC
Permalink
On Feb 21, 2:08pm, ***@Pescadero.dsg.stanford.edu (Jonathan Stone) wrote:
-- Subject: Re: IPSEC in GENERIC

| >He is merging inpcb and in6pcb, the same way FreeBSD and OpenBSD have done.
| >
| >| Or to turn that around: what else is is in-progress that that'd break?
| >
| >This will break his branch,
|
| Sure. But if we finished clean-implmented code to using the struct
| inpcbhdr, and removes all the pointer-jumping, then we don't *need*
| Rui's branch, do we?
|
| To turn my question around: after my first pass at this, we discussed
| various approaches: merging pcbs, or introducing a common prefix.
|
| What changed beetwen the introduction of struct inpbcbhdr and now, so
| as to make a unified inpcb/in6pcb desirable now, when back then, that
| wasn't a desirable approach?

1. Makes our code similar to other BSD's.
2. Without it we cannot make mapped addresses work easily and this breaks
java.
3. It eliminates a lot of common code.

| >and it will make us look very different than
| >FreeBSD and OpenBSD which have a merged inpcb/in6pbc.
|
| FreeBSD's networking stack is already so different (SMP locking, ...)
| that as I see it, this marginal compatibility with FreeBSD buys us
| nothing, in practical terms. Is there something in OpenBSD which this
| buys us?

The major issues are SMP locking and the function dispatch instead of switch in
ctlinput(). I like the function dispatcher more and I think it is simpler
to read and maintain. Once we have merged pcb's it will be easy to move
to that.

Practically making our code diverge from the others for no good reason
increases our maintenance cost. There aren't many people who have the
time and expertise to devote to improving the NetBSD stack...

| I guess I'm partly confused becuase, from where I'm sitting, a common
| header followed by IPv4 or IPv6-specific portions seems cleaner and
| more in our spirit of "doing things right".

The problem here is that you end up with 2 pcb structures that need to
be passed back and forth between v4 and v6 code in the mapped address
case.

christos
Rui Paulo
2006-02-21 22:45:28 UTC
Permalink
Post by Christos Zoulas
-- Subject: Re: IPSEC in GENERIC
| >He is merging inpcb and in6pcb, the same way FreeBSD and OpenBSD have done.
| >
| >| Or to turn that around: what else is is in-progress that that'd break?
| >
| >This will break his branch,
|
| Sure. But if we finished clean-implmented code to using the struct
| inpcbhdr, and removes all the pointer-jumping, then we don't *need*
| Rui's branch, do we?
|
| To turn my question around: after my first pass at this, we discussed
| various approaches: merging pcbs, or introducing a common prefix.
|
| What changed beetwen the introduction of struct inpbcbhdr and now, so
| as to make a unified inpcb/in6pcb desirable now, when back then, that
| wasn't a desirable approach?
1. Makes our code similar to other BSD's.
2. Without it we cannot make mapped addresses work easily and this breaks
java.
3. It eliminates a lot of common code.
| >and it will make us look very different than
| >FreeBSD and OpenBSD which have a merged inpcb/in6pbc.
|
| FreeBSD's networking stack is already so different (SMP locking, ...)
| that as I see it, this marginal compatibility with FreeBSD buys us
| nothing, in practical terms. Is there something in OpenBSD which this
| buys us?
The major issues are SMP locking and the function dispatch instead of switch in
ctlinput(). I like the function dispatcher more and I think it is simpler
to read and maintain. Once we have merged pcb's it will be easy to move
to that.
Practically making our code diverge from the others for no good reason
increases our maintenance cost. There aren't many people who have the
time and expertise to devote to improving the NetBSD stack...
| I guess I'm partly confused becuase, from where I'm sitting, a common
| header followed by IPv4 or IPv6-specific portions seems cleaner and
| more in our spirit of "doing things right".
The problem here is that you end up with 2 pcb structures that need to
be passed back and forth between v4 and v6 code in the mapped address
case.
On HEAD, we have 3 structs. The inpcbhdr, inpcb and in6pcb.

What I am doing is merging then in one, 'inpcb', struct.

http://cvsweb.netbsd.org/bsdweb.cgi/~checkout~/src/sys/netinet/in_pcb.h?rev=1.41.4.7&content-type=text/plain
--
Rui Paulo <rpaulo@{NetBSD{,-PT}.org,fnop.net}>
Jonathan Stone
2006-02-21 23:19:57 UTC
Permalink
Post by Rui Paulo
On HEAD, we have 3 structs. The inpcbhdr, inpcb and in6pcb.
What I am doing is merging then in one, 'inpcb', struct.
Yes, I understood that much quite plainly from Christos' message.
What I still don't understand is *why* we'd want to do that.

Or more precisely, I don't know what has changed from the last time we
tried to do clean-up in this area, beacuse back then the consensus was
to create the inpcbhdr, and to push that change further through the code.
(I vaguley recall that merging the pcbs was ruled out, but I'm not 100% sure).

Reading between the lines of what Christos wrote, I'm guessing that:

0. We have a strong desire to ship prebuilt, NetBSD-native
binary JVM executables forall our ports;

1. We need to support IPv4-mapped IPv6 addresses to pass the Java
conformance suite which we have to pass to acheive #1;

2. Core has therefore considerably softened the earlier stance,
which was along the lines of "IPv4 mapped IPv6 addresses
considered harmfull." [NB, that's the acutal title
of a repeatedly-issued internet draft written by a former Core member]

3. Somehow, the inpbhdr/inpcb/in6pcb method doesn't cut it for the
IPv4-mapped PF_INET6 addreses which [as I try to read between the lines]
we need to ship prebuilt Java binaries.


Am I close?
Jonathan Stone
2006-02-21 21:12:11 UTC
Permalink
Post by Christos Zoulas
-- Subject: Re: IPSEC in GENERIC
| I didn't keep careful records, I found the results too depressing.
I agree here.
Since kre asked for quantitative datapoints...

what I recall is that turning on IPSEC gave roughly a factor of 2 hit
in UDP packets received and delivered to userspace. I honestly don't
recall if that was FAST_IPSEC, or KAME IPsec.
Bill Studenmund
2006-02-21 21:18:09 UTC
Permalink
Post by Jonathan Stone
Post by Christos Zoulas
-- Subject: Re: IPSEC in GENERIC
| I didn't keep careful records, I found the results too depressing.
I agree here.
Since kre asked for quantitative datapoints...
what I recall is that turning on IPSEC gave roughly a factor of 2 hit
in UDP packets received and delivered to userspace. I honestly don't
recall if that was FAST_IPSEC, or KAME IPsec.
And how recent was that? I have been informed that recent changes make
KAME (in NetBSD) not suck as bad with the SPD lookup, so number could need
reevaluating.

Take care,

Bill
Thor Lancelot Simon
2006-02-21 19:55:26 UTC
Permalink
Post by Jonathan Stone
Christos... I'm also wondering about Thor's comment about packet
fowarding. I'm assuming Thor's comment is independent of any of my
ad-hoc measurements. My, er, nasty suspicious mind is wondering if
Thor's results are from a low-end or embedded machine with a small
I-cache (say, 16k or less).
That's right -- I saw approximately a 20% drop in throughput with 1K
datagrams flowing across a Soekris net4501.

I dug into it a little bit, but I'm actually having trouble remembering
what I found. I think, among other things, at the time when I looked
(well over a year ago) options IPSEC may have disabled the fast-path
forwarding code entirely.

Thor
j***@dsg.stanford.edu
2006-02-20 19:43:59 UTC
Permalink
Post by Thor Lancelot Simon
IPsec hooks in all over the network code -- it is anything _but_ a "bump
in the stack" implementation. That makes it useful for more than toy
VPN applications (unlike many BITS implementations) but also means that
it is extremely difficult to cleanly separate out into a module, _and_
that just including it in the kernel causes a measurable decrease in
forwarding performance. Which is why it's not in the kernel by default.
Thor,

If there are no SPDs (or non except the compiled-in default SPDs),
then surely the forwarding code could do an inline check the SPD
count, and if no IPsec processing is possible (because there are no
non-default SPDs, and thus no IPsec processing to do), then just skip
over all IPsec processing entirely?

Surely the overhead of comparing an int against a constant, and one
extra branch, shouldn't add _that_ much overhead. (or two checks,
if we need to check for a matching SA also?).

Am I missing something?
Bill Studenmund
2006-02-21 19:19:25 UTC
Permalink
Post by Thor Lancelot Simon
Post by Garrett D'Amore
Post by j***@britannica.bec.de
But back to the original question -- this doesn't affect IPSec at all,
since it can't be made a module without a lot of efforts in any case.
true, perhaps. but if so, then why? it seems a lot of ipsec at least
could be -- e.g. encryption and hash routines, etc.
Except that those routines are almost always in anyway.
IPsec hooks in all over the network code -- it is anything _but_ a "bump
in the stack" implementation. That makes it useful for more than toy
VPN applications (unlike many BITS implementations) but also means that
it is extremely difficult to cleanly separate out into a module, _and_
that just including it in the kernel causes a measurable decrease in
forwarding performance. Which is why it's not in the kernel by default.
This issue is one thing that I think Linux got right. Most features are
not binary but tertiary, with "Yes" (compile in), "No" (not at all), and
"Module." The latter adds all of the hooks you mention but does not add
the actual code. The hooks default to a do-nothing (or do extremely
little) behavior and only turn on when the module is present.

I think we could gain a lot by following a similar path. Some things just
can't totally drop in, provision had to have been made before hand.

Take care,

Bill
Igor Sobrado
2006-02-20 18:50:20 UTC
Permalink
At some time in the near future, IPsec will be required in GENERIC
kernels even if it affects packet forwarding performance. I would
suggest GENERIC and GENERIC_LAPTOP uniprocessor kernels with ACPI
support too. These kernels should not waste a lot of space on the
NetBSD ISO images and are certainly useful. On the other hand,
I do not know how mature is ACPI code now.

IPsec is becoming the standard for VPNs really fast and it is a
required component for IPv6. IPsec is only optional for IPv4.

Cheers,
Igor.
Martin S. Weber
2006-02-20 19:03:14 UTC
Permalink
Post by Igor Sobrado
At some time in the near future, IPsec will be required in GENERIC
kernels (...)
IPsec is becoming the standard for VPNs really fast and it is a
required component for IPv6. IPsec is only optional for IPv4.
(aha. Thus ipsec is only optional but will be required ? Why ?)

Rather the question should be, why is INET6 in GENERIC?

(to achieve a deployment it doesn't have and never will ?)

-Martin
Steven M. Bellovin
2006-02-20 19:14:28 UTC
Permalink
Post by Martin S. Weber
Rather the question should be, why is INET6 in GENERIC?
(to achieve a deployment it doesn't have and never will ?)
I'm probably going to regret posting this...

A certain large vendor in the upper-left corner of the US is strongly
committed to IPv6. Its next release, Vista, will *prefer* v6
connections if they're available. Rumor has it that their future plans
*really* need v6. We're better off beating on our stack now,
especially anything to do with operating in a mixed environment.

--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
j***@dsg.stanford.edu
2006-02-20 19:20:32 UTC
Permalink
Post by Steven M. Bellovin
A certain large vendor in the upper-left corner of the US is strongly
committed to IPv6. Its next release, Vista, will *prefer* v6
connections if they're available. Rumor has it that their future plans
*really* need v6. We're better off beating on our stack now,
especially anything to do with operating in a mixed environment.
Steven, in all honesty, that's got to be the most compelling
argument for turning our IPv6 support into a loadable module
that I've heard to date.
Steven M. Bellovin
2006-02-20 19:30:08 UTC
Permalink
Post by j***@dsg.stanford.edu
Post by Steven M. Bellovin
A certain large vendor in the upper-left corner of the US is strongly
committed to IPv6. Its next release, Vista, will *prefer* v6
connections if they're available. Rumor has it that their future plans
*really* need v6. We're better off beating on our stack now,
especially anything to do with operating in a mixed environment.
Steven, in all honesty, that's got to be the most compelling
argument for turning our IPv6 support into a loadable module
that I've heard to date.
I'm not sure how you mean that, but if you mean it's easier to do
development and testing when it's a much shorter cycle I certainly
agree. Of course, I'm all for LKMs as well.

Off to class now, and just in time...

--Steven M. Bellovin, http://www.cs.columbia.edu/~smb
Martin S. Weber
2006-02-20 19:23:35 UTC
Permalink
Post by Steven M. Bellovin
Post by Martin S. Weber
Rather the question should be, why is INET6 in GENERIC?
(to achieve a deployment it doesn't have and never will ?)
I'm probably going to regret posting this...
A certain large vendor in the upper-left corner of the US
. ^^^^^^^^^^ hehe :)
Post by Steven M. Bellovin
is strongly
committed to IPv6. Its next release, Vista, will *prefer* v6
connections if they're available.
If they're available. I'd have to tunnel somewhere out of
country to find someone to talk ipv6 to and then tunnel all
the way back on a trustworthy protocol which has grown in and
by use and whose abusability is what makes it still live. I
doubt something "properly designed" will survive as long :)
Post by Steven M. Bellovin
Rumor has it that their future plans
*really* need v6.
Sure. If every mac needs to be traced back so you can see
if they paid for their visas, er, vistas, then you probably
need it. "We don't route you. Go pay for your OS. :P"
Post by Steven M. Bellovin
We're better off beating on our stack now,
especially anything to do with operating in a mixed environment.
Still, you need someone to talk to for a communication protocol.
And I'm not the one calling my isp "Hey. I want TIA-compatible
surveillance id obvious to the rest of the world, PLEASE."

Hell, I can make my lan run ipv6 (wtf ??)

-Martin
Ignatios Souvatzis
2006-02-20 20:00:30 UTC
Permalink
I'd have to tunnel somewhere out of country to find someone to talk ipv6
[...]
If I read your mail header correclty, you'd have to possibly select
a different ADSL access network, then choose among at least two ISPs who
offer dual IPv4/IPv6 on top of that.
Sure. If every mac needs to be traced back so you can see
if they paid for their visas, er, vistas, then you probably
need it. "We don't route you. Go pay for your OS. :P"
The OS vendors don't need IPv6 for that, anyway. They one we're talking
about already does it with their current OS version even if you're only
connected to the legacy Internet, or not at all.
[...]
And I'm not the one calling my isp "Hey. I want TIA-compatible
surveillance id obvious to the rest of the world, PLEASE."
You seem not to have noticed that

a) they can do that anyway with IPv4 and other means with _their_ OS, as
you can't control what happens inside their kernel

b) you can give your machine any IPv6 address suffix you like, if you
configure it manually or via DHCPv6

c) they're the only ones I'm aware of who actually implemented the "create
a new randomized suffix every few minutes" (IPv6 privacy option)
mechanism.

Whatever evil they might be thinking of in other areas, this isn't one of
them.

-is
der Mouse
2006-02-21 00:26:34 UTC
Permalink
Post by Steven M. Bellovin
*really* need v6.
Sure. If every mac needs to be traced back [...]
And I'm not the one calling my isp "Hey. I want TIA-compatible
surveillance id obvious to the rest of the world, PLEASE."
Just because some of the crazier v6 proponents want to take 128 bits of
address and promptly throw half of them away (using them for autoconfig
instead of addressing) doesn't mean *you* have to. My own v6 addresses
have nothing to do with my machines' MAC addresses (if any); my
upstream assigned me a /56 out of their /48 and I'm actually using all
of a /96 of it.

/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML ***@rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
David Young
2006-02-20 19:53:13 UTC
Permalink
Post by Steven M. Bellovin
Post by Martin S. Weber
Rather the question should be, why is INET6 in GENERIC?
(to achieve a deployment it doesn't have and never will ?)
I'm probably going to regret posting this...
A certain large vendor in the upper-left corner of the US is strongly
committed to IPv6. Its next release, Vista, will *prefer* v6
connections if they're available. Rumor has it that their future plans
*really* need v6. We're better off beating on our stack now,
especially anything to do with operating in a mixed environment.
Just to amplify what Steven is saying, it's no secret that Windows XP
already supports IPv6 and one or more migration protocols (Teredo? 6to4?).
Mac OS X, too, supports IPv6 and 6to4.

My OS X laptop, my NetBSD desktop, and my various development servers
have global IPv6 reachability, in spite of my IPv4-only ISP, thanks in
no small part to Microsoft's 6to4 gateway.

Speaking as one who had to wrangle a NAT firewall recently to get an
application to work, I look forward to the day when an IPv6 connection
is the norm, and NAT is a distant memory.

Dave
--
David Young OJC Technologies
***@ojctech.com Urbana, IL * (217) 278-3933
Pavel Cahyna
2006-02-20 19:57:10 UTC
Permalink
Post by David Young
Post by Steven M. Bellovin
Post by Martin S. Weber
Rather the question should be, why is INET6 in GENERIC?
(to achieve a deployment it doesn't have and never will ?)
I'm probably going to regret posting this...
A certain large vendor in the upper-left corner of the US is strongly
committed to IPv6. Its next release, Vista, will *prefer* v6
connections if they're available. Rumor has it that their future plans
*really* need v6. We're better off beating on our stack now,
especially anything to do with operating in a mixed environment.
Just to amplify what Steven is saying, it's no secret that Windows XP
already supports IPv6 and one or more migration protocols (Teredo? 6to4?).
Both.

Pavel
Igor Sobrado
2006-02-20 19:57:48 UTC
Permalink
Post by Martin S. Weber
Post by Igor Sobrado
At some time in the near future, IPsec will be required in GENERIC
kernels (...)
IPsec is becoming the standard for VPNs really fast and it is a
required component for IPv6. IPsec is only optional for IPv4.
(aha. Thus ipsec is only optional but will be required ? Why ?)
IPsec is an optional component on IPv4, but a required component
on IPv6. Security services are an integral part of IPv6, IPv4 was
not designed with security in mind. That is the reason a lot of
incompatible security proposals are available for IPv4.
Post by Martin S. Weber
Rather the question should be, why is INET6 in GENERIC?
IPv6 is a fine and powerful protocol, much faster[*] and expandable
than IPv4. It is certainly a protocol we should start using very soon.
On the other hand, KAME project did a fine implementation of IPv6 even if,
as observed in this thread, IPsec requires some work to be done yet for
performance and scalability improvements.

[*] there are few fields in the default IPv6 header and flow labels
allow routers to send traffic without a lot of calculation. Only
the first package on a flow needs to be fully processed by routers.

In fact, IPv6 simplifies network management and protocol specification:
for example, the current broadcast addressing scheme is a special
case of multicast addressing in IPv6. There are a lot of new and
very useful features, as the new anycast addressing model.
Post by Martin S. Weber
(to achieve a deployment it doesn't have and never will ?)
We need a serious migration effort. I think that the _true reasons_
IPv6 is not being widely used yet are economical and not technical.
ISPs ask for money to people that want static IP addresses for their
servers. DDNS servers are a poor workaround. With IPv6 this requirement
will dissapear (as NAT itself). Recent NICs support IEEE EUI-64 identifiers
and there is a method to map 48-bit IEEE MAC addresses to the IPv6 host ID
field for those NICs that do not provide EUI-64 identifiers (e.g., with
the surface mount component DS2502P-E64 from Dallas Semiconductor).
In both cases, we can assure that if our machine has a NIC we have a valid
static IPv6 address on any network. With more than three million IPv6
addresses on each square cm, there is not a requirement for dynamic addresses
anymore!

Cheers,
Igor.
Loading...