Discussion:
patching for 2010-002: I feel like an idiot, but this isn't working
Anne Bennett
2010-03-11 23:05:23 UTC
Permalink
I'm finally catching up with security advisories applicable
to 5.0.1 since I installed it the first week of January, but I
can't seem to get the updates to openssl for advisory 2010-002.
# cd src
# cvs update -r <branch_name> -d -P crypto/dist/openssl/ssl
Here's what happened:
# cd /usr/src
# cvs update -r netbsd-5-0-1-RELEASE -d -P crypto/dist/openssl/ssl
cvs update: Updating crypto/dist/openssl/ssl

... and no files were updated.

Furthermore, when I used the web-based CVS browser to check for the
files, none of the files listed at this URL:

http://cvsweb.netbsd.org/bsdweb.cgi/src/crypto/dist/openssl/ssl/?hideattic=0&sortby=date&only_with_tag=netbsd-5-0-1-RELEASE#dirlist

were recent at all.

I feel like an idiot, but... what obvious thing have I misunderstood here?

Anne.
Andy Ruhl
2010-03-11 23:12:19 UTC
Permalink
On Thu, Mar 11, 2010 at 4:05 PM, Anne Bennett
Post by Anne Bennett
I'm finally catching up with security advisories applicable
to 5.0.1 since I installed it the first week of January, but I
can't seem to get the updates to openssl for advisory 2010-002.
               # cd src
               # cvs update -r <branch_name> -d -P crypto/dist/openssl/ssl
 # cd /usr/src
 # cvs update -r netbsd-5-0-1-RELEASE -d -P crypto/dist/openssl/ssl
 cvs update: Updating crypto/dist/openssl/ssl
... and no files were updated.
Furthermore, when I used the web-based CVS browser to check for the
 http://cvsweb.netbsd.org/bsdweb.cgi/src/crypto/dist/openssl/ssl/?hideattic=0&sortby=date&only_with_tag=netbsd-5-0-1-RELEASE#dirlist
were recent at all.
Caution: Low effort answer

Did you check against release-5 by chance? Maybe it's a pullup problem
or something.

Andy
Andy Ruhl
2010-03-11 23:55:28 UTC
Permalink
On Thu, Mar 11, 2010 at 4:44 PM, Anne Bennett
No, I had not; I don't really follow the difference between
all these branch numebrs (sorry to be lazy; I'm not a regular
user of CVS), so I just stick to my known branch (per CVS/Tags)
unless otherwise instructed in the advisory.
My general opinion is to just follow the release branch. I can't even
remember the last time the NetBSD team released something that
introduced a big problem for me as part of this stream. So I mostly
just go to one of the daily builds on that branch, for example:

ftp://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/201003110000Z/i386/installation/

I'll grab install media, or even just an install kernel and reboot
into the installer and do a binary upgrade of the entire system. It
takes, what, 30 minutes or less these days?

And I maintain the release-5 code through CVS and use my kernel config
file to build a new kernel as per my config at the same time.

Seems much simpler and cleaner than messing around with patching
whatever it is that they patched...

Andy
Anne Bennett
2010-03-11 23:44:47 UTC
Permalink
Post by Andy Ruhl
I can't seem to get the updates to openssl for advisory 2010-002.
[...]
Post by Andy Ruhl
 # cd /usr/src
 # cvs update -r netbsd-5-0-1-RELEASE -d -P crypto/dist/openssl/ssl
 cvs update: Updating crypto/dist/openssl/ssl
... and no files were updated.
Did you check against release-5 by chance?
No, I had not; I don't really follow the difference between
all these branch numebrs (sorry to be lazy; I'm not a regular
user of CVS), so I just stick to my known branch (per CVS/Tags)
unless otherwise instructed in the advisory.

Thanks for your suggestion; I looked at various branches
on cvsweb (netbsd-5-0-RELEASE, netbsd-5-0, netbsd-5-base),
and finally found recently modified files in "netbsd-5",
so I used that one:

# cvs update -r netbsd-5 -d -P crypto/dist/openssl/ssl
U crypto/dist/openssl/ssl/d1_both.c
U crypto/dist/openssl/ssl/d1_pkt.c
U crypto/dist/openssl/ssl/s3_lib.c
U crypto/dist/openssl/ssl/s3_pkt.c
U crypto/dist/openssl/ssl/s3_srvr.c
U crypto/dist/openssl/ssl/ssl.h
U crypto/dist/openssl/ssl/ssl_err.c
U crypto/dist/openssl/ssl/ssl_locl.h
Post by Andy Ruhl
Maybe it's a pullup problem or something.
Presumably so (whatever that is!). ;-)

Anyway, I trust that if possible this will be fixed by someone
who sees this message, or the advisory will be updated to show
more detailed instructions if that's more appropriate.

With luck the compilation part will be trivial! (Going well so far.)

Thanks very much,

Anne.
Greg Troxel
2010-03-12 00:49:16 UTC
Permalink
No, I had not; I don't really follow the difference between
all these branch numebrs (sorry to be lazy; I'm not a regular
user of CVS), so I just stick to my known branch (per CVS/Tags)
unless otherwise instructed in the advisory.

CVS has the concept of branch tags and point tags. Point tags denote a
specific version of each file and (more or less) never change. Branch
tags denote a logical collection of files which can be updated.
NetBSD's policy is that release branches are updated with care.

current is the trunk of CVS, and thus not really a branch. This is not
necessarily stable, and not intended for production.

netbsd-5 is a branch created before NetBSD 5.0 was released. When 5.0
was released, the point tag netbsd-5-0-RELEASE was created. The branch
netbsd-5-0 was created rooted at that point tag.

5.1 will be tagged along netbsd-5. Then netbsd-5-1 will be created at
that point.

So, you can follow netbsd-5-0 and get software versions along an arc
From 5.0 to 5.0.1 to 5.0.2 and so on. Updates to branches like
netbsd-5-0 are extremely conservative.

Or you can follow netbsd-5, which is on an arc from 5.0 to 5.1 etc.
These updates are still conservative, and I regularly update along
netbsd-5, rebuild and reinstall the entire system.

I think it is best to update the entire tree to the new branch and do a
full rebuild, but it's probably safe to update parts, as the basic
release engineering policy is not to have interface changes on release
branches.
Dennis den Brok
2010-03-11 23:22:35 UTC
Permalink
Post by Anne Bennett
I feel like an idiot, but... what obvious thing have I misunderstood here?
The branch you want is "netbsd-5-0", not "netbsd-5-0-1-RELEASE".
The latter contains precisely the release version of 5.0.1.

HTH,

Dennis den Brok
Greg Troxel
2010-03-12 00:14:46 UTC
Permalink
# cvs update -r netbsd-5-0-1-RELEASE -d -P crypto/dist/openssl/ssl

Dennis answered to use -r 'netbsd-5-0', but another question is: why do
you want to track netbsd-5-0 instead of netbsd-5? netbsd-5 has more
fixes but my experience has been that it is very stable.
Anne Bennett
2010-03-12 01:16:54 UTC
Permalink
Post by Anne Bennett
# cvs update -r netbsd-5-0-1-RELEASE -d -P crypto/dist/openssl/ssl
Dennis answered to use -r 'netbsd-5-0', but another question is: why do
you want to track netbsd-5-0 instead of netbsd-5? netbsd-5 has more
fixes but my experience has been that it is very stable.
Here's a long answer to a short question...

<begin mild rant, please do not take offense; I am a little
frustrated, but willing to listen to reason>

Holy cow, not to sound like a newbie, but I don't *know*
what I should track! After my exchange with Andy on this
list, I re-read the advisory and realized that it had *told*
me to "update from the netbsd-5 or netbsd-5-0 branches",
but I had not grokked that those were the "<branch_name>"
values to substitute in the given cvs command, nor would I have
known which one was more appropriate in my case. The patching
instructions given in the advisories are probably crystal-clear
to someone who understands CVS and the release numbering system,
but to a relative outsider basically just following the recipe,
I humbly submit that they are insufficient. :-(

After years - wow, soon to be decades - (at work) of cursing
DEC (er, HP) and Sun for their arcane O/S patching methods,
I admit that I've been rather spoiled in the past few years by
the (cue doomsday music!) "RedHat and cousins" simplicity of
O/S patching (up2date or yum). The NetBSD cvsweb shows ten
branches whose name starts with "netbsd-5", and even ruling
out the ones with "RC" in their names, I have a choice of:

netbsd-5-base
netbsd-5-0-RELEASE
netbsd-5-0-2-RELEASE
netbsd-5-0-1-RELEASE
netbsd-5-0
netbsd-5

How much research is a simple sysadmin supposed to do to
figure out how to maintain a system? I installed 5.0.1 so I
(mistakenly, it turns out) assumed that if there were patches,
I should track 5.0.1.

I have only one NetBSD machine nowadays: my home workstation.
I periodically curse at it because I can't see Flash content
in Firefox (I'm working on that now, actually, since it's
embarrassing to have to borrow my SO's Slackware box for
that purpose!), my "all-in-one" printer's scanner function
isn't working, I've had problems in the past keeping my "pkg"
software up to date (I bit the bullet in January and figured
out that for most purposes, "pkg_rolling-replace" is my friend,
and that I can even use it after replacing my pkgsrc tree with
the latest quarterly release), and I dread security updates.
I've been basically upgrading the O/S every two or three years,
which is really not acceptable to me. The shoemaker herself
is definitely ill-shod!!!

Every few years when I *do* replace the O/S, I wonder if it's
time to throw in the towel and admit that much as I approve
of the philosophy behind how NetBSD is maintained, and much
as I appreciate the security and stability, and much as I
*really* appreciate the helpful and highly clueful nature of
the help I invariably get when I ask for it, maybe it's just
too time-consuming for me to maintain as a one-off. And each
time I chide myself for sounding like a Windows user instead
of a professional sysadmin, and tell myself that it's just
my own lack of knowledge that's making a simple thing seem
difficult, and I resolve to come up to speed so I can manage
this box with the ease with which my group manages hundreds
of Linux boxes at work.

<end of mild rant>

Anyway, as I said, after an intensive few days of learning
a couple of months ago, and some nifty Perl coding, I am now
managing my "pkg" software in a timely but not time-consuming
manner, and I am pleased.

I need to do something similar for the O/S itself. I am fairly
conservative with my updates because this is my workstation
and if I screw it up, I am very unhappy indeed, and I don't
necessarily have time at that moment (or possibly for days)
to figure out what I did wrong and how to reverse it.

I swear that back in January I re-read significant sections
of the "NetBSD Guide", but somehow I still don't know how to
keep this workstation up to date in a straightforward way.
Perhaps I managed to miss a crucial chapter, and if so, please
tell me to RTFM.

Andy Ruhl posts that he periodically gets a daily build of
"netbsd-5", makes install media, and does a binary upgrade
of the entire system, which takes him about half an hour.
In practice that's reasonable for just one machine, but to me
the idea of having to sit at the console and take the system
down seems philosophically wrong (as I said, at work we manage
hundreds of machines, so methods have to scale).
Post by Anne Bennett
Post by Andy Ruhl
And I maintain the release-5 code through CVS and use my
kernel config file to build a new kernel as per my config at
the same time.
Is there a way to use CVS to track and maintain a very stable
version of the O/S, both sources and binaries? If so, I'd have
to take "special action" only when I want a new kernel, which
isn't all that often. I suppose I could even write a cronjob to
"cvs update" and then "make install" overnight every couple of
weeks, if the make isn't too ridiculously time-consuming, but
it would be a lot handier if I could "cvs update" the binaries!

And how would I figure out which branch to track?

Anne.
Andy Ruhl
2010-03-12 01:49:31 UTC
Permalink
On Thu, Mar 11, 2010 at 6:16 PM, Anne Bennett
(lots of other stuff deleted for brevity by Andy)
And how would I figure out which branch to track?
See, you already sound like you're ahead of me in many (most?)
respects. I just know this one particular thing.

General life lesson type thoughts:

Self doubt is a good thing, it keeps you fresh. You're doing fine, don't worry.

Nobody can know everything about everything. I think there are only a
handfull of people who can answer every question you have about
NetBSD. The rest of us wing it and hope that the documentation is good
enough.

Back to NetBSD rambling by me:

You have to decide what branch you want to follow based on what you're
doing. Take note that the "base" branch is frozen, it never changes.
The 3 digit release ones (5-0-2 for example) don't change much either,
and I don't even really bother with them. If you're using those, you
are probably following binary distributions and maybe patching once in
a while if you can't stand downtime (sounds like your situation). I
have no idea what the 2 digit ones are for, but you could probably
find out here:

http://www.netbsd.org/releases/release-map.html

Once again, even if you are managing hundreds of servers, I would
probably still follow the netbsd-5 branch if I was doing what you're
doing. This is really stable stuff, unless maybe you're using bleeding
edge functions in which case you're going to have instability no
matter what you do until it settles out.

Find a directory somewhere, then do this stuff to track the netbsd-5 branch:

mkdir src tools obj (<--- tools and obj to store stuff when building kernels)
cvs -d :pserver:***@anoncvs.netbsd.org:/cvsroot checkout -PA
-rnetbsd-5 src (<--- for initial download of src dir)
cvs -d :pserver:***@anoncvs.netbsd.org:/cvsroot update -PAd
-rnetbsd-5 src (<--- to keep source dir updated)

Notes:
The initial download is called "checkout". "update" is what you do to
keep it current.
Substitute :pserver:***@anoncvs.netbsd.org:/cvsroot for a mirror
near you, if you like.
Standard disclaimer: everything is configurable and you can only know
what to do if you read all the man pages and docs :) But this should
work

You slightly misunderstood what I do. What I do is I make sure my
netbsd-5 tree is up to date, then I edit the kernel config file and
build the toolchain and kernel like this:

cd src
./build.sh -T ../tools -O ../obj tools
./build.sh -u -T ../tools -O ../obj kernel=CONFIG
cd ../obj/sys/arch/<arch>/compile/CONFIG/
ls -l netbsd
-rwxr-xr-x 1 whoever whoever 4778224 Jan 14 11:31 netbsd

And there it is. I put copy the old one to netbsd.old, and put the new
one in place, then reboot to make sure it works (since the userland
is older than the kernel, it will be compatible). Then go to a
netbsd-5 branch directory on the ftp server and download an install
kernel, for example this one:

ftp://nyftp.netbsd.org/pub/NetBSD-daily/netbsd-5/201003110000Z/i386/binary/kernel/netbsd-INSTALL_FLOPPY.gz
(Make sure you pick a directory that is older than the cvs update you
did to the source tree. You don't want to build a kernel from source
downloaded on Tuesday and then take a daily build from Thursday since
that is going to be newer than the kernel you built)

Then I gunzip it and stick it in the / and rename it to "netbsdi".

Making a backup now would be a good idea. If you're not intimately
familiar with recovering from backups, become familiar. It's not
really all that difficult once you understand disks...

Reboot, then stop the bootloader and give it this:

boot hd0a:netbsdi

And there you go, a nice NetBSD install kernel is running and you can upgrade.

You need to take note of the ftp directory you got the install kernel
from (write it down if you have to) and pass it into the ftp
information when it comes time to download sets. Unfortunately the
install kernels are not aware of which directory they came from on the
FTP server so they don't automatically know where to download the sets
that were built at the same time.

Play with this on a test system. I think you'll pick it up in a short
period of time. And if not, I'll help you out. I've been doing this a
while now. I don't claim to be that smart about it, I just know this
works. I've had a set of NetBSD server machines on various
architectures running constantly since about 2002 with few problems
that I can't blame on shoddy hardware :)

Andy
Dennis den Brok
2010-03-12 02:22:24 UTC
Permalink
Andy Ruhl <***@gmail.com> wrote:
[A proper, but relatively complicated way to stay up to date]

If your demands on security are not all too high (that is, you
trust NetBSD's build cluster not to distribute malicious binaries),
I would recommend against using CVS at all. Just use prebuilt bin-
aries from nyftp.netbsd.org.

First, determine which branch's kernel supports all the hardware
you want it to. The choice is basically netbsd-5-0, netbsd-5, or
HEAD (which is the bleeding edge). Then decide whether you need a
firewall on your workstation, or whether you want to use kernel
modules (up to 6.0, this should not be necessary in many cases;
when using a -current kernel, you could use a MONOLITHIC one).
If neither is the case, install the latest official release version
(5.0.2 at the moment) and put the kernel you chose on top of it.
Else, install the entire system from the your kernel's branch.

Next, subscribe to the security-announce mailing list. Whenever
you receive a security advisory, go to nyftp.netbsd.org and download
either a new kernel, if the advisory concerns the kernel, or the
binary sets (to be found in binary/sets), for the branch you chose
before.

In the kernel case, make sure it works before installing it
permanently.

In the userland case, install the userland by untarring the sets
(leaving out etc.tgz and xetc.tgz, as they would overwrite your
existing configuration) to /. Make sure to preserve file permissions;
with tar, this is done by using the "-p" flag. Finally, update your
configuration to make sure it is compatible with the new userland.
You do this by running etcupdate and postinstall (you need the
etc.tgz and xetc.tgz then). Please refer to the respective man
pages for details (but it is really simple).

That's it.

For pkgsrc, there are a few guides on wiki.netbsd.se, but if
pkg-rolling_replace does a good job for you, that's fine.

HTH,

Dennis den Brok
Andy Ruhl
2010-03-12 02:30:01 UTC
Permalink
Post by Dennis den Brok
[A proper, but relatively complicated way to stay up to date]
Yeah, there's always more than one way to do it. The reason why I do
it like I do is this:

o I can't use a "stock" kernel usually.
o I used modules a while back but I stopped. I just build it
monolithic, and I do it every few months or so mostly for the heck of
it.
o Why bother with untarring sets when you can use the install kernel
to upgrade? It manages the upgrade pretty well. You can point it at
local sets too, it's very nice.

Whatever works. Install CDs are pretty nice too.

Andy
Anne Bennett
2010-03-12 04:59:03 UTC
Permalink
Greg,

Thank you for the explanation of the different kinds of tags, and
Post by Greg Troxel
So, you can follow netbsd-5-0 and get software versions along an arc
From 5.0 to 5.0.1 to 5.0.2 and so on. Updates to branches like
netbsd-5-0 are extremely conservative.
Or you can follow netbsd-5, which is on an arc from 5.0 to 5.1 etc.
These updates are still conservative, and I regularly update along
netbsd-5, rebuild and reinstall the entire system.
Now I understand why you suggested that I track netbsd-5. I will do
so, and with luck, save myself having to reinstall the O/S for a few
years. ;-)
Post by Greg Troxel
I think it is best to update the entire tree to the new branch
and do a full rebuild,
Okay, I will try that. If I understand things correctly, it would
work to:

- Run a nightly cronjob that updates all of /usr/src to netbsd-5
nightly, runs the equivalent of "make" (but *not* "make install"),
and even recompiles the kernel (but does *not* install it).

- On an occasional basis, when there's a security update or when the
changes seem to be piling up, install the new kernel and reboot,
*then* install the new userland.

Does that make sense?


Andy,

Thanks for your continuing help and supportive comments.
Post by Greg Troxel
http://www.netbsd.org/releases/release-map.html
I had scanned that some time ago, but it was fruitful to look
at it again in light of Greg's explanation.
Post by Greg Troxel
mkdir src tools obj (<--- tools and obj to store stuff when building kernels)
-rnetbsd-5 src (<--- for initial download of src dir)
I suspect I don't need most of the above; I already have a
full source tree (symlinked from /usr/src), and I can build
kernels just fine without "tools" or "obj" - I'm enough of
a dinosaur to "config" and "make depend && make" manually
without the newfangled "build.sh". ;-)

I'm not sure any of it is needed to recompile "everything" natively
(no cross-compiling) - perhaps just /usr/obj?
Post by Greg Troxel
-rnetbsd-5 src (<--- to keep source dir updated)
Or just:

cd /usr/src
/usr/bin/cvs -q update -r netbsd-5 -PAd
Post by Greg Troxel
You slightly misunderstood what I do. What I do is I make sure my
netbsd-5 tree is up to date, then
[build and install kernel]
Post by Greg Troxel
then reboot to make sure it works (since the userland
is older than the kernel, it will be compatible). Then go to a
netbsd-5 branch directory on the ftp server and download an install
kernel,
[install it, boot from it, select "upgrade", making sure you know
where your sets are]

Well, if I'm going to keep my sources up to date, I might as
well just rebuild.

Greg, I've been looking at /usr/src/Makefile and
/usr/share/mk/bsd.README, trying to figure out what target(s) to
use, and any options to set. That's a *lot* of documentation,
more than I can absorb in a couple of hours.

I think that what I want is something like:

make USETOOLS=no MKUPDATE=yes DESTDIR=/some/path buildworld
make USETOOLS=no MKUPDATE=yes DESTDIR=/some/path installworld

... except that the "world" stuff seems to make installation
sets, and I don't really want that. Also, it's not clear
to me that this (the installation part) won't stomp on my
config files. And, does this build X as well, or do I have
to do that separately in xsrc?

Perhaps you could tell me what commands you use to "rebuild and
reinstall the entire system"? It would probably help a lot!

Anne.
Andy Ruhl
2010-03-12 14:10:16 UTC
Permalink
On Thu, Mar 11, 2010 at 9:59 PM, Anne Bennett
Post by Anne Bennett
I suspect I don't need most of the above; I already have a
full source tree (symlinked from /usr/src), and I can build
kernels just fine without "tools" or "obj" - I'm enough of
a dinosaur to "config" and "make depend && make" manually
without the newfangled "build.sh".  ;-)
I'm pretty much a dinosaur too, I used to do the whole "make world"
stuff in NetBSD and FreeBSD, but build.sh is a revelation. It's really
one of the absolute best things about NetBSD. Take a look at the
"release" option for example.

I know you said you don't want to make install sets, but it can be
useful to have them in case you need to rebuild the system (if you
don't have backups), or build another test system which you know will
have the exact same software.

I really don't understand the need to continue using "make install" or
"make world' or whatever, other than for some targeted re-build, but
maybe I'm missing something. I can't help you there.

Unfortunately I run headless servers for the most part, so I don't
build xsrc. I know build.sh will do it though. If I'm using X on a
desktop, I'm installing and upgrading from CD... I'm not normally
building the X stuff.

Andy
Anne Bennett
2010-03-12 18:30:40 UTC
Permalink
Post by Andy Ruhl
I'm pretty much a dinosaur too, I used to do the whole "make world"
stuff in NetBSD and FreeBSD, but build.sh is a revelation. It's really
one of the absolute best things about NetBSD. Take a look at the
"release" option for example.
Okay, I will. Nuts, I just discovered "Chapter 33. Updating
an existing system from sources" in "The NetBSD guide".
If I'd found that in the first place, I would have saved you
all some time. Off to RTFM...

Anne.
Greg Troxel
2010-03-12 19:09:48 UTC
Permalink
Post by Anne Bennett
Okay, I will. Nuts, I just discovered "Chapter 33. Updating
an existing system from sources" in "The NetBSD guide".
If I'd found that in the first place, I would have saved you
all some time. Off to RTFM...
Also, you may or may not find it useful to install
pkgsrc/sysutils/etcmanage. This is basically

a wrapper around build.sh to organize obj/tool/release how I want
(BUILD-NetBSD)

a script to install a release on a running system (INSTALL-NetBSD)

a perl script to update files in /etc from the new release, but only
if you haven't also changed them

I use this to update systems along netbsd-5 all the time. It also lets
me build on one system and install on another.
Anne Bennett
2010-03-12 21:30:04 UTC
Permalink
Post by Greg Troxel
Post by Anne Bennett
I just discovered "Chapter 33. Updating
an existing system from sources" in "The NetBSD guide".
Also, you may or may not find it useful to install
pkgsrc/sysutils/etcmanage.
By some interesting coincidence, the arrival of your message
caused a beep exactly as I was reading its DESCR file. :-)
Post by Greg Troxel
I use this to update systems along netbsd-5 all the time. It also lets
me build on one system and install on another.
It may turn out to be be overkill for me, but I have a note
to compare it to plain "etcupdate". Thank you.

I have updated my src and xsrc from netbsd-5, and have
carefully prepared, after much reading, an appropriate version
of the "update recipe" from chapter 33, along with a bunch of
questions, most of which I can safely answer by playing with
the commands. I have two small questions about the "install"
phase, though, that might be appropriate to ask here:

In "./build.sh [...] install=/"

(a) Do I need to specify "-x" to have X updated/installed too?
(b) Why doesn't the example in chapter 33 use "-u" to make sure
that MKUPDATE=yes is set so that we don't unnecessarily
replace unchanged files? Or am I misunderstanding the use
of "-u" in the context of "install"?

Anne.

Loading...