[fossil-users] Passwords stored in cleartext in 'user' table

[Ron Aaron]

I didn't see an option, perhaps it's not even on the list of requests... but
when I look at the 'user' table, the user's password is stored in cleartext.

Having my fossil file on a shared server, this makes me a bit nervous. Anyone
who has access to that file can read all the user passwords.

It would be trivial to change the password stored to sha1(login+pw). In that
case it would also be difficult to hack, since different users with the same
password would have wildly different values saved in the user table.

--
Sending me something private?
Use my GPG public key: AD29415D

[Jeremy Cowgar]

I noticed this the other day. I am not on a shared server and it makes me a bit nervous. Also, it's not really fair to people working on your project as many people have a "common" password (good or bad).

Jeremy

Ron Aaron <***@ronware.org> wrote:
> I didn't see an option, perhaps it's not even on the list of requests... but
> when I look at the 'user' table, the user's password is stored in cleartext.
>
> Having my fossil file on a shared server, this makes me a bit nervous. Anyone
> who has access to that file can read all the user passwords.
>
> It would be trivial to change the password stored to sha1(login+pw). In that
> case it would also be difficult to hack, since different users with the same
> password would have wildly different values saved in the user table.
>
> --
> Sending me something private?
> Use my GPG public key: AD29415D

[D. Richard Hipp]

On Jan 9, 2010, at 11:04 AM, Ron Aaron wrote:

> I didn't see an option, perhaps it's not even on the list of
> requests... but
> when I look at the 'user' table, the user's password is stored in
> cleartext.
>
> Having my fossil file on a shared server, this makes me a bit
> nervous. Anyone
> who has access to that file can read all the user passwords.
>
> It would be trivial to change the password stored to sha1(login
> +pw). In that
> case it would also be difficult to hack, since different users with
> the same
> password would have wildly different values saved in the user table.


There is a trade-off.

You can store an cryptographic checksum of the password in the user
table. But then the password has to be sent in the clear with each
HTTP request. Or you can store the cleartext password in the user
table and send a cryptographic checksum of the password in the HTTP
request. I went with the latter because to break it you must hack a
specific server. With the former, any packet sniffer on the internet
can get the password.

There is another option: Drop HTTP support all together and force
HTTPS for everything.

Note that even with option 3 (HTTPS for everything) you still store
passwords on the client side to enable auto-sync.

D. Richard Hipp
***@hwaci.com

[Ron Aaron]

On Saturday 09 January 2010 18:21:00 D. Richard Hipp wrote:

> There is a trade-off.
>
> You can store an cryptographic checksum of the password in the user
> table. ...
> Or you can store the cleartext password in the user
> table and send a cryptographic checksum of the password...

There is another option: send a crypto checksum over the wire, and store a
different sum in the user table. Then the server file does not have a
cleartext password, nor is one sent on the wire.

> Note that even with option 3 (HTTPS for everything) you still store
> passwords on the client side to enable auto-sync.

Right; I'm less concerned about the local file which only has my password in
it than I am with the server file with potentially dozens or hundreds of
passwords.


--
Sending me something private?
Use my GPG public key: AD29415D

[D. Richard Hipp]

On Jan 9, 2010, at 11:29 AM, Ron Aaron wrote:

> On Saturday 09 January 2010 18:21:00 D. Richard Hipp wrote:
>
>> There is a trade-off.
>>
>> You can store an cryptographic checksum of the password in the user
>> table. ...
>> Or you can store the cleartext password in the user
>> table and send a cryptographic checksum of the password...
>
> There is another option: send a crypto checksum over the wire, and
> store a
> different sum in the user table. Then the server file does not have a
> cleartext password, nor is one sent on the wire.


I'm not familiar with that algorithm. Can you explain or provide a link?


D. Richard Hipp
***@hwaci.com

[D. Richard Hipp]

On Jan 9, 2010, at 11:35 AM, D. Richard Hipp wrote:

>
> On Jan 9, 2010, at 11:29 AM, Ron Aaron wrote:
>
>> On Saturday 09 January 2010 18:21:00 D. Richard Hipp wrote:
>>
>>> There is a trade-off.
>>>
>>> You can store an cryptographic checksum of the password in the user
>>> table. ...
>>> Or you can store the cleartext password in the user
>>> table and send a cryptographic checksum of the password...
>>
>> There is another option: send a crypto checksum over the wire, and
>> store a
>> different sum in the user table. Then the server file does not
>> have a
>> cleartext password, nor is one sent on the wire.
>
>
> I'm not familiar with that algorithm. Can you explain or provide a
> link?


Wait - I think I get it. Feed the user-supplied password through a
cryptographic hash to convert the "real" password that is the shared
secret. Store only the shared secret on the server. Then use the
current algorithm to security authenticate using the shared secret.
Breaking into the server still allows an attacker to recover the
shared secret and then log into the server in the future, but they
cannot recover the original password text which might be used on other
unrelated systems.

OK. I'll work on that.

D. Richard Hipp
***@hwaci.com

[Ron Aaron]

On Saturday 09 January 2010 18:35:51 D. Richard Hipp wrote:


> I'm not familiar with that algorithm. Can you explain or provide a link?

Do the same thing as at present, in that the client sends the password hashed
and not in cleartext.

The server takes that hashed value and the user name, hashes again (perhaps
with a different algorithm) and compares to what it has stored. Neither the
original password (stored in the local machine or entered by the user) nor
the value sent on the wire (first hash) are stored in the server.

I'm not a cryptographer so I don't know if there are stunning weaknesses in
this algorithm, but it seems to me better than storing cleartext in the
server file.

--
Sending me something private?
Use my GPG public key: AD29415D

[Twylite]

Hi,
> Do the same thing as at present, in that the client sends the password hashed
> and not in cleartext.
>
> The server takes that hashed value and the user name, hashes again (perhaps
> with a different algorithm) and compares to what it has stored. Neither the
> original password (stored in the local machine or entered by the user) nor
> the value sent on the wire (first hash) are stored in the server.
>
Cryptographically this is equivalent to sending a cleartext password
over HTTP and storing a hash of the password.

C: username, password --> S
S: compute H(salt, username, password) and compare against stored hash

What you are proposing is to substitute password = H(password) in the
above protocol. If you can sniff the interaction then you will recover
H(password), which is sufficient for you to log in to the server. The
only thing this change buys you is that the attacker would need a
modified client (so that he can enter H(password) directly, instead of
entering password).

The existing (now "old") fossil protocol secures the wire interaction
against compromise. It is generally believed that it is easier to sniff
HTTP traffic than the compromise a server and retrieve a file with
passwords.

Also, unless I am misunderstanding drh's followup message, the "new"
fossil protocol is similarly no more secure that the old protocol, just
very slightly more irritating for an attacker. Instead of the client
and server sharing the secret cleartext password p, they share the
secret hash H(p). If you can compromise the server then you know the
shared secret H(p), and can use it with a small modification to the client.

The only way to solve this problem - securing both the wire and the
server storage - is to move to a more advanced cryptographic protocol
(preferably asymmetric crypto). That is not workable however, as it
would leave no way to log on through the web interface (which is limited
to the mechanisms reasonably supported by browsers). Fossil is not
alone in this dilemma.

Regards,
Twylite

[Ron Aaron]

On Sunday 10 January 2010 13:32:31 Twylite wrote:
> Hi,
...
> Cryptographically this is equivalent to sending a cleartext password
> over HTTP and storing a hash of the password.

Yes, you are correct that from the standpoint of someone intercepting wire
traffic, such a person can still log in. However, that person will *not*
have intercepted the password itself, which is what I and others were
concerned about.

In the current scenario, the password used by the user is neither stored on
the remote server nor sent on the wire. Otherwise, the protocol is the same
as before in terms of overall security.


> The only way to solve this problem - securing both the wire and the
> server storage - is to move to a more advanced cryptographic protocol
> (preferably asymmetric crypto).  

That was one reason I suggested the possibility of allowing the option of
a 'whitelist' of GPG keys, against which potential checkin manifests could be
checked. That doesn't prevent a malicious user from logging in as someone
else, but it does prevent such a person from (easily) compromising the source
repository.

--
For privacy, my GPG key signature is: AD29415D

[Wilson, Ronald]

I agree with your conclusions. for my part I'm just glad that my
password is not stored in plain text in the fossil database -
especially when I don't control the repository. I know that the
transmissions are no more secure than before.

many people, for better or worse, reuse usernames and passwords at
other online portals. i wouldn't want someone to inspect the
reopository and try my username and password at other online portals.

rw

from my mobile [phone]

On Jan 10, 2010, at 6:32 AM, "Twylite" <***@crypt.co.za> wrote:

>
> Hi,
>> Do the same thing as at present, in that the client sends the
>> password hashed
>> and not in cleartext.
>>
>> The server takes that hashed value and the user name, hashes again
>> (perhaps
>> with a different algorithm) and compares to what it has stored.
>> Neither the
>> original password (stored in the local machine or entered by the
>> user) nor
>> the value sent on the wire (first hash) are stored in the server.
>>
> Cryptographically this is equivalent to sending a cleartext password
> over HTTP and storing a hash of the password.
>
> C: username, password --> S
> S: compute H(salt, username, password) and compare against stored hash
>
> What you are proposing is to substitute password = H(password) in the
> above protocol. If you can sniff the interaction then you will
> recover
> H(password), which is sufficient for you to log in to the server. The
> only thing this change buys you is that the attacker would need a
> modified client (so that he can enter H(password) directly, instead of
> entering password).
>
> The existing (now "old") fossil protocol secures the wire interaction
> against compromise. It is generally believed that it is easier to
> sniff
> HTTP traffic than the compromise a server and retrieve a file with
> passwords.
>
> Also, unless I am misunderstanding drh's followup message, the "new"
> fossil protocol is similarly no more secure that the old protocol,
> just
> very slightly more irritating for an attacker. Instead of the client
> and server sharing the secret cleartext password p, they share the
> secret hash H(p). If you can compromise the server then you know the
> shared secret H(p), and can use it with a small modification to the
> client.
>
> The only way to solve this problem - securing both the wire and the
> server storage - is to move to a more advanced cryptographic protocol
> (preferably asymmetric crypto). That is not workable however, as it
> would leave no way to log on through the web interface (which is
> limited
> to the mechanisms reasonably supported by browsers). Fossil is not
> alone in this dilemma.
>
> Regards,
> Twylite
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[Daniel Clark]

Twylite wrote:
> The only way to solve this problem - securing both the wire and the
> server storage - is to move to a more advanced cryptographic protocol
> (preferably asymmetric crypto). That is not workable however, as it
> would leave no way to log on through the web interface (which is limited
> to the mechanisms reasonably supported by browsers). Fossil is not
> alone in this dilemma.

Client-side SSL certificates are indeed supported by almost all
browsers, it's just that for some reason almost no one uses them (in y
knowledge except for MIT, where they seem to work quite well for
authenticating to webapps). I'm pretty sure they even work with
http-only sites - and indeed on many sites it make sense to only spend
computational resources to encrypt the authentication process, and not
looking at web pages / downloads / etc.

Client SSL Certs are actually pretty easy to use, just like having to
have your ssh id_rsa file on computers you ssh from. There are web
interfaces to generate them at many sites, or you can generate yourself
easily. It's possible the SSL server code already linked in could also
be used for verifying SSL client certs.

Another option, which is what ikiwiki does, is to use OpenID. That way
each username is just associated with a URL, and the response from that
URL determines access; so no password information of any kind needs to
be stored locally (and indeed, the user may choose an OpenID provider
that supports non-password authentication mechanisms, such as Client SSL
certs or multiple-factor email / web browser / picture array choices auth).

[1] Client Certificate FAQ: Web Browser
http://wiki.cacert.org/Technology/KnowledgeBase/ClientCerts?action=show&redirect=ClientCerts#Web_Browser

--
Daniel JB Clark | http://pobox.com/~dclark

[D. Richard Hipp]

On Jan 9, 2010, at 11:04 AM, Ron Aaron wrote:

> I didn't see an option, perhaps it's not even on the list of
> requests... but
> when I look at the 'user' table, the user's password is stored in
> cleartext.
>
> Having my fossil file on a shared server, this makes me a bit
> nervous. Anyone
> who has access to that file can read all the user passwords.
>
> It would be trivial to change the password stored to sha1(login
> +pw). In that
> case it would also be difficult to hack, since different users with
> the same
> password would have wildly different values saved in the user table.


OK. Beginning with http://www.fossil-scm.org/fossil/ci/cfe33dcf92
Fossil will store passwords on servers as either cleartext or as a
SHA1 hash of the password. It tells the difference by looking at the
length of the password. A password in the USER table that is exactly
40 characters long is assumed to be a SHA1 hash. Otherwise, the
password is assumed to be cleartext.

Whenever you change a password, the new password is stored as the SHA1
hash. When you create new users, the password is stored as the SHA1
hash. There is no mechanism to force the password to be cleartext.
You can force all cleartext passwords to become SHA1 hashes using this
command:

fossil test-hash-password REPOSITORY

Converting from cleartext to SHA1 hash is irreversible, of course.

The client always uses the SHA1 hash as the shared secret, unless the
password for a sync operation begins with '*'. If the password for a
sync begins with '*', then the characters after the '*' are taken to
be the cleartext password used as the shared secret. This allows
newer clients to communicate with legacy servers that do not know
about the password format change. If you have a new fossil client
and you want to sync against a legacy server, do it this way:

fossil sync http://userid:****@legacy-server.com/

The new server accepts both the cleartext passwords and the SHA1 has
as the shared secret, assuming the cleartext is stored in the USER
table. That means that newer servers will work with older clients as
long as you do not update the USER table to store hashes. Once a hash
is stored in the USER table, the sync protocol will only work with
newer clients.

So, older clients will work with newer servers as long as cleartext
passwords are stored in the USER table, and older servers will work
with newer clients by adding '*' before the password in the URL.

The simplest upgrade path is probably just to upgrade all clients and
servers all at once.

The second simplest upgrade path is:

(1) Upgrade servers, but do not modify the USER table.
(2) Start upgrading clients.
(3) After all clients are upgraded, run [fossil test-hash-password] to
convert the USER table to use hashes instead of cleartext.

I have not yet done step (1) on the server that runs www.fossil-
scm.org. I'll do that after additional testing and after you, gentle
readers, have reviewed my changes and informed me that my changes are
free of new security boo-boos. I eagerly await the results of your
review. Tnx.

D. Richard Hipp
***@hwaci.com

[Ron Aaron]

On Sunday 10 January 2010 00:24:14 D. Richard Hipp wrote:

> I have not yet done step (1) on the server that runs www.fossil-
> scm.org. I'll do that after additional testing and after you, gentle
> readers, have reviewed my changes and informed me that my changes are
> free of new security boo-boos. I eagerly await the results of your
> review. Tnx.

Thank you! It does work for me, so far ...

--
For privacy, my GPG key signature is: AD29415D

[D. Richard Hipp]

On Jan 9, 2010, at 5:24 PM, D. Richard Hipp wrote:
>
>
> OK. Beginning with http://www.fossil-scm.org/fossil/ci/cfe33dcf92
> Fossil will store passwords on servers as either cleartext or as a
> SHA1 hash of the password. ...

I have tagged this change "experimental" for now. I'm planning to
make some additional (incompatible) changes to make the password
handling more secure. You can experiment with this version, but you
should create backups to restore from after my upcoming incompatible
changes.....

***@hwaci.com

[Gé Weijers]

The most common approaches I know of:

Unix-style password file: store salt || H(salt || password)

The 'salt' value is a random string used to make attacks that precompute
the hash values of common passwords harder. The downside is that the
client has to send the password in plaintext. The hash is often iterated
a few times to make brute force attacks more time consuming.

CHAP: (originally from PPP)

send a challenge string from server to client. The client sends back
H(challenge || password). Someone now has to brute-force the password if
the communication is intercepted. The downside is that the password is
stored in plaintext on the server.

There are methods that solve both problems (not storing the password on
the server, and not sending the password over the wire), but you enter a
major patent mine field if you try to use them.

If I had to choose I'd store the password as a salted hash. Break-ins
into servers are much more common than password intercepts (credit card
numbers are stolen from servers, not by intercepting traffic). You can
always add HTTPS if interception is an issue.

Ge'

On 01/10/2010 04:43 AM, D. Richard Hipp wrote:
> On Jan 9, 2010, at 5:24 PM, D. Richard Hipp wrote:
>
>>
>> OK. Beginning with http://www.fossil-scm.org/fossil/ci/cfe33dcf92
>> Fossil will store passwords on servers as either cleartext or as a
>> SHA1 hash of the password. ...
>>
> I have tagged this change "experimental" for now. I'm planning to
> make some additional (incompatible) changes to make the password
> handling more secure. You can experiment with this version, but you
> should create backups to restore from after my upcoming incompatible
> changes.....
>
> ***@hwaci.com
>
>
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>


--
Gé Weijers email: ***@weijers.org <mailto:***@weijers.org>

[Twylite]

Hi,
> CHAP: (originally from PPP)
>
> send a challenge string from server to client. The client sends back
> H(challenge || password). Someone now has to brute-force the password
> if the communication is intercepted. The downside is that the password
> is stored in plaintext on the server.
Another downside: when using HTTP as your transport, you always require
an initial HTTP request the get the challenge. In addition the server
must track the tuple (username, challenge, clientIp, timeIssued), or
more generally implement sessions using cookies.

You _can_ turn this into a stateless protocol (ISO 9798-2 protocol 1) by
deriving a shared secret K = KDF(password) for some Key Derivation
Function KDF, and sending something like E[K]( T || S || username ) or
username || T || HMAC(k, T || S || username ), where T is a timestamp
and S is the distinguished identity of the server. The server must
store T on every transaction, and reject an authentication attempt if T
is older than or equal to the stored T.

Note that you must store KDF(password) on the server, and it cannot be
salted as the client must compute KDF(password) on demand.

To ensure that you can still support a web interface, it must be
possible for the server to accept a plaintext password (via a form POST
over HTTP or HTTPS) and compare it against a stored value. KDF() will
allow this, but because it cannot be salted it may susceptible to
rainbow table attacks (depending on the function used).
> There are methods that solve both problems (not storing the password
> on the server, and not sending the password over the wire), but you
> enter a major patent mine field if you try to use them.
Moreover, many are not compatible with submitting a plaintext password
in a form.

Regards,
Twylite

[Mark Janssen]

It's been quite a while since I looked into the details, but I do know
that the SyncML (or OMA DM as it is known now) protocol provide
authentication with HTTP as one of the available transport protocols.
I think it uses a shared secret. Might be worth looking into. See
http://member.openmobilealliance.org/ftp/public_documents/dm/Permanent_documents/OMA-DM-Security-V1_2_0-20050120-D.zip
for the general architecture and details.

Regards,
Mark

On Mon, Jan 11, 2010 at 12:48 PM, Twylite <***@crypt.co.za> wrote:
> Hi,
>> CHAP: (originally from PPP)
>>
>> send a challenge string from server to client. The client sends back
>> H(challenge || password). Someone now has to brute-force the password
>> if the communication is intercepted. The downside is that the password
>> is stored in plaintext on the server.
> Another downside: when using HTTP as your transport, you always require
> an initial HTTP request the get the challenge.  In addition the server
> must track the tuple (username, challenge, clientIp, timeIssued), or
> more generally implement sessions using cookies.
>
> You _can_ turn this into a stateless protocol (ISO 9798-2 protocol 1) by
> deriving a shared secret K = KDF(password) for some Key Derivation
> Function KDF, and sending something like E[K]( T || S || username ) or
> username || T || HMAC(k, T || S || username ), where T is a timestamp
> and S is the distinguished identity of the server.  The server must
> store T on every transaction, and reject an authentication attempt if T
> is older than or equal to the stored T.
>
> Note that you must store KDF(password) on the server, and it cannot be
> salted as the client must compute KDF(password) on demand.
>
> To ensure that you can still support a web interface, it must be
> possible for the server to accept a plaintext password (via a form POST
> over HTTP or HTTPS) and compare it against a stored value.  KDF() will
> allow this, but because it cannot be salted it may susceptible to
> rainbow table attacks (depending on the function used).
>> There are methods that solve both problems (not storing the password
>> on the server, and not sending the password over the wire), but you
>> enter a major patent mine field if you try to use them.
> Moreover, many are not compatible with submitting a plaintext password
> in a form.
>
> Regards,
> Twylite
>
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>

[D. Richard Hipp]

A description of current password handling can be seen at

http://www.fossil-scm.org/fossil/doc/tip/www/password.wiki

Please comment and if there are no problems seen, I will commit the
latest changes to the trunk and put new binaries on the website.

D. Richard Hipp
***@hwaci.com

[Tiago Dionizio]

Hi Richard,

Because the file was checked-in in the experimental branch, it is not
accessible through that url.

Regards,
Tiago


On Mon, Jan 11, 2010 at 3:10 PM, D. Richard Hipp <***@hwaci.com> wrote:

> A description of current password handling can be seen at
>
> http://www.fossil-scm.org/fossil/doc/tip/www/password.wiki
>
> Please comment and if there are no problems seen, I will commit the
> latest changes to the trunk and put new binaries on the website.
>
> D. Richard Hipp
> ***@hwaci.com
>
>
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>

[D. Richard Hipp]

On Jan 11, 2010, at 12:21 PM, Tiago Dionizio wrote:

> Hi Richard,
>
> Because the file was checked-in in the experimental branch, it is
> not accessible through that url.
>


It was before Jeremy added http://www.fossil-scm.org/fossil/vinfo/1bf6cf832d
on the trunk. But regardless, you can always reach the file here:

http://www.fossil-scm.org/fossil/doc/261e55346d/www/password.wiki

D. Richard Hipp
***@hwaci.com

[Jeremy Cowgar]

I get:

No such document: tip/www/password.wiki

for the link below.

Jeremy

"D. Richard Hipp" <***@hwaci.com> wrote:
> A description of current password handling can be seen at
>
> http://www.fossil-scm.org/fossil/doc/tip/www/password.wiki
>
> Please comment and if there are no problems seen, I will commit the
> latest changes to the trunk and put new binaries on the website.
>
> D. Richard Hipp
> ***@hwaci.com
>
>
>
> _______________________________________________
> fossil-users mailing list
> fossil-***@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[D. Richard Hipp]

I have updated all of my fossil servers to the latest code that
includes support for the new SHA1 passwords. New precompiled binaries
are available at

http://www.fossil-scm.org/download.html

I have not yet converted passwords to SHA1. Once I do that, if you
have a login, the login will stop working for "sync" until you upgrade
your client.

D. Richard Hipp
***@hwaci.com