Discussion:
[hybi] I-D Action:draft-ietf-hybi-thewebsocketprotocol-01.txt
I***@ietf.org
2010-09-01 22:45:02 UTC
Permalink
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.


Title : The WebSocket protocol
Author(s) : I. Fette
Filename : draft-ietf-hybi-thewebsocketprotocol-01.txt
Pages : 57
Date : 2010-09-01

The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code. The
security model used for this is the Origin-based security model
commonly used by Web browsers. The protocol consists of an initial
handshake followed by basic message framing, layered over TCP. The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).

Please send feedback to the ***@ietf.org mailing list.Note

This draft is meant to reflect changes in direction in the HyBi
working group. There is not yet consensus on everything in this
draft. Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft. There have also been proposals
to change the handshake, so the handshake is also not in a final
form.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
Ian Fette (イアンフェッティ)
2010-09-01 22:55:09 UTC
Permalink
I'd like to give a quick explanation of what has changed in this draft.

This draft was begun by taking the -00 version, not what was referred to as
"draft-ietf-hybi-thewebsocketprotocol-latest". The -latest version had a
number of changes after -00, some of which had been discussed on the list
and some of which hadn't. (You can see the changes at
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol-latest)
Many of these changes are editorial (e.g. "the server must not send any
further data to the server" -> "the server must not send any further data to
the client"), but there were also a large number of non-editorial changes,
e.g. to the handshake.

The handshake in -01 is the same as in -00. There are certainly open issues
against the handshake, and I do not mean to dismiss them, they simply are
not addressed in this draft.

The focus of this draft was to provide an updated draft that more closely
reflects the framing that the group seems to be discussing. The framing in
this draft is based on "take vii" that john tamplin sent out, and subsequent
discussion. It is not meant to say "Everyone is happy with this version" but
rather to give us a common document which to reference and base discussion
from. I fully expect that there will be changes.

I am hopeful that we will be able to produce a -02 much more quickly, e.g.
perhaps in 2-3 weeks, that reflects further refinement and consensus that
this group reaches on the mailing list.

On an editorial note, if you diff -00 and -01,
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-01.txt,
you can see the full list of changes. It may look at first glance that there
are significant changes to the handshake sections, I assure you these are
editorial, and you should see that if you look at the text of the handshake
sections itself. The reason it doesn't diff cleanly is that the handshake
was previously split across sections called "Client-side requirements" and
"server-side requirements", while it is now in a section called "Opening
Handshake" with sub-sections for client- and server-side requirements.

-Ian
Post by I***@ietf.org
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP
Working Group of the IETF.
Title : The WebSocket protocol
Author(s) : I. Fette
Filename : draft-ietf-hybi-thewebsocketprotocol-01.txt
Pages : 57
Date : 2010-09-01
The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code. The
security model used for this is the Origin-based security model
commonly used by Web browsers. The protocol consists of an initial
handshake followed by basic message framing, layered over TCP. The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).
This draft is meant to reflect changes in direction in the HyBi
working group. There is not yet consensus on everything in this
draft. Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft. There have also been proposals
to change the handshake, so the handshake is also not in a final
form.
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
ftp://ftp.ietf.org/internet-drafts/
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Simon Pieters
2010-09-02 10:42:40 UTC
Permalink
On Thu, 02 Sep 2010 00:55:09 +0200, Ian Fette (イアンフェッティ)
Post by Ian Fette (イアンフェッティ)
I'd like to give a quick explanation of what has changed in this draft.
This draft was begun by taking the -00 version, not what was referred to as
"draft-ietf-hybi-thewebsocketprotocol-latest". The -latest version had a
number of changes after -00, some of which had been discussed on the list
and some of which hadn't. (You can see the changes at
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol-latest)
Many of these changes are editorial (e.g. "the server must not send any
further data to the server" -> "the server must not send any further data to
the client"), but there were also a large number of non-editorial changes,
e.g. to the handshake.
The handshake in -01 is the same as in -00. There are certainly open issues
against the handshake, and I do not mean to dismiss them, they simply are
not addressed in this draft.
Some of those changes were in response to feedback from me (mostly sent to
whatwg). I was happy with the changes Ian Hickson made. What's the plan?
Is there something I should do if I want those changes in?

Cheers,
--
Simon Pieters
Opera Software
Ian Fette (イアンフェッティ)
2010-09-02 16:24:58 UTC
Permalink
On Thu, 02 Sep 2010 00:55:09 +0200, Ian Fette (むアンフェッティ) <
I'd like to give a quick explanation of what has changed in this draft.
Post by Ian Fette (イアンフェッティ)
This draft was begun by taking the -00 version, not what was referred to as
"draft-ietf-hybi-thewebsocketprotocol-latest". The -latest version had a
number of changes after -00, some of which had been discussed on the list
and some of which hadn't. (You can see the changes at
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol-latest
)
Many of these changes are editorial (e.g. "the server must not send any
further data to the server" -> "the server must not send any further data to
the client"), but there were also a large number of non-editorial changes,
e.g. to the handshake.
The handshake in -01 is the same as in -00. There are certainly open issues
against the handshake, and I do not mean to dismiss them, they simply are
not addressed in this draft.
Some of those changes were in response to feedback from me (mostly sent to
whatwg). I was happy with the changes Ian Hickson made. What's the plan? Is
there something I should do if I want those changes in?
Cheers,
IMO best thing would be to send an email to this list explaining what the
change is, and why it was needed. I don't want to lose these changes, I just
want to make sure they are discussed on this list.
--
Simon Pieters
Opera Software
Simon Pieters
2010-09-03 09:01:38 UTC
Permalink
On Thu, 02 Sep 2010 18:24:58 +0200, Ian Fette (イアンフェッティ)
Post by Ian Fette (イアンフェッティ)
Post by Simon Pieters
Some of those changes were in response to feedback from me (mostly sent to
whatwg). I was happy with the changes Ian Hickson made. What's the plan? Is
there something I should do if I want those changes in?
Cheers,
IMO best thing would be to send an email to this list explaining what the
change is, and why it was needed. I don't want to lose these changes, I just
want to make sure they are discussed on this list.
Ok. Here are the changes I could find since -00 and associated emails/bugs
ignoring things that are moot with -01 and things that IIRC were discussed
on hybi:

http://html5.org/tools/web-apps-tracker?from=4893&to=4894
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9149

http://html5.org/tools/web-apps-tracker?from=5169&to=5170
https://bugzilla.mozilla.org/show_bug.cgi?id=472529#c180 (can't find the
relevant email)

http://html5.org/tools/web-apps-tracker?from=5174&to=5189
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-July/027245.html

http://html5.org/tools/web-apps-tracker?from=5241&to=5244
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027673.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027674.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027675.html

http://html5.org/tools/web-apps-tracker?from=5287&to=5288
http://html5.org/tools/web-apps-tracker?from=5288&to=5289
I don't know what this was in response to.


Let me know if there's a change you'd like discussed here that's not
listed above and I can try to dig it up.

HTH,
--
Simon Pieters
Opera Software
Simon Pieters
2010-09-09 13:21:57 UTC
Permalink
Post by Simon Pieters
On Thu, 02 Sep 2010 18:24:58 +0200, Ian Fette (イアンフェッティ)
Post by Ian Fette (イアンフェッティ)
Post by Simon Pieters
Some of those changes were in response to feedback from me (mostly sent to
whatwg). I was happy with the changes Ian Hickson made. What's the plan? Is
there something I should do if I want those changes in?
Cheers,
IMO best thing would be to send an email to this list explaining what the
change is, and why it was needed. I don't want to lose these changes, I just
want to make sure they are discussed on this list.
Ok. Here are the changes I could find since -00 and associated
emails/bugs ignoring things that are moot with -01 and things that IIRC
http://html5.org/tools/web-apps-tracker?from=4893&to=4894
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9149
http://html5.org/tools/web-apps-tracker?from=5169&to=5170
https://bugzilla.mozilla.org/show_bug.cgi?id=472529#c180 (can't find the
relevant email)
http://html5.org/tools/web-apps-tracker?from=5174&to=5189
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-July/027245.html
http://html5.org/tools/web-apps-tracker?from=5241&to=5244
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027673.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027674.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027675.html
http://html5.org/tools/web-apps-tracker?from=5287&to=5288
http://html5.org/tools/web-apps-tracker?from=5288&to=5289
I don't know what this was in response to.
Let me know if there's a change you'd like discussed here that's not
listed above and I can try to dig it up.
HTH,
I haven't seen any further discussion here. If there are no objections,
could you merge in the changes (everything in
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol-latest
except the framing stuff which is different in -01) in your draft?

Cheers,
--
Simon Pieters
Opera Software
Ian Fette (イアンフェッティ)
2010-09-09 19:48:31 UTC
Permalink
I am on vacation this week. Next week I will work on producing a -02 draft
based on this and other comments to the WG, barring any substantive
objections.
On Thu, 02 Sep 2010 18:24:58 +0200, Ian Fette (むアンフェッティ) <
Post by Simon Pieters
Some of those changes were in response to feedback from me (mostly sent
Post by Ian Fette (イアンフェッティ)
to
whatwg). I was happy with the changes Ian Hickson made. What's the plan? Is
there something I should do if I want those changes in?
Cheers,
IMO best thing would be to send an email to this list explaining what the
change is, and why it was needed. I don't want to lose these changes, I just
want to make sure they are discussed on this list.
Ok. Here are the changes I could find since -00 and associated emails/bugs
ignoring things that are moot with -01 and things that IIRC were discussed
http://html5.org/tools/web-apps-tracker?from=4893&to=4894
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9149
http://html5.org/tools/web-apps-tracker?from=5169&to=5170
https://bugzilla.mozilla.org/show_bug.cgi?id=472529#c180 (can't find the
relevant email)
http://html5.org/tools/web-apps-tracker?from=5174&to=5189
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-July/027245.html
http://html5.org/tools/web-apps-tracker?from=5241&to=5244
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027673.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027674.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027675.html
http://html5.org/tools/web-apps-tracker?from=5287&to=5288
http://html5.org/tools/web-apps-tracker?from=5288&to=5289
I don't know what this was in response to.
Let me know if there's a change you'd like discussed here that's not
listed above and I can try to dig it up.
HTH,
I haven't seen any further discussion here. If there are no objections,
could you merge in the changes (everything in
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol-latestexcept the framing stuff which is different in -01) in your draft?
Cheers,
--
Simon Pieters
Opera Software
Simon Pieters
2010-09-22 21:44:09 UTC
Permalink
Hi Ian,

On Thu, 09 Sep 2010 21:48:31 +0200, Ian Fette (イアンフェッティ)
Post by Ian Fette (イアンフェッティ)
I am on vacation this week. Next week I will work on producing a -02 draft
based on this and other comments to the WG, barring any substantive
objections.
Any progress on this?

Cheers,
Post by Ian Fette (イアンフェッティ)
On Thu, 02 Sep 2010 18:24:58 +0200, Ian Fette (イアンフェッティ) <
Post by Simon Pieters
Some of those changes were in response to feedback from me (mostly sent
Post by Ian Fette (イアンフェッティ)
Post by Simon Pieters
to
whatwg). I was happy with the changes Ian Hickson made. What's the plan?
Is
there something I should do if I want those changes in?
Cheers,
IMO best thing would be to send an email to this list explaining what the
change is, and why it was needed. I don't want to lose these changes,
I
just
want to make sure they are discussed on this list.
Ok. Here are the changes I could find since -00 and associated emails/bugs
ignoring things that are moot with -01 and things that IIRC were discussed
http://html5.org/tools/web-apps-tracker?from=4893&to=4894
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9149
http://html5.org/tools/web-apps-tracker?from=5169&to=5170
https://bugzilla.mozilla.org/show_bug.cgi?id=472529#c180 (can't find the
relevant email)
http://html5.org/tools/web-apps-tracker?from=5174&to=5189
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-July/027245.html
http://html5.org/tools/web-apps-tracker?from=5241&to=5244
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027673.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027674.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027675.html
http://html5.org/tools/web-apps-tracker?from=5287&to=5288
http://html5.org/tools/web-apps-tracker?from=5288&to=5289
I don't know what this was in response to.
Let me know if there's a change you'd like discussed here that's not
listed above and I can try to dig it up.
HTH,
I haven't seen any further discussion here. If there are no objections,
could you merge in the changes (everything in
http://tools.ietf.org/tools/rfcdiff/rfcdiff.pyht?url1=http://tools.ietf.org/id/draft-ietf-hybi-thewebsocketprotocol-00.txt&url2=http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol-latestexcept
the framing stuff which is different in -01) in your draft?
Cheers,
--
Simon Pieters
Opera Software
--
Simon Pieters
Opera Software
John Tamplin
2010-09-22 21:51:54 UTC
Permalink
Post by Simon Pieters
Post by Ian Fette (イアンフェッティ)
I am on vacation this week. Next week I will work on producing a -02 draft
based on this and other comments to the WG, barring any substantive
objections.
Any progress on this?
Ian's chat status says that he is recovering from a minor surgery, so
he may not be online for a few days.

--
John A. Tamplin
Software Engineer (GWT), Google
Ian Fette (イアンフェッティ)
2010-09-22 22:18:19 UTC
Permalink
Apologies. I did have minor surgery on my foot Monday. Everything is fine,
but I'm taking a bit of time to recover. It's also performance management
season at Google so we're all tied down a bit with that. I am going to try
to set aside some time for this, but to be honest this week and next are a
bit rough.

-Ian
Post by John Tamplin
Post by Simon Pieters
Post by Ian Fette (イアンフェッティ)
I am on vacation this week. Next week I will work on producing a -02
draft
Post by Simon Pieters
Post by Ian Fette (イアンフェッティ)
based on this and other comments to the WG, barring any substantive
objections.
Any progress on this?
Ian's chat status says that he is recovering from a minor surgery, so
he may not be online for a few days.
--
John A. Tamplin
Software Engineer (GWT), Google
Anne van Kesteren
2010-09-23 08:30:44 UTC
Permalink
On Thu, 23 Sep 2010 00:18:19 +0200, Ian Fette (=E3=82=A4=E3=82=A2=E3=83=B3=
=E3=83=95=E3=82=A7=E3=83=83=E3=83=86=E3=82=A3) =
Apologies. I did have minor surgery on my foot Monday. Everything is =
fine, but I'm taking a bit of time to recover. It's also performance =
management
season at Google so we're all tied down a bit with that. I am going to=
=
try to set aside some time for this, but to be honest this week and ne=
xt =
are a bit rough.
That sucks. :/ I certainly do not want to put blame on you but I do hope=
=

we all realize we missed the four-week deadline by about a month. I also=
=

have not seen any indication that Firefox 4 will not ship with -76 suppo=
rt =

(apart from that one email to this list). Apart from that, having to wai=
t =

even longer than Firefox 4 for WebSocket to get deployed would not be ni=
ce =

either.


-- =

Anne van Kesteren
http://annevankesteren.nl/
Ian Fette (イアンフェッティ)
2010-09-23 15:36:34 UTC
Permalink
On Thu, 23 Sep 2010 00:18:19 +0200, Ian Fette (むアンフェッティ) <
Post by Ian Fette (イアンフェッティ)
Apologies. I did have minor surgery on my foot Monday. Everything is fine,
but I'm taking a bit of time to recover. It's also performance management
season at Google so we're all tied down a bit with that. I am going to try
to set aside some time for this, but to be honest this week and next are a
bit rough.
That sucks. :/ I certainly do not want to put blame on you but I do hope we
all realize we missed the four-week deadline by about a month. I also have
not seen any indication that Firefox 4 will not ship with -76 support (apart
from that one email to this list). Apart from that, having to wait even
longer than Firefox 4 for WebSocket to get deployed would not be nice
either.
Anne,

With respect, I am happy to put in some extra time to get a -02 out, but I
don't think that has been a blocking factor for e.g. Firefox or others, nor
a major contributor to us missing the 4 week deadline. A large part of our
four week deadline was devoted to some rapid changes at first, then we got
into a period where certain personalities clashed on the list for a while
and a bunch of people dropped off, and now we seem to have quieted down, but
still have a number of open issues on e.g. the handshake hanging on certain
intermediaries that have still yet to be resolved. I don't think we're quite
at the "stable" point yet, which is what I interpreted the emails from
Firefox as being desired by them to ship.

I will try to put together a -02 by the end of the week and see how far I
get, and then let's see how we can move forward. In my mind, some of the
pressing issues remaining are Issue 4 (
http://trac.tools.ietf.org/wg/hybi/trac/ticket/4), and any ideas anyone has
(including NPN, or proposing an alternate that still meets the required
properties) would be helpful.
--
Anne van Kesteren
http://annevankesteren.nl/
John Tamplin
2010-09-23 15:47:55 UTC
Permalink
On Thu, Sep 23, 2010 at 11:36 AM, Ian Fette (イアンフェッティ)
I will try to put together a -02 by the end of the week and see how far I get, and then let's see how we can move forward.
I can help with that if you need it as well.
In my mind, some of the pressing issues remaining are Issue 4 (http://trac.tools.ietf.org/wg/hybi/trac/ticket/4), and any
ideas anyone has (including NPN, or proposing an alternate that still meets the required properties) would be helpful.
I thought a couple of people posted that the simple solution of
requiring the WS server to not read the random bytes until it had
written the 101 response header solved the problem of otherwise
functional proxies causing the WS server to hang?

--
John A. Tamplin
Software Engineer (GWT), Google
Greg Wilkins
2010-09-23 16:53:21 UTC
Permalink
Post by John Tamplin
On Thu, Sep 23, 2010 at 11:36 AM, Ian Fette (イアンフェッティ)
I will try to put together a -02 by the end of the week and see how far I get, and then let's see how we can move forward.
I can help with that if you need it as well.
In my mind, some of the pressing issues remaining are Issue 4 (http://trac.tools.ietf.org/wg/hybi/trac/ticket/4), and any
ideas anyone has (including NPN, or proposing an alternate that still meets the required properties) would be helpful.
I thought a couple of people posted that the simple solution of
requiring the WS server to not read the random bytes until it had
written the 101 response header solved the problem of otherwise
functional proxies causing the WS server to hang?
It may solve "the problem" of a particular intermediary, but I don't
think it justifies the existence of those random bytes on the wire as
not HTTP and not Websocket data.

The inclusion of those bytes was never discussed or justified. There
is still no rigorous explanation of what benefit they provide that
could not be provided with an early ping/pong like:

Client Handshake (with nonce) --->
<---- Server handshake
<---- Ping (with hash of nonce)
onOpen called
Pong ------>

This give no extra round trip before the client onopen is called and
verifies that at least server->client websocket data can be sent. If
client->server data cannot be sent, then the pong will not be received
and the connection will be shutdown. Note that a ws connection can
fail at any time, so this provides no extra error handling burden on
applications as they have to handle this case anyway.


cheers
Willy Tarreau
2010-09-23 18:52:29 UTC
Permalink
Post by Greg Wilkins
On Thu, Sep 23, 2010 at 11:36 AM, Ian Fette (????????????????????????)
I will try to put together a -02 by the end of the week and see how far I get, and then let's see how we can move forward.
I can help with that if you need it as well.
In my mind, some of the pressing issues remaining are Issue 4 (http://trac.tools.ietf.org/wg/hybi/trac/ticket/4), and any
ideas anyone has (including NPN, or proposing an alternate that still meets the required properties) would be helpful.
I thought a couple of people posted that the simple solution of
requiring the WS server to not read the random bytes until it had
written the 101 response header solved the problem of otherwise
functional proxies causing the WS server to hang?
It may solve "the problem" of a particular intermediary, but I don't
think it justifies the existence of those random bytes on the wire as
not HTTP and not Websocket data.
The inclusion of those bytes was never discussed or justified.
The only valid advantage that I've seen (explained by hixie) was to
validate the ability to talk in both directions. But with the new
framing, you proposed the ping+pong which appeared a lot better to
me because not only does it validate the ability to talk, it also
validates the ability to speak the WebSocket protocol, which is
even more important.
Post by Greg Wilkins
There
is still no rigorous explanation of what benefit they provide that
Client Handshake (with nonce) --->
<---- Server handshake
<---- Ping (with hash of nonce)
onOpen called
Pong ------>
This give no extra round trip before the client onopen is called and
verifies that at least server->client websocket data can be sent. If
client->server data cannot be sent, then the pong will not be received
and the connection will be shutdown.
The connection will not necessarily be shutdown in this case. Gateways
that don't understand "upgrade" might only see a GET, and not forward
anything from the client to the server, but only a respose from the
server to the client until the server closes. And unfortunately, the
client has no way to know if its data are delivered.

I think it is reasonable to send the ping from the client first,
provided that we add "Connection: close" in order to prevent anyone
in between from considering the added data as a possible pipelined
request.

Another thing I really like with a ping+pong opening is that it can
easily be reused and extended for non-HTTP handshakes (even raw TCP).
Post by Greg Wilkins
Note that a ws connection can
fail at any time, so this provides no extra error handling burden on
applications as they have to handle this case anyway.
100% agree, and that's something that should clearly be stated in the doc.
I too often hear people say "my application *requires* HTTP keep-alive", so
we should be clear that anyone can break the connection without prior notice.

Regards,
Willy
S Moonesamy
2010-09-23 16:05:05 UTC
Permalink
Hi Ian,
Post by Ian Fette (イアンフェッティ)
Apologies. I did have minor surgery on my foot Monday. Everything is
fine, but I'm taking a bit of time to recover. It's also performance
management season at Google so we're all
I wish you a prompt recovery. I understand that you may need some
time before you can get back to editing the draft.

Regards,
S. Moonesamy
Ian Fette (イアンフェッティ)
2010-09-24 21:14:04 UTC
Permalink
On Thu, 02 Sep 2010 18:24:58 +0200, Ian Fette (むアンフェッティ) <
Some of those changes were in response to feedback from me (mostly sent to
Post by Ian Fette (イアンフェッティ)
whatwg). I was happy with the changes Ian Hickson made. What's the plan? Is
there something I should do if I want those changes in?
Cheers,
IMO best thing would be to send an email to this list explaining what the
change is, and why it was needed. I don't want to lose these changes, I just
want to make sure they are discussed on this list.
Ok. Here are the changes I could find since -00 and associated emails/bugs
ignoring things that are moot with -01 and things that IIRC were discussed
Revisions below refer to revisions of
http://svn.tools.ietf.org/svn/wg/hybi/websocket/draft-ietf-hybi-thewebsocketprotocol.xml
http://html5.org/tools/web-apps-tracker?from=4893&to=4894
http://www.w3.org/Bugs/Public/show_bug.cgi?id=9149
This change as best as I can tell is already in -01
http://html5.org/tools/web-apps-tracker?from=5169&to=5170
https://bugzilla.mozilla.org/show_bug.cgi?id=472529#c180 (can't find the
relevant email)
Committed as r15
http://html5.org/tools/web-apps-tracker?from=5174&to=5189
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-July/027245.html
As best as I can tell, the below corresponds to the part of the source file
that was the JavaScript API, not the protocol. This text is not present in
the HyBi document. I have not incorporated this change.
--- source (revision 5174)
+++ source (revision 5189)
@@ -80897,8 +80897,9 @@
title="">host</var>, on port <var title="">port</var> (if one was
specified), from <var title="">origin</var>, with the flag <var
title="">secure</var>, with <var title="">resource name</var> as
- the resource name, and with <var title="">protocols</var> as the
- (possibly empty) list of protocols.</p>
+ the resource name, with <var title="">protocols</var> as the
+ (possibly empty) list of protocols, and with the <var
+ title="">defer cookies</var> flag set.</p>


Ditto with the below:
<p class="note">If the "<span>establish a WebSocket
connection</span>" algorithm fails, it triggers the "<span>fail
@@ -81137,10 +81138,13 @@
<p>When the <i>WebSocket connection is established</i>, the user
agent must <span>queue a task</span> to first change the <code
title="dom-WebSocket-readyState">readyState</code> attribute's value
- to <code title="dom-WebSocket-OPEN">OPEN</code> (1); then change the
- <code title="dom-WebSocket-protocol">protocol</code> attribute's
- value to the <span>selected WebSocket subprotocol</span>, if there
- is one; and then <span>fire a simple event</span> named <code
+ to <code title="dom-WebSocket-OPEN">OPEN</code> (1); <span>apply the
+ cookies</span> that were collected in the <var title="">list of
+ cookies</var> when the <span title="WebSocket connection is
+ established">connection was established</span>; change the <code
+ title="dom-WebSocket-protocol">protocol</code> attribute's value to
+ the <span>selected WebSocket subprotocol</span>, if there is one;
+ and then <span>fire a simple event</span> named <code
title="event-open">open</code> at the <code>WebSocket</code>
object.</p>

Again, the below seems to be in the HTML/JS api, not the protocol, so I have
not incorporated any of the below.
@@ -81215,10 +81219,40 @@

<h5>Garbage collection</h5>

- <p>A <code>WebSocket</code> object with an open connection must not
- be garbage collected if there are any event listeners registered for
- <code title="event-message">message</code> events.</p>
+ <p>A <code>WebSocket</code> object whose <code
+ title="dom-WebSocket-readyState">readyState</code> attribute's value
+ was set to <code title="dom-WebSocket-CONNECTING">CONNECTING</code>
+ (0) as of the last time the <span>event loop</span> started
+ executing a <span title="concept-task">task</span> must not be
+ garbage collected if there are any event listeners registered for
+ <code title="event-open">open</code> events, <code
+ title="event-message">message</code> events, <code
+ title="event-error">error</code> events, or <code
+ title="event-close">close</code> events.</p>

+ <p>A <code>WebSocket</code> object whose <code
+ title="dom-WebSocket-readyState">readyState</code> attribute's value
+ was set to <code title="dom-WebSocket-OPEN">OPEN</code> (1) as of
+ the last time the <span>event loop</span> started executing a <span
+ title="concept-task">task</span> must not be garbage collected if
+ there are any event listeners registered for <code
+ title="event-message">message</code> events, <code
+ title="event-error">error</code> events, or <code
+ title="event-close">close</code> events.</p>
+
+ <p>A <code>WebSocket</code> object whose <code
+ title="dom-WebSocket-readyState">readyState</code> attribute's value
+ was set to <code title="dom-WebSocket-CLOSING">CLOSING</code> (2) as
+ of the last time the <span>event loop</span> started executing a
+ <span title="concept-task">task</span> must not be garbage collected
+ if there are any event listeners registered for <code
+ title="event-close">close</code> events.</p>
+
+ <p>A <code>WebSocket</code> object with <span title="WebSocket
+ connection is established">an established connection</span> that has
+ data queued to be transmitted to the network must not be garbage
+ collected.</p>
+
<p>If a <code>WebSocket</code> object is garbage collected while its
connection is still open, the user agent must <span>close the
WebSocket connection</span>.</p>

Committed as r16
@@ -81540,7 +81574,7 @@

<p>The third piece of information is given after the fields, in the
last eight bytes of the handshake, expressed here as they would be
- seen if interpreted as ASCII:</p>
+ seen if interpreted as UTF-8:</p>

<pre>Tm[K T2u</pre>

Also in r16
@@ -81597,7 +81631,7 @@
<em>set</em> cookies, as in HTTP.</p>

<p>After the fields, the server sends the aforementioned MD5 sum, a
- 16 byte (128 bit) value, shown here as if interpreted as ASCII:</p>
+ 16 byte (128 bit) value, shown here as if interpreted as UTF-8:</p>

<pre>fQJ,fN/4F4!~K~MH</pre>


Incorporated in r17
@@ -81935,6 +81969,13 @@
<p><i>This section only applies to user agents, not to
servers.</i></p>

+ <p>User agents running in controlled environments, e.g. browsers on
+ mobile handsets tied to specific carriers, may offload the
+ management of the connection to another agent on the network. In
+ such a situation, the user agent for the purposes of conformance is
+ considered to include both the handset software and any such
+ agents.</p>
+
<p class="note">This specification doesn't currently define a limit
to the number of simultaneous connections that a client can
establish to a server.</p>

Also in r17
@@ -81947,17 +81988,18 @@
title="">port</var>, from an origin whose <span title="ASCII
serialization of an origin">ASCII serialization</span> is <var
title="">origin</var>, with a flag <var title="">secure</var>, with
- a string giving a <var title="">resource name</var>, and with a
+ a string giving a <var title="">resource name</var>, with a
(possibly empty) list of strings giving the <var
- title="">protocols</var>, it must run the following steps. The <var
- title="">host</var> must be ASCII-only (i.e. it must have been
- punycode-encoded already if necessary). The <var
- title="">origin</var> must not contain characters in the range
+ title="">protocols</var>, and optionally with a <var title="">defer
+ cookies</var> flag, it must run the following steps. The <var
+ title="">host</var> must have been punycode-encoded already if
+ necessary (i.e. it does not contain characters above U+007E). The
+ <var title="">origin</var> must not contain characters in the range
U+0041 to U+005A (i.e. LATIN CAPITAL LETTER A to LATIN CAPITAL
LETTER Z). The <var title="">resource name</var> string must be a
- non-empty string of ASCII characters in the range U+0021 to U+007E
- that starts with a U+002F SOLIDUS character (/). The various strings
- in <var title="">protocols</var> must all be non-empty strings with
+ non-empty string of characters in the range U+0021 to U+007E that
+ starts with a U+002F SOLIDUS character (/). The various strings in
+ <var title="">protocols</var> must all be non-empty strings with
characters in the range U+0021 to U+007E, and must all be unique. <a
href="#refsORIGIN">[ORIGIN]</a></p>

Added as r18
@@ -82194,6 +82236,16 @@
considered a non-HTTP API for the purpose of cookie
processing.</p>

+ <p>When one or more HTTP headers are to be added to <var
+ title="">fields</var> for this step, each header must be added
+ separately, and each header must be added as one entry consisting
+ of the header's name in its canonical case, followed by a U+003A
+ COLON character (:) and a U+0020 SPACE character, followed by the
+ value with no use of continuation lines (i.e. containing no U+000A
+ LINE FEED characters).</p>
+
+ <!-- (Cookie headers only use one header for multiple cookies.) -->
+
<!--(v2-ws-auth)
<div class="example">

r19
@@ -82398,6 +82452,7 @@

</li>

+<!--(removed since the next step is even stricter)
<li>

<p>If <var title="">code</var> is not three bytes long, or if any

r19
@@ -82406,6 +82461,7 @@
these steps.</p>

</li>
+-->

<li>

r19
@@ -82380,8 +82432,10 @@

<p>If <var title="">field</var> is not at least seven bytes long,
or if the last two bytes aren't 0x0D and 0x0A respectively, or if
- it does not contain at least two 0x20 bytes, then <span>fail the
- WebSocket connection</span> and abort these steps.</p>
+ <var title="">field</var> contains any 0x0D bytes other than the
+ penultimate byte, or if <var title="">field</var> does not contain
+ at least two 0x20 bytes, then <span>fail the WebSocket
+ connection</span> and abort these steps.</p>

<p>User agents may apply a timeout to this step, <spa




r20
@@ -82463,7 +82519,7 @@

<dl class="switch">

- <dt>If the byte is 0x0D (ASCII CR)</dt>
+ <dt>If the byte is 0x0D (UTF-8 CR)</dt>

<dd>If the <var title="">name</var> byte array is empty, then
jump to the <a href="#ws-ua-fields-processing">fields

r20
@@ -82471,18 +82527,18 @@
connection</span> and abort these steps.</dd>


- <dt>If the byte is 0x0A (ASCII LF)</dt>
+ <dt>If the byte is 0x0A (UTF-8 LF)</dt>

<dd><span>Fail the WebSocket connection</span> and abort these
steps.</dd>


- <dt>If the byte is 0x3A (ASCII :)</dt>
+ <dt>If the byte is 0x3A (UTF-8 :)</dt>

<dd>Move on to the next step.</dd>


- <dt>If the byte is in the range 0x41 to 0x5A (ASCII A-Z)</dt>
+ <dt>If the byte is in the range 0x41 to 0x5A (UTF-8 A-Z)</dt>

<dd>Append a byte whose value is the byte's value plus 0x20 to
the <var title="">name</var> byte array and redo this step for

r20
@@ -82497,8 +82553,8 @@
</dl>

<p class="note">This reads a field name, terminated by a colon,
- converting upper-case ASCII letters to lowercase, and aborting if
- a stray CR or LF is found.</p>
+ converting upper-case letters in the range A-Z to lowercase, and
+ aborting if a stray CR or LF is found.</p>

</li>

r20
@@ -82525,17 +82581,17 @@

<dl class="switch">

- <dt>If the byte is 0x20 (ASCII space) and <var title="">count</var>
equals 1</dt>
+ <dt>If the byte is 0x20 (UTF-8 space) and <var title="">count</var>
equals 1</dt>

<dd>Ignore the byte and redo this step for the next byte.</dd>


- <dt>If the byte is 0x0D (ASCII CR)</dt>
+ <dt>If the byte is 0x0D (UTF-8 CR)</dt>

<dd>Move on to the next step.</dd>


- <dt>If the byte is 0x0A (ASCII LF)</dt>
+ <dt>If the byte is 0x0A (UTF-8 LF)</dt>

<dd><span>Fail the WebSocket connection</span> and abort these
steps.</dd>

r20
@@ -82558,7 +82614,8 @@
<p>Read a byte from the server.</p>

<p>If the connection closes before this byte is received, or if
- the byte is not a 0x0A byte (ASCII LF), then <span>fail the WebSocket
connection</span> and abort these steps.</p>
+ the byte is not a 0x0A byte (UTF-8 LF), then <span>fail the
+ WebSocket connection</span> and abort these steps.</p>

<p class="note">This skips past the LF byte of the CRLF after the
field.</p>

r20
@@ -82587,13 +82644,16 @@
<p><i>Fields processing</i>: Read a byte from the server.</p>

<p>If the connection closes before this byte is received, or if
- the byte is not a 0x0A byte (ASCII LF), then <span>fail the WebSocket
connection</span> and abort these steps.</p>
+ the byte is not a 0x0A byte (UTF-8 LF), then <span>fail the
+ WebSocket connection</span> and abort these steps.</p>

<p class="note">This skips past the LF byte of the CRLF after the
blank line after the fields.</p>

</li>

+ <li><p>Let the <var title="">list of cookies</var> be empty.</p></li>
+
<li>

<p>

r21
@@ -82629,9 +82689,9 @@
<dt>If the entry's name is "<code
title="http-upgrade">upgrade</code>"</dt>

- <dd><p>If the value is not exactly equal to the string
- "WebSocket", then <span>fail the WebSocket connection</span> and
- abort these steps.</p></dd>
+ <dd><p>If the value, <span>converted to ASCII lowercase</span>,
+ is not exactly equal to the string "websocket", then <span>fail
+ the WebSocket connection</span> and abort these steps.</p></dd>


<dt>If the entry's name is "<code

r21
@@ -82690,8 +82750,9 @@
<dd>

<p>If the relevant specification is supported by the user agent,
- handle the cookie as defined by the appropriate specification,
- with the resource being the one with the host <var
+ add the cookie, interpreted as defined by the appropriate
+ specification, to the <var title="">list of cookies</var>, with
+ the resource being the one with the host <var
title="">host</var>, the port <var title="">port</var>, the path
(and possibly query parameters) <var title="">resource
name</var>, and the scheme <code title="">http</code> if <var

r21
@@ -82708,6 +82769,11 @@
<p>If the relevant specification is not supported by the user
agent, then the field must be ignored.</p>

+ <p class="note">The cookies added to the <var title="">list of
+ cookies</var> are discarded if the connection fails to be
+ established. Only if and when the connection is established do
+ the cookies actually get applied.</p>
+
</dd>

***** This text is relating to a portion of the specification
(redirects/location: header) that is no longer in the spec *****
@@ -82764,6 +82830,9 @@
<li><p>Close the connection if the server has not already done
so.</p></li>

+ <li><p><span>Apply the cookies</span> in the <var title="">list
+ of cookies</var>.</p></li>
+
<li><p>Jump back to the first step of the overall algorithm
(the very top of the handshake).</p></li>

***** ditto -- not in the spec *****
@@ -82794,19 +82863,24 @@
<dt>If the entry's name is "<code
title="">www-authenticate</code>"</dt>

- <dd><p>Obtain credentials in a manner consistent with the
- requirements for handling the <code>WWW-Authenticate</code>
- header in HTTP, and then close the connection (if the server has
- not already done so) and jump back to the step labeled
- <i>connect</i>, including the relevant authentication headers in
- the new request.
+ <dd>
+
+ <p><span>Apply the cookies</span> in the <var title="">list of
+ cookies</var>, then obtain credentials in a manner consistent
+ with the requirements for handling the
+ <code>WWW-Authenticate</code> header in HTTP, and then close
+ the connection (if the server has not already done so) and jump
+ back to the step labeled <i>connect</i>, including the relevant
+ authentication headers in the new request.
--><!--END complete--><!--END epub--><!--
- <a href="#refsRFC2616">[RFC2616]</a>
+ <a href="#refsRFC2616">[RFC2616]</a>
--><!--START complete--><!--START epub--><!--END websocket-protocol--><!--
- <a href="#refsHTTP">[HTTP]</a>
+ <a href="#refsHTTP">[HTTP]</a>
--><!--START websocket-protocol--><!--
- </p></dd>
+ </p>

+ </dd>
+
<dt>Any other name</dt>

<dd>Ignore it.</dd>

r22
@@ -82847,7 +82921,7 @@

<p class="example">Using the examples given earlier, this leads to
the 16 bytes 0x30 0x73 0x74 0x33 0x52 0x6C 0x26 0x71 0x2D 0x32
- 0x5A 0x55 0x5E 0x77 0x65 0x75. In ASCII, these bytes correspond to
+ 0x5A 0x55 0x5E 0x77 0x65 0x75. In UTF-8, these bytes correspond to
the string "0st3Rl&amp;q-2ZU^weu".</p>

</li>

r22
@@ -82871,17 +82945,58 @@

</li>

+ <li><p>If the <var title="">defer cookies</var> flag is not set,
+ <span>apply the cookies</span> in the <var title="">list of
+ cookies</var>.</p></li>
+
<li>

<p>The <dfn>WebSocket connection is established</dfn>. Now the
user agent must send and receive to and from the connection as
described in the next section.</p>

+ <p>If the <var title="">defer cookies</var> flag is set, store the
+ <var title="">list of cookies</var> for use by the component that
+ invoked this algorithm.</p>
+
</li>

</ol>

+ <p>Where the algorithm above requires that a user agent <span>fail
+ the WebSocket connection</span>, the user agent may first read an
+ arbitrary number of further bytes from the connection (and then
+ discard them) before actually <span title="fail the WebSocket
+ connection">failing the WebSocket connection</span>. Similarly, if a
+ user agent can show that the bytes read from the connection so far
+ are such that there is no subsequent sequence of bytes that the
+ server can send that would not result in the user agent being
+ required to <span>fail the WebSocket connection</span>, the user
+ agent may immediately <span>fail the WebSocket connection</span>
+ without waiting for those bytes.</p>

+ <p class="note">The previous paragraph is intended to make it
+ conforming for user agents to implement the algorithm in subtlely
+ different ways that are equivalent in all ways except that they
+ terminate the connection at earlier or later points. For example, it
+ enables an implementation to buffer the entire handshake response
+ before checking it, or to verify each field as it is received rather
+ than collecting all the fields and then checking them as a
+ block.</p>
+
+ <p>When the user agent is to <dfn>apply the cookies</dfn> in a <var
+ title="">list of cookies</var>, it must handle each cookie in the
+ <var title="">list of cookies</var> as defined by the appropriate
+ specification.
+<!--END complete--><!--END epub-->
+ <a href="#refsRFC2109">[RFC2109]</a>
+ <a href="#refsRFC2965">[RFC2965]</a>
+<!--START complete--><!--START epub--><!--END websocket-protocol-->
+ <a href="#refsCOOKIES">[COOKIES]</a>
+<!--START websocket-protocol-->
+ </p>
+
+
<h6>Data framing</h6>

<p>Once a <span>WebSocket connection is established</span>, the user


**** dead text *****
@@ -82986,8 +83101,14 @@

</ol>

- <p>Otherwise, let <var title="">error</var> be true.</p>
+ </li>

+ <li>
+
+ <p>Otherwise, if the <var title="">frame type</var> is not
+ 0xFF or if the <var title="">length</var> was not 0, let <var
+ title="">error</var> be true.</p>
+
</li>

</ol>

**** dead text *****
@@ -83096,9 +83217,10 @@

</ol>

- <p class="note">The closing handshake <span title="the WebSocket
- closing handshake has finished">finishes</span> once the server
- returns the 0xFF packet, as described above.</p>
+ <p class="note">If the user agent initiates it, the closing
+ handshake <span title="the WebSocket closing handshake has
+ finished">finishes</span> once the server returns the 0xFF frame, as
+ described above.</p>

<hr>

r23
@@ -83125,7 +83247,25 @@

<p><i>This section only applies to servers.</i></p>

+ <p>Servers may offload the management of the connection to other
+ agents on the network, for example load balancers and reverse
+ proxies. In such a situation, the server for the purposes of
+ conformance is considered to include all parts of the server-side
+ infrastructure from the first device to terminate the TCP connection
+ all the way to the server that processes requests and sends
+ responses.</p>

+ <div class="example">
+
+ <p>For example, a data center might have a server that responds to
+ Web Socket requests with an appropriate handshake, and then passes
+ the connection to another server to actually process the data
+ frames. For the purposes of this specification, the "server" is the
+ combination of both computers.</p>
+
+ </div>
+
+
<h6>Reading the client's opening handshake</h6>

<p>When a client starts a WebSocket connection, it sends its part of

r24
@@ -83355,7 +83495,10 @@
requests to multiple hosts (e.g. in a virtual hosting
environment), then the value should be derived from the client's
handshake, specifically from the "<code
- title="http-host">Host</code>" field.</dd>
+ title="http-host">Host</code>" field. The <var
+ title="">host</var> value must be lowercase (not containing
+ characters in the range U+0041 LATIN CAPITAL LETTER A to U+005A
+ LATIN CAPITAL LETTER Z).</dd>

<dt><var title="">port</var></dt>


r25
@@ -83560,7 +83703,7 @@

<p class="example">In the example above, this would be the 16
bytes 0x6E 0x60 0x39 0x65 0x42 0x6B 0x39 0x7A 0x24 0x52 0x38 0x70
- 0x4F 0x74 0x56 0x62, or "n`9eBk9z$R8pOtVb" in ASCII.
+ 0x4F 0x74 0x56 0x62, or "n`9eBk9z$R8pOtVb" in UTF-8.

</li>

r25
@@ -83640,7 +83783,7 @@

<li>

- <p>Send two bytes 0x0D 0x0A (ASCII CRLF).</p>
+ <p>Send two bytes 0x0D 0x0A (UTF-8 CRLF).</p>

</li>


r26
@@ -83655,9 +83798,9 @@
<p>This completes the server's handshake. If the server finishes
these steps without <span title="abort the WebSocket
connection">aborting the WebSocket connection</span>, and if the
- client does not then <span>fail the connection</span>, then the
- connection is established and the server may begin and receiving
- sending data, as described in the next section.</p>
+ client does not then <span>fail the WebSocket connection</span>,
+ then the connection is established and the server may begin and
+ receiving sending data, as described in the next section.</p>


**** dead text ***
@@ -83801,7 +83944,9 @@
<hr>

<p>At any time, the server may decide to terminate the WebSocket
- connection by running through the following steps:</p>
+ connection by running through the following steps. These steps
+ should be run when the <var title="">client terminated</var> flag is
+ set, if they aren't already running.</p>

<ol>


*** irrelevant***
@@ -83903,7 +84048,6 @@
connection</span> arbitrarily.</p>


-
<h5>Security considerations</h5>

<p>While this protocol is intended to be used by scripts in Web


http://html5.org/tools/web-apps-tracker?from=5241&to=5244
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027673.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027674.html
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-August/027675.html
Most of this did not apply as it was all framing. I applied the following in
r27

@@ -82946,8 +82937,8 @@
<li>

<p>Let <var title="">key<sub>3</sub></var> be a string consisting
- of eight random bytes (or equivalently, a random 64 bit integer
- encoded in big-endian order).</p>
+ of eight random bytes (or equivalently, a random 64 bit unsigned
+ integer encoded in big-endian order).</p>

<p class="example">For example, 0x47 0x30 0x22 0x2D 0x5A 0x3F 0x47
0x58.</p>
http://html5.org/tools/web-apps-tracker?from=5287&to=5288
I really want to add this in (NPN), but there has been enough discussion
about the state of NPN and the fact that it's not yet a recommendation that
I am leaving this out for now pending further discussion.
http://html5.org/tools/web-apps-tracker?from=5288&to=5289
I don't know what this was in response to.
ditto, pending more NPN discussion.
Let me know if there's a change you'd like discussed here that's not listed
above and I can try to dig it up.
HTH,
--
Simon Pieters
Opera Software
Patrick McManus
2010-09-02 19:00:39 UTC
Permalink
Ian,

Thanks for the new draft! I think it is great that we all have a
document to focus on moving forward; this is a big help.

Websockets Group,

I've read through the document and I wanted to provide a list of mostly
editorial-style updates to the portions that deal with framing. They are
generally not meant to change the substance of the document but rather
to improve clarity and help us focus on the real issues. Very much a
refinement of what Ian achieved with -01.

However, some of the changes do go beyond syntax changes (see control
messages) in order to address some basic problems. But any really
significant changes in meaning will go into separate email threads.

There is good material in section 1 and especially 4 and my goal is to
especially strengthen the normative material in section 4 and get it at
least a little closer to ietf-standards speak.

Like your revision, I have limited myself to the framing considerations.
Basically that's section 4 and parts of section 1. The other stuff
obviously still needs to be addressed.

Each of my proposed changes is described below, broken apart by
subsection. I've also attached copies of the draft with my edits
included and a diff between that and -01, so people can easily review
them.

-Patrick (proposed changes below)

Section 1.2 - Protocol Overview. There is good normative content here
that is too detailed for the overview and that should be in section 4
instead (as it is wasted in the non-normative section 1).

* Change the text "On the network layer, a message may be represented
as one or more frames." to be "The websocket message does not
necessarily correspond to any network layer framing.".. This is more
correct as the network layer may coalesce as well as fragment.

* strike the paragraph beginning "to close" as it too low level of a
detail to be considered part of the overview and is redundant to the
normative text.

* For similar reasons, strike the paragraph "the protocol is designed"
but add the more summary oriented sentence "This version of the protocol
defines six frame types and leaves ten reserved for future use." to the
end of the section 1.2 paragraph that begins "Data is sent". The
original sentence doesn't reflect the current allocation of op codes.

* move the ABNF portion to normative section 4.2.

Section 4.1 - normative Data Framing Overview

* strike the sentence "The base framing protocol is deliberately kept
simple so that simple implementations may ignore advanced features."
There is no definition of simple implementations, no definition of
advanced features, and no definition of what ignore might mean in this
context. What is an implementor to do with it? :).. I personally don't
think we need the text at all, but if we want it move it to a
non-normative section.

* update the remainder of 4.1 to standards-speak: "In the absence of
extensions negotiated during the opening handshake (Section 5), all
reserved bits MUST be 0 and reserved opcode values MUST NOT be used."..
actually this probably belongs somewhere other than the overview.

* Add to that 4.1 paragraph the text "A data frame MAY be transmitted
by either the client or the server at any time after handshake
completion and before that host has generated a close message." That
probably means we need a normative definition of client and server
somewhere - but I think this is an important thing to say in a place
other than section 1.

Section 4.2 -

* as noted the ABNF from 1.2 is moved here

* Payload length - before "The payload length is" .. insert the
sentence "Multibyte length quantities are expressed in network byte
order." - that's an impt omission.

* I had trouble understanding where the extension data length came
from. I now (think) I understand that it is part of the extension
definiton or negotiation itself. I changed the Extension Data definition
to reflect this requirement more clearly:

"The extension data is 0 bytes unless there is a reserved op-code
or reserved bit present in the frame which indicates an
extension. Any extension MUST specify the length of the
extension data and its use MUST be negotiated during the
handshake."


Section 4.3 - Fragmentation

* Change the paragraph that begins "a sender MAY arbitrarily fragment a
single message (which allows generation of dynamic content without
having to buffer the data in order to count it)." to read more simply "A
sender MAY create fragments of any size for non control messages." If we
need to explain why fragments are important, which I don't think we need
to do, that can live outside of normative section 4.

* Change the paragraph "A receiver MUST be prepared to accept
arbitrarily fragmented messages, even if the sender sent the message in
a single frame." to the more direct "A receiver MUST support receipt of
fragmented messages.". .. I assume the text about "sender sent .. in
single frame" is referring to intermediaries, but it can't really happen
like that - the intermediary is a sender too. To express the concept we
would have to define something like origin-server/user-agent which are
subsets of senders.. but I really don't see why this paragraph needs the
concept at all.

* Change the paragraph "An intermediary MAY fragment .." to read "An
intermediary MAY change the fragmentation of a message if the message
uses an opcode and reserved bit values known to the intermediary." This
change allows coalescing as well as fragmentation, and using the "if"
keeps us out of the awkward "may.. except must not" language the text
was using, and it restricts the section 4.3 requirement to only deal
with fragmentation.

Section 4.4 - Control Frames

I've done some more significant cleanups here that go a bit beyond the
editorial. Specifically I want to include some of James Graham's
concerns.

As drafted a ping and pong are meant to be paired together via their
content.. but the pong is allowed to truncate the content but isn't
given any mechanism to indicate that it has done so.. in light of that
how can the pinger determine if there is a match? If those truncated
bytes aren't relevant to the match, why did it send them in the first
place? My answer to that is to disallow truncation but limit the message
size to the tiny 125 byte frame size. If people want to argue for a
larger but still reasonable constant I don't have a problem with that.
Furthermore the draft made responding to the ping optional (SHOULD
level) which really makes it kind of useless as a keep-alive.. so I
changed that to MUST respond, but SHOULD do it soonish. (see text).

We've been calling them control frames, not control messages, so I
assume that means they shouldn't be fragmented and I made that a
requirement. The alternative is to try and consistently call them
control messages. Personally, I don't see a reason to let them be
fragmented into multiple frames.

With that being said - this is the proposed replacement for section 4.4:

=============

4.4. Control Frames

Control frames have opcodes of 0x01, 0x02, or 0x03. They are used
to communicate state about the websocket.

All control frames MUST be 125 bytes or less in length and MUST NOT
be fragmented.

4.4.1 Close

The Close message contains an opcode of 0x01.

The application MUST NOT send any more data messages after
sending a close message.

A recevied close message is deemed to be an acknowledgement if the
message body matches the body of a close message previously sent by
the receiver. Otherwise the close message is a close initiated by
the sender.

Upon receipt of an initiated close the endpoint MUST send a close
acknowledgment. It should do so as soon as is practical.

The websocket is considered fully closed when an endpoint has either
received a close acknowledgment or sent a close acknowledgment.


4.4.2 Ping

The Ping message contains an opcode of 0x02.

Upon receipt of a Ping message, an endpoint MUST send a Pong
message in response. It SHOULD do so as soon as is practical. The
message bodies of the Ping and Pong MUST be the same.

4.4.3 Pong

The Pong message contains an opcode of 0x03.

Upon receipt of a Ping message, an endpoint MUST send a Pong
message in response. It SHOULD do so as soon as is practical. The
message bodies of the Ping and Pong MUST be the same.

======================

Section 4.5 Data Frames

* replace "not listed above" with "not listed in Section 4.4"

* strike the paragraph "Additional data frame types" .. its just not
necessary for this part of the normative document and is redundant with
the overview section which describes future use.

Section 4.7 Extensibility

With respect to section 4 framing there isn't a lot to say about
extensibility. I replaced the existing section 4.7 text with this:

" The endpoints MUST negotiate the use of any extensions during the
handshake. This specification provides opcodes 0x6 through 0xF, the
extension data field, and the frame-rsv1, frame-rsv2,
frame-rsv3, and frame-rsv4 bits of the frame header for use by
extensions. "
John Tamplin
2010-09-02 20:06:07 UTC
Permalink
Leaving out the suggestions I agree completely with.
Post by Patrick McManus
Section 4.1 - normative Data Framing Overview
* strike the sentence "The base framing protocol is deliberately kept
simple so that simple implementations may ignore advanced features."
There is no definition of simple implementations, no definition of
advanced features, and no definition of what ignore might mean in this
context. What is an implementor to do with it? :).. I personally don't
think we need the text at all, but if we want it move it to a
non-normative section.
Maybe there needs to be a separate document with the rationale, or perhaps
they can be kept inline in the draft with the understanding that they will
not be present in the final version, but this and other statements were
intended to make clear the rationale for the choice and to alleviate
concerns held by those wanting a simple protocol and those wanting to make
sure their more complex use cases would be covered.
Post by Patrick McManus
* Change the paragraph that begins "a sender MAY arbitrarily fragment a
single message (which allows generation of dynamic content without
Post by Patrick McManus
having to buffer the data in order to count it)." to read more simply "A
sender MAY create fragments of any size for non control messages." If we
need to explain why fragments are important, which I don't think we need
to do, that can live outside of normative section 4.
This is another case like above, giving the rationale to head-off
objections. It can go inline annotated as such, a separate non-normative
section collecting all the rationale bits, or a separate document.
Post by Patrick McManus
* Change the paragraph "A receiver MUST be prepared to accept
arbitrarily fragmented messages, even if the sender sent the message in
a single frame." to the more direct "A receiver MUST support receipt of
fragmented messages.". .. I assume the text about "sender sent .. in
single frame" is referring to intermediaries, but it can't really happen
like that - the intermediary is a sender too. To express the concept we
would have to define something like origin-server/user-agent which are
subsets of senders.. but I really don't see why this paragraph needs the
concept at all.
As mentioned in an earlier reply on the list, it was to make clear that even
if someone wrote both the receiver and the sender endpoints, the receiver
must support receiving fragments even if the sender didn't send them. If it
is sufficiently clear that requiring a receiver to support fragmentation
means even if the sending endpoint (as differentiated from an intermediary)
didn't fragment the message, then I am fine with that.
Post by Patrick McManus
As drafted a ping and pong are meant to be paired together via their
content.. but the pong is allowed to truncate the content but isn't
given any mechanism to indicate that it has done so.. in light of that
how can the pinger determine if there is a match? If those truncated
bytes aren't relevant to the match, why did it send them in the first
place? My answer to that is to disallow truncation but limit the message
size to the tiny 125 byte frame size. If people want to argue for a
larger but still reasonable constant I don't have a problem with that.
Furthermore the draft made responding to the ping optional (SHOULD
level) which really makes it kind of useless as a keep-alive.. so I
changed that to MUST respond, but SHOULD do it soonish. (see text).
The idea was that we require a minimum size for truncation, which would be
sufficient to include an id and/or timestamp. I would be fine with your
suggestion as well.
Post by Patrick McManus
We've been calling them control frames, not control messages, so I
assume that means they shouldn't be fragmented and I made that a
requirement. The alternative is to try and consistently call them
control messages. Personally, I don't see a reason to let them be
fragmented into multiple frames.
I agree with the general principle of not fragmenting control messages (and
it also solves the problem of interjecting control messages in the middle of
a fragmented message, since you don't have to keep a stack of "open"
messages). However, I worry if later on we will want to have a control
message that may need to be larger, in which case not being able to fragment
it may cause other problems.
With the above caveats, I am fine with your replacement text.
Post by Patrick McManus
Section 4.7 Extensibility
With respect to section 4 framing there isn't a lot to say about
Again, this section was primarily intended to alleviate fears that the base
protocol was not extensible enough to support some of the ideas that have
been discussed.
--
John A. Tamplin
Software Engineer (GWT), Google
Patrick McManus
2010-09-02 21:11:13 UTC
Permalink
Hi John, thanks for reading the long list and providing feedback.
Maybe there needs to be a separate document with the rationale, [..]
but this and other statements were intended to make clear the
rationale for the choice
right, that's a recurring theme of the proposed edits. I think the
result is that an implementor reading the document has a more concise
view of what she needs to do to get it right.

The document is not trying to win a debate, explain the history of every
bit, or change minds - the mailing list archive or (as you suggest) a
separate non standards track document can persist for those purposes.

I'm not opposed to a few statements of rationale here and there when
they provide useful advice to implementors, but the primary focus of a
standards track protocol specification is to provide a clear definition
of what the protocol requirements are for interoperation and quality
implementations.

In cases where a specific element can benefit from some nuance (for
instance a hypothetical "SHOULD create large fragments") then go for the
explanation, but in most cases it really just opens up a can of worms
and clarity is added by constraining the document to what is necessary.

here is an example:

"The base framing protocol is deliberately kept simple so that simple
implementations may ignore advanced features."

Hypothetically I begin my implementation of a websocket server from the
specification and I find 64 bit lengths difficult to work with. Suddenly
ambiguity sets in. Is 64 bit lengths an advanced feature? Can I ignore
it? Am I a simple implementation - what's the definition of that? If I
know I'm an advanced implementation in some other aspects does that
prohibit me from ignoring this requirement? Obviously, I figure out that
I cannot ignore it and interoperate on the web even though this text
tells me I can ignore advanced features! now the standard is internally
inconsistent. yech.

We could try and fix those problems in the text, or we can ask in what
way the goals of interoperation and quality implementations damaged by
just removing it from the specification.

The most borderline of the cases in my mind involves removing the
rationale from "a sender MAY fragment a message". The way the text was
written created one interpretation where a sender may fragment it in
order to generate dyanmic content but it isn't clear if there is then
permission from the spec to do it for other reasons. I think it states
the requirements more clearly if the rationale is left out, the sender
MAY do it. Period. But of course the rationale can always be reworked if
it provides useful advice and as I said this was probably the most
borderline of the ones I cited.
or perhaps they can be kept inline in the draft with the understanding
that they will not be present in the final version,
I would object to this approach - we have lots of issue trackers that
can track the reasons behind various decisions. Its high time to start
moving the document towards its submission state and using the document
and specific proposed changes to it as a focal point for working group
discussions.

-Patrick
Greg Wilkins
2010-09-06 13:24:42 UTC
Permalink
Post by John Tamplin
Leaving out the suggestions I agree completely with.
 * Change the paragraph that begins "a sender MAY arbitrarily fragment a
single message (which allows generation of dynamic content without
having to buffer the data in order to count it)." to read more simply "A
sender MAY create fragments of any size for non control messages." If we
need to explain why fragments are important, which I don't think we need
to do, that can live outside of normative section 4.
Fragments should probably be non-zero size, else we can get silly
infinite fragmentation.
Post by John Tamplin
As drafted a ping and pong are meant to be paired together via their
content.. but the pong is allowed to truncate the content but isn't
given any mechanism to indicate that it has done so.. in light of that
how can the pinger determine if there is a match? If those truncated
bytes aren't relevant to the match, why did it send them in the first
place? My answer to that is to disallow truncation but limit the message
size to the tiny 125 byte frame size. If people want to argue for a
larger but still reasonable constant I don't have a problem with that.
Furthermore the draft made responding to the ping optional (SHOULD
level) which really makes it kind of useless as a keep-alive.. so I
changed that to MUST respond, but SHOULD do it soonish. (see text).
The idea was that we require a minimum size for truncation, which would be
sufficient to include an id and/or timestamp.  I would be fine with your
suggestion as well.
I'm +1 on limiting ping/pong to a single tiny frame is a good idea.

However there was a use-case of sending a large ping to determine the
mtu of a path. I think it would be great to be able to discover mtu,
but I'm not sure that sending a big ping is a good way of doing that.
So if we go with a tiny frame ping/pong, then we need to capture the
mtu use-case elsewhere.
Post by John Tamplin
We've been calling them control frames, not control messages, so I
assume that means they shouldn't be fragmented and I made that a
requirement. The alternative is to try and consistently call them
control messages. Personally, I don't see a reason to let them be
fragmented into multiple frames.
I agree with the general principle of not fragmenting control messages (and
it also solves the problem of interjecting control messages in the middle of
a fragmented message, since you don't have to keep a stack of "open"
messages).  However, I worry if later on we will want to have a control
message that may need to be larger, in which case not being able to fragment
it may cause other problems.
There are op-codes available to be define for large messages that are
not going to be directed at the existing text API. These should be
sufficient for future large non data messages.




cheers
Patrick McManus
2010-09-06 13:51:41 UTC
Permalink
Post by Greg Wilkins
Fragments should probably be non-zero size, else we can get silly
infinite fragmentation.
we can add this - "The frame-more bit MUST NOT be used on any frame with
a 0 application data length." Would you like to propose that (or other)
text?

The zero-chunk on the end might be necessary if the sender didn't know
there wouldn't be more fragments when it sent the last one.
Greg Wilkins
2010-09-06 23:34:01 UTC
Permalink
Post by Patrick McManus
The zero-chunk on the end might be necessary if the sender didn't know
there wouldn't be more fragments when it sent the last one.
Very good point.
Post by Patrick McManus
"The frame-more bit MUST NOT be used on any frame with a 0 application data length."
That is a reasonable compromise.
Willy Tarreau
2010-09-03 15:25:57 UTC
Permalink
Hi Ian,
Post by Ian Fette (イアンフェッティ)
The handshake in -01 is the same as in -00. There are certainly open issues
against the handshake, and I do not mean to dismiss them, they simply are
not addressed in this draft.
FWIW, today I got another report of a user who had to modify his server
implementation of the handshake to support a reverse-proxy. He just had
to make the server respond to the headers and then read the 8-bytes key3
and emit the hash and that fixed the issue for him.

Just wanted to let you know.

Regards,
Willy
Brian McKelvey
2010-09-14 20:54:36 UTC
Permalink
I did exactly the same thing for my project. Modified Socket.IO-node to work behind haproxy. github.com/theturtle32

Sent from my iPhone
Post by Willy Tarreau
Hi Ian,
Post by Ian Fette (イアンフェッティ)
The handshake in -01 is the same as in -00. There are certainly open issues
against the handshake, and I do not mean to dismiss them, they simply are
not addressed in this draft.
FWIW, today I got another report of a user who had to modify his server
implementation of the handshake to support a reverse-proxy. He just had
to make the server respond to the headers and then read the 8-bytes key3
and emit the hash and that fixed the issue for him.
Just wanted to let you know.
Regards,
Willy
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Brian
2010-09-14 21:00:14 UTC
Permalink
Apologies... better link:
http://github.com/Worlize/Socket.IO-node/commit/9982232032771574ceb68e2bccee4e43fd5af887

Brian
I did exactly the same thing for my project.  Modified Socket.IO-node to work behind haproxy.  github.com/theturtle32
Sent from my iPhone
Post by Willy Tarreau
Hi Ian,
Post by Ian Fette (イアンフェッティ)
The handshake in -01 is the same as in -00. There are certainly open issues
against the handshake, and I do not mean to dismiss them, they simply are
not addressed in this draft.
FWIW, today I got another report of a user who had to modify his server
implementation of the handshake to support a reverse-proxy. He just had
to make the server respond to the headers and then read the 8-bytes key3
and emit the hash and that fixed the issue for him.
Just wanted to let you know.
Regards,
Willy
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Greg Wilkins
2010-09-01 23:09:30 UTC
Permalink
Ian,

thanks for this first draft, and I hope it indicates a new phase of
progress for the WG.


I'll be updating Jetty to support this draft immediately, but we have
an issue that there is no version information in the handshake. Thus
in order to support this version of the draft, I risk breaking support
for previous versions, which is difficult as there are already
deployed experimental applications in the wild, interacting with real
clients with browsers beyond the control of the deployers.

Currently user-agent is not sent, so I can't even use that to
determine what version of the draft to use.

I know a standard version header is not widely accepted, but could we
implementers perhaps agree a temporary header just during this draft
phase. Something like:

Sec-WebSocket-Draft: 1

Unless there are howls of complaint, I'll implement my server to look
for such a field and to use it as a hint when selecting the parser to
use.

cheers
Post by I***@ietf.org
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.
       Title           : The WebSocket protocol
       Author(s)       : I. Fette
       Filename        : draft-ietf-hybi-thewebsocketprotocol-01.txt
       Pages           : 57
       Date            : 2010-09-01
The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code.  The
security model used for this is the Origin-based security model
commonly used by Web browsers.  The protocol consists of an initial
handshake followed by basic message framing, layered over TCP.  The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).
This draft is meant to reflect changes in direction in the HyBi
working group.  There is not yet consensus on everything in this
draft.  Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft.  There have also been proposals
to change the handshake, so the handshake is also not in a final
form.
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
ftp://ftp.ietf.org/internet-drafts/
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Adam Barth
2010-09-01 23:14:14 UTC
Permalink
Please don't add versioning to the protocol. Versioning is a
anti-pattern for the web.

Adam
Post by Greg Wilkins
Ian,
thanks for this first draft, and I hope it indicates a new phase of
progress for the WG.
I'll be updating Jetty to support this draft immediately, but we have
an issue that there is no version information in the handshake.  Thus
in order to support this version of the draft, I risk breaking support
for previous versions, which is difficult as there are already
deployed experimental applications in the wild, interacting with real
clients with browsers beyond the control of the deployers.
Currently user-agent is not sent, so I can't even use that to
determine what version of the draft to use.
I know a standard version header is not widely accepted, but could we
implementers perhaps agree a temporary header just during this draft
 Sec-WebSocket-Draft: 1
Unless there are howls of complaint, I'll implement my server to look
for such a field and to use it as a hint when selecting the parser to
use.
cheers
Post by I***@ietf.org
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.
       Title           : The WebSocket protocol
       Author(s)       : I. Fette
       Filename        : draft-ietf-hybi-thewebsocketprotocol-01.txt
       Pages           : 57
       Date            : 2010-09-01
The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code.  The
security model used for this is the Origin-based security model
commonly used by Web browsers.  The protocol consists of an initial
handshake followed by basic message framing, layered over TCP.  The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).
This draft is meant to reflect changes in direction in the HyBi
working group.  There is not yet consensus on everything in this
draft.  Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft.  There have also been proposals
to change the handshake, so the handshake is also not in a final
form.
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
ftp://ftp.ietf.org/internet-drafts/
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Greg Wilkins
2010-09-01 23:18:54 UTC
Permalink
Adam,

my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.

I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.

How else do you expect servers to track the drafts and deal with the
various browser implementations out there?

regards
Please don't add versioning to the protocol.  Versioning is a
anti-pattern for the web.
Adam
Post by Greg Wilkins
Ian,
thanks for this first draft, and I hope it indicates a new phase of
progress for the WG.
I'll be updating Jetty to support this draft immediately, but we have
an issue that there is no version information in the handshake.  Thus
in order to support this version of the draft, I risk breaking support
for previous versions, which is difficult as there are already
deployed experimental applications in the wild, interacting with real
clients with browsers beyond the control of the deployers.
Currently user-agent is not sent, so I can't even use that to
determine what version of the draft to use.
I know a standard version header is not widely accepted, but could we
implementers perhaps agree a temporary header just during this draft
 Sec-WebSocket-Draft: 1
Unless there are howls of complaint, I'll implement my server to look
for such a field and to use it as a hint when selecting the parser to
use.
cheers
Post by I***@ietf.org
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.
       Title           : The WebSocket protocol
       Author(s)       : I. Fette
       Filename        : draft-ietf-hybi-thewebsocketprotocol-01.txt
       Pages           : 57
       Date            : 2010-09-01
The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code.  The
security model used for this is the Origin-based security model
commonly used by Web browsers.  The protocol consists of an initial
handshake followed by basic message framing, layered over TCP.  The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).
This draft is meant to reflect changes in direction in the HyBi
working group.  There is not yet consensus on everything in this
draft.  Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft.  There have also been proposals
to change the handshake, so the handshake is also not in a final
form.
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
ftp://ftp.ietf.org/internet-drafts/
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Adam Barth
2010-09-01 23:22:59 UTC
Permalink
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft.   So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
You're of course free to implement whatever you like, but I don't
think the working group should sanction a versioning mechanism.
Post by Greg Wilkins
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
No one should be using this protocol for anything that can't receive a
breaking change.

Adam
John Tamplin
2010-09-01 23:36:33 UTC
Permalink
Post by Adam Barth
No one should be using this protocol for anything that can't receive a
breaking change.
Then how do we get real world data? Should everyone go modify Firefox
or Chrome themselves and run against only their servers to see how it
works?
--
John A. Tamplin
Software Engineer (GWT), Google
Adam Barth
2010-09-01 23:42:31 UTC
Permalink
Post by Adam Barth
No one should be using this protocol for anything that can't receive a
breaking change.
Then how do we get real world data?  Should everyone go modify Firefox
or Chrome themselves and run against only their servers to see how it
works?
If you're running an experiment, you should either:

1) Upgrade your clients and servers.
2) Break folks who are using old versions of the protocol.

(or ideally both)

If you can't either upgrade the old code or break the use of the old
code, you'll be calcified into using that version protocol.
Versioning isn't the solution. Avoiding calcification in the
solution.

Adam
Greg Wilkins
2010-09-01 23:54:02 UTC
Permalink
Post by Adam Barth
Post by Adam Barth
No one should be using this protocol for anything that can't receive a
breaking change.
Then how do we get real world data?  Should everyone go modify Firefox
or Chrome themselves and run against only their servers to see how it
works?
1) Upgrade your clients and servers.
2) Break folks who are using old versions of the protocol.
I'm not running the experiment.
I'm providing infrastructure to many different organizations and
developers that are running the experiments.
Some of them are using real users in their experiments.

There is no single point of control.

regards
John Tamplin
2010-09-01 23:56:03 UTC
Permalink
Post by Adam Barth
1) Upgrade your clients and servers.
2) Break folks who are using old versions of the protocol.
(or ideally both)
If you can't either upgrade the old code or break the use of the old
code, you'll be calcified into using that version protocol.
Versioning isn't the solution.  Avoiding calcification in the
solution.
If the clients are the general public, it is not feasible to have them
all upgrade at the same time. If you want to test a big service, you
can't afford to break people running older browsers. So your solution
boils down to, test only in controlled environments which means no
real world experience.
--
John A. Tamplin
Software Engineer (GWT), Google
Willy Tarreau
2010-09-02 05:25:09 UTC
Permalink
Post by Adam Barth
Post by Adam Barth
No one should be using this protocol for anything that can't receive a
breaking change.
Then how do we get real world data?  Should everyone go modify Firefox
or Chrome themselves and run against only their servers to see how it
works?
1) Upgrade your clients and servers.
2) Break folks who are using old versions of the protocol.
(or ideally both)
If you can't either upgrade the old code or break the use of the old
code, you'll be calcified into using that version protocol.
Versioning isn't the solution. Avoiding calcification in the
solution.
Adam, I disagree. We're running an experimentation and we want to get
measurable results. This experimentation consists in :
- getting server-side implementations
- getting client-side implementations
- see how they interoperate

If we state that every upgrade will break, noone will want to make the
first move. Browsers will wait for servers to be ready to support it,
and servers will wait for browsers to be ready. With a version, you can
change the servers first, and decide to break support for the old version
as soon as the browsers have support for the new protocol (eg: 1-2 weeks
later max). Also, when we get success or failure reports, it's important
to know if this is because clients can't communicate through the internet
or if it is because they're just not updated and don't talk the same
protocol anymore (in which case these samples must be dropped).

Willy
John Tamplin
2010-09-01 23:26:55 UTC
Permalink
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft.   So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to. Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.

I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
Greg Wilkins
2010-09-01 23:52:19 UTC
Permalink
Post by John Tamplin
The problem is that the framing of v75, v76, v00, and v01 are
incompatible.
Tell me about it!
Post by John Tamplin
 If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to.
We tried initially just to support v75 waiting for a sane v00, but
users were having problems. So we switched to v76 and got howls of
despair from v75 users.
So now Jetty supports both and we use the presence of Sec headers to
pick between v75 and v76 implementations.

I don't know of any wide deployments of the server, but I do know that
those experimenting with it have other constraints that prevent them
arbitrarily changing browsers at the same time they upgrade their
servers.
Post by John Tamplin
 Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.
Exactly - currently there is no change in the handshake for 01 draft,
so I will have to inspect the first packet and guess. Of course that
does not work if the server sends the first packet.
Post by John Tamplin
I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
thanks!
Ian Fette (イアンフェッティ)
2010-09-01 23:37:59 UTC
Permalink
+1. Adam, I share your concerns about versions on the web, but the reality
is there is no other way right now. Suggestions welcome as to alternatives.
Post by John Tamplin
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to. Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Gabriel Montenegro
2010-09-02 00:22:11 UTC
Permalink
+1. I’m all for a version as part of the protocol itself, but recognize that the discussion right now is limited to a versioning field only during draft stage.

From: hybi-***@ietf.org [mailto:hybi-***@ietf.org] On Behalf Of Ian Fette (????????)
Sent: Wednesday, September 01, 2010 4:38 PM
To: John Tamplin
Cc: hybi
Subject: Re: [hybi] I-D Action:draft-ietf-hybi-thewebsocketprotocol-01.txt


+1. Adam, I share your concerns about versions on the web, but the reality is there is no other way right now. Suggestions welcome as to alternatives.
Post by John Tamplin
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to. Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Takeshi Yoshino
2010-09-02 05:35:24 UTC
Permalink
+1

Aligning implementation is desirable but not realistic for real world
experiment. Welcoming older implementation users to experiment, we can get
more participant and more feedback on WebSocket. People who doesn't care
protocol detail but interested in WebSocket's basic features and API want
not to be blocked by protocol level change, I think.

We implemented some heuristics that look for some -76 (== -00) specific
header in handshake request. Now implementors have no other choice than
using frame inspection to tell -00 and -01 apart as Greg mentioned. At least
in draft stage where we'll publish number of drafts with drastic change as
milestones every time we get consensus on some topic, we should have some
way to tell the version which the client is using explicitly rather than
requiring implementor to adopt such heuristics.

Takeshi
+1. I’m all for a version as part of the protocol itself, but recognize
that the discussion right now is limited to a versioning field only during
draft stage.
*Ian Fette (????????)
*Sent:* Wednesday, September 01, 2010 4:38 PM
*To:* John Tamplin
*Cc:* hybi
*Subject:* Re: [hybi] I-D
Action:draft-ietf-hybi-thewebsocketprotocol-01.txt
+1. Adam, I share your concerns about versions on the web, but the reality
is there is no other way right now. Suggestions welcome as to alternatives.
Post by John Tamplin
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to. Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Olli Pettay
2010-09-02 14:18:27 UTC
Permalink
Post by Ian Fette (イアンフェッティ)
+1. Adam, I share your concerns about versions on the web, but the
reality is there is no other way right now. Suggestions welcome as to
alternatives.
Couldn't web applications use different subprotocols for different
versions of the web socket protocol.
That wouldn't be the nicest solution, but could work
well enough while we're trying to get some stable protocol.


-Olli


(I haven't read properly the current draft, but the framing looks too
complicated. I'll send comments once I've reviewed the draft properly.)
Post by Ian Fette (イアンフェッティ)
Post by John Tamplin
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to. Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Ian Fette (イアンフェッティ)
2010-09-02 16:53:30 UTC
Permalink
Post by Olli Pettay
Post by Ian Fette (イアンフェッティ)
+1. Adam, I share your concerns about versions on the web, but the
reality is there is no other way right now. Suggestions welcome as to
alternatives.
Couldn't web applications use different subprotocols for different
versions of the web socket protocol.
That wouldn't be the nicest solution, but could work
well enough while we're trying to get some stable protocol.
-Olli
(I haven't read properly the current draft, but the framing looks too
complicated. I'll send comments once I've reviewed the draft properly.)
Subprotocols are intended to be an application level indication, and IMO
should affect base framing. This seems to be conflating issues.

Also, in addition to reviewing the draft, I would ask that you try to review
the comments that led up the draft being what it is, otherwise we are likely
to just revisit old discussions.
Post by Olli Pettay
Post by Ian Fette (イアンフェッティ)
Post by John Tamplin
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I know some
think it is an anti pattern.
I just want to be able to implement the next draft without breaking
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by implementers as
non-standard convenience at this time. The lack of a draft version can
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal with the
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that interoperates with
browsers implementing these drafts (I know at least v75 and v76 are in
the wild), then you have to rely on heuristics to figure out which one
you are talking to. Even if there isn't a version in the 1.0 spec,
any attempt to get real-world data while the spec is changing means
you either have a version number or you heuristically determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to implementations of
this draft, and that header will only be present before the spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
John Tamplin
2010-09-02 17:22:10 UTC
Permalink
Post by Ian Fette (イアンフェッティ)
Subprotocols are intended to be an application level indication, and IMO
should affect base framing. This seems to be conflating issues.
Did you mean "should not affect"? If not, I think I am
misunderstanding your point.
--
John A. Tamplin
Software Engineer (GWT), Google
Ian Fette (イアンフェッティ)
2010-09-02 17:27:59 UTC
Permalink
Yes, sorry. Should NOT affect.
Post by John Tamplin
Post by Ian Fette (イアンフェッティ)
Subprotocols are intended to be an application level indication, and IMO
should affect base framing. This seems to be conflating issues.
Did you mean "should not affect"? If not, I think I am
misunderstanding your point.
--
John A. Tamplin
Software Engineer (GWT), Google
Olli Pettay
2010-09-02 17:31:09 UTC
Permalink
Post by Ian Fette (イアンフェッティ)
+1. Adam, I share your concerns about versions on the web, but the
reality is there is no other way right now. Suggestions welcome as to
alternatives.
Couldn't web applications use different subprotocols for different
versions of the web socket protocol.
That wouldn't be the nicest solution, but could work
well enough while we're trying to get some stable protocol.
-Olli
(I haven't read properly the current draft, but the framing looks too
complicated. I'll send comments once I've reviewed the draft properly.)
Subprotocols are intended to be an application level indication, and IMO
should affect base framing. This seems to be conflating issues.
I understand what subprotocols are meant for, but while we're trying to
stabilize the protocol, subprotocols could be used for versioning too.
Post by Ian Fette (イアンフェッティ)
Also, in addition to reviewing the draft, I would ask that you try to
review the comments that led up the draft being what it is, otherwise we
are likely to just revisit old discussions.
Yeah, I know that there is a risk that I will complain about things what
others have complained too, and what others like about.
In general it is not clear to me why we have to change framing so much.
The old hixie-style framing allow all sorts for extensions. There are
plenty of bits left for different kinds of frame types.

(And for things like multiplexing shared workers some
people have just said that it doesn't work without even trying.)


-Olli
Post by Ian Fette (イアンフェッティ)
On Wed, Sep 1, 2010 at 7:18 PM, Greg Wilkins
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I
know some
Post by Greg Wilkins
think it is an anti pattern.
I just want to be able to implement the next draft without
breaking
Post by Greg Wilkins
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by
implementers as
Post by Greg Wilkins
non-standard convenience at this time. The lack of a draft
version can
Post by Greg Wilkins
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal
with the
Post by Greg Wilkins
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that
interoperates with
browsers implementing these drafts (I know at least v75 and
v76 are in
the wild), then you have to rely on heuristics to figure out
which one
you are talking to. Even if there isn't a version in the 1.0
spec,
any attempt to get real-world data while the spec is changing
means
you either have a version number or you heuristically
determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to
implementations of
this draft, and that header will only be present before the
spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
John Tamplin
2010-09-02 17:40:49 UTC
Permalink
Post by Olli Pettay
I understand what subprotocols are meant for, but while we're trying to
stabilize the protocol, subprotocols could be used for versioning too.
So what if you want to experiment with an application subprotocol? If
the framing is altered based on the subprotocol, then the application
can't use subprotocols itself.
Post by Olli Pettay
The old hixie-style framing allow all sorts for extensions. There are
plenty of bits left for different kinds of frame types.
Well, the first problem is it doesn't support fragmentation, which is
a problem given that has been accepted as a requirement. Second, it
uses two different types of framing for text and binary data, and many
negative opinions of having two types of framing were expressed here.
Finally, the only possible method of extension would be to define
nested opcodes, and there have been many discussions here about the
virtue of that choice relative to others. The point of the current
framing is that it answers most of the objections that have been
raised while retaining the flexibility to define the various extension
mechanisms that have been proposed on top of it. That allows us to
agree on the base framing, in such a way that those who don't ever
want to handle extensions can just refuse to negotiate them and ignore
them, and then those who do care can debate the merits of different
extension mechanisms.
Post by Olli Pettay
(And for things like multiplexing shared workers some
 people have just said that it doesn't work without even trying.)
How can writing shared workers ever handle the case of mutliplexing
traffic in different apps? Or aggregating traffic across an expensive
wireless link, which will then get split on the other side? Even if
shared workers can be used to solve some of the problems multiplexing
solves, it can't solve all of them and IMHO is more complex rather
than just implementing it once in the framing layer.
--
John A. Tamplin
Software Engineer (GWT), Google
Ian Fette (イアンフェッティ)
2010-09-02 17:34:01 UTC
Permalink
Post by Olli Pettay
Post by Ian Fette (イアンフェッティ)
+1. Adam, I share your concerns about versions on the web, but the
reality is there is no other way right now. Suggestions welcome as to
alternatives.
Couldn't web applications use different subprotocols for different
versions of the web socket protocol.
That wouldn't be the nicest solution, but could work
well enough while we're trying to get some stable protocol.
-Olli
(I haven't read properly the current draft, but the framing looks too
complicated. I'll send comments once I've reviewed the draft properly.)
Subprotocols are intended to be an application level indication, and IMO
should affect base framing. This seems to be conflating issues.
I understand what subprotocols are meant for, but while we're trying to
stabilize the protocol, subprotocols could be used for versioning too.
Post by Ian Fette (イアンフェッティ)
Also, in addition to reviewing the draft, I would ask that you try to
review the comments that led up the draft being what it is, otherwise we
are likely to just revisit old discussions.
Yeah, I know that there is a risk that I will complain about things what
others have complained too, and what others like about.
In general it is not clear to me why we have to change framing so much.
The old hixie-style framing allow all sorts for extensions. There are
plenty of bits left for different kinds of frame types.
I really suggest you read the archive before trying to re-hash debates that
have gone on for hundreds of email messages
Post by Olli Pettay
(And for things like multiplexing shared workers some
people have just said that it doesn't work without even trying.)
-Olli
Post by Ian Fette (イアンフェッティ)
On Wed, Sep 1, 2010 at 7:18 PM, Greg Wilkins
Post by Greg Wilkins
my proposal was not to add versioning to the protocol - as I
know some
Post by Greg Wilkins
think it is an anti pattern.
I just want to be able to implement the next draft without
breaking
Post by Greg Wilkins
all my users that have been working on the last draft. So my
proposal is only for a draft version to be added by
implementers as
Post by Greg Wilkins
non-standard convenience at this time. The lack of a draft
version can
Post by Greg Wilkins
be taken to mean the final stable version.
How else do you expect servers to track the drafts and deal
with the
Post by Greg Wilkins
various browser implementations out there?
The problem is that the framing of v75, v76, v00, and v01 are
incompatible. If you want to write a server that
interoperates with
browsers implementing these drafts (I know at least v75 and
v76 are in
the wild), then you have to rely on heuristics to figure out
which one
you are talking to. Even if there isn't a version in the 1.0
spec,
any attempt to get real-world data while the spec is changing
means
you either have a version number or you heuristically
determine one --
the latter is obviously more prone to error.
I would suggest adding Sec-WebSocket-Draft: 01 to
implementations of
this draft, and that header will only be present before the
spec is
finalized.
--
John A. Tamplin
Software Engineer (GWT), Google
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Roy T. Fielding
2010-09-02 00:00:25 UTC
Permalink
Post by Adam Barth
Please don't add versioning to the protocol. Versioning is a
anti-pattern for the web.
FTR, that statement is false. There is a very small group of
inexperienced developers that think client-side rendering of
uncontrolled content somehow defines universal principles for
all software engineering. The rest of us use whatever techniques
we can to reduce entropy. Network protocols, in particular,
should always have a version number to distinguish them from
other similar protocols that might appear in the same context.

Both HTTP and Waka have their version in the first line of
every message, and I can assure you that the Web would not have
survived without it.

....Roy
Willy Tarreau
2010-09-02 05:19:29 UTC
Permalink
Post by Roy T. Fielding
Post by Adam Barth
Please don't add versioning to the protocol. Versioning is a
anti-pattern for the web.
FTR, that statement is false. There is a very small group of
inexperienced developers that think client-side rendering of
uncontrolled content somehow defines universal principles for
all software engineering. The rest of us use whatever techniques
we can to reduce entropy. Network protocols, in particular,
should always have a version number to distinguish them from
other similar protocols that might appear in the same context.
Both HTTP and Waka have their version in the first line of
every message, and I can assure you that the Web would not have
survived without it.
I would add that protocol naming and versioning is also an efficient
way to protect against cross-protocol attacks, which is regularly
brought to the table here... If SMTP required a version on its first
line instead of ignoring all unparsable lines, we would not be looking
for ways to prevent a web client from sending emails over SMTP through
POST requests.

Willy
Hector Santos
2010-09-02 06:07:29 UTC
Permalink
Post by Willy Tarreau
I would add that protocol naming and versioning is also an efficient
way to protect against cross-protocol attacks, which is regularly
brought to the table here... If SMTP required a version on its first
line instead of ignoring all unparsable lines, we would not be looking
for ways to prevent a web client from sending emails over SMTP through
POST requests.
Speaking with my SMTP developer hat on, I don't follow Willy how this
is a problem.

Many MTA already trap and count unknown commands. If the first command
is POST, you can reject the client immediately or prevent it from
going into the next state until it issues QUIT or drop the line for
excessive out of state commands.
--
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com
Willy Tarreau
2010-09-02 06:16:13 UTC
Permalink
Post by Hector Santos
Post by Willy Tarreau
I would add that protocol naming and versioning is also an efficient
way to protect against cross-protocol attacks, which is regularly
brought to the table here... If SMTP required a version on its first
line instead of ignoring all unparsable lines, we would not be looking
for ways to prevent a web client from sending emails over SMTP through
POST requests.
Speaking with my SMTP developer hat on, I don't follow Willy how this
is a problem.
Many MTA already trap and count unknown commands. If the first command
is POST, you can reject the client immediately or prevent it from
going into the next state until it issues QUIT or drop the line for
excessive out of state commands.
Well, this is precisely a consequence of the problem I talked about. POST
has never been part of the SMTP protocol, and it's because the SMTP
protocol is prone to such attacks that these checks have been added in
implementations. Maybe some updates to the protocol have since been emitted
to suggest this, but I always had the feeling that these protections were
just good implementers' choices in face of a real threat.

Regards,
Willy
Hector Santos
2010-09-02 07:03:53 UTC
Permalink
Post by Willy Tarreau
Well, this is precisely a consequence of the problem I talked about. POST
has never been part of the SMTP protocol, and it's because the SMTP
protocol is prone to such attacks that these checks have been added in
implementations. Maybe some updates to the protocol have since been emitted
to suggest this, but I always had the feeling that these protections were
just good implementers' choices in face of a real threat.
Since this behavior is no different that a client batching its
commands with disregard for server responses (not intentional
pipelining), this has long been mitigated in good MTAs.

The same will be true with websockets - the server always need to be
control or it will be open to all sorts of issues.

The main thing here is that we are talking about hosting servers and
whatever protocol is put together, we need to make sure that we don't
make it hard to resolve the known formulas for threat entry points.

For example, one way to authenticate the WS client is to use IP such
as it done with POP3-B4-SMTP methods where a POP3 host records an IP
for another SMTP host to open a time window for SMTP IP-based allow
relay.

For Websockets, it may work pretty well based on the fact most
applications will mostly start with a browser client and ws client
from the same IP. The same idea can be applied here:

C1: HTTP request (possibly start HTTP or Cookie AUTH login)
S2: HTTP page is served. IP recorded (optional TTL)
C3: WS Client begin WS open and handshake
S4: WS Server authenticates recorded IP, continues with handshake

S2 MAY opens a limited time window (TTL) for the 2nd session to
appear. It helps resolve the unsolicited WS client request issue for
servers that expect WS clients to begin with an HTTP server prepared
client.

Lets keep in mind something here folks - the mindset has changed to
even allowing a TCP socket connection to be part of the browser (and
many of the HTML5 aspect with LOCAL I/O).

Its taken a long time to even view this as "acceptable" but that
should not be a taken lightly or become a reason to relax with all the
security concerns. Its too easy to fall trap in this open generation
of meshed connections.

If we fail with that, WEBSOCKETS will be turned off which is something
a QA conscious browser will probably soon need to add to its user
setup interface in the same way Javacript can be still be turned off.
If not done right, expect the NoWebSocket plug-ins to appear.
--
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com
Willy Tarreau
2010-09-02 20:22:03 UTC
Permalink
Post by Hector Santos
For example, one way to authenticate the WS client is to use IP such
as it done with POP3-B4-SMTP methods where a POP3 host records an IP
for another SMTP host to open a time window for SMTP IP-based allow
relay.
IP is often used in POP+SMTP auth because the purpose is to grant a
service to some users during a short period of time, with very little
risk if the match fails.

In web environment, the IP is useless for anything other than logging.
Depending on the environments and web sites, I have observed up to as
much as 5% of the users showing a different IP address at least once
in a session. The most common cases are on sites used from work, because
many companies have large outgoing proxy farms. And for the same reason
you can't rely on that at all because when 1000 users come to your site
via the same IP address, it's a real issue.

And I predict that with IPv6 it will be even worse, because right now
this issue is hidden by the fact that many internet access points are
natted due to IP addresses shortage. But right now it's already common
to see various types of requests pass through different outgoing proxies.
And here, we will certainly have some users showing one (or a few) IP
address(es) for the main HTTP traffic, and another set for the WS traffic.
I'm not even talking about mobile users !

And that's what is really nice with HTTP : it allows such architectures
to reliably work without having to wonder about IP addresses!

Regars,
Willy
Dave Cridland
2010-09-02 20:35:37 UTC
Permalink
Post by Hector Santos
For example, one way to authenticate the WS client is to use IP
such as it done with POP3-B4-SMTP methods where a POP3 host records
an IP for another SMTP host to open a time window for SMTP IP-based
allow relay.
That's been long discredited in email circles, and is generally not
supported, in favour of SMTP AUTH (and preferably a dedicated
submission service on 587 with SASL and TLS).

The reason it's been discredited is actually because it's in a
general class of error referred to as "Time of check to time of use".

Dave.
--
Dave Cridland - mailto:***@cridland.net - xmpp:***@dave.cridland.net
- acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
- http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Hector Santos
2010-09-02 21:30:04 UTC
Permalink
Post by Dave Cridland
Post by Hector Santos
For example, one way to authenticate the WS client is to use IP such
as it done with POP3-B4-SMTP methods where a POP3 host records an IP
for another SMTP host to open a time window for SMTP IP-based allow
relay.
That's been long discredited in email circles, and is generally not
supported, in favour of SMTP AUTH (and preferably a dedicated submission
service on 587 with SASL and TLS).
The reason it's been discredited is actually because it's in a general
class of error referred to as "Time of check to time of use".
Dave.
Sure, hut I've long been within the email circles and I do not ever
recall any outright conclusion.

But I do agree with your last statement. Nonetheless, it is still an
option in many integrated mail software and whether people have turned
it off - don't be surprise if many have not because one of the primary
reasons for it was that it reduced technical support cost - lesser
need to help/setup the layman users.

ESMTP AUTH is not a required standard and RFC 4409 (only 4 years old)
was not widely supported. But when the shift did begin with ISPs
requiring ESMTP AUTH for their users, some immediately fell back to
allowing IP authorization because of the huge tech support burden it
created. I specifically recall this exact situation with my home
Bellsouth account when they issued a deadline notification to all
users to begin switching to software using ESMTP AUTH. After the
deadline, I would say it was maybe less than a week when a 2nd
notification indicating it was no longer a requirement and still isn't
today - although I always had it personally setup for ESMTP AUTH.

Now of course, the ISP doesn't need to use POP4-B4-SMTP because the
ISP user is already "IP authenticated" on their network and SMTP Relay
is allowed.

Another subtle point regarding the SUBMISSION protocol that has it
began to show a problem for the growing home or soho market with nats
and 2nd, 3rd MUAs on LAN or wireless machines.

Some SMTP clients residing on a LAN or wireless machine sending mail
will use a EHLO [IP-LITERAL] where the ip is private. This will fail
RFC 4409 tight EHLO validation requirement for implementations that
will check the ip literal against the connecting IP which would be the
NAT public IP.

I reported this issue to the Thunderbird people which now has a config
option to set the HELLO string used. Klensin and Gellens were also
informed of this growing issue. My recommendation was to add
semantics to suggest skipping any EHLO IP-literal verification since
AUTH was already a session requirement, not an option under PORT 587
connections.

In any case, the point was that IP authentication *can* be a valid
server side consideration for secondary web sockets connections.

When the HTTP session authenticates the user with HTTP/COOKIE auth,
the binding to the IP is set and this can be used for any pending
web-socket clients on the same IP. Whether a TTL is required, I don't
know if its necessary or not since IMV, there is a greater
predictability and timeline of events with the HTTP session and WS
session than it was with POP3 and SMTP.

Thanks
--
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com
Dave Cridland
2010-09-03 08:22:10 UTC
Permalink
I'll skip the email discussion as it's not relevant here.
Post by Hector Santos
In any case, the point was that IP authentication *can* be a valid
server side consideration for secondary web sockets connections.
And my point is that it is not.
Post by Hector Santos
When the HTTP session authenticates the user with HTTP/COOKIE auth,
the binding to the IP is set and this can be used for any pending
web-socket clients on the same IP. Whether a TTL is required, I
don't know if its necessary or not since IMV, there is a greater
predictability and timeline of events with the HTTP session and WS
session than it was with POP3 and SMTP.
Not especially. An HTTP request may be passed through several
proxies, a WebSocket request is less likely to.

It's important to consider that if the HTTP service provides a
cookie, then that should be sufficient if it would be sufficient in
HTTP. I would argue that the whole question of user authentication on
"the web" is in dire need of a re-examination, but again, this really
isn't the forum to do it in.

One point you made does apply here too - an IP address, if within the
AS or local network, is sufficient to authenticate as an otherwise
anonymous local user. This is the case used to allow SMTP relaying to
ISP customers. I don't think this will be an option for the vast
majrotiy of WebSocket services.

Dave.
--
Dave Cridland - mailto:***@cridland.net - xmpp:***@dave.cridland.net
- acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
- http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Hector Santos
2010-09-03 12:16:05 UTC
Permalink
Good morning,
Post by Dave Cridland
Post by Hector Santos
In any case, the point was that IP authentication *can* be a valid
server side consideration for secondary web sockets connections.
And my point is that it is not.
Come on David, surely you are more open minded than that. :)
Post by Dave Cridland
An HTTP request may be passed through several proxies,
a WebSocket request is less likely to.
Sure.
Post by Dave Cridland
One point you made does apply here too - an IP address, if within the AS
or local network, is sufficient to authenticate as an otherwise
anonymous local user. This is the case used to allow SMTP relaying to
ISP customers. I don't think this will be an option for the vast
majrotiy of WebSocket services.
Well, David, you already made a decision on what the majority services
will be and to (mildly) suggest there is no utility for the minority
cases. :)

We long had a dual/multi client session framework since 1996 so my
perspective does have a basis. WebSocket offers an alternative,
perhaps simplified "client/wiring" framework. For us, its more about
single sourcing the client interface devices - again. It may also show
to be "not ready" for prime time.

Nonetheless, while off hand the session management will naturally
carry over which is not IP associated, IP may|could be part of it. 3rd
party applets have leveraged "IP" in different ways, i.e. "call back
verifiers" ideas. Overall, the reasons have been more strategic in
some aspect, but its all about different levels of session management
considerations.

WebSocket simply highlights these design considerations more - at
least for us.

IMO, the majority market place, at least in the short term, will be
more transforming existing applications including private, intranet,
etc. The longer term may be more public service hosting, a wider mesh
network or some combo of decentralization/centralization server ideas.

Thanks for your input.
--
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com
Daniel Stenberg
2010-09-02 18:25:53 UTC
Permalink
Please don't add versioning to the protocol. Versioning is a anti-pattern
for the web.
We've seen this mentioned before on this list but without a lot of
clarifications and I'm curious:

What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?

Or isn't it so that it truly is an anti-pattern for the *web* (with the
emphasis added on the web word) and we're not really doing web here, we're
discussing a transfer protocol.

HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
--
/ daniel.haxx.se
Tim Bray
2010-09-02 18:42:16 UTC
Permalink
Versioning requres you to make all sorts of REALLY HARD choices, and runs
the risk of imposing really onerous costs on implementors. Thus, you need
to be really convinced that there are really significant benefits before you
embark on this difficult and risky course of action. -Tim
Please don't add versioning to the protocol. Versioning is a anti-pattern
Post by Adam Barth
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Or isn't it so that it truly is an anti-pattern for the *web* (with the
emphasis added on the web word) and we're not really doing web here, we're
discussing a transfer protocol.
HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
--
/ daniel.haxx.se
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
John Tamplin
2010-09-02 18:50:00 UTC
Permalink
Post by Tim Bray
Versioning requres you to make all sorts of REALLY HARD choices, and runs
the risk of imposing really onerous costs on implementors.  Thus, you need
to be really convinced that there are really significant benefits before you
embark on this difficult and risky course of action.  -Tim
Not versioning requires an implementor to make the same really hard
choices, plus the added complexity of trying to figure out which
version of the protocol the other version is speaking.

Once we get to the final version of the spec, I am fine with leaving
out a formal version identifier, and then when we get to the point of
wanting to add some non-backwards-compatible feature we can have the
discussion about adding versioning then.

However, I remain totally convinced that we have to have some version
identifier in the draft protocol, so we can tell different versions of
the draft apart. We already have mutually incompatible
implementations in the wild, and having to come up with heuristics to
detect each new version that might be out there is going to get more
and more problematic.
--
John A. Tamplin
Software Engineer (GWT), Google
Dave Cridland
2010-09-02 18:52:30 UTC
Permalink
Post by Tim Bray
Versioning requres you to make all sorts of REALLY HARD choices, and runs
the risk of imposing really onerous costs on implementors. Thus, you need
to be really convinced that there are really significant benefits before you
embark on this difficult and risky course of action. -Tim
To put it another way, versioning is for flag days, whereas feature
negotiation is for, well, negotiating features.

An example of a protocol with both is XMPP. It has a version, which
is always 1.0. It also has stream features for peer-to-peer feature
negotiation.

Jabber, its predecessor, had no version - but it also lacked the
feature negotiation. Essentially, XMPP's version indicates to the
receiver that it's okay to announce what features you offer.

So in our release protocol, I'd say we SHOULD NOT have a version, but
we MAY have a static, never to be changed, version.

For now, though, when we're quite likely to want to change our
feature negotiation mechanisms, we probably MUST have a version.

The key item is not that versions are bad, but that *changing* them
is.

Dave.
--
Dave Cridland - mailto:***@cridland.net - xmpp:***@dave.cridland.net
- acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
- http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Hector Santos
2010-09-02 18:52:48 UTC
Permalink
+1
Post by Tim Bray
Versioning requres you to make all sorts of REALLY HARD choices, and runs
the risk of imposing really onerous costs on implementors. Thus, you need
to be really convinced that there are really significant benefits before you
embark on this difficult and risky course of action. -Tim
Please don't add versioning to the protocol. Versioning is a anti-pattern
Post by Adam Barth
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Or isn't it so that it truly is an anti-pattern for the *web* (with the
emphasis added on the web word) and we're not really doing web here, we're
discussing a transfer protocol.
HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
--
/ daniel.haxx.se
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
------------------------------------------------------------------------
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
--
Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com
David Orchard
2010-09-02 21:42:07 UTC
Permalink
I think the expression that versioning is an anti-pattern for the web
is in the context of human authored HTML documents, where people
notoriously view source and cut 'n paste that results in the wrong
DTDs, etc. I don't think that a large number of people will be
cutting and pasting text containing versioning information to hybi
protocol messages, so that advice doesn't apply to hybi imo.

However, protocols that have many optional features might not do well
with single string version identifiers. HTTP's use of version numbers
asserting the capabilities of the server rather than the features used
by the message has worked pretty well at a coarse grained level but
the different feature negotiations have had varying degrees of
success.

A lack of versioning, and I don't mean just using identifiers, is
pretty risky. Interesting question as to whether HTTP version
identifiers have helped or not, and I tend to think they have helped.

Cheers,
Dave
Post by Daniel Stenberg
Please don't add versioning to the protocol.  Versioning is a anti-pattern
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Or isn't it so that it truly is an anti-pattern for the *web* (with the
emphasis added on the web word) and we're not really doing web here, we're
discussing a transfer protocol.
HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
--
 / daniel.haxx.se
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Greg Wilkins
2010-09-02 23:34:08 UTC
Permalink
Post by David Orchard
A lack of versioning, and I don't mean just using identifiers, is
pretty risky.  Interesting question as to whether HTTP version
identifiers have helped or not, and I tend to think they have helped.
HTTP servers deployed in the wild stills sees a significant proportion
of HTTP/1.0 traffic, but mostly HTTP/1.1. There is still even a few
HTTP/0.9 (== no version) requests, but those are mostly me using
telnet to get a REST resource (because I can't remember the args to
curl).

The difference in protocol behaviour between these 3 versions is
significant and it would be neigh impossible for a server to service
the variety of clients deployed without a version. Supporting 3
versions of the protocol does complicate the server, but there is no
other choice as you cannot force web clients to upgrade (nor me to
remember the args to curl)

The only time I've had real problems with different dialects of HTTP
was during the change from rfc2068 to rfc2616, both of which said they
were HTTP/1.1 (why not HTTP/1.2? So maybe now, years later it
is great they we don't have to maintain 2068 support, but that was
only lucky as there was not wide deployment of rfc2068 clients.

cheers
James Graham
2010-09-03 06:47:19 UTC
Permalink
Post by Greg Wilkins
Post by David Orchard
A lack of versioning, and I don't mean just using identifiers, is
pretty risky.  Interesting question as to whether HTTP version
identifiers have helped or not, and I tend to think they have helped.
HTTP servers deployed in the wild stills sees a significant proportion
of HTTP/1.0 traffic, but mostly HTTP/1.1. There is still even a few
HTTP/0.9 (== no version) requests, but those are mostly me using
telnet to get a REST resource (because I can't remember the args to
curl).
The difference in protocol behaviour between these 3 versions is
significant and it would be neigh impossible for a server to service
the variety of clients deployed without a version. Supporting 3
versions of the protocol does complicate the server, but there is no
other choice as you cannot force web clients to upgrade (nor me to
remember the args to curl)
So, to makw sure I understand you correctly, you are proposing that future
clients and servers support *all* the versions of the draft that have ever
seen non-trivial deployment (possibly including hixie-75, likely including
hixie-76, and presumably including more future versions that happen to
line up with significant client releases)?
John Tamplin
2010-09-03 06:56:50 UTC
Permalink
Post by James Graham
So, to makw sure I understand you correctly, you are proposing that future
clients and servers support *all* the versions of the draft that have ever
seen non-trivial deployment (possibly including hixie-75, likely including
hixie-76, and presumably including more future versions that happen to line
up with significant client releases)?
Speaking for myself, I think we are already likely in a position of
supporting v76 for some extended period of time. Chrome tends to get
updated pretty quickly, though certain platforms may lag getting a new
release into stable. Unless the final spec makes FF4.0 (and I don't know
how much lead time it would take to get it in), then it is likely to exist
there for some non-trivial period of time. I suspect at least one more
intermediate version will see some deployment so we can get real-world data.

The point is, these clients will still exist whether there is a version in
new drafts or not. Even the question of whether you care about making them
work is orthogonal to the question of whether to put a version on them or
not. The only thing having a version does for you is to know which version
of the protocol the other end is speaking, rather than having to guess from
the information it sent. Unless you just send something and hope it works,
you still ultimately decide "the remote endpoint is speaking v76" in either
case -- what you do with it after that is up to you -- it could be that you
drop the connection as incompatible, or it could be you speak v76 to it.
--
John A. Tamplin
Software Engineer (GWT), Google
Julian Reschke
2010-09-03 07:31:44 UTC
Permalink
Post by John Tamplin
...
The point is, these clients will still exist whether there is a version
in new drafts or not. Even the question of whether you care about
making them work is orthogonal to the question of whether to put a
version on them or not. The only thing having a version does for you is
to know which version of the protocol the other end is speaking, rather
than having to guess from the information it sent. Unless you just send
...
Which might be impossible to do.
Post by John Tamplin
...
something and hope it works, you still ultimately decide "the remote
endpoint is speaking v76" in either case -- what you do with it after
that is up to you -- it could be that you drop the connection as
incompatible, or it could be you speak v76 to it.
...
Yes. Sounds like a feature to me, at least before we get to 1.0.

Another benefit is that it might be easier to do comparisons between two
draft protocols.

Best regards, Julian
Patrick McManus
2010-09-03 12:34:45 UTC
Permalink
Unless the final spec makes FF4.0 (and I don't know how much lead time
it would take to get it in), then it is likely to exist there for some
non-trivial period of time.
For those that aren't aware, I'm working with the Mozilla Websockets
team. And with that hat on I can say FF is unlikely to deploy -76 in FF
4 - except perhaps under a clearly for-experiment-only namespace which
is subject to breakage at any time.

That being said, I'd love to get the specification to a state where we
could ship something based on draft-ietf-* with some confidence that at
least the basics were stable. We've got some work to do.
John Tamplin
2010-09-03 15:11:57 UTC
Permalink
Post by Patrick McManus
For those that aren't aware, I'm working with the Mozilla Websockets
team. And with that hat on I can say FF is unlikely to deploy -76 in FF
4 - except perhaps under a clearly for-experiment-only namespace which
is subject to breakage at any time.
That being said, I'd love to get the specification to a state where we
could ship something based on draft-ietf-* with some confidence that at
least the basics were stable. We've got some work to do.
So FF40 final will not ship WebSockets support under the final name unless
the spec is finalized before then? I thought 4.0b4 was running v76 and
interoperable with Chrome from the client side API?

Since most of the comments on the framing in draft 01 have been about the
wording rather than the technical details in general, I think we are pretty
close to having the framing done. However, I expect there to be significant
changes in the handshake.
--
John A. Tamplin
Software Engineer (GWT), Google
Scott Ferguson
2010-09-03 15:41:46 UTC
Permalink
Post by Patrick McManus
That being said, I'd love to get the specification to a state where we
could ship something based on draft-ietf-* with some confidence that at
least the basics were stable. We've got some work to do.
Since most of the comments on the framing in draft 01 have been about
the wording rather than the technical details in general, I think we
are pretty close to having the framing done. However, I expect there
to be significant changes in the handshake.
I do have a minor, aesthetic suggestion for the framing to reorder the
bits so the reserved bits are contiguous.

* more(1)
* length(7)
* res(4)
* opcode(4)

It's not a substantive or important proposal, of course, but looks a bit
cleaner than splitting the reserved bits.

-- Scott
Post by Patrick McManus
--
John A. Tamplin
Software Engineer (GWT), Google
------------------------------------------------------------------------
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
John Tamplin
2010-09-03 15:44:50 UTC
Permalink
I do have a minor, aesthetic suggestion for the framing to reorder the bits
so the reserved bits are contiguous.
* more(1)
* length(7)
* res(4)
* opcode(4)
It's not a substantive or important proposal, of course, but looks a bit
cleaner than splitting the reserved bits.
I dislike having the bits of the length separated from each other (since the
16/63-bit case it will come after the opcode). In particular, it makes it
hard to write ABNF which is clear.
--
John A. Tamplin
Software Engineer (GWT), Google
Scott Ferguson
2010-09-03 16:15:50 UTC
Permalink
Post by Scott Ferguson
I do have a minor, aesthetic suggestion for the framing to reorder
the bits so the reserved bits are contiguous.
* more(1)
* length(7)
* res(4)
* opcode(4)
It's not a substantive or important proposal, of course, but looks
a bit cleaner than splitting the reserved bits.
I dislike having the bits of the length separated from each other
(since the 16/63-bit case it will come after the opcode). In
particular, it makes it hard to write ABNF which is clear.
There's a value in putting more and length together since they're
related. That would help debugging because a fragment would be easily
recognizable as %xFE or %xFF instead of varying based on the reserved
bits as the current frame encoding requires. For short packets, having
the high bit clear will make the length easier to scan (assuming it's a
non-fragment) than scanning would be if the high bit varies for
unrelated extension reasons.

* res(4)
* opcode(4)
* more(1)
* length(7)

It's a matter of taste, really. Having the reserved split in two also
makes the frame header look more complicated because it implies that
RES1 and RES2 are reserved for different purposes rather than just being
generic reserved bits.

But it really doesn't matter much, so I'll just leave the point out
there and not argue for it further.

-- Scott
Post by Scott Ferguson
--
John A. Tamplin
Software Engineer (GWT), Google
John Tamplin
2010-09-03 16:26:10 UTC
Permalink
It's a matter of taste, really. Having the reserved split in two also makes
the frame header look more complicated because it implies that RES1 and RES2
are reserved for different purposes rather than just being generic reserved
bits.
Actually in the draft they are named individually (I thought it would be
easier to define future extensions rather than saying the 2nd-highest bit of
reserved1 etc), and it seems highly likely that we will want at least one
stand-alone bit for a flag in some extension.

I also don't care much, but I prefer the current formulation.
--
John A. Tamplin
Software Engineer (GWT), Google
Greg Wilkins
2010-09-06 13:03:28 UTC
Permalink
Post by James Graham
Post by Greg Wilkins
The difference in protocol behaviour between these 3 versions is
significant and it would be neigh impossible for a server to service
the variety of clients deployed without a version. Supporting 3
versions of the protocol does complicate the server, but there is no
other choice as you cannot force web clients to upgrade (nor me to
remember the args to curl)
So, to makw sure I understand you correctly, you are proposing that future
clients and servers support *all* the versions of the draft that have ever
seen non-trivial deployment (possibly including hixie-75, likely including
hixie-76, and presumably including more future versions that happen to line
up with significant client releases)?
James,

all I'm proposing at the moment is that we identify draft versions
while we are in draft.

However, in general, I expect that servers will need to support
multiple versions of the specification, even if they are not actually
identified with version numbers (or user-agents or heuristics or
whatever ). That is just the reality of the situation - successful
servers must support a wide range of deployed clients and it takes
many years for clients with significant deployment to fade away
(although it is not as slow as it once was).

I would hope that we will never have the need for clients to have to
handle multiple versions.

cheers
Adam Barth
2010-09-04 02:06:32 UTC
Permalink
Post by Daniel Stenberg
Please don't add versioning to the protocol.  Versioning is a anti-pattern
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Versioning has been a big pain point for TLS. Here's a presentation
that outlines some recent implementation experience:

http://www.ietf.org/proceedings/65/slides/tls-6.pdf

I encourage you to read through the presentation (it's pretty short).
Post by Daniel Stenberg
Or isn't it so that it truly is an anti-pattern for the *web* (with the
emphasis added on the web word) and we're not really doing web here, we're
discussing a transfer protocol.
It's a continual annoyance in dealing with the IETF to have to deal
with this "browsers don't matter" attitude. In he case of this
working group, we're working on a protocol expressly for use by
browsers, which would seem to indicate that browser implementation
experience is relevant.
Post by Daniel Stenberg
HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
The version number in HTTP hasn't been change in a very long time. I
suspect we'd encounter tremendous difficulty if we tried to change it
today.

Adam
Martin J. Dürst
2010-09-06 05:41:26 UTC
Permalink
Hello Adam,
Post by Adam Barth
Post by Daniel Stenberg
Please don't add versioning to the protocol. Versioning is a anti-pattern
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Versioning has been a big pain point for TLS. Here's a presentation
http://www.ietf.org/proceedings/65/slides/tls-6.pdf
I encourage you to read through the presentation (it's pretty short).
This is indeed discouraging. TLS server side implementers seem to be
rather challenged. For a security-related protocol, that's rather bad
indeed. I very much hope WS implementers can do a better job.
Post by Adam Barth
Post by Daniel Stenberg
Or isn't it so that it truly is an anti-pattern for the *web* (with the
emphasis added on the web word) and we're not really doing web here, we're
discussing a transfer protocol.
It's a continual annoyance in dealing with the IETF to have to deal
with this "browsers don't matter" attitude.
Of course browsers do matter. But servers do matter, too.
Post by Adam Barth
In he case of this
working group, we're working on a protocol expressly for use by
browsers, which would seem to indicate that browser implementation
experience is relevant.
Of course the browser implementation experience is highly relevant! But
that doesn't mean that necessarily the browser implementation experience
with HTML (or TLS) is relevant to pre-RFC WS implementations.

The way I understand it, there are three things that could potentially
be versioned:

- Handshake
- Framing
- Extensions


Of these, the handshake is versioned almost by definition (though not
with an explicit number), given that a change of the handshake means
different header field names (I guess there's very wide agreement that
it would be extremely dumb to keep the same header field names with
different semantics).


The framing is where versioning has been proposed, but it is very
important to note that this is only for when the framing actually
changes. As soon as the framing is 'baked', I think there's wide
agreement that there is no need for versioning. Frankly speaking, I'd
also prefer to have *some* kind of indication of whether the first few
bits of a frame are a length or a frame type or what, even if that
doesn't have to be called a version number.


For extensions, as far as I understand, there is no proposal for
versioning in the sense that there is no plan to bundle certain
extensions under a certain version number. (there may be some base
protocol features (maybe multiplexing could be one of these, we don't
know yet) that are implemented by using the extension mechanism, but
that's a separate matter).


There is of course the possibility for each extension (as well as for
each subprotocol) to do versioning as they please (which hopefully will
mean as little as possible.


So overall, it seems to be that the browser experience of "be careful,
versioning, it may not work as well as you might think" is very much heeded.


Also, versioning has been requested for framing, and in particular for
the first few bits/bytes of framing. Squeezing information tightly into
bytes creates a different context with respect to versioning than a
textual protocol with lots of redundancy such as HTML.
Post by Adam Barth
Post by Daniel Stenberg
HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
The version number in HTTP hasn't been change in a very long time. I
suspect we'd encounter tremendous difficulty if we tried to change it
today.
Yes, indeed. The version number would only be changed if there were some
very basic changes in the protocol. The tremendous difficulty isn't so
much coming from a fundamental problem with versioning (if there indeed
were some very basic protocol changes, there would probably be wide
agreement with changing the version number) but from the fact that
nobody currently is interested in very basic changes in the HTTP
protocol (if you exclude this group :-).

In a similar way, it seems the big majority of contributors on this list
agree that the version number (or the absence of a version number)
should not change in a very, very long time *once the WS protocol, and
in particular the framing part, is baked*.


In some ways, this reminds me of the AtomPub WG. It kept the version
number at 0.3 while working on the spec. Everybody knew in advance that
the final version would be version 1.0. That way, it tried to make sure
that it had one shot at a clean spec, and reduced feature creep and
early lock-in. I don't know how successful that were.


Regards, Martin.
--
#-# Martin J. Dürst, Professor, Aoyama Gakuin University
#-# http://www.sw.it.aoyama.ac.jp mailto:***@it.aoyama.ac.jp
David Orchard
2010-09-06 05:50:18 UTC
Permalink
Post by Martin J. Dürst
Yes, indeed. The version number would only be changed if there were some
very basic changes in the protocol. The tremendous difficulty isn't so much
coming from a fundamental problem with versioning (if there indeed were some
very basic protocol changes, there would probably be wide agreement with
changing the version number) but from the fact that nobody currently is
interested in very basic changes in the HTTP protocol (if you exclude this
group :-).
Which is accurate, but I think that is partially because of what I
mentioned before, that the HTTP version identifier semantics weren't
defined enough. If it was "easy" to version HTTP from 1.1 to say 1.2
with some optional components, then we may have done so. But because
HTTP did not do anything like what I mentioned before, such as
something akin to "Minor versions are guaranteed to be compatible
changes and any unknown components may be ignored and must not cause a
fault in a receiver" or somesuch wording, than 1.1 made it very
difficult to create a version of HTTP that was forwards compatible
with existing receivers.

Cheers,
Dave
Julian Reschke
2010-09-06 07:17:44 UTC
Permalink
Post by David Orchard
Post by Martin J. Dürst
Yes, indeed. The version number would only be changed if there were some
very basic changes in the protocol. The tremendous difficulty isn't so much
coming from a fundamental problem with versioning (if there indeed were some
very basic protocol changes, there would probably be wide agreement with
changing the version number) but from the fact that nobody currently is
interested in very basic changes in the HTTP protocol (if you exclude this
group :-).
Which is accurate, but I think that is partially because of what I
mentioned before, that the HTTP version identifier semantics weren't
defined enough. If it was "easy" to version HTTP from 1.1 to say 1.2
with some optional components, then we may have done so. But because
HTTP did not do anything like what I mentioned before, such as
something akin to "Minor versions are guaranteed to be compatible
changes and any unknown components may be ignored and must not cause a
fault in a receiver" or somesuch wording, than 1.1 made it very
difficult to create a version of HTTP that was forwards compatible
with existing receivers.
"HTTP uses a "<major>.<minor>" numbering scheme to indicate versions of
the protocol. The protocol versioning policy is intended to allow the
sender to indicate the format of a message and its capacity for
understanding further HTTP communication, rather than the features
obtained via that communication. No change is made to the version number
for the addition of message components which do not affect communication
behavior or which only add to extensible field values. The <minor>
number is incremented when the changes made to the protocol add features
which do not change the general message parsing algorithm, but which may
add to the message semantics and imply additional capabilities of the
sender. The <major> number is incremented when the format of a message
within the protocol is changed. See RFC 2145 [36] for a fuller
explanation." --
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.3.1>

I think another answer is that we didn't need to, because HTTP/1.1 is
already extensible enough, thus didn't need any change to the message
format.

Best regards, Julian
Willy Tarreau
2010-09-06 05:59:24 UTC
Permalink
Post by Martin J. Dürst
Hello Adam,
Post by Adam Barth
Post by Daniel Stenberg
Please don't add versioning to the protocol. Versioning is a anti-pattern
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Versioning has been a big pain point for TLS. Here's a presentation
http://www.ietf.org/proceedings/65/slides/tls-6.pdf
I encourage you to read through the presentation (it's pretty short).
This is indeed discouraging. TLS server side implementers seem to be
rather challenged. For a security-related protocol, that's rather bad
indeed. I very much hope WS implementers can do a better job.
This presentation exactly outlines what is wrong when versions are used :
to make it mandatory to support a certain feature set without full
upgradability from the previous protocol. Early adopters do not completely
implement the spec and cause the trouble. In my opinion, a version should
be used *in conjunction* with a negociation, in order to announce :
- a behaviour on the wire (could have been useful for WS with the new
framing)
- the highest, fully supported standard set of feature, in order not
to have to announce them all.

There should be no problem always announcing v1.0 with a set of 20
extensions or announcing v1.1 with a set of 2 extensions. And in turn,
the spec should not make complex extensions mandatory, or it risks
having some users not support them once a protocol upgrade requires a
version bump.

Willy
Adam Barth
2010-09-06 07:36:00 UTC
Permalink
On Sun, Sep 5, 2010 at 10:41 PM, "Martin J. Dürst"
Post by Daniel Stenberg
Please don't add versioning to the protocol.  Versioning is a anti-pattern
for the web.
We've seen this mentioned before on this list but without a lot of
What are the other transfer protocols for which versioning have failed so
significantly that versioning in protocols can be called an anti-pattern?
Versioning has been a big pain point for TLS.  Here's a presentation
http://www.ietf.org/proceedings/65/slides/tls-6.pdf
I encourage you to read through the presentation (it's pretty short).
This is indeed discouraging. TLS server side implementers seem to be rather
challenged. For a security-related protocol, that's rather bad indeed. I
very much hope WS implementers can do a better job.
I wish you the best of luck. I'm not optimistic that WebSocket
servers will have a higher implementation quality than TLS servers.

Adam
Adam Barth
2010-09-01 23:14:14 UTC
Permalink
Please don't add versioning to the protocol. Versioning is a
anti-pattern for the web.

Adam
Post by Greg Wilkins
Ian,
thanks for this first draft, and I hope it indicates a new phase of
progress for the WG.
I'll be updating Jetty to support this draft immediately, but we have
an issue that there is no version information in the handshake.  Thus
in order to support this version of the draft, I risk breaking support
for previous versions, which is difficult as there are already
deployed experimental applications in the wild, interacting with real
clients with browsers beyond the control of the deployers.
Currently user-agent is not sent, so I can't even use that to
determine what version of the draft to use.
I know a standard version header is not widely accepted, but could we
implementers perhaps agree a temporary header just during this draft
 Sec-WebSocket-Draft: 1
Unless there are howls of complaint, I'll implement my server to look
for such a field and to use it as a hint when selecting the parser to
use.
cheers
Post by I***@ietf.org
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.
       Title           : The WebSocket protocol
       Author(s)       : I. Fette
       Filename        : draft-ietf-hybi-thewebsocketprotocol-01.txt
       Pages           : 57
       Date            : 2010-09-01
The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code.  The
security model used for this is the Origin-based security model
commonly used by Web browsers.  The protocol consists of an initial
handshake followed by basic message framing, layered over TCP.  The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).
This draft is meant to reflect changes in direction in the HyBi
working group.  There is not yet consensus on everything in this
draft.  Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft.  There have also been proposals
to change the handshake, so the handshake is also not in a final
form.
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
ftp://ftp.ietf.org/internet-drafts/
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Adam Barth
2010-09-01 23:14:14 UTC
Permalink
Please don't add versioning to the protocol. Versioning is a
anti-pattern for the web.

Adam
Post by Greg Wilkins
Ian,
thanks for this first draft, and I hope it indicates a new phase of
progress for the WG.
I'll be updating Jetty to support this draft immediately, but we have
an issue that there is no version information in the handshake.  Thus
in order to support this version of the draft, I risk breaking support
for previous versions, which is difficult as there are already
deployed experimental applications in the wild, interacting with real
clients with browsers beyond the control of the deployers.
Currently user-agent is not sent, so I can't even use that to
determine what version of the draft to use.
I know a standard version header is not widely accepted, but could we
implementers perhaps agree a temporary header just during this draft
 Sec-WebSocket-Draft: 1
Unless there are howls of complaint, I'll implement my server to look
for such a field and to use it as a hint when selecting the parser to
use.
cheers
Post by I***@ietf.org
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.
       Title           : The WebSocket protocol
       Author(s)       : I. Fette
       Filename        : draft-ietf-hybi-thewebsocketprotocol-01.txt
       Pages           : 57
       Date            : 2010-09-01
The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code.  The
security model used for this is the Origin-based security model
commonly used by Web browsers.  The protocol consists of an initial
handshake followed by basic message framing, layered over TCP.  The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).
This draft is meant to reflect changes in direction in the HyBi
working group.  There is not yet consensus on everything in this
draft.  Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft.  There have also been proposals
to change the handshake, so the handshake is also not in a final
form.
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
ftp://ftp.ietf.org/internet-drafts/
Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Joe Hildebrand
2010-09-01 23:34:35 UTC
Permalink
Post by John Tamplin
I would suggest adding Sec-WebSocket-Draft: 01 to
implementations of
this draft, and that header will only be present before the
Post by John Tamplin
spec is
finalized.

Let's at least agree that it shouldn't be a header that looks like something
that we had standardized as a part of the WebSocket protocol.
--
Joe Hildebrand
(as individual)
John Tamplin
2010-09-01 23:37:17 UTC
Permalink
Post by Joe Hildebrand
Let's at least agree that it shouldn't be a header that looks like something
that we had standardized as a part of the WebSocket protocol.
Fine, all I care about is that I don't have to write code that tries
to guess what version of the draft a remote endpoint is trying to use
to talk to me and hope I get it right.
--
John A. Tamplin
Software Engineer (GWT), Google
Willy Tarreau
2010-09-02 05:20:42 UTC
Permalink
Post by John Tamplin
Post by Joe Hildebrand
Let's at least agree that it shouldn't be a header that looks like something
that we had standardized as a part of the WebSocket protocol.
Fine, all I care about is that I don't have to write code that tries
to guess what version of the draft a remote endpoint is trying to use
to talk to me and hope I get it right.
Let's use something like "X-WebSocket-Experimental: xxx" or something
clear enough on this subject.

Willy
Joe Hildebrand
2010-09-02 05:50:50 UTC
Permalink
Post by Willy Tarreau
Let's use something like "X-WebSocket-Experimental: xxx" or something
clear enough on this subject.
Well... One of our ADs just wrote this:

http://tools.ietf.org/html/draft-saintandre-xdash-considered-harmful

Perhaps Peter would give his opinion on this use of X-?
--
Joe Hildebrand
Willy Tarreau
2010-09-02 06:03:06 UTC
Permalink
Post by Joe Hildebrand
Post by Willy Tarreau
Let's use something like "X-WebSocket-Experimental: xxx" or something
clear enough on this subject.
http://tools.ietf.org/html/draft-saintandre-xdash-considered-harmful
Perhaps Peter would give his opinion on this use of X-?
I see, but probably that an experiment is precisely the place where it
would have a valid place, as by definition this header will not appear
in any standard : we're sure it quickly disappears when the experiment
ends and the standard is released.

Willy
Alexey Melnikov
2010-09-02 10:44:20 UTC
Permalink
Post by Joe Hildebrand
Post by Willy Tarreau
Let's use something like "X-WebSocket-Experimental: xxx" or something
clear enough on this subject.
http://tools.ietf.org/html/draft-saintandre-xdash-considered-harmful
Perhaps Peter would give his opinion on this use of X-?
As somebody who discussed this with Peter, I think reserving a new
header for this is fine and I agree that it shouldn't be a X- header.
James Graham
2010-09-02 11:47:51 UTC
Permalink
Post by I***@ietf.org
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
A few initial comments:

The framing sections seem to have a lot of SHOULDs. This is worrying as
SHOULD-level conditions can't really be tested (it is not an error to
violate them) and can be a source of interoperability problems. I would
prefer that we make all behaviour mandatory unless there is a good
reason to do otherwise.

Some specific clauses seem problematic. For example:

"""A receiver MUST be prepared to accept arbitrarily fragmented
messages, even if the sender sent the message in a single frame."""

"be prepared to accept" seems like poor wording. I would just say
"Clients and servers MUST support recieving both fragmented and
unfragmented messages". The clause "even if the sender sent the
message..." seems odd because it is not clear how the recipient can know
how the message is originally sent. In any case it seems redundant. This
whole clause could be avoided with normative processing requirements
that require support for fragmented and unfragmented messages (see below).

"""Ping

Upon receipt of a Ping message, an endpoint SHOULD send a Pong
response as soon as is practical. The Pong response MUST contain
the payload provided in the Ping message, though an implementation
MAY truncate the message at an implementation-defined size which
MUST be at least 8 _(TBD)_ bytes."""

It seems simpler and less error prone to require truncation always in
case servers come to depend on the behaviour of specific implementations
here.

""" Ping frames MAY be sent as a keep-alive mechanism, but if so
the interval SHOULD be configurable."""

I have no idea what this SHOULD is trying to require. Is it supposed to
be a requirement on browser UI? On server implementations? Something
else? In any case, we can't really require that things are configurable.

"""Close:

Upon receipt of a close frame, an endpoint SHOULD send a Close
frame to the remote recipient, if it has not already done so,
deliver a close event to the application if necessary, and then
close the WebSocket."""

It is not clear what the scope of the SHOULD here is. Since this is
nested inside a MUST clause I guess "and then close the connection" is
supposed to be a requirement. I'm not sure why sending the close frame
is "SHOULD". I don't think it is necessary to talk about "deliver[ing] a
close event to the application" since the interaction between the
application and the protocol is for the application to determine. For
the specific case of the JS API we just need to ensure we use the right
terminology so that the description in the API document is correct (i.e.
this needs to match section 7.3 of the protocol draft and section 5 of
the API draft).


There seems to be a general lack of processing requirements. I don't
know if this is just something that is missing from the first draft, but
without such requirements there are a lot of missing details. For
example it is unclear what happens if the first frame a client receives
has opcode=0 (aside: it seems like we can design around this particular
problem by e.g. ditching opcode=0 and just ignoring opcode after the
first frame in a fragment). It is also not defined what should happen if
any of the reserved bits are set or if the opcode is in the reserved
range. All of these things are essential for interoperable implementations.
Julian Reschke
2010-09-02 12:04:33 UTC
Permalink
Post by James Graham
Post by I***@ietf.org
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
The framing sections seem to have a lot of SHOULDs. This is worrying as
SHOULD-level conditions can't really be tested (it is not an error to
violate them) and can be a source of interoperability problems. I would
...
Nothing stops you from testing them; you just need to either state that
you reported SHOULD failures as violations, or need to report different
compliance levels.

Best regards, Julian
Gabriel Montenegro
2010-09-02 16:57:25 UTC
Permalink
The real issue here is that MUST is the right keyword if something is required for interoperability. A SHOULD means you are recommended to do a certain thing, but you may depart from that recommendation if you know better or have a good reason. MAY is for optional stuff.

My take on it is that the basic framing is absolutely a MUST. Optional stuff like multiplexing or compression are MAYs (when and if we get to them). Fragmentation support sounds like a SHOULD.
-----Original Message-----
Julian Reschke
Sent: Thursday, September 02, 2010 5:05 AM
To: James Graham
Subject: Re: [hybi] I-D Action:draft-ietf-hybi-thewebsocketprotocol-01.txt
Post by James Graham
Post by I***@ietf.org
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketproto
col-01.txt
The framing sections seem to have a lot of SHOULDs. This is worrying
as SHOULD-level conditions can't really be tested (it is not an error
to violate them) and can be a source of interoperability problems. I
would
...
Nothing stops you from testing them; you just need to either state that you
reported SHOULD failures as violations, or need to report different compliance
levels.
Best regards, Julian
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
John Tamplin
2010-09-02 17:23:18 UTC
Permalink
On Thu, Sep 2, 2010 at 12:57 PM, Gabriel Montenegro
Post by Gabriel Montenegro
The real issue here is that MUST is the right keyword if something is required for interoperability. A SHOULD means you are recommended to do a certain thing, but you may depart from that recommendation if you know better or have a good reason. MAY is for optional stuff.
My take on it is that the basic framing is absolutely a MUST. Optional stuff like multiplexing or compression are MAYs (when and if we get to them). Fragmentation support sounds like a SHOULD.
I think fragmentation has to be optional for the sender, and mandatory
for the receiver. So, I think that is a sender MAY fragment a message
and a receiver MUST be prepared to receive fragmented messages.
--
John A. Tamplin
Software Engineer (GWT), Google
Julian Reschke
2010-09-02 17:39:07 UTC
Permalink
Post by Gabriel Montenegro
The real issue here is that MUST is the right keyword if something is required for interoperability. A SHOULD means you are recommended to do a certain thing, but you may depart from that recommendation if you know better or have a good reason. MAY is for optional stuff.
"may know better" certainly isn't an excuse.
Post by Gabriel Montenegro
My take on it is that the basic framing is absolutely a MUST. Optional stuff like multiplexing or compression are MAYs (when and if we get to them). Fragmentation support sounds like a SHOULD.
I agree that there probably shouldn't be any wiggle room wrt framing.

Best regards, Julian
Scott Ferguson
2010-09-02 17:55:45 UTC
Permalink
Post by Gabriel Montenegro
The real issue here is that MUST is the right keyword if something is required for interoperability. A SHOULD means you are recommended to do a certain thing, but you may depart from that recommendation if you know better or have a good reason. MAY is for optional stuff.
My take on it is that the basic framing is absolutely a MUST. Optional stuff like multiplexing or compression are MAYs (when and if we get to them). Fragmentation support sounds like a SHOULD.
Fragmentation is a MUST, based on past experience with non-browser
clients and HTTP.

Since a non-browser client might only send small queries, but receive
large data, a lazy implementation with would skip fragmenting support
entirely (because they never need to send fragments) if it's SHOULD,
because it's slightly easier for them to force the server to buffer all
data, which defeats the whole point of fragmenting. Since the receiving
code for fragmentation is actually small, allowing that kind of laziness
is harmful.

There's no requirement that anyone send fragments, of course, but they
MUST be able to receive fragmented data, or server writers will go insane.

-- Scott
Post by Gabriel Montenegro
-----Original Message-----
Julian Reschke
Sent: Thursday, September 02, 2010 5:05 AM
To: James Graham
Subject: Re: [hybi] I-D Action:draft-ietf-hybi-thewebsocketprotocol-01.txt
Post by James Graham
Post by I***@ietf.org
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketproto
col-01.txt
The framing sections seem to have a lot of SHOULDs. This is worrying
as SHOULD-level conditions can't really be tested (it is not an error
to violate them) and can be a source of interoperability problems. I
would
...
Nothing stops you from testing them; you just need to either state that you
reported SHOULD failures as violations, or need to report different compliance
levels.
Best regards, Julian
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
Ian Fette (イアンフェッティ)
2010-09-02 16:50:55 UTC
Permalink
James -

Good feedback, thank you. FWIW I agree with what you said below, but I
expect that we will flush out details once we have agreement on the base
framing. In putting this together my primary concern was to get an updated
draft that reflected the recent framing discussions so that we can settle
that issue. Once we have some reasonable level agreement that yes, this
framing is correct, then I think we can fully specify (with MUSTs) behaviour
in all of the circumstances you mention. However, it's a bit like trying to
discuss whether you want the car in red or black when your spouse is trying
to decide between a prius and a F150 -- certain things have to settle down
first :)

-Ian
Post by James Graham
Post by I***@ietf.org
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt
The framing sections seem to have a lot of SHOULDs. This is worrying as
SHOULD-level conditions can't really be tested (it is not an error to
violate them) and can be a source of interoperability problems. I would
prefer that we make all behaviour mandatory unless there is a good reason to
do otherwise.
"""A receiver MUST be prepared to accept arbitrarily fragmented
messages, even if the sender sent the message in a single frame."""
"be prepared to accept" seems like poor wording. I would just say "Clients
and servers MUST support recieving both fragmented and unfragmented
messages". The clause "even if the sender sent the message..." seems odd
because it is not clear how the recipient can know how the message is
originally sent. In any case it seems redundant. This whole clause could be
avoided with normative processing requirements that require support for
fragmented and unfragmented messages (see below).
"""Ping
Upon receipt of a Ping message, an endpoint SHOULD send a Pong
response as soon as is practical. The Pong response MUST contain
the payload provided in the Ping message, though an implementation
MAY truncate the message at an implementation-defined size which
MUST be at least 8 _(TBD)_ bytes."""
It seems simpler and less error prone to require truncation always in case
servers come to depend on the behaviour of specific implementations here.
""" Ping frames MAY be sent as a keep-alive mechanism, but if so the
interval SHOULD be configurable."""
I have no idea what this SHOULD is trying to require. Is it supposed to be
a requirement on browser UI? On server implementations? Something else? In
any case, we can't really require that things are configurable.
Upon receipt of a close frame, an endpoint SHOULD send a Close
frame to the remote recipient, if it has not already done so,
deliver a close event to the application if necessary, and then
close the WebSocket."""
It is not clear what the scope of the SHOULD here is. Since this is nested
inside a MUST clause I guess "and then close the connection" is supposed to
be a requirement. I'm not sure why sending the close frame is "SHOULD". I
don't think it is necessary to talk about "deliver[ing] a close event to the
application" since the interaction between the application and the protocol
is for the application to determine. For the specific case of the JS API we
just need to ensure we use the right terminology so that the description in
the API document is correct (i.e. this needs to match section 7.3 of the
protocol draft and section 5 of the API draft).
There seems to be a general lack of processing requirements. I don't know
if this is just something that is missing from the first draft, but without
such requirements there are a lot of missing details. For example it is
unclear what happens if the first frame a client receives has opcode=0
(aside: it seems like we can design around this particular problem by e.g.
ditching opcode=0 and just ignoring opcode after the first frame in a
fragment). It is also not defined what should happen if any of the reserved
bits are set or if the opcode is in the reserved range. All of these things
are essential for interoperable implementations.
_______________________________________________
hybi mailing list
https://www.ietf.org/mailman/listinfo/hybi
John Tamplin
2010-09-02 17:19:12 UTC
Permalink
Post by James Graham
The framing sections seem to have a lot of SHOULDs. This is worrying as
SHOULD-level conditions can't really be tested (it is not an error to
violate them) and can be a source of interoperability problems. I would
prefer that we make all behaviour mandatory unless there is a good reason to
do otherwise.
I think we can tighten those up as we get consensus. As an example,
there were people that wanted keep-alives, and people that didn't.
The language in Ping/Pong is intended to mean that those who want them
can have them, while the ones who don't, don't. I agree it would be
better to have fewer options left up to the implementation, but I
think we need more debate to get there.
Post by James Graham
"""A receiver MUST be prepared to accept arbitrarily fragmented
     messages, even if the sender sent the message in a single frame."""
"be prepared to accept" seems like poor wording. I would just say "Clients
and servers MUST support recieving both fragmented and unfragmented
messages". The clause "even if the sender sent the message..." seems odd
because it is not clear how the recipient can know how the message is
originally sent. In any case it seems redundant. This whole clause could be
avoided with normative processing requirements that require support for
fragmented and unfragmented messages (see below).
So, the case I was thinking of was someone writing the JS code for
their app and their own stand-alone server. Without some statement to
that effect, they might assume that if they send a single message the
server will receive it in a single frame, but the browser or some
intermediary might decide to fragment it anyway. I am open to
different ways of wording it, but I think there needs to be some
clarification that just because a message was sent in one fragment
doesn't meant it will stay that way.
Post by James Graham
"""Ping
     Upon receipt of a Ping message, an endpoint SHOULD send a Pong
     response as soon as is practical.  The Pong response MUST contain
     the payload provided in the Ping message, though an implementation
     MAY truncate the message at an implementation-defined size which
     MUST be at least 8 _(TBD)_ bytes."""
It seems simpler and less error prone to require truncation always in case
servers come to depend on the behaviour of specific implementations here.
So what should that size be? Also, the simplest case is you already
have the ping frame in memory, and you just immediately send it back
out -- it would actually be more work to truncate it first.
Post by James Graham
"""      Ping frames MAY be sent as a keep-alive mechanism, but if so the
interval SHOULD be configurable."""
I have no idea what this SHOULD is trying to require. Is it supposed to be a
requirement on browser UI? On server implementations? Something else? In any
case, we can't really require that things are configurable.
See above. If we can get agreement on what a keep-alive mechanism
should look like (which I think would be good so that they could be
aggregated on muxed connections or rate-limited on mobile networks,
for example), then I think it would be great to specify exactly what
that happens.

The attempt here was to capture that part of the framing which we
appear to have agreed on, and use that as a base for finishing the
rest of it.
Post by James Graham
     Upon receipt of a close frame, an endpoint SHOULD send a Close
     frame to the remote recipient, if it has not already done so,
     deliver a close event to the application if necessary, and then
     close the WebSocket."""
It is not clear what the scope of the SHOULD here is. Since this is nested
inside a MUST clause I guess "and then close the connection" is supposed to
be a requirement. I'm not sure why sending the close frame is "SHOULD". I
don't think it is necessary to talk about "deliver[ing] a close event to the
application" since the interaction between the application and the protocol
is for the application to determine. For the specific case of the JS API we
just need to ensure we use the right terminology so that the description in
the API document is correct (i.e. this needs to match section 7.3 of the
protocol draft and section 5 of the API draft).
I don't know if it is reasonable to require the close frame in all
cases -- some implementations may not make it easy to close inbound
while keeping outbound traffic alive to send the close frame, or the
architecture makes it awkward. The original closer has to be prepared
to not get a response anyway, in case the connection goes down.

Regarding the application layer, it seems like you would want to
define the interaction between the WebSocket layer and the application
layer, but I agree perhaps the discussion of framing isn't the right
place for it.
Post by James Graham
For example it is unclear what happens if the first frame a client receives has
opcode=0 (aside: it seems like we can design around this particular problem by
e.g. ditching opcode=0 and just ignoring opcode after the first frame in a
fragment).
That seems ripe for interoperability problems -- some implementations
may wind up just using the opcode of the last frame if they are all
sent the same, and an attacker could take advantage of differing
implementations to sneak some data past a filtering intermediary by
having a different opcode on an intermediate frame. It seems better
to avoid those problems by mandating a Continuation opcode, and it
also gets you useful error checking (similar to when two bits were
used for fragmentation support).
Post by James Graham
It is also not defined what should happen if any of the reserved
bits are set or if the opcode is in the reserved range. All of these things
are essential for interoperable implementations.
Ok. I would suggest mandating that any framing error, such as a
non-Continuation opcode on a non-intitial frame, using reserved bits
if no extension was negotiated which defined them, etc, the connection
should simply be dropped. Maybe we want to have a Control frame for
"Framing error", which might be part of the Close frame, or maybe just
drop it.
--
John A. Tamplin
Software Engineer (GWT), Google
I***@ietf.org
2010-09-01 22:45:02 UTC
Permalink
A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the BiDirectional or Server-Initiated HTTP Working Group of the IETF.


Title : The WebSocket protocol
Author(s) : I. Fette
Filename : draft-ietf-hybi-thewebsocketprotocol-01.txt
Pages : 57
Date : 2010-09-01

The WebSocket protocol enables two-way communication between a user
agent running untrusted code running in a controlled environment to a
remote host that has opted-in to communications from that code. The
security model used for this is the Origin-based security model
commonly used by Web browsers. The protocol consists of an initial
handshake followed by basic message framing, layered over TCP. The
goal of this technology is to provide a mechanism for browser-based
applications that need two-way communication with servers that does
not rely on opening multiple HTTP connections (e.g. using
XMLHttpRequest or <iframe>s and long polling).

Please send feedback to the ***@ietf.org mailing list.Note

This draft is meant to reflect changes in direction in the HyBi
working group. There is not yet consensus on everything in this
draft. Specifically, details about the framing are still under
discussion, however this draft is much closer to what the group is
discussing than the previous draft. There have also been proposals
to change the handshake, so the handshake is also not in a final
form.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-hybi-thewebsocketprotocol-01.txt

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
Benjamin Black
2010-09-04 02:50:14 UTC
Permalink
Versioning has been a big pain point for TLS.  Here's a presentation
http://www.ietf.org/proceedings/65/slides/tls-6.pdf
I encourage you to read through the presentation (it's pretty short).
Having dealt extensively with exactly this problem, I don't see an
"anti-pattern" in versioning. To the contrary, properly specified and
implemented versioning is very effective in making protocols that are
interoperable and extensible (one or the other alone is much easier
and does not require versioning). The anti-pattern, as evidenced in
TLS and documented in the presentation you reference, is in
_under-specifying_ the protocol and not producing a way to _validate
compliance_ such that you can produce implementations that are
compliant and incompatible. See slide 7 of the presentation for a
specific example.

Consider BGP4: it is ostensibly versioned, but almost all change
effort goes into extensions. In turn, incompatible implementations
blow up in exciting, globally destructive ways every few years. Look
at the introduction of the capabilities negotiation mechanism (which,
admittedly, did not cause _unexpected_ problems) 10 years ago, or the
recent IOS bug tickled by researchers (at Duke?) for specific
examples.
Post by Daniel Stenberg
HTTP is the primary "web protocol" and AFAIK, its versioning is usually not
considered a failure.
The version number in HTTP hasn't been change in a very long time.  I
suspect we'd encounter tremendous difficulty if we tried to change it
today.
Without delving into the realm of attempting to prove a negative, I
suggest this argument is invalid. The fact is the change from 1.0 to
1.1 with HTTP did not result in nearly the raft of incompatible but
compliant implementation problems that were seen with SSL and TLS.
Protocol versioning can absolutely be done properly. The existence of
improper specifications and implementations does not damn versioning
in general.


Back to lurking,
b

ps - This sort of voluntary rejection of rigorous specification and
compliance is exactly what I dislike about the current Websockets
protocol upgrade handshake. It's kind of HTTP, but not quite.
Hilarity ensues.
David Orchard
2010-09-04 04:09:26 UTC
Permalink
Post by Benjamin Black
Versioning has been a big pain point for TLS.  Here's a presentation
http://www.ietf.org/proceedings/65/slides/tls-6.pdf
I encourage you to read through the presentation (it's pretty short).
Having dealt extensively with exactly this problem, I don't see an
"anti-pattern" in versioning.  To the contrary, properly specified and
implemented versioning is very effective in making protocols that are
interoperable and extensible (one or the other alone is much easier
and does not require versioning).  The anti-pattern, as evidenced in
TLS and documented in the presentation you reference, is in
_under-specifying_ the protocol and not producing a way to _validate
compliance_ such that you can produce implementations that are
compliant and incompatible.  See slide 7 of the presentation for a
specific example.
There are many "rules" that can help define a robust versioning
strategy. I've worked on versioning protocols and document formats
for many years. The one thing that I've learned, which isn't a
surprise, is that it's hard. If you'd like to spend an hour or so
reading, I worked a lot on the W3C TAG on versioning and some of the
conclusions as best I could write are at
http://www.w3.org/2001/tag/doc/versioning. An example nugget: "All
systems that include inputs and outputs must consider both when making
changes and determining compatibility. For full compatibility, a
revised consumer or producer must ensure any input message changes are
backwards compatible and any output message changes are forwards
compatible."

A really big part of the problem is that documents or data that are
exchanged as part of the protocol contain version identifiers and it's
often unclear whether they are identifying the containing document or
data version information or the version identiifier is
protocol/client/server versioning that is "smuggled" into the
document. I think in the hybi scenario, there is little likelihood of
somebody picking up a bunch of hybi packets on a disk somewhere and
constructing a document - which is very much the web situation - so I
think probably any version identifiers in hybi exchanges will be for
hybi components, not the underlying languages/documents/formats.

Having dispensed with that, I think some further reading at
http://www.w3.org/2001/tag/doc/versioning-compatibility-strategies
could help. Section 5.3 describes a few good practices.

"When producers use the highest possible version identifier, languages
SHOULD provide a default model for unknown version identifiers for
forwards-compatible evolution.

The handling model could be an algorithmic approach. For version
numbers, one could say that version numbers will only have a "major"
change if there is an incompatible change. For example, version 1.1 of
a language is by definition compatible with version 1.0 and version
2.0 is incompatible. Then, when the producer puts 1.0, 1.1, or 2.0, a
consumer at any level will know whether it can process the content.
This also means that there is a choice about which version number to
put in, the lowest or the highest. A document that contains "1.1"
means that any 1.X processor can process it. A "2.0" document means
that a 1.X processor cannot process it, but any "2.X" processor can.

Then the language should have wording about processing unknown version
numbers. Sample wording for a handling model for version identifiers
is, "A processor of this version MUST not fault if it receives a
document that contains the same major version number." This rule would
be in conjunction with forwards-compatible design for the texts, such
as "Must Accept Unknowns"
"

"When producers use the lowest possible version identifiers,
languagues should only change the version identifer for incompatible
changes."

There's much more, but I hope those snippets help. If version
identifiers are in hybi, and I think they should be because the
messages will not be constructed by millions of people cut 'n pasting,
then please provide rigorous rules for what processors do with various
version identifiers they don't know about, and about messages that
contain content they don't know about, where know about typically
means that the implementor didn't know and programmed accordingly.

Cheers,
Dave
Shelby Moore
2010-09-04 13:32:23 UTC
Permalink
Fundamental law may help. Separating interface from implementation breaks
Liskov Substitution Principle as explained here:

http://esr.ibiblio.org/?p=2491#comment-277013

Thus version numbers are nothing more than orthogonal interfaces, i.e.
replacement. And any assumptions about relationships between the
interfaces of two versions are bound to fail at some granularity
(complexity) of interaction. To assume you can wall off the complexity
(e.g. that this or that won't happen) is not a sufficient defense against
a Turing-complete machine.

The solution is to make interfaces orthogonal and more finely grained, so
you only have to version (replace) the affected portions of your system. I
have given up trying to push for those principles here, as I see for
example the MORE bit is still the concensus choice. I respect the right of
people to learn by fire.

COM uses a QueryInterface() for example. You may be tempted to think this
does not apply to a protocol, but it does just the same.

Good luck to everyone and may the efforts of this WG be interesting times.
Loading...