joerg van den hoff
2018-08-06 21:52:12 UTC
Here is the technical problem I am having difficulty with. My
currently solution is (probably) what is causing the delays and errors
you are seeing. Suggestions from you are any mailing list reader on
how to solve this problem are appreciated.
I've read through your explanations below but there are sure nocurrently solution is (probably) what is causing the delays and errors
you are seeing. Suggestions from you are any mailing list reader on
how to solve this problem are appreciated.
suggestion from my side... I hardly understand the nature of the
problem. especially I am at a loss where all this backoffice machinery
interferes with CLI/ssh communication between two repos (or cloning, see
below).
however, I would like to augment the list of problems I am experiencing
right now (this is with version [71260ba2] as proposed by you earlier
today):
* fossil uv sync no longer works correctly. I did try it this evening
(still accessing our server via ssh but now over the wire from home
rather than from wihtin the file system (if this makes any difference)
and was not able to get one new uv file that was added to the server
repo sync to my labtop.
* I then tried to re-clone the whole repo. cloning failed completely with
multiple calls to backoffice_run() received: 113
server did not reply
Clone done, sent: 676 received: 364935 ip: {our_server}
server returned an error - clone aborted
I also want to emphasize that these problems all happen without ever
opening the fossil ui or trying to communicate via https. this is a pure
CLI/ssh scenario.
so this all is rather bad. it is also the first time in years that I can
remember that the tip of trunk is not working essentially without any
serious problems.
I would appreciate being pointed to the most recent (or thereabouts)
checkin on trunk that is not experiencing these problems in order to
get fossil back to working as nicely as it usually does.
If I can provide any further information or help otherwise in tracking
the present problems down, I would of course be happy to do so.
overall, I really hope that not too much complexity is currently added
to fossil that could lead to a situation where fossil no longer excels
by ease of use and stability/absence of problems or serious bugs as it
has done now for so many years, thankfully.
thank you,
joerg
Basically: Fossil needs to run some operations in the background.
With the introduction of email notification, some computations need to
be performed in the background on the Fossil server. Right now, the
only background operation is sending email alerts to subscribers. In
the future, I'll probably want to add automatic synchronization to
peer servers and maybe other things.
I call the module that does background processing the "backoffice".
The name comes from the analogy of a business where an order is placed
at the counter in the front of the store, but the actually fulfillment
of the order takes place in the "back office", out of sight of the
customer.
(1) Only a single process should be running backoffice processing at a
time. There can be multiple processes serving up web pages, but there
is only a single process sending email notification. Serialization of
backoffice processing is handled by making atomic updates to the entry
in the repository CONFIG table with name='backoffice'.
(2) Backoffice processing should happen independently of webpage
generation. The results of an HTTP request should not need to wait
for some backoffice process to complete.
(3) Once one backoffice process completes, no other should run for
another 60 seconds or so. In other words, backoffice processes should
be rate limited.
(4) At least one backoffice process should be run within about 60
seconds after the any successful HTTP request. A single backoffice
run can satisfy this requirement for any number of HTTP requests. So,
for example, if there is a flurry of 1000 HTTP requests then 5 seconds
later there is a single backoffice run, then that one run of the
backoffice is sufficient for all 1000 HTTP requests.
(5) If there are no HTTP request in the past minute or two, then there
should not be any backoffice processes running or waiting to run. The
idea here is that a website like sqlite.org has literally about a
hundred separate Fossil repositories. SQLite is very busy and it
would be ok to have a persistent backoffice process running for it.
But many of the other 99 repos are accessed much less frequently, and
we don't want 99 processes waiting around for activity for days on
end. Also, I want to be able to upgrade the Fossil executable by
simply overwriting it, and then have any backoffice processes
eventually (within a minute or two) switch over to using the new
executable.
(6) In keeping with the easy-to-setup goal of Fossil, running the
backoffice process should not require any special setup or
configuration by the site administrator. It should just work.
The way the above is implemented now is that after every successful
HTTP request is completed, the process that generated the reply checks
to see if a backoffice process is needed, and if so it morphs itself
into a backoffice process. If there is already one backoffice process
running, then the new one might become the "on-deck" process that is
to run next. The HTTP reply has been sent, so you would think that
would be sufficient to avoid long delays in the user interface. But
somehow, keeping that process around for a little longer to do
backoffice processing is causing delays in the HTTP reply. A lot
depends on webserver and (surprisingly) the webbrowser.
Anyhow, I'm looking for a better and more reliable way to implement
the backoffice such that it satisfies the six requirements listed
above.
Alternative design ideas are welcomed.
With the introduction of email notification, some computations need to
be performed in the background on the Fossil server. Right now, the
only background operation is sending email alerts to subscribers. In
the future, I'll probably want to add automatic synchronization to
peer servers and maybe other things.
I call the module that does background processing the "backoffice".
The name comes from the analogy of a business where an order is placed
at the counter in the front of the store, but the actually fulfillment
of the order takes place in the "back office", out of sight of the
customer.
(1) Only a single process should be running backoffice processing at a
time. There can be multiple processes serving up web pages, but there
is only a single process sending email notification. Serialization of
backoffice processing is handled by making atomic updates to the entry
in the repository CONFIG table with name='backoffice'.
(2) Backoffice processing should happen independently of webpage
generation. The results of an HTTP request should not need to wait
for some backoffice process to complete.
(3) Once one backoffice process completes, no other should run for
another 60 seconds or so. In other words, backoffice processes should
be rate limited.
(4) At least one backoffice process should be run within about 60
seconds after the any successful HTTP request. A single backoffice
run can satisfy this requirement for any number of HTTP requests. So,
for example, if there is a flurry of 1000 HTTP requests then 5 seconds
later there is a single backoffice run, then that one run of the
backoffice is sufficient for all 1000 HTTP requests.
(5) If there are no HTTP request in the past minute or two, then there
should not be any backoffice processes running or waiting to run. The
idea here is that a website like sqlite.org has literally about a
hundred separate Fossil repositories. SQLite is very busy and it
would be ok to have a persistent backoffice process running for it.
But many of the other 99 repos are accessed much less frequently, and
we don't want 99 processes waiting around for activity for days on
end. Also, I want to be able to upgrade the Fossil executable by
simply overwriting it, and then have any backoffice processes
eventually (within a minute or two) switch over to using the new
executable.
(6) In keeping with the easy-to-setup goal of Fossil, running the
backoffice process should not require any special setup or
configuration by the site administrator. It should just work.
The way the above is implemented now is that after every successful
HTTP request is completed, the process that generated the reply checks
to see if a backoffice process is needed, and if so it morphs itself
into a backoffice process. If there is already one backoffice process
running, then the new one might become the "on-deck" process that is
to run next. The HTTP reply has been sent, so you would think that
would be sufficient to avoid long delays in the user interface. But
somehow, keeping that process around for a little longer to do
backoffice processing is causing delays in the HTTP reply. A lot
depends on webserver and (surprisingly) the webbrowser.
Anyhow, I'm looking for a better and more reliable way to implement
the backoffice such that it satisfies the six requirements listed
above.
Alternative design ideas are welcomed.