Discussion:
Ken Coar's OpenVMS Lock utility sources are available on Sourceforge.
(too old to reply)
Jon Pinkley
2020-11-05 08:03:00 UTC
Permalink
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.

LOCK was submitted twice first LOCK010 on VAX86C, and LOCK011 on VAX87A both under the GENDYN directory.

https://www.digiater.nl/openvms/decus/vax86c/gendyn/
https://www.digiater.nl/openvms/decus/vax87a/gendyn/

But these won't help much unless you are using VAX/VMS

It had source code for LOCK (in BLISS) in the VAX87A LOCK011.A installation saveset, but was distributed with some support routines in VAX specific Bliss library (KENLIB.L32) and an VAX object library SHRLIB.OLB

In 1998, I was able to get an Alpha version of KENLIB.L32 and SHRLIB.OLB directly from Ken Coar, and I also added some features to support some of the new DLM features, specifically /expedite of null lock request and support for QUECVT which uses /serialise (Ken's choice of spelling).

LOCK uses supervisor mode locks that survive image rundown, and the lock ids are help in process kernel mode logical names.

I find it to be an extremely useful utility.

If you are interested in it, here is where it lives.

https://sourceforge.net/projects/vms-otools/

and to download a zip file of everything, go to

https://sourceforge.net/p/vms-otools/code/HEAD/tree/
and then click download snapshot.

I used unzip -a when extracting to create "standard" text files instead of stream_lf.

I was able to regenerate a lock.exe from the source I downloaded, without any of my additions.

$ analyze/image lock_main.exe

Analyze Image 5-NOV-2020 02:44:01.18 Page 1
ROOT$USERS:[JON.SCR.LOCK_TEST_BUILD]LOCK_MAIN.EXE;1
ANALYZ A01-07

This is an OpenVMS Alpha image file

---snip---

Image Identification Information

image name: "LOCK_MAIN"
image file identification: "V1.2-012"
image file build identification: ""
link date/time: 4-NOV-2020 03:14:49.03
linker identification: "A13-03"

Patch Information

There are no patches at this time.

The following is the version I have been using for the last 22 years, probably compiled on an AS2000.

$ pipe analyze/image sys$system:lock_main.exe | search sys$pipe "image ident"/win=(0,10)
Image Identification Information

image name: "LOCK_MAIN"
image file identification: "T1.2-013JLP0.1"
image file build identification: ""
link date/time: 21-AUG-1998 00:37:27.63
linker identification: "A11-20"

Patch Information

There are no patches at this time.
$

My additions are not in the code that is on sourceforge, and I didn't find a command file to build the support files. There is no LCLDEF.SDL file for the LDLDEF.R32 file, but it should work as long as you use bliss32. I don't know if Ken could even find the original SDL file, because he did include the LOCK_DEF.SDL file. But for LCLDEF.R32 (used by KENLIB.B32), the LCLDEF.SDL was not provided.

I can't redistribute modified source without Ken's permission, but I could create a DIFFERENCES/SLP file with the changes needed to create what I have using EDIT/SUMSLP and the sources from sourceforge.

It is a utility that gets used frequently here, so it is one of the must have working when we migrate to the Itanium, so rebuilding from source was on my todo list.
Jon Pinkley
2020-11-06 05:32:55 UTC
Permalink
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
That was a typo. It was on the VAX86C and VAX87A tapes. 1998 is when I built the version I am running now.
Dave Froble
2020-11-06 05:33:42 UTC
Permalink
Post by Jon Pinkley
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
That was a typo. It was on the VAX86C and VAX87A tapes. 1998 is when I built the version I am running now.
What does this utility do?
--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
Jan-Erik Söderholm
2020-11-06 08:26:10 UTC
Permalink
Post by Dave Froble
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
That was a typo.  It was on the VAX86C and VAX87A tapes.  1998 is when I
built the version I am running now.
What does this utility do?
https://www.digiater.nl/openvms/decus/vax87a/gendyn/aaareadme.txt
Jon Pinkley
2020-11-06 10:21:42 UTC
Permalink
Post by Dave Froble
Post by Jon Pinkley
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
That was a typo. It was on the VAX86C and VAX87A tapes. 1998 is when I built the version I am running now.
What does this utility do?
The LOCK utility is nothing more than a DCL interface to the distributed lock manager system services SYS$ENQ and SYS$DEQ. LOCK uses supervisor mode locks, so the locks granted to the process don't disappear with image rundown.

It allows you to use the OpenVMS distributed lock manager from DCL. So you can use it to do things the VMS DLM allows, but from DCL. That can be for synchronizing access to some object that can't be shared safely, or to enforce FIFO, to detect/prevent a multi-step job from running concurrently, to allow failover to another node in the cluster (with finer granularity than autostart queues).

Or perhaps you have a poorly designed program that uses fixed filenames, so you can't run multiple instances concurrently without problems. With LOCK you can avoid the problem by forcing single threading as long as you can make everyone use your wrapper to use the program.

Or if you have a single concurrent use compiler license, and you don't want jobs to fail halfway through a build when someone else compiles something.

You can also implement "door bell" request, to wake a process up when there is new work to be done. This can be done by requesting a lock with /surrender, which causes LOCL to request a blocking AST if there is an request that forces another request to wait.

It is quite general in purpose, and you can specify the lock mode (and scope, by default group resource domain, but you can specify /scope=system which means cluster wide.

It doesn't prevent access, it just gives your the ability to have e

It does not support sublocks, or resource domains, 64 byte value blocks, or "system owned locks" (but those are not for general use anyway).

You need to understand the lock manager, and how the different modes interact.

It is a good way to learn how the lock manager works, i.e. that a new lock that is compatible with the current lock won't be granted if there are other locks waiting to convert, even if the new lock is NULL. In some version old version of VMS (v5?) the LCK$M_EXPEDITE and LCK$M_QUECVT flags were added. With LCK$M_EXPEDITE you can always get a new NULL lock, and then you can convert the granted NULL lock to another mode, and if that mode is compatible, it will be granted immediately.

You and also specify /nowait and if the lock can't be granted synchronously, lock will return a fatal status, which you can see with $status or $severity. If you specify /nowait, no blocking AST will be sent, that is just the way the DLM works.

All described in the Programming Concepts manual.

It is a bit like ethernet vlans, if you don't know you need them, you probably won't miss them. But once you see how useful they can be, you will miss them if you need them but don't have a vlan-aware switch.

Here's an example showing two processes one with prompt "P1$ " and one with prompt "P2$ ". Pasting from two terminal sessions with a blank line between.

P2$ lock abc
%LOCK-S-OBTAINED, 6-NOV-2020 05:11:10.64, obtained EXCLUSIVE lock on ABC
P2$ show symbol $status
$STATUS == "%X1AB78001"
P2$

P1$ lock abc
%LOCK-I-LOCKED, 6-NOV-2020 05:11:55.25, resource ABC locked by another user, waiting

P2$ lock abc/mod=null
%LOCK-S-CONVERTED, 6-NOV-2020 05:12:09.72, EXCLUSIVE lock on ABC converted to NULL
P2$

%LOCK-S-OBTAINED, 6-NOV-2020 05:12:09.72, obtained EXCLUSIVE lock on ABC
P1$ show symbol $status
$STATUS == "%X1AB79003"
P1$

P2$ lock abc/nowait ! note default /mode is EXCLUSIVE
%LOCK-F-LOCKED, 6-NOV-2020 05:12:47.98, resource ABC locked by another user, aborting
P2$ show symbol $status
$STATUS == "%X1AB7C004"
P2$
Dave Froble
2020-11-06 22:31:26 UTC
Permalink
Post by Jon Pinkley
Post by Dave Froble
Post by Jon Pinkley
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
That was a typo. It was on the VAX86C and VAX87A tapes. 1998 is when I built the version I am running now.
What does this utility do?
The LOCK utility is nothing more than a DCL interface to the distributed lock manager system services SYS$ENQ and SYS$DEQ.
Thanks, that is what I was looking for.

I'm rather familiar with the DLM, using it from within programs. Never
thought of using it from DCL.
--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
Simon Clubley
2020-11-09 13:14:22 UTC
Permalink
Post by Dave Froble
Post by Jon Pinkley
Post by Dave Froble
Post by Jon Pinkley
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
That was a typo. It was on the VAX86C and VAX87A tapes. 1998 is when I built the version I am running now.
What does this utility do?
The LOCK utility is nothing more than a DCL interface to the distributed lock manager system services SYS$ENQ and SYS$DEQ.
Thanks, that is what I was looking for.
I'm rather familiar with the DLM, using it from within programs. Never
thought of using it from DCL.
There are a good number of ideas in Unix which can be used to improve
VMS, but the ability to use the DLM to notify everyone else that some
activity or program is running is a far better approach than having to
use lock files to indicate that.

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Stephen Hoffman
2020-11-09 15:34:22 UTC
Permalink
...but the ability to use the DLM to notify everyone else that some
activity or program is running is a far better approach than having to
use lock files to indicate that.
You're asking for a job scheduler.

DLM might well be part of that scheduler implementation. But as an
interprocess notification and coordination mechanism, the DLM API...
does not deal easily with the common cases.
--
Pure Personal Opinion | HoffmanLabs LLC
Simon Clubley
2020-11-09 18:24:44 UTC
Permalink
Post by Stephen Hoffman
...but the ability to use the DLM to notify everyone else that some
activity or program is running is a far better approach than having to
use lock files to indicate that.
You're asking for a job scheduler.
No, I'm not.

Imagine if, for example, a webserver took out a lock at startup so you
know it's running. Or if a program took out a lock during a critical
section so that another instance of the program can't run the same
critical section at the same time.

If the web server (for example) crashes, the lock disappears without any
manual cleanup of a lock file required. Same if the system crashes
or there's a power failure.
Post by Stephen Hoffman
DLM might well be part of that scheduler implementation. But as an
interprocess notification and coordination mechanism, the DLM API...
does not deal easily with the common cases.
I can imagine a large number of common cases it deals with, including
the examples above.

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Stephen Hoffman
2020-11-09 19:43:51 UTC
Permalink
This post might be inappropriate. Click to display it.
Jon Pinkley
2020-11-06 10:33:44 UTC
Permalink
Post by Dave Froble
What does this utility do?
Assuming you have access to a VMS system with runoff

https://sourceforge.net/p/vms-otools/code/HEAD/tree/tags/lock-1.2/lock_help.rnh
Stephen Hoffman
2020-11-06 16:36:02 UTC
Permalink
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
...
FWIW...

There've been a few other DLM-from-DCL utilities available over the
years, too. Here are two more:

https://www.digiater.nl/openvms/freeware/v80/lck/freeware_readme.txt
http://mail.digiater.nl/openvms/decus/vax92b/netlock/aaareadme.txt

More than a few other apps use lock files for this purpose, as that
tends to be somehwat more portable. And that approach can work from
DCL, too.
--
Pure Personal Opinion | HoffmanLabs LLC
Jon Pinkley
2020-11-06 23:12:17 UTC
Permalink
Post by Stephen Hoffman
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
...
FWIW...
There've been a few other DLM-from-DCL utilities available over the
https://www.digiater.nl/openvms/freeware/v80/lck/freeware_readme.txt
http://mail.digiater.nl/openvms/decus/vax92b/netlock/aaareadme.txt
More than a few other apps use lock files for this purpose, as that
tends to be somehwat more portable. And that approach can work from
DCL, too.
Of the above two, the LCK is more like LOCK, in fact it almost looks like it was written to replace LOCK which was never released for the Alpha on the DECUS tapes or freeware. From the Docs, it appears to be a proper subset of what LOCK provides. For example, the only mode is EXCLUSIVE. It doesn't support passing info with the value block. Still useful for the most used case, and my intent isn't to denigrate the LCK or NETLOCK utilities, they are useful and the authors were kind to share their work.

As far as portability, my argument is that if you are using DCL, it probably isn't a big issue, because you have already decided that portability isn't the most important issue.

I am showing my ignorance here, but how do you implement the equivalent of waiting for a lock with a lock file (queuing a lock request, $ENQ)? I can understand how the equivalent of lock/nowait/mode=exclusive could be done with a lock file, but waiting? It seems to me that would involve polling.
Craig A. Berry
2020-11-06 23:45:26 UTC
Permalink
Post by Jon Pinkley
Post by Stephen Hoffman
Post by Jon Pinkley
In 1998, Ken Coar submitted the LOCK utility to the DECUS VAXSIG tape.
More than a few other apps use lock files for this purpose, as that
tends to be somehwat more portable. And that approach can work from
DCL, too.
I am showing my ignorance here, but how do you implement the
equivalent of waiting for a lock with a lock file (queuing a lock
request, $ENQ)? I can understand how the equivalent of
lock/nowait/mode=exclusive could be done with a lock file, but
waiting? It seems to me that would involve polling.
You wouldn't want to poll, but you might use select() or something from
the libevent library or similar. Which are also non-portable.
Marc Van Dyck
2020-11-07 11:10:28 UTC
Permalink
Post by Jon Pinkley
Of the above two, the LCK is more like LOCK, in fact it almost looks like it
was written to replace LOCK which was never released for the Alpha on the
DECUS tapes or freeware. From the Docs, it appears to be a proper subset of
what LOCK provides. For example, the only mode is EXCLUSIVE. It doesn't
support passing info with the value block. Still useful for the most used
case, and my intent isn't to denigrate the LCK or NETLOCK utilities, they are
useful and the authors were kind to share their work.
I'm the author of LCK and would love to find a place to store more
recent versions... It also works on Itanium now. If there is a demand
for locks in other modes than exclusive, I'm ready to give it a try.
--
Marc Van Dyck
Jon Pinkley
2020-11-08 04:09:51 UTC
Permalink
Post by Marc Van Dyck
I'm the author of LCK and would love to find a place to store more
recent versions... It also works on Itanium now. If there is a demand
for locks in other modes than exclusive, I'm ready to give it a try.
--
Marc Van Dyck
The two places that come to mind would be Hunter Goatley's fileserv at ftp.process.com which may be mirrored and Eisner.decus.org, which has a repository of many of the decus tape contents. You can telnet to eisner.decus.org and read the sys$annouce for how to get a free account there.

For example see ftp://ftp.process.com/vms-freeware/fileserv.readme
and ftp://ftp.process.com/vms-freeware/fileserv

Now a question about LCK. Had you ever seen Ken's LOCK utility before you wrote LCK? It has a very similar interface, although Ken's doesn't support a programming library to simplify the use of locks.

Ken's is written in Bliss which until it was made available on the freeware disk, was prohibitively expensive. Pascal is probably better known than Bliss, but it isn't a "popular" language either, and isn't free to use, so you would need to supply at least .obj (and hopefully source).
Marc Van Dyck
2020-11-08 17:17:12 UTC
Permalink
This post might be inappropriate. Click to display it.
Hunter Goatley
2020-11-09 04:48:06 UTC
Permalink
Post by Marc Van Dyck
I tried to arrange a hosting on ftp.process.com with Hunter several
months ago, He agreed, I sent him kits and sources, but I don't know
why, it never appeared on the site.
I'm embarrassed to say that I got busy after you sent it, and then I
forgot all about it. 8-( Sorry about that! I'll try to get it posted on
Monday!
--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/
Hunter Goatley
2020-11-09 16:23:06 UTC
Permalink
Post by Hunter Goatley
Post by Marc Van Dyck
I tried to arrange a hosting on ftp.process.com with Hunter several
months ago, He agreed, I sent him kits and sources, but I don't know
why, it never appeared on the site.
I'm embarrassed to say that I got busy after you sent it, and then I
forgot all about it. 8-( Sorry about that! I'll try to get it posted
on Monday!
Finally, MBX and LCK have been added to my archive:

http://vms.process.com/scripts/fileserv/fileserv.com?LCK

http://vms.process.com/scripts/fileserv/fileserv.com?MBX

http://www.process.com/resources/openvms/index.html

Thanks, Marc, and again, sorry for the way-too-long delay!
--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/
Marc Van Dyck
2020-11-09 22:33:02 UTC
Permalink
Post by Hunter Goatley
Post by Hunter Goatley
Post by Marc Van Dyck
I tried to arrange a hosting on ftp.process.com with Hunter several
months ago, He agreed, I sent him kits and sources, but I don't know
why, it never appeared on the site.
I'm embarrassed to say that I got busy after you sent it, and then I
forgot all about it. 8-( Sorry about that! I'll try to get it posted
on Monday!
http://vms.process.com/scripts/fileserv/fileserv.com?LCK
http://vms.process.com/scripts/fileserv/fileserv.com?MBX
http://www.process.com/resources/openvms/index.html
Thanks, Marc, and again, sorry for the way-too-long delay!
No need to be embarassed. Thank you for offering me a home...
I'll start working on an enhanced LCK as soon as I can.
--
Marc Van Dyck
Stephen Hoffman
2020-11-07 16:11:23 UTC
Permalink
Post by Jon Pinkley
I am showing my ignorance here, but how do you implement the equivalent
of waiting for a lock with a lock file (queuing a lock request, $ENQ)?
I can understand how the equivalent of lock/nowait/mode=exclusive could
be done with a lock file, but waiting? It seems to me that would
involve polling.
Yeah, "polling is bad". We're also operating on GHz-class processors.
The performance concerns for checking something in a loop once a
~second are negligible. Particularly if the DCL loop code is both
simple and is never going to fast-loop. Premature optimization, etc.
Put differently, I poll. It's simple, undemanding, with very low
overhead at even an "aggressive" one-second polling loop, the resulting
code is clear and easy to maintain and self-contained, and it works.

If I were polling more frequently or needed higher performance or
faster responsiveness to changes, well, I probably also wouldn't be
using DCL either.

As for alternatives, I've seen and have used both coordinated access
with an indexed file (with records for tasks and PIDs and times) and
that with polling from DCL to implement a "job scheduler", and have
implemented a DECnet-based "job scheduler" from DCL.

As for what apps can do with files and what happens with source code
that gets ported around, C offers file locking via fcntl et al. On
OpenVMS, that C locking support uses DLM.

And some other shells can have some locking analog, such as flock:
https://linux.die.net/man/1/flock

I've seen and/or used scheduling tools such as Xgrid and distcc and
ZooKeeper used for this on other platforms (and CMake and Ninja have
capabilities here too), as well as job schedulers on OpenVMS including
DECscheduler.

I'm skeptical of the value of incremental modifications to DCL given
its oft-discussed limitations and issues. For various reasons. I would
like prefer asynchronous capabilities in whatever might be the DCL
upgrade path. Or in whatever might supplement DCL for scripting, if
Python or Lua or some other scripting language might eventually ship in
the base OS. DCL just doesn't do quadword math, or asynchronous I/O, or
try-catch-punt error handling (yes, I know about ON and SET), or TLS,
or a job scheduler, or... all that well. And I'd doubt that VSI would
benefit from a DCL2 replacement or whatever as compared with the return
on that investment. And that's ignoring what would traditionally then
be a slow and messy and not-seamless migration from DCL to
DCL2/zsh/whatever. Supplementing DCL with some other scripting tooling
avoids most of that, without having to create a wholly new DCL-flavored
domain-specific language and tooling. But then how quickly OpenVMS
folks would adopt Python or Lua or whatever is itself an open question.
So here we are...

Various of these smaller feature-related discussions are symptoms of
larger missing features. This is one.
--
Pure Personal Opinion | HoffmanLabs LLC
Tom Wade
2020-11-09 22:41:22 UTC
Permalink
Post by Stephen Hoffman
There've been a few other DLM-from-DCL utilities available over the
https://www.digiater.nl/openvms/freeware/v80/lck/freeware_readme.txt
http://mail.digiater.nl/openvms/decus/vax92b/netlock/aaareadme.txt
And yet another one at www.tomwade.eu/software/superlock.html

Tom Wade
tom dot wade at tomwade dot eu

Loading...