Post by Dirk MunkPost by Bill GunshannonPost by Dirk MunkPost by Dirk MunkNo, of course Unix is not immune. Using <lf> or <cr> (Windows) as
record terminator is a rather silly idea. It means that you can't use
those characters in a record, and you have to scan the contents of a
file for those characters. Simply writing the length of a record at the
beginning of that record is far better solution.
Having a <LF> or a <CR> in text files seems rather logical to me. What
else, if you want either a line feed or a carriage return?
But yes, there are other ways to specify and delimiting a "line of
text", if you have a system suporting that.
Now, if that "record" is something else than a "line of text"...
The problem is that in Unix and Windows land there is no difference
between the metadata of a file, and the actual contents of a file. The
metadata should define the file and the records in the file, that
should be completely separate from the actual data contents of the file.
Can't speak for Windows, but Unix has no meta-data. Unix has only one
file type, a stream of bytes. Everything else is application layer.
Which means you don't have a clue about the contents of a file, until
you know the internals of the application. Standard VMS applications
produce structured files, so you only have to worry about the data
contents. It is possible to write your own applications using the files
of another application. The application can be in any language, because
RMS is the layer between the application and the file. This is a
structured approach, instead of producing a diarrhea of bytes, and
calling it a file.
Most of us use file magic to see what sort of app file we're looking
at. And those same file magic tools common on Unix are also pretty good
at identifying common OpenVMS-format files, too.
RMS is a database that emulates punched cards and provides the related
sorting and arrays from that era, and it still works pretty well where
punched cards and related access is a workable data store for an app.
RMS is problematic when it comes to updates, modifications, or pretty
much anything past punched cards. And RMS knows nothing about the data
and data format and encoding used in the record.
Nobody's suggesting removing RMS. But a database that tops out with
key-value access and DEC MCS support is not going to be viewed as a
product differentiator.
Post by Dirk MunkPost by Bill GunshannonPost by Dirk MunkSuppose I have a file with binary data, and one byte has the binary
(ascii) value of <lf>, then Unix will use it as a record separator,
even if it is in the middle of the actual data of that record.
As stated below, Unix doesn't work that way any more than PRINT
SYS$SYSTEM:AUTHORIZE.EXE "works".
Post by Dirk MunkPost by Bill GunshannonUnix has no records. If you cat the file it will line break at the
<lf>. If you od -c the file it will identify the <lf> as just that.
Wonderful. However, it is clear that in many applications the notion of
a data record is present, and that the <lf> is used as record
separator, even if Unix formally doesn't have records.
I'm finding myself poking around in the databases of other apps rather
less often. OpenVMS or elsewhere.
Interestingly—when poking around in an app data store is
required—that's often gotten much easier on Unix, as the data stores
are increasingly using the local analog to RMS.
That local equivalent tends to be SQLite, in the environments I'm often
working in. And SQLite provides much better clues about "record"
formats and fields and field relationships, too.
Post by Dirk MunkPost by Bill GunshannonPost by Dirk MunkSuppose you have a VMS file with fixed record size. That file has no
records separators what so ever, it is one long stream of data. VMS can
calculate where the records start and end in the file. Suppose it
consists out of sets of three records of 100 bytes that belong
together. Then you can change the attributes of that file to records of
300 bytes, and in one read operation you will have all the data that
belongs together. I've actually used this in the past.
And that would be an application concept, not really an OS thing.
Actually not, since this can only be done because of the way RMS stores
data, and RMS is part of the OS.
Which is part and parcel of what some of us have been grumbling about
for years; that OpenVMS, well, stopped enhancing its data access in the
1980s, and given the efforts to embed the Rdb database support run-time
ended with the sale of Rdb to Oracle.
SQLite would be one option here for future integration alongside RMS,
and there are others.
Post by Dirk MunkPost by Bill GunshannonPost by Dirk MunkSuppose you want to print such a file, then VMS will send a <cr> and a
<lf> to the printer after each record. Simple.
VMS won't. Whatever application actually prints it will.
Obviously, this is functionality of the spooler, and that is part of VMS.
OpenVMS printing really shouldn't be printing non-printable files—see
file magic above, see PRINT SYS$SYSTEM:AUTHORIZE.EXE above, etc.
Printing needs work. But here we are.
Post by Dirk MunkPost by Bill GunshannonPost by Dirk MunkThe DEC software engineers understood very well why it is a bad idea to
mix up contents of a file with the structure of a file, and that's why
they did not use stream files as standard RMS files in applications.
They are just there for compatibly with Unix, Windows etc.
And Unix made all files streams of bytes and lets the applications
decide what to do with them. Not really an OS problem.
Ayup.. And I've found the Unix approach works very well. OpenVMS folks
do the same wad-of-bytes thing here with the add-on databases, too;
with Oracle Rdb, SQLite, and other such.
Post by Dirk MunkExchanging data between applications is rather important. Those
applications can be written in many languages, can come from different
sources. It is obvious that well structured files are paramount for
exchanging data between applications. That is why something like RMS is
in fact a very modern approach to structured software engineering,
instead of producing a an unstructured diarrhea of bytes, and calling
it a file.
Can't say I really want to document my internal data store as my
import-export API, but you do you. If it's SQLite, that access does
work decently well, as SQLite databases are themselves quite portable
including across endianness differences. But even on OpenVMS, poking
directly into an app's RMS database is close to reverse-engineering,
and not really an interface that most app developers want to support.
Poking directly into SYSUAF isn't recommended, even if SYSUAF currently
uses RMS, Providing a supported and documented data import-export being
much more typical. That interchange format might be YAML or XML or any
number of other interfaces, and frameworks and tooling are available
for all of the common formats. Put differently, we've moved from
abstracting at the RMS or SQLite or other layer to a higher-level
abstraction or data import-export interface.
And RMS has other gaps here beyond its inability to import and export
its common files, too. RMS never got around to providing support for
consistent live backups, though various add-on databases do. RMS lacks
data definitions within records, too. CDD/Repository went to Oracle,
and that and SDL and other data definition tooling. RMS gets you the
record yes, but is less than useful with the app data within the
record, and the character encoding for the data, and related details.
And RMS itself—and BACKUP has similar issues—stinks at identifying and
repairing issues of metadata. Which was the reason for this thread.
Again, RMS was great in the last millennium. Its age is showing.
There's absolutely no reason to remove RMS, but RMS is comparatively
limited. But if RMS works for you, have at. Streams-of-bytes file
systems, and SQLite and other databases, and other tooling work well
for others. RMS is just not something I miss, when working on other
platforms.
--
Pure Personal Opinion | HoffmanLabs LLC