[H390-MVS] MVS38J IND$FILE DFT/DDM and WSF

[[e-mail] [H390-MVS]]

Guys, after many years of hanging in the background, I finally have some time to work on IND$FILE again.

The new version will have DFT/DDM support as well as the old CUT mode transfer support.

However I've run into a bit of an issue running under MVS3.8J with WSF data streams using buffers larger than about 2K in size. With a 4K buffer and WSF style data streams sent/received via TPUT/TGET, the TPUT calls seem to work as expected, however the TGET with a 4K buffer fails and causes a terminal disconnect (not nice). So I may be forced to limit the buffer size to 2K which is far less than optimal for IND$FILE transfers.

From a system maintenance side, I have ZP60008 and ZP60009 applied using the most recent downloads as those have received a few modification over the many years since they first appeared on the web http://www.prycroft6.com.au/vs2mods/ http://www.prycroft6.com.au/vs2mods/ .

My guess about the larger buffers causing disconnects would be an issue with the internal buffers available in TSO or VTAM. If you have some ideas on how we can use larger buffers for the WSF data streams please feel free to make suggestions.

Thanks.
--Mike

[[e-mail] [H390-MVS]]

Hi Mike


good to see you back again!


Concerning your buffer size problem: Could you, as a cross check, verify that you are seeing the same problem on TK4-? You can download the current version (Update 07) from http://wotho.ethz.ch/tk4-/tk4-_v1.00_current.zip http://wotho.ethz.ch/tk4-/tk4-_v1.00_current.zip. It's just an unzip and go, so the effort to try it should be minimal. Basically, you can say: If it does happen there too, then it's an internal VTAM or TSO/VTAM limitation, as I've optimized both for largest possible buffer sizes.



Concerning your new IND$FILE version: TK4- Update 08 is on the ramp. I'm expecting to run a short beta phase starting mid September and would of course very much appreciate being able to take the new IND$FILE in...



Cheers
JÃŒrgen

---In H390-***@yahoogroups.com, <***@...> wrote :


Guys, after many years of hanging in the background, I finally have some time to work on IND$FILE again.

The new version will have DFT/DDM support as well as the old CUT mode transfer support.

However I've run into a bit of an issue running under MVS3.8J with WSF data streams using buffers larger than about 2K in size. With a 4K buffer and WSF style data streams sent/received via TPUT/TGET, the TPUT calls seem to work as expected, however the TGET with a 4K buffer fails and causes a terminal disconnect (not nice). So I may be forced to limit the buffer size to 2K which is far less than optimal for IND$FILE transfers.

From a system maintenance side, I have ZP60008 and ZP60009 applied using the most recent downloads as those have received a few modification over the many years since they first appeared on the web http://www.prycroft6.com.au/vs2mods/ http://www.prycroft6.com.au/vs2mods/ .

My guess about the larger buffers causing disconnects would be an issue with the internal buffers available in TSO or VTAM. If you have some ideas on how we can use larger buffers for the WSF data streams please feel free to make suggestions.

Thanks.
--Mike

[Mike Rayborn [e-mail] [H390-MVS]]

I don't have a packaged new version of IND$FILE to install on the TK4
system just now. That code is a work in progress on the old TK3 system
and it would take some effort to move it all to the TK4 system.

That said, I downloaded the TK4 system and fired it up without issue.
So that may end up as the new development platform once I can get things
moved from TK3 to TK4.

--Mike

Post by ***@id.ethz.ch [H390-MVS]
Hi Mike
good to see you back again!
Concerning your buffer size problem: Could you, as a cross check,
verify that you are seeing the same problem on TK4-? You can download
the current version (Update 07) from
http://wotho.ethz.ch/tk4-/tk4-_v1.00_current.zip. It's just an unzip
and go, so the effort to try it should be minimal. Basically, you can
say: If it does happen there too, then it's an internal VTAM or
TSO/VTAM limitation, as I've optimized both for largest possible
buffer sizes.
Concerning your new IND$FILE version: TK4- Update 08 is on the ramp.
I'm expecting to run a short beta phase starting mid September and
would of course very much appreciate being able to take the new
IND$FILE in...
Cheers
JÃŒrgen
Guys, after many years of hanging in the background, I finally have
some time to work on IND$FILE again.
The new version will have DFT/DDM support as well as the old CUT mode transfer support.
However I've run into a bit of an issue running under MVS3.8J with WSF
data streams using buffers larger than about 2K in size. With a 4K
buffer and WSF style data streams sent/received via TPUT/TGET, the
TPUT calls seem to work as expected, however the TGET with a 4K buffer
fails and causes a terminal disconnect (not nice). So I may be forced
to limit the buffer size to 2K which is far less than optimal for
IND$FILE transfers.
From a system maintenance side, I have ZP60008 and ZP60009 applied
using the most recent downloads as those have received a few
modification over the many years since they first appeared on the web
http://www.prycroft6.com.au/vs2mods/ .
My guess about the larger buffers causing disconnects would be an
issue with the internal buffers available in TSO or VTAM. If you have
some ideas on how we can use larger buffers for the WSF data streams
please feel free to make suggestions.
Thanks.
--Mike

[[e-mail] [H390-MVS]]

Post by ***@comcast.net [H390-MVS]
Guys, after many years of hanging in the background, I finally have
some time to work on IND$FILE again.

Good. I hope that the new code will check for RECFM=U before
checking for RECFM=F or RECFM=V since RECFM=U has both
the RECFM=F and RECFM=V bits on.

Post by ***@comcast.net [H390-MVS]
The new version will have DFT/DDM support as well as the old CUT mode transfer support.
However I've run into a bit of an issue running under MVS3.8J with
WSF data streams using buffers larger than about 2K in size.
With a 4K buffer and WSF style data streams sent/received via
TPUT/TGET, the TPUT calls seem to work as expected, however
the TGET with a 4K buffer fails and causes a terminal disconnect
(not nice). So I may be forced to limit the buffer size to 2K which
is far less than optimal for IND$FILE transfers.

How many bits are you using and expecting in your 3270 buffer
addresses?

Post by ***@comcast.net [H390-MVS]
From a system maintenance side, I have ZP60008 and ZP60009
applied using the most recent downloads as those have received
a few modification over the many years since they first appeared
on the web http://www.prycroft6.com.au/vs2mods/
My guess about the larger buffers causing disconnects would be
an issue with the internal buffers available in TSO or VTAM.
If you have some ideas on how we can use larger buffers for the
WSF data streams please feel free to make suggestions.
Thanks.
--Mike

I don't use WSF but plenty of software written by other people does
both use WSF and address screens larger than 1920.

[Mike Rayborn [e-mail] [H390-MVS]]

I'd like to here more about the issue/symptom regarding RECFM=U.

There may not be a lot I can do about it, but I'd like to understand the
symptoms in any case.

My IND$FILE is written C. The new version uses the PDPCLIB for
file/dataset IO and as such kind of does its own thing regarding record
formats. Generally this works out just fine and simplifies reading and
writing datasets.

Note that the old IND$FILE used the Dignus Systems/C cross platform
compiler and whatever tweaks I had made back then. I'm not placing any
blame upon Dignus as they have a fine product, in fact excellent
product. I'm only mentioning this because it was a different build
environment and the runtime code is very different.

The first version of IND$FILE that most of you have, is purely a hack.
The behavior was derived from GTF traces and a best guess on my part of
what was expected. I know that in some cases my code fell short of
expectations, and in some cases didn't work as expected or the same as
IBM's IND$FILE program.

For better or worse, I seem to be the guy for IND$FILE on MVS38J and I
have a pretty good idea of what can be done in that environment with
*modern* TN3270 emulators.

The new version of IND$FILE will support both CUT and DFT mode file
transfers (assuming I can solve the buffer issue with WSF data streams
and large (32K) buffers).

--Mike

Post by ***@comcast.net [H390-MVS]
Guys, after many years of hanging in the background, I finally have
some time to work on IND$FILE again.

Good. I hope that the new code will check for RECFM=U before
checking for RECFM=F or RECFM=V since RECFM=U has both
the RECFM=F and RECFM=V bits on.

Post by ***@comcast.net [H390-MVS]
The new version will have DFT/DDM support as well as the old CUT mode transfer support.
However I've run into a bit of an issue running under MVS3.8J with
WSF data streams using buffers larger than about 2K in size.
With a 4K buffer and WSF style data streams sent/received via
TPUT/TGET, the TPUT calls seem to work as expected, however
the TGET with a 4K buffer fails and causes a terminal disconnect
(not nice). So I may be forced to limit the buffer size to 2K which
is far less than optimal for IND$FILE transfers.

How many bits are you using and expecting in your 3270 buffer
addresses?

Post by ***@comcast.net [H390-MVS]
From a system maintenance side, I have ZP60008 and ZP60009
applied using the most recent downloads as those have received
a few modification over the many years since they first appeared
on the web http://www.prycroft6.com.au/vs2mods/
My guess about the larger buffers causing disconnects would be
an issue with the internal buffers available in TSO or VTAM.
If you have some ideas on how we can use larger buffers for the
WSF data streams please feel free to make suggestions.
Thanks.
--Mike

I don't use WSF but plenty of software written by other people does
both use WSF and address screens larger than 1920.

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
I'd like to here more about the issue/symptom regarding RECFM=U.

- - - remainder snipped - - -

I reported the RECFM=U issue years ago and was
waiting for a fix before using IND$FILE again.
I don't remember the details. But even my simple
following test shows that RECFM=U processing
has an issue.

Does your code understand that if the RECFM F
and V bits are both on, then it is neither but
instead is RECFM=U . That is why RECFM=U
must be checked for first.

I just installed what I already had available.
I don't have it installed normally so can switch to
whatever you like.

ind$file ?
READY
ind$file
Free File Transfer Program, version 1.1.1, Compiled: Jan 10 2006 08:39:46
Copyright(c) 2002-2006 Mike Rayborn, ***@comcast.net

Usage: IND$FILE {GET|PUT} 'dataset.name' options

If the dataset name is specified without quotes, then the TSO userid
will be prepended to the dataset as 'userid.dataset.name'.

options: ASCII CRLF APPEND CODEPAGE cdpg ALT DEBUG TRACE
ASCII The dataset will be converted to/from ASCII and EBCDIC.
CRLF The end of line will be represented by CR and LF characters.
APPEND Records will be appended to the dataset (PUT only).
CODEPAGE The cdpg name represents the name of an external code page
translation table which will be used for translating ASCII
characters to/from EBCDIC. The default cdpg is CDPG1047.
ALT Use Erase Write Alternate for screen buffer setup (Experimental).
DEBUG Sets the internal debug flag (deprecated).
TRACE Sends trace information to the dataset allocated to INDTRACE.
If the INDTRACE DD is not pre-allocated then a trace dataset will be
allocated dynamically using 'userid.INDTRACE' as the trace dataset.
READY

===> Warning, copy/paste didn't work for the error message so I manually typed or typoed:<===

File Transfer Status
Transfer successfully completed
0 bytes transferred to PC file
h:\MVS\IEFBR14.LMOD
from host file
'sys1.linklib(IEFBR14)'
TRANS03File transfer complete$

H:\MVS>dir iefb*
Directory of H:\MVS
08/20/2016 02:42 PM 0 IEFBR14.LMOD

Identical results for IEBCOPY

[Mike Rayborn [e-mail] [H390-MVS]]

Fair enough. Transferring binary datasets, like IEFBR14 *should* produce
a file of some size and not 0 bytes unless we actually failed to send
anything at all.

I'll add it to the testing when I get that far in the new code.

Thanks.

--Mike

Post by ***@yahoo.com [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
I'd like to here more about the issue/symptom regarding RECFM=U.

- - - remainder snipped - - -
I reported the RECFM=U issue years ago and was
waiting for a fix before using IND$FILE again.
I don't remember the details. But even my simple
following test shows that RECFM=U processing
has an issue.
Does your code understand that if the RECFM F
and V bits are both on, then it is neither but
instead is RECFM=U . That is why RECFM=U
must be checked for first.
I just installed what I already had available.
I don't have it installed normally so can switch to
whatever you like.
ind$file ?
READY
ind$file
Free File Transfer Program, version 1.1.1, Compiled: Jan 10 2006 08:39:46
Usage: IND$FILE {GET|PUT} 'dataset.name' options
If the dataset name is specified without quotes, then the TSO userid
will be prepended to the dataset as 'userid.dataset.name'.
options: ASCII CRLF APPEND CODEPAGE cdpg ALT DEBUG TRACE
ASCII The dataset will be converted to/from ASCII and EBCDIC.
CRLF The end of line will be represented by CR and LF characters.
APPEND Records will be appended to the dataset (PUT only).
CODEPAGE The cdpg name represents the name of an external code page
translation table which will be used for translating ASCII
characters to/from EBCDIC. The default cdpg is CDPG1047.
ALT Use Erase Write Alternate for screen buffer setup (Experimental).
DEBUG Sets the internal debug flag (deprecated).
TRACE Sends trace information to the dataset allocated to INDTRACE.
If the INDTRACE DD is not pre-allocated then a trace dataset will be
allocated dynamically using 'userid.INDTRACE' as the trace dataset.
READY
===> Warning, copy/paste didn't work for the error message so I
manually typed or typoed:<===
File Transfer Status
Transfer successfully completed
0 bytes transferred to PC file
h:\MVS\IEFBR14.LMOD
from host file
'sys1.linklib(IEFBR14)'
TRANS03File transfer complete$
H:\MVS>dir iefb*
Directory of H:\MVS
08/20/2016 02:42 PM 0 IEFBR14.LMOD
Identical results for IEBCOPY

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com [H390-MVS]
Does your code understand that if the RECFM F
and V bits are both on, then it is neither but
instead is RECFM=U . That is why RECFM=U
must be checked for first.

Note that I think that is more likely to be
a bug in the Dignus C runtime, not IND$FILE.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com [H390-MVS]
File Transfer Status
Transfer successfully completed
0 bytes transferred to PC file
h:\MVS\IEFBR14.LMOD
from host file
'sys1.linklib(IEFBR14)'
TRANS03File transfer complete$

You didn't show the command you
used to do the transfer. And the output
doesn't mention whether a binary
transfer is being done.

If you are doing a text transfer, then
a 0-byte output file may make sense.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com [H390-MVS]
File Transfer Status
Transfer successfully completed
0 bytes transferred to PC file
h:\MVS\IEFBR14.LMOD
from host file
'sys1.linklib(IEFBR14)'
TRANS03File transfer complete$

You didn't show the command you
used to do the transfer. And the output
doesn't mention whether a binary
transfer is being done.
If you are doing a text transfer, then
a 0-byte output file may make sense.
BFN. Paul.

The "command" was whatever the TN3270 menu generated.
i.e. IND$FILE GET 'sys1.linklib(IEFBR14)' NOTRUNC
The NOTRUNC of course makes no difference.
RECFM=U fails for IEBGENER also and every file that
I ever tried for years and years.

Even if I had done a text transfer for RECFM=U,
the zero bytes transfer still wouldn't make sense for
IEFBR14 and certainly not for IEBCOPY nor IEBGENER.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com [H390-MVS]
RECFM=U fails for IEBGENER also and every file that
I ever tried for years and years.

Ok.

Post by ***@yahoo.com [H390-MVS]
Even if I had done a text transfer for RECFM=U,
the zero bytes transfer still wouldn't make sense for
IEFBR14 and certainly not for IEBCOPY nor IEBGENER.

If a text-processing program encounters
a non-text character, such as x'00', then
all bets are off and empty files do not
really surprise me.

As an example, a text-processing program
could do a 1 MiB fread, stick a NUL
character at the end, and then start
writing up until it hits the NUL terminator.
If there's a NUL as the first character,
you can kiss your data goodbye. :-)

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com [H390-MVS]
RECFM=U fails for IEBGENER also and every file that
I ever tried for years and years.

Ok.

Post by ***@yahoo.com [H390-MVS]
Even if I had done a text transfer for RECFM=U,
the zero bytes transfer still wouldn't make sense for
IEFBR14 and certainly not for IEBCOPY nor IEBGENER.

If a text-processing program encounters
a non-text character, such as x'00', then
all bets are off and empty files do not
really surprise me.
As an example, a text-processing program
could do a 1 MiB fread, stick a NUL
character at the end, and then start
writing up until it hits the NUL terminator.

Null terminated memory buffers are supposed
to be null terminated memory buffers.

Post by ***@yahoo.com.au [H390-MVS]
If there's a NUL as the first character,
you can kiss your data goodbye. :-)
BFN. Paul.

If some or many bytes in IEBCOPY don't download
because IND$FILE mistaken believes that IEBCOPY
has some null terminated lines that should be dropped,
then other data would still down load because not all
"text" records start with nulls.

The tests included proper binary down loads anyway.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com [H390-MVS]
If some or many bytes in IEBCOPY don't download
because IND$FILE mistaken believes that IEBCOPY
has some null terminated lines that should be dropped,
then other data would still down load because not all
"text" records start with nulls.

In the example I gave, all it needs is
for the first text record to start with a
NUL, and the next 1 MiB of data will
be ignored. The multiple records in
the input file, if less than 1 MiB in
size total, will all be read via a single
fread, and all ignored.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com [H390-MVS]
If some or many bytes in IEBCOPY don't download
because IND$FILE mistaken believes that IEBCOPY
has some null terminated lines that should be dropped,
then other data would still down load because not all
"text" records start with nulls.

In the example I gave, all it needs is
for the first text record to start with a
NUL, and the next 1 MiB of data will
be ignored. The multiple records in
the input file, if less than 1 MiB in
size total, will all be read via a single
fread, and all ignored.
BFN. Paul.

Main frames don't always go out of their way to destroy user data.
Main frames don't always read 1MB blocks of text data.

If IND$FILE is dropping RECFM FB, VB, and U text records
that begin with a null byte, then IND$FILE is broken.

If IND$FILE is dropping entire files because the first byte
could have started with a null, it is broken.
Even it IND$FILE was that brain dead, it wouldn't have
affected the binary down load tests.

Just like your code is broken because it arbitrarily drops
NL does not mean that all code tries to corrupt user data
without a specific request from the application.

P.S. No block in IEFBR14 starts with a null byte.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com [H390-MVS]
If IND$FILE is dropping RECFM FB, VB, and U text records
that begin with a null byte, then IND$FILE is broken.

I disagree. If you give binary rubbish to
a text-processing program, you shouldn't
expect anything useful at all. It's all
"implementation defined".

Post by ***@yahoo.com [H390-MVS]
Even it IND$FILE was that brain dead, it wouldn't have
affected the binary down load tests.

Sure. No disagreement there.

Post by ***@yahoo.com [H390-MVS]
Just like your code is broken because it arbitrarily drops
NL does not mean that all code tries to corrupt user data
without a specific request from the application.

Sorry, what code of mine drops NL?
When writing text data to RECFM=U
it is IBM who drop the NL, not me.
PDPCLIB preserves it.

BFN. Paul.

[Robert Prins [e-mail] [H390-MVS]]

RECFM=U fails for IEBGENER also and every file that I ever tried for years
and years.

Ok.

Even if I had done a text transfer for RECFM=U, the zero bytes transfer
still wouldn't make sense for IEFBR14 and certainly not for IEBCOPY nor
IEBGENER.

If a text-processing program encounters a non-text character, such as x'00',
then all bets are off and empty files do not really surprise me.

This should of course read: If a text-processing program written in a
abomination called "C"...

Robert

[Greg Price [e-mail] [H390-MVS]]

Post by Robert Prins ***@gmail.com [H390-MVS]
This should of course read: If a text-processing program written in a
abomination called "C"...

Well, I might agree with you, Rob, but we might be in the minority these
days...

:/

Cheers,
Greg

[[e-mail] [H390-MVS]]

Hi Greg & Rob


Well, I second... and I'm sure we are not the only ones. Although fluency in C is basically a requirement today, one nonetheless doesn't have to like it .


Cheers
JÃŒrgen

Post by Robert Prins ***@gmail.com [H390-MVS]
This should of course read: If a text-processing program written in a
abomination called "C"...

Well, I might agree with you, Rob, but we might be in the minority these
days...

:/

Cheers,
Greg

[Kevin Monceaux [e-mail] [H390-MVS]]

Post by ***@id.ethz.ch [H390-MVS]
Well, I second... and I'm sure we are not the only ones. Although fluency
in C is basically a requirement today, one nonetheless doesn't have to
like it .

Over the years I've tried to develop a fondness for C. It hasn't happened
yet. I'm no where near fluent in it, which probably doesn't help.

[[e-mail] [H390-MVS]]

Post by ***@id.ethz.ch [H390-MVS]
Well, I second... and I'm sure we are not the only ones. Although fluency
in C is basically a requirement today, one nonetheless doesn't have to
like it .

Over the years I've tried to develop a fondness for C. It hasn't happened
yet. I'm no where near fluent in it, which probably doesn't help.

It is certainly interesting that there is
a love/hate relationship with C, instead
of some sort of bell curve.

Regardless, for those of you being
discouraged from using C on MVS,
here is what Louis (an experienced
mainframe assembler programmer)
had to say about C after I had tutored
him for a few months via email (a
lot of it was "in assembler I can do
this, how can I do it in C?":


C language has a compiler very efficient
and is pleasant to use and more readable.
I definitely decided to use C language.

And about IBM C (METAL) in particular:

My feeling is that the generated code is
as efficient as a good assembly programmer.


If anyone would like help in learning C,
I am happy to answer any email questions,
doesn't matter how basic. My email
address is
mutazilah^^^at^^^gmail^^^dot^^^com

Don't forget that after you've made the
effort to learn C on MVS etc, you
suddenly gain the ability to write
utilities on Windows/Unix too. It's the
*exact same language*.

BFN. Paul.

[Kevin Monceaux [e-mail] [H390-MVS]]

It is certainly interesting that there is a love/hate relationship with C,
instead of some sort of bell curve.

I'm probably somewhere in the middle. I don't actually hate C. I do like
C's general syntax. But being a novice programmer, at best, I find C's lack
of string handling, pointers, and dynamic memory management a challenge at
times.

Most of my programming projects are either web site related, simple command
line apps and/or ncurses apps. I'd love to find one language that works
well for all of the above. Web development seems to be especially
challenging in that area. I often find a web framework I like, but don't
like the language the web framework uses. All the sites I manage are
currently Django based. I like Django well enough, but I like Python even
less than C. My personal web site has been reimplemented in several
frameworks over the years. I'm not sure if I can remember them all. I
think the evolution went something like:

The original site was done in plain PHP with no web framework

Catalyst (Perl)

PSP (Pascal Server Pages - Pascal)

Rails (Ruby)

Drupal 6 (PHP)

Django (Python)

At one point I implemented another web site I manage in ASP.NET (C#), before
there was an ASP.NET MVC, running under Mono on FreeBSD. Of the above
languages I think I probably like C# the best, with Perl and PHP a close
second for their similarities to C syntax. Drupal 8 has come a long way
since Drupal 6. I might be moving back to Drupal and PHP, or if not maybe
Symfony and PHP. ASP.NET MVC might be another possibility now that they're
making it more command line and Linux friendly.

Probably my most adventurous C project to date started out in Perl. I'm a
TV and movie fan. I love all the information available on the IMDb, but
they keep cluttering up their web interface with a bunch of unnecessary
stuff. I wrote a Perl program that web scrapes info from the IMDb and
displays it as plain text. I intended to eventually add an ncurses
interface to it. But its HTML parsing was much slower than it should have
been. I started thinking that it would probably be better to rewrite it in
a compiled language, so I took a stab at rewriting it in C. I have the
basic IMDb search listing and TV episode listing ncurses interfaces done. I
have part of the IMDb title information parsing done, but not enough to
display the full title details. At the moment it falls back to using the
Perl program to display the title details, reviews, trivia, etc.. I
eventually discovered the bottleneck with the Perl program was using
HTML::TreeBuilder::XPath to parse the HTML from the IMDb. I switched to
XML::LibXML and its speed is much improved.

To bring this back slightly on topic, if MVS 3.8J had a network stack, and a
library that could parse HTML via xpath, I'd love to implement my IMDb app
as a KICKS app. I know, I'm odd. I'm an amateur web developer who doesn't
particularly like web interfaces. Actually, I don't like any interface that
tries to force me reach for a mouse. That has a lot to do with me disliking
windows.

['Jason Froebe' [e-mail] [H390-MVS]]

I must call out: VIVA LA COMMAND LINE!

;-)



Sorry. I had to.



jason



From: H390-***@yahoogroups.com [mailto:H390-***@yahoogroups.com]
Sent: Tuesday, August 23, 2016 11:07 AM
To: Hercules MVS Mailing List <H390-***@YahooGroups.com>
Subject: [H390-MVS] Re: OT: C, and other Languages (Was: MVS38J IND$FILE
DFT/DDM and WSF)

It is certainly interesting that there is a love/hate relationship with C,
instead of some sort of bell curve.

I'm probably somewhere in the middle. I don't actually hate C. I do like
C's general syntax. But being a novice programmer, at best, I find C's lack
of string handling, pointers, and dynamic memory management a challenge at
times.

Most of my programming projects are either web site related, simple command
line apps and/or ncurses apps. I'd love to find one language that works
well for all of the above. Web development seems to be especially
challenging in that area. I often find a web framework I like, but don't
like the language the web framework uses. All the sites I manage are
currently Django based. I like Django well enough, but I like Python even
less than C. My personal web site has been reimplemented in several
frameworks over the years. I'm not sure if I can remember them all. I
think the evolution went something like:

The original site was done in plain PHP with no web framework

Catalyst (Perl)

PSP (Pascal Server Pages - Pascal)

Rails (Ruby)

Drupal 6 (PHP)

Django (Python)

At one point I implemented another web site I manage in ASP.NET (C#), before
there was an ASP.NET MVC, running under Mono on FreeBSD. Of the above
languages I think I probably like C# the best, with Perl and PHP a close
second for their similarities to C syntax. Drupal 8 has come a long way
since Drupal 6. I might be moving back to Drupal and PHP, or if not maybe
Symfony and PHP. ASP.NET MVC might be another possibility now that they're
making it more command line and Linux friendly.

Probably my most adventurous C project to date started out in Perl. I'm a
TV and movie fan. I love all the information available on the IMDb, but
they keep cluttering up their web interface with a bunch of unnecessary
stuff. I wrote a Perl program that web scrapes info from the IMDb and
displays it as plain text. I intended to eventually add an ncurses
interface to it. But its HTML parsing was much slower than it should have
been. I started thinking that it would probably be better to rewrite it in
a compiled language, so I took a stab at rewriting it in C. I have the
basic IMDb search listing and TV episode listing ncurses interfaces done. I
have part of the IMDb title information parsing done, but not enough to
display the full title details. At the moment it falls back to using the
Perl program to display the title details, reviews, trivia, etc.. I
eventually discovered the bottleneck with the Perl program was using
HTML::TreeBuilder::XPath to parse the HTML from the IMDb. I switched to
XML::LibXML and its speed is much improved.

To bring this back slightly on topic, if MVS 3.8J had a network stack, and a
library that could parse HTML via xpath, I'd love to implement my IMDb app
as a KICKS app. I know, I'm odd. I'm an amateur web developer who doesn't
particularly like web interfaces. Actually, I don't like any interface that
tries to force me reach for a mouse. That has a lot to do with me disliking
windows.

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

Post by 'Jason Froebe' ***@gmail.com [H390-MVS]
I must call out: VIVA LA COMMAND LINE!
;-)
Sorry. I had to.

As a long time mainframer who grew up with DOS/VS(E) and VM/SP and fell in love with Rexx, ...

*FUCK* the command line! >;-)

The first day I saw a Macintosh after having worked with DOS on a PC I moved away from the command line so fast I probably broke the sound barrier. :)

Perhaps it's the inner artist in me but I personally find using a well designed and well written GUI program to be *FAR* superior to using some poorly name command line utility with options I can never remember and whose syntax is often very unforgiving. To me a picture is worth a thousand words.

And given its popularity among personal computer operating systems (Linux included) I'm apparently not along regarding that opinion either.

Now, all that being said I'm certainly not *averse* to using the command line mind you. It certainly has its place and I do find myself using it quite often for certain things, so I'm not suggesting it should be abandoned. I'm simply saying, FOR ME, (and apparently most others as well), accomplishing something with a few mouse clicks is faster and easier (and for me certainly more fun!) than using the command line.

(And I hate the white text on a black background too! And no, before you ask, switching it to white background with black text isn't any better. The only thing that's better is well designed, functional dialog with buttons and whatnot.)

[Ivan Warren [e-mail] [H390-MVS]]

Post by '\'Fish\' (David B. Trout)' ***@gmail.com [H390-MVS]

Post by 'Jason Froebe' ***@gmail.com [H390-MVS]
I must call out: VIVA LA COMMAND LINE!
;-)
Sorry. I had to.

As a long time mainframer who grew up with DOS/VS(E) and VM/SP and fell in love with Rexx, ...
*FUCK* the command line! >;-)

Dear Fish,

I disagree :P (you know I would).

I have absolutely NOTHING against well conceived GUIs and assisted entry
(aka wizards)....

As long as there IS a command line alternative.

That's why I always liked AIX's "SMIT" : You can always see WHAT command
is used and it will even log all the command lines used to achieve such
and such task (even making it scriptable).

I become worried when anything you can do in a GUI cannot also be done
by just typing a line in a B/W terminal or script it (or at least be
described in a documented API).

--Ivan

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
Dear Fish,
I disagree :P (you know I would).

:)

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
I have absolutely NOTHING against well conceived GUIs
and assisted entry (aka wizards)....
As long as there IS a command line alternative.

Agreed. For scripting purposes as well as for user preference there should always be a non-GUI (command line) way to do the same thing.

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
That's why I always liked AIX's "SMIT" : You can always
see WHAT command is used and it will even log all the
command lines used to achieve such and such task (even
making it scriptable).

Sounds nice!

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
I become worried when anything you can do in a GUI cannot also
be done by just typing a line in a B/W terminal or script it
(or at least be described in a documented API).

Agreed. You've seen the batch scripts I've written for Windows. Using the command line absolutely does have its place and for many tasks absolutely *is* the best way to get certain things done.

But for simple day to day stuff? Nah. Give me a good GUI. :)

[[e-mail] [H390-MVS]]

GUI's are nice toys, but let's see someone install a change reconfiguring 200 file definitions and a handful of other resources in 2000 CICS regions, do it perfectly, AND validate that it all went in correctly -- within a 10 minute change window -- AND have enough time left over to back it all out (also perfectly with no mistakes) in the event that something goes wrong (such as the group who asked for the change forgetting that they also needed to have some security authorizations done) -- using some GUI.

Ain't gonna happen.

GUI's in such a production scenario are worthless. Scripted batch rules.

Rob

['Dave Wade' [e-mail] [H390-MVS]]

Which is why Microsoft now insist that all new product can be configured via PowerShell scripts.
Yes there is a GUI for some products, but in general PowerShell rules.

Dave
G4UGM

From: H390-***@yahoogroups.com [mailto:H390-***@yahoogroups.com]
Sent: 23 August 2016 22:26
To: H390-***@yahoogroups.com
Subject: RE: [H390-MVS] Re: OT: C, and other Languages (Was: MVS38J IND$FILE DFT/DDM and WSF)





GUI's are nice toys, but let's see someone install a change reconfiguring 200 file definitions and a handful of other resources in 2000 CICS regions, do it perfectly, AND validate that it all went in correctly -- within a 10 minute change window -- AND have enough time left over to back it all out (also perfectly with no mistakes) in the event that something goes wrong (such as the group who asked for the change forgetting that they also needed to have some security authorizations done) -- using some GUI.

Ain't gonna happen.

GUI's in such a production scenario are worthless. Scripted batch rules.

Rob

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

Post by 'Dave Wade' ***@gmail.com [H390-MVS]
Which is why Microsoft now insist that all new product can be
configured via PowerShell scripts.
Yes there is a GUI for some products, but in general PowerShell rules.

I never bothered to learn PowerShell, just as I've never bothered to learn C# either. I'm sure they're both nice just as I'm sure Python or other high level scripting languages are.

I just wish everyone would settle on ONE so I wouldn't have to learn 47 different ones, you know? Why can't everyone agree to use Rexx as their shell/scripting language? THAT'S a nice language!

(Even as skilled as I am at writing Windows batch scripts, the Windows batch language has to be the most arcane, weird, fucked up and hard to use batch scripting languages in existence! I swear! I hate it.)

[Ivan Warren [e-mail] [H390-MVS]]

Post by '\'Fish\' (David B. Trout)' ***@gmail.com [H390-MVS]
(Even as skilled as I am at writing Windows batch scripts, the Windows batch language has to be the most arcane, weird, fucked up and hard to use batch scripting languages in existence! I swear! I hate it.)

If you want some really arcane, fucked up language try the brainfuck
(and it's even Turing complete).

https://en.wikipedia.org/wiki/Brainfuck

--Ivan

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

[...]

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
https://en.wikipedia.org/wiki/Brainfuck

What about Whitespace?

https://en.wikipedia.org/wiki/Whitespace_(programming_language)

[Ivan Warren [e-mail] [H390-MVS]]

Post by '\'Fish\' (David B. Trout)' ***@gmail.com [H390-MVS]

Post by 'Dave Wade' ***@gmail.com [H390-MVS]
Which is why Microsoft now insist that all new product can be
configured via PowerShell scripts.
Yes there is a GUI for some products, but in general PowerShell rules.

I never bothered to learn PowerShell, just as I've never bothered to learn C# either. I'm sure they're both nice just as I'm sure Python or other high level scripting languages are.
I just wish everyone would settle on ONE so I wouldn't have to learn 47 different ones, you know? Why can't everyone agree to use Rexx as their shell/scripting language? THAT'S a nice language!
(Even as skilled as I am at writing Windows batch scripts, the Windows batch language has to be the most arcane, weird, fucked up and hard to use batch scripting languages in existence! I swear! I hate it.)

47 ? Think more like 1500 !

http://www.99-bottles-of-beer.net/

--Ivan

['Dave Wade' [e-mail] [H390-MVS]]

-----Original Message-----
Sent: 24 August 2016 03:40
Subject: RE: [H390-MVS] Re: OT: C, and other Languages (Was: MVS38J
IND$FILE DFT/DDM and WSF)

Post by 'Dave Wade' ***@gmail.com [H390-MVS]
Which is why Microsoft now insist that all new product can be
configured via PowerShell scripts.
Yes there is a GUI for some products, but in general PowerShell rules.

I never bothered to learn PowerShell, just as I've never bothered to learn C#
either. I'm sure they're both nice just as I'm sure Python or other high level
scripting languages are.

I suspect you could produce a Python interface to the .NET objects that Powershell uses.

I just wish everyone would settle on ONE so I wouldn't have to learn 47
different ones, you know? Why can't everyone agree to use Rexx as their
shell/scripting language? THAT'S a nice language!

Well again Objective Rexx should be able to interface to the same object library ....

(Even as skilled as I am at writing Windows batch scripts, the Windows batch
language has to be the most arcane, weird, fucked up and hard to use batch
scripting languages in existence! I swear! I hate it.)

Well everything post XP has Powershell and its been open sourced to Unix.

--
"Fish" (David B. Trout)
Software Development Laboratories
http://www.softdevlabs.com

Dave
G4UGM

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

Rob wrote:

[...]

Post by ***@troop57garland.org [H390-MVS]
GUI's in such a production scenario are worthless.
Scripted batch rules.

Which of course is why I said "It has its place". In a scenario such as you described the command line is absolutely the correct tool to use. It would be nuts to use a GUI to do that.

But for everyday stuff? Nah. Give me a GUI. :)

[Mike Stramba [e-mail] [H390-MVS]]

Post by ***@troop57garland.org [H390-MVS]
GUI's are nice toys, but let's see someone install a change reconfiguring
200 file definitions and a handful of other resources in 2000 CICS regions,
do it perfectly, AND validate that it all went in correctly -- within a 10
minute change window -- AND have enough time left over to back it all out
(also perfectly with no mistakes) in the event that something goes wrong
(such as the group who asked for the change forgetting that they also needed
to have some security authorizations done) -- using some GUI.
Ain't gonna happen.
GUI's in such a production scenario are worthless. Scripted batch rules.
Rob

But what did you use to WRITE that batch file ? ;)

Probably the ISPF (or similar) FULL SCREEN editor, right ? ;)

Mike

[Ivan Warren [e-mail] [H390-MVS]]

Post by Mike Stramba ***@gmail.com [H390-MVS]
But what did you use to WRITE that batch file ? ;)
Probably the ISPF (or similar) FULL SCREEN editor, right ? ;)
Mike

Fullscreen editors are for sissies !

We real system programmers poke bits into real memory to write a machine
language program to change disk records directly !

(That's when we don't take the HDA appart to write the file using a very
fine magnetized needle and change the bits directly on the platter)

--Ivan

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
Fullscreen editors are for sissies !
We real system programmers poke bits into real memory to write a machine
language program to change disk records directly !
(That's when we don't take the HDA appart to write the file using a very
fine magnetized needle and change the bits directly on the platter)
--Ivan

ROTFLMAO! :)))

[Denis Molony [e-mail] [H390-MVS]]

I think this is the reference - https://xkcd.com/378/ <https://xkcd.com/378/>

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
Fullscreen editors are for sissies !
We real system programmers poke bits into real memory to write a machine
language program to change disk records directly !
(That's when we don't take the HDA appart to write the file using a very
fine magnetized needle and change the bits directly on the platter)
--Ivan

ROTFLMAO! :)))
--
"Fish" (David B. Trout)
Software Development Laboratories
http://www.softdevlabs.com <http://www.softdevlabs.com/>

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

Post by '\'Fish\' (David B. Trout)' ***@gmail.com [H390-MVS]

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
Fullscreen editors are for sissies !
We real system programmers poke bits into real memory to write a machine
language program to change disk records directly !
(That's when we don't take the HDA appart to write the file using a very
fine magnetized needle and change the bits directly on the platter)
--Ivan

ROTFLMAO! :)))

Reminds me of the Monty Python "Four Yorkshiremen" sketch:

['Bill Turner, WB4ALM' [e-mail] [H390-MVS]]

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]

Post by Mike Stramba ***@gmail.com [H390-MVS]
But what did you use to WRITE that batch file ? ;)
Probably the ISPF (or similar) FULL SCREEN editor, right ? ;)
Mike

Fullscreen editors are for sissies !
We real system programmers poke bits into real memory to write a
machine language program to change disk records directly !

*Using the switches on the front console...
/s/ Bill, wb4alm
*

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
(That's when we don't take the HDA appart to write the file using a
very fine magnetized needle and change the bits directly on the platter)
--Ivan

[Gerhard Postpischil [e-mail] [H390-MVS]]

Post by 'Bill Turner, WB4ALM' ***@arrl.net [H390-MVS]

Post by Ivan Warren ***@vmfacility.fr [H390-MVS]
We real system programmers poke bits into real memory to write a
machine language program to change disk records directly !

*Using the switches on the front console...

Don't laugh. Back in the late sixties, our management had a 360/40
running DOS, but soon found that most service bureau customers wanted
MVT. After some travails, including frequent Rollout/Rollin overhead,
they decided to spring for a 365/50. This was a wonderful upgrade,
except that several times a day it would crash on a machine check. The
C.E. spent a lot of time running diagnostics, but was unable to find a
problem. After a lot of work poring over console logs, etc., I found
out that the crash occurred whenever there was a Ring-bell CCW issued to
the console (my BFF and colleague had decided that it was necessary to
wake the operators when they didn't respond to a WTOR or mount request
before the wait time limit was reached [minor IEFUTL change]). The C.E.
didn't believe me, and blamed the system. I had him clear memory, and
used the front panel switches to enter a CAW, CCW, and simple SIO/TIO
loop, and had him hit Start. Voila, instant machine check. He had the
bell repaired within the hour; but we both kept wondering why IBM did
not check the bell in the diagnostics.


Gerhard Postpischil
Bradford, VT

[[e-mail] [H390-MVS]]

I believe a system I was on did the same thing, a 360/50 but not running MVT. It was running some kind of DOS clone. Maybe someone out there remembers this, I believe it allowed DOS to run 6 partitions instead of 3. Anyway, I recall that someone (me?) wrote an assembler program to issue the console alarm CCW and boom, the system crashed. I don't believe the CE was ever called in to fix it - they just banned the use of the console alarm CCW and gave us programmers a really dirty look.

[Gary [e-mail] [H390-MVS]]

I was an operator at a shop that had a 360/65 and it was running
Software Pursuit's DOS/MVT. Could this be what you were thinking of?

Unlike the IBM DOS/VS which had fixed partitions, it had variable
partitions that could be resized on the fly. That generally required
deactivating 1 or more partitions to create a larger partition for a
weekend or month-end job.

Gary

Post by ***@yahoo.com [H390-MVS]
I believe a system I was on did the same thing, a 360/50 but not
running MVT. It was running some kind of DOS clone. Maybe someone
out there remembers this, I believe it allowed DOS to run 6 partitions
instead of 3. Anyway, I recall that someone (me?) wrote an assembler
program to issue the console alarm CCW and boom, the system crashed. I
don't believe the CE was ever called in to fix it - they just banned
the use of the console alarm CCW and gave us programmers a really
dirty look.

[[e-mail] [H390-MVS]]

Me too! I was the sysprog for BP in Anchorage, and we ran EDOS (6 partitions) on a 360/40. I wrote a 'setup' program to alert the ops to tape mounts, and it rang the bell after listing all the tapes they were gonna need. Worked great until we ugraded to a 50, then machine check time! CE told me ringing the bell was reserved for IBM utilities & I had to take it out!!!

[[e-mail] [H390-MVS]]

I forgot the name until you mentioned it (wow, was it really 40 or 45 years ago?!) but yes, it was EDOS. That was in the days when DOS source was free. As I recall, EDOS did everything it set out to do, and worked pretty well. I don't recall ever having problems specifically caused by EDOS.

['\'Fish\' (David B. Trout)' [e-mail] [H390-MVS]]

I forgot the name until you mentioned it [...]
but yes, it was EDOS.

[...]

That was in the days when DOS source was free.

<snip>

Yep. The company I worked at 30 yrs. ago was originally running EDOS when I arrived but then switched to DOS/VS(E) release 35 shortly thereafter.

DOS/VS(E) r35 was the last free version of DOS that came with source and was the version I cut my system programmer teeth on, allowing me to grow from being a wet behind the ears system programmer to skilled systems internal guru.

I VERY MUCH wish I could find it. I've been looking for it for *years*.

Does anyone out there have a copy or know where a copy might be obtained?

[[e-mail] [H390-MVS]]

I started using C in the mid '80s after getting tired of learning a new assembler every time I ran across a new PC. Sadly I found I still had to learn the new assembler(s) eventually, but it did give me something of a jump start. Then I discovered GCCMVS (or is it MVSGCC?). First one I tried *after* knowing the assembler. I think it is quite a usable tool and certainly faster to implement a large project than (entirely) in assembler. Paul deserves much credit for his development and advocacy. Even for MVS380 (talk about love/hate!).
I took a statistics course a few years ago & one thing I learned is you need to be a little careful generalizing observations of a small sample. I speculate there is a 'bell curve' for C preference; but people in the middle don't care enough to comment one way or the other, thus skewing the results towards the extremes.
But put me on the 'like it' side!

[[e-mail] [H390-MVS]]

Post by ***@gmail.com [H390-MVS]
I started using C in the mid '80s after
getting tired of learning a new assembler
every time I ran across a new PC. Sadly
I found I still had to learn the new
assembler(s) eventually, but it did give
me something of a jump start.

Agreed about the jump start being
valuable.

On a more modern PC I don't think
it is necessary to learn assembler
at all. Hercules is written entirely
in C as far as I am aware.

Post by ***@gmail.com [H390-MVS]
Then I discovered GCCMVS (or
is it MVSGCC?).

I named it GCCMVS and have always
used that name. For some reason,
from memory, on one of the TK distributions,
maybe TK3SU1, someone renamed
GCCMVS to MVSGCC for no purpose
I know of.

Post by ***@gmail.com [H390-MVS]
First one I tried *after* knowing the assembler.

Ok, interesting.

Post by ***@gmail.com [H390-MVS]
I think it is quite a usable tool and certainly
faster to implement a large project than
(entirely) in assembler.

Exactly. I personally am probably about
30 times more productive in C than
assembler.

And thanks to the enormous effort
that Gerhard put into MVSSUPA,
which every single C program picks
up without any effort at all from the
C programmer, I would argue that
the ANY/ANY modules that GCCMVS
produces are far superior to most,
and probably ALL, pure assembler
programs.

Post by ***@gmail.com [H390-MVS]
Paul deserves much credit for his development

Thanks. And it wasn't a sole effort, so
thanks to everyone else, including you,
who helped in development.

Post by ***@gmail.com [H390-MVS]
and advocacy.

That's the first time anyone has praised
me for promoting C on MVS! :-)

Post by ***@gmail.com [H390-MVS]
Even for MVS380 (talk about love/hate!).

Yes. :-)

Post by ***@gmail.com [H390-MVS]
I took a statistics course a few years ago &
one thing I learned is you need to be a
little careful generalizing observations of
a small sample. I speculate there is a
'bell curve' for C preference; but people
in the middle don't care enough to comment
one way or the other, thus skewing the
results towards the extremes.

Thanks. I never thought of that concept.

I wondered (just now) if that is also
responsible for the 2003 Iraq war
statistics of people being divided
50/50 on whether it is right or
wrong to use force. But I have made
a very active effort to investigate
average people to find out where
they stand, and pretty much
everyone has an opinion one side
or the other, I don't meet people
in the middle who don't care
either way.

Post by ***@gmail.com [H390-MVS]
But put me on the 'like it' side!

Cool!

BFN. Paul.

[[e-mail] [H390-MVS]]

There is also the matter of how you ask the question. If you ask whether someone is for or against something you will always get a binary answer! In order to observe a broader spectrum (to see a possible bell curve) you have to ask how much for or how much against. Or perhaps ask for approval on a 1 to 10 scale, etc.

[[e-mail] [H390-MVS]]

Post by ***@gmail.com [H390-MVS]
There is also the matter of how you ask
the question. If you ask whether someone
is for or against something you will always
get a binary answer! In order to observe
a broader spectrum (to see a possible
bell curve) you have to ask how much for
or how much against. Or perhaps ask for
approval on a 1 to 10 scale, etc.

Thankyou again for giving me yet another
concept. Sometimes I fail to see what is
bleeding obvious in hindsight.

BFN. Paul.

[Mike Rayborn [e-mail] [H390-MVS]]

Back on subject.

IND$FILE is currently in ALPHA testing this past week. A few minor
issues have been corrected and a new build created for our alpha
testers. So we're now in the ALPHA/BETA stage of testing with fingers
crossed.

Barring any major issues I'll be releasing IND$FILE into the wild later
this week.

What's new?
The new release will support both CUT and DFT mode file transfers. If
your terminal emulator supports DFT/DDM/WSF style transfers then this
*should* be a lot faster. You'll need to be on TK4- *or* have the
appropriate modes for VTAM and TSO as well as changes to the VTAM buffer
assignments. For me it was easier to just switch to TK4- and be done
with it, YMMV.

Several issues have been corrected regarding binary downloads of RECFM=U
PDS members (think load modules).

Question for you guys, is Yahoo the preferred place for hosting IND$FILE
as a download? or would some other wed site/service be better?

--Mike

[[e-mail] [H390-MVS]]

- - - In H390-***@yahoogroups.com, <***@...> wrote:
0 0 0 beginning snipped - - -

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Several issues have been corrected regarding binary downloads of
RECFM=U PDS members (think load modules).

The issue was not only load modules. That was just a quick example.
RECFM=U was broken for everything that I tried.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Question for you guys, is Yahoo the preferred place for hosting IND$FILE
as a download? or would some other wed site/service be better?
--Mike

Yahoo group files area is better than scattered all over the web.
Later, CBT might be appropriate?

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Several issues have been corrected
regarding binary downloads of RECFM=U
PDS members (think load modules).

Note that zip files are most appropriately
stored as RECFM=U, whether sequential
or PDS. Transferring a load module is
not something is useful because it uses
metadata which cannot be lost. It would
be better to make sure a zip file can be
transferred from PC to mainframe as
RECFM=U and opened by MINIUNZ,
preferably the SEASIK version of
MINIUNZ, which I assume is included
as part of TK4- (ie check HLQ of MINIZIP).

If you are using PDPCLIB (GCCMVS),
I am shocked that there was anything
in your code that needed to be "fixed"
to support RECFM=U. All you need to
do is open normally with "rb", and
write normally with "wb". Could you
please explain what the bug was and
what you needed to do to fix it?

Thanks. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Several issues have been corrected
regarding binary downloads of RECFM=U
PDS members (think load modules).

Note that zip files are most appropriately
stored as RECFM=U, whether sequential
or PDS.

- - - ending snipped - - -

zip files are just a byte stream.
They can be stored in RECFM=U, RECFM=VB,
or BSAM-RECFM=FBx1
RECFM=FB can be used because unzip programs
will read before the padding but why corrupt data
when there is no reason to.

Warning: there is at least one piece of braindead
software that treats RECFM=VB data incorrectly.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Several issues have been corrected
regarding binary downloads of RECFM=U
PDS members (think load modules).

Note that zip files are most appropriately
stored as RECFM=U, whether sequential
or PDS.

zip files are just a byte stream.
They can be stored in RECFM=U, RECFM=VB,
or BSAM-RECFM=FBx1
RECFM=FB can be used because unzip programs
will read before the padding but why corrupt data
when there is no reason to.

No, zip files can't be stored in VB datasets
unless you are willing to take away
the ability to store binary SMF files in VB
datasets.

At least under any sensible paradigm.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
No, zip files can't be stored in VB datasets
unless you are willing to take away
the ability to store binary SMF files in VB
datasets.

Specifically, what do you expect IND$FILE
to do when transferring an SMF files (VB)
in binary mode from mainframe to PC?

I expect it to be sensible. How about you?

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
Specifically, what do you expect IND$FILE
to do when transferring an SMF file (VB)
in binary mode from mainframe to PC?
I expect it to be sensible. How about you?

And in particular, when that file is
transferred back to the mainframe,
ie going from PC to mainframe, I
expect IND$FILE to preserve the
original SMF dataset.

This is as basic as expecting that
3 + 2 - 2 = 3, and any other concept
is extremely weird.

What do you expect?

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]
Specifically, what do you expect IND$FILE
to do when transferring an SMF file (VB)
in binary mode from mainframe to PC?
I expect it to be sensible. How about you?

And in particular, when that file is
transferred back to the mainframe,
ie going from PC to mainframe, I
expect IND$FILE to preserve the
original SMF dataset.
This is as basic as expecting that
3 + 2 - 2 = 3, and any other concept
is extremely weird.
What do you expect?
BFN. Paul.

When record or block boundaries need to
be preserved when a PC is used for
intermediate storage, the data is copied
to some format that preserves the record
or block boundary.

Some methods include
.aws tape, TSO XMIT, BDT, IEHMOVE

But are just trying to reverse the subject.

*up loaded* PC zip files can be put in
RECFM=VB, BSAM-FBx1, or U format
with no harm except for your program or
where you went out of the way to mess
with the data.

['Bill Turner, WB4ALM' [e-mail] [H390-MVS]]

IND$FILE --MUST-- transmit any file "bit for bit" unchanged when
transmitting in binary mode.
/s/ Bill Turner, wb4alm

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]
No, zip files can't be stored in VB datasets
unless you are willing to take away
the ability to store binary SMF files in VB
datasets.

Specifically, what do you expect IND$FILE
to do when transferring an SMF files (VB)
in binary mode from mainframe to PC?
I expect it to be sensible. How about you?
BFN. Paul.

[[e-mail] [H390-MVS]]

Can you please elaborate on that?

If you have an SMF file that contains

4-byte RDW, value 7, with data "AAA"

and

4-byte RDW, value 8, with data "BBBB"

what do you expect to see on the PC file?

And what do you expect to see when the
PC file is transferred back to the mainframe?

Thanks. Paul.




---In H390-***@yahoogroups.com, <***@...> wrote :

IND$FILE --MUST-- transmit any file "bit for bit" unchanged when transmitting in binary mode.
/s/ Bill Turner, wb4alm

Post by ***@yahoo.com.au [H390-MVS]
No, zip files can't be stored in VB datasets
unless you are willing to take away
the ability to store binary SMF files in VB
datasets.

Specifically, what do you expect IND$FILE
to do when transferring an SMF files (VB)
in binary mode from mainframe to PC?

I expect it to be sensible. How about you?

BFN. Paul.

['Bill Turner, WB4ALM' [e-mail] [H390-MVS]]

I would expect to see '0000 0007 C1C1 0000 0010 C2C2C2C2'x

on the PC and also back at the mainframe.

the character values would still be in EBCDIC, binary says no translation.
the bit sequences would still be in the same order as on the mainframe.

/s/ Bill Turner, wb4alm

Post by ***@yahoo.com.au [H390-MVS]
Can you please elaborate on that?
If you have an SMF file that contains
4-byte RDW, value 7, with data "AAA"
and
4-byte RDW, value 8, with data "BBBB"
what do you expect to see on the PC file?
And what do you expect to see when the
PC file is transferred back to the mainframe?

[[e-mail] [H390-MVS]]

Hi Bill.

Thanks for that.

Post by 'Bill Turner, WB4ALM' ***@arrl.net [H390-MVS]
I would expect to see '0000 0007 C1C1 0000 0010 C2C2C2C2'x

RDWs are:

xxyy (4 bytes), where xx = 2-byte length
in big endian format, while yy is always
0 for a VB file.

So did you mean to say:

0007 0000 C1C1 C1 0008 0000 C2C2C2C2

Note that I also added another "C1"
and I also changed the 0010 to 0008.

BFN. Paul.




---In H390-***@yahoogroups.com, <***@...> wrote :

I would expect to see '0000 0007 C1C1 0000 0010 C2C2C2C2'x

on the PC and also back at the mainframe.

the character values would still be in EBCDIC, binary says no translation.
the bit sequences would still be in the same order as on the mainframe.

/s/ Bill Turner, wb4alm



On 08/29/2016 07:34 AM, ***@... mailto:***@... [H390-MVS] wrote:

Can you please elaborate on that?

If you have an SMF file that contains

4-byte RDW, value 7, with data "AAA"

and

4-byte RDW, value 8, with data "BBBB"

what do you expect to see on the PC file?

And what do you expect to see when the
PC file is transferred back to the mainframe?

[[e-mail] [H390-MVS]]

Post by 'Bill Turner, WB4ALM' ***@arrl.net [H390-MVS]
I would expect to see '0000 0007 C1C1 0000 0010 C2C2C2C2'x

You counted wrong. RDW plus X'C1C1' is not seven.

If FTP specifies RDW, then it is treated as data but the
default is to not consider BDW, SDW, or RDW as data.

IBM IND$FILE will ask no questions and only transfer
data down. That is no BDW, no SDW, and no RDW

Post by 'Bill Turner, WB4ALM' ***@arrl.net [H390-MVS]
on the PC and also back at the mainframe.

Then you have messed up the data even more.
You just took two records of RECFM=VBS data that
can only be up loaded as RECFM=U/UB with no
record or block boundaries.

Your data is trash.

Post by 'Bill Turner, WB4ALM' ***@arrl.net [H390-MVS]
the character values would still be in EBCDIC, binary says
no translation.
the bit sequences would still be in the same order as on
the mainframe.
/s/ Bill Turner, wb4alm

- - - old notes snipped - - -

Now do you understand why RECFM=U, just like RECFM=VB,
PDSes, ISAM files, VSAM files main frame data is copied to
an unloaded format before using a PC for intermediate storage.

What does your strange main frame data being doubly trashed
have to do with UP, that is UP loading zip files, that is zip files?

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
Can you please elaborate on that?
If you have an SMF file that contains
4-byte RDW, value 7, with data "AAA"
and
4-byte RDW, value 8, with data "BBBB"
what do you expect to see on the PC file?

Why do you keep trying to push strange ways
to down load when up loaded zip files are being
discussed. Can we count the number of times
in the dozens now?
What is so fearful of discussion up loaded
zip files?

If you need record or block boundaries preserved
for RECFM=U or RECFM=V/VB/VBS then you
put the data in an unloaded format with .aws,
XMIT BDT, IEHMOVE or others that have been
used for years.

Post by ***@yahoo.com.au [H390-MVS]
And what do you expect to see when the
PC file is transferred back to the mainframe?
Thanks. Paul.

- - - old notes snipped - - -

With your methods, you will need a non-standard
utility to try to recover your data.

The rest of the world will use standard utilities
to restore what was in unloaded format.

[[e-mail] [H390-MVS]]

Post by 'Bill Turner, WB4ALM' ***@arrl.net [H390-MVS]
IND$FILE --MUST-- transmit any file "bit for bit" unchanged
when transmitting in binary mode.
/s/ Bill Turner, wb4alm

- - - ending snipped - - -

The question was never anything about the data.
It was converting SMF data from RECFM=VBS
to RECFM=VB and forcing applications to accept
the non-data RDW because of some dream or
nightmare about C applications needing a record
length twice if one would ever read SMF data..

Does your "bit for bit" statement include:
RECFM=VBS BDW and SDW ?
RECFM=VB BDW and RDW ?
( Which are the unrelated red herrings that Paul is on. )
Partition data set including the directory?
An ISAM data set that is not in unloaded format?
KSDS, ESDS, RRDS, LINEAR ?

None of the above garbage is related to zip files.

[Mike Rayborn [e-mail] [H390-MVS]]

I don't usually bite on comments like this, and maybe your asking
nicely, but it's sure hard to tell from the perceived tone of your comments.

Let me say that I have taken my time and effort to create an IND$FILE
from zero specs. Maybe you've done likewise for something in your
past. And if so, you'll know it's not an easy task. In any case, it
was my choice to do so. If anyone finds what I have done useful that's
great and they're welcome to share in what I've done. For those who
don't like it, well they can go and build something that they do like,
and maybe share it if they so desire.

Now I'll go back to playing with IND$FILE. If I choose to share it
later then you are welcome to use it, or not. You won't hurt my
feelings if you decide that IND$FILE is not right for you.

Bye for now.

--Mike

Post by ***@yahoo.com [H390-MVS]

IND$FILE --MUST-- transmit any file "bit for bit" unchanged
when transmitting in binary mode.
/s/ Bill Turner, wb4alm

- - - ending snipped - - -
The question was never anything about the data.
It was converting SMF data from RECFM=VBS
to RECFM=VB and forcing applications to accept
the non-data RDW because of some dream or
nightmare about C applications needing a record
length twice if one would ever read SMF data..
RECFM=VBS BDW and SDW ?
RECFM=VB BDW and RDW ?
( Which are the unrelated red herrings that Paul is on. )
Partition data set including the directory?
An ISAM data set that is not in unloaded format?
KSDS, ESDS, RRDS, LINEAR ?
None of the above garbage is related to zip files.

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
I don't usually bite on comments like this, and maybe your asking
nicely, but it's sure hard to tell from the perceived tone of your comments.

- - - remainder snipped - - -

If my responding to Bill Turner has upset you,
I won't respond to him for a couple of days.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]
No, zip files can't be stored in VB datasets
unless you are willing to take away
the ability to store binary SMF files in VB
datasets.

The world doesn't end because an application
selected one of two method to read a file.
The second method ( SDW, RDW, BDW)
doesn't magically end for all time.

Post by ***@yahoo.com.au [H390-MVS]
Specifically, what do you expect IND$FILE
to do when transferring an SMF files (VB)
in binary mode from mainframe to PC?

Either run IEBGENER or IDCAMS REPRO to
convert to RECFM=VB and then change to
unload format with XMIT, BDT, or other method.
Another option is to convert to unload format
with IEHMOVE and down load that.
No different than MVS RECFM=U data sets.

If you try your method and reject normal utilities,
you will need custom utilities to recover the data.

Post by ***@yahoo.com.au [H390-MVS]
I expect it to be sensible. How about you?
BFN. Paul.

I would also expect it to be sensible so applications
can both handle byte streams and handle R$CFM=U,
FB, VB but that is not what you created.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Several issues have been corrected
regarding binary downloads of RECFM=U
PDS members (think load modules).

Note that zip files are most appropriately
stored as RECFM=U, whether sequential
or PDS.

zip files are just a byte stream.
They can be stored in RECFM=U, RECFM=VB,
or BSAM-RECFM=FBx1
RECFM=FB can be used because unzip programs
will read before the padding but why corrupt data
when there is no reason to.

No, zip files can't be stored in VB datasets
unless you are willing to take away
the ability to store binary SMF files in VB
datasets.
At least under any sensible paradigm.
BFN. Paul.

IEBGENER and IDCAMS can convert SMF data
to RECFM=VB but that has nothing with allowing
C applications of not having RDWs shoved down
their throat when they didn't request it.
C applications don't need to be told the amount
of bytes read a second time on each read.

If you want to allow applications to see an SDW,
RDW, or BDW that is not an issue. Easily done.

[Mike Rayborn [e-mail] [H390-MVS]]

There was no bug in PDPCLIB regarding RECFM=U. The bug was in IND$FILE.

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Several issues have been corrected
regarding binary downloads of RECFM=U
PDS members (think load modules).

Note that zip files are most appropriately
stored as RECFM=U, whether sequential
or PDS. Transferring a load module is
not something is useful because it uses
metadata which cannot be lost. It would
be better to make sure a zip file can be
transferred from PC to mainframe as
RECFM=U and opened by MINIUNZ,
preferably the SEASIK version of
MINIUNZ, which I assume is included
as part of TK4- (ie check HLQ of MINIZIP).
If you are using PDPCLIB (GCCMVS),
I am shocked that there was anything
in your code that needed to be "fixed"
to support RECFM=U. All you need to
do is open normally with "rb", and
write normally with "wb". Could you
please explain what the bug was and
what you needed to do to fix it?
Thanks. Paul.

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
There was no bug in PDPCLIB regarding
RECFM=U. The bug was in IND$FILE.

I am very interested in the technical
details of this, if you could elaborate.
People have speculated about this
for years. There was speculation
that the problem with IND$FILE
RECFM=U was due to a bug in
Dignus C (not PDPCLIB).

PDPCLIB provides no official way for
you to know that you are processing
a RECFM=U dataset.

So I am curious as to an answer to
all the speculation. What was the
problem with IND$FILE (your
code), and did it affect both Dignus
and PDPCLIB?

If it's too much work to explain,
that's fine, we'll all just look
to the future instead of wondering
about the past. :-)

Thanks. Paul.

[Mike Rayborn [e-mail] [H390-MVS]]

I really can't remember as that issue must have been from a long time
ago. That is, when I tested using the current code I didn't see the
problem of 0 bytes being downloaded for a recfm=u dataset/member, so I
really can't say what changed along the way. Over the life of IND$FILE,
I moved from Dignus based code to GCC370 and PDPCLIB and several
incarnations of IND$FILE occurred, most of those version stayed in
testing and never got released to the wild.

If we can, I'd like to focus on the binary file transfer for just a
bit. Many of you have stated that you like/want/need to preserve the
RDW in the records being transferred. And I understand why, since I'm
guessing you intend to post process the downloaded binary data and need
those RDW's so that you can processed the resulting file. Without those
RDW's you can't easily "see" where one record in the stream of bytes
begins and ends. Makes sense and I get it.

So I'd like to understand the expected processing for uploading (PUT) of
a binary file (let's say a GIF or JPG type file) to a recfm=vb dataset.
As best I know, binary files on a PC don't have RDW's, so would you
expect an upload of a PC binary file to a recfm=vb style dataset to
succeed. And as far as I know it does except in some specific cases.

Let me give you an example. Let's say I have a PC file that contains
256 characters, the first character is x'00', then x'01' and so on,
ending with x'FF'. I can upload this file in binary mode and write
those 256 bytes to a recfm=vb dataset.

Here's dump of the received buffer, the file data start at offset x'11'
below:
Dump of 00149288 "received buffer" (273 bytes)
+00000 880110D0 46056306 00000001 C0806101 :h..}........{./.:
:....F.c.......a.:
+00010 05000102 03040506 0708090A 0B0C0D0E :................:
:................:
+00020 0F101112 13141516 1718191A 1B1C1D1E :................:
:................:
+00030 1F202122 23242526 2728292A 2B2C2D2E :................: :.
!"#$%&'()*+,-.:
+00040 2F303132 33343536 3738393A 3B3C3D3E :................:
:/0123456789:;<=>:
+00050 3F404142 43444546 4748494A 4B4C4D4E :. .........¢.<(+:
:?@ABCDEFGHIJKLMN:
+00060 4F505152 53545556 5758595A 5B5C5D5E :|&.........!$*);:
:OPQRSTUVWXYZ¢\|¬:
+00070 5F606162 63646566 6768696A 6B6C6D6E :¬-/........Š,%_>:
:_`abcdefghijklmn:
+00080 6F707172 73747576 7778797A 7B7C7D7E :?.........`:#@'=:
:opqrstuvwxyz{Š}~:
+00090 7F808182 83848586 8788898A 8B8C8D8E :".abcdefghi.....:
:................:
+000A0 8F909192 93949596 9798999A 9B9C9D9E :..jklmnopqr.....:
:................:
+000B0 9FA0A1A2 A3A4A5A6 A7A8A9AA ABACADAE :..~stuvwxyz...[.:
:................:
+000C0 AFB0B1B2 B3B4B5B6 B7B8B9BA BBBCBDBE :..............].:
:................:
+000D0 BFC0C1C2 C3C4C5C6 C7C8C9CA CBCCCDCE :.{ABCDEFGHI.....:
:................:
+000E0 CFD0D1D2 D3D4D5D6 D7D8D9DA DBDCDDDE :.}JKLMNOPQR.....:
:......[.........:
+000F0 DFE0E1E2 E3E4E5E6 E7E8E9EA EBECEDEE :.\.STUVWXYZ.....:
:......].........:
+00100 EFF0F1F2 F3F4F5F6 F7F8F9FA FBFCFDFE :.0123456789.....:
:................:
+00110 FF :. :
:. :

The fwrite of the 256 bytes (x'00' - x'FF') returns 256 as expected.
fwrite() rc=256

However when I browse the dataset using RFE browse it looks like an
empty file. So I'm guessing the fwrite() interpreted the x'00010203' as
a RDW?
HERC01.ALLCHARS.TXT MVSCAT 1 PS VB 0 1 255 14790

When browsed in RFE is looks like:
HERC01.ALLCHARS.TXT on MVSCAT --------------------------------- Line 1
Col 1 80
Command ===> Scroll ===> CS
1 10 20 30 40 50 60 70 80
+---+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
******EOF-TTR=000001************ BOTTOM OF DATA
********************************

So maybe this is a bug/feature of PDPCLIB or something I've
misunderstood in fwrite() behavior for binary files.

That all being said, I have uploaded and then downloaded a PC GIF file
in binary to a recfm=vb dataset and the file was unchanged by the
process and could be viewed without issue on the PC following the download.

In any case, it's all been very interesting and educational.

73, KK4WGD
--Mike

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
There was no bug in PDPCLIB regarding
RECFM=U. The bug was in IND$FILE.

I am very interested in the technical
details of this, if you could elaborate.
People have speculated about this
for years. There was speculation
that the problem with IND$FILE
RECFM=U was due to a bug in
Dignus C (not PDPCLIB).
PDPCLIB provides no official way for
you to know that you are processing
a RECFM=U dataset.
So I am curious as to an answer to
all the speculation. What was the
problem with IND$FILE (your
code), and did it affect both Dignus
and PDPCLIB?
If it's too much work to explain,
that's fine, we'll all just look
to the future instead of wondering
about the past. :-)
Thanks. Paul.

[Mike Rayborn [e-mail] [H390-MVS]]

I'll provide a little more info and current thoughts.

My earlier email was a bit misleading as the GIF file I transferred to
the mainframe and back again was in fact to a RECFM=U dataset, and not a
RECFM=VB as I had claimed. So of course that worked as expected, as
RECFM=U was the correct dataset type for raw binary files like a GIF or
ZIP file. I had forgotten that my IND$FILE creates new BINARY datasets
as RECFM=U (I blame my faulty memory). I need to review my own help
text once in a while.

So I've come around to thinking that binary transfers of the RDW style
RECFM=VB datasets should preserve and expect RDW byte sequence as the
first 4 bytes. For ASCII transfers to a RECFM=VB dataset, an RDW will
not exist (at least not externally). For some this will seem strange
and different from what IBM's IND$FILE does. I can't be sure if it is
or isn't, just that I don't remember IBM doing it this way, so maybe
that will cause some confusion.

In any case, handling RECFM=VB datasets this way for BINARY transfers
can be very handy for downloading SMF datasets.

In most cases, if you're uploading a binary dataset from the PC to MVS,
the target should be a RECFM=U dataset. So files like ZIP, GIF, JPG,
etc, remain consistent across platforms as long as they reside on the
mainframe as RECFM=U datasets.

I'll try to do a new build later and get it into the files area so
everyone else can take a shot at it.

Thanks to all for the feedback and suggestions.

--Mike

Post by Mike Rayborn ***@comcast.net [H390-MVS]
I really can't remember as that issue must have been from a long time
ago. That is, when I tested using the current code I didn't see the
problem of 0 bytes being downloaded for a recfm=u dataset/member, so I
really can't say what changed along the way. Over the life of IND$FILE,
I moved from Dignus based code to GCC370 and PDPCLIB and several
incarnations of IND$FILE occurred, most of those version stayed in
testing and never got released to the wild.
If we can, I'd like to focus on the binary file transfer for just a
bit. Many of you have stated that you like/want/need to preserve the
RDW in the records being transferred. And I understand why, since I'm
guessing you intend to post process the downloaded binary data and need
those RDW's so that you can processed the resulting file. Without those
RDW's you can't easily "see" where one record in the stream of bytes
begins and ends. Makes sense and I get it.
So I'd like to understand the expected processing for uploading (PUT) of
a binary file (let's say a GIF or JPG type file) to a recfm=vb dataset.
As best I know, binary files on a PC don't have RDW's, so would you
expect an upload of a PC binary file to a recfm=vb style dataset to
succeed. And as far as I know it does except in some specific cases.
Let me give you an example. Let's say I have a PC file that contains
256 characters, the first character is x'00', then x'01' and so on,
ending with x'FF'. I can upload this file in binary mode and write
those 256 bytes to a recfm=vb dataset.
Here's dump of the received buffer, the file data start at offset x'11'
Dump of 00149288 "received buffer" (273 bytes)
+00030 1F202122 23242526 2728292A 2B2C2D2E :................: :.
The fwrite of the 256 bytes (x'00' - x'FF') returns 256 as expected.
fwrite() rc=256
However when I browse the dataset using RFE browse it looks like an
empty file. So I'm guessing the fwrite() interpreted the x'00010203' as
a RDW?
HERC01.ALLCHARS.TXT MVSCAT 1 PS VB 0 1 255 14790
HERC01.ALLCHARS.TXT on MVSCAT --------------------------------- Line 1
Col 1 80
Command ===> Scroll ===> CS
1 10 20 30 40 50 60 70 80
+---+----+----+----+----+----+----+----+----+----+----+----+----+----+----+----+
******EOF-TTR=000001************ BOTTOM OF DATA
********************************
So maybe this is a bug/feature of PDPCLIB or something I've
misunderstood in fwrite() behavior for binary files.
That all being said, I have uploaded and then downloaded a PC GIF file
in binary to a recfm=vb dataset and the file was unchanged by the
process and could be viewed without issue on the PC following the download.
In any case, it's all been very interesting and educational.
73, KK4WGD
--Mike

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
There was no bug in PDPCLIB regarding
RECFM=U. The bug was in IND$FILE.

I am very interested in the technical
details of this, if you could elaborate.
People have speculated about this
for years. There was speculation
that the problem with IND$FILE
RECFM=U was due to a bug in
Dignus C (not PDPCLIB).
PDPCLIB provides no official way for
you to know that you are processing
a RECFM=U dataset.
So I am curious as to an answer to
all the speculation. What was the
problem with IND$FILE (your
code), and did it affect both Dignus
and PDPCLIB?
If it's too much work to explain,
that's fine, we'll all just look
to the future instead of wondering
about the past. :-)
Thanks. Paul.

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
I'll provide a little more info and current thoughts.
My earlier email was a bit misleading as the GIF file I transferred to
the mainframe and back again was in fact to a RECFM=U dataset,
and not a RECFM=VB as I had claimed.

- - - ending snipped - - -

Makes no difference.

IBM IND$FILE will neither drop data nor add any RDW for
down loads of either RECFM=U or RECFM=VB data.

Up and down load any .gif or other PC byte stream binary file to
MVS as many times as you like as RECFM=U or RECFM=VB and
when on your PC, it will be "bit for bit" identical with the original.
The MVS designers and current maintainers pride themselves on
not corrupting data.

[[e-mail] [H390-MVS]]

- - - In H390-***@yahoogroups.com, <***@...> wrote:
- - - beginning snipped - - -

If we can, I'd like to focus on the binary file transfer for just a bit.
Many of you have stated that you like/want/need to preserve the
RDW in the records being transferred.

That at least sounds okay to me, but if you want to diverge from
IBM's lead, it might come back to bite.

I would suggest that you make preserving and restoring RDWs
an option.

And I understand why, since I'm guessing you intend to post process
the downloaded binary data and need those RDW's so that you can
processed the resulting file. Without those RDW's you can't easily
"see" where one record in the stream of bytes begins and ends.
Makes sense and I get it.

People have been down and up loading files for decades.
There is no issue with preserving record and block boundaries.

If you provide still another another option, then transferring data
to real systems will need to avoid your code and will need to use
tried and true methods.
Perhaps a warning that you are creating incompatible data?

So I'd like to understand the expected processing for uploading (PUT) of
a binary file (let's say a GIF or JPG type file) to a recfm=vb dataset.
As best I know, binary files on a PC don't have RDW's, so would you
expect an upload of a PC binary file to a recfm=vb style dataset to
succeed. And as far as I know it does except in some specific cases.

- - - ending snipped - - -

Like I mentioned above, you will need a option to indicate that
non-standard processing is being used.
IBM's FTP has such an option that can be specified when down
loading data but it doesn't work for up load. The IBM default is to
not pretend that an RDW is part of the data. IBM IND$FILE and
FTP have no useable option to use RECFM=VB data that need
record boundaries in PC intermediate files.
Use .aws, XMIT, BDT, IEHMOVE to put the data in unloaded
format before download.

Okay, you can FTP RECFM=VB to a PC with the RDW option,
upload as RECFM=U, and run a crappy non-standard program
to add BDWs back while copying to RECFM=VB .
Wouldn't if have been simpler to just put the data in standard
unloaded format for down load and use standard methods to
load the data back to standard format? Is standard that bad?

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
If we can, I'd like to focus on the binary file transfer for just a
bit. Many of you have stated that you like/want/need to preserve the
RDW in the records being transferred. And I understand why, since I'm
guessing you intend to post process the downloaded binary data and need
those RDW's so that you can processed the resulting file. Without those
RDW's you can't easily "see" where one record in the stream of bytes
begins and ends. Makes sense and I get it.

Yes, for a genuine binary VB file, the RDWs
are an integral part of the data. And even if
it is a text file, not a binary file, that is being
transferred in binary mode, for whatever
reason, then the text file will be trashed
if the RDWs are lost. That prevents the
text file from being transferred back up
to the mainframe (in either binary or
text mode).

Post by Mike Rayborn ***@comcast.net [H390-MVS]
So I'd like to understand the expected processing for uploading (PUT) of
a binary file (let's say a GIF or JPG type file) to a recfm=vb dataset.

I personally consider it totally inappropriate
to be transferring a JPG to a RECFM=VB
dataset. That is *exactly* what RECFM=U
is for.

VB is for text files, like source code, or the
binary equivalent of that. Or genuine
binary variable-length data, such as SMF
records.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
As best I know, binary files on a PC don't have RDW's, so would you
expect an upload of a PC binary file to a recfm=vb style dataset to
succeed.

It is true that the RDW-format files you
see on the mainframe are almost
non-existent, but the concept of using
RDW files on the PC is perfectly valid.

There is nothing wrong with writing a
program to process an RDW-file. Note
that IBM themselves recognized the
need for an RDW-file, and they added
an "rdw" option for their ftp program
to preserver the RDW.

Even if you never actually process this
file on the PC, at least the data remains
intact so that you can transfer it to
another system and reupload it.

BTW, this is a really long debate we've
been having in the hercules-os380 group
(again!).

Post by Mike Rayborn ***@comcast.net [H390-MVS]
The fwrite of the 256 bytes (x'00' - x'FF') returns
256 as expected. fwrite() rc=256

Ok, that looks like a bug in PDPCLIB, it
should have returned 0.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
However when I browse the dataset using RFE browse it looks like an
empty file. So I'm guessing the fwrite() interpreted the x'00010203' as
a RDW?

Correct. The length of x'0001' is an
error, and the '0203' is also an error,
so the error indicator is set and no
more writing is possible to the
dataset.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
So maybe this is a bug/feature of PDPCLIB or something I've
misunderstood in fwrite() behavior for binary files.

It's a feature of PDPCLIB. IBM C will
generate its own fixed-length RDW
to store inappropriate data in the VB
dataset.

When I was designing PDPCLIB, I
normally would have just copied IBM,
but when I realized that I couldn't do
a binary copy of a VB text file (or
perhaps a gzip and gunzip), because
all the RDWs got stripped, and thus
all the data ran together, ie a file like:

AAA
BBBB

got converted into a single line:

AAABBBB

irretrievably trashed, I wasn't willing to
code PDPCLIB to behave like that, as
I considered that it complicated MVS
datasets for someone moving from
Unix or MSDOS to the mainframe.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
My earlier email was a bit misleading as the GIF file I transferred to
the mainframe and back again was in fact to a RECFM=U dataset, and not a
RECFM=VB as I had claimed. So of course that worked as expected, as
RECFM=U was the correct dataset type for raw binary files like a GIF or
ZIP file.

Exactly correct.

It's actually very strange, in my opinion, to
expect VB files to behave the same as
RECFM=U. Surely there should be some
difference between these two formats, from
the point of view of a C program?

Post by Mike Rayborn ***@comcast.net [H390-MVS]
So I've come around to thinking that binary transfers of the RDW style
RECFM=VB datasets should preserve and expect RDW byte sequence as the
first 4 bytes.

Cool!

Post by Mike Rayborn ***@comcast.net [H390-MVS]
For ASCII transfers to a RECFM=VB dataset, an RDW will
not exist (at least not externally).

Sure. There is no dispute about that. The
only dispute is about VB binary files and
RECFM=U text files. You might want to
check RECFM=U text files as well, and
see what you think. IBM C again runs
all the text together if you read a text
file in binary mode. But this time for a
different reason. It is because for text
files it uses the block boundary as a
line terminator, but binary mode ignores
that.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
For some this will seem strange
and different from what IBM's IND$FILE does. I can't be sure if it is
or isn't, just that I don't remember IBM doing it this way, so maybe
that will cause some confusion.

There will be no confusion if people simply
store their ZIP files in RECFM=U where
they belong, instead of RECFM=VB where
there is a dispute.

I don't know any valid reason why people
should insist that VB behave like U and
then insist that they want to use VB
instead of U.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
In any case, handling RECFM=VB datasets this way for BINARY transfers
can be very handy for downloading SMF datasets.

Yes, and it extends in general. It's not
just IND$FILE, it's anything that is
processing a RECFM=VB dataset on
the mainframe. Do you expect a
gzip/gunzip sequence to preserve or
trash both an SMF file and a text file?
PDPCLIB preserves both. IBM trashes
both.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
In most cases, if you're uploading a binary dataset from the PC to MVS,
the target should be a RECFM=U dataset. So files like ZIP, GIF, JPG,
etc, remain consistent across platforms as long as they reside on the
mainframe as RECFM=U datasets.

Yes, exactly correct, in my opinion.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
I don't usually bite on comments like this, and maybe your asking
nicely, but it's sure hard to tell from the perceived tone of your comments.

Please ignore somitcw. His dispute is
with me, not you. We've been clashing
over this and other issues for many
years.

BFN. Paul.

[Denis Molony [e-mail] [H390-MVS]]

Can anyone explain the reason why FTP allows variable length records to be downloaded (with the RDW option), but doesn’t allow those same records to be uploaded back to the same dataset? It always struck me as a one-way only solution.

Or is there another (simple) option that I am missing?

[[e-mail] [H390-MVS]]

Post by Denis Molony ***@iinet.net.au [H390-MVS]
Can anyone explain the reason why FTP
allows variable length records to be
downloaded (with the RDW option), but
doesn’t allow those same records to be
uploaded back to the same dataset?

We're still waiting for IBM to answer
that exact question:

https://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=63596

I can't verify that the link still works
because it's offline right now.

Post by Denis Molony ***@iinet.net.au [H390-MVS]
It always struck me as a one-way only solution.

Yes, that's right.

Post by Denis Molony ***@iinet.net.au [H390-MVS]
Or is there another (simple) option that I am missing?

There is no simple option available, but
the simplest option I know of is to transfer
the file back as RECFM=U and then use
COPYFILE to put it into RECFM=V.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by Denis Molony ***@iinet.net.au [H390-MVS]
Can anyone explain the reason why FTP allows variable length records
to be downloaded (with the RDW option), but doesn’t allow those same
records to be uploaded back to the same dataset? It always struck me
as a one-way only solution.

I agree with you but IBM does nothing that they can't sell.
A user requirements may help?

Post by Denis Molony ***@iinet.net.au [H390-MVS]
Or is there another (simple) option that I am missing?

I would prefer if IBM honoured MODE B for TYPE B
as in MODE Block for a TYPE Binary that PCs recognize.
Since TYPE Binary is logically is the same as TYPE Ebcdic,
it should be a tiny zap. Perhaps a one byte zap?

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
The fwrite of the 256 bytes (x'00' - x'FF') returns
256 as expected. fwrite() rc=256

Ok, that looks like a bug in PDPCLIB, it
should have returned 0.

I have confirmed that the bug still
exists in current dev PDPCLIB, and
have made a quite simple fix and
it tests ok now if you want to apply it.

http://pdos.cvs.sourceforge.net/viewvc/pdos/pdos/pdpclib/stdio.c?r1=1.234&r2=1.235

Apologies for any inconvenience.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
it tests ok now if you want to apply it.

Alternatively, you should be able to
just call ferror() to see if an error has
occurred.

BFN. Paul.

[Mike Rayborn [e-mail] [H390-MVS]]

Paul, I reviewed the STDIO source code as well, and noticed the way it
handled the "error". So I'll be checking with ferror() following calls
to fwrite() just to be safe.

As you've likely seen in this long running email exchange, there are
some who do not appreciate the reasoning behind the way recfm=vb files
are handled in PDPCLIB. It seems to be different from the other C
runtimes handle binary reads and writes. But I can appreciate the
reasoning that went into the design of that code and the solution it
provides for those who might use a C program to processes variable
length binary data. So Paul, you get both a tip of the hat and a raised
eyebrow. It is/was a creative solution to the problem in that context.

Paul, thanks for the fix, but for now I'll just work around any issues,
or make the same fix in the PDPCLIB stdio code.

--Mike

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
The fwrite of the 256 bytes (x'00' - x'FF') returns
256 as expected. fwrite() rc=256

Ok, that looks like a bug in PDPCLIB, it
should have returned 0.

I have confirmed that the bug still
exists in current dev PDPCLIB, and
have made a quite simple fix and
it tests ok now if you want to apply it.
http://pdos.cvs.sourceforge.net/viewvc/pdos/pdos/pdpclib/stdio.c?r1=1.234&r2=1.235
Apologies for any inconvenience.
BFN. Paul.

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Paul, I reviewed the STDIO source code as well, and noticed the way it
handled the "error". So I'll be checking with ferror() following calls
to fwrite() just to be safe.
As you've likely seen in this long running email exchange, there are
some who do not appreciate the reasoning behind the way recfm=vb files
are handled in PDPCLIB. It seems to be different from the other C
runtimes handle binary reads and writes. But I can appreciate the
reasoning that went into the design of that code and the solution it
provides for those who might use a C program to processes variable
length binary data. So Paul, you get both a tip of the hat and a raised
eyebrow. It is/was a creative solution to the problem in that context.

It was a poor method to redesign MVS for the worse.

It would be much simpler to do it right but that would make Paul's
software compatible with the rest of the world.

If an application program wants an RDW, then and only then should
it get a RDW. If the application program was not designed to accept
an RDW, then an RDW should not be forced on it. Why an application
program wants a second length is weird but up to the programmer.

The application program RDW option could be specified in the program
or in JCL or both . What is specified in the program should override
what is specified in the JCL .

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Paul, thanks for the fix, but for now I'll just work around any issues,
or make the same fix in the PDPCLIB stdio code.
--Mike

- - - old notes snipped - - -

[Mike Rayborn [e-mail] [H390-MVS]]

Have no fear. I've heard all of you and I understand, or at least I
hope I do :)

I'm going to take a step back and design/build a replacement for the
fread()/fwrite() functions that IND$FILE uses so that we can have it
both ways for variable binary datasets. For those who want/need a RDW,
we'll have an option for that, but the default will be to not have the
RDW in the records that are read/written in binary mode. This *should*
please everyone, right?

This change will likely delay the pending IND$FILE roll out as I work
through the issues and get a new round of testing with our alpha/beta
testers.

Thanks to everyone for contributing to the discussion.

--Mike

Post by ***@yahoo.com [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Paul, I reviewed the STDIO source code as well, and noticed the way it
handled the "error". So I'll be checking with ferror() following calls
to fwrite() just to be safe.
As you've likely seen in this long running email exchange, there are
some who do not appreciate the reasoning behind the way recfm=vb files
are handled in PDPCLIB. It seems to be different from the other C
runtimes handle binary reads and writes. But I can appreciate the
reasoning that went into the design of that code and the solution it
provides for those who might use a C program to processes variable
length binary data. So Paul, you get both a tip of the hat and a raised
eyebrow. It is/was a creative solution to the problem in that context.

It was a poor method to redesign MVS for the worse.
It would be much simpler to do it right but that would make Paul's
software compatible with the rest of the world.
If an application program wants an RDW, then and only then should
it get a RDW. If the application program was not designed to accept
an RDW, then an RDW should not be forced on it. Why an application
program wants a second length is weird but up to the programmer.
The application program RDW option could be specified in the program
or in JCL or both . What is specified in the program should override
what is specified in the JCL .

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Paul, thanks for the fix, but for now I'll just work around any issues,
or make the same fix in the PDPCLIB stdio code.
--Mike

- - - old notes snipped - - -

[[e-mail] [H390-MVS]]

Have no fear. I've heard all of you and I understand, or at least I
hope I do :)
I'm going to take a step back and design/build a replacement for the
fread()/fwrite() functions that IND$FILE uses so that we can have it
both ways for variable binary datasets. For those who want/need a RDW,
we'll have an option for that, but the default will be to not have the
RDW in the records that are read/written in binary mode. This *should*
please everyone, right?

Hallelujah

This change will likely delay the pending IND$FILE roll out as I work
through the issues and get a new round of testing with our alpha/beta
testers.
Thanks to everyone for contributing to the discussion.
--Mike

- - - old notes snipped - - -

Allowing a non-default option of RDW would be a nice enhancement to
IND$FILE but IBM might also do that and may pick a different method.
For all I know, IBM may already have the option?
My knowledge is out-of-date.

Questions:

Do you plan to use a two byte record length and two bytes for a spanned
record indicator or do you plan to go with a four byte record length plus four?
IBM FTP RDW uses the RECFM=VB style RDW but I believe that IBM FTP
type e mode b uses a four byte record length?.

Is your change also for RECFM=FB records or do RECFM=U and
RECFM=VB/VBS have all of the fun?

Big-endian or little-endian or magically based on the type of PC?

Will TRUNC/NOTRUNC then apply for binary transfers?
Will truncation be for spaces or binary zeroes?
I suggest that for binary transfers you continue to force NOTRUNC .

Wouldn't allowing an .aws option be easier and have less risk of
being incompatible with future IBM options and formats?

[Bert Lindeman [e-mail] [H390-MVS]]

Hi all,

As far as I know a more or less current IND$FILE on z/OS dates from 1995.
Do you have any news on PTFs on it?

Have a nice day,

Bert

Post by ***@yahoo.com [H390-MVS]

Have no fear. I've heard all of you and I understand, or at least I
hope I do :)
I'm going to take a step back and design/build a replacement for the
fread()/fwrite() functions that IND$FILE uses so that we can have it
both ways for variable binary datasets. For those who want/need a RDW,
we'll have an option for that, but the default will be to not have the
RDW in the records that are read/written in binary mode. This *should*
please everyone, right?

Hallelujah

This change will likely delay the pending IND$FILE roll out as I work
through the issues and get a new round of testing with our alpha/beta
testers.
Thanks to everyone for contributing to the discussion.
--Mike

- - - old notes snipped - - -
Allowing a non-default option of RDW would be a nice enhancement to
IND$FILE but IBM might also do that and may pick a different method.
For all I know, IBM may already have the option?
My knowledge is out-of-date.
Do you plan to use a two byte record length and two bytes for a spanned
record indicator or do you plan to go with a four byte record length plus four?
IBM FTP RDW uses the RECFM=VB style RDW but I believe that IBM FTP
type e mode b uses a four byte record length?.
Is your change also for RECFM=FB records or do RECFM=U and
RECFM=VB/VBS have all of the fun?
Big-endian or little-endian or magically based on the type of PC?
Will TRUNC/NOTRUNC then apply for binary transfers?
Will truncation be for spaces or binary zeroes?
I suggest that for binary transfers you continue to force NOTRUNC .
Wouldn't allowing an .aws option be easier and have less risk of
being incompatible with future IBM options and formats?

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
If we can, I'd like to focus on the binary file transfer for just a
bit. Many of you have stated that you like/want/need to preserve the
RDW in the records being transferred. And I understand why, since I'm
guessing you intend to post process the downloaded binary data and need
those RDW's so that you can processed the resulting file. Without those
RDW's you can't easily "see" where one record in the stream of bytes
begins and ends. Makes sense and I get it.

Yes, for a genuine binary VB file, the RDWs
are an integral part of the data. And even if
it is a text file, not a binary file, that is being
transferred in binary mode, for whatever
reason, then the text file will be trashed
if the RDWs are lost. That prevents the
text file from being transferred back up
to the mainframe (in either binary or
text mode).

I question your ownership of everyone's data and your
authority to dictate how they store their data.

Your redefinition of RECFM=VB is not compatible
with what people do.

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
So I'd like to understand the expected processing for
uploading (PUT) of a binary file (let's say a GIF or JPG
type file) to a recfm=vb dataset.

I personally consider it totally inappropriate
to be transferring a JPG to a RECFM=VB
dataset. That is *exactly* what RECFM=U
is for.

Your dictates are beyond belief.

Post by ***@yahoo.com.au [H390-MVS]
VB is for text files, like source code, or the
binary equivalent of that. Or genuine
binary variable-length data, such as SMF
records.

Back to your red herring of your secret issue with
SMF data that you keep secret.
You must keep the discussion away from up loading
byte stream binary data like zip files in RECFM=VB
format like people have been doing for decades.

Post by ***@yahoo.com.au [H390-MVS]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
As best I know, binary files on a PC don't have
RDW's, so would you expect an upload of a PC
binary file to a recfm=vb style dataset to succeed.

It is true that the RDW-format files you
see on the mainframe are almost
non-existent, but the concept of using
RDW files on the PC is perfectly valid.

Redesigning how data is stored on a main
frame because you believe that how data
is stored on a PC needs to be changed.
Does that not sound weird to you?

Post by ***@yahoo.com.au [H390-MVS]
There is nothing wrong with writing a
program to process an RDW-file. Note
that IBM themselves recognized the
need for an RDW-file, and they added
an "rdw" option for their ftp program
to preserver the RDW.

That is a one-way option.

Because IBM optionally allows RDWs to be
stored on a PC interspersed with binary data
is not a valid reason to redesign main frame
RECFM=VB

Down load of record oriented files is not
related to up load of byte stream binary
data other than for your red herring.

Post by ***@yahoo.com.au [H390-MVS]
Even if you never actually process this
file on the PC, at least the data remains
intact so that you can transfer it to
another system and reupload it.

What does up loading byte stream binary
data to the type of file that people want
and people have been doing for decades
have to do with processing data on a PC
for your red herring?

Post by ***@yahoo.com.au [H390-MVS]
BTW, this is a really long debate we've
been having in the hercules-os380 group
(again!).

- - - remaining snipped - - -

Yes, and you keep switch the subject of the
debate from up loading byte stream binary
data to some secret issue that you have with
down loading SMF record format binary data.

Why not come clean and stop switching the
subject from UP loading byte stream binary
data or giving up your secret issue with
down loading SMF record orientated
big-endian binary data?

Also please stop telling everyone that the way
that they have been using RECFM=VB for
decades is inappropriate.

Also please stop trying to have IND$FILE
changed to default to being incompatible
with IBM's IND$FILE.

The craziness of not allowing byte stream
binary data in a RECFM=VB file is craziness
and does not match what has been done for
decades. Your secret SMF data red herring
that you keep secret is not related.

[Manuel Escobar [e-mail] [H390-MVS]]

Well, I have no experience with MVS programming in C, I just learned the
language at the end of the 80's and then participated to several projects
on both Windows/Unix/Linux platforms. Definitely on the 'like it' side.

Recently I was in need to run a small utility under z/Os and I was very
impressed seeing how quick the port was completed (with the standard IBM
tools). Apart from the (usual?) difficulties with the dataset format (fixed
or variable length recs?) and the LF/CR line terminators I didn't had to
modify the code, it simply run at the first launch.

To return to our previous discussion, I think it's not about languages but
about programmers. Every skilled C program should know that a NUL
terminated buffer must be used differently than a binary buffer. But I also
understand that sometimes you're forced to use a wrench as a hammer...

[[e-mail] [H390-MVS]]

My IND$FILE is written C. The new version uses the PDPCLIB for
file/dataset IO

Some points:

1. In the last month there has been a massive
argument in hercules-os380 as to whether
PDPCLIB behaves sensibly or not, given
that its behavior is different from IBM C.
Rightly or wrongly, most people seem to
like IBM C's behavior. I believe JCC behaves
the same as IBM C, and JCC now ships
with TK4-, so that gives you another option.

2. Unfortunately PDPCLIB does not yet
add the TSO prefix to the dataset name.
That is waiting for someone to volunteer
to write a "simple" assembler routine
to call IKJPARS to just handle a single
dataset name as input and add the
prefix as output. I don't know whether
JCC allows running as a TSO CP, and
whether the prefix is added.

and as such kind of does its own thing regarding record
formats. Generally this works out just fine and simplifies reading and
writing datasets.

Yes, I personally think that PDPCLIB's
default behavior with regard to record
formats is exactly as you said: works
just fine and simplifies things to be as
simple as e.g. MSDOS.

For better or worse, I seem to be the guy for IND$FILE on MVS38J

:-) That's how things work!


BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
2. Unfortunately PDPCLIB does not yet
add the TSO prefix to the dataset name.
That is waiting for someone to volunteer
to write a "simple" assembler routine
to call IKJPARS to just handle a single

Actually, in the short term, a routine
that just scanned the control blocks
and returned the prefix would be
good enough to overcome this
problem.

BFN. Paul.

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]
2. Unfortunately PDPCLIB does not yet
add the TSO prefix to the dataset name.
That is waiting for someone to volunteer
to write a "simple" assembler routine
to call IKJPARS to just handle a single

Actually, in the short term, a routine
that just scanned the control blocks
and returned the prefix would be
good enough to overcome this
problem.

I revisited this message from Gerhard:

https://groups.yahoo.com/neo/groups/hercules-os380/conversations/messages/10225

and thought I'd have a stab at doing
this relatively simple design myself,
and the below code does indeed
work, and I should have this prefixing
issue fixed in PDPCLIB soon.

BFN. Paul.



**********************************************************************
* *
* GETPFX - get TSO prefix *
* *
**********************************************************************
ENTRY @@GETPFX
@@GETPFX EQU *
SAVE (14,12),,@@GETPFX
LR R12,R15
USING @@GETPFX,R12
*
LA R0,0 Not really needed, just looks nice
USING PSA,R0
L R2,PSATOLD
USING TCB,R2
L R3,TCBJSCB
USING IEZJSCB,R3
L R4,JSCBPSCB
USING PSCB,R4
L R5,PSCBUPT
USING UPT,R5
LA R6,UPTPREFX
LR R15,R6
*
RETURNGP DS 0H
RETURN (14,12),RC=(15)
LTORG
*
*
*

...

IEFJESCT ,

[Gerhard Postpischil [e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
https://groups.yahoo.com/neo/groups/hercules-os380/conversations/messages/10225
and thought I'd have a stab at doing
this relatively simple design myself,
and the below code does indeed
work, and I should have this prefixing
issue fixed in PDPCLIB soon.
**********************************************************************
* *
* GETPFX - get TSO prefix *
* *
**********************************************************************
@@GETPFX EQU *
LR R12,R15
*
LA R0,0 Not really needed, just looks nice
USING PSA,R0
L R2,PSATOLD
USING TCB,R2
L R3,TCBJSCB
USING IEZJSCB,R3
L R4,JSCBPSCB
USING PSCB,R4
L R5,PSCBUPT
USING UPT,R5
LA R6,UPTPREFX
LR R15,R6
*
RETURNGP DS 0H
RETURN (14,12),RC=(15)
LTORG

1) I'd also add validity checks to prevent 0Cx abends:

L R1,PSAAOLD
USING ASCB,R1
ICM R1,15,ASCBTSB
BZ NOTTMP

and for each Load, either change to an ICM r,15,fld/BZ or add LTR/BZ
to check that the field exists. In an unpredictable environment, only
the PSA is guaranteed; the TSB is zero in SCHEDULE code, etc.

2) Mapping macros required are IHAASCB, IKJTCB, IEZJSCB, IKJPSCB,
IKJUPT. The only one you don't need is IEFJESCT. Also for this fragment,
you don't need a LTORG.



Gerhard Postpischil
Bradford, VT

[[e-mail] [H390-MVS]]

ICM R1,15,ASCBTSB
BZ NOTTMP

Ok, thanks. See the code below, which
is now up to pdpclib-stage199

Please let me know if I did anything
incorrectly. I did test to see that the
new code does indeed return NULL
when not run from TSO.

2) Mapping macros required are

Most of those were already in mvssupa.asm,
which is what made it easy enough
for me to do myself.

Also for this fragment, you don't need a LTORG.

I like having that for any possible
expansion of any function.

BFN. Paul.



**********************************************************************
* *
* GETPFX - get TSO prefix *
* *
**********************************************************************
ENTRY @@GETPFX
@@GETPFX EQU *
SAVE (14,12),,@@GETPFX
LR R12,R15
USING @@GETPFX,R12
*
LA R15,0
LA R0,0 Not really needed, just looks nice
USING PSA,R0
ICM R2,15,PSATOLD
BZ RETURNGP
USING TCB,R2
ICM R3,15,TCBJSCB
BZ RETURNGP
USING IEZJSCB,R3
ICM R4,15,JSCBPSCB
BZ RETURNGP
USING PSCB,R4
ICM R5,15,PSCBUPT
BZ RETURNGP
USING UPT,R5
LA R6,UPTPREFX
LR R15,R6
*
RETURNGP DS 0H
RETURN (14,12),RC=(15)
LTORG

[[e-mail] [H390-MVS]]

Post by ***@yahoo.com.au [H390-MVS]
work, and I should have this prefixing
issue fixed in PDPCLIB soon.

Ok, the prefixing issue is now committed
to CVS as pdpclib-stage198 and can
be found here:

http://pdos.cvs.sourceforge.net/viewvc/pdos/pdos/pdpclib/

Mike, let me know if you need this new
code, and if so, how you would like it
delivered.

I can give you a (beta) ASCII zip file of
the dev PDPCLIB, or I can give you the
latest dev GCCMVS package as a
zipped XMIT.

BFN. Paul.

[Mike Rayborn [e-mail] [H390-MVS]]

Not to dash anyone's efforts, but I don't really need code to obtain the
TSO PREFIX. IND$FILE doesn't obey the TSO PREFIX setting and never
has. It simply checks for a quoted or unquoted dataset name. If the
dataset name is not quoted, then the TSO userid is prefixed to the
dataset name. It's simple and it works as expected.

My 2 cents on "extending" the PDPCLIB code base.

If we/you/me wish to add new code to PDPCLIB, we must make sure to not
add these extensions to existing libraries, like STDLIB, STDIO, etc.
They *should* be new modules with appropriate headers so that developers
can choose to use or not use any of these extensions.

And as you've shown in your code, these extensions to the PDPCLIB code
base *can* and probably *should* use a "@@" style entry point to avoid
collisions with any application object/load decks.

Just my 2 cents.

Now back to IND$FILE. I've got my DASD moved over to the TK4 system and
was able to do a small test. The DFT/DDM style transfer using WSF data
streams now works as expected. So I'm back on track for IND$FILE
development. Lots of code cleanup is needed and a bunch of testing
before I can release the new module.

Thanks for everyone's help in resolving the disconnect issue, and for
providing me with an upgraded MVS38J on which to do my code development.

--Mike

Post by ***@yahoo.com.au [H390-MVS]

Post by ***@yahoo.com.au [H390-MVS]
work, and I should have this prefixing
issue fixed in PDPCLIB soon.

Ok, the prefixing issue is now committed
to CVS as pdpclib-stage198 and can
http://pdos.cvs.sourceforge.net/viewvc/pdos/pdos/pdpclib/
Mike, let me know if you need this new
code, and if so, how you would like it
delivered.
I can give you a (beta) ASCII zip file of
the dev PDPCLIB, or I can give you the
latest dev GCCMVS package as a
zipped XMIT.
BFN. Paul.

[[e-mail] [H390-MVS]]

Post by Mike Rayborn ***@comcast.net [H390-MVS]
Not to dash anyone's efforts, but I don't really need code to obtain the
TSO PREFIX. IND$FILE doesn't obey the TSO PREFIX setting and never
has. It simply checks for a quoted or unquoted dataset name. If the
dataset name is not quoted, then the TSO userid is prefixed to the
dataset name. It's simple and it works as expected.

Can you tell me how you are getting
the TSO userid using GCCMVS?

Post by Mike Rayborn ***@comcast.net [H390-MVS]
My 2 cents on "extending" the PDPCLIB code base.

Note that I don't consider adding a prefix
to be an "extension". It's just a decent
C90 implementation.

Post by Mike Rayborn ***@comcast.net [H390-MVS]
If we/you/me wish to add new code to PDPCLIB, we must make sure to not
add these extensions to existing libraries, like STDLIB, STDIO, etc.
They *should* be new modules with appropriate headers so that developers
can choose to use or not use any of these extensions.

I don't provide any extensions to C90
at all. All functions starting with @@
(ie __ in C) are for internal use only,
so not considered to be extensions.

BFN. Paul.

[Greg Price [e-mail] [H390-MVS]]

Post by ***@comcast.net [H390-MVS]
With a 4K buffer and WSF style data streams sent/received via
TPUT/TGET, the TPUT calls seem to work as expected, however the TGET
with a 4K buffer fails and causes a terminal disconnect (not nice).

Hi Mike,

Yes, sounds like what used to happen to me with large screen sizes.

With TPUT, the application provides the data length, and so VTAM knows
how many buffers are required, and after suitable provisioning performs
long data stream writes successfully.

With TGET, VTAM does not know how much data will be sent inbound by the
terminal, and so when the buffer limit is exceeded VTAM will disconnect
the session.

I put up with this problem for many years (seemed like it, anyway) after
the failure of ZP60010 to fix it. I even wrote ZP60011 so I could
effectively do a CCW trace in an effort to try to track down the
problem, but to no avail.

Eventually someone who had more understanding of VTAM buffer parameter
settings than I did (somitcw IIRC) posted superior settings for VTAMLST,
and I haven't had a problem since.

I'll second JÃŒrgen's suggestion of downloading his TK4- package and
trying it out. If it works there and you want to go back to your
current system (which you may or may not after you try it out) you can
copy his VTAM settings to your system.

Here's what I run with in *SYS1.VTAMLST(ATCSTR00)*, but they may not be
the best settings around:

*CONFIG=00, **
**SSCPID=01, **
**NETSOL=YES, **
**MAXSUBA=31, **
**NOPROMPT, **
**SUPP=NOSUP, **
**COLD, **
**APBUF=(128,,064), **
**CRPLBUF=(256,,44), **
**IOBUF=(020,3992,10,F), **
**LFBUF=(016,,16,F), **
**LPBUF=(032,,32,F), **
**NPBUF=(032,,32,F), **
**PPBUF=(020,3992,10,F), **
**SFBUF=(032,,32,F), **
**SPBUF=(032,,32,F), **
**UECBUF=(32,,16,F), **
**WPBUF=(64,,64,F) *

Cheers,
Greg

[Mike Rayborn [e-mail] [H390-MVS]]

Thanks Greg.

My TPUT's with large buffers work just fine.

My TGET's with large buffers cause disconnects after a few attempts,
just like you described, so I'm betting this is a configuration or
maintenance issue.

I think the attack plan will be for me to move my build datasets to the
TK4 environment. If that works out, then we'll just say that TK4 (or
higher) is the expected MVS environment for the new IND$FILE release,
whenever that is :)

--Mike

Post by Greg Price ***@optusnet.com.au [H390-MVS]

Post by ***@comcast.net [H390-MVS]
With a 4K buffer and WSF style data streams sent/received via
TPUT/TGET, the TPUT calls seem to work as expected, however the TGET
with a 4K buffer fails and causes a terminal disconnect (not nice).

Hi Mike,
Yes, sounds like what used to happen to me with large screen sizes.
With TPUT, the application provides the data length, and so VTAM knows
how many buffers are required, and after suitable provisioning
performs long data stream writes successfully.
With TGET, VTAM does not know how much data will be sent inbound by
the terminal, and so when the buffer limit is exceeded VTAM will
disconnect the session.
I put up with this problem for many years (seemed like it, anyway)
after the failure of ZP60010 to fix it. I even wrote ZP60011 so I
could effectively do a CCW trace in an effort to try to track down the
problem, but to no avail.
Eventually someone who had more understanding of VTAM buffer parameter
settings than I did (somitcw IIRC) posted superior settings for
VTAMLST, and I haven't had a problem since.
I'll second JÃŒrgen's suggestion of downloading his TK4- package and
trying it out. If it works there and you want to go back to your
current system (which you may or may not after you try it out) you can
copy his VTAM settings to your system.
Here's what I run with in *SYS1.VTAMLST(ATCSTR00)*, but they may not
*CONFIG=00, **
**SSCPID=01, **
**NETSOL=YES, **
**MAXSUBA=31, **
**NOPROMPT, **
**SUPP=NOSUP, **
**COLD, **
**APBUF=(128,,064), **
**CRPLBUF=(256,,44), **
**IOBUF=(020,3992,10,F), **
**LFBUF=(016,,16,F), **
**LPBUF=(032,,32,F), **
**NPBUF=(032,,32,F), **
**PPBUF=(020,3992,10,F), **
**SFBUF=(032,,32,F), **
**SPBUF=(032,,32,F), **
**UECBUF=(32,,16,F), **
**WPBUF=(64,,64,F) *
Cheers,
Greg

[Graeme Houston [e-mail] [H390-MVS]]

[MTaylorIBM360 [e-mail] [H390-MVS]]

Hello Group,
I have been following the discussion on moving datasets using IND$FILE. When I had to ship any thing to a customer ( file, Loadlib etc ) I would use XMIT to create a 80/3120 file and move that to the PC binary mode. This can then be attached to email, provided it's not too large. I have used TERSE to create the same fixed format file, but I don't think that program is available

With XMIT files one can then use XMIT MANAGER to view the uploaded data.

Thats my two pen nth worth

Martin.

[Bert Lindeman [e-mail] [H390-MVS]]

Hi Martin,

You describe the practice I used most of the time.
XMIT -> into *.XM
Terse XM
Transfer binary to PC or linux server or such.
Transfer binary

Bert

Post by MTaylorIBM360 ***@yahoo.com [H390-MVS]
Hello Group,
I have been following the discussion on moving datasets using
IND$FILE. When I had to ship any thing to a customer ( file, Loadlib
etc ) I would use XMIT to create a 80/3120 file and move that to the
PC binary mode. This can then be attached to email, provided it's not
too large. I have used TERSE to create the same fixed format file, but
I don't think that program is available
With XMIT files one can then use XMIT MANAGER to view the uploaded data.
Thats my two pen nth worth
Martin.