John Brooks' Improvements to ProDOS KFest Session

[Hugh Hood]

John,

I just watched the YouTube video of your presentation on improving
ProDOS 8 from KansasFest 2017.

Great job!

For anyone else who might be interested, here's the general link to the
KansasFest session videos, of which there are several.

<https://www.youtube.com/user/KansasFest/videos>

I didn't hear many details on your server-side 230kB serial image
server. Can you share any information about whether it's written in C,
or Java, or something else, and the extent to which it will be platform
independent?

Thanks.




Hugh Hood

[John Brooks]

Post by Hugh Hood
John,
I just watched the YouTube video of your presentation on improving
ProDOS 8 from KansasFest 2017.
Great job!
For anyone else who might be interested, here's the general link to the
KansasFest session videos, of which there are several.
<https://www.youtube.com/user/KansasFest/videos>
I didn't hear many details on your server-side 230kB serial image
server. Can you share any information about whether it's written in C,
or Java, or something else, and the extent to which it will be platform
independent?
Thanks.
Hugh Hood

Hi Hugh.

The server is written in generic cross-platform C. The Mac executable size is currently 14KB.

xHD server uses the cross-platform libserialport library from the sigrok project to abstract the platform-specific complexities of serial over USB.

https://sigrok.org/wiki/Libserialport

My intention with the design and Mac implementation was that it should be small, simple and easy port to popular modern platforms.

-JB
@JBrooksBSI

[Hugh Hood]

Post by John Brooks
The server is written in generic cross-platform C. The Mac executable size is currently 14KB.

Very nice. I was hoping you'd say that.

I'm looking forward to using it.

Allow me to throw you a question from left field -- I frequently
manipulate the bits in the IIGS Slot Register at $C02D (SLTROMSEL) from
within programs (primarily AppleWorks) to switch between the built-in
SCC ports and plug-in cards, which are typically either serial or
parallel printer cards.

Assuming I'm using one of the built-in SCC ports with your server as a
virtual ProDOS drive, and then change the slot register to access the
plug-in card, will I destroy the server-client link to the point that it
will not recover when I toggle the slot register back to the SCC port?

Obviously, I couldn't change this _during_ virtual disk access, but
barring that, it is feasible?

Thanks.





Hugh Hood

[John Brooks]

Post by Hugh Hood

Post by John Brooks
The server is written in generic cross-platform C. The Mac executable size is currently 14KB.

Very nice. I was hoping you'd say that.
I'm looking forward to using it.
Allow me to throw you a question from left field -- I frequently
manipulate the bits in the IIGS Slot Register at $C02D (SLTROMSEL) from
within programs (primarily AppleWorks) to switch between the built-in
SCC ports and plug-in cards, which are typically either serial or
parallel printer cards.
Assuming I'm using one of the built-in SCC ports with your server as a
virtual ProDOS drive, and then change the slot register to access the
plug-in card, will I destroy the server-client link to the point that it
will not recover when I toggle the slot register back to the SCC port?
Obviously, I couldn't change this _during_ virtual disk access, but
barring that, it is feasible?
Thanks.
Hugh Hood

I frequently manipulate the bits in the IIGS Slot Register at $C02D (SLTROMSEL) from within programs (primarily AppleWorks) to switch between the built-in SCC ports and plug-in cards, which are typically either serial or parallel printer cards.

Heads-up that it's generally not safe to manipulate $C02D without rebooting as both the GS firmware and interface card firmware can trash each other's screen hole data, or generate interrupts, etc. If you know the slot has a disk drive with no interrupts and you save/restore the screen holes, it might be safe.

Post by Hugh Hood
Assuming I'm using one of the built-in SCC ports with your server as a

virtual ProDOS drive, and then change the slot register to access the
plug-in card, will I destroy the server-client link to the point that it
will not recover when I toggle the slot register back to the SCC port?

No, xHD does not use the GS SCC firmware or screen holes, so I recommend just leaving slot 2 to always be 'Your Card'. The xHD client driver in ProDOS directly accesses the SCC soft-switches at $C038-$C03B and is unaffected by $C02D.

Similarly, there is no keep-alive or 'chatting' in the current protocol. The GS driver initiates all communication via reads/writes block requests to the server.

It should be possible (but I haven't tested this), to have both xHD and a terminal/communications program share the modem port, assuming you have an external port switcher box.

The xHD client can detect that the GS firmware has reconfigured the modem port and if so, xHD will reset it to 230Kbps on the next block transfer. I'm not sure if the GS firmware will be as flexible about resetting the modem port to 9600 baud or whatever after xHD has done a transfer, but this might be something to test. Of course, the GS SCC firmware might crash and burn having to share time with another driver. :)

-JB

[Hugh Hood]

Post by John Brooks
Similarly, there is no keep-alive or 'chatting' in the current
protocol. The GS driver initiates all communication via reads/writes
block requests to the server.
The xHD client can detect that the GS firmware has reconfigured the
modem port and if so, xHD will reset it to 230Kbps on the next block
transfer.

Very good. I like the sound of both of those features. May I assume the
driver fits entirely within the space normally used by the Disk ][ driver?

On the server side -- how are you currently starting the server on your
Mac? Is it one of the Logon items, or a daemon of some sort?

Post by John Brooks
Heads-up that it's generally not safe to manipulate $C02D without
rebooting as both the GS firmware and interface card firmware can
trash each other's screen hole data, or generate interrupts, etc. If
you know the slot has a disk drive with no interrupts and you
save/restore the screen holes, it might be safe.

Thanks for that heads-up, as I'll investigate this further to determine
whether I've just been lucky in my use of the Slot Register, or whether
my use of discrete jobs avoids potential problems here. I've never
experienced any craziness when switching Slots 1 and 2 back and forth
between the built-in SCC and a parallel printer card without an
intervening reboot, and I've been doing it for many years. I'll see
whether saving and restoring the screen holes, as you suggested, might
be warranted in my case.




Hugh Hood

[John Brooks]

Post by Hugh Hood

Post by John Brooks
Similarly, there is no keep-alive or 'chatting' in the current
protocol. The GS driver initiates all communication via reads/writes
block requests to the server.
The xHD client can detect that the GS firmware has reconfigured the
modem port and if so, xHD will reset it to 230Kbps on the next block
transfer.

Very good. I like the sound of both of those features. May I assume the
driver fits entirely within the space normally used by the Disk ][ driver?
On the server side -- how are you currently starting the server on your
Mac? Is it one of the Logon items, or a daemon of some sort?

Post by John Brooks
Heads-up that it's generally not safe to manipulate $C02D without
rebooting as both the GS firmware and interface card firmware can
trash each other's screen hole data, or generate interrupts, etc. If
you know the slot has a disk drive with no interrupts and you
save/restore the screen holes, it might be safe.

Thanks for that heads-up, as I'll investigate this further to determine
whether I've just been lucky in my use of the Slot Register, or whether
my use of discrete jobs avoids potential problems here. I've never
experienced any craziness when switching Slots 1 and 2 back and forth
between the built-in SCC and a parallel printer card without an
intervening reboot, and I've been doing it for many years. I'll see
whether saving and restoring the screen holes, as you suggested, might
be warranted in my case.
Hugh Hood

The driver is about 2 pages and currently lives at $E0/BD00-$E0/BEDD. I haven't optimized for size yet and hope to get down to about half it's current size, to reduce the size of ProDOS and load time more than save GS memory in bank $E0.

Since the driver is in $E0, all the other ProDOS devices are unaffected and xHD data can be copied to/from floppy disks.

Post by Hugh Hood
On the server side -- how are you currently starting the server on your
Mac? Is it one of the Logon items, or a daemon of some sort?

Since I've just been using it for personal software-dev use so far, I've just been running it from the xcode project. It's currently a console application. A fellow kfest attendee has volunteered to help port it to linux and make it more complete, so it should at a faster clip than when I was the sole contributor.

Post by Hugh Hood
I've never experienced any craziness when switching Slots 1 and 2 back and forth

between the built-in SCC and a parallel printer card without an intervening reboot, and I've been doing it for many years.

It may be that the GS SCC firmware does not use the screen holes. If that is true, banking out the slot card firmware should be safe as long as the slot card does not generate an interrupt while it's card ROM is banked out.

Out of curiosity, what is your use-case for the $C02D swap? Slot 1 parallel port printer and SCC port A modem?

-JB
@JBrooksBSI

[Hugh Hood]

Post by John Brooks
The driver is about 2 pages and currently lives at $E0/BD00-$E0/BEDD. I haven't optimized for size yet and hope to get down to about half it's current size, to reduce the size of ProDOS and load time more than save GS memory in bank $E0.
Since the driver is in $E0, all the other ProDOS devices are unaffected and xHD data can be copied to/from floppy disks.

Very clever.

Post by John Brooks
Out of curiosity, what is your use-case for the $C02D swap? Slot 1 parallel port printer and SCC port A modem?

Remember, you asked. ;-)


I use both internal serial ports and (2) parallel printer cards.

Slot 1 Internal Serial Port:
--> Mac OS X Serial Terminal for Unix Work and Zmodem file transfers
(e.g. AppleWorks automatically 'logs on' to the Mac, 'prints'
documents using its PCL/LaserJet driver to the Mac where
the GhostScript/GhostPCL utility 'PCL6' automatically
converts the job to a text-searchable 'pdf' file on the
Mac's Desktop, after which AppleWorks 'logs off' the Mac).

Slot 1 Parallel Card (7-bit Epson APL):
--> Tractor-Fed Impact Printer for multi-part business forms
(e.g. Primarily checks - yes, I'm old-fashioned that way).


Slot 2 Internal Serial Port:
--> HP LaserJet 4050TN Printer with (3) bins + envelope feeder
for PCL and (some) PostScript print jobs

Slot 2 Parallel Card (7-bit Epson APL):
--> Same HP LaserJet Printer when I need a quick way to disregard
printing the high bit
(e.g. from BASIC and on occasion an old CAD program I have,
as the LaserJet also understands HPGL).


Some 'trivia' for those also using the Slot Register ($C02D) in this
fashion:

In Bank $E1, Page 2 ($E102Cx) the IIGS stores the 'specific' setting for
Slots 1 and 2 (e.g. Printer/Modem/AppleTalk/Your Card).

_IF_ you start out with one of the Slots set to 'Your Card' and then you
use the Slot Register to change to the Internal Port, you must also Poke
in the appropriate value at $E102Cx before initializing the built-in
serial firmware, or else the IIGS gets very confused. I documented this
in a post several years ago (with help from Antoine Vignau, IIRC), but I
don't have the reference handy. FWIW, I have an AppleWorks macro that
takes care of it for me.


I'm sure that by now you (and everyone else) have regretted asking me. ;-)





Hugh Hood

[Jorge]

Post by Hugh Hood
John,
I just watched the YouTube video of your presentation on improving
ProDOS 8 from KansasFest 2017.

I can't find that video, which one is it, exactly?

[John Brooks]

Post by Jorge

Post by Hugh Hood
John,
I just watched the YouTube video of your presentation on improving
ProDOS 8 from KansasFest 2017.

I can't find that video, which one is it, exactly?
--
Jorge.

The high-quality video & audio should go up on archive.org in a few days.

Here is the low-quality livestream video:

-JB

[Jorge]

Post by John Brooks
The high-quality video & audio should go up on archive.org in a few days.
http://youtu.be/ZS7yrqNV1YM

Thanks!

[Michael 'AppleWin Debugger Dev']

Thanks for the link John. Great talk -- lots of great info and ideas!

One detail that you didn't discuss was how exactly are you patching ProDOS -- are you doing a binary patch? Or are you working off a set of asm (source) files?

[John Brooks]

Post by Michael 'AppleWin Debugger Dev'
Thanks for the link John. Great talk -- lots of great info and ideas!
One detail that you didn't discuss was how exactly are you patching ProDOS -- are you doing a binary patch? Or are you working off a set of asm (source) files?

This was discussed (briefly) in the Q&A after the talk:

http://youtu.be/ZS7yrqNV1YM

-JB
@JBrooksBSI

[Michael 'AppleWin Debugger Dev']

Ah! I _knew_ I should have watched the Q&A. :-)

When do you plan to make the source available? After v2.5 ?

[D Finnigan]

Post by Michael 'AppleWin Debugger Dev'
Ah! I _knew_ I should have watched the Q&A. :-)
When do you plan to make the source available? After v2.5 ?

If he's smart, he'll never release the source. Same with the people doing
the GS/OS System 6.0.x updates. Apple Computer may have officially released
DOS 3.x source code, but I don't believe they've ever sanctioned ProDOS
source public release.

[Michael 'AppleWin Debugger Dev']

Couldn't we the community petition Steve Woz to ask Apple to donate the ProDOS source into the Public Domain? Or at least allow it to be distributed for educational / non-commercial use?

It is not like Apple is _ever_ going to release any code for a 8-bit system that it stopped selling decades ago.

What is the value in hoarding the rights to a "dead" Operating System???

[Steven Hirsch]

Post by Michael 'AppleWin Debugger Dev'
Couldn't we the community petition Steve Woz to ask Apple to donate the ProDOS source into the Public Domain? Or at least allow it to be distributed for educational / non-commercial use?
It is not like Apple is _ever_ going to release any code for a 8-bit system that it stopped selling decades ago.
What is the value in hoarding the rights to a "dead" Operating System???

Probably no value, but the safe, default answer from overworked IP legal staff
is always "no" and that's as far as it will ever get.

[Jorge]

Post by Steven Hirsch

Post by Michael 'AppleWin Debugger Dev'
Couldn't we the community petition Steve Woz to ask Apple to donate the ProDOS source into the Public Domain? Or at least allow it to be distributed for educational / non-commercial use?
It is not like Apple is _ever_ going to release any code for a 8-bit system that it stopped selling decades ago.
What is the value in hoarding the rights to a "dead" Operating System???

Probably no value, but the safe, default answer from overworked IP legal staff
is always "no" and that's as far as it will ever get.

Is it illegal to modify/improve something and publish the modifications?

[Michael 'AppleWin Debugger Dev']

Post by Jorge
Is it illegal to modify/improve something and publish the modifications?

No -- as long as you publish ONLY the changes.
How do you think Ms. Pacman came about? It was a _hardware mod_ to Pacman!

https://www.fastcompany.com/3067296/retrology/the-mit-dropouts-who-created-ms-pac-man-a-35th-anniversary-oral-history

[Jorge]

Post by Michael 'AppleWin Debugger Dev'

Post by Jorge
Is it illegal to modify/improve something and publish the modifications?

No -- as long as you publish ONLY the changes.
How do you think Ms. Pacman came about? It was a _hardware mod_ to Pacman!
https://www.fastcompany.com/3067296/retrology/the-mit-dropouts-who-created-ms-pac-man-a-35th-anniversary-oral-history

Then is it illegal to disassemble a binary and publish that? Are you sure?

[Michael 'AppleWin Debugger Dev']

Post by Jorge
Then is it illegal to disassemble a binary and publish that?

That is not illegal -- with caveats on how you are defining "publish"; I'm assuming you are using the term as in "writing"?

1) Reverse Engineering and
2) Documenting your findings

are 100% legal, protected rights in the US. (All though companies try to pretend you don't have them with the DMCA abuse/shenanigans. /sarcasm Because breaking encryption is such as crime against humanity^H^H^H^ corporations ^H^H^H profits. )

It _may_ be illegal to share a number, aka, distribute the binary. It depends on who "owns" the copyright, and what rights are given to you.

The MIT guys were able to distribute (and sell) their Ms. Pacman ROMs because it required the owner to have a legal copy of Pacman first. All they were doing was "patching" the _existing_ ROM to now have their own set of numbers -- of which they owned the copyright to distribute. How the owner got a legal copy of Pacman was not their problem.

i.e.
You can legally distribute public domain binaries.
You can legally distribute GPL/BSD binaries.
You can legally distribute "shareware" binaries.

Whether the software was commercially produced or not is a red herring. i.e. Red Hat publishes Red Hat Linux and that has always been free to distribute. Just because a company produced it doesn't imply that it is legal to share, nor illegal to share -- all through the later is probably the safer default position. You can't assume anything -- you always have to verify what is being "Licensed" vs what is being "Sold", if anything.

i.e.
You can distribute proprietary programs once the company/author has given permission to do so. There are many out-dated commercial programs that companies have made "freeware" long after they served their purpose of making money. You can find a list here:

https://en.wikipedia.org/wiki/List_of_formerly_proprietary_software

That is why it is important that we are legally allowed to preserve our computing past. Holding 1980's programs "hostage" to due to shitty copyright laws is destroying culture -- along with disks. This concept of "Greed before Need" is a cancer to society.

I, and many others, would argue that if a company have not been selling a program for 25 years then it should default to public domain -- because the only value it has is to a) fans, and b) historians -- obviously it has no commercial value or they would still be selling it!

Does anyone actually care that a few old farts are distributing Operating Systems, Games, etc. that are almost 30 years old and haven't been commercially available for decades? No, except for scumbag lawyers. The rest of us are getting on with our life and enjoying the software that we grew up with.

The law is completely out of touch with reality -- and practicality.

This is why we need someone famous, like Steve Woz, to ask Apple to release all copyrights over all Apple 2 software for the benefit of everyone. We are going to keep distributing binaries regardless of their permission but it would be nice to be in "good legal standing" instead of having to worry about asinine and corrupt laws.

[Jorge]

Post by Michael 'AppleWin Debugger Dev'

Post by Jorge
Then is it illegal to disassemble a binary and publish that?

That is not illegal -- with caveats on how you are defining "publish"; I'm assuming you are using the term as in "writing"?
1) Reverse Engineering and
2) Documenting your findings
are 100% legal, protected rights in the US. (All though companies try to pretend you don't have them with the DMCA abuse/shenanigans. /sarcasm Because breaking encryption is such as crime against humanity^H^H^H^ corporations ^H^H^H profits. )
It _may_ be illegal to share a number, aka, distribute the binary. It depends on who "owns" the copyright, and what rights are given to you.
The MIT guys were able to distribute (and sell) their Ms. Pacman ROMs because it required the owner to have a legal copy of Pacman first. All they were doing was "patching" the _existing_ ROM to now have their own set of numbers -- of which they owned the copyright to distribute. How the owner got a legal copy of Pacman was not their problem.
i.e.
You can legally distribute public domain binaries.
You can legally distribute GPL/BSD binaries.
You can legally distribute "shareware" binaries.
Whether the software was commercially produced or not is a red herring. i.e. Red Hat publishes Red Hat Linux and that has always been free to distribute. Just because a company produced it doesn't imply that it is legal to share, nor illegal to share -- all through the later is probably the safer default position. You can't assume anything -- you always have to verify what is being "Licensed" vs what is being "Sold", if anything.
i.e.
https://en.wikipedia.org/wiki/List_of_formerly_proprietary_software
That is why it is important that we are legally allowed to preserve our computing past. Holding 1980's programs "hostage" to due to shitty copyright laws is destroying culture -- along with disks. This concept of "Greed before Need" is a cancer to society.
I, and many others, would argue that if a company have not been selling a program for 25 years then it should default to public domain -- because the only value it has is to a) fans, and b) historians -- obviously it has no commercial value or they would still be selling it!
Does anyone actually care that a few old farts are distributing Operating Systems, Games, etc. that are almost 30 years old and haven't been commercially available for decades? No, except for scumbag lawyers. The rest of us are getting on with our life and enjoying the software that we grew up with.
The law is completely out of touch with reality -- and practicality.
This is why we need someone famous, like Steve Woz, to ask Apple to release all copyrights over all Apple 2 software for the benefit of everyone. We are going to keep distributing binaries regardless of their permission but it would be nice to be in "good legal standing" instead of having to worry about asinine and corrupt laws.

I wholeheartedly agree with everything you've written above!

Has any lawyer said anything about all those .dsk images of DOS and PRODOS (and etc., I mean, all the other images of Apple's Apple II software floating on the internet), ever?

If after all this time they have not, it perhaps means we are safe :-)

[James Davis]

Post by Michael 'AppleWin Debugger Dev'

Post by Jorge
Then is it illegal to disassemble a binary and publish that?

That is not illegal -- with caveats on how you are defining "publish"; I'm assuming you are using the term as in "writing"?
1) Reverse Engineering and
2) Documenting your findings
are 100% legal, protected rights in the US. (All though companies try to pretend you don't have them with the DMCA abuse/shenanigans. /sarcasm Because breaking encryption is such as crime against humanity^H^H^H^ corporations ^H^H^H profits. )
It _may_ be illegal to share a number, aka, distribute the binary. It depends on who "owns" the copyright, and what rights are given to you.
The MIT guys were able to distribute (and sell) their Ms. Pacman ROMs because it required the owner to have a legal copy of Pacman first. All they were doing was "patching" the _existing_ ROM to now have their own set of numbers -- of which they owned the copyright to distribute. How the owner got a legal copy of Pacman was not their problem.
i.e.
You can legally distribute public domain binaries.
You can legally distribute GPL/BSD binaries.
You can legally distribute "shareware" binaries.
Whether the software was commercially produced or not is a red herring. i.e. Red Hat publishes Red Hat Linux and that has always been free to distribute. Just because a company produced it doesn't imply that it is legal to share, nor illegal to share -- all through the later is probably the safer default position. You can't assume anything -- you always have to verify what is being "Licensed" vs what is being "Sold", if anything.
i.e.
https://en.wikipedia.org/wiki/List_of_formerly_proprietary_software
That is why it is important that we are legally allowed to preserve our computing past. Holding 1980's programs "hostage" to due to shitty copyright laws is destroying culture -- along with disks. This concept of "Greed before Need" is a cancer to society.
I, and many others, would argue that if a company have not been selling a program for 25 years then it should default to public domain -- because the only value it has is to a) fans, and b) historians -- obviously it has no commercial value or they would still be selling it!
Does anyone actually care that a few old farts are distributing Operating Systems, Games, etc. that are almost 30 years old and haven't been commercially available for decades? No, except for scumbag lawyers. The rest of us are getting on with our life and enjoying the software that we grew up with.
The law is completely out of touch with reality -- and practicality.
This is why we need someone famous, like Steve Woz, to ask Apple to release all copyrights over all Apple 2 software for the benefit of everyone. We are going to keep distributing binaries regardless of their permission but it would be nice to be in "good legal standing" instead of having to worry about asinine and corrupt laws.

Speculation: I think that Apple Computer, Inc.'s Lawyers just don't want to create any Precedence in Law to support Copyright Infringement. They probably don't really care about what people are doing with their 25-40 year old software. If they did, they would be wasting their money pursuing anyone and everyone who is doing it, e.g., like the music industry does.

[Jorge]

Post by John Brooks
http://youtu.be/ZS7yrqNV1YM

Now that I've seen it (even the Q&A :-)) I can only say: WOW AMAZING !

I always hated PRODOS, for several reasons:

1.- It came too late (because Apple would have liked to kill the II line)
2.- The silly short file names
3.- After a couple of versions the dropped support for the IIs (only for IIe/IIc and better).

I said thank you, but no, thank you. I wanted to keep my II as it was, didn't want to ditch it for anything else.

Now the "John Brooks" PRODOS is something different, I appreciate very much that you've made it 6502 and Apple II compatible, I love that, thanks !!!

[Jorge]

Post by John Brooks
The high-quality video & audio should go up on archive.org in a few days.

By the way, archive.org is the best thing ever...

[Antoine Vignau]

I've always liked ProDOS because it is fast, reliable, supports hierarchical volumes, supports HDD and other "modern" devices!

1. It came in 83/84
2. Agree
3. Stay with v1.9 then if you are a II/II+ users. 65c02 kits were available for the IIe

Antoine

[Jorge]

Post by Antoine Vignau
I've always liked ProDOS because it is fast, reliable, supports hierarchical volumes, supports HDD and other "modern" devices!
1. It came in 83/84

Yeah, that, exactly that... I got my Disk II in 1978, then they completely "forgot" us while busy making the Apple III SOS, the Lisa and the Mac... huh. Those silly guys still using the II? Fuck them.

[Jorge]

Post by Antoine Vignau
I've always liked ProDOS because it is fast

And there were "fast" DOSes too !!!