Discussion:
[gem5-dev] Let's retire some ISAs
Andreas Sandberg
2016-06-03 16:28:39 UTC
Permalink
Hi Everyone,

You have probably all heard my complaining about tests taking /ages/ to
run. One way we could reduce overhead (mainly at compile time) would be to
retire some of the poorly supported ISAs. This would serve three purposes:

1. It reduces overhead for developers.

2. It reduces the cost of testing if/when we move regression tests to a
cloud solution. One of the problems poorly supported architecture pose is
that we still need to compile them in three different modes whenever we
run regressions. This consumes many cores hours (== money).

3. It reduces confusion among users. Having a handful of architectures
that are clearly incomplete and don¹t support full-system is not very
helpful to anyone.

I would propose that we plan to phase out the architectures based on the
following criteria:

1. Completeness: We should phase out ISAS that don¹t support full-system
unless there is a clear plan to add that support. (The NULL ISA is an
exception) Users expect to be able to run an OS if an ISA is supported. In
practice, most ISAs in this category probably lack good SE mode support
since they have few or no users.

2. Ecosystem: We should phase out ISAs that don¹t have compiler support.
Being able to run code isn¹t very useful if you can¹t compile code for the
platform. Using old compilers isn¹t very helpful either since that is
/very/ likely going to lead to meaningless performance results.

3. Maintainers: We should /consider/ phasing out ISAs where there is no
clear maintainer. Only ISAs with clear maintainers are likely to be even
close to functionally correct.


Currently, ALPHA, POWER, and MIPS fail at least two of these criteria.
SPARC is borderline. You could argue that SPARC fails 1 since OS boot
isn¹t normally tested. I would suggest that these architectures are
scheduled for a two step phase out.

Short term, we phase out POWER and MIPS. They don¹t have clear maintainers
or users submitting patches for them and they lack full-system support.
They won¹t be missed.

Medium to long term, we plan to phase out ALPHA within the next 6 months
and phase out SPARC unless someone steps up and want to care for it
long-term. For SPARC to stay, it needs to have at least one working
full-system test that can be redistributed (i.e., not Solaris).

Thoughts or objections?

Cheers,
Andreas



IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Jakub Jermář
2016-06-03 18:42:52 UTC
Permalink
Hi,
Post by Andreas Sandberg
and phase out SPARC unless someone steps up and want to care for it
long-term. For SPARC to stay, it needs to have at least one working
full-system test that can be redistributed (i.e., not Solaris).
The thought of losing the only existing and basically working open
source sun4v simulator is simply unbareable :-) I will try to put
together a HelenOS testcase for SPARC.

Jakub
Andreas Sandberg
2016-06-03 22:27:05 UTC
Permalink
On 03/06/2016, 19:42, "gem5-dev on behalf of Jakub Jermář"
Post by Jakub Jermář
Hi,
Post by Andreas Sandberg
and phase out SPARC unless someone steps up and want to care for it
long-term. For SPARC to stay, it needs to have at least one working
full-system test that can be redistributed (i.e., not Solaris).
The thought of losing the only existing and basically working open
source sun4v simulator is simply unbareable :-) I will try to put
together a HelenOS testcase for SPARC.
Fair enough. If we have at least one full-system test case and someone who
cares about it, we should keep it.

//Andreas

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Jakub Jermář
2016-06-06 20:05:23 UTC
Permalink
Post by Andreas Sandberg
On 03/06/2016, 19:42, "gem5-dev on behalf of Jakub Jermář"
Post by Jakub Jermář
Post by Andreas Sandberg
and phase out SPARC unless someone steps up and want to care for it
long-term. For SPARC to stay, it needs to have at least one working
full-system test that can be redistributed (i.e., not Solaris).
The thought of losing the only existing and basically working open
source sun4v simulator is simply unbareable :-) I will try to put
together a HelenOS testcase for SPARC.
Fair enough. If we have at least one full-system test case and someone who
cares about it, we should keep it.
In the meantime, I've put together a HelenOS branch that boots halfway
through the init process (involves the kernel code and about a dozen
userspace servers) and then issues the m5_exit instruction to abort the
simulation. In the future, it should be possible to issue the m5_exit
later when all the pending issues are resolved. The branch also comes
with an easy-to-use script to start the simulation.

In general, one should at least make the following steps:

1. clone the gem5-sparc-fs-regression-test repository:
======================================================
$ bzr clone lp:~jakub/helenos/gem5-sparc-fs-regression-test
$ cd gem5-sparc-fs-regression-test


2. prepare a sparc64 toolchain for building HelenOS/sparc64:
============================================================
$ cd tools/
$ sudo ./toolchain.sh sparc64


3. build gem5 for HelenOS:
==========================
$ cd ../contrib/gem5
$ ./build-from-scratch.sh


4. build HelenOS:
=================
$ cd ../..
$ make all HANDS_OFF=y PROFILE=sparc64/niagara


With everything in place, one just types:

M5_PATH=/path/to/gem5/installation/ tools/ew.py

and waits for the simulation to abort on m5_exit.

As for the next steps, I plan to look into the gem5 regression testing
framework and try to integrate the above with it.

Cheers,
Jakub
S***@labware.com
2016-06-06 18:24:20 UTC
Permalink
Hi Andreas,
Post by Andreas Sandberg
POWER and MIPS.
[...]
They won't be missed.
This is certainly not so here in my project where POWER and MIPS
are the primary available options, and dropping them would be pretty
much equivalent to discontinuing GEM5 altogether.

What feels even more sad, is that I can see how the very presence of
this discussion within the Overton window could be used by certain
closed-source advocates to undermine our "community support" argument
and thus try to weaken the position of not only our choice of GEM5
but some neighbouring open-source choices as well.
Post by Andreas Sandberg
users submitting patches for them
This one should easily exclude MIPS and POWER from candidates for
phasing out, well at least I do submit MIPS and POWER patches which
get merged into origin.
Post by Andreas Sandberg
Ecosystem: We should phase out ISAs that don't have compiler
support. Being able to run code isn't very useful if you can't
compile code for the platform.
True; and the above's implied connection to MIPS and/or POWER, sounds
disturbing to me. Unless we assume a Redmond-like position that there
is one OS and one compiler and one software stack (based on an 80%
bell curve argument?), different people do different things and
so the definition of "compiler support" varies from project to
project. For example, in retargetable compilers (which is what I am
interested in) there is more than a single project based on ArchC
processor descriptions, and in that world, it's pretty much a
three-ISA world of ARM, MIPS and POWER. I certainly hope SPARC
to be equally usable soon too, but right now it's just barely limping;
Alpha, yes there is a one-in-a-million chance; RISC-V, that's a very
sweet dream indeed; and x86 seems even less likely.
So yeah, three ISAs with compiler support that I could use, and like
I said above, losing two of the three would not be much different
from giving up on GEM5 altogether.
Post by Andreas Sandberg
We should phase out ISAS that don't support full-system
unless there is a clear plan to add that support.
(The NULL ISA is an exception)
Users expect to be able to run an OS if an ISA is supported.
From my chair it appears like this:
In another kingdom far away, there are those other people doing
those other extremely exciting things (including FS).

For me, the catastrophic showstopper was broken remote debugging
(gdb failing to even initiate the session) on every platform I tried,
so I had to fix that. This was the majority of the defining
discriminator between "working" and "not working"; FS didn't even
enter into the picture.

On the other hand, I do completely agree with you about functional
verification: I need to run a tall enough software stack (such as an
OS kernel) to believe in the simulation. But I can have enormous
stacks completely in SE. E.g. Java would serve as a good illustration
(not that's what we use here) of such a huge SW stack having no
relation to FS. I guess our situation here, is similar across the
board: there is Jakub's system on SPARC which is obviously tall enough
but not quite ready yet; then there is our stack which isn't there
yet; and then I wouldn't be surprised to learn about other people
with nontrivial workloads which are not the FS Linux kernel.
Post by Andreas Sandberg
It reduces confusion among users. Having a handful of architectures
that are clearly incomplete and don't support full-system is not
very helpful to anyone.
Does enabling projects which result in new compilers being written
and papers being published in journals, count as helpful?
A number of these things definitely wouldn't have happened if GEM5
lacked MIPS and PPC.

Boris
Beckmann, Brad
2016-06-06 19:05:27 UTC
Permalink
Hi Andreas,

Though I work for a company that only produces ARM/x86-compatible processors and I do live in Redmond, WA :), I am firmly in the camp that we should not eliminate any ISAs. The gem5 simulator is a tremendously valuable tool to the entire community, not just ARM and AMD. We need to maintain whatever features we can to sustain and grow our user and developer base.

As others have already pointed out, there is still significant industry support for MIPS, SPARC, and POWER. Perhaps an argument could be made against ALPHA, but how hard is that ISA to maintain? Also there is some historical significance to ALPHA, since it was the first ISA. As Boris just pointed out, if anything, we should actively be encouraging someone to add RISC-V. We only drive people away from the simulator when we discuss removing features.

Brad


-----Original Message-----
From: gem5-dev [mailto:gem5-dev-***@gem5.org] On Behalf Of ***@labware.com
Sent: Monday, June 06, 2016 11:24 AM
To: gem5 Developer List <gem5-***@gem5.org>
Subject: Re: [gem5-dev] Let's retire some ISAs

Hi Andreas,
Post by Andreas Sandberg
POWER and MIPS.
[...]
They won't be missed.
This is certainly not so here in my project where POWER and MIPS are the primary available options, and dropping them would be pretty much equivalent to discontinuing GEM5 altogether.

What feels even more sad, is that I can see how the very presence of this discussion within the Overton window could be used by certain closed-source advocates to undermine our "community support" argument and thus try to weaken the position of not only our choice of GEM5 but some neighbouring open-source choices as well.
Post by Andreas Sandberg
users submitting patches for them
This one should easily exclude MIPS and POWER from candidates for phasing out, well at least I do submit MIPS and POWER patches which get merged into origin.
Post by Andreas Sandberg
Ecosystem: We should phase out ISAs that don't have compiler support.
Being able to run code isn't very useful if you can't compile code for
the platform.
True; and the above's implied connection to MIPS and/or POWER, sounds disturbing to me. Unless we assume a Redmond-like position that there is one OS and one compiler and one software stack (based on an 80% bell curve argument?), different people do different things and so the definition of "compiler support" varies from project to project. For example, in retargetable compilers (which is what I am interested in) there is more than a single project based on ArchC processor descriptions, and in that world, it's pretty much a three-ISA world of ARM, MIPS and POWER. I certainly hope SPARC to be equally usable soon too, but right now it's just barely limping; Alpha, yes there is a one-in-a-million chance; RISC-V, that's a very sweet dream indeed; and x86 seems even less likely.
So yeah, three ISAs with compiler support that I could use, and like I said above, losing two of the three would not be much different from giving up on GEM5 altogether.
Post by Andreas Sandberg
We should phase out ISAS that don't support full-system unless there
is a clear plan to add that support.
(The NULL ISA is an exception)
Users expect to be able to run an OS if an ISA is supported.
From my chair it appears like this:
In another kingdom far away, there are those other people doing those other extremely exciting things (including FS).

For me, the catastrophic showstopper was broken remote debugging (gdb failing to even initiate the session) on every platform I tried, so I had to fix that. This was the majority of the defining discriminator between "working" and "not working"; FS didn't even enter into the picture.

On the other hand, I do completely agree with you about functional
verification: I need to run a tall enough software stack (such as an OS kernel) to believe in the simulation. But I can have enormous stacks completely in SE. E.g. Java would serve as a good illustration (not that's what we use here) of such a huge SW stack having no relation to FS. I guess our situation here, is similar across the
board: there is Jakub's system on SPARC which is obviously tall enough but not quite ready yet; then there is our stack which isn't there yet; and then I wouldn't be surprised to learn about other people with nontrivial workloads which are not the FS Linux kernel.
Post by Andreas Sandberg
It reduces confusion among users. Having a handful of architectures
that are clearly incomplete and don't support full-system is not very
helpful to anyone.
Does enabling projects which result in new compilers being written and papers being published in journals, count as helpful?
A number of these things definitely wouldn't have happened if GEM5 lacked MIPS and PPC.

Boris

_______________________________________________
gem5-dev mailing list
gem5-***@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
Bjoern A. Zeeb
2016-06-06 20:27:26 UTC
Permalink
Hi,
Post by Beckmann, Brad
As others have already pointed out, there is still significant industry support for MIPS, SPARC, and POWER. Perhaps an argument could be made against ALPHA, but how hard is that ISA to maintain? Also there is some historical significance to ALPHA, since it was the first ISA. As Boris just pointed out, if anything, we should actively be encouraging someone to add RISC-V. We only drive people away from the simulator when we discuss removing features.
I looked at MIPS (FS) support a few months back; it’s just not there; I mean literally, what is there is not even a start to get it to working in any reasonable timeframe. I’d love to see 64bit MIPS support but that’s quite a lot of man hours.

One thing people need to remember however is that retiring (currently) unusable code does not mean that the code disappears, it sits in attic and could be brought back; the effort to get it working is needed now and will be then. The question simply is how likely is it going to happen and when does the cost of keeping it around by far exceed the cost of resurrecting it.

My experience with x86_64 on non-Linux was not entirely happy the last 9 months, that I’d rather wish the “advertising” would have been correct or I might have never gone down that path; you need to be realistic in terms of what people can expect and I’d rather have 3 properly working ISAs than 7 out of which I waste a year of time to get anywhere. In that way I think Andreas is right: you need at least 3 people per ISA that feel responsible and commit themselves to (a) review patches and get them in, (b) keep enough Open Source bits available for tests and reproducibility, and (c) fee actively responsible for further development, documentation, keeping things going.

Just my 3cts
Bjoern
Andreas Hansson
2016-06-06 20:49:48 UTC
Permalink
Hi all,

Firstly: Jakub, great work getting HelenOS going, and Boris many thanks
for helping things along.

Secondly: No one is suggesting removing things for the sake of removing
things. It is simply software-development hygiene.

I hope we all agree that unused or non-working parts of the code base are
not bringing value to anyone (in their current form), and are merely
slowing down the progress of future _useful_ changes and additions. If we
can identify and remove any unused or non-working code it benefits
everyone: less confusion, less maintenance, and less testing. Removing
stale code also gives more freedom to tune things for what is really
working, and more focus on things that actually _do_ add value.

More half-supported ISAs is clearly _not_ equivalent to a more useful
simulation tool. I actually feel sorry for those users that struggle
unnecessarily, similar to what Bjoern describes. The key question is what
should be removed? Suggestions are welcome.


As Bjoern points out, if someone feels compelled to put in the effort the
code is still available as part of the repo and can be re-surrected.

Andreas

On 06/06/2016, 21:27, "gem5-dev on behalf of Bjoern A. Zeeb"
Post by Bjoern A. Zeeb
Hi,
Post by Beckmann, Brad
As others have already pointed out, there is still significant industry
support for MIPS, SPARC, and POWER. Perhaps an argument could be made
against ALPHA, but how hard is that ISA to maintain? Also there is some
historical significance to ALPHA, since it was the first ISA. As Boris
just pointed out, if anything, we should actively be encouraging someone
to add RISC-V. We only drive people away from the simulator when we
discuss removing features.
I looked at MIPS (FS) support a few months back; it’s just not there; I
mean literally, what is there is not even a start to get it to working in
any reasonable timeframe. I’d love to see 64bit MIPS support but that’s
quite a lot of man hours.
One thing people need to remember however is that retiring (currently)
unusable code does not mean that the code disappears, it sits in attic
and could be brought back; the effort to get it working is needed now
and will be then. The question simply is how likely is it going to
happen and when does the cost of keeping it around by far exceed the cost
of resurrecting it.
My experience with x86_64 on non-Linux was not entirely happy the last 9
months, that I’d rather wish the “advertising” would have been correct or
I might have never gone down that path; you need to be realistic in terms
of what people can expect and I’d rather have 3 properly working ISAs
than 7 out of which I waste a year of time to get anywhere. In that way
I think Andreas is right: you need at least 3 people per ISA that feel
responsible and commit themselves to (a) review patches and get them in,
(b) keep enough Open Source bits available for tests and reproducibility,
and (c) fee actively responsible for further development, documentation,
keeping things going.
Just my 3cts
Bjoern
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Steve Reinhardt
2016-06-07 16:27:34 UTC
Permalink
On Mon, Jun 6, 2016 at 1:27 PM Bjoern A. Zeeb <
Post by Bjoern A. Zeeb
One thing people need to remember however is that retiring (currently)
unusable code does not mean that the code disappears, it sits in attic and
could be brought back; the effort to get it working is needed now and will
be then. The question simply is how likely is it going to happen and when
does the cost of keeping it around by far exceed the cost of resurrecting
it.
I agree with Bjoern's last sentence about it being an issue of likelihoods
and relative costs. I think it's worth noting that (1) the decision to
retire an ISA will have a (possibly large) impact on the likelihood of it
being used or improved, and (2) the cost of incremental updates as changes
are made to keep something from regressing functionally is much lower than
the cost of deferring those same updates to some future point in time.

As far as #1: As long as an ISA is available in the head, and functional
enough to pass the current regression tests, a potential user may come
along and find that the existing level of support is good enough for them,
or close enough that it's worth their effort to make it so. (And it's this
latter case that's the only hope for these ISAs to move forward.)

Once the bar for reaching this point is raised by adding the steps of
determining that gem5 *used to* support the ISA of interest, finding and
extracting the old revision that still had that support in it, and very
likely needing to do who knows how much work to bring the ISA support back
in sync with the head of the repo before being able to reasonably make
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.

Which leads to #2: Basically the only way you win by retiring code is if
that code is never resurrected. I'd say the lower bound on the cost of
resurrecting the code is the same as the incremental cost of keeping the
code alive, and that's a very optimistic lower bound. In practice, whoever
faces the task of resurrecting the code won't know which past changes
actually broke things, won't be the person who made those changes and thus,
once the guilty changes are identified, will have to reverse-engineer them
to see how to apply them to the old ISA, and will be faced with the task of
fixing a number of things en masse rather than one at a time. In addition,
there's a good chance that someone considering taking this on is new to
gem5 and unfamiliar with the code to begin with, unlike the person who is
committing patches and could perform the incremental updates.

Please note that I'm not saying we should never retire anything; I just
don't want us to use the "well we can always resurrect it from the repo"
argument to take the decision more lightly than we ought.

It would also be nice to keep in mind that some of the issues raised here
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about the
level of support for various ISAs:
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that information
up to date and maybe feature it more prominently so that new users are
aware of it. Perhaps less-supported ISAs could print a warning and a link
to these pages when they are used.

Steve
Andreas Sandberg
2016-06-08 15:12:37 UTC
Permalink
On 07/06/2016, 17:27, "gem5-dev on behalf of Steve Reinhardt"
Post by Steve Reinhardt
On Mon, Jun 6, 2016 at 1:27 PM Bjoern A. Zeeb <
Post by Bjoern A. Zeeb
One thing people need to remember however is that retiring (currently)
unusable code does not mean that the code disappears, it sits in attic and
could be brought back; the effort to get it working is needed now and will
be then. The question simply is how likely is it going to happen and when
does the cost of keeping it around by far exceed the cost of
resurrecting
it.
I agree with Bjoern's last sentence about it being an issue of likelihoods
and relative costs. I think it's worth noting that (1) the decision to
retire an ISA will have a (possibly large) impact on the likelihood of it
being used or improved, and (2) the cost of incremental updates as changes
are made to keep something from regressing functionally is much lower than
the cost of deferring those same updates to some future point in time.
As far as #1: As long as an ISA is available in the head, and functional
enough to pass the current regression tests, a potential user may come
along and find that the existing level of support is good enough for them,
or close enough that it's worth their effort to make it so. (And it's this
latter case that's the only hope for these ISAs to move forward.)
Once the bar for reaching this point is raised by adding the steps of
determining that gem5 *used to* support the ISA of interest, finding and
extracting the old revision that still had that support in it, and very
likely needing to do who knows how much work to bring the ISA support back
in sync with the head of the repo before being able to reasonably make
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your the
conclusion.

For complex work, say reworking register interfaces (e.g., Nathanael¹s
patch), modifying every single ISA adds a significant development
overhead. There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support". There
are two things we could do to improve the situation without completely
retiring ISAs:

* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core framework
once instead of for each ISA/Ruby configuration.

* We should get rid of long tests for unsupported ISAs and make the
remaining tests functional only (i.e., not diff stats or out files). This
would make stat refreshes (which pretty much every single non-trivial
change requires) much more manageable. I¹m planning to do similar things
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code is if
that code is never resurrected. I'd say the lower bound on the cost of
resurrecting the code is the same as the incremental cost of keeping the
code alive, and that's a very optimistic lower bound. In practice, whoever
faces the task of resurrecting the code won't know which past changes
actually broke things, won't be the person who made those changes and thus,
once the guilty changes are identified, will have to reverse-engineer them
to see how to apply them to the old ISA, and will be faced with the task of
fixing a number of things en masse rather than one at a time. In addition,
there's a good chance that someone considering taking this on is new to
gem5 and unfamiliar with the code to begin with, unlike the person who is
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal. The
likelihood that someone will ever need it for research seems very low and
having to resurrect it seems extremely unlikely. A simulator such as SimH
seems more suitable for what can only be described as retro computing
(which I enjoy, but it¹s not gem5).
Post by Steve Reinhardt
It would also be nice to keep in mind that some of the issues raised here
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about the
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that information
up to date and maybe feature it more prominently so that new users are
aware of it. Perhaps less-supported ISAs could print a warning and a link
to these pages when they are used.
Updating these pages is definitely something we should do regardless of
whether we retire existing ISAs or not. We should make sure that both the
build system and the generated binary print a warning if a user wants to
use one of the unmaintained ISAs. This should help in setting
expectations.

//Andreas

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Steve Reinhardt
2016-06-08 16:11:31 UTC
Permalink
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps of
determining that gem5 *used to* support the ISA of interest, finding and
extracting the old revision that still had that support in it, and very
likely needing to do who knows how much work to bring the ISA support back
in sync with the head of the repo before being able to reasonably make
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your the
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying that
disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g., Nathanael¹s
patch), modifying every single ISA adds a significant development
overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is always
easy. My point is that, first, however much effort was required of
Nathanael, the effort required for someone else to come back later,
identify his change as one that needs to be applied to a defunct ISA,
reverse engineer what he did, and then apply it, is going to be greater;
and second, that greater effort, multiplied by the number of such changes
that accumulate, will significantly reduce the likelihood that anyone ever
does bother to resurrect an ISA, compared with the smaller effort of taking
a functioning but incomplete ISA and addressing the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support".
Yes, I didn't mean to imply that there weren't other costs. Your solutions
below are some of the things I was thinking of when I said "some of the
issues raised here could be addressed in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without completely
* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core framework
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make the
remaining tests functional only (i.e., not diff stats or out files). This
would make stat refreshes (which pretty much every single non-trivial
change requires) much more manageable. I¹m planning to do similar things
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code is if
that code is never resurrected. I'd say the lower bound on the cost of
resurrecting the code is the same as the incremental cost of keeping the
code alive, and that's a very optimistic lower bound. In practice, whoever
faces the task of resurrecting the code won't know which past changes
actually broke things, won't be the person who made those changes and thus,
once the guilty changes are identified, will have to reverse-engineer them
to see how to apply them to the old ISA, and will be faced with the task of
fixing a number of things en masse rather than one at a time. In addition,
there's a good chance that someone considering taking this on is new to
gem5 and unfamiliar with the code to begin with, unlike the person who is
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal. The
likelihood that someone will ever need it for research seems very low and
having to resurrect it seems extremely unlikely. A simulator such as SimH
seems more suitable for what can only be described as retro computing
(which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should rationally
retire. For sentimental reasons I'd hate to see that happen, but I won't
claim they are much more than that. Though perhaps there is pedagogical
value in providing architecture students, who might not download SimH on
their own, the opportunity to look at the ISA and weep for what could have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised here
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about the
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that information
up to date and maybe feature it more prominently so that new users are
aware of it. Perhaps less-supported ISAs could print a warning and a link
to these pages when they are used.
Updating these pages is definitely something we should do regardless of
whether we retire existing ISAs or not. We should make sure that both the
build system and the generated binary print a warning if a user wants to
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.

Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
Andreas Hansson
2016-06-08 21:20:06 UTC
Permalink
Hi all,

It is worth remembering that the argument about “low" effort needs to take
long-term usefulness into account. Yes, the incremental effort may seem
small per commit, but if the incremental cost per ISA is 5% on average,
and the likelihood of that effort being useful long term is 20%, then
after one year we have effectively wasted 3 months of gem5 contributions.
That is 90 commits.

We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things around has
noticeable implications on the overall progress.

AndreasH


On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
Post by Steve Reinhardt
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps of
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and very
likely needing to do who knows how much work to bring the ISA support
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably make
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your the
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying that
disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g., Nathanael¹s
patch), modifying every single ISA adds a significant development
overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is always
easy. My point is that, first, however much effort was required of
Nathanael, the effort required for someone else to come back later,
identify his change as one that needs to be applied to a defunct ISA,
reverse engineer what he did, and then apply it, is going to be greater;
and second, that greater effort, multiplied by the number of such changes
that accumulate, will significantly reduce the likelihood that anyone ever
does bother to resurrect an ISA, compared with the smaller effort of taking
a functioning but incomplete ISA and addressing the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support".
Yes, I didn't mean to imply that there weren't other costs. Your solutions
below are some of the things I was thinking of when I said "some of the
issues raised here could be addressed in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without completely
* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core framework
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make the
remaining tests functional only (i.e., not diff stats or out files). This
would make stat refreshes (which pretty much every single non-trivial
change requires) much more manageable. I¹m planning to do similar things
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code is
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost of
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past changes
actually broke things, won't be the person who made those changes and thus,
once the guilty changes are identified, will have to reverse-engineer
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is new to
gem5 and unfamiliar with the code to begin with, unlike the person who
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal. The
likelihood that someone will ever need it for research seems very low and
having to resurrect it seems extremely unlikely. A simulator such as SimH
seems more suitable for what can only be described as retro computing
(which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should rationally
retire. For sentimental reasons I'd hate to see that happen, but I won't
claim they are much more than that. Though perhaps there is pedagogical
value in providing architecture students, who might not download SimH on
their own, the opportunity to look at the ISA and weep for what could have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised here
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about the
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users are
aware of it. Perhaps less-supported ISAs could print a warning and a
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless of
whether we retire existing ISAs or not. We should make sure that both the
build system and the generated binary print a warning if a user wants to
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Steve Reinhardt
2016-06-08 22:05:06 UTC
Permalink
Hi Andreas,

I don't know that I want a whole spreadsheet, but it's not clear to me how
a 5% overhead turns into wasting 25% of our time (3 months out of 12).
Partly that may be because I'm not sure what an "incremental cost of [...]
5% on average" means either. Can you elaborate please?

Thanks,

Steve
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort needs to take
long-term usefulness into account. Yes, the incremental effort may seem
small per commit, but if the incremental cost per ISA is 5% on average,
and the likelihood of that effort being useful long term is 20%, then
after one year we have effectively wasted 3 months of gem5 contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things around has
noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
Post by Steve Reinhardt
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps of
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and very
likely needing to do who knows how much work to bring the ISA support
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably make
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your the
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying that
disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g., Nathanael¹s
patch), modifying every single ISA adds a significant development
overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is always
easy. My point is that, first, however much effort was required of
Nathanael, the effort required for someone else to come back later,
identify his change as one that needs to be applied to a defunct ISA,
reverse engineer what he did, and then apply it, is going to be greater;
and second, that greater effort, multiplied by the number of such changes
that accumulate, will significantly reduce the likelihood that anyone ever
does bother to resurrect an ISA, compared with the smaller effort of taking
a functioning but incomplete ISA and addressing the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support".
Yes, I didn't mean to imply that there weren't other costs. Your solutions
below are some of the things I was thinking of when I said "some of the
issues raised here could be addressed in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without completely
* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core framework
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make the
remaining tests functional only (i.e., not diff stats or out files). This
would make stat refreshes (which pretty much every single non-trivial
change requires) much more manageable. I¹m planning to do similar things
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code is
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost of
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past changes
actually broke things, won't be the person who made those changes and thus,
once the guilty changes are identified, will have to reverse-engineer
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is new to
gem5 and unfamiliar with the code to begin with, unlike the person who
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal. The
likelihood that someone will ever need it for research seems very low and
having to resurrect it seems extremely unlikely. A simulator such as SimH
seems more suitable for what can only be described as retro computing
(which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should rationally
retire. For sentimental reasons I'd hate to see that happen, but I won't
claim they are much more than that. Though perhaps there is pedagogical
value in providing architecture students, who might not download SimH on
their own, the opportunity to look at the ISA and weep for what could have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised here
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about the
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users are
aware of it. Perhaps less-supported ISAs could print a warning and a
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless of
whether we retire existing ISAs or not. We should make sure that both the
build system and the generated binary print a warning if a user wants to
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
Andreas Hansson
2016-06-08 22:20:38 UTC
Permalink
Hi Steve,

In short: If we want to put in effort X to achieve something, we end up
having to put in 1.25 X (overhead times #ISAs), but it is only 1.05 X that
is actually useful (due to low utility). The 3 months was merely me
translating the wasted effort into a relatable number using the current
months commit count.

Again, let’s not obsess about the numbers. Let us instead focus on making
the most of our efforts and decide what is needed and what is not.

AndreasH

On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt"
Post by S***@labware.com
Hi Andreas,
I don't know that I want a whole spreadsheet, but it's not clear to me how
a 5% overhead turns into wasting 25% of our time (3 months out of 12).
Partly that may be because I'm not sure what an "incremental cost of [...]
5% on average" means either. Can you elaborate please?
Thanks,
Steve
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort needs to take
long-term usefulness into account. Yes, the incremental effort may seem
small per commit, but if the incremental cost per ISA is 5% on average,
and the likelihood of that effort being useful long term is 20%, then
after one year we have effectively wasted 3 months of gem5
contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things around has
noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps
of
Post by Andreas Sandberg
Post by Steve Reinhardt
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and
very
Post by Andreas Sandberg
Post by Steve Reinhardt
likely needing to do who knows how much work to bring the ISA
support
Post by Andreas Sandberg
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably
make
Post by Andreas Sandberg
Post by Steve Reinhardt
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your
the
Post by Andreas Sandberg
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying that
disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g.,
Nathanael¹s
Post by Andreas Sandberg
patch), modifying every single ISA adds a significant development
overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is always
easy. My point is that, first, however much effort was required of
Nathanael, the effort required for someone else to come back later,
identify his change as one that needs to be applied to a defunct ISA,
reverse engineer what he did, and then apply it, is going to be
greater;
and second, that greater effort, multiplied by the number of such
changes
that accumulate, will significantly reduce the likelihood that anyone
ever
does bother to resurrect an ISA, compared with the smaller effort of taking
a functioning but incomplete ISA and addressing the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support".
Yes, I didn't mean to imply that there weren't other costs. Your solutions
below are some of the things I was thinking of when I said "some of the
issues raised here could be addressed in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without
completely
Post by Andreas Sandberg
* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core
framework
Post by Andreas Sandberg
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make the
remaining tests functional only (i.e., not diff stats or out files). This
would make stat refreshes (which pretty much every single non-trivial
change requires) much more manageable. I¹m planning to do similar
things
Post by Andreas Sandberg
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code
is
Post by Andreas Sandberg
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost
of
Post by Andreas Sandberg
Post by Steve Reinhardt
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past
changes
Post by Andreas Sandberg
Post by Steve Reinhardt
actually broke things, won't be the person who made those changes
and
Post by Andreas Sandberg
Post by Steve Reinhardt
thus,
once the guilty changes are identified, will have to
reverse-engineer
Post by Andreas Sandberg
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is
new to
Post by Andreas Sandberg
Post by Steve Reinhardt
gem5 and unfamiliar with the code to begin with, unlike the person
who
Post by Andreas Sandberg
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal. The
likelihood that someone will ever need it for research seems very low and
having to resurrect it seems extremely unlikely. A simulator such as SimH
seems more suitable for what can only be described as retro computing
(which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should
rationally
retire. For sentimental reasons I'd hate to see that happen, but I
won't
claim they are much more than that. Though perhaps there is pedagogical
value in providing architecture students, who might not download SimH
on
their own, the opportunity to look at the ISA and weep for what could
have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised
here
Post by Andreas Sandberg
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about
the
Post by Andreas Sandberg
Post by Steve Reinhardt
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users
are
Post by Andreas Sandberg
Post by Steve Reinhardt
aware of it. Perhaps less-supported ISAs could print a warning and
a
Post by Andreas Sandberg
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless
of
Post by Andreas Sandberg
whether we retire existing ISAs or not. We should make sure that both the
build system and the generated binary print a warning if a user
wants to
Post by Andreas Sandberg
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
Post by Andreas Sandberg
contents to any other person, use it for any purpose, or store or
copy
Post by Andreas Sandberg
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Steve Reinhardt
2016-06-08 23:01:00 UTC
Permalink
Yes, I could quibble with your model, but I agree, I don't think it's
worthwhile ;). I do think we've already outlined a number of steps (plus a
couple others I thought of) that we could take immediately that are (I
believe) totally uncontroversial:

1. Let's prune a lot of the long regressions that don't add much value,
particularly for less-supported ISAs. Concretely, getting rid of all the
long SE-mode ALPHA tests is pretty low-hanging fruit here.
2. Let's make sure the wiki is up-to-date on the level of support provided
for various ISAs.
3. Let's add warnings, ideally at both compile time and run time, letting
users know when they are using a less-supported ISA.
4. Let's make the default Ruby protocol something useful, so that we can do
basic testing of both classic and Ruby memory systems with a single compile.
5. Let's only compile that one version of less-supported ISAs (read:
ALPHA), instead of compiling it with four (!) different extra Ruby
protocols. If we really want to test multiple Ruby protocols, we might as
well at least do it using a well-supported ISA.
6. Let's give it another few days, but unless someone responds to my
pleading EIO email from 6/3, let's go ahead and ditch EIO support.

I suggest we go ahead and do the things we all agree on and see where that
leaves us before we debate about additional steps.

Steve
Post by Andreas Hansson
Hi Steve,
In short: If we want to put in effort X to achieve something, we end up
having to put in 1.25 X (overhead times #ISAs), but it is only 1.05 X that
is actually useful (due to low utility). The 3 months was merely me
translating the wasted effort into a relatable number using the current
months commit count.
Again, let’s not obsess about the numbers. Let us instead focus on making
the most of our efforts and decide what is needed and what is not.
AndreasH
On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt"
Post by S***@labware.com
Hi Andreas,
I don't know that I want a whole spreadsheet, but it's not clear to me how
a 5% overhead turns into wasting 25% of our time (3 months out of 12).
Partly that may be because I'm not sure what an "incremental cost of [...]
5% on average" means either. Can you elaborate please?
Thanks,
Steve
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort needs to take
long-term usefulness into account. Yes, the incremental effort may seem
small per commit, but if the incremental cost per ISA is 5% on average,
and the likelihood of that effort being useful long term is 20%, then
after one year we have effectively wasted 3 months of gem5
contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things around has
noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps
of
Post by Andreas Sandberg
Post by Steve Reinhardt
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and
very
Post by Andreas Sandberg
Post by Steve Reinhardt
likely needing to do who knows how much work to bring the ISA
support
Post by Andreas Sandberg
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably
make
Post by Andreas Sandberg
Post by Steve Reinhardt
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your
the
Post by Andreas Sandberg
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying that
disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g.,
Nathanael¹s
Post by Andreas Sandberg
patch), modifying every single ISA adds a significant development
overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is always
easy. My point is that, first, however much effort was required of
Nathanael, the effort required for someone else to come back later,
identify his change as one that needs to be applied to a defunct ISA,
reverse engineer what he did, and then apply it, is going to be
greater;
and second, that greater effort, multiplied by the number of such
changes
that accumulate, will significantly reduce the likelihood that anyone
ever
does bother to resurrect an ISA, compared with the smaller effort of taking
a functioning but incomplete ISA and addressing the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support".
Yes, I didn't mean to imply that there weren't other costs. Your solutions
below are some of the things I was thinking of when I said "some of the
issues raised here could be addressed in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without
completely
Post by Andreas Sandberg
* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core
framework
Post by Andreas Sandberg
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make the
remaining tests functional only (i.e., not diff stats or out files). This
would make stat refreshes (which pretty much every single non-trivial
change requires) much more manageable. I¹m planning to do similar
things
Post by Andreas Sandberg
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code
is
Post by Andreas Sandberg
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost
of
Post by Andreas Sandberg
Post by Steve Reinhardt
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past
changes
Post by Andreas Sandberg
Post by Steve Reinhardt
actually broke things, won't be the person who made those changes
and
Post by Andreas Sandberg
Post by Steve Reinhardt
thus,
once the guilty changes are identified, will have to
reverse-engineer
Post by Andreas Sandberg
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is
new to
Post by Andreas Sandberg
Post by Steve Reinhardt
gem5 and unfamiliar with the code to begin with, unlike the person
who
Post by Andreas Sandberg
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal. The
likelihood that someone will ever need it for research seems very low and
having to resurrect it seems extremely unlikely. A simulator such as SimH
seems more suitable for what can only be described as retro computing
(which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should
rationally
retire. For sentimental reasons I'd hate to see that happen, but I
won't
claim they are much more than that. Though perhaps there is pedagogical
value in providing architecture students, who might not download SimH
on
their own, the opportunity to look at the ISA and weep for what could
have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised
here
Post by Andreas Sandberg
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to user
expectations, there is already some documentation on the wiki about
the
Post by Andreas Sandberg
Post by Steve Reinhardt
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users
are
Post by Andreas Sandberg
Post by Steve Reinhardt
aware of it. Perhaps less-supported ISAs could print a warning and
a
Post by Andreas Sandberg
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless
of
Post by Andreas Sandberg
whether we retire existing ISAs or not. We should make sure that both the
build system and the generated binary print a warning if a user
wants to
Post by Andreas Sandberg
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
Post by Andreas Sandberg
contents to any other person, use it for any purpose, or store or
copy
Post by Andreas Sandberg
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
Beckmann, Brad
2016-06-08 23:13:27 UTC
Permalink
I just want to chime in and say that we definitely want to continue to test the same number of Ruby protocols as before. Let test them with x86. I suspect that is the most common CPU ISA used with Ruby.

Thanks,

Brad



-----Original Message-----
From: gem5-dev [mailto:gem5-dev-***@gem5.org] On Behalf Of Steve Reinhardt
Sent: Wednesday, June 08, 2016 4:01 PM
To: gem5 Developer List <gem5-***@gem5.org>
Subject: Re: [gem5-dev] Let's retire some ISAs

Yes, I could quibble with your model, but I agree, I don't think it's worthwhile ;). I do think we've already outlined a number of steps (plus a couple others I thought of) that we could take immediately that are (I
believe) totally uncontroversial:

1. Let's prune a lot of the long regressions that don't add much value, particularly for less-supported ISAs. Concretely, getting rid of all the long SE-mode ALPHA tests is pretty low-hanging fruit here.
2. Let's make sure the wiki is up-to-date on the level of support provided for various ISAs.
3. Let's add warnings, ideally at both compile time and run time, letting users know when they are using a less-supported ISA.
4. Let's make the default Ruby protocol something useful, so that we can do basic testing of both classic and Ruby memory systems with a single compile.
5. Let's only compile that one version of less-supported ISAs (read:
ALPHA), instead of compiling it with four (!) different extra Ruby protocols. If we really want to test multiple Ruby protocols, we might as well at least do it using a well-supported ISA.
6. Let's give it another few days, but unless someone responds to my pleading EIO email from 6/3, let's go ahead and ditch EIO support.

I suggest we go ahead and do the things we all agree on and see where that leaves us before we debate about additional steps.

Steve
Post by Andreas Hansson
Hi Steve,
In short: If we want to put in effort X to achieve something, we end
up having to put in 1.25 X (overhead times #ISAs), but it is only 1.05
X that is actually useful (due to low utility). The 3 months was
merely me translating the wasted effort into a relatable number using
the current months commit count.
Again, let’s not obsess about the numbers. Let us instead focus on
making the most of our efforts and decide what is needed and what is not.
AndreasH
On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt"
Post by S***@labware.com
Hi Andreas,
I don't know that I want a whole spreadsheet, but it's not clear to
me how a 5% overhead turns into wasting 25% of our time (3 months out of 12).
Partly that may be because I'm not sure what an "incremental cost of
[...] 5% on average" means either. Can you elaborate please?
Thanks,
Steve
On Wed, Jun 8, 2016 at 2:20 PM Andreas Hansson
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort needs
to take long-term usefulness into account. Yes, the incremental
effort may seem small per commit, but if the incremental cost per
ISA is 5% on average, and the likelihood of that effort being
useful long term is 20%, then after one year we have effectively
wasted 3 months of gem5 contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things
around has noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps
of
Post by Andreas Sandberg
Post by Steve Reinhardt
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and
very
Post by Andreas Sandberg
Post by Steve Reinhardt
likely needing to do who knows how much work to bring the ISA
support
Post by Andreas Sandberg
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to
reasonably
make
Post by Andreas Sandberg
Post by Steve Reinhardt
further enhancements, then the likelihood of anyone crossing
that threshold is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your
the
Post by Andreas Sandberg
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying
that disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g.,
Nathanael¹s
Post by Andreas Sandberg
patch), modifying every single ISA adds a significant
development overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is
always easy. My point is that, first, however much effort was
required of Nathanael, the effort required for someone else to
come back later, identify his change as one that needs to be
applied to a defunct ISA, reverse engineer what he did, and then
apply it, is going to be
greater;
and second, that greater effort, multiplied by the number of such
changes
that accumulate, will significantly reduce the likelihood that anyone
ever
does bother to resurrect an ISA, compared with the smaller effort
of taking a functioning but incomplete ISA and addressing the
gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we ³support".
Yes, I didn't mean to imply that there weren't other costs. Your
solutions below are some of the things I was thinking of when I
said "some of the issues raised here could be addressed in other
ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without
completely
Post by Andreas Sandberg
* Improved build system reuse. We could significantly shorten
the build/test cycle by only compiling device models and the
core
framework
Post by Andreas Sandberg
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make
the remaining tests functional only (i.e., not diff stats or out files).
This
would make stat refreshes (which pretty much every single
non-trivial change requires) much more manageable. I¹m planning
to do similar
things
Post by Andreas Sandberg
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code
is
Post by Andreas Sandberg
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost
of
Post by Andreas Sandberg
Post by Steve Reinhardt
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past
changes
Post by Andreas Sandberg
Post by Steve Reinhardt
actually broke things, won't be the person who made those changes
and
Post by Andreas Sandberg
Post by Steve Reinhardt
thus,
once the guilty changes are identified, will have to
reverse-engineer
Post by Andreas Sandberg
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is
new to
Post by Andreas Sandberg
Post by Steve Reinhardt
gem5 and unfamiliar with the code to begin with, unlike the person
who
Post by Andreas Sandberg
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal.
The likelihood that someone will ever need it for research seems
very low and having to resurrect it seems extremely unlikely. A
simulator such as SimH seems more suitable for what can only be
described as retro computing (which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should
rationally
retire. For sentimental reasons I'd hate to see that happen, but I
won't
claim they are much more than that. Though perhaps there is
pedagogical value in providing architecture students, who might
not download SimH
on
their own, the opportunity to look at the ISA and weep for what could
have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised
here
Post by Andreas Sandberg
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to
user expectations, there is already some documentation on the
wiki about
the
Post by Andreas Sandberg
Post by Steve Reinhardt
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users
are
Post by Andreas Sandberg
Post by Steve Reinhardt
aware of it. Perhaps less-supported ISAs could print a warning and
a
Post by Andreas Sandberg
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless
of
Post by Andreas Sandberg
whether we retire existing ISAs or not. We should make sure that
both the build system and the generated binary print a warning
if a user
wants to
Post by Andreas Sandberg
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do
not disclose
the
Post by Andreas Sandberg
contents to any other person, use it for any purpose, or store or
copy
Post by Andreas Sandberg
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do not
disclose the contents to any other person, use it for any purpose,
or store or copy the information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the contents to any other person, use it for any purpose, or store or
copy the information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
gem5-***@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
Steve Reinhardt
2016-06-08 23:16:05 UTC
Permalink
Brad, does that include MI_example?

Steve
Post by Beckmann, Brad
I just want to chime in and say that we definitely want to continue to
test the same number of Ruby protocols as before. Let test them with x86.
I suspect that is the most common CPU ISA used with Ruby.
Thanks,
Brad
-----Original Message-----
Sent: Wednesday, June 08, 2016 4:01 PM
Subject: Re: [gem5-dev] Let's retire some ISAs
Yes, I could quibble with your model, but I agree, I don't think it's
worthwhile ;). I do think we've already outlined a number of steps (plus a
couple others I thought of) that we could take immediately that are (I
1. Let's prune a lot of the long regressions that don't add much value,
particularly for less-supported ISAs. Concretely, getting rid of all the
long SE-mode ALPHA tests is pretty low-hanging fruit here.
2. Let's make sure the wiki is up-to-date on the level of support provided
for various ISAs.
3. Let's add warnings, ideally at both compile time and run time, letting
users know when they are using a less-supported ISA.
4. Let's make the default Ruby protocol something useful, so that we can
do basic testing of both classic and Ruby memory systems with a single
compile.
ALPHA), instead of compiling it with four (!) different extra Ruby
protocols. If we really want to test multiple Ruby protocols, we might as
well at least do it using a well-supported ISA.
6. Let's give it another few days, but unless someone responds to my
pleading EIO email from 6/3, let's go ahead and ditch EIO support.
I suggest we go ahead and do the things we all agree on and see where that
leaves us before we debate about additional steps.
Steve
Post by Andreas Hansson
Hi Steve,
In short: If we want to put in effort X to achieve something, we end
up having to put in 1.25 X (overhead times #ISAs), but it is only 1.05
X that is actually useful (due to low utility). The 3 months was
merely me translating the wasted effort into a relatable number using
the current months commit count.
Again, let’s not obsess about the numbers. Let us instead focus on
making the most of our efforts and decide what is needed and what is not.
AndreasH
On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt"
Post by S***@labware.com
Hi Andreas,
I don't know that I want a whole spreadsheet, but it's not clear to
me how a 5% overhead turns into wasting 25% of our time (3 months out
of 12).
Post by Andreas Hansson
Post by S***@labware.com
Partly that may be because I'm not sure what an "incremental cost of
[...] 5% on average" means either. Can you elaborate please?
Thanks,
Steve
On Wed, Jun 8, 2016 at 2:20 PM Andreas Hansson
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort needs
to take long-term usefulness into account. Yes, the incremental
effort may seem small per commit, but if the incremental cost per
ISA is 5% on average, and the likelihood of that effort being
useful long term is 20%, then after one year we have effectively
wasted 3 months of gem5 contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things
around has noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps
of
Post by Andreas Sandberg
Post by Steve Reinhardt
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and
very
Post by Andreas Sandberg
Post by Steve Reinhardt
likely needing to do who knows how much work to bring the ISA
support
Post by Andreas Sandberg
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably
make
Post by Andreas Sandberg
Post by Steve Reinhardt
further enhancements, then the likelihood of anyone crossing
that threshold is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your
the
Post by Andreas Sandberg
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying
that disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g.,
Nathanael¹s
Post by Andreas Sandberg
patch), modifying every single ISA adds a significant
development overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is
always easy. My point is that, first, however much effort was
required of Nathanael, the effort required for someone else to
come back later, identify his change as one that needs to be
applied to a defunct ISA, reverse engineer what he did, and then
apply it, is going to be
greater;
and second, that greater effort, multiplied by the number of such
changes
that accumulate, will significantly reduce the likelihood that anyone
ever
does bother to resurrect an ISA, compared with the smaller effort
of taking a functioning but incomplete ISA and addressing the
gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we
³support".
Post by Andreas Hansson
Post by S***@labware.com
Post by Andreas Hansson
Yes, I didn't mean to imply that there weren't other costs. Your
solutions below are some of the things I was thinking of when I
said "some of the issues raised here could be addressed in other
ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without
completely
Post by Andreas Sandberg
* Improved build system reuse. We could significantly shorten
the build/test cycle by only compiling device models and the
core
framework
Post by Andreas Sandberg
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make
the remaining tests functional only (i.e., not diff stats or out
files).
Post by Andreas Hansson
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
This
would make stat refreshes (which pretty much every single
non-trivial change requires) much more manageable. I¹m planning
to do similar
things
Post by Andreas Sandberg
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code
is
Post by Andreas Sandberg
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost
of
Post by Andreas Sandberg
Post by Steve Reinhardt
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past
changes
Post by Andreas Sandberg
Post by Steve Reinhardt
actually broke things, won't be the person who made those changes
and
Post by Andreas Sandberg
Post by Steve Reinhardt
thus,
once the guilty changes are identified, will have to
reverse-engineer
Post by Andreas Sandberg
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is
new to
Post by Andreas Sandberg
Post by Steve Reinhardt
gem5 and unfamiliar with the code to begin with, unlike the person
who
Post by Andreas Sandberg
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal.
The likelihood that someone will ever need it for research seems
very low and having to resurrect it seems extremely unlikely. A
simulator such as SimH seems more suitable for what can only be
described as retro computing (which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should
rationally
retire. For sentimental reasons I'd hate to see that happen, but I
won't
claim they are much more than that. Though perhaps there is
pedagogical value in providing architecture students, who might
not download SimH
on
their own, the opportunity to look at the ISA and weep for what could
have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised
here
Post by Andreas Sandberg
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to
user expectations, there is already some documentation on the
wiki about
the
Post by Andreas Sandberg
Post by Steve Reinhardt
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users
are
Post by Andreas Sandberg
Post by Steve Reinhardt
aware of it. Perhaps less-supported ISAs could print a warning and
a
Post by Andreas Sandberg
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless
of
Post by Andreas Sandberg
whether we retire existing ISAs or not. We should make sure that
both the build system and the generated binary print a warning
if a user
wants to
Post by Andreas Sandberg
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do
not disclose
the
Post by Andreas Sandberg
contents to any other person, use it for any purpose, or store or
copy
Post by Andreas Sandberg
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do not
disclose the contents to any other person, use it for any purpose,
or store or copy the information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the contents to any other person, use it for any purpose, or store or
copy the information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
Beckmann, Brad
2016-06-08 23:36:40 UTC
Permalink
As the name suggests, I think MI_example still provides an excellent simple example for those people getting started with Ruby. I would like to see it maintained.

Brad


-----Original Message-----
From: gem5-dev [mailto:gem5-dev-***@gem5.org] On Behalf Of Steve Reinhardt
Sent: Wednesday, June 08, 2016 4:16 PM
To: gem5 Developer List <gem5-***@gem5.org>
Subject: Re: [gem5-dev] Let's retire some ISAs

Brad, does that include MI_example?

Steve
Post by Beckmann, Brad
I just want to chime in and say that we definitely want to continue to
test the same number of Ruby protocols as before. Let test them with x86.
I suspect that is the most common CPU ISA used with Ruby.
Thanks,
Brad
-----Original Message-----
Sent: Wednesday, June 08, 2016 4:01 PM
Subject: Re: [gem5-dev] Let's retire some ISAs
Yes, I could quibble with your model, but I agree, I don't think it's
worthwhile ;). I do think we've already outlined a number of steps
(plus a couple others I thought of) that we could take immediately
that are (I
1. Let's prune a lot of the long regressions that don't add much
value, particularly for less-supported ISAs. Concretely, getting rid
of all the long SE-mode ALPHA tests is pretty low-hanging fruit here.
2. Let's make sure the wiki is up-to-date on the level of support
provided for various ISAs.
3. Let's add warnings, ideally at both compile time and run time,
letting users know when they are using a less-supported ISA.
4. Let's make the default Ruby protocol something useful, so that we
can do basic testing of both classic and Ruby memory systems with a
single compile.
ALPHA), instead of compiling it with four (!) different extra Ruby
protocols. If we really want to test multiple Ruby protocols, we
might as well at least do it using a well-supported ISA.
6. Let's give it another few days, but unless someone responds to my
pleading EIO email from 6/3, let's go ahead and ditch EIO support.
I suggest we go ahead and do the things we all agree on and see where
that leaves us before we debate about additional steps.
Steve
On Wed, Jun 8, 2016 at 3:20 PM Andreas Hansson
Post by Andreas Hansson
Hi Steve,
In short: If we want to put in effort X to achieve something, we end
up having to put in 1.25 X (overhead times #ISAs), but it is only
1.05 X that is actually useful (due to low utility). The 3 months
was merely me translating the wasted effort into a relatable number
using the current months commit count.
Again, let’s not obsess about the numbers. Let us instead focus on
making the most of our efforts and decide what is needed and what is not.
AndreasH
On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt"
Post by S***@labware.com
Hi Andreas,
I don't know that I want a whole spreadsheet, but it's not clear to
me how a 5% overhead turns into wasting 25% of our time (3 months out
of 12).
Post by Andreas Hansson
Post by S***@labware.com
Partly that may be because I'm not sure what an "incremental cost
of [...] 5% on average" means either. Can you elaborate please?
Thanks,
Steve
On Wed, Jun 8, 2016 at 2:20 PM Andreas Hansson
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort
needs to take long-term usefulness into account. Yes, the
incremental effort may seem small per commit, but if the
incremental cost per ISA is 5% on average, and the likelihood of
that effort being useful long term is 20%, then after one year we
have effectively wasted 3 months of gem5 contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want
the spreadsheet...), but let’s not fool ourselves. Keeping things
around has noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the steps
of
Post by Andreas Sandberg
Post by Steve Reinhardt
determining that gem5 *used to* support the ISA of interest, finding
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it, and
very
Post by Andreas Sandberg
Post by Steve Reinhardt
likely needing to do who knows how much work to bring the ISA
support
Post by Andreas Sandberg
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably
make
Post by Andreas Sandberg
Post by Steve Reinhardt
further enhancements, then the likelihood of anyone crossing
that threshold is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with your
the
Post by Andreas Sandberg
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're
saying that disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g.,
Nathanael¹s
Post by Andreas Sandberg
patch), modifying every single ISA adds a significant
development overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is
always easy. My point is that, first, however much effort was
required of Nathanael, the effort required for someone else to
come back later, identify his change as one that needs to be
applied to a defunct ISA, reverse engineer what he did, and then
apply it, is going to be
greater;
and second, that greater effort, multiplied by the number of such
changes
that accumulate, will significantly reduce the likelihood that anyone
ever
does bother to resurrect an ISA, compared with the smaller
effort of taking a functioning but incomplete ISA and addressing
the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we
³support".
Post by Andreas Hansson
Post by S***@labware.com
Post by Andreas Hansson
Yes, I didn't mean to imply that there weren't other costs.
Your solutions below are some of the things I was thinking of
when I said "some of the issues raised here could be addressed
in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without
completely
Post by Andreas Sandberg
* Improved build system reuse. We could significantly shorten
the build/test cycle by only compiling device models and the
core
framework
Post by Andreas Sandberg
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and
make the remaining tests functional only (i.e., not diff stats
or out
files).
Post by Andreas Hansson
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
This
would make stat refreshes (which pretty much every single
non-trivial change requires) much more manageable. I¹m
planning to do similar
things
Post by Andreas Sandberg
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring code
is
Post by Andreas Sandberg
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the cost
of
Post by Andreas Sandberg
Post by Steve Reinhardt
resurrecting the code is the same as the incremental cost of keeping
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In practice,
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past
changes
Post by Andreas Sandberg
Post by Steve Reinhardt
actually broke things, won't be the person who made those changes
and
Post by Andreas Sandberg
Post by Steve Reinhardt
thus,
once the guilty changes are identified, will have to
reverse-engineer
Post by Andreas Sandberg
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with the
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is
new to
Post by Andreas Sandberg
Post by Steve Reinhardt
gem5 and unfamiliar with the code to begin with, unlike the person
who
Post by Andreas Sandberg
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal.
The likelihood that someone will ever need it for research
seems very low and having to resurrect it seems extremely
unlikely. A simulator such as SimH seems more suitable for
what can only be described as retro computing (which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should
rationally
retire. For sentimental reasons I'd hate to see that happen, but I
won't
claim they are much more than that. Though perhaps there is
pedagogical value in providing architecture students, who might
not download SimH
on
their own, the opportunity to look at the ISA and weep for what could
have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues raised
here
Post by Andreas Sandberg
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect
to user expectations, there is already some documentation on
the wiki about
the
Post by Andreas Sandberg
Post by Steve Reinhardt
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix Regardless of the outcome here,
it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new users
are
Post by Andreas Sandberg
Post by Steve Reinhardt
aware of it. Perhaps less-supported ISAs could print a warning and
a
Post by Andreas Sandberg
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do regardless
of
Post by Andreas Sandberg
whether we retire existing ISAs or not. We should make sure
that both the build system and the generated binary print a
warning if a user
wants to
Post by Andreas Sandberg
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any
attachments are confidential and may also be privileged. If
you are not the intended recipient, please notify the sender
immediately and do not disclose
the
Post by Andreas Sandberg
contents to any other person, use it for any purpose, or store or
copy
Post by Andreas Sandberg
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments
are confidential and may also be privileged. If you are not the
intended recipient, please notify the sender immediately and do
not disclose the contents to any other person, use it for any
purpose, or store or copy the information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the contents to any other person, use it for any purpose, or store
or copy the information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
gem5-***@gem5.org
http://m5sim.org/mailman/listinfo/gem5-dev
Andreas Sandberg
2016-06-10 09:09:05 UTC
Permalink
Hi Steve,

This sounds like a solid plan that we could get behind. I’d still like to
plan for Alpha deprecation though. But let’s give it some more time.

Another incremental change that I would like to see (which should be
relatively easy to accomplish) is to reuse the gem5 same core binaries for
all of the Ruby protocols. That is, compile ARM/X86 once and then link it
with the relevant Ruby protocol. This would significantly decrease
compilation time. I don’t think we’ll be able to do that ourselves anytime
soon though. Scons is a bit of a time sink.

Cheers,
Andreas


On 09/06/2016, 00:01, "gem5-dev on behalf of Steve Reinhardt"
Post by Steve Reinhardt
Yes, I could quibble with your model, but I agree, I don't think it's
worthwhile ;). I do think we've already outlined a number of steps (plus a
couple others I thought of) that we could take immediately that are (I
1. Let's prune a lot of the long regressions that don't add much value,
particularly for less-supported ISAs. Concretely, getting rid of all the
long SE-mode ALPHA tests is pretty low-hanging fruit here.
2. Let's make sure the wiki is up-to-date on the level of support provided
for various ISAs.
3. Let's add warnings, ideally at both compile time and run time, letting
users know when they are using a less-supported ISA.
4. Let's make the default Ruby protocol something useful, so that we can do
basic testing of both classic and Ruby memory systems with a single compile.
ALPHA), instead of compiling it with four (!) different extra Ruby
protocols. If we really want to test multiple Ruby protocols, we might as
well at least do it using a well-supported ISA.
6. Let's give it another few days, but unless someone responds to my
pleading EIO email from 6/3, let's go ahead and ditch EIO support.
I suggest we go ahead and do the things we all agree on and see where that
leaves us before we debate about additional steps.
Steve
Post by Andreas Hansson
Hi Steve,
In short: If we want to put in effort X to achieve something, we end up
having to put in 1.25 X (overhead times #ISAs), but it is only 1.05 X that
is actually useful (due to low utility). The 3 months was merely me
translating the wasted effort into a relatable number using the current
months commit count.
Again, let’s not obsess about the numbers. Let us instead focus on making
the most of our efforts and decide what is needed and what is not.
AndreasH
On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt"
Post by S***@labware.com
Hi Andreas,
I don't know that I want a whole spreadsheet, but it's not clear to me
how
Post by S***@labware.com
a 5% overhead turns into wasting 25% of our time (3 months out of 12).
Partly that may be because I'm not sure what an "incremental cost of
[...]
Post by S***@labware.com
5% on average" means either. Can you elaborate please?
Thanks,
Steve
On Wed, Jun 8, 2016 at 2:20 PM Andreas Hansson
Post by Andreas Hansson
Hi all,
It is worth remembering that the argument about “low" effort needs to take
long-term usefulness into account. Yes, the incremental effort may
seem
Post by S***@labware.com
Post by Andreas Hansson
small per commit, but if the incremental cost per ISA is 5% on
average,
Post by S***@labware.com
Post by Andreas Hansson
and the likelihood of that effort being useful long term is 20%, then
after one year we have effectively wasted 3 months of gem5
contributions.
That is 90 commits.
We can of course debate these numbers (let me know if you want the
spreadsheet...), but let’s not fool ourselves. Keeping things around
has
Post by S***@labware.com
Post by Andreas Hansson
noticeable implications on the overall progress.
AndreasH
On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt"
On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg
Post by Andreas Sandberg
Post by Steve Reinhardt
Once the bar for reaching this point is raised by adding the
steps
Post by S***@labware.com
Post by Andreas Hansson
of
Post by Andreas Sandberg
Post by Steve Reinhardt
determining that gem5 *used to* support the ISA of interest,
finding
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
and
Post by Steve Reinhardt
extracting the old revision that still had that support in it,
and
Post by S***@labware.com
Post by Andreas Hansson
very
Post by Andreas Sandberg
Post by Steve Reinhardt
likely needing to do who knows how much work to bring the ISA
support
Post by Andreas Sandberg
back
Post by Steve Reinhardt
in sync with the head of the repo before being able to reasonably
make
Post by Andreas Sandberg
Post by Steve Reinhardt
further enhancements, then the likelihood of anyone crossing that threshold
is going to be dramatically lowered.
You raise a valid point. However, I don¹t completely agree with
your
Post by S***@labware.com
Post by Andreas Hansson
the
Post by Andreas Sandberg
conclusion.
Hi Andreas, can you elaborate? I don't see anything you're saying
that
Post by S***@labware.com
Post by Andreas Hansson
disagrees with what I think I said.
Post by Andreas Sandberg
For complex work, say reworking register interfaces (e.g.,
Nathanael¹s
Post by Andreas Sandberg
patch), modifying every single ISA adds a significant development
overhead.
Yes, I didn't say that incrementally maintaining all the ISAs is
always
Post by S***@labware.com
Post by Andreas Hansson
easy. My point is that, first, however much effort was required of
Nathanael, the effort required for someone else to come back later,
identify his change as one that needs to be applied to a defunct
ISA,
Post by S***@labware.com
Post by Andreas Hansson
reverse engineer what he did, and then apply it, is going to be
greater;
and second, that greater effort, multiplied by the number of such
changes
that accumulate, will significantly reduce the likelihood that
anyone
Post by S***@labware.com
Post by Andreas Hansson
ever
does bother to resurrect an ISA, compared with the smaller effort of taking
a functioning but incomplete ISA and addressing the gaps.
Post by Andreas Sandberg
There is also a very real cost associated with having to
re-compile gem5 test all of the ISA/Ruby configuration we
³support".
Post by S***@labware.com
Post by Andreas Hansson
Yes, I didn't mean to imply that there weren't other costs. Your solutions
below are some of the things I was thinking of when I said "some of
the
Post by S***@labware.com
Post by Andreas Hansson
issues raised here could be addressed in other ways".
Post by Andreas Sandberg
There
are two things we could do to improve the situation without
completely
Post by Andreas Sandberg
* Improved build system reuse. We could significantly shorten the
build/test cycle by only compiling device models and the core
framework
Post by Andreas Sandberg
once instead of for each ISA/Ruby configuration.
* We should get rid of long tests for unsupported ISAs and make
the
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
remaining tests functional only (i.e., not diff stats or out
files).
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
This
would make stat refreshes (which pretty much every single
non-trivial
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
change requires) much more manageable. I¹m planning to do similar
things
Post by Andreas Sandberg
for some of the ARM tests that are more functional in nature.
Post by Steve Reinhardt
Which leads to #2: Basically the only way you win by retiring
code
Post by S***@labware.com
Post by Andreas Hansson
is
Post by Andreas Sandberg
if
Post by Steve Reinhardt
that code is never resurrected. I'd say the lower bound on the
cost
Post by S***@labware.com
Post by Andreas Hansson
of
Post by Andreas Sandberg
Post by Steve Reinhardt
resurrecting the code is the same as the incremental cost of
keeping
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
the
Post by Steve Reinhardt
code alive, and that's a very optimistic lower bound. In
practice,
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
whoever
Post by Steve Reinhardt
faces the task of resurrecting the code won't know which past
changes
Post by Andreas Sandberg
Post by Steve Reinhardt
actually broke things, won't be the person who made those changes
and
Post by Andreas Sandberg
Post by Steve Reinhardt
thus,
once the guilty changes are identified, will have to
reverse-engineer
Post by Andreas Sandberg
them
Post by Steve Reinhardt
to see how to apply them to the old ISA, and will be faced with
the
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
task
Post by Steve Reinhardt
of
fixing a number of things en masse rather than one at a time. In
addition,
Post by Steve Reinhardt
there's a good chance that someone considering taking this on is
new to
Post by Andreas Sandberg
Post by Steve Reinhardt
gem5 and unfamiliar with the code to begin with, unlike the
person
Post by S***@labware.com
Post by Andreas Hansson
who
Post by Andreas Sandberg
is
Post by Steve Reinhardt
committing patches and could perform the incremental updates.
In that case, we Alpha would still be a candidate for removal.
The
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
likelihood that someone will ever need it for research seems very
low
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
and
having to resurrect it seems extremely unlikely. A simulator such
as
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
SimH
seems more suitable for what can only be described as retro
computing
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
(which I enjoy, but it¹s not gem5).
Yes, I don't dispute that Alpha looks like something we should
rationally
retire. For sentimental reasons I'd hate to see that happen, but I
won't
claim they are much more than that. Though perhaps there is
pedagogical
Post by S***@labware.com
Post by Andreas Hansson
value in providing architecture students, who might not download
SimH
Post by S***@labware.com
Post by Andreas Hansson
on
their own, the opportunity to look at the ISA and weep for what
could
Post by S***@labware.com
Post by Andreas Hansson
have
been...
Post by Andreas Sandberg
It would also be nice to keep in mind that some of the issues
raised
Post by S***@labware.com
Post by Andreas Hansson
here
Post by Andreas Sandberg
Post by Steve Reinhardt
could be addressed in other ways. For example, with respect to
user
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
Post by Steve Reinhardt
expectations, there is already some documentation on the wiki
about
Post by S***@labware.com
Post by Andreas Hansson
the
Post by Andreas Sandberg
Post by Steve Reinhardt
http://gem5.org/Architecture_Support
http://gem5.org/Status_Matrix
Regardless of the outcome here, it would be good to keep that
information
Post by Steve Reinhardt
up to date and maybe feature it more prominently so that new
users
Post by S***@labware.com
Post by Andreas Hansson
are
Post by Andreas Sandberg
Post by Steve Reinhardt
aware of it. Perhaps less-supported ISAs could print a warning
and
Post by S***@labware.com
Post by Andreas Hansson
a
Post by Andreas Sandberg
link
Post by Steve Reinhardt
to these pages when they are used.
Updating these pages is definitely something we should do
regardless
Post by S***@labware.com
Post by Andreas Hansson
of
Post by Andreas Sandberg
whether we retire existing ISAs or not. We should make sure that
both
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
the
build system and the generated binary print a warning if a user
wants to
Post by Andreas Sandberg
use one of the unmaintained ISAs. This should help in setting
expectations.
Yes, I agree with this too.
Steve
Post by Andreas Sandberg
//Andreas
IMPORTANT NOTICE: The contents of this email and any attachments
are
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
confidential and may also be privileged. If you are not the
intended
Post by S***@labware.com
Post by Andreas Hansson
Post by Andreas Sandberg
recipient, please notify the sender immediately and do not
disclose
Post by S***@labware.com
Post by Andreas Hansson
the
Post by Andreas Sandberg
contents to any other person, use it for any purpose, or store or
copy
Post by Andreas Sandberg
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose
the
Post by S***@labware.com
Post by Andreas Hansson
contents to any other person, use it for any purpose, or store or
copy
Post by S***@labware.com
Post by Andreas Hansson
the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
_______________________________________________
gem5-dev mailing list
http://m5sim.org/mailman/listinfo/gem5-dev
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Andreas Sandberg
2016-06-07 10:59:01 UTC
Permalink
Hi Bors,

Apologis if my email came across as a bit harsh. I don¹t want to prune
things that are in active use. What I want to do is mainly to find out
which architectures people care about and get a better understanding of
who can/will maintain them.

One of the problems with a lot of ISAs in gem5 is unclear maintainership.
gem5 currently doesn¹t have a clear notion of architecture maintainers,
which I think is a bit of a problem. There are de facto maintainers for
x86 and ARM, but the other ISAs are for all practical purposes
unmaintained. I can, as you can probably understand, only really work on
ARM-related and core bits of the simulator. Similarly, AMD mainly works on
the x86 bits and the GPU bits.

The ideal outcome of this discussion would be that we formalise
maintainers for all of the ISAs and come up with a plan for unmaintained
ISAs.

To give you some background, I did some data mining on the repo to get an
estimate of the number of commits per ISA since 2012-01-01. These are the
stats:

ALPHA: 8 commits
ARM: 240 commits
MIPS: 11 commits
POWER: 11 commits (only ~5 of these are related to the POWER ISA, the rest
are related to power/energy modelling)
SPARC: 7 commits
x86: 158 commits

(for A in arm alpha x86 mips sparc power; do echo $A `git shortlog
--after=2012-01-01 -n | grep -i "^.*${A}.*:" |wc`; done)


This is obviously not a fool proof measure of the number of users, but it
does give you some background to what prompted us to start the discussion.
Post by S***@labware.com
Hi Andreas,
Post by Andreas Sandberg
POWER and MIPS.
[...]
They won't be missed.
This is certainly not so here in my project where POWER and MIPS
are the primary available options, and dropping them would be pretty
much equivalent to discontinuing GEM5 altogether.
What feels even more sad, is that I can see how the very presence of
this discussion within the Overton window could be used by certain
closed-source advocates to undermine our "community support" argument
and thus try to weaken the position of not only our choice of GEM5
but some neighbouring open-source choices as well.
Similar discussions have been held in the Debian and Fedora communities.
These sort of discussions are not new, if a component isn¹t maintained,
it¹s not unusual for open source projects to officially drop support for
it. At least until someone steps up to maintain the component. If you are
interested in maintaining these ISAs, we obviously shouldn¹t drop them.

The whole notion of community support hinges on there being a community.
Judging by the commit history for the past 4 years, you can only really
claim that there is community support for ARM and x86.
Post by S***@labware.com
Post by Andreas Sandberg
users submitting patches for them
This one should easily exclude MIPS and POWER from candidates for
phasing out, well at least I do submit MIPS and POWER patches which
get merged into origin.
If you care about them enough to maintain them, then we should definitely
keep them. I don¹t want to scrap architectures for the sake of scrapping
them. It¹s about setting user expectations and knowing that someone cares
enough to fix issues as they crop up.
Post by S***@labware.com
Post by Andreas Sandberg
Ecosystem: We should phase out ISAs that don't have compiler
support. Being able to run code isn't very useful if you can't
compile code for the platform.
True; and the above's implied connection to MIPS and/or POWER, sounds
disturbing to me. Unless we assume a Redmond-like position that there
is one OS and one compiler and one software stack (based on an 80%
bell curve argument?), different people do different things and
so the definition of "compiler support" varies from project to
project. For example, in retargetable compilers (which is what I am
interested in) there is more than a single project based on ArchC
processor descriptions, and in that world, it's pretty much a
three-ISA world of ARM, MIPS and POWER. I certainly hope SPARC
to be equally usable soon too, but right now it's just barely limping;
Alpha, yes there is a one-in-a-million chance; RISC-V, that's a very
sweet dream indeed; and x86 seems even less likely.
So yeah, three ISAs with compiler support that I could use, and like
I said above, losing two of the three would not be much different
from giving up on GEM5 altogether.
The ecosystem argument is mainly aiming at Alpha. Both MIPS and POWER have
actively supported toolchains.
Post by S***@labware.com
Post by Andreas Sandberg
We should phase out ISAS that don't support full-system
unless there is a clear plan to add that support.
(The NULL ISA is an exception)
Users expect to be able to run an OS if an ISA is supported.
Actually, this statement was probably not fair assessment of the state of
things. SE mode can be very useful, so please ignore it. :)
Post by S***@labware.com
In another kingdom far away, there are those other people doing
those other extremely exciting things (including FS).
For me, the catastrophic showstopper was broken remote debugging
(gdb failing to even initiate the session) on every platform I tried,
so I had to fix that. This was the majority of the defining
discriminator between "working" and "not working"; FS didn't even
enter into the picture.
That was much appreciated! It has been /extremely/ helpful for debugging a
few kernel issues I was encountering a couple of months back.
Post by S***@labware.com
On the other hand, I do completely agree with you about functional
verification: I need to run a tall enough software stack (such as an
OS kernel) to believe in the simulation. But I can have enormous
stacks completely in SE. E.g. Java would serve as a good illustration
(not that's what we use here) of such a huge SW stack having no
relation to FS. I guess our situation here, is similar across the
board: there is Jakub's system on SPARC which is obviously tall enough
but not quite ready yet; then there is our stack which isn't there
yet; and then I wouldn't be surprised to learn about other people
with nontrivial workloads which are not the FS Linux kernel.
I completely agree with you. There are plenty of interesting things you
can do in SE mode. However, that depends on SE mode being functionally
correct having up-to-date kernel API support. We should definitely keep
SPARC (but consider retiring the Solaris test case) if we can get Jakub¹s
test case in there.

Functional correctness is probably my biggest pet peeve. During my PhD, I
had very painful experiences with gem5 and x86 which was far from
functionally correct. The kind of functional correctness issues I had
meant that only ~1/3 of SPEC could be run to completion and verify. I
simply cannot know how performance was affected by functional issues
affected convergence behaviour in the third that verified, but I suspect
it wasn¹t pretty. The bigger problem was that the breakdown of benchmarks
with issues was timing dependent, so changing the timing meant a new
benchmarks were braking. This was despite AMD caring about x86.
Post by S***@labware.com
Post by Andreas Sandberg
It reduces confusion among users. Having a handful of architectures
that are clearly incomplete and don't support full-system is not
very helpful to anyone.
Does enabling projects which result in new compilers being written
and papers being published in journals, count as helpful?
It¹s definitely helpful, but only as long as the behaviours of the
simulator are representative of real hardware. Again, users need to be
able to trust the simulator. A paper is only helpful if it reports
realistic behaviour. If the simulator is too broken to be representative
of the system you¹re simulating, the paper will be at best useless or at
worst misleading.

I hope that clarifies my position a bit. I really don¹t want to prune
things that are still useful, but we need to think about who maintains the
architectures that currently fall between the cracks.

Cheers,
Andreas


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Loading...