Discussion:
qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...
Paul Brook
2007-04-07 18:14:41 UTC
Permalink
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41

Modified files:
. : Makefile.target vl.h
hw : acpi.c adlib.c apb_pci.c arm_gic.c arm_pic.c
arm_pic.h arm_timer.c cs4231.c cuda.c
eepro100.c es1370.c fdc.c grackle_pci.c
gt64xxx.c heathrow_pic.c i8254.c i8259.c ide.c
integratorcp.c lsi53c895a.c m48t59.c m48t59.h
mc146818rtc.c mips_int.c mips_malta.c
mips_r4k.c mips_timer.c ne2000.c openpic.c
parallel.c pc.c pci.c pckbd.c pcnet.c pcspk.c
piix_pci.c pl011.c pl050.c pl080.c pl110.c
pl181.c pl190.c ppc.c ppc_chrp.c ppc_prep.c
prep_pci.c realview.c rtl8139.c sb16.c serial.c
shix.c slavio_intctl.c slavio_misc.c
slavio_serial.c slavio_timer.c smc91c111.c
sparc32_dma.c sun4m.c sun4u.c unin_pci.c
usb-ohci.c usb-uhci.c usb.h versatile_pci.c
versatilepb.c
target-mips : cpu.h
target-ppc : cpu.h
Added files:
hw : irq.c irq.h

Log message:
Unify IRQ handling.

CVSWeb URLs:
http://cvs.savannah.gnu.org/viewcvs/qemu/Makefile.target?cvsroot=qemu&r1=1.158&r2=1.159
http://cvs.savannah.gnu.org/viewcvs/qemu/vl.h?cvsroot=qemu&r1=1.207&r2=1.208
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/acpi.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/adlib.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/apb_pci.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_gic.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_pic.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_pic.h?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/arm_timer.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/cs4231.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/cuda.c?cvsroot=qemu&r1=1.11&r2=1.12
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/eepro100.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/es1370.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/fdc.c?cvsroot=qemu&r1=1.21&r2=1.22
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/grackle_pci.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/gt64xxx.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/heathrow_pic.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/i8254.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/i8259.c?cvsroot=qemu&r1=1.21&r2=1.22
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ide.c?cvsroot=qemu&r1=1.57&r2=1.58
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/integratorcp.c?cvsroot=qemu&r1=1.14&r2=1.15
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/lsi53c895a.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/m48t59.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/m48t59.h?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mc146818rtc.c?cvsroot=qemu&r1=1.7&r2=1.8
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_int.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_malta.c?cvsroot=qemu&r1=1.22&r2=1.23
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_r4k.c?cvsroot=qemu&r1=1.41&r2=1.42
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/mips_timer.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ne2000.c?cvsroot=qemu&r1=1.24&r2=1.25
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/openpic.c?cvsroot=qemu&r1=1.11&r2=1.12
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/parallel.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pc.c?cvsroot=qemu&r1=1.73&r2=1.74
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pci.c?cvsroot=qemu&r1=1.36&r2=1.37
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pckbd.c?cvsroot=qemu&r1=1.17&r2=1.18
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pcnet.c?cvsroot=qemu&r1=1.9&r2=1.10
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pcspk.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/piix_pci.c?cvsroot=qemu&r1=1.9&r2=1.10
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl011.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl050.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl080.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl110.c?cvsroot=qemu&r1=1.7&r2=1.8
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl181.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/pl190.c?cvsroot=qemu&r1=1.1&r2=1.2
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ppc.c?cvsroot=qemu&r1=1.13&r2=1.14
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ppc_chrp.c?cvsroot=qemu&r1=1.30&r2=1.31
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/ppc_prep.c?cvsroot=qemu&r1=1.32&r2=1.33
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/prep_pci.c?cvsroot=qemu&r1=1.3&r2=1.4
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/realview.c?cvsroot=qemu&r1=1.7&r2=1.8
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/rtl8139.c?cvsroot=qemu&r1=1.8&r2=1.9
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sb16.c?cvsroot=qemu&r1=1.22&r2=1.23
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/serial.c?cvsroot=qemu&r1=1.15&r2=1.16
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/shix.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_intctl.c?cvsroot=qemu&r1=1.10&r2=1.11
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_misc.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_serial.c?cvsroot=qemu&r1=1.12&r2=1.13
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/slavio_timer.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/smc91c111.c?cvsroot=qemu&r1=1.6&r2=1.7
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sparc32_dma.c?cvsroot=qemu&r1=1.2&r2=1.3
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sun4m.c?cvsroot=qemu&r1=1.31&r2=1.32
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/sun4u.c?cvsroot=qemu&r1=1.15&r2=1.16
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/unin_pci.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/usb-ohci.c?cvsroot=qemu&r1=1.5&r2=1.6
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/usb-uhci.c?cvsroot=qemu&r1=1.14&r2=1.15
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/usb.h?cvsroot=qemu&r1=1.12&r2=1.13
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/versatile_pci.c?cvsroot=qemu&r1=1.4&r2=1.5
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/versatilepb.c?cvsroot=qemu&r1=1.12&r2=1.13
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/irq.c?cvsroot=qemu&rev=1.1
http://cvs.savannah.gnu.org/viewcvs/qemu/hw/irq.h?cvsroot=qemu&rev=1.1
http://cvs.savannah.gnu.org/viewcvs/qemu/target-mips/cpu.h?cvsroot=qemu&r1=1.28&r2=1.29
http://cvs.savannah.gnu.org/viewcvs/qemu/target-ppc/cpu.h?cvsroot=qemu&r1=1.35&r2=1.36
J. Mayer
2007-04-07 18:32:21 UTC
Permalink
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Furthermore, this kind of patch that break other guys work would likely
to be discussed and not beeing imposed.

Please revert NOW.
--
J. Mayer <***@magic.fr>
Never organized
Paul Brook
2007-04-07 19:10:04 UTC
Permalink
Post by J. Mayer
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Can you give specific examples?
The CHRP code looks a bit broken, but no more so than before I started.
Post by J. Mayer
Furthermore, this kind of patch that break other guys work would likely
to be discussed and not beeing imposed.
It's been mentioned several times on this list (by Fabrice, specifically) that
this is the way to go.

I applied this now specifically because there are big ARM patches waiting to
be applied, and wanted to get this cleanup done before they went in.

Paul
Blue Swirl
2007-04-07 19:32:28 UTC
Permalink
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Can you give specific examples?
The CHRP code looks a bit broken, but no more so than before I started.
Post by J. Mayer
Furthermore, this kind of patch that break other guys work would likely
to be discussed and not beeing imposed.
It's been mentioned several times on this list (by Fabrice, specifically) that
this is the way to go.
I applied this now specifically because there are big ARM patches waiting to
be applied, and wanted to get this cleanup done before they went in.
Sparc seems undamaged, devices using IRQs work as before. SMP BogoMIPS gets
calculated and that needs per-CPU IRQs.

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Paul Brook
2007-04-07 19:46:56 UTC
Permalink
Post by Blue Swirl
Sparc seems undamaged, devices using IRQs work as before. SMP BogoMIPS gets
calculated and that needs per-CPU IRQs.
For the record I tested win2k, knoppix and x86 debian, mips test image, sparc
debian installer, arm test image, and a preinstalled ARM linux.

I didn't test ppc (other than that it built) because it didn't work to start
with. I've never managed to get the ppc target to successfully boot anything,
and there is no test image.

Paul
J. Mayer
2007-04-07 20:28:46 UTC
Permalink
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Can you give specific examples?
I'm talking about the CPU code.
There is NO notion of external IRQ allocation in the PowerPC
specification.
IRQ are completely out of the scope of the CPU emulation so the table
of 32 void *IRQ pointers is the CPU structure is a complete nonsense.
Where do you see in the PowerPC specification that those CPU have any
notion of how the EXTERNAL IRQ controler works ? Where do you see that a
machine with a PowerPC cannot manage more than 32 IRQ ? Where do you see
ANY NOTION OF EXTERNAL IRQ MANAGEMENT in the PowerPC specification ?
EXTERNAL IRQ MANAGEMENT IS NO WAY RELATED WITH CPU ! It's private to
each IRQ controller.
Saying anything else is just completely ignoring how real hardware
works.
SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
HAS TO BE REVERTED.
If you don't, I'LL REVERT ALL POWERPC CODE AFFECTED BY THIS PATCH.
Post by Paul Brook
The CHRP code looks a bit broken, but no more so than before I started.
Post by J. Mayer
Furthermore, this kind of patch that break other guys work would likely
to be discussed and not beeing imposed.
It's been mentioned several times on this list (by Fabrice, specifically) that
this is the way to go.
I did not received any single mail AT ALL saying "we're going to break
your code, you can now throw away all the work you're doing". NOT ONE.
So don't say "it's been discussed several times".
Post by Paul Brook
I applied this now specifically because there are big ARM patches waiting to
be applied, and wanted to get this cleanup done before they went in.
I don't contest you that you can do what is needed for ARM. JUST DON'T
DO WEIRD THINGS IN CODE YOU
SHOULD NOT MODIFY.

J. Mayer <***@magic.fr>
Never organized
Paul Brook
2007-04-07 20:45:26 UTC
Permalink
Post by J. Mayer
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Can you give specific examples?
I'm talking about the CPU code.
There is NO notion of external IRQ allocation in the PowerPC
specification.
Quoting from ppc.c (both before and after my patch):

/*****************************************************************************/
/* PowerPC internal fake IRQ controller
* used to manage multiple sources hardware events

My patch did not change how this worked at all. It just changed it from using
an ad-hoc system of callpacks and opaque parameters to using a standard
mechanism for connecting interrupt sources, be they internal or external.

MIPS works exactly the same way.
Post by J. Mayer
Where do you see that a
machine with a PowerPC cannot manage more than 32 IRQ ?
Any PPC that needs more that 32 internal interrupt sources is already broken.
Notice how the old bitmask is a uint32_t.
Post by J. Mayer
SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
HAS TO BE REVERTED.
I can only fix the code that I have access to.

IMHO your request that I revert changes because they allegedly break code that
noone else has access to is unreasonable. My changes are a significant
cleanup to qemu internals.

However if there is agreement from other maintainers I will revert my patch.

Paul
J. Mayer
2007-04-07 22:18:13 UTC
Permalink
It seems that you don't figure out how real hardware works AT ALL.
IRQ callbacks and private data CANNOT BE SHARED as they are INTERNAL TO
ONE IRQ CONTROLLER. IRQ management, once again, is completelly specific
to an IRQ controller technology and absolutely not related to the CPU
you may use on the same board. So it's a complete nonsense to try to
share IRQ notion in the CPU. IRQ notions and management is completely
different to one IRQ controller technology to another.
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Can you give specific examples?
I'm talking about the CPU code.
There is NO notion of external IRQ allocation in the PowerPC
specification.
/*****************************************************************************/
/* PowerPC internal fake IRQ controller
* used to manage multiple sources hardware events
My patch did not change how this worked at all. It just changed it from using
an ad-hoc system of callpacks and opaque parameters to using a standard
mechanism for connecting interrupt sources, be they internal or external.
MIPS works exactly the same way.
No. MIPS CPU have an internal IRQ controller and often one or many
external ones. It's completely different and cannot be easily compared
to what done internally in a PowerPC.
Post by Paul Brook
Post by J. Mayer
Where do you see that a
machine with a PowerPC cannot manage more than 32 IRQ ?
Any PPC that needs more that 32 internal interrupt sources is already broken.
Notice how the old bitmask is a uint32_t.
You don't understand a single word of what I wrote or you don't
understand at all what is the difference between a CPU and an IRQ
controller.
The code you quote concerns INTERNAL EXCEPTION MANAGEMENT. It's no way
related with an IRQ controller and will never be. One or more exception,
like in any CPU, can be connected to an IRQ controller (or more,
cascaded, for example). The IRQ management is done inside the IRQ
controller, not in the CPU and all the hardware IRQ interface are
SPECIFIC to the IRQ controller. Then, this is to be emulated internally
in the controller and has no notion to be shared in the CPU (as there is
none in the real hardware).
The existing code ACTUALLY WORKS with more than 32 IRQs as you can
cascade any number of IRQ controller one to the other. Each IRQ
controller handle its own events and manage the callbacks to the
cascade. There is only one controller in the chain connected to the CPU.
Then, you can virually have an infinite number of IRQ, managed by a
chain of controllers, using different technologies then cannot share any
data execpt the IRQ events on the cascade. THAT's HOW REAL HARDWARE
WORKS AND THAT'S HOW IT'S EMULATED FOR POWERPC IN THE CODE BEFORE YOU
BREAK IT !
Post by Paul Brook
Post by J. Mayer
SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
HAS TO BE REVERTED.
I can only fix the code that I have access to.
You broke the commited code.... And you also completely break any new
development for the PowerPC target as most real machine cannot be
emulated now (even PREP is broken, using 3 IRQ controllers).
Post by Paul Brook
However if there is agreement from other maintainers I will revert my patch.
I only ask you to revert all that affects the PowerPC target. If the
other developpers want their target to follow this broken way, it does
not hurt me as long as it does not break fundamental concept for PowerPC
emulation.
--
J. Mayer <***@magic.fr>
Never organized
Thiemo Seufer
2007-04-07 22:49:35 UTC
Permalink
J. Mayer wrote:
[snip]
Post by J. Mayer
Post by Paul Brook
Post by J. Mayer
SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
HAS TO BE REVERTED.
I can only fix the code that I have access to.
You broke the commited code.... And you also completely break any new
development for the PowerPC target as most real machine cannot be
emulated now (even PREP is broken, using 3 IRQ controllers).
It might help here if you could provide a PREP sample image which can be
used as a basic smoketest, like the ones for other architectures on
Fabrice's page.


Thiemo
Paul Brook
2007-04-07 23:13:14 UTC
Permalink
Post by J. Mayer
It seems that you don't figure out how real hardware works AT ALL.
IRQ callbacks and private data CANNOT BE SHARED as they are INTERNAL TO
ONE IRQ CONTROLLER. IRQ management, once again, is completelly specific
to an IRQ controller technology and absolutely not related to the CPU
you may use on the same board. So it's a complete nonsense to try to
share IRQ notion in the CPU. IRQ notions and management is completely
different to one IRQ controller technology to another.
<snip>

You appear to have misunderstood what my patch does. The arguments you use in
the part of the email I snipped are the same resons why I created this patch.


The most important thing to understand is that the "qemu_irq" object is simply
an abstract interface for connecting two devices. One of which produces a
single-bit output, and the other does something in response to that signal.
qemu_irq isn't an ideal name because it's used for things other than physical
IRQ pins. However it's the best we (Me, Theimo, or anyone else on #qemu about
24 hours ago) could come up with.

The purpose of this abstraction is precisely so that interrupt sources and
sinks can be chained and rearranged in arbitrary fashion.


In the specific case of PPC a typical system, as you say, has several cascaded
levels of interrupt handling. You have multiple external interrupt
controllers, including interrupts from PCI bus based devices.

Contrary to your assertion, a PPC cpu does contains an internal vectored
interrupt controller. This handles the external IRQ pin, plus various
internal interrupt sources (as defined by the PPC_INTERRUPT_* constants).
This is what the IRQ array in the cpu state structure is for. I see no
fundamental reason why this should be treated any differently to an external
interrupt controller.

Because these are currently implemented in the same file I didn't bother
converting them fully and they mostly still use ppc_set_irq dircetly instead
of going via qemu_set_irq. If/when ppc.c is split up and/or cpus with more
than one external IRQ pin are added the generic IRQ routing mechanisms allow
this to happen without having to export ppc_set_irq.

PPC is a slightly bad example because it still has ppc_openpic_irq. This is
exactly the sort of horrible system specific hack I'm trying to replace.


Before my patch chained interrupts were handled with a messy ad-hoc
combination of callbacks and opaque parameters. Some devices were properly
parameterized, others assumed that their IRQ output went to a specific device
or CPU.

My patch introduces a generic mechanism for devices to talk to an interrupt
controller. It means that the code signalling the interrupt doesn't need to
know or care where it ends up. The logic for implementing a particular
interrupt controller is still private to that device.

I have not created a single flat IRQ super-controller, just a consistent
mechanism for glueing the existing bits of emulation together.
Post by J. Mayer
You broke the commited code.... And you also completely break any new
development for the PowerPC target as most real machine cannot be
emulated now (even PREP is broken, using 3 IRQ controllers).
Can you send me a testcase of something that worked before my patch and
doesn't afterwards? As I said above, handling of multiple IRQ controllers is
significantly better after my patch than it was before.

Paul
J. Mayer
2007-04-07 23:54:08 UTC
Permalink
Post by Paul Brook
Post by J. Mayer
It seems that you don't figure out how real hardware works AT ALL.
IRQ callbacks and private data CANNOT BE SHARED as they are INTERNAL TO
ONE IRQ CONTROLLER. IRQ management, once again, is completelly specific
to an IRQ controller technology and absolutely not related to the CPU
you may use on the same board. So it's a complete nonsense to try to
share IRQ notion in the CPU. IRQ notions and management is completely
different to one IRQ controller technology to another.
<snip>
You appear to have misunderstood what my patch does. The arguments you use in
the part of the email I snipped are the same resons why I created this patch.
So your patch is not useful for the PowerPC target as cascading could
already works and does not need anything hardcoded in the CPU (once
again, IRQ sources is no way related to the CPU).
Post by Paul Brook
The most important thing to understand is that the "qemu_irq" object is simply
an abstract interface for connecting two devices. One of which produces a
single-bit output, and the other does something in response to that signal.
qemu_irq isn't an ideal name because it's used for things other than physical
IRQ pins. However it's the best we (Me, Theimo, or anyone else on #qemu about
24 hours ago) could come up with.
The purpose of this abstraction is precisely so that interrupt sources and
sinks can be chained and rearranged in arbitrary fashion.
Which was already done. Why change what works, so ? Why BREAK ?
Post by Paul Brook
In the specific case of PPC a typical system, as you say, has several cascaded
levels of interrupt handling. You have multiple external interrupt
controllers, including interrupts from PCI bus based devices.
Contrary to your assertion, a PPC cpu does contains an internal vectored
interrupt controller. This handles the external IRQ pin, plus various
internal interrupt sources (as defined by the PPC_INTERRUPT_* constants).
This is what the IRQ array in the cpu state structure is for. I see no
fundamental reason why this should be treated any differently to an external
interrupt controller.
I guess you make a confusion between IRQ, which are external and managed
with an IRQ controller and exceptions, which are handled inside the
PowerPC core. There is no IRQ controller inside the PowerPC. You can
read the specification in any way you want, you will find none.
IRQ controllers are peripheral devices. So, nothing about them can be
managed in the CPU structure. What when there is more than one CPU ? IRQ
number is a notion internal to one IRQ controller and has no meaning
outside of it. How can you even think of using it to reference
Post by Paul Brook
Because these are currently implemented in the same file I didn't bother
converting them fully and they mostly still use ppc_set_irq dircetly instead
of going via qemu_set_irq. If/when ppc.c is split up and/or cpus with more
than one external IRQ pin are added the generic IRQ routing mechanisms allow
this to happen without having to export ppc_set_irq.
PPC is a slightly bad example because it still has ppc_openpic_irq. This is
exactly the sort of horrible system specific hack I'm trying to replace.
Once again, I guess you don't understand what this code does. Openpic,
as many IRQ controllers, has a lot of inputs and not only a single
output.This code is there to connect external IRQ sources comming from
an OpenPIC to internal exceptions of the PowerPC. As this is hardwired
in the PowerPC core and cannot be changed, it's not especially a hack.
To be _really_ generic and do exactly what is done is real hardware
devices, it would need another cascaded stage. I did not add this stage
because it would have just add performance penalties but would not have
added any real advantage, imho.
Post by Paul Brook
Before my patch chained interrupts were handled with a messy ad-hoc
combination of callbacks and opaque parameters. Some devices were properly
parameterized, others assumed that their IRQ output went to a specific device
or CPU.
So, you just have to fix the messy devices. Not to break the CPU
emulation code or structure.
Post by Paul Brook
My patch introduces a generic mechanism for devices to talk to an interrupt
controller. It means that the code signalling the interrupt doesn't need to
know or care where it ends up.
That's exactly why some IRQ callbacks have been added in some devices,
just not to have to care where the interrupt ends up or even if the IRQ
is actually raised. No need to break anything. Just fix the old devices
that did not use this logic.
Post by Paul Brook
The logic for implementing a particular
interrupt controller is still private to that device.
Your patch is even worse than what already exists and work, add
confusion by breaking concepts (the IRQ controller is an external
device, whatever the CPU you use), add garbage in the CPU structure, add
a mess to handle multi-cpu/multi PIC systems (which are more than
common, these days), add arbitrary limits with no justifications, ....
So, no new features, broken concepts, new mess, new limitations and more
complicated code...
It's unacceptable.

To give you an real example why arbitrary limits are not acceptable AT
ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
and about 500 IRQ sources. How can you even pretend add a limited
structure in the CPUState structure when this is exactly the kind of
device some people want to emulate in Qemu ? Your concept is completely
broken, you have to admit it. You can never put peripheral informations
in the CPUState structure. What if I decide to put the NIC registers in
the CPU structure ? It's exactly the same as what you did...


The funny thing is that you even did not fix the only bug that makes the
OpenPic not really cascadable: the last argument of the openpic_init has
to be changed to be void ** instead of CPUState ** (same in
the .IRQ_dst_t structure). No need for more (but this patch is needed, I
did not commit it because I don't need it just now...)

[...]
--
J. Mayer <***@magic.fr>
Never organized
Thiemo Seufer
2007-04-08 00:04:20 UTC
Permalink
J. Mayer wrote:
[snip]
Post by J. Mayer
To give you an real example why arbitrary limits are not acceptable AT
ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
and about 500 IRQ sources.
Care to tell which one this is?
Post by J. Mayer
How can you even pretend add a limited
structure in the CPUState structure when this is exactly the kind of
device some people want to emulate in Qemu ? Your concept is completely
broken, you have to admit it. You can never put peripheral informations
in the CPUState structure.
At least for MIPS it makes sense to put the CPU-internal controller
in exactly that place.


Thiemo
J. Mayer
2007-04-08 07:49:55 UTC
Permalink
Post by Thiemo Seufer
[snip]
Post by J. Mayer
To give you an real example why arbitrary limits are not acceptable AT
ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
and about 500 IRQ sources.
Care to tell which one this is?
I'm sorry, I'm no sure I can (NDA rules....).
Let's say it's a chips used in some consumer electronics products.
Post by Thiemo Seufer
Post by J. Mayer
How can you even pretend add a limited
structure in the CPUState structure when this is exactly the kind of
device some people want to emulate in Qemu ? Your concept is completely
broken, you have to admit it. You can never put peripheral informations
in the CPUState structure.
At least for MIPS it makes sense to put the CPU-internal controller
in exactly that place.
It does not. If you look well, the IRQ controller is not in the CPU.
Only the exception are managed in the CPU. The "internal" IRQ controller
is a peripheral device located in the CP0. OK, the CP0 is tightly
coupled to the CPU so it's easier to consider it as part of the CPU,
when emulating it. But it seems like you could imagine a MIPS CPU
without a CP0 coprocessor (even if it will never happen in the real
world), no ?

The problem is also: what does this patch adds, but complexity and
arbitrary limitations ?
What do you need to route an IRQ ?
-> A peripheral destination
What we got now ?
-> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
state
Is more needed to have a generic routing approach ?
-> no. This can work to route any signal
Can we do with less ?
-> no. You need the 3 parameters.
Can we share data ?
-> It seems not. The opaque is used internally by the IRQ controller.
The PIN number has a meaning only inside the IRQ controller. And the
meaning of the state depends on the IRQ controller state.
The only thing that could be changed to hide the complexity is define a
structure that contain all information, like the struct IRQState.
But this structure cannot be shared as it has no signification outside
of an IRQ controller.
What can be done (but once again, it changes nothing, just hide the
"complexity"), is to replace the { callback, n_IRQ } in devices
structure by a IRQState structure and have the inline functions.

static inline void qemu_irq_set_state(struct IRQState *st, int level)
{
if (st->handler != NULL)
(*st->handler)(st->opaque, st->n, level);
}
static inline void qemu_irq_raise(struct IRQState *st)
{
qemu_irq_set_state(st, 1);
}
static inline void qemu_irq_lower(struct IRQState *st)
{
qemu_irq_set_state(st, 0);
}

If you want to be generic, as it is said it's supposed to be (but is not) to be able to manage any signal, then this is not sufficient.
One will also have to get the value. If you want to emulate GPIOs, for example, you need to store the current state
to be able to get it back easily. And for GPIO, you may prefer that changing the state has no action but just be able
to read back the current PIN level at any time.

To achieve this, you have to have a structure:
struct PINState {
qemu_pin_handler handler;
void *opaque;
int n;
int level;
};

static inline void qemu_pin_set_state(struct PINState *st, int level)
{
if (st->handler != NULL && level != st->level != level) {
(*st->handler)(st->opaque, st->n, level);
st->level = level;
}
}
static inline int qemu_pin_get_state(struct PINState *st)
{
return st->level;
}

static inline void qemu_ping_raise(struct PINState *st)
{
qemu_pin_pin_state(st, 1);
}
static inline void qemu_pin_lower(struct PINState *st)
{
qemu_pin_set_state(st, 0);
}

The hw/irq.c code is not needed and meaningless. The first function should be inlined, unless the goal is to slowdown the emulator.
The second function has no meaning: you can never globally allocate resources that have no meaning outside of a limited
scope, which is the case for IRQ or any physical connection on a real hardware.

The PINState structure can be store only in the peripheral device that will change the PIN state and the peripheral that will read this state.
Trying to store it in a generic place is completely meaningless: a PIN state (I talk about real hardware, now) has absolutelly
no generic meaning. It's only significant between two devices. You eventually can have more than one device changing the
PIN state (shared IRQ, I2C bus, ....) and more than one device using this PIN state (I2C, once again). But outside of the scope
of those devices, it means nothing. So the information is to be stored inside those devices, anywhere else is meaningless.
--
J. Mayer <***@magic.fr>
Never organized
J. Mayer
2007-04-08 08:38:46 UTC
Permalink
Post by J. Mayer
I'm sorry, I'm no sure I can (NDA rules....).
Let's say it's a chips used in some consumer electronics products.
I gave this example to show that we cannot pre-determine any limit to
the number of PINs we have to manage, even if it's a poor design and
that no one need those 500 IRQ sources (50 would be sufficient if it was
well designed but that the way the actual hardware has been
designed...).

Here's an example which is documented:
A CHRP machine is supposed to have an OpenPIC controller (up to 32 IRQ
sources and up to 4 output IRQ pins) and 2 i8259. This simple machine
already has more than 32 IRQs.
Some may also have 2 cascaded OpenPICs.
It will also have one or more PCI controllers (which have it's own IRQ
controller, n inputs, 4 outputs, on actual hardware).
Some complex PCI devices may also have an internal IRQ controller if
they can generate a lot of different hardware events. Those devices can
even be hotplugged.

Then, can you pre-determine the maximum number of IRQ of a hardware
platform ? I can't, because it's conceptually impossible.

[....]

Ooops... Lots of typo in what I wrote....
Post by J. Mayer
static inline void qemu_pin_set_state(struct PINState *st, int level)
{
if (st->handler != NULL && level != st->level != level) {
I meant .... && level != st->level) {

[...]
Post by J. Mayer
static inline void qemu_ping_raise(struct PINState *st)
I meant qemu_pin_raise (not ping).
You can even have an allocator to ease sharing the information between
the devices, like:
static inline PINState *qemu_pin_alloc (void *opaque, int n, int
initial_level)
{
PINState *st;

st = qemu_mallocz(sizeof(PINState));
if (st != NULL) {
st->opaque = opaque;
st->n = n;
st->level = initial_level;
}
}

And note that if you really want to emulate GPIOs, you may also need to
add a default_level field to the structure and a bitmask to know which
devices are really driving the PIN (as it reaches its default state only
when no one is driving it). and a new function to signal the case where
the PIN is beeing released (not drived) by a peripheral. And a
peripheral need to identify itself when calling the pin_raise,
pin_lower, pin_release function. The only limitation of this approach
may be the number of peripheral driving one single signal on the board,
which can not be > of the size of the bitfield. But you can have as many
handled signals as you need too.

Once again, one of the two structure is not needed (qemu_irq /
IRQState): there is only one physical state so having 2 different
description of the same thing is conceptually broken. Nothing can be
stored in a generic place as a PIN connection is a completely local
concept between a set of devices. And once again, IRQ are not bound to a
CPU. Storing any informations about IRQs in the CPU structure is
meaningless. Where do you store the IRQ informations when you have 2, 4
CPUs connected to one OpenPIC ? Choose a random CPU ? Only use one ?

Then, we can see the code that has been commited does not solve the
problems of generic signal routing and is conceptually completelly
broken.

One last point:
I thought more about the ppc_openpic_irq function and I have to admit
it's not the good approach.
Yes, this has to be changed to store the IRQ destination inside the
OpenPIC controller and directly call the ppc_set_irq callback. In fact,
there should be a function replacing the ppc_openpic_irq one that
handles the external exception sources and callback ppc_set_irq (this
function also manages internal hardware exception sources). The code in
hw/ppc.c would not be changed a lot but the OpenPIC would be more
generic and may allow some crazy schemes where the outputs would not be
connected to the same destination device (usually the CPU). And it may
make easier to use OpenPIC on hardware that are not PowerPC base (with
no hack), which is imho more important (OpenPIC was designed for and
used on AMD SMP PC platforms, originally). I'll change this, once the
IRQ problems will be solved (which seems urgent, more than anything
else).
--
J. Mayer <***@magic.fr>
Never organized
Thiemo Seufer
2007-04-08 14:41:03 UTC
Permalink
Post by J. Mayer
Post by Thiemo Seufer
[snip]
Post by J. Mayer
To give you an real example why arbitrary limits are not acceptable AT
ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
and about 500 IRQ sources.
Care to tell which one this is?
I'm sorry, I'm no sure I can (NDA rules....).
Let's say it's a chips used in some consumer electronics products.
Post by Thiemo Seufer
Post by J. Mayer
How can you even pretend add a limited
structure in the CPUState structure when this is exactly the kind of
device some people want to emulate in Qemu ? Your concept is completely
broken, you have to admit it. You can never put peripheral informations
in the CPUState structure.
At least for MIPS it makes sense to put the CPU-internal controller
in exactly that place.
It does not. If you look well, the IRQ controller is not in the CPU.
Only the exception are managed in the CPU. The "internal" IRQ controller
is a peripheral device located in the CP0.
It is not a peripheral but an integral part of any MIPS-compatible CPU.
The architecture allows since MIPS{32,64}R2 to optionally externalize
interupts (the so-called VEIC mode), but even those devices have to
implement the traditional "compatibility mode" interrupt handling.

This is spelt out e.g. in MD00091, page 32, as available from
http://www.mips.com/products/resource_library/product_materials/
Post by J. Mayer
OK, the CP0 is tightly
coupled to the CPU so it's easier to consider it as part of the CPU,
when emulating it. But it seems like you could imagine a MIPS CPU
without a CP0 coprocessor (even if it will never happen in the real
world), no ?
No. Since MIPS{32,64}R2 the CP0 is standardized and a mandatory part of
a MIPS compatible CPU.
Post by J. Mayer
The problem is also: what does this patch adds, but complexity and
arbitrary limitations ?
For MIPS it adds an abstraction layer between the interrupt controller
and the CP0 registers which will be useful to implement support for
SMP devices.
Post by J. Mayer
What do you need to route an IRQ ?
-> A peripheral destination
What we got now ?
-> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
state
A pin number doesn't look like a reasonable abstraction for a packetized
interrupt on an interconnection fabric (like it is used on e.g O2000).
It may do for the machines we currently emulate, though.


Thiemo
J. Mayer
2007-04-08 16:31:27 UTC
Permalink
Post by Thiemo Seufer
Post by J. Mayer
Post by Thiemo Seufer
[snip]
Post by J. Mayer
To give you an real example why arbitrary limits are not acceptable AT
ALL: I know an embedded Mips device (widely used !) with 2 CPU, 8 PIC
and about 500 IRQ sources.
Care to tell which one this is?
I'm sorry, I'm no sure I can (NDA rules....).
Let's say it's a chips used in some consumer electronics products.
Post by Thiemo Seufer
Post by J. Mayer
How can you even pretend add a limited
structure in the CPUState structure when this is exactly the kind of
device some people want to emulate in Qemu ? Your concept is completely
broken, you have to admit it. You can never put peripheral informations
in the CPUState structure.
At least for MIPS it makes sense to put the CPU-internal controller
in exactly that place.
For the internal IRQ controller, OK. For the external ones, it makes no
sense to put any informations about them into the CPU.
Post by Thiemo Seufer
Post by J. Mayer
It does not. If you look well, the IRQ controller is not in the CPU.
Only the exception are managed in the CPU. The "internal" IRQ controller
is a peripheral device located in the CP0.
It is not a peripheral but an integral part of any MIPS-compatible CPU.
The architecture allows since MIPS{32,64}R2 to optionally externalize
interupts (the so-called VEIC mode), but even those devices have to
implement the traditional "compatibility mode" interrupt handling.
This is spelt out e.g. in MD00091, page 32, as available from
http://www.mips.com/products/resource_library/product_materials/
Post by J. Mayer
OK, the CP0 is tightly
coupled to the CPU so it's easier to consider it as part of the CPU,
when emulating it. But it seems like you could imagine a MIPS CPU
without a CP0 coprocessor (even if it will never happen in the real
world), no ?
No. Since MIPS{32,64}R2 the CP0 is standardized and a mandatory part of
a MIPS compatible CPU.
Yes, I know MIPS want always CP0 to be present. I should have put a
smiley somewhere. I wanted just to point the fact that the CPU itself
does not need the CP0 controller to run and that one could easily
imagine designing a core without the CP0 controller. Yes, this would not
follow the specification but it would be able to run....
The biggest conceptual problem would you have to read a CP0 register when you want to know what coprocessor are actually implemented !
But all this is a foolish idea....
Post by Thiemo Seufer
Post by J. Mayer
The problem is also: what does this patch adds, but complexity and
arbitrary limitations ?
For MIPS it adds an abstraction layer between the interrupt controller
and the CP0 registers which will be useful to implement support for
SMP devices.
Could you please explain me what you can do now that you were not able
too without this patch ?
Post by Thiemo Seufer
Post by J. Mayer
What do you need to route an IRQ ?
-> A peripheral destination
What we got now ?
-> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
state
A pin number doesn't look like a reasonable abstraction for a packetized
interrupt on an interconnection fabric (like it is used on e.g O2000).
It may do for the machines we currently emulate, though.
Could you please tell me more about this problem ?
And could you please explain me how the patch solves this, as it just
does not seem to me that it uses more than an IRQ number to identify an
IRQ ?

I though a little bit more about a generic IRQ and/or IO implementation
and it seems that the commited patch is far from solving any pending
problem about it.
What appears to me is that a IO can have 3 "natural state", when not
driven:
- pulled up
- pulled low
- floating
An IO driver can then have 3 actions on an IO line:
- drive it up
- drive it low
- release it
It appears to me that all the possible combinations of thoses natural
states / driving states have to be managed properly to emulate IRQ.
For ISA like IRQ, that are not sharable, IRQ lines are naturally
floating and can be drived up or low
For PCI like IRQs, that are sharable, IRQ lines are naturally pulled up
and can be drived down or released.
Some other designs may use (less often) pulled down sharable IRQ lines.
The first idea is to say that floating IOs can only be driven up or low
and pulled up/low IO can only be driven or not driven. It appears that
because of poorly designed hardware or (more often) software bugs, weird
situations can happen and must be handled: floating IO not driven (which
may gives undefined state on real hardware) or pulled up IO drived up
(same for down, this may give intermediate states between logical 0 and
1 thus product "strange" results).
All those situations can be seen on IRQ line so a generic IRQ
implementation have to manage all those cases.
Now, let's see the other side of the connection. A simplistic approach
would be to say we just need a callback or a stored value. Thinking a
little more about it shows that we cannot do without having at least 2
possible outputs. The reason is that you often have the possibility for
the driving device to read back the value of the IRQ pin and that you
cannot rely on the device internal state for this as an IO may be shared
or may not have the awaited value, especially if not driven properly by
the software.
Even worse, if you are to drive an IRQ with a pure GPIO pin you will
have to set this pin as an bidirectional I/O if you want to be able to
read its actual value back. On some embedded hardware, where I/O pins
cannot be set in a bidirectional mode, you may have to use 2 I/O to
drive and read back an IRQ line state. And it's very useful to be able
to read back hardware states, and quite usual.
My conclusion about it is that it seems that it is mandatory to manage
the 3 possible natural states of an IO, the 3 "driven" state, multiple
sources and multiple outputs if you want to have a correct emulation of
physical IRQ lines behavior. It seems all the work is still to be done
to achieve this.
And doing this would provide a correct emulation for IRQ line and it
seems to me that it will be perfectly usable to emulate any kind of GPIO
line.
But I still cannot agree that the informations about an IO line is to be
stored anywhere else than in the driver device and the receiver one. CPU
and other devices even don't have any "knowledge" of an IO line that is
not connected to them, thus should never have to carry any information
about it.
And I still cannot see why two different structures would be needed to
express one single simple concept of an IO connection.
Please explain me about those points if I'm wrong.
--
J. Mayer <***@magic.fr>
Never organized
Natalia Portillo
2007-04-08 20:43:20 UTC
Permalink
Hi all,

I have a huge list of operating systems (both closed and open source, that
works and that doesn't work under QEMU) that can be used to check that QEMU
doesn't broke (or even, that it corrects a non-working state).

I've already discussed that with Fabrice.

And I think it is better to check an installed image that to test it only
boots or installs (faster at least).

Fabrice and me discussed about taking screenshot per some seconds to compare
with known took screenshots and if something fails shutdown the VM and send
an email.

But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.

The best solution I think is to get a way to send QEMU the screenshot to
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)

What do you think?

I have enough spare space to hold the boot images (600Gibibytes free), and a
machine that should be able to automatically checkout and compile QEMU
(Athlon XP 2000+, 768Mb RAM, 600GiB free, Gentoo Linux).

Just, I have not the knowledge to make the script that boots qemu, says qemu
to take the screenshot, compares the took one with the last one, stops qemu,
sends the last screenshot by email, compresses all took screenshots, goes to
next VM, so on. (Preferibly without X11 running, as the machine is a mostly
headless P2P and File server)

If anyone can do this, I will make the boot images for all my OSes and start
it.

I think, it is a great idea (and a better way to update my -currently dead
but no more spammed- official OS support list).

Regards,
Natalia Portillo
Eduardo Felipe
2007-04-08 22:07:55 UTC
Permalink
Post by Natalia Portillo
But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.
It does exist... it's called VNC server. Maybe you can have a look at:

http://www.sodan.org/~penny/vncrec/ <http://www.sodan.org/%7Epenny/vncrec/>

or

http://suif.stanford.edu/vncplay/freenix05-html/

I don't know any of them, but they can be useful.

Also this one:

http://www.unixuser.org/~euske/vnc2swf/pyvnc2swf.html
<http://www.unixuser.org/%7Eeuske/vnc2swf/pyvnc2swf.html>

It's vncrec compatible and can record and replay sessions.

The best solution I think is to get a way to send QEMU the screenshot to
Post by Natalia Portillo
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)
pyvnc2swf has the ability to create image files from VNC sessions. It is
easily customizable and a special instruction could be recorded into the
.vnc files meaning:

1) Request a full screen update
2) Turn screen buffer into an image
3) Compare image to reference

Control points could be chosen in recording time and inserted into the .vnc
by some key combination.

Regards,
Eduardo
Natalia Portillo
2007-04-08 23:53:12 UTC
Permalink
It doesn't fullfil the requirements for an automated testing,
however, it is a good idea, but not inside the plannings I proposed.
Post by Natalia Portillo
But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.
http://www.sodan.org/~penny/vncrec/
or
http://suif.stanford.edu/vncplay/freenix05-html/
I don't know any of them, but they can be useful.
http://www.unixuser.org/~euske/vnc2swf/pyvnc2swf.html
It's vncrec compatible and can record and replay sessions.
The best solution I think is to get a way to send QEMU the
screenshot to
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be
checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)
pyvnc2swf has the ability to create image files from VNC sessions.
It is easily customizable and a special instruction could be
1) Request a full screen update
2) Turn screen buffer into an image
3) Compare image to reference
Control points could be chosen in recording time and inserted into
the .vnc by some key combination.
Regards,
Eduardo
Eduardo Felipe
2007-04-09 09:36:00 UTC
Permalink
Sorry, I'm afraid I badly mixed up things in my mind.

vncrec and pyvnc2swf record VNC server responses, not client actions, so
they will hardly be of any help.

To record and replay client actions these tools are more appropriate:

http://cyberelk.net/tim/rfbproxy/
http://cyberelk.net/tim/rfbplaymacro/

Although they seem not to be available at this moment...

Regards,
Eduardo
Rob Landley
2007-04-09 21:19:30 UTC
Permalink
Post by Natalia Portillo
Hi all,
I have a huge list of operating systems (both closed and open source, that
works and that doesn't work under QEMU) that can be used to check that QEMU
doesn't broke (or even, that it corrects a non-working state).
I vaguely intend to get Firmware Linux to rebuild itself under qemu. Right
now it's building target images for nine different platforms. (Although
that's not as impressive as it sounds. Two of them, sparc and powerpc, don't
actually run under qemu yet. i586 and armv5l are trivial variants of i686
and armv4l. And and mips big endian just got added today and wasn't in the
last release.)

Anyway, I've found compiling large software packages to be a reasonable stress
test for most systems, and it's entirely scriptable. Not much of a test of
any of the peripheral hardware, though.

Rob
--
Penguicon 5.0 Apr 20-22, Linux Expo/SF Convention. Bruce Schneier, Christine
Peterson, Steve Jackson, Randy Milholland, Elizabeth Bear, Charlie Stross...
Jamie Lokier
2007-04-10 11:24:20 UTC
Permalink
Post by Natalia Portillo
But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.
The best solution I think is to get a way to send QEMU the screenshot to
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)
If there was a way to get QEMU to stop when the screen matches a
particular stored screenshot, that would make it a lot more scriptable.

For example, you could record the steps to install a particular OS
from installation media as a series of screenshots showing dialog
boxes, and script commands like pressing keys or clicking at x, y when
those screens match.

Another useful event to stop on would be when the processor hits an
x86 HLT instruction or whatever is the equivalent on other
architectures, or particular recognised waiting loops - in conjunction
with there being no outstanding I/O. Maybe that's an appropriate time
to check for matching screenshots.

That kind of script would be nice when you want to create a new fresh
installation of some OS, but with a few options changed. The script
would not be too hard to stop at a certain point, and set different
option in the installation, then continue. Currently I do this
manually and it's quite tedious.

Similarly for actually running applications in a VM.

-- Jamie
Pierre d'Herbemont
2007-04-10 12:00:49 UTC
Permalink
Post by Natalia Portillo
But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.
There is cxtest (GPL) which has been created for that matter, it
should be interesting to give it a try:
http://www.cxtest.org/

"Cxtest is an open source project that provides visual regression
testing facilities for X11 based systems. It can automate basic X
Window functionality, including finding X windows by title or by
graphic picture"


Pierre.
Dan Shearer
2007-07-27 14:21:42 UTC
Permalink
On Sun, Apr 08, 2007 at 09:43:20PM +0100, Natalia Portillo wrote:

Whoops, slightly late reply :-)
Post by Natalia Portillo
I have a huge list of operating systems (both closed and open source, that
works and that doesn't work under QEMU) that can be used to check that QEMU
doesn't broke (or even, that it corrects a non-working state).
Where the problem with QEMU is not something basic (eg doesn't boot) it
can be useful to use nested virtualisation. Although QEMU in QEMU is of
course slow because that case hasn't been optimised, if you have a
single image within which are many other images savevm'd just before
the error condition occurs, you can run a repeatable testsuite quite
quickly. And since you can snapshot the outer image, you know that you
are running these tests in exactly the same way every time. (There are a
lot of other uses for recursive virtualisation, I'm a fan of it!)
Post by Natalia Portillo
And I think it is better to check an installed image that to test it only
boots or installs (faster at least).
Agreed. As to installs I had to do something similar for different VM
systems a while ago and I found it useful to have a frozen image just
before the typical "detecting hardware" phase of an installation. The
point of this was not to compare the hardware that was detected, just to
detect major malfunctions. Hardware probing like this is uniquely
stressful. It isn't hard to notice if an installer crashes, or the VM
crashes, or there's lot's of errors.
Post by Natalia Portillo
Fabrice and me discussed about taking screenshot per some seconds to compare
with known took screenshots and if something fails shutdown the VM and send
an email.
But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.
Why can't you redirect the QEMU monitor to something easily accessible,
eg a virtual serial port, and then use mouse_move and sendkey monitor
commands to drive testing, and screendump to save images for comparison?
Those commands have been in the monitor for years, so is there something
I'm not understanding here?
Post by Natalia Portillo
The best solution I think is to get a way to send QEMU the screenshot to
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)
What do you think?
The problem is synchronisation given that QEMU has no guaranteed time
domain, so you don't know if your target is failing or just being slow
so you keep on taking screenshots for a ridiculously long time. You can
take fewer screenshots if you have some out-of-band signal. For example
knowing to expect a particular screenshot around the time that a
particular file is touched or a particular hardware device is
initialised, or a magic QEMU-specific CPU instruction is executed.
Post by Natalia Portillo
I have enough spare space to hold the boot images (600Gibibytes free), and a
machine that should be able to automatically checkout and compile QEMU
(Athlon XP 2000+, 768Mb RAM, 600GiB free, Gentoo Linux).
Compiling is really important. Even without testing targets, this would
be a very good community service if you want to do even just this much.
Compiling a simulator under itself in all supported configurations is an
important set of tests (compile/run MIPS under IA32, IA32 under PPC32,
etc especially 32/64 mixes, then the different supported compiler
versions, and then a couple of different Linux distributions with their
different choices of library versions and configurations.) This is
probably 24 hours of hard work. I have seen simulators choke on these
sorts of tests.
Post by Natalia Portillo
Just, I have not the knowledge to make the script that boots qemu, says qemu
to take the screenshot, compares the took one with the last one, stops qemu,
sends the last screenshot by email, compresses all took screenshots, goes to
next VM, so on. (Preferibly without X11 running, as the machine is a mostly
headless P2P and File server)
I think this can be simplified. Care to send a copy offlist of what
you've done and i'll have a look at it?

--
Dan Shearer
***@shearer.org
Anthony Liguori
2007-07-27 14:29:11 UTC
Permalink
FYI, I've started building a VNC based automated tester. You provide it
a series of screenshots with masks of data that's likely to be
semi-random and it'll wait for each screen shot. It also has the smarts
to find where the mouse is so that you can move it to an exact location.

I'll have something functional (along with a paper) within a month.

Regards,

Anthony Liguori
Post by Dan Shearer
Whoops, slightly late reply :-)
Post by Natalia Portillo
I have a huge list of operating systems (both closed and open source, that
works and that doesn't work under QEMU) that can be used to check that QEMU
doesn't broke (or even, that it corrects a non-working state).
Where the problem with QEMU is not something basic (eg doesn't boot) it
can be useful to use nested virtualisation. Although QEMU in QEMU is of
course slow because that case hasn't been optimised, if you have a
single image within which are many other images savevm'd just before
the error condition occurs, you can run a repeatable testsuite quite
quickly. And since you can snapshot the outer image, you know that you
are running these tests in exactly the same way every time. (There are a
lot of other uses for recursive virtualisation, I'm a fan of it!)
Post by Natalia Portillo
And I think it is better to check an installed image that to test it only
boots or installs (faster at least).
Agreed. As to installs I had to do something similar for different VM
systems a while ago and I found it useful to have a frozen image just
before the typical "detecting hardware" phase of an installation. The
point of this was not to compare the hardware that was detected, just to
detect major malfunctions. Hardware probing like this is uniquely
stressful. It isn't hard to notice if an installer crashes, or the VM
crashes, or there's lot's of errors.
Post by Natalia Portillo
Fabrice and me discussed about taking screenshot per some seconds to compare
with known took screenshots and if something fails shutdown the VM and send
an email.
But that required some macro interface "click at x,y, wait some seconds,
press 'k' key", that is not currently under QEMU.
Why can't you redirect the QEMU monitor to something easily accessible,
eg a virtual serial port, and then use mouse_move and sendkey monitor
commands to drive testing, and screendump to save images for comparison?
Those commands have been in the monitor for years, so is there something
I'm not understanding here?
Post by Natalia Portillo
The best solution I think is to get a way to send QEMU the screenshot to
file command some times and stop the VM when both are equal, then send the
last took screenshot to a mail address so breakability can be checked. (If
it is the login window, great, it BOOTS!, if not, it doesn't)
What do you think?
The problem is synchronisation given that QEMU has no guaranteed time
domain, so you don't know if your target is failing or just being slow
so you keep on taking screenshots for a ridiculously long time. You can
take fewer screenshots if you have some out-of-band signal. For example
knowing to expect a particular screenshot around the time that a
particular file is touched or a particular hardware device is
initialised, or a magic QEMU-specific CPU instruction is executed.
Post by Natalia Portillo
I have enough spare space to hold the boot images (600Gibibytes free), and a
machine that should be able to automatically checkout and compile QEMU
(Athlon XP 2000+, 768Mb RAM, 600GiB free, Gentoo Linux).
Compiling is really important. Even without testing targets, this would
be a very good community service if you want to do even just this much.
Compiling a simulator under itself in all supported configurations is an
important set of tests (compile/run MIPS under IA32, IA32 under PPC32,
etc especially 32/64 mixes, then the different supported compiler
versions, and then a couple of different Linux distributions with their
different choices of library versions and configurations.) This is
probably 24 hours of hard work. I have seen simulators choke on these
sorts of tests.
Post by Natalia Portillo
Just, I have not the knowledge to make the script that boots qemu, says qemu
to take the screenshot, compares the took one with the last one, stops qemu,
sends the last screenshot by email, compresses all took screenshots, goes to
next VM, so on. (Preferibly without X11 running, as the machine is a mostly
headless P2P and File server)
I think this can be simplified. Care to send a copy offlist of what
you've done and i'll have a look at it?
--
Dan Shearer
Dan Shearer
2007-07-27 14:34:04 UTC
Permalink
Post by Anthony Liguori
FYI, I've started building a VNC based automated tester. You provide it
a series of screenshots with masks of data that's likely to be
That's excellent.

I see this as a different class of testing to the tinderbox-style
testing Natalia Portillo was talking about, checking out and compiling
under different circumstances. That's very valuable, and will help
address a problem that drives away a lot of potential QEMU users - they
just can't compile it.
--
Dan Shearer
***@shearer.org
Sunil Amitkumar Janki
2007-07-27 14:58:19 UTC
Permalink
Post by Dan Shearer
Post by Anthony Liguori
FYI, I've started building a VNC based automated tester. You provide it
a series of screenshots with masks of data that's likely to be
That's excellent.
I see this as a different class of testing to the tinderbox-style
testing Natalia Portillo was talking about, checking out and compiling
under different circumstances. That's very valuable, and will help
address a problem that drives away a lot of potential QEMU users - they
just can't compile it.
That would be great. I am regularly compiling and using QEMU CVS
snapshots because features get added all the time. A lot of the time
it happens that these build fine on x86 whereas they don't work on
other architectures because the developers don't have access to those.

I have been running self-built Slackware 12.0 on MIPS for the last
few months and would like to run QEMU on that architecture. previously
it built but didn't run, but now it doesn't even compile anymore. The
Post by Dan Shearer
gcc -O2 -march=r4600 -Wall -O2 -g -fno-strict-aliasing -I. -I..
-I/root/tmp/qemu-20070727/target-i386 -I/root/tmp/qemu-20070727
-I/root/tmp/qemu-20070727/linux-user
-I/root/tmp/qemu-20070727/linux-user/i386 -D_GNU_SOURCE
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-I/root/tmp/qemu-20070727/fpu -DHAS_AUDIO -DHAS_AUDIO_CHOICE
-I/root/tmp/qemu-20070727/slirp -c -o gdbstub.o
/root/tmp/qemu-20070727/gdbstub.c
gcc -g -Wl,-T,/root/tmp/qemu-20070727/mipsel.ld -o qemu-i386 main.o
syscall.o mmap.o signal.o path.o osdep.o thunk.o elfload.o linuxload.o
vm86.o libqemu.a gdbstub.o -lm -lrt -lasound
/usr/bin/ld: cannot find -lm
collect2: ld returned 1 exit status
make[1]: *** [qemu-i386] Error 1
make[1]: Leaving directory `/root/tmp/qemu-20070727/i386-linux-user'
make: *** [subdir-i386-linux-user] Error 2
make -C i386-linux-user all
make[1]: Entering directory `/root/tmp/qemu-20070727/i386-linux-user'
gcc -g -Wl,-T,/root/tmp/qemu-20070727/mipsel.ld -o qemu-i386 main.o
syscall.o mmap.o signal.o path.o osdep.o thunk.o elfload.o linuxload.o
vm86.o libqemu.a gdbstub.o -lm -lrt -lasound
/usr/bin/ld: cannot find -lm
collect2: ld returned 1 exit status
make[1]: *** [qemu-i386] Error 1
make[1]: Leaving directory `/root/tmp/qemu-20070727/i386-linux-user'
make: *** [subdir-i386-linux-user] Error 2
I will try to get SPARC Linux and Solaris running on my newly acquired Sun
Enterprise 250 server and compile there as well so I can test snapshots on
all the architectures I have at my disposal.

Sunil
Dan Shearer
2007-07-27 15:12:11 UTC
Permalink
Post by Sunil Amitkumar Janki
Post by Dan Shearer
I see this as a different class of testing to the tinderbox-style
testing Natalia Portillo was talking about, checking out and compiling
under different circumstances. That's very valuable, and will help
address a problem that drives away a lot of potential QEMU users - they
just can't compile it.
That would be great. I am regularly compiling and using QEMU CVS
snapshots because features get added all the time. A lot of the time
it happens that these build fine on x86 whereas they don't work on
other architectures because the developers don't have access to those.
I have been running self-built Slackware 12.0 on MIPS for the last
few months and would like to run QEMU on that architecture. previously
it built but didn't run, but now it doesn't even compile anymore. The
This is one of the three reasons why I say it is better to do
tinerbox-type testing inside a VM. If you could make the precise QEMU
MIPS target image that fails to build QEMU available online then it is
easier for everyone to see the context. (The other two reasons are that
it is a good workout for the simulator, and general dogfood-ness for the
developers who are forced to become aware of problems that would
otherwise only be noticed by minority users such as you :-)
Post by Sunil Amitkumar Janki
I will try to get SPARC Linux and Solaris running on my newly acquired Sun
Enterprise 250 server and compile there as well so I can test snapshots on
all the architectures I have at my disposal.
You do also have the architectures provided to you by QEMU :-) :-)
--
Dan Shearer
***@shearer.org
Sunil Amitkumar Janki
2007-07-27 15:50:05 UTC
Permalink
Post by Dan Shearer
You do also have the architectures provided to you by QEMU :-) :-)
I have been able to build modular Xorg for Armedslack in QEMU without
having the hardware but when I tried to port Slackware 12.0 to SPARC in
QEMU I found it was way too slow and started looking for some real hardware.

Maybe in the future where we have all kinds of cheap multicore processors
this will not matter much but for the moment limited resources are still
some
kind of hindrance to realizing the benefits of multi-architecture emulation.

Maybe I should trade in my single-core Athlon system for some serious
quadcore x86_64 chip but I'd rather wait for 16-core Loongson 3 :-).
Dan Shearer
2007-07-27 16:04:50 UTC
Permalink
Post by Sunil Amitkumar Janki
Post by Dan Shearer
You do also have the architectures provided to you by QEMU :-) :-)
Maybe in the future where we have all kinds of cheap multicore processors
this will not matter much but for the moment limited resources are still
some
kind of hindrance to realizing the benefits of multi-architecture emulation.
In the special case of repeat automated test compiles where the only
output is a short status message from time to time, the lack of speed is
less of a problem. You do end up with a machine that creates the maximum
amount of noise the hardware is capable of, and runs very hot :-)
--
Dan Shearer
***@shearer.org
Jan Marten Simons
2007-07-27 16:50:51 UTC
Permalink
Post by Sunil Amitkumar Janki
Post by Dan Shearer
You do also have the architectures provided to you by QEMU :-) :-)
I have been able to build modular Xorg for Armedslack in QEMU without
having the hardware but when I tried to port Slackware 12.0 to SPARC in
QEMU I found it was way too slow and started looking for some real hardware.
Maybe in the future where we have all kinds of cheap multicore processors
this will not matter much but for the moment limited resources are still
some
kind of hindrance to realizing the benefits of multi-architecture emulation.
Maybe I should trade in my single-core Athlon system for some serious
quadcore x86_64 chip but I'd rather wait for 16-core Loongson 3 :-).
As Qemu cannot use multicore CPUs (partially due to missing thread
safety), yet, you won't benefit from this unless you want to run
multiple instances of Qemu in parallel.

regs,
Jan
Andreas Färber
2007-07-27 18:54:49 UTC
Permalink
Post by Sunil Amitkumar Janki
Post by Dan Shearer
I see this as a different class of testing
[...], checking out and compiling
under different circumstances. That's very valuable, and will help
address a problem that drives away a lot of potential QEMU users - they
just can't compile it.
That would be great. I am regularly compiling and using QEMU CVS
snapshots because features get added all the time. A lot of the time
it happens that these build fine on x86 whereas they don't work on
other architectures because the developers don't have access to those.
I have been running self-built Slackware 12.0 on MIPS for the last
few months and would like to run QEMU on that architecture. previously
it built but didn't run, but now it doesn't even compile anymore. The
error message I get is the following: [...]
I recently reported build failure of the ppc target on OS X due to an
unsupported GCC feature but no-one has shown any interest in fixing
it, neither applying my workaround patch nor proposing a different
solution.

It's a strange phenomenon that many other open source projects don't
suffer - qemu seems to expect everyone to patch it herself instead of
fixing problems in the repository at their origin. Major example
being qemu clinging on to GCC 3.x instead of supporting both 3 and 4
by updating and integrating the existing patches that worked for 0.9.0.

Andreas
Thiemo Seufer
2007-07-28 10:36:00 UTC
Permalink
Post by Andreas Färber
Post by Sunil Amitkumar Janki
Post by Dan Shearer
I see this as a different class of testing
[...], checking out and compiling
under different circumstances. That's very valuable, and will help
address a problem that drives away a lot of potential QEMU users - they
just can't compile it.
That would be great. I am regularly compiling and using QEMU CVS
snapshots because features get added all the time. A lot of the time
it happens that these build fine on x86 whereas they don't work on
other architectures because the developers don't have access to those.
I have been running self-built Slackware 12.0 on MIPS for the last
few months and would like to run QEMU on that architecture. previously
it built but didn't run, but now it doesn't even compile anymore. The
error message I get is the following: [...]
I recently reported build failure of the ppc target on OS X due to an
unsupported GCC feature but no-one has shown any interest in fixing it,
neither applying my workaround patch nor proposing a different solution.
It's a strange phenomenon that many other open source projects don't suffer
- qemu seems to expect everyone to patch it herself instead of fixing
problems in the repository at their origin. Major example being qemu
clinging on to GCC 3.x instead of supporting both 3 and 4 by updating and
integrating the existing patches that worked for 0.9.0.
The patches I remember did either break other compilers or were a
maintenance nightmare of postprocessed assembler output.


Thiemo
Andreas Färber
2007-07-29 15:31:21 UTC
Permalink
Post by Thiemo Seufer
Post by Andreas Färber
qemu seems to expect everyone to patch it herself instead of fixing
problems in the repository at their origin. Major example being qemu
clinging on to GCC 3.x instead of supporting both 3 and 4 by
updating and
integrating the existing patches that worked for 0.9.0.
The patches I remember did either break other compilers or were a
maintenance nightmare of postprocessed assembler output.
I didn't say the particular patch was perfect, I can't tell. All I
know is that there are platforms that don't have a GCC 3.x (the Intel
Macs for instance), and syncing the existing 0.9.0 patches with CVS
HEAD is like shooting for a running target - I didn't manage to
successfully update the patch, it crashed badly. Having some initial,
conditional support for GCC 4 (e.g. --enable-gcc4 or depending on $CC
version) would be good. Do you know where exactly the problem is
between the versions and what needs to be done to have it working
(again)?

Andreas

Thiemo Seufer
2007-07-27 18:51:54 UTC
Permalink
Sunil Amitkumar Janki wrote:
[snip]
Post by Dan Shearer
gcc -g -Wl,-T,/root/tmp/qemu-20070727/mipsel.ld -o qemu-i386 main.o
syscall.o mmap.o signal.o path.o osdep.o thunk.o elfload.o linuxload.o
vm86.o libqemu.a gdbstub.o -lm -lrt -lasound
/usr/bin/ld: cannot find -lm
collect2: ld returned 1 exit status
make[1]: *** [qemu-i386] Error 1
A missing libm sounds like a build environment problem.


Thiemo
Sunil Amitkumar Janki
2007-07-27 19:55:16 UTC
Permalink
Post by Thiemo Seufer
A missing libm sounds like a build environment problem.
Thiemo
Do you have a test for this because I reinstalled the entire
system from scratch and have had no problems building
lots of packages over the last week except for those
applications that assume that all the world is an x86?

The only thing I have had to do is getting a GCC 3.4.6
compiler setup which I did by configuring and packaging
it to "/opt/gcc/3.4.6" and including it in the path and
making it the default using a script in "/usr/bin" and
reversing that again.

I *do* have it configured with "--with-arch=r4600", which
may have led to problems. It's really painful for me to
go back to some hacked up previous os release, that was
not done by me, on which I can rebuild GCC 3.4.6 without
this option, but I will if I really have to.

It works well for several packages that don't compile with
GCC 4.1.2, which is the system compiler and seems to
be the accepted default nowadays in all distributions.

I guess the best solution would be to change QEMU to
make it build with GCC4 but I understand that may not
be an easy endeavour to undertake because of fundamental
design issues.

Sunil
Thiemo Seufer
2007-07-28 10:17:19 UTC
Permalink
Post by Sunil Amitkumar Janki
Post by Thiemo Seufer
A missing libm sounds like a build environment problem.
Thiemo
Do you have a test for this because I reinstalled the entire
system from scratch and have had no problems building
lots of packages over the last week except for those
applications that assume that all the world is an x86?
The only thing I have had to do is getting a GCC 3.4.6
compiler setup which I did by configuring and packaging
it to "/opt/gcc/3.4.6" and including it in the path and
making it the default using a script in "/usr/bin" and
reversing that again.
The problem is the linker (presumably /usr/bin/ld) does not
find the math library (likely name is /lib/libm.so.6). You can
add "-v" to the gcc command line to find out what it looks for.

JFTR, building qemu on Debian mips worked for me some days ago,
but the resulting binary was broken (as expected with current CVS).
Post by Sunil Amitkumar Janki
I *do* have it configured with "--with-arch=r4600", which
may have led to problems. It's really painful for me to
go back to some hacked up previous os release, that was
not done by me, on which I can rebuild GCC 3.4.6 without
this option, but I will if I really have to.
A plain upstream tarball and the classic
"./configure && make && make install"
should work for that.


Thiemo
Sunil Amitkumar Janki
2007-07-28 11:41:59 UTC
Permalink
Post by Thiemo Seufer
The problem is the linker (presumably /usr/bin/ld) does not
find the math library (likely name is /lib/libm.so.6). You can
add "-v" to the gcc command line to find out what it looks for.
JFTR, building qemu on Debian mips worked for me some days ago,
but the resulting binary was broken (as expected with current CVS).
I have rebuilt in a pure Slackware 12.0 chroot in which I have replaced the
GCC 4.1.2 system compiler that runs from /usr/bin with a previously
built GCC
3.4.6 that also runs from /usr/bin and I agree that it builds this way.
I will keep
building in this chroot then for the foreseeable future until QEMU is
ready for
GCC 4.

Do you have any idea of what else is needed to get QEMU to a usable state on
MIPS hosts? Are there any other ideas for virtualisation such as Xen or
VMware/
SimOS? I have asked on the Xen-devel list but nobody gave any feedback
there.
Stefan Weil
2007-07-28 12:43:16 UTC
Permalink
QEMU can build and run on a MIPS32 host - it just needs a few patches
(which were already sent to the list but got lost in CVS head).

The patched sources I use to run QEMU MIPS Malta with Debian stable
inside of a QEMU Malta emulation on x86 hardware are available from URL
http://svn.berlios.de/viewcvs/ar7-firmware/qemu/branches/head/

Use this command to get them:
svn checkout
http://svn.berlios.de/svnroot/repos/ar7-firmware/qemu/branches/head/

Any combination of big and little endian works with pcnet32:
qemu-system-mipsel in qemu-system-mips,
qemu-system-mipsel in qemu-system-mipsel,
qemu-system-mips in qemu-system-mips,
qemu-system-mips in qemu-system-mipsel.

Other network emulations only work with some of these combinations.

User mode emulation won't work, but it does not work on any
platform - not even x86.

I did not check MIPS64 emulation.

Stefan
Post by Sunil Amitkumar Janki
Do you have any idea of what else is needed to get QEMU to a usable state on
MIPS hosts? Are there any other ideas for virtualisation such as Xen
or VMware/
SimOS? I have asked on the Xen-devel list but nobody gave any feedback
there.
Jamie Lokier
2007-04-10 11:17:20 UTC
Permalink
Post by J. Mayer
Post by Thiemo Seufer
No. Since MIPS{32,64}R2 the CP0 is standardized and a mandatory part of
a MIPS compatible CPU.
Yes, I know MIPS want always CP0 to be present. I should have put a
smiley somewhere. I wanted just to point the fact that the CPU itself
does not need the CP0 controller to run and that one could easily
imagine designing a core without the CP0 controller.
I just want to say I have programmed a MIPS-ish core which did not
have a CP0 controller. It was the Alteon AceNIC gigabit ethernet
controller, and it had two CPU cores implementing the MIPS integer
instruction set. We used the standard mips-elf GCC to compile for it.

I'm under the impression that copying the basic MIPS instruction set
is not uncommon for the odd ASIC here and there, as it doesn't bump
into the licensing problems that copying ARM instructions does, and
it's quite simple.

-- Jamie
Paul Brook
2007-04-09 00:41:05 UTC
Permalink
[replying to a couple of different mails]
Post by J. Mayer
What do you need to route an IRQ ?
-> A peripheral destination
Agreed.
Post by J. Mayer
What we got now ?
-> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
state
We have this in some places. Other places only have some parts.
Post by J. Mayer
Is more needed to have a generic routing approach ?
-> no. This can work to route any signal
Agreed.
Post by J. Mayer
Can we do with less ?
-> no. You need the 3 parameters.
Agreed.


In summary the IRQ source (ie. device raising the IRQ) needs to keep track of
4 values:
1) Callback
2) Opaque callback argument
3) PIN number
4) IRQ state.

In most cases we get (4) for free because it's a direct function of device
state, so I'll ignore it for now.

I believe (1) and (2) are inherently linked, and it makes no sense to
set/change them individually.

In some cases code has multiple instances of (3) sharing a single (2). This
is, in general, incorrect as a device may have outputs connected to different
interrupt controllers. I think there are examples of this on the ARM boards.

Thus an IRQ source can treat (1), (2) and (3) as a single block of
information, with no loss of flexibility.
Post by J. Mayer
The problem is also: what does this patch adds, but complexity
I believe my patch concentrates the (necessary) complexity in a single place.
For the record, the net effect of my patch was to remove approximately 32
lines of code (71 files changed, 594 insertions(+), 626 deletions(-))
Post by J. Mayer
and arbitrary limitations ?
You have stated several times that my patch adds arbitrary limitations.
I refute this assertion absolutely.

There are no limits on the number of IRQs, or the topology of entities
(devices, interrupt handlers, and CPUs) that can be supported.

Hotplugging is not a problem, neither are systems with thousands of IRQs. I
have local patches for an arm-based core with several hundred IRQ lines.
Convention is that IRQ objects are created at the same time as the rest of
the device state for the IRQ sink (ie. interrupt controller or CPU).

In practice this means a particular device needs to know how many IRQ inputs
it has when it is instantiated. I believe this is entirely reasonable. Last
time I checked it wasn't feasible to dynamically solder new pins onto an IC
while it was running [*1]. Note that this is the number of IRQs*per instance
of a device. It's entirely possible to have different instances of the "same"
device with arbitrarily different numbers of IRQs, and an arbitrary number of
devices/IRQs in a system. [*2]

If you want to do anything other that simple 1-1 connections (eg. shared IRQ
lines) you can create a fake device to perform the appropriate multiplexing.
This is what the PCI code does. It creates a "PCI bus" interrupt controller
that maps individual device IRQ pins onto the host interface IRQs.
Post by J. Mayer
What can be done (but once again, it changes nothing, just hide the
"complexity"), is to replace the { callback, n_IRQ } in devices
structure by a IRQState structure and have the inline functions.
This is what I did, except I chose to make it an opaque structure, to prevent
devices for meddling with it directly. I'd be amazed if inlining qemu_set_irq
made any measurable difference to execution speed.

You seem to be saying that making this change has no benefit. I disagree quite
strongly.

Having each device keep track of 3 values (callback, opaque and nIRQ; see
earlier) is a real mess, evidenced by the fact that devices don't do this
consistently, the PCI code has grown its own slightly different mechanisms
for signalling IRQs, and the ARM boards had their own partially generic
implementation. Adding simple and consistent infrastructure for signalling
interrupts is IMHO a worthwhile change in its own right.
Post by J. Mayer
..
struct PINState {
qemu_pin_handler handler;
void *opaque;
int n;
int level;
};
Yes, and the existing code can be extended to implement this without wasting
any of the current changes.

What you're talking about (and in later emails with tristate pins) is a
generic mechanism for emulating single-bit buses. I don't claim that my
implementation can do this as-is.

I have implemented sufficient infrastructure for a single-master single-slave
bus. The most common example of which is a IRQ line. I believe it also covers
a usefully large subset GPIO pin uses.

I say that my changes are a necessary first step in implementing a fully
generic single-bit bus framework. My implementation adds infrastructure and
abstraction for the "master" device (IRQ source), while leaving the "slave"
(IRQ sink) device code largely unchanged.

Paul

[*1] I guess you could theoretically do this with a self-modifying FPGA SoC.
It's not impossible to model, just a bit hairy. Effectively an extreme case of
hotplugging.

[*2] Technically you're limited by available memory on the host. However
struct IRQState is very small, so you have other much larger problems before
you even come close to that limit.
J. Mayer
2007-04-09 11:11:55 UTC
Permalink
Post by Paul Brook
[replying to a couple of different mails]
Post by J. Mayer
What do you need to route an IRQ ?
-> A peripheral destination
Agreed.
Post by J. Mayer
What we got now ?
-> a callback with 3 parameters: an opaque, a PIN (the n_IRQ) and a
state
We have this in some places. Other places only have some parts.
So, here was the first point to fix. Imho, no need to change all the
code for this.
Post by Paul Brook
Post by J. Mayer
Is more needed to have a generic routing approach ?
-> no. This can work to route any signal
Agreed.
Post by J. Mayer
Can we do with less ?
-> no. You need the 3 parameters.
Agreed.
In summary the IRQ source (ie. device raising the IRQ) needs to keep track of
1) Callback
2) Opaque callback argument
3) PIN number
4) IRQ state.
In most cases we get (4) for free because it's a direct function of device
state, so I'll ignore it for now.
I don't agree. There are a lot of cases where the hardware state is not
equivalent to the device internal state. If an IRQ line is not driven,
then you cannot predict what the hardware state is, from the source. It
seems to me this is the most usual case for IRQs lines.
Post by Paul Brook
I believe (1) and (2) are inherently linked, and it makes no sense to
set/change them individually.
OK.
Post by Paul Brook
In some cases code has multiple instances of (3) sharing a single (2). This
is, in general, incorrect as a device may have outputs connected to different
interrupt controllers. I think there are examples of this on the ARM boards.
Thus an IRQ source can treat (1), (2) and (3) as a single block of
information, with no loss of flexibility.
Post by J. Mayer
The problem is also: what does this patch adds, but complexity
I believe my patch concentrates the (necessary) complexity in a single place.
For the record, the net effect of my patch was to remove approximately 32
lines of code (71 files changed, 594 insertions(+), 626 deletions(-))
Post by J. Mayer
and arbitrary limitations ?
You have stated several times that my patch adds arbitrary limitations.
I refute this assertion absolutely.
There are no limits on the number of IRQs, or the topology of entities
(devices, interrupt handlers, and CPUs) that can be supported.
OK, then I'm wrong on this point.
Post by Paul Brook
Hotplugging is not a problem, neither are systems with thousands of IRQs. I
have local patches for an arm-based core with several hundred IRQ lines.
Convention is that IRQ objects are created at the same time as the rest of
the device state for the IRQ sink (ie. interrupt controller or CPU).
In practice this means a particular device needs to know how many IRQ inputs
it has when it is instantiated. I believe this is entirely reasonable. Last
time I checked it wasn't feasible to dynamically solder new pins onto an IC
while it was running [*1].
Quite hard to achieve, I got to admit it.
Post by Paul Brook
Note that this is the number of IRQs*per instance
of a device. It's entirely possible to have different instances of the "same"
device with arbitrarily different numbers of IRQs, and an arbitrary number of
devices/IRQs in a system. [*2]
If you want to do anything other that simple 1-1 connections (eg. shared IRQ
lines) you can create a fake device to perform the appropriate multiplexing.
This is what the PCI code does. It creates a "PCI bus" interrupt controller
that maps individual device IRQ pins onto the host interface IRQs.
So, this does not change anything from the previous situation.
Post by Paul Brook
Post by J. Mayer
What can be done (but once again, it changes nothing, just hide the
"complexity"), is to replace the { callback, n_IRQ } in devices
structure by a IRQState structure and have the inline functions.
This is what I did, except I chose to make it an opaque structure, to prevent
devices for meddling with it directly. I'd be amazed if inlining qemu_set_irq
made any measurable difference to execution speed.
You seem to be saying that making this change has no benefit. I disagree quite
strongly.
Having each device keep track of 3 values (callback, opaque and nIRQ; see
earlier) is a real mess, evidenced by the fact that devices don't do this
consistently, the PCI code has grown its own slightly different mechanisms
for signalling IRQs, and the ARM boards had their own partially generic
implementation. Adding simple and consistent infrastructure for signalling
interrupts is IMHO a worthwhile change in its own right.
So, you wanted just to hide all the mess in an opaque ?
But I still cannot see what functionality this could add, how this could
make SMP feasible, for example (I do think there was no problem for this
before), ...
Post by Paul Brook
Post by J. Mayer
..
struct PINState {
qemu_pin_handler handler;
void *opaque;
int n;
int level;
};
Yes, and the existing code can be extended to implement this without wasting
any of the current changes.
What you're talking about (and in later emails with tristate pins) is a
generic mechanism for emulating single-bit buses. I don't claim that my
implementation can do this as-is.
I have implemented sufficient infrastructure for a single-master single-slave
bus. The most common example of which is a IRQ line. I believe it also covers
a usefully large subset GPIO pin uses.
I don't think so. For most hardware, you need to emulate the actual
hardware behavior if you want to emulate what happens on the GPIO pins.
And I maintain you need to read back the hardware state if you want to
know the logical level of an IO. Think about implementing IIC,
SPI, .... using GPIO, which is not efficient but quite a usual
situation. You absolutely need to handle multiple sources - multiple
destinations cases to achieve this. And knowing the state of an IO
source is not sufficient to know the state of the IO line.
Even for IRQ, I think it should be done: this would avoid the kind of
local mess we have to handle PCI shared IRQ (I just did look the PREP
PCI 1.2 patch to say this, it may be done in a better way for other PCI
controllers). And PCI IRQ is exactly a case where you cannot read back
the IRQ line state from the state of the IRQ source. And it's not rare
that an IRQ source has a register where you can read back the hardware
state of its IRQ line(s).
And you also have many cases you need to know the level of an IO at some
precise points but you do not want to track all the IO changes (the IO
line may be floating most of the time, for example), then not define any
callback but just read the actual IO value when you need to know it.
Post by Paul Brook
I say that my changes are a necessary first step in implementing a fully
generic single-bit bus framework. My implementation adds infrastructure and
abstraction for the "master" device (IRQ source), while leaving the "slave"
(IRQ sink) device code largely unchanged.
I will try to sanitize the openpic, as you did suggest. I have to admit
what I did is not so good as I initially though it was.
As I already told here, I have added a fake internal IRQ controller in
the PowerPC core just to use the same API (the SetIRQ callback / nIRQ /
level) as the rest of the code to easily plug other IRQ controllers to a
PowerPC core. This fake IRQ controller is not in the PowerPC
specification and does not have any counterpart in actual
implementations:
"While the specific definition of an external input exception is
implementation dependant, it would typically be caused by the activation
of an asynchronous signal that is part of the processing system".
The same definition is used for other exception sources (critical
interrupt, ...) not generated by the PowerPC core itself. One should
take care that the word "interrupt" in the PowerPC specification is not
related to hardware IRQ (IBM likes to use standard vocabulary with
different meanings), which is quite confusing: "an interrupt is the
action in which the processor saves its context and begins execution at
a pre-determined interrupt-handler address. Exception are the events
that will, if enabled, cause the processor to take an interrupt".
This fake interrupt controller is just an IRQ sink, not a source, so I
maintain the presence of IRQ structure in the PowerPC CPU structure is
not to be. In addition of using the API of an IRQ controller, it also
cover a problem in the CPU emulation which is Qemu does not handle
exception queues and/or has not any "clean" support for multiple
exception sources raising at the same time. As suggested by Fabrice,
instead of adding mess in the Qemu core code, as I needed to add more
asynchronous exception sources, I decided to hide this using a bitmask
in the CPU structure, which seemed to be cleaner to me.
Post by Paul Brook
[*1] I guess you could theoretically do this with a self-modifying FPGA SoC.
It's not impossible to model, just a bit hairy. Effectively an extreme case of
hotplugging.
Quite extreme, true.
Post by Paul Brook
[*2] Technically you're limited by available memory on the host. However
struct IRQState is very small, so you have other much larger problems before
you even come close to that limit.
;-)
--
J. Mayer <***@magic.fr>
Never organized
Paul Brook
2007-04-09 13:58:49 UTC
Permalink
Post by J. Mayer
Post by Paul Brook
In summary the IRQ source (ie. device raising the IRQ) needs to keep
1) Callback
2) Opaque callback argument
3) PIN number
4) IRQ state.
...
Post by Paul Brook
I believe (1) and (2) are inherently linked, and it makes no sense to
set/change them individually.
OK.
Post by Paul Brook
Having each device keep track of 3 values (callback, opaque and nIRQ; see
earlier) is a real mess, evidenced by the fact that devices don't do this
consistently, the PCI code has grown its own slightly different
mechanisms for signalling IRQs, and the ARM boards had their own
partially generic implementation. Adding simple and consistent
infrastructure for signalling interrupts is IMHO a worthwhile change in
its own right.
So, you wanted just to hide all the mess in an opaque ?
Yes. As you agreed above, there's no need for the device to know the details.
Post by J. Mayer
But I still cannot see what functionality this could add, how this could
make SMP feasible, for example (I do think there was no problem for this
before), ...
In its current implementation it doesn't provide any functional enhancements
over a {callpack, opaque, nIRQ} triplet. What it does do is isolate the
device (IRQ source) emulation from the implementation details. As you agreed
above, there's no need for the device to know these details.

It could be extended to do clever things like multiple sources, shared IRQ
lines and tristate pins, but I have not done this. For the current code it is
sufficient to model these cases as a virtual fixed-function IRQ controller.
Post by J. Mayer
Post by Paul Brook
I have implemented sufficient infrastructure for a single-master
single-slave bus. The most common example of which is a IRQ line. I
believe it also covers a usefully large subset GPIO pin uses.
I don't think so. For most hardware, you need to emulate the actual
hardware behavior if you want to emulate what happens on the GPIO pins.
And I maintain you need to read back the hardware state if you want to
know the logical level of an IO. Think about implementing IIC,
SPI,  .... using GPIO, which is not efficient but quite a usual
We obviously have a different idea of what a "usefully large subset of GPIO
pin uses" entails. Most of the hardware I'm familiar with has dedicated I2C
hardware, and uses GPIO pins for interrupts and simple signals (eg. card
present).

I agree that in some cases you may want to model eg. an I2C bus connected to a
tristate GPIO pin. I don't have a good solution worked out for that. However
I am fairly confident that the current infrastructure could be used as a base
for whatever we do come up with.
Post by J. Mayer
I have added a fake internal IRQ controller in
the PowerPC core just to use the same API (the SetIRQ callback / nIRQ /
level) as the rest of the code to easily plug other IRQ controllers to a
PowerPC core.
...
As suggested by Fabrice,
instead of adding mess in the Qemu core code, as I needed to add more
asynchronous exception sources, I decided to hide this using a bitmask
in the CPU structure, which seemed to be cleaner to me.
Right, This is that same as what MIPS does, and probably ARM will soon.

All I did was to change the "same API" to be abstract object instead of
messing about with pointless triplets of information. For PPC this conversion
is incomplete because I don't understand the details well enough to tell how
bogus the current code is. eg. the openpic emulation looks like it is capable
of raising several outputs, but only one is ever used.

Paul
Paul Brook
2007-04-09 16:57:52 UTC
Permalink
Post by Paul Brook
In its current implementation it doesn't provide any functional
enhancements over a {callpack, opaque, nIRQ} triplet. What it does do is
isolate the device (IRQ source) emulation from the implementation
details.
So, I was confused by some remarks saying that this would give new
features.
I probably meant functionality improvements to the devices that didn't
previously implement the full triplet. Sorry for any confusion.
Yes, the implementation is incomplete. I need to use more than one
outputs for embedded PowerPC emulation but the code is not finished.
I just started to rework all this code. If I want to follow your new
scheme, I have to add an emulation for the implementation dependant
hardware internal IRQ controller (fortunately, most usual PowerPC
implementations use the same scheme) and try to change the OpenPIC so it
can easily map its outputs pins to the input pins of the PowerPC, in a
machine dependant way.
That is what I would recommend doing.

Paul
J. Mayer
2007-04-09 14:56:04 UTC
Permalink
On Mon, 2007-04-09 at 14:58 +0100, Paul Brook wrote:
[...]
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
Having each device keep track of 3 values (callback, opaque and nIRQ; see
earlier) is a real mess, evidenced by the fact that devices don't do this
consistently, the PCI code has grown its own slightly different
mechanisms for signalling IRQs, and the ARM boards had their own
partially generic implementation. Adding simple and consistent
infrastructure for signalling interrupts is IMHO a worthwhile change in
its own right.
So, you wanted just to hide all the mess in an opaque ?
Yes. As you agreed above, there's no need for the device to know the details.
Post by J. Mayer
But I still cannot see what functionality this could add, how this could
make SMP feasible, for example (I do think there was no problem for this
before), ...
In its current implementation it doesn't provide any functional enhancements
over a {callpack, opaque, nIRQ} triplet. What it does do is isolate the
device (IRQ source) emulation from the implementation details. As you agreed
above, there's no need for the device to know these details.
So, I was confused by some remarks saying that this would give new
features.
Post by Paul Brook
It could be extended to do clever things like multiple sources, shared IRQ
lines and tristate pins, but I have not done this. For the current code it is
sufficient to model these cases as a virtual fixed-function IRQ controller.
Post by J. Mayer
Post by Paul Brook
I have implemented sufficient infrastructure for a single-master
single-slave bus. The most common example of which is a IRQ line. I
believe it also covers a usefully large subset GPIO pin uses.
I don't think so. For most hardware, you need to emulate the actual
hardware behavior if you want to emulate what happens on the GPIO pins.
And I maintain you need to read back the hardware state if you want to
know the logical level of an IO. Think about implementing IIC,
SPI, .... using GPIO, which is not efficient but quite a usual
We obviously have a different idea of what a "usefully large subset of GPIO
pin uses" entails. Most of the hardware I'm familiar with has dedicated I2C
hardware, and uses GPIO pins for interrupts and simple signals (eg. card
present).
I agree a lot of embedded micro-controllers implement hardware I2C but I
also know a lot that lacks support for some specialised busses, like
SPI, then force developpers to emulate those interfaces using GPIOs, or
share some IO pins for multiple functions.
Post by Paul Brook
I agree that in some cases you may want to model eg. an I2C bus connected to a
tristate GPIO pin. I don't have a good solution worked out for that. However
I am fairly confident that the current infrastructure could be used as a base
for whatever we do come up with.
Post by J. Mayer
I have added a fake internal IRQ controller in
the PowerPC core just to use the same API (the SetIRQ callback / nIRQ /
level) as the rest of the code to easily plug other IRQ controllers to a
PowerPC core.
...
As suggested by Fabrice,
instead of adding mess in the Qemu core code, as I needed to add more
asynchronous exception sources, I decided to hide this using a bitmask
in the CPU structure, which seemed to be cleaner to me.
Right, This is that same as what MIPS does, and probably ARM will soon.
All I did was to change the "same API" to be abstract object instead of
messing about with pointless triplets of information. For PPC this conversion
is incomplete because I don't understand the details well enough to tell how
bogus the current code is. eg. the openpic emulation looks like it is capable
of raising several outputs, but only one is ever used.
Yes, the implementation is incomplete. I need to use more than one
outputs for embedded PowerPC emulation but the code is not finished.
I just started to rework all this code. If I want to follow your new
scheme, I have to add an emulation for the implementation dependant
hardware internal IRQ controller (fortunately, most usual PowerPC
implementations use the same scheme) and try to change the OpenPIC so it
can easily map its outputs pins to the input pins of the PowerPC, in a
machine dependant way.
--
J. Mayer <***@magic.fr>
Never organized
Fabrice Bellard
2007-04-07 23:26:03 UTC
Permalink
Hi,
Post by Paul Brook
I can only fix the code that I have access to.
IMHO your request that I revert changes because they allegedly break code that
noone else has access to is unreasonable. My changes are a significant
cleanup to qemu internals.
However if there is agreement from other maintainers I will revert my patch.
I don't think you need to revert your patch. Personally, I would have
changed the following:

1) The code is more generic than IRQs, it can be applied to any logic
signal, so the naming should show it.

2) I don't like hiding pointers in types, so I would use for example
'QEMUSignal *' instead of 'qemu_irq'.

Regarding the PPC problem, maybe you could just revert the patches in
the MIPS and PowerPC CPUs structures, as I don't think it is necessary
to move the IRQ code at this level now.

BTW, I have a small PREP test image that I can publish.

I am planning since some time to add to the QEMU web site an automatic
regression testing system which will automatically launch many OSes with
the current CVS version. I had no time to do it recently, but if no one
does it before I'll try to complete it !

Regards,

Fabrice.
Wang Cheng Yeh
2007-04-08 13:06:42 UTC
Permalink
Post by Fabrice Bellard
BTW, I have a small PREP test image that I can publish.
I am planning since some time to add to the QEMU web site an automatic
regression testing system which will automatically launch many OSes with
the current CVS version. I had no time to do it recently, but if no one
does it before I'll try to complete it !
Regards,
Fabrice.
I think the installation will cover more than just kernel booting.
I have debian auto installation files for regression test.
They are just a auto config file and a script to do regression test.
The only problem is if the system hangs, the script will also hang.
If anyone needs these files, I will release here.
The installation just follows the URL
http://www.aurel32.net/info/debian_arm_qemu.php
Thiemo Seufer
2007-04-08 13:56:29 UTC
Permalink
Post by Wang Cheng Yeh
Post by Fabrice Bellard
BTW, I have a small PREP test image that I can publish.
I am planning since some time to add to the QEMU web site an automatic
regression testing system which will automatically launch many OSes with
the current CVS version. I had no time to do it recently, but if no one
does it before I'll try to complete it !
Regards,
Fabrice.
I think the installation will cover more than just kernel booting.
I have debian auto installation files for regression test.
They are just a auto config file and a script to do regression test.
The only problem is if the system hangs, the script will also hang.
If anyone needs these files, I will release here.
The installation just follows the URL
http://www.aurel32.net/info/debian_arm_qemu.php
Maybe the debian-installer regression testing is helpful as well. It uses
qemu for some of its configs.

http://people.debian.org/~joeyh/d-i/test-logs.html is the results page,
http://d-i.alioth.debian.org/svn/debian-installer/scripts/digress/ points
to the source of the test harness.


Thiemo
Paul Brook
2007-04-08 22:45:27 UTC
Permalink
Post by Fabrice Bellard
2) I don't like hiding pointers in types, so I would use for example
'QEMUSignal *' instead of 'qemu_irq'.
FWIW I did it this way so that devices are agnostic about whether it was
implemented as a pointer or an integer index.

Paul
Thiemo Seufer
2007-04-07 21:20:47 UTC
Permalink
Post by J. Mayer
Post by Paul Brook
Post by J. Mayer
Post by Paul Brook
CVSROOT: /sources/qemu
Module name: qemu
Changes by: Paul Brook <pbrook> 07/04/07 18:14:41
The patches in the PowerPC target seem complete nonsense.
Can you give specific examples?
I'm talking about the CPU code.
There is NO notion of external IRQ allocation in the PowerPC
specification.
IRQ are completely out of the scope of the CPU emulation so the table
of 32 void *IRQ pointers is the CPU structure is a complete nonsense.
Where do you see in the PowerPC specification that those CPU have any
notion of how the EXTERNAL IRQ controler works ? Where do you see that a
machine with a PowerPC cannot manage more than 32 IRQ ? Where do you see
ANY NOTION OF EXTERNAL IRQ MANAGEMENT in the PowerPC specification ?
EXTERNAL IRQ MANAGEMENT IS NO WAY RELATED WITH CPU ! It's private to
each IRQ controller.
Saying anything else is just completely ignoring how real hardware
works.
SO your patch is a complete nonsense and YES IT BREAKS MY WORKS SO IT
HAS TO BE REVERTED.
So you are saying the old code is similiarily flawed? Why would reverting
to the old version be an improvement then?
Post by J. Mayer
If you don't, I'LL REVERT ALL POWERPC CODE AFFECTED BY THIS PATCH.
I assume you are close to have working code which solves the aforementioned
problems. In that case it might make most sense to stick with the current
version for the moment and replace it with yours as soon as it works.
Post by J. Mayer
Post by Paul Brook
The CHRP code looks a bit broken, but no more so than before I started.
Post by J. Mayer
Furthermore, this kind of patch that break other guys work would likely
to be discussed and not beeing imposed.
It's been mentioned several times on this list (by Fabrice, specifically) that
this is the way to go.
I did not received any single mail AT ALL saying "we're going to break
your code, you can now throw away all the work you're doing". NOT ONE.
So don't say "it's been discussed several times".
It is a bit hard to figure that out when the existence of such code isn't
known. Btw, I fail to see why it amounts to "throw away all work".
Presumably you could "revert" the offending bits in you local copy and
merge the remaining interface changes.


Thiemo
n schembr
2007-07-27 17:15:55 UTC
Permalink
----- Original Message ----
From: Jan Marten Simons <***@xtal.rwth-aachen.de>
To: qemu-***@nongnu.org
Sent: Friday, July 27, 2007 12:50:51 PM
Subject: Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)
Post by Sunil Amitkumar Janki
Post by Dan Shearer
You do also have the architectures provided to you by QEMU :-) :-)
I have been able to build modular Xorg for Armedslack in QEMU without
having the hardware but when I tried to port Slackware 12.0 to SPARC in
QEMU I found it was way too slow and started looking for some real
hardware.
Maybe in the future where we have all kinds of cheap multicore processors
this will not matter much but for the moment limited resources are still
some
kind of hindrance to realizing the benefits of multi-architecture
emulation.
Maybe I should trade in my single-core Athlon system for some serious
quadcore x86_64 chip but I'd rather wait for 16-core Loongson 3 :-).
As Qemu cannot use multicore CPUs (partially due to missing thread
safety), yet, you won't benefit from this unless you want to run
multiple instances of Qemu in parallel.
regs,
Jan
how close is thread safey?
Paul Brook
2007-07-27 17:38:19 UTC
Permalink
Post by n schembr
Post by Jan Marten Simons
As Qemu cannot use multicore CPUs (partially due to missing thread
safety), yet, you won't benefit from this unless you want to run
multiple instances of Qemu in parallel.
how close is thread safey?
In a useful form: a fair way off.

It's relatively simple to hack something together than runs. Making it work
correctly and go fast is much harder though. My current prototype (running on
2 cores) runs about a quarter the speed of normal qemu, and dies shortly
after booting because the guest atomic synchronisation primitives don't work
right.

Either way, for a tinderbox or automated testing yo *do* have lots of
different tests you can run in parallel, so it doesn't matter that each qemu
instance only uses one core. We're only talking about small SMP here - maybe
16 cores, but not thousands.

Paul
Anthony Liguori
2007-07-27 17:55:08 UTC
Permalink
Post by Paul Brook
Post by n schembr
Post by Jan Marten Simons
As Qemu cannot use multicore CPUs (partially due to missing thread
safety), yet, you won't benefit from this unless you want to run
multiple instances of Qemu in parallel.
how close is thread safey?
In a useful form: a fair way off.
It's relatively simple to hack something together than runs. Making it work
correctly and go fast is much harder though. My current prototype (running on
2 cores) runs about a quarter the speed of normal qemu, and dies shortly
after booting because the guest atomic synchronisation primitives don't work
right.
This later problem seems like the hardest to solve to me. Did you have
any ideas here that don't involve hand coding the translation for atomic
instructions?

Regards,

Anthony Liguori
Post by Paul Brook
Either way, for a tinderbox or automated testing yo *do* have lots of
different tests you can run in parallel, so it doesn't matter that each qemu
instance only uses one core. We're only talking about small SMP here - maybe
16 cores, but not thousands.
Paul
Paul Brook
2007-07-27 22:03:15 UTC
Permalink
Post by Anthony Liguori
Post by Paul Brook
Post by n schembr
how close is thread safey?
In a useful form: a fair way off.
It's relatively simple to hack something together than runs. Making it
work correctly and go fast is much harder though. My current prototype
(running on 2 cores) runs about a quarter the speed of normal qemu, and
dies shortly after booting because the guest atomic synchronisation
primitives don't work right.
This later problem seems like the hardest to solve to me. Did you have
any ideas here that don't involve hand coding the translation for atomic
instructions?
Yes. I do exclusive access locking at the TLB level. i.e. creating a TLB entry
for a writable page forces that page to be flushed from all the other CPU
TLBs.

There's some wiggle room in the definition of a "writable" page. If other
critera are met it should be sufficient to just do this exclusion for atomic
accesses.

If necessary the same technique can be used to avoid write ordering and
coherency problems without having to accurately map guest barriers onto the
equivalent host operations. This is handy when most of the guest barriers are
implicit, e.g. when emulating a strictly ordered guest on a weakly ordered
host.

In theory this could be taken to extremes and used to split emulation of a
single machine over multiple address spaces/nodes. In practice the
contention on a normal SMP operating system is high enough that this is not
practical.

Paul
n schembr
2007-07-27 22:59:15 UTC
Permalink
----- Original Message ----
From: Paul Brook <***@codesourcery.com>
To: qemu-***@nongnu.org
Cc: n schembr <***@yahoo.com>
Sent: Friday, July 27, 2007 1:38:19 PM
Subject: Re: QEMU Automated Testing (was [Qemu-devel] qemu Makefile.target vl.h hw/acpi.c hw/adlib.c ...)
Post by n schembr
Post by Jan Marten Simons
As Qemu cannot use multicore CPUs (partially due to missing thread
safety), yet, you won't benefit from this unless you want to run
multiple instances of Qemu in parallel.
how close is thread safey?
In a useful form: a fair way off.

It's relatively simple to hack something together than runs. Making it work
correctly and go fast is much harder though. My current prototype (running on
2 cores) runs about a quarter the speed of normal qemu, and dies shortly
after booting because the guest atomic synchronisation primitives don't work
right.

Either way, for a tinderbox or automated testing yo *do* have lots of
different tests you can run in parallel, so it doesn't matter that each qemu
instance only uses one core. We're only talking about small SMP here - maybe
16 cores, but not thousands.

Paul
-------------------------------------------------------------

Is the the issue I had with qemu moving from cpu to cpu? taskset -c 1 qemu or notsc

http://lists.gnu.org/archive/html/qemu-devel/2007-03/msg00652.html http://qemu-forum.ipi.fi/viewtopic.php?t=3714&highlight=taskset



What about preforking the processes one per virtual cpu?
Loading...