Eric Dumazet
2011-04-07 04:58:47 UTC
Le jeudi 07 avril 2011 =C3=A0 10:16 +0800, Wei Gu a =C3=A9crit :
> Hi Eric,
> Testing with ixgbe Linux 2.6.38 driver:
> We have a little better thruput figure with this driver, but it looks
> not scalling at all, I always stressed one CPU core/24.
> And when look the perf report for ksoftirqd/24, the most cost functio=
n
> is still "_raw_spin_unlock_irqstore" and the IRQ/s is huge, it's
> somehow conflicts with desgin of NAPI. On linux 2.6.32 while the CPU
> was stressed the IRQ will descreased while the NAPI will running much
> on the polling mode. I don't know why on 2.6.38 the IRQ was keep
> increasing.
CC netdev and Intel guys, since they said it should not happen (TM)
IF you dont use DCA (make sure ioatdma module is not loaded), how comes
alloc_iova() is called at all ?
IF you use DCA, how comes its called, since the same CPU serves a given
interrupt ?
> =20
> CONFIG_TICK_ONESHOT=3Dy
> CONFIG_NO_HZ=3Dy
> =20
> PerfTop: 512417 irqs/sec kernel:91.3% exact: 0.0% [1000Hz cpu-clo=
ck-msecs], (all, 64 CPUs)
> ---------------------------------------------------------------------=
-----------------------------------------------------------------------=
----------
> - 0.82% ksoftirqd/24 [kernel.kallsyms] [k] _raw_sp=
in_unlock_irqrestore =
=
=20
> \u2592 - _raw_spin_unlock_irqrestore =
=
=20
> \u2592 - 44.27% alloc_iova =
=20
> \u2592 intel_alloc_iova =
=
=
=20
> \u2592 __intel_map_single =
=20
> \u2592 intel_map_page =
=
=20
> \u2592 - ixgbe_init_interrupt_scheme =
=
=20
> \u2592 - 59.97% ixgbe_alloc_rx_buffers =
=
=20
> \u2592 ixgbe_clean_rx_irq =
=
=20
> \u2592 0xffffffffa033a5 =
=20
> \u2592 net_rx_action =
=
=20
> u2592 __do_softirq =
=20
> \u2592 + call_softirq =
=
=20
> \u2592 - 40.03% ixgbe_change_mtu =
=
=
=20
> \u2592 ixgbe_change_mtu =
=20
> \u2592 dev_hard_start_xmit =
=20
> \u2592 sch_direct_xmit =
=20
> \u2592 dev_queue_xmit =
=20
> \u2592 vlan_dev_hard_start_xmit =
=
=
=20
> \u2592 hook_func =
=
=
=20
> \u2592 nf_iterate =
=
=
=20
> \u2592 nf_hook_slow =
=
=
=20
> \u2592 NF_HOOK.clone.1 =
=
=
=20
> \u2592 ip_rcv =
=
=
=20
> \u2592 __netif_receive_skb =
=
=
=20
> \u2592 __netif_receive_skb =
=
=
=20
> \u2592 netif_receive_skb =
=
=
=20
> \u2592 napi_skb_finish =
=
=
=20
> \u2592 napi_gro_receive =
=
=
=20
> \u2592 ixgbe_clean_rx_irq =
=
=
=20
> \u2592 0xffffffffa033a5 =
=
=
=20
> \u2592 net_rx_action =
=
=
=20
> \u2592 __do_softirq =
=
=
=20
> \u2592 + call_softirq =
=
=
=20
> \u2592 + 35.85% find_iova =
=
=
=20
> \u2592 + 19.44% add_unmap =20
> =20
> =20
> Thanks
> WeiGu
> =20
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
> Hi Eric,
> Testing with ixgbe Linux 2.6.38 driver:
> We have a little better thruput figure with this driver, but it looks
> not scalling at all, I always stressed one CPU core/24.
> And when look the perf report for ksoftirqd/24, the most cost functio=
n
> is still "_raw_spin_unlock_irqstore" and the IRQ/s is huge, it's
> somehow conflicts with desgin of NAPI. On linux 2.6.32 while the CPU
> was stressed the IRQ will descreased while the NAPI will running much
> on the polling mode. I don't know why on 2.6.38 the IRQ was keep
> increasing.
CC netdev and Intel guys, since they said it should not happen (TM)
IF you dont use DCA (make sure ioatdma module is not loaded), how comes
alloc_iova() is called at all ?
IF you use DCA, how comes its called, since the same CPU serves a given
interrupt ?
> =20
> CONFIG_TICK_ONESHOT=3Dy
> CONFIG_NO_HZ=3Dy
> =20
> PerfTop: 512417 irqs/sec kernel:91.3% exact: 0.0% [1000Hz cpu-clo=
ck-msecs], (all, 64 CPUs)
> ---------------------------------------------------------------------=
-----------------------------------------------------------------------=
----------
> - 0.82% ksoftirqd/24 [kernel.kallsyms] [k] _raw_sp=
in_unlock_irqrestore =
=
=20
> \u2592 - _raw_spin_unlock_irqrestore =
=
=20
> \u2592 - 44.27% alloc_iova =
=20
> \u2592 intel_alloc_iova =
=
=
=20
> \u2592 __intel_map_single =
=20
> \u2592 intel_map_page =
=
=20
> \u2592 - ixgbe_init_interrupt_scheme =
=
=20
> \u2592 - 59.97% ixgbe_alloc_rx_buffers =
=
=20
> \u2592 ixgbe_clean_rx_irq =
=
=20
> \u2592 0xffffffffa033a5 =
=20
> \u2592 net_rx_action =
=
=20
> u2592 __do_softirq =
=20
> \u2592 + call_softirq =
=
=20
> \u2592 - 40.03% ixgbe_change_mtu =
=
=
=20
> \u2592 ixgbe_change_mtu =
=20
> \u2592 dev_hard_start_xmit =
=20
> \u2592 sch_direct_xmit =
=20
> \u2592 dev_queue_xmit =
=20
> \u2592 vlan_dev_hard_start_xmit =
=
=
=20
> \u2592 hook_func =
=
=
=20
> \u2592 nf_iterate =
=
=
=20
> \u2592 nf_hook_slow =
=
=
=20
> \u2592 NF_HOOK.clone.1 =
=
=
=20
> \u2592 ip_rcv =
=
=
=20
> \u2592 __netif_receive_skb =
=
=
=20
> \u2592 __netif_receive_skb =
=
=
=20
> \u2592 netif_receive_skb =
=
=
=20
> \u2592 napi_skb_finish =
=
=
=20
> \u2592 napi_gro_receive =
=
=
=20
> \u2592 ixgbe_clean_rx_irq =
=
=
=20
> \u2592 0xffffffffa033a5 =
=
=
=20
> \u2592 net_rx_action =
=
=
=20
> \u2592 __do_softirq =
=
=
=20
> \u2592 + call_softirq =
=
=
=20
> \u2592 + 35.85% find_iova =
=
=
=20
> \u2592 + 19.44% add_unmap =20
> =20
> =20
> Thanks
> WeiGu
> =20
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html