Hi Jeremy,
please have a look at Linpack and Stream results
http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux...
and
http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux...
We have collected /proc/sched_debug every 10 seconds - there are ""*scheddebug" files for every run. We are also collecting mpstat statistics every 2 seconds.
For example during a test when 64 stream jobs were running in parallel there are corresponding /proc/sched_debug logs:
4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug
4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug
However, we are not sure how to process these log files. Is there any tool to do so or could you share some hints what to look for?
Thanks a lot! Jirka
On Tue, May 3, 2016 at 4:18 PM, Jeremy Eder jeder@redhat.com wrote:
Yikes. 20% off...why? Do you have any system stats? Particularly /proc/sched_debug collected at some interval.
On Mon, May 2, 2016 at 12:39 PM, Jirka Hladky jhladky@redhat.com wrote:
Hi all,
we have compared scheduler performance for kernel-4.6.0-0.rc5.git0.1.wastedcores against kernel-4.6.0-0.rc5.git0.1.el7.
*Summary: *against all expectations* kernel-4.6.0-0.rc5.git0.1.wastedcores performs on average 20% WORSE than the regular kernel. *
Results for different servers:
SPECjbb2005:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
SPECjvm2008:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
Linpack and Stream:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
Any comments, questions, suggestions?
Jirka and Kamil
On Thu, Apr 28, 2016 at 12:25 AM, Jirka Hladky jhladky@redhat.com wrote:
Hi Jeremy,
I have just read
http://www.i3s.unice.fr/~jplozi/wastedcores/files/extended_talk.pdf http://www.ece.ubc.ca/~sasha/papers/eurosys16-final29.pdf
We will definitely test it and share the results here.
Jirka
--
-- Jeremy Eder
Hi Jeremy,
I think the reason why we observe the performance degradation after applying wastedcores patches is this issue:
https://github.com/jplozi/wastedcores/issues/6
With HT turned OFF we see that wastedcores kernel compares as following to mainline kernel
- performance regression in order of 1-5% for SPECjbb2005 and for Linpack and Stream - SPECjvm shows some minor improvements for some subtests but also some regressions for other subtests
I think we will need to test the four proposed "wastedcores" patches one by one.
Could you please prepare 4 different kernels where always just one patch from
https://github.com/Freeaqingme/wastedcores
will be applied?
Jirka
On Tue, May 3, 2016 at 11:43 PM, Jirka Hladky jhladky@redhat.com wrote:
Hi Jeremy,
please have a look at Linpack and Stream results
http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux...
and
http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux...
We have collected /proc/sched_debug every 10 seconds - there are ""*scheddebug" files for every run. We are also collecting mpstat statistics every 2 seconds.
For example during a test when 64 stream jobs were running in parallel there are corresponding /proc/sched_debug logs:
4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug
4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug
However, we are not sure how to process these log files. Is there any tool to do so or could you share some hints what to look for?
Thanks a lot! Jirka
On Tue, May 3, 2016 at 4:18 PM, Jeremy Eder jeder@redhat.com wrote:
Yikes. 20% off...why? Do you have any system stats? Particularly /proc/sched_debug collected at some interval.
On Mon, May 2, 2016 at 12:39 PM, Jirka Hladky jhladky@redhat.com wrote:
Hi all,
we have compared scheduler performance for kernel-4.6.0-0.rc5.git0.1.wastedcores against kernel-4.6.0-0.rc5.git0.1.el7.
*Summary: *against all expectations* kernel-4.6.0-0.rc5.git0.1.wastedcores performs on average 20% WORSE than the regular kernel. *
Results for different servers:
SPECjbb2005:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
SPECjvm2008:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
Linpack and Stream:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
Any comments, questions, suggestions?
Jirka and Kamil
On Thu, Apr 28, 2016 at 12:25 AM, Jirka Hladky jhladky@redhat.com wrote:
Hi Jeremy,
I have just read
http://www.i3s.unice.fr/~jplozi/wastedcores/files/extended_talk.pdf http://www.ece.ubc.ca/~sasha/papers/eurosys16-final29.pdf
We will definitely test it and share the results here.
Jirka
--
-- Jeremy Eder
Hi all,
do you have tools mentioned in articles about wastecores to visualize the idle and overloaded cores, producing images like this one?
https://adriancolyer.files.wordpress.com/2016/04/wasted-cores-fig-3.png
I have searched for it on GITHUB
https://github.com/jplozi/wastedcores/tree/master/tools
but it says just coming soon (update old already 3 months:-( )
Jirka
On Wed, May 4, 2016 at 3:55 AM, Jirka Hladky jhladky@redhat.com wrote:
Hi Jeremy,
I think the reason why we observe the performance degradation after applying wastedcores patches is this issue:
https://github.com/jplozi/wastedcores/issues/6
With HT turned OFF we see that wastedcores kernel compares as following to mainline kernel
- performance regression in order of 1-5% for SPECjbb2005 and for
Linpack and Stream
- SPECjvm shows some minor improvements for some subtests but also
some regressions for other subtests
I think we will need to test the four proposed "wastedcores" patches one by one.
Could you please prepare 4 different kernels where always just one patch from
https://github.com/Freeaqingme/wastedcores
will be applied?
Jirka
On Tue, May 3, 2016 at 11:43 PM, Jirka Hladky jhladky@redhat.com wrote:
Hi Jeremy,
please have a look at Linpack and Stream results
http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux...
and
http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux...
We have collected /proc/sched_debug every 10 seconds - there are ""*scheddebug" files for every run. We are also collecting mpstat statistics every 2 seconds.
For example during a test when 64 stream jobs were running in parallel there are corresponding /proc/sched_debug logs:
4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.el7.x86_64/throughput-performance/Results-2016-Apr-28_14h01m32s/RawData/stream.affinityRun.064streams.loop01.scheddebug
4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug http://perf-desktop.brq.redhat.com/LinpackAndStream/Red-Hat-Enterprise-Linux-Server-release-7.2/maja.lab.eng.brq.redhat.com-htON/4.6.0-0.rc5.git0.1.wastedcores.el7.x86_64/throughput-performance/Results-2016-Apr-29_15h37m17s/RawData/stream.affinityRun.064streams.loop01.scheddebug
However, we are not sure how to process these log files. Is there any tool to do so or could you share some hints what to look for?
Thanks a lot! Jirka
On Tue, May 3, 2016 at 4:18 PM, Jeremy Eder jeder@redhat.com wrote:
Yikes. 20% off...why? Do you have any system stats? Particularly /proc/sched_debug collected at some interval.
On Mon, May 2, 2016 at 12:39 PM, Jirka Hladky jhladky@redhat.com wrote:
Hi all,
we have compared scheduler performance for kernel-4.6.0-0.rc5.git0.1.wastedcores against kernel-4.6.0-0.rc5.git0.1.el7.
*Summary: *against all expectations* kernel-4.6.0-0.rc5.git0.1.wastedcores performs on average 20% WORSE than the regular kernel. *
Results for different servers:
SPECjbb2005:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
SPECjvm2008:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
Linpack and Stream:
- karkulka-02.lab.eng.brq.redhat.com-htON
- kiff-01.cluster-qe.lab.eng.brq.redhat.com-htON
Any comments, questions, suggestions?
Jirka and Kamil
On Thu, Apr 28, 2016 at 12:25 AM, Jirka Hladky jhladky@redhat.com wrote:
Hi Jeremy,
I have just read
http://www.i3s.unice.fr/~jplozi/wastedcores/files/extended_talk.pdf http://www.ece.ubc.ca/~sasha/papers/eurosys16-final29.pdf
We will definitely test it and share the results here.
Jirka
--
-- Jeremy Eder
kernel@lists.fedoraproject.org