RSB Performance Test API: Issueshttps://code.cor-lab.de/https://code.cor-lab.de/favicon.ico?14019720732016-07-21T10:42:07ZOpen Source Collaboration Platform
Redmine Bug #2619 (Resolved): Last events missed during recordinghttps://code.cor-lab.de/issues/26192016-07-21T10:42:07ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>It might happen that the recording actually doesn't contain the last phase end message because the recording process is stopped too early.</p> Bug #2618 (Resolved): Exception if the last phase end message was not receivedhttps://code.cor-lab.de/issues/26182016-07-21T10:36:57ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<pre>
Traceback (most recent call last):
File "/vol/perfmon/0.3/bin/perfmoneval", line 9, in <module>
load_entry_point('perfmonvalidation==0.3', 'console_scripts', 'perfmoneval')()
File "build/bdist.linux-x86_64/egg/perfmonvalidation/__init__.py", line 290, in main
File "build/bdist.linux-x86_64/egg/perfmonvalidation/__init__.py", line 298, in main_dump
File "build/bdist.linux-x86_64/egg/perfmonvalidation/bag.py", line 281, in load_test_cases
File "build/bdist.linux-x86_64/egg/perfmonvalidation/bag.py", line 188, in finish_test_case
File "/vol/perfmon/0.3/local/lib/python2.7/site-packages/pandas/core/frame.py", line 224, in __init__
mgr = self._init_dict(data, index, columns, dtype=dtype)
File "/vol/perfmon/0.3/local/lib/python2.7/site-packages/pandas/core/frame.py", line 360, in _init_dict
return _arrays_to_mgr(arrays, data_names, index, columns, dtype=dtype)
File "/vol/perfmon/0.3/local/lib/python2.7/site-packages/pandas/core/frame.py", line 5241, in _arrays_to_mgr
return create_block_manager_from_arrays(arrays, arr_names, axes)
File "/vol/perfmon/0.3/local/lib/python2.7/site-packages/pandas/core/internals.py", line 4004, in create_block_manager_from_arrays
construction_error(len(arrays), arrays[0].shape, axes, e)
File "/vol/perfmon/0.3/local/lib/python2.7/site-packages/pandas/core/internals.py", line 3970, in construction_error
passed, implied))
ValueError: Shape of passed values is (3, 17), indices imply (3, 18)
</pre>
<p>Be resilient against this or at least display a real error message.</p> Enhancement #2599 (Resolved): Better colors and legend for debug plotshttps://code.cor-lab.de/issues/25992016-07-12T12:39:21ZJ. Wienkejwienke@techfak.uni-bielefeld.deEnhancement #2598 (Resolved): Remove date-based indexing of debug plotshttps://code.cor-lab.de/issues/25982016-07-12T12:39:06ZJ. Wienkejwienke@techfak.uni-bielefeld.deFeature #2597 (Resolved): Always print specification codeshttps://code.cor-lab.de/issues/25972016-07-11T14:58:49ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>This will help debugging version change issues.</p> Bug #2595 (Rejected): Warn if current validation argument is not the most recent onehttps://code.cor-lab.de/issues/25952016-07-11T14:07:32ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>In the validate call do this.</p> Feature #2589 (Resolved): Exponential Parameter rangeshttps://code.cor-lab.de/issues/25892016-07-11T08:35:02ZJ. Moringenjmoringe@cor-lab.uni-bielefeld.de
<p><code>[ x**y for y in some-range ]</code></p> Bug #2542 (Resolved): Make test phases and their actions uniquely identifiablehttps://code.cor-lab.de/issues/25422016-05-09T13:53:31ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>In order to associate test results from previous system and test versions, changes in the plan need to be trackable and detectable.</p> Tasks #2540 (Resolved): Send test status via RSBhttps://code.cor-lab.de/issues/25402016-05-06T11:21:20ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>To have information about which test phase and which parameter set created which kind of performance output, generating status events to record would be a good idea.</p> Feature #2481 (Resolved): Add junit tests for the api methods to test their functionhttps://code.cor-lab.de/issues/24812015-12-11T14:12:22ZH. Oestreichhoestreich@techfak.uni-bielefeld.de
<p>At least the more advanced methods like data payload scaling should be tested.</p> Feature #2480 (Resolved): Implement test suite interfacehttps://code.cor-lab.de/issues/24802015-12-11T14:10:47ZH. Oestreichhoestreich@techfak.uni-bielefeld.de
The test suite should be able to:
<ol>
<li>start a spread daemon</li>
<li>start a specific component</li>
<li>start a process-monitor (<a class="external" href="https://code.cor-lab.de/projects/rsb-performance-monitor">https://code.cor-lab.de/projects/rsb-performance-monitor</a>)</li>
<li>start rsbag</li>
<li>start generator class to produce test data</li>
<li>wait for finish, close all components, stop recording the data, print path to recorded data</li>
</ol>
<ul>
<li>the paths were the applications are found should be configurable by a config file (ini or something similar)</li>
<li>configuration of the programs should also be loaded from a file</li>
</ul> Feature #2448 (Resolved): Implement varargs for some methodshttps://code.cor-lab.de/issues/24482015-11-27T12:50:53ZH. Oestreichhoestreich@techfak.uni-bielefeld.de
<p>ParallelSimplePhase.addTestPhase()<br />LoopPhase.addTestPhase()<br />TestPhase.addTiming()<br />TestPhase.addTransaction()<br />Generator.addTestPhase()</p> Tasks #2447 (Resolved): Design API to generate different payload sizeshttps://code.cor-lab.de/issues/24472015-11-26T12:09:56ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>We restrict this API to protobuf types.</p>
The following aspects of a message can scale:
<ul>
<li>size of a string/byte payload</li>
<li>Number of elements in a repeated field</li>
</ul>
<p>Add special phases (or a generator for an existing phase) which can scale the payload size of a message. The user probably has to provide some kind of object which realizes at least the creation of data in an appropriate size.</p> Tasks #2446 (Resolved): Include different payload sizes in the ObjectBuilder examplehttps://code.cor-lab.de/issues/24462015-11-26T12:06:50ZJ. Wienkejwienke@techfak.uni-bielefeld.de
<p>Iterate different numbers of legs to test out how this can currently be written with the API.</p>