1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<title>org.eclipse.test.performance.ui readme</title>
</head>
<p><b><font size="+2">README</font></b><br> </p><p>last updated: October 26, 2005<br>
</p><p><b>Introduction</b><br> <br> This readme describes how to create performance
charts and tables similar to those available on the eclipse platform download
pages using the stand-alone java program <b>org.eclipse.test.performance.ui.Main</b>
stored in this project. Although the program is currently geared to generating
results for the Eclipse Platform builds, the program may work for you given the
following conditions:</p><ul> <li>The "eclipse.perf.config" system property
used to run tests used three keys config, build and jvm <ul> <li>for example,
"eclipse.perf.config=config=relengbuildwin2;build=I20050219-1500;jvm=sun1.4.2_06".
<br> <br> </li></ul></li><li>Values for the "build" key begin with either an 'I'
or 'N' with the exception of the reference builds (for example, 3.0). <ul> <li>The
line graphs plot the reference build, all builds starting with 'I' and the seven
'N' builds preceding the current build.</li><li>This can be overriden with the use of the "-current.prefix" parameter where you can specify a comma-separated list of prefixes for build ids to include in the main line graph.<br> <br> </li></ul></li><li>Scenario names
all contain a prefix "<component>.test" <ul> <li>For example,
it is assumed that the scenario "org.eclipse.help.tests.performance.appserver.TestAppserver#testAppserverCycle()"
belongs to the "org.eclipse.help" component. </li><li>Unique scenario
prefixes are derived from all the scenarios in the database and are used to group
results.</li></ul></li></ul><p><br> For purposes of this document, the term "<a href="images/descriptions.html#fp">fingerprint</a>"
will be used to refer to a bar graph which represents the performance summary
at a global or component level for each test configuration.<br> <br> The term
"<a href="images/descriptions.html#sst">scenario status table</a>" will
be used to refer to a table of scenarios with green checks and red x indicators
below a component fingerprint. This table gives an overall view of the status
for all scenarios for a component for all configurations.<br> </p><p>The term
"<a href="images/descriptions.html#sr">scenario results</a>" will be
used to refer to a web page comparing current performance measurements to the
baseline measurements for one scenario on one configuration. The web page displays
the raw data for the current and baseline measurements and also displays line
graphs showing the measured values over the course of builds between the current
and baseline.</p><p><br> <b>Requirements</b> </p><ul> <li>Windows or Linux (x86)
operating system</li><li>Cloudscape or Derby plug-in</li><li>org.eclipse.releng.basebuilder</li><li>1.4
level jvm installed and on path</li></ul><p><b>Setup</b><br> </p><ul> <li>Checkout
org.eclipse.releng.basebuilder from cvs repository dev.eclipse.org:/home/eclipse
(HEAD should be fine).</li><li>Create a Cloudscape or derby plug-in as described
<A HREF="http://dev.eclipse.org/viewcvs/index.cgi/*checkout*/org.eclipse.test.performance/doc/Performance%20Tests%20HowTo.html?rev=HEAD&content-type=text/html#cloudscape">here</A>.</li><li>Set
the environment variable "dbloc" to the location of the database containing
the performance data<br> For example:<br> <ul> <li> export dbloc=net://minsky.ottawa.ibm.com
(Linux)</li><li>set dbloc==net://minsky.ottawa.ibm.com (Windows)<br> </li></ul></li></ul><p><b>Running
the program</b></p><p>From a terminal or dos prompt,<br> </p><OL><LI>Set the environment
variable "dbloc" to the location of the database containing the performance
data<BR> For example:<UL><LI>export dbloc=net://minsky.ottawa.ibm.com (Linux)</LI><LI>set
dbloc==net://minsky.ottawa.ibm.com (Windows)<BR><BR></LI></UL></LI><LI>cd
to org.eclipse.releng.basebuilder\plugins\org.eclipse.test.performance.ui\scripts<BR></LI><LI>If
running on Linux, execute "chmod 755 genresults"</LI><LI>Execute the
following command:<b><br> <br> genresults -baseline <baseline build id>
-current <current build id> -config <comma-separated list of configs>
-jvm <jvm name> -output <output dir> [-config.properties <properties>]
[highlight <buildid patterns>] [scenario.pattern <scenario prefix
patterns>] [-fingerprints][-scenarioresults][-current.prefix <comma-separated list of build id prefixes>][-baseline.prefix <comma-separated list of build id prefixes>]</b></LI></OL><br> Parameter descriptions:
<blockquote> <p>-baseline <baseline build id. A previous value used in the
"build" key in the eclipse.perf.config system property or the eclipse.perf.assertAgainst
property. > <br> -current <current build id. The value used in the "build"
key in the eclipse.perf.config system property.><br> -config <a comma-separated
list of names of configurations for which to produce results. The values used
should be the same as values specified for the "config" key in the eclipse.perf.config
system property.><br> -jvm <jvm description. The value used in the "jvm"
key in the eclipse.perf.config system property.><br> -output <path to an
output directory for fingerprints and scenarios for each configuration specified
in -config parameter. Fingerprint gifs and html files are generated in the directory
specified, configuration specific subdirectories are created to store scenario
results and line graphs.></p></blockquote><p><br> Optional arguments:</p><blockquote>
<p>-config.properties <semi-colon separated list of: config, alternate config
description. The value should be quoted if there are spaces in the value.><BR>of builds which match any prefix in this list. When omitted, a magenta line is draw on the graph which represents the baseline value specified in the -baseline parameter.><br> -highlight <Comma-separated
list of build Id prefixes(excluding 'I' and 'N'), which are used to add and highlight
the most recent matching buildId on line graphs."><br> -scenario.pattern
<Scenario prefix pattern used to query performance results database. Can be
used to generate results for a single component or subset of scenarios.><br>
-fingerprints or -scenarioresults <use one or the other to generate fingerprints
or scenario results only. Not specifying either will execute both.><BR>-baseline.prefix <semi-colon separated list of: build id prefixes used in eclipse.perf.assertAgainst property or eclipse.perf.config system property. Values used to generate a second line graph representing changes in repeated baseline test runs>.<BR>-current.prefix <semi-colon separated list of: build id prefixes used in eclipse.perf.config system property. Values here used to override default of "N,I". Used to select build id's to include in main line graph.></p></blockquote>
</html>
|
