On 2021-03-26 1:50 p.m., Richard Purdie wrote:

[Please note: This e-mail is from an EXTERNAL e-mail address]

On Thu, 2021-03-25 at 20:00 -0400, Randy MacLeod wrote:

quoted

Thanks for the update Sakib.

We're planning to follow Steve's example of getting commit access to
contrib branches of y-a-h and poky and using the YP AB to start custom
workers/job initially.  We'll also keep working to duplicate the
AB behaviour locally. Trevor has asked Konrad for an additional system
or two since when jobs are running, the web UI is
painfully slow since it's runnig on the same node right now.

The one part that Sakib didn't ask about is how to get all the workers
to do the data collection but that's not urgent since we'll be doing
some smaller scale experiments tomorrow. If you can point out a helpful
example, that'd be good.

The easiest is the fact that the workers all share an NFS mount which
is at BASE_SHAREDDIR in config.json. See also WEBPUBLISH_DIR and
WEBPUBLISH_URL.

Ok. We have not gone there, yet.

We use this to share the reproducible build failures and to share the
reproducible build failures/diffoscope and test results, e.g.:
OEQA_DEBUGGING_SAVED_OUTPUT=${BASE_SHAREDDIR}/pub/repro-fail/

or perhaps more useful is scripts/collect-results:

"""
#!/bin/bash
WORKDIR=$1
DEST=$2
target=$3

RESFILE=$WORKDIR/tmp/log/oeqa/testresults.json

if [ -e $RESFILE ]; then
     mkdir -p $DEST/$target
     cp $WORKDIR/tmp/log/oeqa/testresults.json  $DEST/$target/
fi
"""

would let you do something similar and DEST here is specific to a given
build (date stamp + number). An example would be:

https://autobuilder.yocto.io/pub/non-release/20210129-3/

and then you could create a folder per target. You may need to
include the worker name in the log.

We were able to follow the example and collect results in:

https://autobuilder.yocto.io/pub/non-release/20210331-18/testresults/qa-extras2/2021-03-31--20-32/

host_stats.0

host_stats.1

host_stats.2

We are renaming the files to host_stats_<step>.txt so that it would open 
on the browser.


We were looking at generate-testresult-index.py and would like to append 
a new column called "AB INT" and hyperlink each file in the folder where 
"top" was triggered. Normally there should not be any links. 
Occasionally, we should see 1 or 2 but not more unless things go really 
bad. Do you agree this is a reasonable approach? Who do we contact about 
testing the changes? Trevor has an internal instance running, we could 
try it out there.


Below is the diff of the changes we plan to make for the index. We do 
not completely understand the script yet, we are working on it.

autobuilder/yocto-autobuilder-helper$ git diff 
./scripts/generate-testresult-index.py

diff --git a/scripts/generate-testresult-index.py 

b/scripts/generate-testresult-index.py
index d9d577e..27fe06f 100755

--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py

@@ -33,6 +33,7 @@ index_templpate = """
    <th>Performance Reports</th>
    <th>ptest Logs</th>
    <th>Buildhistory</th>
+  <th>AB INT</th>
  </tr>
  </thead>
  <tdata>

@@ -57,6 +58,11 @@ index_templpate = """
       <a href="{{bh[0]}}">{{bh[1]}}</a>
     {% endfor %}
     </td>
+   <td>
+   {% for abint in entry[8] %}
+     <a href="{{abint[0]}}">{{abint[1]}}</a>
+   {% endfor %}
+   </td>
  </tr>
  {% endfor %}
  </tdata>

@@ -145,6 +151,11 @@ for build in sorted(os.listdir(path), key=keygen, 

reverse=True):
      if os.path.exists(buildpath + "/qemuarm/buildhistory.txt"):
          buildhistory.append((reldir + 
"testresults/qemuarm/buildhistory.txt", "qemuarm"))

+    abint = []
+    for p in glob.glob(buildpath + "/ab-int*/host_stats*"):
+        abintname = os.path.basename(os.path.dirname(p))
+        abintreports.append((reldir + "testresults/" + abintname + "/" 
+ os.path.basename(p), perfname.replace("buildperf-",""))) #todo
+
      branch = get_build_branch(buildpath)


Regards,

Sakib, Randy

Cheers,

Richard