Re: Autobuilder data collection for intermittent bugs
From: Richard Purdie <hidden>
Date: 2021-03-31 22:01:51
On Wed, 2021-03-31 at 17:45 -0400, Sakib Sajal wrote:
quoted hunk ↗ jump to hunk
We were able to follow the example and collect results in: https://autobuilder.yocto.io/pub/non-release/20210331-18/testresults/qa-extras2/2021-03-31--20-32/ host_stats.0 host_stats.1 host_stats.2 We are renaming the files to host_stats_<step>.txt so that it would open on the browser. We were looking at generate-testresult-index.py and would like to append a new column called "AB INT" and hyperlink each file in the folder where "top" was triggered. Normally there should not be any links. Occasionally, we should see 1 or 2 but not more unless things go really bad. Do you agree this is a reasonable approach? Who do we contact about testing the changes? Trevor has an internal instance running, we could try it out there. Below is the diff of the changes we plan to make for the index. We do not completely understand the script yet, we are working on it. autobuilder/yocto-autobuilder-helper$ git diff ./scripts/generate-testresult-index.pydiff --git a/scripts/generate-testresult-index.pyb/scripts/generate-testresult-index.py index d9d577e..27fe06f 100755--- a/scripts/generate-testresult-index.py +++ b/scripts/generate-testresult-index.py@@ -33,6 +33,7 @@ index_templpate = """<th>Performance Reports</th> <th>ptest Logs</th> <th>Buildhistory</th> + <th>AB INT</th> </tr> </thead> <tdata>@@ -57,6 +58,11 @@ index_templpate = """<a href="{{bh[0]}}">{{bh[1]}}</a> {% endfor %} </td> + <td> + {% for abint in entry[8] %} + <a href="{{abint[0]}}">{{abint[1]}}</a> + {% endfor %} + </td> </tr> {% endfor %} </tdata>@@ -145,6 +151,11 @@ for build in sorted(os.listdir(path), key=keygen,reverse=True): if os.path.exists(buildpath + "/qemuarm/buildhistory.txt"): buildhistory.append((reldir + "testresults/qemuarm/buildhistory.txt", "qemuarm")) + abint = [] + for p in glob.glob(buildpath + "/ab-int*/host_stats*"): + abintname = os.path.basename(os.path.dirname(p)) + abintreports.append((reldir + "testresults/" + abintname + "/" + os.path.basename(p), perfname.replace("buildperf-",""))) #todo + branch = get_build_branch(buildpath)
Rather than messing with the main index which is "production", could you just create your own for now for testing? :) FWIW I added tmpfs testing for qemu images into master-next (needs ab-helper master-next too) so it will be interesting to compare builds running with that with the previous build bug trends. Cheers, Richard