Re: [PATCH v5 4/5] oeqa/selftest/cases/rust.py: Rust oe-selftest script.

Hi Alex,

Is the below comment about execution time is ok ?

+# Total time taken for testing is of about 2hr 20min, with
PARALLEL_MAKE set to 40 number of jobs.
 class RustSelfTestBase(OESelftestTestCase, OEPTestResultTestCase):

And, should I send v8 version of patches (all 5 patches) or only the
patch of rust.py  with this comment as v8 patch ?

Regards,
Vinay

On Sat, May 29, 2021 at 2:40 AM Alexander Kanavin
[off-list ref] wrote:

Thanks Vinay, the code looks nice and clean now, and I have no further comments about it.

I would only ask that you put the amount of time it takes to (successfully) execute as a comment just before the class definition, so that anyone who needs to allocate CI resources for this test knows what is the reasonable way to run it.

Alex

On Fri, 28 May 2021 at 18:13, Vinay Kumar [off-list ref] wrote:

quoted

Hi Alex,

Please ignore the v6 patch set. As there is minor correction in rust.py
- cmd = cmd + " cd %s;  python3 src/bootstrap/bootstrap.py test
codegen %s --target %s ;" % (builddir, testargs, targetsys)
+ cmd = cmd + " cd %s;  python3 src/bootstrap/bootstrap.py test %s
--target %s ;" % (builddir, testargs, targetsys)

Sent v7 patches with corrections of runCmd and ssh.

For copying "remote-test-server" used copy_to from sshcontrol.py, and
for ssh background execution used "subprocess" module.

Regarding result.output, whenever there is failure it gets printed
test summary and also captured in "oe-selftest-results.log".
And when there is failure with runCmd then no test summary details or
printed.  This is taken care of by runCmd() itself from commands.py.

Verified with 2 independent test modules "compile-fail" and "codegen"
which are part of rust testing.

compile-test : All tests get passed and no errors thrown, so
oe-selftes does not print the test results.

codegen : Some of the tests fail, making the shell exit. In this case
oe-selftest prints test details with stdout and stderr.

Regards,
Vinay


On Thu, May 27, 2021 at 7:05 PM Alexander Kanavin
[off-list ref] wrote:

quoted

Hello Vinay,

sure, no problem. I see that SSHControl.run() isn't designed for starting background processes on the target. But I still think it's then better to use subprocess module for starting the server over ssh, rather than bundle everything into one gigantic runCmd().

Alex

On Thu, 27 May 2021 at 14:18, Vinay Kumar [off-list ref] wrote:

quoted

Hi Alex,

The reason why I changed this is, we have to execute
"remote-test-server" on the image through background ssh.
So that the "remote-test-server" will be alive waiting for connection.
Then in the rust.py  control passed to the next command (cmd) for
execution.
If we use the sshcontrol.py method we have to set a timeout of 600
otherwise default timeout is 300. Even if the test execution completes
early due to failure then we have to wait till this timeout.

I tried reducing the timeout to 60 seconds and some of the tests got skipped.

If that is ok I will use that copy_to and run from sshcontrol.py as we
did in v4 patches and work for the result.output

Regards,
Vinay

On Thu, May 27, 2021 at 5:33 PM Alexander Kanavin
[off-list ref] wrote:

quoted

On Thu, 27 May 2021 at 10:11, Vinay Kumar [off-list ref] wrote:

quoted

+                       # To kill remote-test-server executing through background ssh
+                       killcommand = "kill -9 \$(ps | grep remote-test-server |  grep -v \"grep\" | awk '{print \$1}')"
+                       cmd = cmd + "ssh %s root@%s \"%s\";" % (sshargs, qemu.ip, killcommand)

I don't think you actually need to kill the remote test server. As qemu will be stopped anyway, and with it, all the processes that run inside qemu.

Also, it's better to use cmd only to execute the tests. Starting the server is better done through SSHControl.run(), and copying the binary via SSHControl.copy_to() - see examples in selfttest's imagefeatures.py or runtime_test.py.

Alex