On 24/07/09 01:44AM, Karthik Nayak wrote:

Justin Tobler [off-list ref] writes:

quoted

On 24/07/08 02:16PM, Karthik Nayak wrote:

quoted

Justin Tobler [off-list ref] writes:

quoted

On 24/07/08 11:23AM, Karthik Nayak wrote:

quoted

We don't run style checks on our CI, even though we have a
'.clang-format' setup in the repository. Let's add one, the job will
validate only against the new commits added and will only run on merge
requests. Since we're introducing it for the first time, let's allow
this job to fail, so we can validate if this is useful and eventually
enforce it.

[snip]

quoted

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 37b991e080..65fd261e5e 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml

@@ -123,6 +123,18 @@ check-whitespace:
   rules:
     - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

+check-style:
+  image: ubuntu:latest
+  allow_failure: true
+  variables:
+    CC: clang
+  before_script:
+    - ./ci/install-dependencies.sh
+  script:
+    - ./ci/run-style-check.sh "$CI_MERGE_REQUEST_DIFF_BASE_SHA"

One downside to using $CI_MERGE_REQUEST_DIFF_BASE_SHA is that for GitLab
merge pipeines, commits from the merge that are not part of the MR
changes are also included. This could lead to somewhat confusing
failures.

I'm not sure I follow.

quoted

Example failure occuring on this patch series:
https://gitlab.com/gitlab-org/git/-/jobs/7284442220

If you notice this job, it points to the commit: 1c6551488, and the
parent commit of that commit is: 614dff2011.

The parent commit [1] is a test commit I added to check the failures. So
isn't this working as expected?

Ah ok, I misunderstood the setup of that CI job, but the problem is
still present. Here is an example CI job I've run demonstrating it:

CI - https://gitlab.com/gitlab-org/git/-/jobs/7291829941
MR - https://gitlab.com/gitlab-org/git/-/merge_requests/174

For the MR that spawned this CI job, This patch series is the source
branch and the target branch is a version of master one commit ahead
containing a clang format error. Because this is a merge pipeline, using
$CI_MERGE_REQUEST_DIFF_BASE_SHA will include changes from either side of
the base commit. This means it would be possible for the CI job to fail
due to commits ahead in the target branch, but not in the source branch.
For the check-whitespace CI job, I specifically chose
$CI_MERGE_REQUEST_TARGET_BRANCH_SHA for this reason.j

You're right indeed. I did some more reading about this and I think the
solution lies somewhere in between..

quoted

quoted

quoted

It might be best to use $CI_MERGE_REQUEST_TARGET_BRANCH_SHA instead.

I actually started with $CI_MERGE_REQUEST_TARGET_BRANCH_SHA, it didn't
work, because the value was undefined.

See: https://gitlab.com/gitlab-org/git/-/jobs/7283724903

This is why I also decided to fix and change the whitespace check.

I'm not seeing $CI_MERGE_REQUEST_TARGET_BRANCH_SHA as undefined in the
job. Here is a modified version on the check-style CI job printing the
environment variables:

You can see the output

    $ ./ci/run-style-check.sh "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA"
    fatal: ambiguous argument '': unknown revision or path not in the
working tree.
    Use '--' to separate paths from revisions, like this:
    'git <command> [<revision>...] -- [<file>...]'

This only happens if "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA" is empty.

Ya I noticed this failure, but was wondering if it was maybe due to
something else. I have been unable to reproduce it and in all the jobs I
was running resulted in a merge pipeline with the variable defined. But 
maybe sometimes a regular pipeline gets run for some reason and 
consequently $CI_MERGE_REQUEST_TARGET_BRANCH_SHA is not defined? Was the
pipeline triggered directly from the source branch?

quoted

https://gitlab.com/gitlab-org/git/-/jobs/7291792329#L2470

Do you have an example of the check-whitespace job failing in GitLab CI?
Maybe I'm missing something, but I don't see a problem.

-Justin

So I think I get the issue, GitLab has two kinds of pipelines it runs:
1. merge pipeline: Here the pipeline runs on the source branch (the
feature branch which has to be merged).
2. merged pipeline: Here the pipeline creates a merge commit using the
source and target branch and then runs the pipeline on the merged
commit.

Correct, this is my understanding.

And "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA" is only defined in the 'merged
pipeline'. If you see the pipelines for my branch on GitLab [1]. You'll
see only one of them is marked as 'merge results' and the others being
marked as 'merged results'. The former includes the job I mentioned
above, where "$CI_MERGE_REQUEST_TARGET_BRANCH_SHA" is not defined.

I'm still not sure why it marked only one of the pipelines as such, but
this means there is chance that it could happen.

Huh, I'm guessing the CI job must have been triggered from the source
branch directly. Did you manually run the CI job? I wonder if that could
have caused it.

So I guess the best outcome is to use
"$CI_MERGE_REQUEST_TARGET_BRANCH_SHA", but fallback to
"$CI_MERGE_REQUEST_DIFF_BASE_SHA", if the former is not defined.

This is exactly what I think we should do too. For merge pipelines we 
will want to use $CI_MERGE_REQUEST_TARGET_BRANCH_SHA so that only the 
commits included in the MR are scanned in CI. If that variable is not 
defined, it makes sense to fallback to $CI_MERGE_REQUEST_DIFF_BASE_SHA.
Since it's not a merge pipeline it will only scan commits included from
the MR and therefore work as expected.

This should handle both cases correctly. :)

-Justin