From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-03 13:34:49
From: Derrick Stolee <redacted>
Just like `scalar register` starts the scheduled background maintenance,
`scalar unregister` stops it. Note that we use `git maintenance start`
in `scalar register`, but we do not use `git maintenance stop` in
`scalar unregister`: this would stop maintenance for _all_ repositories,
not just for the one we want to unregister.
The `unregister` command also removes the corresponding entry from the
`[scalar]` section in the global Git config.
Co-authored-by: Victoria Dye [off-list ref]
Signed-off-by: Derrick Stolee <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 50 ++++++++++++++++++++++++++++++++-------
contrib/scalar/scalar.txt | 8 +++++++
2 files changed, 50 insertions(+), 8 deletions(-)
@@ -45,6 +46,13 @@ Note: when this subcommand is called in a worktree that is called `src/`, its parent directory is considered to be the Scalar enlistment. If the worktree is _not_ called `src/`, it itself will be considered to be the Scalar enlistment.+Unregister+~~~~~~~~~~++unregister [<enlistment>]::+ Remove the specified repository from the list of repositories+ registered with Scalar and stop the scheduled background maintenance.+ SEE ALSO -------- linkgit:git-maintenance[1].
From: Johannes Schindelin via GitGitGadget <hidden> Date: 2021-12-03 13:34:51
From: Johannes Schindelin <redacted>
This implements Scalar's opinionated `clone` command: it tries to use a
partial clone and sets up a sparse checkout by default. In contrast to
`git clone`, `scalar clone` sets up the worktree in the `src/`
subdirectory, to encourage a separation between the source files and the
build output (which helps Git tremendously because it avoids untracked
files that have to be specifically ignored when refreshing the index).
Also, it registers the repository for regular, scheduled maintenance,
and configures a flurry of configuration settings based on the
experience and experiments of the Microsoft Windows and the Microsoft
Office development teams.
Note: since the `scalar clone` command is by far the most commonly
called `scalar` subcommand, we document it at the top of the manual
page.
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 201 +++++++++++++++++++++++++++++++
contrib/scalar/scalar.txt | 32 ++++-
contrib/scalar/t/t9099-scalar.sh | 32 +++++
3 files changed, 262 insertions(+), 3 deletions(-)
@@ -251,6 +252,205 @@ static int unregister_dir(void)returnres;}+/* printf-style interface, expects `<key>=<value>` argument */+staticintset_config(constchar*fmt,...)+{+structstrbufbuf=STRBUF_INIT;+char*value;+intres;+va_listargs;++va_start(args,fmt);+strbuf_vaddf(&buf,fmt,args);+va_end(args);++value=strchr(buf.buf,'=');+if(value)+*(value++)='\0';+res=git_config_set_gently(buf.buf,value);+strbuf_release(&buf);++returnres;+}++staticchar*remote_default_branch(constchar*url)+{+structchild_processcp=CHILD_PROCESS_INIT;+structstrbufout=STRBUF_INIT;++cp.git_cmd=1;+strvec_pushl(&cp.args,"ls-remote","--symref",url,"HEAD",NULL);+if(!pipe_command(&cp,NULL,0,&out,0,NULL,0)){+constchar*line=out.buf;++while(*line){+constchar*eol=strchrnul(line,'\n'),*p;+size_tlen=eol-line;+char*branch;++if(!skip_prefix(line,"ref: ",&p)||+!strip_suffix_mem(line,&len,"\tHEAD")){+line=eol+(*eol=='\n');+continue;+}++eol=line+len;+if(skip_prefix(p,"refs/heads/",&p)){+branch=xstrndup(p,eol-p);+strbuf_release(&out);+returnbranch;+}++error(_("remote HEAD is not a branch: '%.*s'"),+(int)(eol-p),p);+strbuf_release(&out);+returnNULL;+}+}+warning(_("failed to get default branch name from remote; "+"using local default"));+strbuf_reset(&out);++child_process_init(&cp);+cp.git_cmd=1;+strvec_pushl(&cp.args,"symbolic-ref","--short","HEAD",NULL);+if(!pipe_command(&cp,NULL,0,&out,0,NULL,0)){+strbuf_trim(&out);+returnstrbuf_detach(&out,NULL);+}++strbuf_release(&out);+error(_("failed to get default branch name"));+returnNULL;+}++staticintcmd_clone(intargc,constchar**argv)+{+constchar*branch=NULL;+intfull_clone=0;+structoptionclone_options[]={+OPT_STRING('b',"branch",&branch,N_("<branch>"),+N_("branch to checkout after clone")),+OPT_BOOL(0,"full-clone",&full_clone,+N_("when cloning, create full working directory")),+OPT_END(),+};+constchar*constclone_usage[]={+N_("scalar clone [<options>] [--] <repo> [<dir>]"),+NULL+};+constchar*url;+char*enlistment=NULL,*dir=NULL;+structstrbufbuf=STRBUF_INIT;+intres;++argc=parse_options(argc,argv,NULL,clone_options,clone_usage,0);++if(argc==2){+url=argv[0];+enlistment=xstrdup(argv[1]);+}elseif(argc==1){+url=argv[0];++strbuf_addstr(&buf,url);+/* Strip trailing slashes, if any */+while(buf.len>0&&is_dir_sep(buf.buf[buf.len-1]))+strbuf_setlen(&buf,buf.len-1);+/* Strip suffix `.git`, if any */+strbuf_strip_suffix(&buf,".git");++enlistment=find_last_dir_sep(buf.buf);+if(!enlistment){+die(_("cannot deduce worktree name from '%s'"),url);+}+enlistment=xstrdup(enlistment+1);+}else{+usage_msg_opt(_("You must specify a repository to clone."),+clone_usage,clone_options);+}++if(is_directory(enlistment))+die(_("directory '%s' exists already"),enlistment);++dir=xstrfmt("%s/src",enlistment);++strbuf_reset(&buf);+if(branch)+strbuf_addf(&buf,"init.defaultBranch=%s",branch);+else{+char*b=repo_default_branch_name(the_repository,1);+strbuf_addf(&buf,"init.defaultBranch=%s",b);+free(b);+}++if((res=run_git("-c",buf.buf,"init","--",dir,NULL)))+gotocleanup;++if(chdir(dir)<0){+res=error_errno(_("could not switch to '%s'"),dir);+gotocleanup;+}++setup_git_directory();++/* common-main already logs `argv` */+trace2_def_repo(the_repository);++if(!branch&&!(branch=remote_default_branch(url))){+res=error(_("failed to get default branch for '%s'"),url);+gotocleanup;+}++if(set_config("remote.origin.url=%s",url)||+set_config("remote.origin.fetch="+"+refs/heads/*:refs/remotes/origin/*")||+set_config("remote.origin.promisor=true")||+set_config("remote.origin.partialCloneFilter=blob:none")){+res=error(_("could not configure remote in '%s'"),dir);+gotocleanup;+}++if(!full_clone&&+(res=run_git("sparse-checkout","init","--cone",NULL)))+gotocleanup;++if(set_recommended_config())+returnerror(_("could not configure '%s'"),dir);++if((res=run_git("fetch","--quiet","origin",NULL))){+warning(_("partial clone failed; attempting full clone"));++if(set_config("remote.origin.promisor")||+set_config("remote.origin.partialCloneFilter")){+res=error(_("could not configure for full clone"));+gotocleanup;+}++if((res=run_git("fetch","--quiet","origin",NULL)))+gotocleanup;+}++if((res=set_config("branch.%s.remote=origin",branch)))+gotocleanup;+if((res=set_config("branch.%s.merge=refs/heads/%s",+branch,branch)))+gotocleanup;++strbuf_reset(&buf);+strbuf_addf(&buf,"origin/%s",branch);+res=run_git("checkout","-f","-t",buf.buf,NULL);+if(res)+gotocleanup;++res=register_dir();++cleanup:+free(enlistment);+free(dir);+strbuf_release(&buf);+returnres;+}+staticintcmd_list(intargc,constchar**argv){if(argc!=1)
@@ -29,12 +30,37 @@ an existing Git worktree with Scalar whose name is not `src`, the enlistment will be identical to the worktree. The `scalar` command implements various subcommands, and different options-depending on the subcommand. With the exception of `list`, all subcommands-expect to be run in an enlistment.+depending on the subcommand. With the exception of `clone` and `list`, all+subcommands expect to be run in an enlistment. COMMANDS --------+Clone+~~~~~++clone [<options>] <url> [<enlistment>]::+ Clones the specified repository, similar to linkgit:git-clone[1]. By+ default, only commit and tree objects are cloned. Once finished, the+ worktree is located at `<enlistment>/src`.+++The sparse-checkout feature is enabled (except when run with `--full-clone`)+and the only files present are those in the top-level directory. Use+`git sparse-checkout set` to expand the set of directories you want to see,+or `git sparse-checkout disable` to expand to all files (see+linkgit:git-sparse-checkout[1] for more details). You can explore the+subdirectories outside your sparse-checkout by using `git ls-tree+HEAD[:<directory>]`.++-b <name>::+--branch <name>::+ Instead of checking out the branch pointed to by the cloned+ repository's HEAD, check out the `<name>` branch instead.++--[no-]full-clone::+ A sparse-checkout is initialized by default. This behavior can be+ turned off via `--full-clone`.+ List ~~~~
@@ -64,7 +90,7 @@ unregister [<enlistment>]:: SEE ALSO ---------linkgit:git-maintenance[1].+linkgit:git-clone[1], linkgit:git-maintenance[1]. Scalar ---
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-03 13:34:54
From: Derrick Stolee <redacted>
The produced list simply consists of those repositories registered under
the multi-valued `scalar.repo` config setting in the user's Git config.
Signed-off-by: Derrick Stolee <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 11 +++++++++++
contrib/scalar/scalar.txt | 11 ++++++++++-
2 files changed, 21 insertions(+), 1 deletion(-)
@@ -251,6 +251,16 @@ static int unregister_dir(void)returnres;}+staticintcmd_list(intargc,constchar**argv)+{+if(argc!=1)+die(_("`scalar list` does not take arguments"));++if(run_git("config","--global","--get-all","scalar.repo",NULL)<0)+return-1;+return0;+}+staticintcmd_register(intargc,constchar**argv){structoptionoptions[]={
@@ -28,11 +29,19 @@ an existing Git worktree with Scalar whose name is not `src`, the enlistment will be identical to the worktree. The `scalar` command implements various subcommands, and different options-depending on the subcommand.+depending on the subcommand. With the exception of `list`, all subcommands+expect to be run in an enlistment. COMMANDS --------+List+~~~~++list::+ List enlistments that are currently registered by Scalar. This+ subcommand does not need to be run inside an enlistment.+ Register ~~~~~~~~
From: Johannes Schindelin via GitGitGadget <hidden> Date: 2021-12-03 13:34:55
From: Johannes Schindelin <redacted>
When a user deleted an enlistment manually, let's be generous and
_still_ unregister it.
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 46 ++++++++++++++++++++++++++++++++
contrib/scalar/t/t9099-scalar.sh | 15 +++++++++++
2 files changed, 61 insertions(+)
From: Johannes Schindelin via GitGitGadget <hidden> Date: 2021-12-03 13:34:55
From: Johannes Schindelin <redacted>
Just like `git clone`, the `scalar clone` command now also offers to
restrict the clone to a single branch.
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 9 +++++++--
contrib/scalar/scalar.txt | 12 +++++++++++-
contrib/scalar/t/t9099-scalar.sh | 6 +++++-
3 files changed, 23 insertions(+), 4 deletions(-)
@@ -327,12 +327,15 @@ static char *remote_default_branch(const char *url)staticintcmd_clone(intargc,constchar**argv){constchar*branch=NULL;-intfull_clone=0;+intfull_clone=0,single_branch=0;structoptionclone_options[]={OPT_STRING('b',"branch",&branch,N_("<branch>"),N_("branch to checkout after clone")),OPT_BOOL(0,"full-clone",&full_clone,N_("when cloning, create full working directory")),+OPT_BOOL(0,"single-branch",&single_branch,+N_("only download metadata for the branch that will "+"be checked out")),OPT_END(),};constchar*constclone_usage[]={
@@ -403,7 +406,9 @@ static int cmd_clone(int argc, const char **argv)if(set_config("remote.origin.url=%s",url)||set_config("remote.origin.fetch="-"+refs/heads/*:refs/remotes/origin/*")||+"+refs/heads/%s:refs/remotes/origin/%s",+single_branch?branch:"*",+single_branch?branch:"*")||set_config("remote.origin.promisor=true")||set_config("remote.origin.partialCloneFilter=blob:none")){res=error(_("could not configure remote in '%s'"),dir);
@@ -57,6 +57,16 @@ HEAD[:<directory>]`. Instead of checking out the branch pointed to by the cloned repository's HEAD, check out the `<name>` branch instead.+--[no-]single-branch::+ Clone only the history leading to the tip of a single branch, either+ specified by the `--branch` option or the primary branch remote's+ `HEAD` points at.+++Further fetches into the resulting repository will only update the+remote-tracking branch for the branch this option was used for the initial+cloning. If the HEAD at the remote did not point at any branch when+`--single-branch` clone was made, no remote-tracking branch is created.+ --[no-]full-clone:: A sparse-checkout is initialized by default. This behavior can be turned off via `--full-clone`.
From: Derrick Stolee via GitGitGadget <hidden> Date: 2021-12-03 13:34:57
From: Derrick Stolee <redacted>
Note: this subcommand is provided primarily for backwards-compatibility,
for existing Scalar uses. It is mostly just a shim for `git
maintenance`, mapping task names from the way Scalar called them to the
way Git calls them.
The reason why those names differ? The background maintenance was first
implemented in Scalar, and when it was contributed as a patch series
implementing the `git maintenance` command, reviewers suggested better
names, those suggestions were accepted before the patches were
integrated into core Git.
Signed-off-by: Derrick Stolee <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 64 +++++++++++++++++++++++++++++++++++++++
contrib/scalar/scalar.txt | 19 ++++++++++++
2 files changed, 83 insertions(+)
@@ -484,6 +484,69 @@ static int cmd_register(int argc, const char **argv)returnregister_dir();}+staticintcmd_run(intargc,constchar**argv)+{+structoptionoptions[]={+OPT_END(),+};+struct{+constchar*arg,*task;+}tasks[]={+{"config",NULL},+{"commit-graph","commit-graph"},+{"fetch","prefetch"},+{"loose-objects","loose-objects"},+{"pack-files","incremental-repack"},+{NULL,NULL}+};+structstrbufbuf=STRBUF_INIT;+constchar*usagestr[]={NULL,NULL};+inti;++strbuf_addstr(&buf,N_("scalar run <task> [<enlistment>]\nTasks:\n"));+for(i=0;tasks[i].arg;i++)+strbuf_addf(&buf,"\t%s\n",tasks[i].arg);+usagestr[0]=buf.buf;++argc=parse_options(argc,argv,NULL,options,+usagestr,0);++if(!argc)+usage_with_options(usagestr,options);++if(!strcmp("all",argv[0])){+i=-1;+}else{+for(i=0;tasks[i].arg&&strcmp(tasks[i].arg,argv[0]);i++)+;/* keep looking for the task */++if(i>0&&!tasks[i].arg){+error(_("no such task: '%s'"),argv[0]);+usage_with_options(usagestr,options);+}+}++argc--;+argv++;+setup_enlistment_directory(argc,argv,usagestr,options,NULL);+strbuf_release(&buf);++if(i==0)+returnregister_dir();++if(i>0)+returnrun_git("maintenance","run",+"--task",tasks[i].task,NULL);++if(register_dir())+return-1;+for(i=1;tasks[i].arg;i++)+if(run_git("maintenance","run",+"--task",tasks[i].task,NULL))+return-1;+return0;+}+staticintremove_deleted_enlistment(structstrbuf*path){intres=0;
@@ -98,6 +99,24 @@ unregister [<enlistment>]:: Remove the specified repository from the list of repositories registered with Scalar and stop the scheduled background maintenance.+Run+~~~++scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) [<enlistment>]::+ Run the given maintenance task (or all tasks, if `all` was specified).+ Except for `all` and `config`, this subcommand simply hands off to+ linkgit:git-maintenance[1] (mapping `fetch` to `prefetch` and+ `pack-files` to `incremental-repack`).+++These tasks are run automatically as part of the scheduled maintenance,+as soon as the repository is registered with Scalar. It should therefore+not be necessary to run this subcommand manually.+++The `config` task is specific to Scalar and configures all those+opinionated default settings that make Git work more efficiently with+large repositories. As this task is run as part of `scalar clone`+automatically, explicit invocations of this task are rarely needed.+ SEE ALSO -------- linkgit:git-clone[1], linkgit:git-maintenance[1].
From: Johannes Schindelin via GitGitGadget <hidden> Date: 2021-12-03 13:34:58
From: Johannes Schindelin <redacted>
This comes in handy during Scalar upgrades, or when config settings were
messed up by mistake.
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 79 +++++++++++++++++++++-----------
contrib/scalar/scalar.txt | 8 ++++
contrib/scalar/t/t9099-scalar.sh | 8 ++++
3 files changed, 67 insertions(+), 28 deletions(-)
@@ -117,6 +118,13 @@ opinionated default settings that make Git work more efficiently with large repositories. As this task is run as part of `scalar clone` automatically, explicit invocations of this task are rarely needed.+Reconfigure+~~~~~~~~~~~++After a Scalar upgrade, or when the configuration of a Scalar enlistment+was somehow corrupted or changed by mistake, this subcommand allows to+reconfigure the enlistment.+ SEE ALSO -------- linkgit:git-clone[1], linkgit:git-maintenance[1].
From: Johannes Schindelin via GitGitGadget <hidden> Date: 2021-12-03 13:34:59
From: Johannes Schindelin <redacted>
After a Scalar upgrade, it can come in really handy if there is an easy
way to reconfigure all Scalar enlistments. This new option offers this
functionality.
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 61 ++++++++++++++++++++++++++++++--
contrib/scalar/scalar.txt | 9 +++--
contrib/scalar/t/t9099-scalar.sh | 3 ++
3 files changed, 67 insertions(+), 6 deletions(-)
@@ -488,22 +488,77 @@ static int cmd_register(int argc, const char **argv)returnregister_dir();}+staticintget_scalar_repos(constchar*key,constchar*value,void*data)+{+structstring_list*list=data;++if(!strcmp(key,"scalar.repo"))+string_list_append(list,value);++return0;+}+staticintcmd_reconfigure(intargc,constchar**argv){+intall=0;structoptionoptions[]={+OPT_BOOL('a',"all",&all,+N_("reconfigure all registered enlistments")),OPT_END(),};constchar*constusage[]={-N_("scalar reconfigure [<enlistment>]"),+N_("scalar reconfigure [--all | <enlistment>]"),NULL};+structstring_listscalar_repos=STRING_LIST_INIT_DUP;+inti,res=0;+structrepositoryr={NULL};+structstrbufcommondir=STRBUF_INIT,gitdir=STRBUF_INIT;argc=parse_options(argc,argv,NULL,options,usage,0);-setup_enlistment_directory(argc,argv,usage,options,NULL);+if(!all){+setup_enlistment_directory(argc,argv,usage,options,NULL);++returnset_recommended_config(1);+}++if(argc>0)+usage_msg_opt(_("--all or <enlistment>, but not both"),+usage,options);++git_config(get_scalar_repos,&scalar_repos);-returnset_recommended_config(1);+for(i=0;i<scalar_repos.nr;i++){+constchar*dir=scalar_repos.items[i].string;++strbuf_reset(&commondir);+strbuf_reset(&gitdir);++if(chdir(dir)<0){+warning_errno(_("could not switch to '%s'"),dir);+res=-1;+}elseif(discover_git_directory(&commondir,&gitdir)<0){+warning_errno(_("git repository gone in '%s'"),dir);+res=-1;+}else{+git_config_clear();++the_repository=&r;+r.commondir=commondir.buf;+r.gitdir=gitdir.buf;++if(set_recommended_config(1)<0)+res=-1;+}+}++string_list_clear(&scalar_repos,1);+strbuf_release(&commondir);+strbuf_release(&gitdir);++returnres;}staticintcmd_run(intargc,constchar**argv)
@@ -32,8 +32,8 @@ an existing Git worktree with Scalar whose name is not `src`, the enlistment will be identical to the worktree. The `scalar` command implements various subcommands, and different options-depending on the subcommand. With the exception of `clone` and `list`, all-subcommands expect to be run in an enlistment.+depending on the subcommand. With the exception of `clone`, `list` and+`reconfigure --all`, all subcommands expect to be run in an enlistment. COMMANDS --------
@@ -125,6 +125,9 @@ After a Scalar upgrade, or when the configuration of a Scalar enlistment was somehow corrupted or changed by mistake, this subcommand allows to reconfigure the enlistment.+With the `--all` option, all enlistments currently registered with Scalar+will be reconfigured. Use this option after each Scalar upgrade.+ SEE ALSO -------- linkgit:git-clone[1], linkgit:git-maintenance[1].
From: Matthew John Cheetham via GitGitGadget <hidden> Date: 2021-12-03 13:35:00
From: Matthew John Cheetham <redacted>
Delete an enlistment by first unregistering the repository and then
deleting the enlistment directory (usually the directory containing the
worktree `src/` directory).
On Windows, if the current directory is inside the enlistment's
directory, change to the parent of the enlistment directory, to allow us
to delete the enlistment (directories used by processes e.g. as current
working directories cannot be deleted on Windows).
Co-authored-by: Victoria Dye [off-list ref]
Signed-off-by: Matthew John Cheetham <redacted>
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 63 ++++++++++++++++++++++++++++++++
contrib/scalar/scalar.txt | 8 ++++
contrib/scalar/t/t9099-scalar.sh | 9 +++++
3 files changed, 80 insertions(+)
@@ -328,6 +330,33 @@ static char *remote_default_branch(const char *url)returnNULL;}+staticintdelete_enlistment(structstrbuf*enlistment)+{+#ifdef WIN32+structstrbufparent=STRBUF_INIT;+#endif++if(unregister_dir())+die(_("failed to unregister repository"));++#ifdef WIN32+/*+*Changethecurrentdirectorytooneoutsideoftheenlistmentso+*thatwemaydeleteeverythingunderneathit.+*/+strbuf_addbuf(&parent,enlistment);+strbuf_parent_directory(&parent);+if(chdir(parent.buf)<0)+die_errno(_("could not switch to '%s'"),parent.buf);+strbuf_release(&parent);+#endif++if(remove_dir_recursively(enlistment,0))+die(_("failed to delete enlistment directory"));++return0;+}+staticintcmd_clone(intargc,constchar**argv){constchar*branch=NULL;
@@ -688,6 +717,39 @@ static int cmd_unregister(int argc, const char **argv)returnunregister_dir();}+staticintcmd_delete(intargc,constchar**argv)+{+char*cwd=xgetcwd();+structoptionoptions[]={+OPT_END(),+};+constchar*constusage[]={+N_("scalar delete <enlistment>"),+NULL+};+structstrbufenlistment=STRBUF_INIT;+intres=0;++argc=parse_options(argc,argv,NULL,options,+usage,0);++if(argc!=1)+usage_with_options(usage,options);++setup_enlistment_directory(argc,argv,usage,options,&enlistment);++if(dir_inside_of(cwd,enlistment.buf)>=0)+res=error(_("refusing to delete current working directory"));+else{+close_object_store(the_repository->objects);+res=delete_enlistment(&enlistment);+}+strbuf_release(&enlistment);+free(cwd);++returnres;+}+staticstruct{constchar*name;int(*fn)(int,constchar**);
@@ -128,6 +129,13 @@ reconfigure the enlistment. With the `--all` option, all enlistments currently registered with Scalar will be reconfigured. Use this option after each Scalar upgrade.+Delete+~~~~~~++delete <enlistment>::+ This subcommand lets you delete an existing Scalar enlistment from your+ local file system, unregistering the repository.+ SEE ALSO -------- linkgit:git-clone[1], linkgit:git-maintenance[1].
From: Johannes Schindelin via GitGitGadget <hidden> Date: 2021-12-03 13:35:03
From: Johannes Schindelin <redacted>
The .NET version of Scalar has a `version` command. This was necessary
because it was versioned independently of Git.
Since Scalar is now tightly coupled with Git, it does not make sense for
them to show different versions. Therefore, it shows the same output as
`git version`. For backwards-compatibility with the .NET version,
`scalar version` prints to `stderr`, though (`git version` prints to
`stdout` instead).
Signed-off-by: Johannes Schindelin <redacted>
---
contrib/scalar/scalar.c | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
On Fri, Dec 3, 2021 at 5:34 AM Johannes Schindelin via GitGitGadget
[off-list ref] wrote:
tl;dr: This series contributes the core part of the Scalar command to the
Git project. This command provides a convenient way to clone/initialize very
large repositories (think: monorepos).
Note: This patch series' focus is entirely on Scalar, on choosing sensible
defaults and offering a delightful user experience around working with
monorepos, and not about changing any existing paradigms for contrib/ (even
if catching up on the mail thread is likely to give interested readers that
false impression).
Changes since v9:
* The patches to build Scalar and run its tests as part of Git's CI/PR,
have been dropped because a recent unrelated patch series does not
interact well with them.
i.e. basically undoing this:
...
Changes since v6:
...
* I added two patches that I had planned on keeping in an add-on patch
series for later, to build and test Scalar as part of the CI. I am still
not 100% certain that it is a good idea to do so already now, but let's
see what the reviewers have to say.
...and returning to the original plan:
...
On top of this patch series, I have lined up a few more:
...
4. A few patches to optionally build and install scalar as part of a
regular Git install (also teaching git help scalar to find the Scalar
documentation
Avoiding the issues and adding the CI later seems reasonable to me.
You addressed the last of my points in v9; I think this version is
good to go. But one quick comment...
These are included in my vfs-with-scalar branch thicket
[https://github.com/dscho/git/commits/vfs-with-scalar]. On top of that, this
branch thicket also includes patches I do not plan on upstreaming, mainly
because they are too specific either to VFS for Git, or they support Azure
Repos (which does not offer partial clones but speaks the GVFS protocol,
which can be used to emulate partial clones).
One other thing is very interesting about that vfs-with-scalar branch
thicket: it contains a GitHub workflow which will run Scalar's quite
extensive Functional Tests suite. This test suite is quite comprehensive and
caught us a lot of bugs in the past, not only in the Scalar code, but also
core Git.
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
From: Junio C Hamano <hidden> Date: 2021-12-05 10:02:41
Elijah Newren [off-list ref] writes:
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
Yeah, we may be initially queuing this without tests for expediency,
but a production code cannot go forever without CI tests to ensure
continued code health. People make changes in other parts of the
system Scalar may depend on and unknowingly break some assumption
Scalar makes on it.
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
Yeah, we may be initially queuing this without tests for expediency,
but a production code cannot go forever without CI tests to ensure
continued code health. People make changes in other parts of the
system Scalar may depend on and unknowingly break some assumption
Scalar makes on it.
In this case there really isn't any reason not to have the tests go in
at the same time. The explanation in the v10 CL is:
Changes since v9:
* The patches to build Scalar and run its tests as part of Git's CI/PR,
have been dropped because a recent unrelated patch series does not
interact well with them.
That assessment isn't correct.
The change in v8->v9 of adding a "make &&" before the "test" was only
necessary because of a logic error in the v8 version. Yes it broke
because the "scalar test" target didn't know how to build its
prerequisites, but the real underlying issue is that it was even trying
at that point. It had no business running in the static-analysis target
where we hadn't built git already.
Now v9->v10 has
dropped the tests entirely, allegedly due to an interaction with my
ab/ci-updates, but there's nothing new there that isn't also the case on
"master".
But we can have our cake and eat it too.
The below patch on top of v9 would make the scalar tests do the right
thing. I.e. whenever we do a "make test" we'll run the scalar tests
too.
The code changes somewhat with ab/ci-updates, but the conflict with
js/scalar is mostly textual, not semantic (and as I've pointed out, to
the extent that ab/ci-updates changed anything it made things a bit
better for js/scalar).
I'd really like to see this scalar series land, but I really don't see
why it's necessary to entirety eject the CI test coverage due to what's
a rather trivilly solved issue.
As I've noted ad-nauseum at this point I think the necessity for the
below patch is rather silly, this should just nicely integrate with
"make test", but <brokenrecord.gif>. But even without that IMO better
approach it's clearly rather trivial to make this series have test
coverage.
It was just broken because it added a test run to the "pedantic" run,
and didn't properly integrate with the multi-"make test" runs on
"master" , both of which are addressed by the patch below.
@@ -15,6 +15,26 @@ thenexportDEVOPTS=pedanticfi+make(){+scalar_tests=+fortarget+do+iftest$target="test"+then+scalar_tests=t+fi+done++# Do whatever we would have done with "make"+commandmake"$@"++# Running tests? Run scalar tests too+iftest-n"$scalar_tests"+then+commandmake-Ccontrib/scalartest+fi+}+ makecase"$jobname"in linux-gcc)
From: Johannes Schindelin <hidden> Date: 2021-12-08 11:16:00
Hi Junio,
On Sun, 5 Dec 2021, Junio C Hamano wrote:
Elijah Newren [off-list ref] writes:
quoted
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
Yeah, we may be initially queuing this without tests for expediency,
but a production code cannot go forever without CI tests to ensure
continued code health. People make changes in other parts of the
system Scalar may depend on and unknowingly break some assumption
Scalar makes on it.
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
I do not know off-hand how entangled the GVFS part is in the test suite,
but from what I recall, every single test starts with cloning a test
repository. From Azure Repos. Using the `gvfs-helper`.
Which means that the `gvfs-helper` would need to be upstreamed and be
maintained in the git.git repository proper.
Previously I was under the impression that that might be met with grumpy
rejection.
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
Ciao,
Dscho
Hi Junio,
On Sun, 5 Dec 2021, Junio C Hamano wrote:
quoted
Elijah Newren [off-list ref] writes:
quoted
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
Yeah, we may be initially queuing this without tests for expediency,
but a production code cannot go forever without CI tests to ensure
continued code health. People make changes in other parts of the
system Scalar may depend on and unknowingly break some assumption
Scalar makes on it.
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
I do not know off-hand how entangled the GVFS part is in the test suite,
but from what I recall, every single test starts with cloning a test
repository. From Azure Repos. Using the `gvfs-helper`.
Which means that the `gvfs-helper` would need to be upstreamed and be
maintained in the git.git repository proper.
Previously I was under the impression that that might be met with grumpy
rejection.
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
An alternate way would be be to have our own tests build git, and then
clone and build those third party repos and test them.
I had a patch to do that for git-annex. I think it would be a good idea
to pursue it in general for prominent downstream projects as part of
some extended integration testing:
https://lore.kernel.org/git/20170516203712.15921-1-avarab@gmail.com/
Hi Junio,
On Sun, 5 Dec 2021, Junio C Hamano wrote:
quoted
Elijah Newren [off-list ref] writes:
quoted
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
Moving the C# test suite over doesn't make a lot of sense. We also
are re-using the test suite from VFS for Git, which is probably overkill
here. Those tests were created due to issues that arose with the virtual
filesystem (paired with the GVFS protocol for finding missing objects)
and most of them probably don't test anything interesting in Scalar.
When we _do_ find something interesting in that suite, we port over the
test as a normal Git test so the regression is avoided in the future.
We work to test the -rc0 version of every release with our custom patches
in microsoft/git and then run them through the Scalar and VFS for Git
functional tests as a necessary step before releasing to our internal
users. Since we are doing that already, it is a better use of time to
port tests that actually matter when they come up rather than port the
entire test suite.
quoted
Yeah, we may be initially queuing this without tests for expediency,
but a production code cannot go forever without CI tests to ensure
continued code health. People make changes in other parts of the
system Scalar may depend on and unknowingly break some assumption
Scalar makes on it.
I think it is important to keep in mind that the Scalar features that
are being submitted here are getting Git-style tests included. The only
thing that is missing right now is a firm link with Git's CI system,
which can be added quickly once things have calmed down in the build
system.
If we are interested in doing something more substantial that is
closer to the Scalar functional tests, then it is important to know
that those tests are running against a production server to clone
data and fetch it dynamically throughout. That is not exactly something
we have done in the Git test suite before.
In fact, I don't think Scalar introduces anything novel here: if we
want to add more coverage of running Git commands while in a
sparse-checkout _and_ partial clone _and_ have a lot of optional config
set, then we can do that independently of Scalar. 'scalar clone' just
sets up a repository in a state that an expert user could do themselves,
so should we spend a lot of effort creating that environment in our
test suite?
We have this already in some form:
1. t1091 and t1092 try to cover important sparse-checkout behavior.
2. t0410, t5616, and others try to cover important partial clone
behavior.
3. Our GIT_TEST_* variables that are enabled in one of our CI runs
test many of the advanced config options enabled by Scalar.
The thing that is missing is "all of these things at once" which
would be difficult to do across the test suite with our current test
design. I'm happy to provide the service of checking the Scalar
functional tests before each release as an expensive way to check
that combination of configuration without adding that cost to every
CI run and developer inner loop.
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
I do not know off-hand how entangled the GVFS part is in the test suite,
but from what I recall, every single test starts with cloning a test
repository. From Azure Repos. Using the `gvfs-helper`.
Which means that the `gvfs-helper` would need to be upstreamed and be
maintained in the git.git repository proper.
Previously I was under the impression that that might be met with grumpy
rejection.
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
I, for one, don't think that has much value for the core Git project.
Thanks,
-Stolee
I know this was directed to Junio, but I feel like it was my earlier
comment that accidentally opened this can of worms, so if my opinion
helps resolve it at all...
On Wed, Dec 8, 2021 at 3:15 AM Johannes Schindelin
[off-list ref] wrote:
Hi Junio,
On Sun, 5 Dec 2021, Junio C Hamano wrote:
quoted
Elijah Newren [off-list ref] writes:
quoted
From your wording it sounds like the plan might not include moving
these tests over. Perhaps it doesn't make sense to move them all
over, but since they've caught problems in both Scalar and core Git,
it would be nice to see many of those tests come to Git as well as
part of a future follow on series.
Yeah, we may be initially queuing this without tests for expediency,
but a production code cannot go forever without CI tests to ensure
continued code health. People make changes in other parts of the
system Scalar may depend on and unknowingly break some assumption
Scalar makes on it.
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
I do not know off-hand how entangled the GVFS part is in the test suite,
but from what I recall, every single test starts with cloning a test
repository. From Azure Repos. Using the `gvfs-helper`.
Which means that the `gvfs-helper` would need to be upstreamed and be
maintained in the git.git repository proper.
Ah, sorry, I was remembering this from an earlier cover letter of yours:
"""
But it was realized that many of these key concepts were independent of the
actual VFS and its projection of the working directory. The Scalar project
was created to make that separation, refine the key concepts, and then
extract those features into the new Scalar command.
"""
when I read
"""
One other thing is very interesting about that vfs-with-scalar branch
thicket: it contains a GitHub workflow which will run Scalar's quite
extensive Functional Tests suite. This test suite is quite comprehensive and
caught us a lot of bugs in the past, not only in the Scalar code, but also
core Git.
"""
and I was thinking (despite the branch name) that you had some scalar
+ git (w/o gvfs) tests that were interesting but not planning to
upstream. I agree that if they're gvfs + scalar + git then they make
sense to keep internal to your work, though I hope that for any bugs
your internal testcases find in git, that you find an upstreamable
testcase to submit. I believe Stolee has done exactly that in the
past, so just more of that would be good.
Previously I was under the impression that that might be met with grumpy
rejection.
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
I know I'm not Junio, but if my opinion matters, I don't think that
needs to be part of the plan.
From: Junio C Hamano <hidden> Date: 2021-12-09 03:52:41
Johannes Schindelin [off-list ref] writes:
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
...
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
Sorry, I do not follow.
What I was lamenting about was the lack of CI test coverage of stuff
that is already being considered to go 'next'. Specifically, since
contrib/scalar/Makefile in 'seen' has a 'test' target, it would be a
shame not to exercise it, when we should be able to do so in the CI
fairly easily.
I fail to see what gvfs-helper has to do with anything in the
context of advancing the js/scalar topic as we have today. If "The
Scalar Functional Tests" that were designed with Azure Repos in mind
is not a good fit to come into contrib/scalar/, it is fine not to
have it here---lack of it would not make the test target you have in
contrib/scalar/Makefile any less valuable, I would think.
Unless you are saying that "make -C contrib/scalar test" is useless,
that is. But I do not think that is the case.
From: Johannes Schindelin <hidden> Date: 2021-12-11 00:29:55
Hi Junio,
On Wed, 8 Dec 2021, Junio C Hamano wrote:
Johannes Schindelin [off-list ref] writes:
quoted
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
...
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
Sorry, I do not follow.
In
https://lore.kernel.org/git/CABPp-BGpe9Q5k22Yu8a=1xwu=pZYSeNQoqEgf+DN07cU4EB1ew@mail.gmail.com/
(i.e. in the great great grand parent of this mail), you specifically
replied to my mentioning Scalar's Functional Test suite:
> > One other thing is very interesting about that vfs-with-scalar
> > branch thicket: it contains a GitHub workflow which will run
> > Scalar's quite extensive Functional Tests suite. This test
> > suite is quite comprehensive and caught us a lot of bugs in
> > the past, not only in the Scalar code, but also core Git.
>
> From your wording it sounds like the plan might not include
> moving these tests over. Perhaps it doesn't make sense to move
> them all over, but since they've caught problems in both Scalar
> and core Git, it would be nice to see many of those tests come
> to Git as well as part of a future follow on series.
I had mentioned a couple of times that I had no intention to move Scalar's
Function Tests into contrib/scalar/, and your wording "it would be nice to
see many of those tests come to Git as well" made it sound as if you
disagreed with that intention.
But it was not a clear "please do port them over" nor a "nah, we don't
want that test suite implemented in C# and requiring, for the most part,
access to a dedicacted Azure Repo".
Hence I was asking for a clear answer to the question whether you want me
to spend time on preparing a patch series to contribute Scalar's
Functional Tests to contrib/scalar/ as well.
I _suspect_ your clear answer, if you are willing to give it as clearly,
to be "no, we do not do integration tests here, and besides, C# is not a
language we want to add to Git's tree".
What I was lamenting about was the lack of CI test coverage of stuff
that is already being considered to go 'next'. Specifically, since
contrib/scalar/Makefile in 'seen' has a 'test' target, it would be a
shame not to exercise it, when we should be able to do so in the CI
fairly easily.
We do have a very different understanding of "fairly easily" in that case.
Three iterations, and three weeks time spent on implementing what you
suggest, only to see broken by the merge of the `ab/ci-updates` patch
series, suggesting a fixup for the incorrect merge, seeing that fixup
rejected, and then more discussing, all of that does not strike me as
"fairly easily". It strikes me as "a lot of time and effort was spent,
mostly stepping on toes".
Granted, if `ab/ci-updates` would not have happened, it would have been
much easier. Or if `ab/ci-updates` had waited until `js/scalar` advanced
to `next`. But the way it happened was (unnecessarily?) un-easy.
I fail to see what gvfs-helper has to do with anything in the
context of advancing the js/scalar topic as we have today.
Okay, okay! I was just asking about gvfs-helper because that would be
required to port over Scalar's Functional Tests. The same Functional Tests
that I heard you mentioning would be "nice to see" to "come to Git as
well".
If "The Scalar Functional Tests" that were designed with Azure Repos in
mind is not a good fit to come into contrib/scalar/, it is fine not to
have it here---lack of it would not make the test target you have in
contrib/scalar/Makefile any less valuable, I would think.
The test target won't go anywhere, no worries. Just like the test target
in contrib/subtree/ does not go anywhere.
And just like `contrib/subtree/`, it does not have to be run as part of
Git's CI build.
Unless you are saying that "make -C contrib/scalar test" is useless,
that is. But I do not think that is the case.
It is as useful as `make -C contrib/subtree test`. Which, as Ævar will
readily offer, is broken, because it does not ensure that top-level `make
all` is executed and therefore in a fresh checkout will fail.
Of course, I disagree that it is "broken". It works as designed. It is in
the contrib/ part of the tree, i.e. safely in the realm of "you have to
build Git first, and then the thing in contrib/". In other words, the idea
to "fix" this kind of "broken"ness is a solution in search of a problem.
And as I have said multiple times, I still think that having Scalar's code
in contrib/ is a good spot to experiment with it. It sends the right
signal of "this is not really something we promise to maintain just yet".
It is a logical place for code that developers can build themselves, but
that is not built and installed with Git by default.
Having it in the Git tree will give interested developers a chance who
want to clone a large repository on Linux, without having to touch
anything with "Microsoft" in its repository name.
Having it in the Git tree will give interested developers a chance to
experiment with things like "let's try to let `scalar clone` _not_
clone into `<enlistment>/src/`, but instead create a bare clone in
`<enlistment>/.git` and make `<enlistment>/src/` a worktree". Things like
that.
I would find those things quite a bit more useful than to force regular
Git contributors who want to change libgit.a (even if it is just pointless
refactoring) to pay attention to contrib/scalar/ in CI, when there is
still no clear answer whether Scalar will even become a first-class Git
command eventually (which I hope it will, of course).
Ciao,
Dscho
Hi Junio,
[...]
We do have a very different understanding of "fairly easily" in that case.
Three iterations, and three weeks time spent on implementing what you
suggest, only to see broken by the merge of the `ab/ci-updates` patch
series, suggesting a fixup for the incorrect merge, seeing that fixup
rejected, and then more discussing, all of that does not strike me as
"fairly easily". It strikes me as "a lot of time and effort was spent,
mostly stepping on toes".
I sent you a working path to a fixup in [1] on the 23rd of November
where we won't go from running zero tests in compile-only to running
just the scalar test.
Junio replied[2] ("the above" referring to [1]):
I think the above shows that it is a bug in the topic itself,
You didn't reply further in that fixup thread, and then your v9 re-roll
a week later still had the same issue[3] discussed therein. I again
pointed that out[4]:
Is it intentional that the previously compile-only "pedantic" job is now
running the scalar tests?
You didn't reply, but in your v10 decided to make the current iteration
of this series have no CI testing at all, and cited the interaction with
ab/ci-updates[4]:
because a recent unrelated patch series does not interact well with them.
Which I think is clearly inaccurate, because...
Granted, if `ab/ci-updates` would not have happened, it would have been
much easier. Or if `ab/ci-updates` had waited until `js/scalar` advanced
to `next`. But the way it happened was (unnecessarily?) un-easy.
...your initial patch to run the scalar tests in CI[5] was part of v7, and
had the issue described above. It pre-dates the v1 of ab/ci-updates
being on-list by a couple of days[6].
So yes, I do think it was "easy", as in that was an easy fix-up. You
just didn't follow up on it and submitted re-rolls with the already
noted breakage.
I don't blame you for that, maybe you were busy, it slipped through
etc.
But I don't accept that delays in this topic are my fault, or something
to the effect that that this whole saga represents some failure of the
review process.
Our topics textually/semantically conflicted, it happens. I offered a
fixup & way forward. Fixing it was trivial, and still is. You just
didn't follow-up.
[...]
quoted
If "The Scalar Functional Tests" that were designed with Azure Repos in
mind is not a good fit to come into contrib/scalar/, it is fine not to
have it here---lack of it would not make the test target you have in
contrib/scalar/Makefile any less valuable, I would think.
The test target won't go anywhere, no worries. Just like the test target
in contrib/subtree/ does not go anywhere.
And just like `contrib/subtree/`, it does not have to be run as part of
Git's CI build.
But unlike contrib/completion, which we do run as part of Git's CI
build[7]?
quoted
Unless you are saying that "make -C contrib/scalar test" is useless,
that is. But I do not think that is the case.
It is as useful as `make -C contrib/subtree test`. Which, as Ævar will
readily offer, is broken, because it does not ensure that top-level `make
all` is executed and therefore in a fresh checkout will fail.
Before the scalar topic there was only one "make" entry point to build
libgit.a, contrib/scalar/Makefile makes that two. That was the immediate
prompt for the fixup discussion in [1].
So no, I won't offer that "make -C contrib/subtree test" is broken, it
doesn't try to build libgit.a and errors out right away if git isn't
built.
Your scalar patches do try, get most of the way there, and fail.
Your bicycle isn't broken if it doesn't make coffee, but if your fridge
has a built-in coffee maker and it doesn't work it's broken, at least as
it pertains to its coffee making function.
I think I made that distinction clear in [8], but apparently not clear
enough, as you seem to be under the impression that I was conveying the
opposite of the idea I was trying to get across.
Of course, I disagree that it is "broken". It works as designed. It is in
the contrib/ part of the tree, i.e. safely in the realm of "you have to
build Git first, and then the thing in contrib/". In other words, the idea
to "fix" this kind of "broken"ness is a solution in search of a problem.
I agree with that, but it's your proposed patches that contain the build
integration you're describing as unnecessary for "contrib/subtree/". In
v8->v8 of the series you changed the CI integration from:
make -C contrib/scalar test
To:
make && make -C contrib/scalar test
While keeping the bits in contrib/scalar/Makefile that made it go most
of the way towards a working "libgit.a" useful for testing, but it
breaks before we get everything we need to run the "test" target.
Which I find to be odd given the above comparison to contib/subtree/. If
you have to build git first at the top level why is it trying and
failing to build git? "contrib/subtree" doesn't.
[...]
I would find those things quite a bit more useful than to force regular
Git contributors who want to change libgit.a (even if it is just pointless
refactoring) to pay attention to contrib/scalar/ in CI, when there is
still no clear answer whether Scalar will even become a first-class Git
command eventually (which I hope it will, of course).
Hi Dscho,
On Fri, Dec 10, 2021 at 4:29 PM Johannes Schindelin
[off-list ref] wrote:
Hi Junio,
On Wed, 8 Dec 2021, Junio C Hamano wrote:
quoted
Johannes Schindelin [off-list ref] writes:
quoted
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
...
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
Sorry, I do not follow.
In
https://lore.kernel.org/git/CABPp-BGpe9Q5k22Yu8a=1xwu=pZYSeNQoqEgf+DN07cU4EB1ew@mail.gmail.com/
(i.e. in the great great grand parent of this mail), you specifically
replied to my mentioning Scalar's Functional Test suite:
> > One other thing is very interesting about that vfs-with-scalar
> > branch thicket: it contains a GitHub workflow which will run
> > Scalar's quite extensive Functional Tests suite. This test
> > suite is quite comprehensive and caught us a lot of bugs in
> > the past, not only in the Scalar code, but also core Git.
>
> From your wording it sounds like the plan might not include
> moving these tests over. Perhaps it doesn't make sense to move
> them all over, but since they've caught problems in both Scalar
> and core Git, it would be nice to see many of those tests come
> to Git as well as part of a future follow on series.
This is me and my email you are quoting; these aren't Junio's words.
I'm afraid my confusion may have snowballed for others here. Sorry
about that.
I simply misunderstood at the time -- I thought there were scalar-only
tests (rather than scalar+gvfs tests) that were not being considered
for upstreaming. As I mentioned before[1], I'm sorry for the
confusion and seemingly opening an unrelated can of worms. I agree
that we don't need gvfs tests, or tests that combine gvfs with other
things like scalar, or c# tests.
[1] https://lore.kernel.org/git/CABPp-BFmNiqY=NfN7Ys3XE8wYBn1EQ_War+0QLq96Tk7FO6zfg@mail.gmail.com/
From: Johannes Schindelin <hidden> Date: 2021-12-11 13:46:15
Hi Elijah,
On Fri, 10 Dec 2021, Elijah Newren wrote:
On Fri, Dec 10, 2021 at 4:29 PM Johannes Schindelin
[off-list ref] wrote:
quoted
On Wed, 8 Dec 2021, Junio C Hamano wrote:
quoted
Johannes Schindelin [off-list ref] writes:
quoted
The Scalar Functional Tests were designed with Azure Repos in mind, i.e.
they specifically verify that the `gvfs-helper` (emulating Partial Clone
using the predecessor of Partial Clone, the GVFS protocol) manages to
access the repositories in the intended way.
...
I do realize, though, that clarity of intention has been missing from this
mail thread all around, so let me ask point blank: Junio, do you want me
to include upstreaming `gvfs-helper` in the overall Scalar plan?
Sorry, I do not follow.
In
https://lore.kernel.org/git/CABPp-BGpe9Q5k22Yu8a=1xwu=pZYSeNQoqEgf+DN07cU4EB1ew@mail.gmail.com/
(i.e. in the great great grand parent of this mail), you specifically
replied to my mentioning Scalar's Functional Test suite:
> > One other thing is very interesting about that vfs-with-scalar
> > branch thicket: it contains a GitHub workflow which will run
> > Scalar's quite extensive Functional Tests suite. This test
> > suite is quite comprehensive and caught us a lot of bugs in
> > the past, not only in the Scalar code, but also core Git.
>
> From your wording it sounds like the plan might not include
> moving these tests over. Perhaps it doesn't make sense to move
> them all over, but since they've caught problems in both Scalar
> and core Git, it would be nice to see many of those tests come
> to Git as well as part of a future follow on series.
This is me and my email you are quoting; these aren't Junio's words.
I'm afraid my confusion may have snowballed for others here. Sorry
about that.
I simply misunderstood at the time -- I thought there were scalar-only
tests (rather than scalar+gvfs tests) that were not being considered
for upstreaming. As I mentioned before[1], I'm sorry for the
confusion and seemingly opening an unrelated can of worms. I agree
that we don't need gvfs tests, or tests that combine gvfs with other
things like scalar, or c# tests.
[1] https://lore.kernel.org/git/CABPp-BFmNiqY=NfN7Ys3XE8wYBn1EQ_War+0QLq96Tk7FO6zfg@mail.gmail.com/
No worries, I am glad it is sorted out now.
Ciao,
Dscho