Re: [RFC] Introducing different handling for small/large transactions

Stefan Beller [off-list ref] writes:

In ref_transaction_commit
	* commit the .lock file to its destination
	* in case this is a deletion:
		* remove the loose ref
		* and repack the packed refs file if necessary

Don't you need to repack and then remove the loose one, though?
Otherwise you would expose a stale packed ref in the middle to the
other readers, no?

The larger transactions would be handled differently by relying
on the packed refs file:
In ref_transaction_update:
	* detect if we transition to a large transaction
	  (by having more than one entry in transaction->updates)
	  if so:
		* Pack all currently existing refs into the packed
		  refs file, commit the packed refs file and delete
		  all loose refs. This will avoid (d/f) conflicts.

		* Keep the packed-refs file locked and move the first
		  transaction update into the packed-refs.lock file

	* Any update(delete, create, update) is put into the locked
	  packed refs file.

I am not sure if you mean (a) keep updates only in-core, to be
flushed at the commit time, or (b) each and every update in the
large transaction results in rewriting the entire packed-refs.lock
file, only to be renamed to the final name at the commit time.
I am hoping it would be the former.

	* Additionally we need to obtain the .lock for the loose refs
	  file to keep guarantees, though we should close the file
	  descriptor as we don't wand to run out of file descriptors.

Yes, this last point is important.

On Thu, Jan 15, 2015 at 3:34 PM, Junio C Hamano [off-list ref] wrote:

Stefan Beller [off-list ref] writes:

quoted

In ref_transaction_commit
      * commit the .lock file to its destination
      * in case this is a deletion:
              * remove the loose ref
              * and repack the packed refs file if necessary

Don't you need to repack and then remove the loose one, though?
Otherwise you would expose a stale packed ref in the middle to the
other readers, no?

You're right.

quoted

The larger transactions would be handled differently by relying
on the packed refs file:
In ref_transaction_update:
      * detect if we transition to a large transaction
        (by having more than one entry in transaction->updates)
        if so:
              * Pack all currently existing refs into the packed
                refs file, commit the packed refs file and delete
                all loose refs. This will avoid (d/f) conflicts.

              * Keep the packed-refs file locked and move the first
                transaction update into the packed-refs.lock file

      * Any update(delete, create, update) is put into the locked
        packed refs file.

I am not sure if you mean (a) keep updates only in-core, to be
flushed at the commit time, or (b) each and every update in the
large transaction results in rewriting the entire packed-refs.lock
file, only to be renamed to the final name at the commit time.
I am hoping it would be the former.

I wasn't sure which I mean. The first one is obviously better to do.

quoted

      * Additionally we need to obtain the .lock for the loose refs
        file to keep guarantees, though we should close the file
        descriptor as we don't wand to run out of file descriptors.

Yes, this last point is important.