[PATCH v3 1/4] odb: use size_t for the length of objects to write
flat view
HOTtoday
From: Elia Pinto <hidden>
Date: 2026-10-31 10:16:43
Subsystem:
the rest · Maintainer:
Linus Torvalds
Revision v3 (3 revisions archived).
Revisions (3)
- v1
- v2 [diff against previous]
- v3 current [diff against previous]
c6a4629e32 (odb: use size_t for object_info.sizep and the size APIs, 2026-06-15) widened the read side of the object database API, and the write_object() callback of odb_source already takes a size_t. But odb_write_object_ext(), odb_write_object() and odb_source_write_object() in front of it, as well as write_loose_object() behind it, still take an "unsigned long", which is only 32 bits wide on Windows. Most callers hand them a size_t from a strbuf or from odb_read_object(), which silently gets truncated. The most visible case is index_mem() and index_stream_convert_blob(): when "git add" or "git hash-object -w" runs a file over 4GiB through a conversion (core.autocrlf, which is on by default in Git for Windows, an eol or ident attribute, or a clean filter), we write a blob made of the first "size mod 2^32" bytes. The index records it with the stat data of the real file, so "git status" stays clean and nobody notices until the file is checked out again. "git hash-object" without "-w" uses hash_object_file(), which takes a size_t, so the two disagree on the object ID of the same file. Use size_t throughout, including for the size kept by the in-memory source. The existing "hash filtered files over 4GB correctly" test in t1007 only hashes, so add a variant that writes the object with "git hash-object -w" and checks its object ID and that it exists. Like the other tests over 4GB, it needs EXPENSIVE and SIZE_T_IS_64BIT. Truncating the length to 32 bits in odb_write_object_ext(), as happens on Windows without this patch, makes it fail with a different object ID. Signed-off-by: Elia Pinto <redacted> --- odb.c | 2 +- odb.h | 4 ++-- odb/source-inmemory.c | 2 +- odb/source-loose.c | 2 +- odb/source.h | 2 +- t/t1007-hash-object.sh | 11 +++++++++++ 6 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/odb.c b/odb.c
index 2c93a9490c..1f173625b7 100644
--- a/odb.c
+++ b/odb.c@@ -640,7 +640,7 @@ void odb_assert_oid_type(struct object_database *odb, } int odb_write_object_ext(struct object_database *odb, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, struct object_id *oid, const struct object_id *compat_oid_in,
diff --git a/odb.h b/odb.h
index dd85b880a7..45182d28cb 100644
--- a/odb.h
+++ b/odb.h@@ -621,14 +621,14 @@ enum odb_write_object_flags { * Returns 0 on success, a negative error code otherwise. */ int odb_write_object_ext(struct object_database *odb, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, struct object_id *oid, const struct object_id *compat_oid, enum odb_write_object_flags flags); static inline int odb_write_object(struct object_database *odb, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, struct object_id *oid) {
diff --git a/odb/source-inmemory.c b/odb/source-inmemory.c
index 75cf29fa9d..c13cffc805 100644
--- a/odb/source-inmemory.c
+++ b/odb/source-inmemory.c@@ -10,7 +10,7 @@ struct inmemory_object { enum object_type type; const void *buf; - unsigned long size; + size_t size; }; static const struct inmemory_object *find_cached_object(struct odb_source_inmemory *source,
diff --git a/odb/source-loose.c b/odb/source-loose.c
index 50cb388c35..60801e70e8 100644
--- a/odb/source-loose.c
+++ b/odb/source-loose.c@@ -758,7 +758,7 @@ static int end_loose_object_common(struct odb_source_loose *loose, static int write_loose_object(struct odb_source_loose *loose, const struct object_id *oid, char *hdr, - int hdrlen, const void *buf, unsigned long len, + int hdrlen, const void *buf, size_t len, const time_t *mtime, unsigned flags) { int fd, ret;
diff --git a/odb/source.h b/odb/source.h
index 8b839caa39..07d5019aa5 100644
--- a/odb/source.h
+++ b/odb/source.h@@ -451,7 +451,7 @@ static inline int odb_source_freshen_object(struct odb_source *source, * object ID and the compatibility object ID, if non-NULL. */ static inline int odb_source_write_object(struct odb_source *source, - const void *buf, unsigned long len, + const void *buf, size_t len, enum object_type type, const struct object_id *oid, const struct object_id *compat_oid,
diff --git a/t/t1007-hash-object.sh b/t/t1007-hash-object.sh
index 463b38f990..e76557981d 100755
--- a/t/t1007-hash-object.sh
+++ b/t/t1007-hash-object.sh@@ -297,4 +297,15 @@ test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \ test_cmp expect actual ' +test_expect_success EXPENSIVE,SIZE_T_IS_64BIT \ + 'write filtered files over 4GB correctly' ' + { test -f big || test-tool genzeros $((5*1024*1024*1024)) >big; } && + test_oid large5GB >expect && + test_config filter.null-filter.clean "cat" && + echo "big filter=null-filter" >.gitattributes && + git hash-object -w -- big >actual && + test_cmp expect actual && + git cat-file -e "$(cat actual)" +' + test_done
--
2.53.0