[JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

STALE3729d

8 messages, 5 authors, 2016-06-15 · open the first message on its own page

[JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Nigel Magnay <hidden>
Date: 2016-06-15 22:46:12

JGit is used as a library in external projects such as build tools.
Some of the representations of git data structures are useful
in these external tools - but - it is often desirable to be able to
either persist these objects, or serialize them across the wire.

Apply Externalizable to URIish and RefSpec, and Serializable to
ObjectId and RemoteConfig (in order to avoid the undesirable
requirement of a public, no-args constructor needed for Externalizable).

 Signed-off-by: Nigel Magnay [off-list ref]

---
 .../src/org/spearce/jgit/lib/ObjectId.java         |   22 +++++-
 .../src/org/spearce/jgit/transport/RefSpec.java    |   77 +++++++++++++------
 .../org/spearce/jgit/transport/RemoteConfig.java   |   79 +++++++++++++++++++-
 .../src/org/spearce/jgit/transport/URIish.java     |   29 +++++++-
 4 files changed, 179 insertions(+), 28 deletions(-)
diff --git a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
index 52ce0d4..7c3b922 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
@@ -38,6 +38,10 @@

 package org.spearce.jgit.lib;

+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+import java.io.Serializable;
 import java.io.UnsupportedEncodingException;

 import org.spearce.jgit.util.NB;
@@ -45,7 +49,7 @@
 /**
  * A SHA-1 abstraction.
  */
-public class ObjectId extends AnyObjectId {
+public class ObjectId extends AnyObjectId implements Serializable {
 	private static final ObjectId ZEROID;

 	private static final String ZEROID_STR;
@@ -269,4 +273,20 @@ protected ObjectId(final AnyObjectId src) {
 	public ObjectId toObjectId() {
 		return this;
 	}
+
+	private void writeObject(ObjectOutputStream os)  throws IOException {
+		os.writeInt(w1);
+		os.writeInt(w2);
+		os.writeInt(w3);
+		os.writeInt(w4);
+		os.writeInt(w5);
+	}
+	
+	private void readObject(ObjectInputStream ois)  throws IOException {
+		w1 = ois.readInt();
+		w2 = ois.readInt();
+		w3 = ois.readInt();
+		w4 = ois.readInt();
+		w5 = ois.readInt();
+	}
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
index 521110b..0ee89b0 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/RefSpec.java
@@ -37,6 +37,11 @@

 package org.spearce.jgit.transport;

+import java.io.Externalizable;
+import java.io.IOException;
+import java.io.ObjectInput;
+import java.io.ObjectOutput;
+
 import org.spearce.jgit.lib.Constants;
 import org.spearce.jgit.lib.Ref;
@@ -46,7 +51,7 @@
  * A ref specification provides matching support and limited rules to rewrite a
  * reference in one repository to another reference in another repository.
  */
-public class RefSpec {
+public class RefSpec implements Externalizable {
 	/**
 	 * Suffix for wildcard ref spec component, that indicate matching all refs
 	 * with specified prefix.
@@ -109,30 +114,7 @@ public RefSpec() {
 	 *             the specification is invalid.
 	 */
 	public RefSpec(final String spec) {
-		String s = spec;
-		if (s.startsWith("+")) {
-			force = true;
-			s = s.substring(1);
-		}
-
-		final int c = s.indexOf(':');
-		if (c == 0) {
-			s = s.substring(1);
-			if (isWildcard(s))
-				throw new IllegalArgumentException("Invalid wildcards " + spec);
-			dstName = s;
-		} else if (c > 0) {
-			srcName = s.substring(0, c);
-			dstName = s.substring(c + 1);
-			if (isWildcard(srcName) && isWildcard(dstName))
-				wildcard = true;
-			else if (isWildcard(srcName) || isWildcard(dstName))
-				throw new IllegalArgumentException("Invalid wildcards " + spec);
-		} else {
-			if (isWildcard(s))
-				throw new IllegalArgumentException("Invalid wildcards " + spec);
-			srcName = s;
-		}
+		initializeFromString(spec);
 	}

 	/**
@@ -161,6 +143,42 @@ private RefSpec(final RefSpec p) {
 	}

 	/**
+	 * Initialize the ref specification from a string.
+	 *
+	 * @param spec
+	 *            string describing the specification.
+	 * @throws IllegalArgumentException
+	 *             the specification is invalid.
+	 */
+	private void initializeFromString(final String spec) {
+		srcName = null;
+		String s = spec;
+		if (s.startsWith("+")) {
+			force = true;
+			s = s.substring(1);
+		}
+
+		final int c = s.indexOf(':');
+		if (c == 0) {
+			s = s.substring(1);
+			if (isWildcard(s))
+				throw new IllegalArgumentException("Invalid wildcards " + spec);
+			dstName = s;
+		} else if (c > 0) {
+			srcName = s.substring(0, c);
+			dstName = s.substring(c + 1);
+			if (isWildcard(srcName) && isWildcard(dstName))
+				wildcard = true;
+			else if (isWildcard(srcName) || isWildcard(dstName))
+				throw new IllegalArgumentException("Invalid wildcards " + spec);
+		} else {
+			if (isWildcard(s))
+				throw new IllegalArgumentException("Invalid wildcards " + spec);
+			srcName = s;
+		}
+	}
+	
+	/**
 	 * Check if this specification wants to forcefully update the destination.
 	 *
 	 * @return true if this specification asks for updates without merge tests.
@@ -421,4 +439,13 @@ public String toString() {
 		}
 		return r.toString();
 	}
+
+	public void readExternal(ObjectInput in) throws IOException,
+			ClassNotFoundException {
+		initializeFromString(in.readUTF());	
+	}
+
+	public void writeExternal(ObjectOutput out) throws IOException {
+		out.writeUTF(toString());
+	}
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
index 5bbf664..899f73f 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
@@ -38,6 +38,10 @@

 package org.spearce.jgit.transport;

+import java.io.IOException;
+import java.io.ObjectInputStream;
+import java.io.ObjectOutputStream;
+import java.io.Serializable;
 import java.net.URISyntaxException;
 import java.util.ArrayList;
 import java.util.Collections;
@@ -53,7 +57,7 @@
  * describing how refs should be transferred between this repository and the
  * remote repository.
  */
-public class RemoteConfig {
+public class RemoteConfig implements Serializable {
 	private static final String SECTION = "remote";

 	private static final String KEY_URL = "url";
@@ -382,4 +386,77 @@ public TagOpt getTagOpt() {
 	public void setTagOpt(final TagOpt option) {
 		tagopt = option != null ? option : TagOpt.AUTO_FOLLOW;
 	}
+	
+	private void writeObject(ObjectOutputStream os) throws IOException {
+
+	    // Name
+	    os.writeUTF(name);
+	
+	    // Key, Value pairs
+	    for (URIish uri : uris) {
+            os.writeUTF(KEY_URL);
+            os.writeUTF(uri.toPrivateString());
+        }
+
+        for (RefSpec refspec : fetch) {
+            os.writeUTF(KEY_FETCH);
+            os.writeUTF(refspec.toString());
+        }
+
+        for (RefSpec refspec : push) {
+        	os.writeUTF(KEY_PUSH);
+            os.writeUTF(refspec.toString());
+        }
+
+        os.writeUTF(KEY_UPLOADPACK);
+        os.writeUTF(uploadpack);
+
+        os.writeUTF(KEY_RECEIVEPACK);
+        os.writeUTF(receivepack);
+
+        os.writeUTF(KEY_TAGOPT);
+        os.writeUTF(tagopt.option());
+
+        // End marker
+        os.writeUTF("");
+    }
+
+    private void readObject(ObjectInputStream ois) throws IOException {
+		uris = new ArrayList<URIish>();
+        fetch = new ArrayList<RefSpec>();
+        push = new ArrayList<RefSpec>();
+        uploadpack = DEFAULT_UPLOAD_PACK;
+        receivepack = DEFAULT_RECEIVE_PACK;
+
+        name = ois.readUTF();
+
+		for (String key = ois.readUTF(); key.length() > 0; key = ois.readUTF()) {
+			String value = ois.readUTF();
+
+			if (key.equals(KEY_URL)) {
+				try {
+					uris.add(new URIish(value));
+				} catch (URISyntaxException e) {
+					throw new IOException("Invalid URI in RemoteConfig : "
+							+ value);
+				}
+			} else if (key.equals(KEY_FETCH)) {
+				fetch.add(new RefSpec(value));
+
+			} else if (key.equals(KEY_PUSH)) {
+				push.add(new RefSpec(value));
+
+			} else if (key.equals(KEY_UPLOADPACK)) {
+				uploadpack = value;
+
+			} else if (key.equals(KEY_RECEIVEPACK)) {
+				receivepack = value;
+
+			} else if (key.equals(KEY_TAGOPT)) {
+				tagopt = TagOpt.fromOption(value);
+			}
+
+		}
+
+    }
 }
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
index b86e00c..6b85f45 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/URIish.java
@@ -38,6 +38,10 @@

 package org.spearce.jgit.transport;

+import java.io.Externalizable;
+import java.io.IOException;
+import java.io.ObjectInput;
+import java.io.ObjectOutput;
 import java.net.URISyntaxException;
 import java.net.URL;
 import java.util.regex.Matcher;
@@ -49,7 +53,7 @@
  * RFC 2396 URI's is that no URI encoding/decoding ever takes place. A space or
  * any special character is written as-is.
  */
-public class URIish {
+public class URIish implements Externalizable {
 	private static final Pattern FULL_URI = Pattern
 			.compile("^(?:([a-z][a-z0-9+-]+)://(?:([^/]+?)(?::([^/]+?))?@)?(?:([^/]+?))?(?::(\\d+))?)?((?:[A-Za-z]:)?/.+)$");
@@ -75,6 +79,16 @@
 	 * @throws URISyntaxException
 	 */
 	public URIish(String s) throws URISyntaxException {
+		initializeFromString(s);
+	}
+	
+	/**
+	 * Set fields from string based URI.
+	 *
+	 * @param s
+	 * @throws URISyntaxException
+	 */
+	private void initializeFromString(String s)  throws URISyntaxException {
 		s = s.replace('\\', '/');
 		Matcher matcher = FULL_URI.matcher(s);
 		if (matcher.matches()) {
@@ -357,4 +371,17 @@ private String format(final boolean includePassword) {

 		return r.toString();
 	}
+
+	public void readExternal(ObjectInput in) throws IOException,
+			ClassNotFoundException {
+	    try {
+			initializeFromString(in.readUTF());
+		} catch (URISyntaxException e) {
+			throw new IOException("Incorrect format URI");
+		}
+	}
+
+	public void writeExternal(ObjectOutput out) throws IOException {
+		out.writeUTF(format(true));
+	}
 }
-- 
1.6.0.2

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Johannes Schindelin <hidden>
Date: 2016-06-15 22:46:12

Hi,

On Mon, 16 Feb 2009, Nigel Magnay wrote:
JGit is used as a library in external projects such as build tools.
Some of the representations of git data structures are useful
in these external tools - but - it is often desirable to be able to
either persist these objects, or serialize them across the wire.
I am still missing the description of the format.
 Signed-off-by: Nigel Magnay [off-list ref]
And I wonder what that leading space is all about.

Ciao,
Dscho

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Shawn O. Pearce <hidden>
Date: 2016-06-15 22:46:14

Nigel Magnay [off-list ref] wrote:
+public class ObjectId extends AnyObjectId implements Serializable {
We should define our own serialVersionUID:

  private static final long serialVersionUID = 1L;

is good enough to make Java happy.
quoted hunk
@@ -269,4 +273,20 @@ protected ObjectId(final AnyObjectId src) {
 	public ObjectId toObjectId() {
 		return this;
 	}
+
+	private void writeObject(ObjectOutputStream os)  throws IOException {
+	private void readObject(ObjectInputStream ois)  throws IOException {
Minor nit: Only 1 space between ) and throws, please.
quoted hunk
diff --git a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
index 5bbf664..899f73f 100644
--- a/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
+++ b/org.spearce.jgit/src/org/spearce/jgit/transport/RemoteConfig.java
@@ -53,7 +57,7 @@
  * describing how refs should be transferred between this repository and the
  * remote repository.
  */
-public class RemoteConfig {
+public class RemoteConfig implements Serializable {
Please set a serialVersionUID.
quoted hunk
@@ -382,4 +386,77 @@ public TagOpt getTagOpt() {
 	public void setTagOpt(final TagOpt option) {
 		tagopt = option != null ? option : TagOpt.AUTO_FOLLOW;
 	}
+	
+	private void writeObject(ObjectOutputStream os) throws IOException {
+
+	    // Name
+	    os.writeUTF(name);
+	
+	    // Key, Value pairs
+	    for (URIish uri : uris) {
+            os.writeUTF(KEY_URL);
+            os.writeUTF(uri.toPrivateString());
+        }
+
+        for (RefSpec refspec : fetch) {
There is some sort of whitespace damage here, the second for loop
is not lined up at the same starting column as the first for loop.
My guess is, you have tabs in here.  We only indent with spaces.
+            os.writeUTF(KEY_FETCH);
+            os.writeUTF(refspec.toString());
+        }
+
+        for (RefSpec refspec : push) {
+        	os.writeUTF(KEY_PUSH);
+            os.writeUTF(refspec.toString());
+        }
Should we maybe allow RefSpec to serialize itself with
os.writeObject() rather than using writeUTF() directly?

FWIW, I did find this new implementation to be much easier to read.
Thanks.

-- 
Shawn.

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Robin Rosenberg <hidden>
Date: 2016-06-15 22:46:14

Shawn wrote:
quoted
+            os.writeUTF(KEY_FETCH);
+            os.writeUTF(refspec.toString());
+        }
+
+        for (RefSpec refspec : push) {
+        	os.writeUTF(KEY_PUSH);
+            os.writeUTF(refspec.toString());
+        }
Should we maybe allow RefSpec to serialize itself with
os.writeObject() rather than using writeUTF() directly?
Doesn't the style above make it easy to define and document
a format that is easy for non-java programs to write and read,
while writeObject introduces java-centric stuff (depending on
the full class name etc).

-- robin

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Shawn O. Pearce <hidden>
Date: 2016-06-15 22:46:14

Robin Rosenberg [off-list ref] wrote:
Shawn wrote:
quoted
quoted
+            os.writeUTF(KEY_FETCH);
+            os.writeUTF(refspec.toString());
+        }
+
+        for (RefSpec refspec : push) {
+        	os.writeUTF(KEY_PUSH);
+            os.writeUTF(refspec.toString());
+        }
Should we maybe allow RefSpec to serialize itself with
os.writeObject() rather than using writeUTF() directly?
Doesn't the style above make it easy to define and document
a format that is easy for non-java programs to write and read,
while writeObject introduces java-centric stuff (depending on
the full class name etc).
Non-Java reading a Java serialization stream?  Seriously?

-- 
Shawn.

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Sverre Rabbelier <hidden>
Date: 2016-06-15 22:46:14

On Wed, Feb 18, 2009 at 22:48, Shawn O. Pearce [off-list ref] wrote:
Non-Java reading a Java serialization stream?  Seriously?
That would be a contender for UJSFWIINI :P (with JS standing for Java
Serialization).

-- 
Cheers,

Sverre Rabbelier

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Robin Rosenberg <hidden>
Date: 2016-06-15 22:46:14

onsdag 18 februari 2009 22:48:59 skrev "Shawn O. Pearce" [off-list ref]:
Robin Rosenberg [off-list ref] wrote:
quoted
Shawn wrote:
quoted
quoted
+            os.writeUTF(KEY_FETCH);
+            os.writeUTF(refspec.toString());
+        }
+
+        for (RefSpec refspec : push) {
+        	os.writeUTF(KEY_PUSH);
+            os.writeUTF(refspec.toString());
+        }
Should we maybe allow RefSpec to serialize itself with
os.writeObject() rather than using writeUTF() directly?
Doesn't the style above make it easy to define and document
a format that is easy for non-java programs to write and read,
while writeObject introduces java-centric stuff (depending on
the full class name etc).
Non-Java reading a Java serialization stream?  Seriously?
No, that was my objection to using writeObject, as that make
it a Java-only stream, but then it might not be worth doing
it via the serialization mechanism.

-- robin

Re: [JGIT PATCH] 1/2 : (reworked) Externalizable/Serializable Items

From: Shawn O. Pearce <hidden>
Date: 2016-06-15 22:46:14

Robin Rosenberg [off-list ref] wrote:
onsdag 18 februari 2009 22:48:59 skrev "Shawn O. Pearce" [off-list ref]:
quoted
Non-Java reading a Java serialization stream?  Seriously?
No, that was my objection to using writeObject, as that make
it a Java-only stream, but then it might not be worth doing
it via the serialization mechanism.
IMHO, if we are talking about either java.io.Serializable or
java.io.Externalizable, there's no point in considering a non
Java peer.

If you want a non-Java format, we'd need to consider a much
more neutral encoding, like Google's protobuf, or *shudder*
XML/JSON, or cooking up our own format.

That wasn't this thread started with.  The original poster just
wanted an easy way to serialize some basic data types from JGit,
as part of some higher level stream being done in the container
application.  Since that higher level stream is a apparently a
Java object serialization stream, we just need to match that.

-- 
Shawn.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help