[DTC PATCH] Add support for decimal, octal and binary based cell values.

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

---

Beginnings of test cases in the next patch.
Don't stress the yylloc too much.  We'll eventually
work to make that better and more general later.


 data.c       |   21 +++++++++++++++++++++
 dtc-lexer.l  |   21 +++++++++++++++------
 dtc-parser.y |   16 ++++++++++++++--
 dtc.h        |    1 +
 4 files changed, 51 insertions(+), 8 deletions(-)

diff --git a/data.c b/data.c
index 1907a1a..c6c2350 100644
--- a/data.c
+++ b/data.c

@@ -19,6 +19,7 @@
  */
 
 #include "dtc.h"
+#include "dtc-parser.tab.h"
 
 void fixup_free(struct fixup *f)
 {

@@ -224,6 +225,26 @@ struct data data_merge(struct data d1, struct data d2)
 	return d;
 }
 
+/*
+ * Convert a string representation of a numberic cell
+ * in the given base into a cell.
+ */
+cell_t data_convert_cell(char *s, unsigned int base)
+{
+	cell_t c;
+	extern YYLTYPE yylloc;
+
+	c = strtoul(s, NULL, base);
+	if (errno == EINVAL || errno == ERANGE) {
+		fprintf(stderr,
+			"Line %d: Invalid cell value '%s'; %d assumed\n",
+			yylloc.first_line, s, c);
+	}
+
+	return c;
+}
+
+
 struct data data_append_cell(struct data d, cell_t word)
 {
 	cell_t beword = cpu_to_be32(word);

diff --git a/dtc-lexer.l b/dtc-lexer.l
index 6146bad..93f3268 100644
--- a/dtc-lexer.l
+++ b/dtc-lexer.l

@@ -83,15 +83,24 @@ REFCHAR		({PROPCHAR}|{UNITCHAR}|[/@])
 			BEGIN(INITIAL);
 			return ';';
 		}
+<CELLDATA>[bodh]# {
+			yylloc.first_line = yylineno;
+			if (*yytext == 'b')
+				yylval.cbase = 2;
+			else if (*yytext == 'o')
+				yylval.cbase = 8;
+			else if (*yytext == 'd')
+				yylval.cbase = 10;
+			else
+				yylval.cbase = 16;
+			DPRINT("Base: %d\n", yylval.cbase);
+			return DT_BASE;
+		}
 
 <CELLDATA>[0-9a-fA-F]+	{
 			yylloc.first_line = yylineno;
-			if (yyleng > 2*sizeof(yylval.cval)) {
-				fprintf(stderr,
-					"Cell value %s too long\n", yytext);
-			}
-			yylval.cval = strtoul(yytext, NULL, 16);
-			DPRINT("Cell: %x\n", yylval.cval);
+			yylval.str = strdup(yytext);
+			DPRINT("Cell: '%s'\n", yylval.str);
 			return DT_CELL;
 		}
 

diff --git a/dtc-parser.y b/dtc-parser.y
index bd725fe..992fdb1 100644
--- a/dtc-parser.y
+++ b/dtc-parser.y

@@ -33,6 +33,7 @@ extern struct boot_info *the_boot_info;
 
 %union {
 	cell_t cval;
+	unsigned int cbase;
 	u8 byte;
 	char *str;
 	struct data data;

@@ -50,7 +51,8 @@ extern struct boot_info *the_boot_info;
 %token <addr> DT_ADDR
 %token <str> DT_PROPNAME
 %token <str> DT_NODENAME
-%token <cval> DT_CELL
+%token <cbase> DT_BASE
+%token <str> DT_CELL
 %token <byte> DT_BYTE
 %token <data> DT_STRING
 %token <str> DT_UNIT

@@ -61,6 +63,7 @@ extern struct boot_info *the_boot_info;
 %type <data> propdataprefix
 %type <re> memreserve
 %type <re> memreserves
+%type <cbase> opt_cell_base
 %type <data> celllist
 %type <data> bytestring
 %type <prop> propdef

@@ -133,7 +136,16 @@ propdataprefix:	propdata ',' { $$ = $1; }
 	|	/* empty */ { $$ = empty_data; }
 	;
 
-celllist:	celllist DT_CELL { $$ = data_append_cell($1, $2); }
+opt_cell_base:
+	  /* empty */
+		{ $$ = 16; }
+	| DT_BASE
+	;
+
+celllist:	celllist opt_cell_base DT_CELL {
+			$$ = data_append_cell($1,
+					      data_convert_cell($3, $2));
+		}
 	|	celllist DT_REF	{
 			$$ = data_append_cell(data_add_fixup($1, $2), -1);
 		}

diff --git a/dtc.h b/dtc.h
index 8d3964c..7ee96a5 100644
--- a/dtc.h
+++ b/dtc.h

@@ -118,6 +118,7 @@ struct data data_copy_mem(char *mem, int len);
 struct data data_copy_escape_string(char *s, int len);
 struct data data_copy_file(FILE *f, size_t len);
 
+cell_t data_convert_cell(char *s, unsigned int base);
 struct data data_append_data(struct data d, void *p, int len);
 struct data data_merge(struct data d1, struct data d2);
 struct data data_append_cell(struct data d, cell_t word);

-- 
1.5.0.rc3.ge4b0e

On Feb 15, 2007, at 11:13 AM, Jon Loeliger wrote:

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

---

What are people's thoughts on supporting '0x' and '0X' for hex?

- k

I like to see no space between the d#, h#, etc and the
numeric value.

Does anyone see a good reason to keep whitespace there?

Stuart

=20

-----Original Message-----
From: linuxppc-dev-bounces+b08248=3Dfreescale.com@ozlabs.org=20
[mailto:linuxppc-dev-bounces+b08248=3Dfreescale.com@ozlabs.org]=20
On Behalf Of Kumar Gala

[snip]

quoted

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

---

=20
What are people's thoughts on supporting '0x' and '0X' for hex?

Are you saying allow d#1234, h#5678, and 0x5678?  Or use 0x instead
of h#?

Stuart

On Thu, Feb 15, 2007 at 10:45:54AM -0700, Yoder Stuart-B08248 wrote:

I like to see no space between the d#, h#, etc and the
numeric value.

Does anyone see a good reason to keep whitespace there?

Does it really matter what color the bike shed is?

http://www.unixguide.net/freebsd/faq/16.19.shtml

(I've seen 0x<hex> <decimal> or 0d<decimal> and 0b<binary> been used
sometimes and work well, but I really don't care if it's what the dtc
uses or not.)

-Olof

=20

-----Original Message-----
From: Olof Johansson [mailto:olof@lixom.net]=20
Sent: Thursday, February 15, 2007 12:00 PM
To: Yoder Stuart-B08248
Cc: Jon Loeliger; linuxppc-dev@ozlabs.org
Subject: Re: [DTC PATCH] Add support for decimal, octal and=20
binary based cell values.
=20
On Thu, Feb 15, 2007 at 10:45:54AM -0700, Yoder Stuart-B08248 wrote:

quoted

=20
I like to see no space between the d#, h#, etc and the
numeric value.
=20
Does anyone see a good reason to keep whitespace there?

=20
Does it really matter what color the bike shed is?

In this case I think it does matter (a little :)).  If
you represent a number as <#h5678> it's clear that the
#h and 5678 are related.

If you represent it as <#h 5678> the relation between
the two is not as clear-- of course once you read the=20
spec you figure it out.

It's not a huge deal, but while we're adding a feature
to dtc why not make the syntax as clear as we can?

Stuart

Yoder Stuart-B08248 wrote:

In this case I think it does matter (a little :)).  If
you represent a number as <#h5678> it's clear that the
#h and 5678 are related.

If you represent it as <#h 5678> the relation between
the two is not as clear-- of course once you read the 
spec you figure it out.

It's not a huge deal, but while we're adding a feature
to dtc why not make the syntax as clear as we can?

It should be noted that the patch Jon posted should accept it either 
with or without the space.

-Scott

On Feb 15, 2007, at 11:49 AM, Yoder Stuart-B08248 wrote:

quoted

-----Original Message-----
From: linuxppc-dev-bounces+b08248=freescale.com@ozlabs.org
[mailto:linuxppc-dev-bounces+b08248=freescale.com@ozlabs.org]
On Behalf Of Kumar Gala

[snip]

quoted

quoted

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

---

What are people's thoughts on supporting '0x' and '0X' for hex?

Are you saying allow d#1234, h#5678, and 0x5678?  Or use 0x instead
of h#?

I'm saying in addition to supporting the d#, h# notation.

The reason I'm suggesting support '0x' is its pretty natural from C  
and I know there have been a number of times when I forget that all  
int constants in .dts are hex.

- k

On Thu, Feb 15, 2007 at 12:41:50PM -0600, Kumar Gala wrote:

On Feb 15, 2007, at 11:49 AM, Yoder Stuart-B08248 wrote:

quoted

quoted

-----Original Message-----
From: linuxppc-dev-bounces+b08248=freescale.com@ozlabs.org
[mailto:linuxppc-dev-bounces+b08248=freescale.com@ozlabs.org]
On Behalf Of Kumar Gala

[snip]

quoted

quoted

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

---

What are people's thoughts on supporting '0x' and '0X' for hex?

Are you saying allow d#1234, h#5678, and 0x5678?  Or use 0x instead
of h#?

I'm saying in addition to supporting the d#, h# notation.

The reason I'm suggesting support '0x' is its pretty natural from C  
and I know there have been a number of times when I forget that all  
int constants in .dts are hex.

I'd prefer not to do this.  I agree it would be nice in some ways, but
I'm worried that if people see 0x all over the place, they'll assume
that things without an 0x are decimal, which they can't be for
compatibility.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

On Thu, Feb 15, 2007 at 11:13:05AM -0600, Jon Loeliger wrote:

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

[snip]

I'd prefer a different name for this - and possibly it should go in a
different file.  The data_* prefix should be for functions that
actually manipulate struct data objects.

+{
+	cell_t c;
+	extern YYLTYPE yylloc;
+
+	c = strtoul(s, NULL, base);
+	if (errno == EINVAL || errno == ERANGE) {
+		fprintf(stderr,
+			"Line %d: Invalid cell value '%s'; %d assumed\n",
+			yylloc.first_line, s, c);
+	}

And I'd really prefer to keep the yyblah junk confined to the parser
code.  I think it would be better to move this function into
dtc-parser.y

Otherwise looks pretty good.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

On Feb 15, 2007, at 9:13 AM, Jon Loeliger wrote:

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Why do we need yet another syntax?  Everyone
else, including most silicon documentation, uses
the standard "C" syntax 0x for hex, 0 for octal, and
nothing for decimal.  If really needed binary, we
could us the '0b'.  Of the few dts files I've done,
anything to make this easier, rather than inventing
something new, would be appreciated.

Thanks.

	-- Dan

On Thu, Feb 15, 2007 at 02:33:47PM -0800, Dan Malek wrote:

On Feb 15, 2007, at 9:13 AM, Jon Loeliger wrote:

quoted

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Why do we need yet another syntax?  Everyone
else, including most silicon documentation, uses
the standard "C" syntax 0x for hex, 0 for octal, and
nothing for decimal.  If really needed binary, we
could us the '0b'.  Of the few dts files I've done,
anything to make this easier, rather than inventing
something new, would be appreciated.

Yes, but hex-by-default is already implemented and out there.  We
can't change it without breaking compatibility horribly.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

On Feb 15, 2007, at 2:57 PM, David Gibson wrote:

Yes, but hex-by-default is already implemented and out there.  We
can't change it without breaking compatibility horribly.

Compatibility with what?  The few dts files that exist
that are still all changing frequently due to other
tool modifications?  I'd prefer to change them now,
rather than propagate this new syntax.

Thanks.

	-- Dan

On Feb 15, 2007, at 4:14 PM, David Gibson wrote:

On Thu, Feb 15, 2007 at 12:41:50PM -0600, Kumar Gala wrote:

quoted

On Feb 15, 2007, at 11:49 AM, Yoder Stuart-B08248 wrote:

quoted

quoted

-----Original Message-----
From: linuxppc-dev-bounces+b08248=freescale.com@ozlabs.org
[mailto:linuxppc-dev-bounces+b08248=freescale.com@ozlabs.org]
On Behalf Of Kumar Gala

[snip]

quoted

quoted

New syntax d#, b#, o# and h# allow for an explicit prefix
on cell values to specify their base.  Eg: <d# 123>

Signed-off-by: Jon Loeliger <redacted>

---

What are people's thoughts on supporting '0x' and '0X' for hex?

Are you saying allow d#1234, h#5678, and 0x5678?  Or use 0x instead
of h#?

I'm saying in addition to supporting the d#, h# notation.

The reason I'm suggesting support '0x' is its pretty natural from C
and I know there have been a number of times when I forget that all
int constants in .dts are hex.

I'd prefer not to do this.  I agree it would be nice in some ways, but
I'm worried that if people see 0x all over the place, they'll assume
that things without an 0x are decimal, which they can't be for
compatibility.

People don't know what to assume, I know I've made a number of errors  
when I forget that all numbers where hex.

I think we should change dtc to follow standard C conventions and do  
it now rather than later.

We've already introduced dtc version compatibilities issues.  I agree  
with Dan that we should fix this now while there are a small handful  
of .dts in existence and we can provide a compat mode flag that  
treats non-prefixed numbers as hex instead of decimal.

(in theory with that we can use dtc to convert old .dts into new .dts)

- k

On Thu, Feb 15, 2007 at 03:37:36PM -0800, Dan Malek wrote:

On Feb 15, 2007, at 2:57 PM, David Gibson wrote:

quoted

Yes, but hex-by-default is already implemented and out there.  We
can't change it without breaking compatibility horribly.

Compatibility with what?  The few dts files that exist
that are still all changing frequently due to other
tool modifications?  I'd prefer to change them now,
rather than propagate this new syntax.

Nonetheless there are enough dts files out there - some of them within
vendors where I can't see them - that I absolutely do not want to
change something som fundamental in the format.

I have considered defining a "new-style" cell format, with delimiters
different from < > that uses C style literals.  But that idea's fairly
ugly too.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Why do we need yet another syntax?  Everyone
else, including most silicon documentation, uses
the standard "C" syntax 0x for hex, 0 for octal, and
nothing for decimal.

[Completely off-topic]

You're joking, right?  Many docs use 1234h for hex and
1011b for binary; or $1234 for hex and %1011 for binary;
or x'1234 and b'1011; or 1234_{(16)} and 1011_{(2)} (TeX
notation -- base as decimal in a parenthesised subscript);
etc. etc. etc.

Almost all computer languages use something more sane
than the C-family stuff too (esp. the dreaded "0 is
octal" thing is not exactly great).

Anyway, really off-topic :-)


Segher

I think we should change dtc to follow standard C conventions and do
it now rather than later.

We've already introduced dtc version compatibilities issues.  I agree
with Dan that we should fix this now while there are a small handful
of .dts in existence and we can provide a compat mode flag that
treats non-prefixed numbers as hex instead of decimal.

(in theory with that we can use dtc to convert old .dts into new .dts)

May I suggest every DTS file start with a version statement?
That way, you can change its format easily whenever you
want.  This would be version of the source language, not the
same thing as version of the generated blob.


Segher