MIME-Version: 1.0
In-Reply-To: <54DBCB51.2080000@ecosensory.com>
References: <CAOFvGD7WSM+qErYxvhe2m9YxwpAUvcEGsxgtYTZZmE_+v3Tpig AT mail DOT gmail DOT com>
	<54DBCB51 DOT 2080000 AT ecosensory DOT com>
Date: Wed, 11 Feb 2015 17:19:23 -0500
Message-ID: <CAOFvGD7jU4amu0AcugiSb24XOT+WKQGcvStJjBAK3am8XpjYUA@mail.gmail.com>
Subject: Re: [geda-user] Using Lua to safely read configuration and layout
 files (program attached)
From: Jason White <whitewaterssoftwareinfo AT gmail DOT com>
To: geda-user AT delorie DOT com
Content-Type: text/plain; charset=UTF-8
Reply-To: geda-user AT delorie DOT com
Errors-To: nobody AT delorie DOT com
Precedence: bulk

John, this is using Lua not JSON.

To iterate through all 200,000 elements in the 10 megabyte file
without printing, the program took 1.14 seconds on my dual core
machine from 2003. So roughly 175,000 elements per second or even more
roughly 8.8 "megabytes per second" given some average size per
element.

I don't know how well with is improve handling in versioning systems.
What I think you would need for git friendliness is to maintain the
order of the elements in between successive versions of the file. How
would you do that? Perhaps you could sort the elements by their
coordinates in the file (since no other unique identifier exists that
I know of).

On Wed, Feb 11, 2015 at 4:36 PM, John Griessen <john AT ecosensory DOT com> wrote:
> On 02/10/2015 02:28 PM, Jason White wrote:
>>
>> Levente, I just [yesterday] tried a script with over 200,000 entries
>> (roughly 9.8
>> megabytes of text) and I am happy to report it worked without error.
>>
>> Here is a link to the file I used
>>
>> https://drive.google.com/file/d/0BwP0qhqyaTIIYVlISkQwMXJtRlU/view?usp=sharing
>
>
> What was the time like on an ordinary computer with just two cores and no
> special parallelizing
> help?  Are there enough benefits to using JSON to offset the larger file
> size?
> I like that it could tame white space differences when stored in version
> control.
>
> How would that work?  How does JSON decrease the impact of white space
> differences
> when stored in version control?
>
> On 02/08/2015 05:47 AM, Christian Riggenbach wrote:> If you use an
> versioning system for the schematics today, you get a mess of
>> chopped "gibberish" now, as the normal line based patches are not human
>> readable.
>
> Would making code commits always based on content between { and } help that?


-- 
Jason White