X-Authentication-Warning:	delorie.com: mail set sender to geda-user-bounces using -f
X-Recipient:	geda-user AT delorie DOT com
DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:content-type;
	bh=e5Du6PilAJxrkWb/ZWRypCzVfMGKBtpWLlDAsXV3bO4=;
	b=YUzBnPnzYrj0ajbophSlzPW/tKbvL8a19K4SV5gBPpCftIxCI6vAfob2I7MjX5c4az
	7yaZOPcmA2CUoC5SLF48ANH2SkLuclKEPu4oH+b/BV6BvouIeaqsdSzXA/GHQDk9EXcP
	uihHfFQNuaB78pL/nuS5kEpgUlAu3+fLiV/ocL4NeysZZIVYJL42dKrhqDnVfcm2i6/k
	0DSwLW0cVvu1nW7d2EiI8a5ORg0pkdT8e2YGxGoWRRIO3g9cD5ZaCSOAeogCxWMWjWDR
	IEs11fOaMGZj3rVCO/4J+Kwu8EHGsfVHasz/PZlmX9+WjC186hfKRClk//Ae9snriRj2
	5pbQ==
MIME-Version:	1.0
X-Received:	by 10.202.168.141 with SMTP id r135mr569526oie.92.1423693163837;
	Wed, 11 Feb 2015 14:19:23 -0800 (PST)
In-Reply-To:	<54DBCB51.2080000@ecosensory.com>
References:	<CAOFvGD7WSM+qErYxvhe2m9YxwpAUvcEGsxgtYTZZmE_+v3Tpig AT mail DOT gmail DOT com>
	<54DBCB51 DOT 2080000 AT ecosensory DOT com>
Date:	Wed, 11 Feb 2015 17:19:23 -0500
Message-ID:	<CAOFvGD7jU4amu0AcugiSb24XOT+WKQGcvStJjBAK3am8XpjYUA@mail.gmail.com>
Subject:	Re: [geda-user] Using Lua to safely read configuration and layout
	files (program attached)
From:	Jason White <whitewaterssoftwareinfo AT gmail DOT com>
To:	geda-user AT delorie DOT com
Reply-To:	geda-user AT delorie DOT com
Errors-To:	nobody AT delorie DOT com
X-Mailing-List:	geda-user AT delorie DOT com
X-Unsubscribes-To:	listserv AT delorie DOT com

John, this is using Lua not JSON.

To iterate through all 200,000 elements in the 10 megabyte file
without printing, the program took 1.14 seconds on my dual core
machine from 2003. So roughly 175,000 elements per second or even more
roughly 8.8 "megabytes per second" given some average size per
element.

I don't know how well with is improve handling in versioning systems.
What I think you would need for git friendliness is to maintain the
order of the elements in between successive versions of the file. How
would you do that? Perhaps you could sort the elements by their
coordinates in the file (since no other unique identifier exists that
I know of).

On Wed, Feb 11, 2015 at 4:36 PM, John Griessen <john AT ecosensory DOT com> wrote:
> On 02/10/2015 02:28 PM, Jason White wrote:
>>
>> Levente, I just [yesterday] tried a script with over 200,000 entries
>> (roughly 9.8
>> megabytes of text) and I am happy to report it worked without error.
>>
>> Here is a link to the file I used
>>
>> https://drive.google.com/file/d/0BwP0qhqyaTIIYVlISkQwMXJtRlU/view?usp=sharing
>
>
> What was the time like on an ordinary computer with just two cores and no
> special parallelizing
> help?  Are there enough benefits to using JSON to offset the larger file
> size?
> I like that it could tame white space differences when stored in version
> control.
>
> How would that work?  How does JSON decrease the impact of white space
> differences
> when stored in version control?
>
> On 02/08/2015 05:47 AM, Christian Riggenbach wrote:> If you use an
> versioning system for the schematics today, you get a mess of
>> chopped "gibberish" now, as the normal line based patches are not human
>> readable.
>
> Would making code commits always based on content between { and } help that?



-- 
Jason White

- Raw text -