Mail Archives: geda-user/2011/12/06/07:29:13
--20cf3074b5feb8609204b36b9148
Content-Type: multipart/alternative; boundary=20cf3074b5feb8608f04b36b9146
--20cf3074b5feb8608f04b36b9146
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
On Mon, Dec 5, 2011 at 10:44 PM, Karl Hammar <karl AT aspodata DOT se> wrote:
> Luigi:
> ...
> > In attachment an improved version of the script.
> ...
>
> That version works fine.
>
> It seems a little strange, though, to use both ftw and opendir.
>
Let me to explain why this decision. In my point of view we have two
targets:
a) build a list with the sub-folders, and
b) skip sub-folder with no files or files other than .sym
So the ftw system is very fast to build the tree of sub-folders, but it
scans all the tree in depth. I haven't found any option to stop it at the
first level of the tree. This is the real reason on the why i use readdir
in the (is-there-any-symbols-in?) procedure. readdir,instead, stops at the
first level of the tree and the procedure exit immediately when found a sym
file.
So the code in (build-symbols-list) does a searching for only the
subfolders containing sym files and exiting as fast as possible.
I thought the overhead of this mechanism is lower than an hash calculation
for each sym file. But I was wrong !
So to verify this i make a test as follow:
i have downloaded all gedasymbols site in the folder gedasymbol and copy it
4 times so i have:
gedasymbol
gedasymbol2
gedasymbol3
gedasymbol4
gedasymbol5
main folders. Each folder has (at now):
716 folder (x5 3580)
4997 files of this 1754 are symbols (x5 24985 8770)
in gafrc i put the following lines:
(load (build-path geda-rchome-path "local-symbols-library.scm")) <--
opendir version
;;;(load (build-path geda-rchome-path "peter.scm")) <-- Peter version
;;;(load (build-path geda-rchome-path "karl.scm")) <-- Karl version
;;;(load (build-path geda-rchome-path
"local-symbols-library-hash-version.scm")) <-- we'll see later
(component-library-add-tree "Own"
"/home/ultrabit/ProgrammingWorkspace/gitrepos/sviluppo-100/gitosis-admin/Fo=
otprints/gEda-sym/personal/")
(component-library-add-tree "gEDA"
"/home/ultrabit/ProgrammingWorkspace/electronics/geda-suite/gedasymbols/www=
/user")
(component-library-add-tree "gEDA1"
"/home/ultrabit/ProgrammingWorkspace/electronics/geda-suite/gedasymbols2/ww=
w/user")
(component-library-add-tree "gEDA2"
"/home/ultrabit/ProgrammingWorkspace/electronics/geda-suite/gedasymbols3/ww=
w/user")
(component-library-add-tree "gEDA3"
"/home/ultrabit/ProgrammingWorkspace/electronics/geda-suite/gedasymbols4/ww=
w/user")
(component-library-add-tree "gEDA4"
"/home/ultrabit/ProgrammingWorkspace/electronics/geda-suite/gedasymbols5/ww=
w/user")
at the prompt shell i run:
ultrabit AT hurricane:~/.gEDA$ time gschem -c "(gschem-exit)" <--- loading
gschem and exit at startup
Invalid path [./sym] passed to component-library
Invalid path [./footprint] passed to component-library
Invalid path [./footprint] passed to component-library-search
real 0m1.427s
user 0m1.100s
sys 0m0.248s
(total gschem loading time (intel core 2 @ 3Ghz and 8 Gb of ram (not all
free))
Step 2 second test some conditions but Peter version:
;;;(load (build-path geda-rchome-path "local-symbols-library.scm")) <--
opendir version
(load (build-path geda-rchome-path "peter.scm")) <-- Peter version
;;;(load (build-path geda-rchome-path "karl.scm")) <-- Karl version
;;;(load (build-path geda-rchome-path
"local-symbols-library-hash-version.scm")) <-- we'll see later
ultrabit AT hurricane:~/.gEDA$ time gschem -c "(gschem-exit)"
Invalid path [./sym] passed to component-library
Invalid path [./footprint] passed to component-library
Invalid path [./footprint] passed to component-library-search
real 0m5.538s
user 0m3.600s
sys 0m1.840s
Step 3 same conditions but with Karl version:
ultrabit AT hurricane:~/.gEDA$ time gschem -c "(gschem-exit)"
Invalid path [./sym] passed to component-library
Invalid path [./footprint] passed to component-library
Invalid path [./footprint] passed to component-library-search
real 0m1.436s
user 0m1.172s
sys 0m0.160s
So, i am surprised about results because i thought the Karl version was
like the Peter version, but this is not true.
I found the problem (many thanks to the "two column" emacs mode) was the
use of (hasx-set!) vs (hashq-set!) and storing symbols instead of strings .
In conclusion i made a very condensed guile code (one procedure) to do all
using only ftw:
Step 4:
;;;(load (build-path geda-rchome-path "local-symbols-library.scm")) <--
opendir version
;;;(load (build-path geda-rchome-path "peter.scm")) <-- Peter version
;;;(load (build-path geda-rchome-path "karl.scm")) <-- Karl version
(load (build-path geda-rchome-path "local-symbols-library.scm")) <--
Optimized hash version
ultrabit AT hurricane:~/.gEDA$ time gschem -c "(gschem-exit)"
Invalid path [./sym] passed to component-library
Invalid path [./footprint] passed to component-library
Invalid path [./footprint] passed to component-library-search
real 0m1.229s
user 0m0.980s
sys 0m0.152s
The better i can do :-D
If we are going to use opendir, we could just as well do the
> traversal ourself and dump ftw.
>
> I see two alternatives:
>
> a) I'm satisfied with my version, and stop here
>
> b) we
> dump ftw
> do the traversal ourself
> clean up your script
> and proposes it for inclusion in geda
>
> What do you think?
>
Karl, please look at code in attachment and say me if can be accepted as
is. I think this can be proposed for inclusion in geda.
>
> Regards,
> /Karl Hammar
>
> -----------------------------------------------------------------------
> Asp=F6 Data
> Lilla Asp=F6 148
> S-742 94 =D6sthammar
> Sweden
> +46 173 140 57
>
>
> Regards,
Luigi
PS:
Peter in bcc as FYI
--20cf3074b5feb8608f04b36b9146
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<br><br><div class=3D"gmail_quote">On Mon, Dec 5, 2011 at 10:44 PM, Karl Ha=
mmar <span dir=3D"ltr"><<a href=3D"mailto:karl AT aspodata DOT se" target=3D"_b=
lank">karl AT aspodata DOT se</a>></span> wrote:<br><blockquote class=3D"gmail_=
quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1=
ex">
Luigi:<br>
<div>...<br>
> In attachment an improved version of the script.<br>
</div>...<br>
<br>
That version works fine.<br>
<br>
It seems a little strange, though, to use both ftw and opendir.<br></blockq=
uote><div>Let me to explain why this decision. In my point of view we have =
two targets:<br><br>a) build a list with the sub-folders, and<br>b) skip su=
b-folder with no files or files other than .sym<br>
<br>So the ftw system is very fast to build the tree of sub-folders, but it=
scans all the tree in depth. I haven't found any option to stop it at =
the first level of the tree. This is the real reason on the why i use readd=
ir in the (is-there-any-symbols-in?) procedure. readdir,instead, stops at t=
he first level of the tree and the procedure exit immediately when found a =
sym file. <br>
So the code in (build-symbols-list) does a searching for only the subfolder=
s containing sym files and exiting as fast as possible. <br>I <span id=3D"r=
esult_box" class=3D"short_text" lang=3D"en"><span class=3D"hps">thought</sp=
an></span> the overhead of this mechanism is lower than an hash calculation=
for each sym file. But I was wrong !<br>
So to verify this i make a test as follow:<br><br>i have downloaded all ged=
asymbols site in the folder gedasymbol and copy it 4 times so i have:<br>ge=
dasymbol<br>gedasymbol2<br>gedasymbol3<br>gedasymbol4<br>gedasymbol5<br>
<br>main folders. Each folder has (at now):<br>
716 folder (x5 3580)<br>4997 files of this 1754 are symbols (x5 24985 8770)=
<br>=A0<br>in gafrc i put the following lines:<br>(load (build-path geda-rc=
home-path "local-symbols-library.scm")) <-- opendir version<br=
>
;;;(load (build-path geda-rchome-path "peter.scm")) <-- Peter=
=A0 version<br>;;;(load (build-path geda-rchome-path "karl.scm"))=
=A0 <-- Karl=A0 version<br>;;;(load (build-path geda-rchome-path "l=
ocal-symbols-library-hash-version.scm")) <-- we'll see later<br=
>
<br>(component-library-add-tree "Own" "/home/ultrabit/Progra=
mmingWorkspace/gitrepos/sviluppo-100/gitosis-admin/Footprints/gEda-sym/pers=
onal/")<br>
(component-library-add-tree "gEDA" "/home/ultrabit/Programmi=
ngWorkspace/electronics/geda-suite/gedasymbols/www/user")<br>(componen=
t-library-add-tree "gEDA1" "/home/ultrabit/ProgrammingWorksp=
ace/electronics/geda-suite/gedasymbols2/www/user")<br>
(component-library-add-tree "gEDA2" "/home/ultrabit/Programm=
ingWorkspace/electronics/geda-suite/gedasymbols3/www/user")<br>(compon=
ent-library-add-tree "gEDA3" "/home/ultrabit/ProgrammingWork=
space/electronics/geda-suite/gedasymbols4/www/user")<br>
(component-library-add-tree "gEDA4" "/home/ultrabit/Programm=
ingWorkspace/electronics/geda-suite/gedasymbols5/www/user")<br><br>at =
the prompt shell i run:<br><br>ultrabit AT hurricane:~/.gEDA$ time gschem -c &=
quot;(gschem-exit)"=A0=A0=A0=A0 <--- loading gschem and exit at sta=
rtup<br>
Invalid path [./sym] passed to component-library<br>Invalid path [./footpri=
nt] passed to component-library<br>Invalid path [./footprint] passed to com=
ponent-library-search<br><br>real=A0=A0=A0 0m1.427s=A0=A0 <br>
user=A0=A0=A0 0m1.100s<br>sys=A0=A0=A0 0m0.248s<br><br>(total gschem loadin=
g time (intel core 2 @ 3Ghz and 8 Gb of ram (not all free)) <br><br>Step 2 =
second test some conditions but Peter version:<br>;;;(load (build-path geda=
-rchome-path "local-symbols-library.scm")) <-- opendir version=
<br>
(load (build-path geda-rchome-path "peter.scm")) <-- Peter=A0 =
version<br>
;;;(load (build-path geda-rchome-path "karl.scm"))=A0 <-- Karl=
=A0 version<br>
;;;(load (build-path geda-rchome-path "local-symbols-library-hash-vers=
ion.scm")) <-- we'll see later<br>=A0<br>ultrabit AT hurricane:~/.=
gEDA$ time gschem -c "(gschem-exit)"<br>Invalid path [./sym] pass=
ed to component-library<br>
Invalid path [./footprint] passed to component-library<br>Invalid path [./f=
ootprint] passed to component-library-search<br><br>real=A0=A0=A0 0m5.538s<=
br>user=A0=A0=A0 0m3.600s<br>sys=A0=A0=A0 0m1.840s<br><br>Step 3 same condi=
tions but with Karl version:<br>
ultrabit AT hurricane:~/.gEDA$ time gschem -c "(gschem-exit)"<br>Inv=
alid path [./sym] passed to component-library<br>Invalid path [./footprint]=
passed to component-library<br>Invalid path [./footprint] passed to compon=
ent-library-search<br>
<br>real=A0=A0=A0 0m1.436s<br>user=A0=A0=A0 0m1.172s<br>sys=A0=A0=A0 0m0.16=
0s<br><br>So, i am surprised about results because i thought the Karl versi=
on was like the Peter version, but this is not true. <br>I found the proble=
m (many thanks to the "two column" emacs mode) was the use of (ha=
sx-set!) vs (hashq-set!) and storing symbols instead of strings .<br>
In conclusion i made a very condensed guile code (one procedure) to do all =
using only ftw:<br><br>Step 4:<br>;;;(load (build-path geda-rchome-path &qu=
ot;local-symbols-library.scm")) <-- opendir version<br>
;;;(load (build-path geda-rchome-path "peter.scm")) <-- Peter=
=A0 version<br>
;;;(load (build-path geda-rchome-path "karl.scm"))=A0 <-- Karl=
=A0 version<br>
(load (build-path geda-rchome-path "local-symbols-library.scm"))=
=A0 <-- Optimized hash version<br><br>ultrabit AT hurricane:~/.gEDA$ time g=
schem -c "(gschem-exit)"<br>Invalid path [./sym] passed to compon=
ent-library<br>
Invalid path [./footprint] passed to component-library<br>Invalid path [./f=
ootprint] passed to component-library-search<br><br>real=A0=A0=A0 0m1.229s<=
br>user=A0=A0=A0 0m0.980s<br>sys=A0=A0=A0 0m0.152s<br><br>The better i can =
do :-D <br><br>
<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0pt 0pt 0pt 0.8=
ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
If we are going to use opendir, we could just as well do the<br>
traversal ourself and dump ftw.<br>
<br>
I see two alternatives:<br>
<br>
a) I'm satisfied with my version, and stop here<br>
<br>
b) we<br>
=A0 =A0dump ftw<br>
=A0 =A0do the traversal ourself<br>
=A0 =A0clean up your script<br>
=A0and proposes it for inclusion in geda<br>
<br>
What do you think?<br></blockquote><div><br>Karl, please look at code in at=
tachment and say me if can be accepted as is. I think this can be proposed =
for inclusion in geda.<br>
<br>=A0</div><blockquote class=3D"gmail_quote" style=3D"margin: 0pt 0pt 0pt=
0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
Regards,<br>
<span><font color=3D"#888888">/Karl Hammar<br>
</font></span><div><div><br>
-----------------------------------------------------------------------<br>
Asp=F6 Data<br>
Lilla Asp=F6 148<br>
S-742 94 =D6sthammar<br>
Sweden<br>
<a href=3D"tel:%2B46%20173%20140%2057" value=3D"+4617314057" target=3D"_bla=
nk">+46 173 140 57</a><br>
<br>
<br>
</div></div></blockquote></div>Regards,<br>Luigi<br><br>PS:<br>Peter in bcc=
as FYI<br>
--20cf3074b5feb8608f04b36b9146--
--20cf3074b5feb8609204b36b9148
Content-Type: text/x-scheme; charset=US-ASCII; name="local-symbols-library.scm"
Content-Disposition: attachment; filename="local-symbols-library.scm"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_gvuvncck0
OyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IC0qLVNjaGVtZS0qLQo7ICAgIFRoaXMgcHJvZ3JhbSBpcyBmcmVlIHNvZnR3YXJlOiB5b3UgY2Fu
IHJlZGlzdHJpYnV0ZSBpdCBhbmQvb3IgbW9kaWZ5CjsgICAgaXQgdW5kZXIgdGhlIHRlcm1zIG9m
IHRoZSBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBhcyBwdWJsaXNoZWQgYnkKOyAgICB0aGUg
RnJlZSBTb2Z0d2FyZSBGb3VuZGF0aW9uLCBlaXRoZXIgdmVyc2lvbiAzIG9mIHRoZSBMaWNlbnNl
LCBvcgo7ICAgIChhdCB5b3VyIG9wdGlvbikgYW55IGxhdGVyIHZlcnNpb24uCjsKOyAgICBUaGlz
IHByb2dyYW0gaXMgZGlzdHJpYnV0ZWQgaW4gdGhlIGhvcGUgdGhhdCBpdCB3aWxsIGJlIHVzZWZ1
bCwKOyAgICBidXQgV0lUSE9VVCBBTlkgV0FSUkFOVFk7IHdpdGhvdXQgZXZlbiB0aGUgaW1wbGll
ZCB3YXJyYW50eSBvZgo7ICAgIE1FUkNIQU5UQUJJTElUWSBvciBGSVRORVNTIEZPUiBBIFBBUlRJ
Q1VMQVIgUFVSUE9TRS4gIFNlZSB0aGUKOyAgICBHTlUgR2VuZXJhbCBQdWJsaWMgTGljZW5zZSBm
b3IgbW9yZSBkZXRhaWxzLgo7CjsgICAgWW91IHNob3VsZCBoYXZlIHJlY2VpdmVkIGEgY29weSBv
ZiB0aGUgR05VIEdlbmVyYWwgUHVibGljIExpY2Vuc2UKOyAgICBhbG9uZyB3aXRoIHRoaXMgcHJv
Z3JhbS4gIElmIG5vdCwgc2VlIDxodHRwOi8vd3d3LmdudS5vcmcvbGljZW5zZXMvPi4KOwo7IFRo
aXMgY29kZSBpcyBiYXNlZCBvbiB0aGUgc291cmNlIGJ5IFBldGVyIEJUIEJyZXR0IGFuZCBLYXJs
IEhhbW1hci4gCjsKOyBDb3B5cmlnaHQgKEMpIDIwMTEgIEwuUy5QLiA8dWx0cmFiaXRAZ21haWwu
Y29tPgo7CjsgY2hhbmdlbG9nIDEyLzA2LzIwMTE6CjsJKiBvcHRpbWl6ZWQgdmVyc2lvbi4KOwkq
IHJlbW92ZWQgYWxsIHVubmVlZGVkIHByb2NlZHVyZXMuCjsJCjsgY2hhbmdlbG9nIDEyLzA1LzIw
MTE6CjsJKiBzb3J0aW5nIGZvbGRlcmxpc3QgYmVmb3JlIGFkZGluZyB0cmVlCjsJKiB1c2luZyBz
dHJpbmctc3VmZml4LWNpIGluc3RlYWQgb2YgcmVnZXggaW4gdGhlIGlzLXRoZXJlLWFueS1zeW1i
b2xzLWluPyBwcm9jZWR1cmUKOwo7IGNoYW5nZWxvZyAxMi8wNC8yMDExOgo7CSogYWRkZWQgKGlz
LXRoZXJlLWFueS1zeW1ib2xzLWluPyBkaXIpIHByb2NlZHVyZSB0byBjaGVjayBpZiBhIGZvbGRl
ciBjb250YWlucyBhbnkgc3ltYm9sIGZpbGUuCjsKOyBjaGFuZ2Vsb2cgMTIvMDIvMjAxMToKOwkq
IHN0YXJ0aW5nIHZlcnNpb24gCjsKOyBUb2RvIDoKOwkqIEltcGxlbWVudCBlbnZpcm9ubWVudCB2
YXJzIHN1YnN0aXR1dGlvbgo7CSogQ29kZSBjbGVhbnVwIGFuZCBvcHRpbWl6YXRpb24KCih1c2Ut
bW9kdWxlcyAoaWNlLTkgZnR3KSkKOwo7IAo7CihkZWZpbmUgY29tcG9uZW50LWxpYnJhcnktYWRk
LXRyZWUKICAobGFtYmRhIChwcmVmaXggbWFpbmZvbGRlcikKICAgIChkZWZpbmUgaHQgKG1ha2Ut
aGFzaC10YWJsZSAzMSkpCiAgICAoc2V0ISBtYWluZm9sZGVyIChzdHJpbmctdHJpbS1yaWdodCAo
c3RyaW5nLXRyaW0tYm90aCBtYWluZm9sZGVyIGNoYXItc2V0OndoaXRlc3BhY2UpICNcLykpCiAg
ICAoYmVnaW4KICAgICAgKGZ0dyBtYWluZm9sZGVyCgkgICAobGFtYmRhIChmaWxlbmFtZSBzdGF0
aW5mbyBmbGFncykKCSAgICAgKGFuZCAoZXE/ICdyZWd1bGFyIChzdGF0OnR5cGUgc3RhdGluZm8p
KQoJCSAgKHN0cmluZy1zdWZmaXgtY2k/ICIuc3ltIiBmaWxlbmFtZSkKCQkgIChoYXNocS1zZXQh
IGh0IChzdHJpbmctPnN5bWJvbCAoZGlybmFtZSBmaWxlbmFtZSkpICN0KSkgI3QpKSkKICAgIChm
b3ItZWFjaAogICAgIChsYW1iZGEgKGRpcikKICAgICAgIChjb21wb25lbnQtbGlicmFyeSBkaXIg
KHN0cmluZy1hcHBlbmQgcHJlZml4IChzdHJpbmctY29weSBkaXIgKHN0cmluZy1sZW5ndGggbWFp
bmZvbGRlcikpKSkKICAgICAgICkKICAgICAoc29ydC1saXN0ISAoaGFzaC1tYXAtPmxpc3QgKGxh
bWJkYSAoa2V5IHZhbCkgKHN5bWJvbC0+c3RyaW5nIGtleSkpIGh0KSBzdHJpbmc+PykpCiAgICAp
CiAgKQo=
--20cf3074b5feb8609204b36b9148--
- Raw text -