Source language: Translate to:

Plugin to StrParse

Questions about NeoBook PlugIns

Moderator: Neosoft Support

Plugin to StrParse

Postby Alex » Thu Mar 08, 2012 10:32 am

Hello,

I use Neobook to strparse 2462 lines of text, but with my 9Mb program it soon like taking 40 seconds to sparse and put the text in [TEXT1], [TEXT2], [TEXT3]... variables. I am searching for a plugin that can strparse very fast (maybe using a dll or something like that).

Thanks


Alex
Alex
 
Posts: 745
Joined: Sat Mar 04, 2006 4:53 am

Postby HPW » Thu Mar 08, 2012 10:38 am

Hello,

Is this the same problem, where you get the newLISP solution for?
Your description is not really clear what you want to do.

Hans-Peter
Hans-Peter
User avatar
HPW
 
Posts: 2520
Joined: Fri Apr 01, 2005 11:24 pm
Location: Germany

Postby Wrangler » Thu Mar 08, 2012 11:08 am

Neobook string parses pretty fast for me. Why not post the strparse code you are using? Maybe someone here can optimize it.
Wrangler
--------------
"You never know about a woman. Whether she'll laugh, cry or go for a gun." - Louis L'Amour

Windows 7 Ultimate SP1 64bit
16GB Ram
Asus GTX 950 OC Strix
Software made with NeoBook
http://highdesertsoftware.com
User avatar
Wrangler
 
Posts: 1507
Joined: Thu Mar 31, 2005 11:40 pm
Location: USA

Postby Alex » Thu Mar 08, 2012 11:30 am

Hi,

Yes, it is the same problem as newLISP but newLISP doesn't load very fast the sentences on my program, it needs 35 seconds. That's why I am searching a way or an other plugin that can load very fast.

I tried:

Code: Select all
dbpOpenAccessDatabase "avoc" "C:\hd.mdb" ""
dbpOpenTable "avoc" "t" ""
dbpFieldToVar "avoc" "t" "t" "[Text]" "Delimiter=[#13][#10];SkipBlanks=No;NoDuplicates=No"
StrParse "[Text]" "[#13][#10]" "[TEXT]" "[Size]"


but same way, very long to load. My application is 9,8Mb, use lot of plugins, and there is approximaticaly 2Mb of codes (so 7,8 Mb of plugins). Maybe there is a way to load the sentences very fast.

I thought this:
Code: Select all
FileRead "!C:\TEXT" "All" "[Text]"
StrParse "[Text]" "[#13][#10]" "[TEXT]" "[Size]"


It is very long to load, but maybe if I load per 500 lines in 5 times it will goes faster, but how to do for that doesn't replace the variables and for that continue to load the rest of the text?

Thanks


Alex
Alex
 
Posts: 745
Joined: Sat Mar 04, 2006 4:53 am

Postby HPW » Thu Mar 08, 2012 11:40 am

Hello,

newlisp needs 35 sec to load what from where?
I do not understand what you do.
First you ask to read a textfile into neobook vars one line in one var.

Now your example show a MDB which puts data into a neobook variable.
From ther you parse it into seperate vars.

Then again a text-file sample.

What is really the problem?
What do you do with thousands of neobook vars?

Hans-Peter
Hans-Peter
User avatar
HPW
 
Posts: 2520
Joined: Fri Apr 01, 2005 11:24 pm
Location: Germany

Postby Alex » Thu Mar 08, 2012 11:53 am

Hi Hans,

I explain:

I have my application in French, my application has 2462 sentenses like Menu, Custom, Exit, Do you really want to exit?, Load a file... I put them all into a file called TEXT because if I want my application in english, then I just have to translate the TEXT file and the application is in english.

I put each sentences from the TEXT file into variables because it is for the application.

I am searching a way to load my 2462 sentences from TEXT to Neobook variables ([TEXT1], [TEXT2], [TEXT3]...) for the applications very quick.

Thanks


Alex
Alex
 
Posts: 745
Joined: Sat Mar 04, 2006 4:53 am

Postby Neosoft Support » Thu Mar 08, 2012 11:57 am

What about the LoadVariables action?
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5603
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

Postby Alex » Thu Mar 08, 2012 12:26 pm

Hi,

I tried, it is very slow and my laptop's fan turned ON.

Any other ideas?

Thanks


Alex
Alex
 
Posts: 745
Joined: Sat Mar 04, 2006 4:53 am

Postby dpayer » Thu Mar 08, 2012 1:48 pm

Alex wrote:Hi,
I tried, it is very slow and my laptop's fan turned ON.
Any other ideas?
Thanks
Alex


Show an exciting video for people to laugh at while the data is loading. They will forget how long it took!

:) :lol:

David P.
User avatar
dpayer
 
Posts: 1383
Joined: Mon Apr 11, 2005 5:55 am
Location: Iowa - USA

Postby HPW » Thu Mar 08, 2012 2:59 pm

Hallo,

I made a little benchmark with a testfile with 2500 lines.

Using newlisp to read each line into a variable: 15 sec.
SaveVariables to write all variables to disk: 80ms
LoadVariables to read back variables from disk: 38 sec.

Code: Select all
TimerStart "Timer3" "1000"
hpwNewLispCall "(setq in-file(open [#34][nlPubdir]Testfile.txt[#34] [#34]read[#34]))(setq linecount 1)(while (setq linestr(read-line in-file))(nbset (string [#34]TEXT[#34] linecount)linestr)(inc linecount))(close in-file)" "[DllRetvar]"
TimerStop "Timer3"

TimerStart "Timer3" "1000"
SaveVariables "!C:\Programme\NeoBook5\Plugins\hpwNewLISP\Testfile1.txt"
TimerStop "Timer3"

TimerStart "Timer3" "1000"
LoadVariables "!C:\Programme\NeoBook5\Plugins\hpwNewLISP\Testfile2.txt"
TimerStop "Timer3"


I wonder about LoadVariables.

Hans-Peter
Hans-Peter
User avatar
HPW
 
Posts: 2520
Joined: Fri Apr 01, 2005 11:24 pm
Location: Germany

Postby HPW » Fri Mar 09, 2012 12:48 am

Testing on my faster office quad core 2.8GHZ:

Using newlisp to read each line into a variable: 5 sec.
SaveVariables to write all variables to disk: 0ms (too fast to measure)
LoadVariables to read back variables from disk: 9 sec.

To be fair Loadvariables does an individual varname instead of a numbered varname. So I add a newLISP alternative:

Table in testfile.lsp:
Code: Select all
(setq inlist (list
(list"TEXT1"    "Testline0001")
(list"TEXT2"    "Testline0002")
..
..
(list"TEXT2499" "Testline2499")
(list"TEXT2500" "Testline2500")
)
)


Code: Select all
TimerStart "Timer3" "1000"
hpwNewLispCall "(silent(load [#34][nlPubdir]Testfile.lsp[#34]))(silent(dolist (linestr inlist)(nbset (linestr 0) (linestr 1))))" "[DllRetvar]"
TimerStop "Timer3"


Using newlisp to set each list into a nb-variable: 5 sec.
So consistent between serial variable and individual variable.
Hans-Peter
User avatar
HPW
 
Posts: 2520
Joined: Fri Apr 01, 2005 11:24 pm
Location: Germany

Postby HPW » Fri Mar 09, 2012 1:29 am

I made a experimental new command:

Code: Select all
hpwFileToVar "!C:\Programme\NeoBook5\Plugins\hpwNewLISP\Testfile.txt" "MyPrefix"


Which does a serial variable with the MyPrefix-String with each line in textfile.
I can not think of a faster plugin function using nbSetVar.
So for the textfile on my quadcore it needs 4.8 sec

It is still a bit out-performed by the newlisp solutions with 4.4 sec

So for users who do not want to include newLISP I could add this command to hpwUtility.

Interested?
Hans-Peter
User avatar
HPW
 
Posts: 2520
Joined: Fri Apr 01, 2005 11:24 pm
Location: Germany

Postby Luiz Alfredo » Fri Mar 09, 2012 9:18 am

It wll be good Hans. Thanks.
L.A.G.M.
Luiz Alfredo
 
Posts: 195
Joined: Thu Apr 19, 2007 6:58 am
Location: Brazil

Postby Alex » Fri Mar 09, 2012 9:56 am

Hi HPW,

Thank you very much for all your try.

I would like to try your new function into HPWUtility

David, your joke is cool, thanks.


Alex
Alex
 
Posts: 745
Joined: Sat Mar 04, 2006 4:53 am

Postby HPW » Fri Mar 09, 2012 10:46 am

Try out:

http://www.neosoftware.com/forum/viewto ... 2#11205532

By the way: Did you consider not to load all data on startup.
Maybe you can load on demand: On page 1 the data for page 2 etc.
Hans-Peter
User avatar
HPW
 
Posts: 2520
Joined: Fri Apr 01, 2005 11:24 pm
Location: Germany

Next

Return to PlugIn Discussions

Who is online

Users browsing this forum: Yahoo [Bot] and 1 guest