Source language: Translate to:

memory issue

General questions about NeoBook

Moderator: Neosoft Support

Re: memory issue

Postby Gaev » Sun Jan 19, 2014 6:58 am

Fred:

1) Thank you for the subroutine code ... but what I was asking in (a) was the code/logic that is used to (repeatedly) invoke this subroutine i.e. is there a Timer that invokes a GoSub every so often or is there another criteria/event that causes this to happen ? ... I don't need to see all of the "severe digestable spagetty like" code; just an explanation will do.

2) Regarding the BATch file ...

a) not sure about the purpose of the ping localhost -n 10 >nul command ... is it some kind of delay mechanism ?

b) does the file meetgegevens.txt accumulate all the readings ? ... or does each iteration of the loop write over the previous reading ? ... note that if it does not, you can use the Type command to do so ...

Type meetgegevens.txt>>accumulated.txt

... so one resolution (workaround) would be to ...

- invoke the BATch file just once
- when required, fetch the accumulated contents
- enhance the BATch file to check for the presence of a stopnow.txt file ... and stop the loop when found


But still a useful exercise to find the root cause of the "memory loss" ... happy hunting.
User avatar
Gaev
 
Posts: 3737
Joined: Fri Apr 01, 2005 7:48 am
Location: Toronto, Canada

Re: memory issue

Postby frednet » Sun Jan 19, 2014 10:38 am

I have brought the program down to this:
Code: Select all
:lus
While "[meet]" "=" "1"

Delay "1000"
. measure temperatures and put data in inputfile
Run "digitemp" "-o2 -a -q -l[naaminputfile]" "RunOnce+Wait+Hidden" "" ""
Delay "4300"
endwhile

Return


Where [Meet] and [naaminputfile] are set at the start of the book

As David suggested I also ran this version:
Code: Select all
:doslus
While "[meet]" "=" "1"
Delay "1000"
. measure temperatures and put data in inputfile
DOSCommand "digitemp.exe" "-o2 -a -q -l[naaminputfile]" "RunOnce+Wait+Hidden"
Delay "4300"
endwhile
Return


Both are called from a simple gosub through a button command and show the same memory loss.

However i stumbled on something I accidentally made a long pause in the loop about ten seconds and now, it seems that the leaking is far less than to be expected seen the number of measurements taken. I'll have to test more to see what happens.
Fred
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue

Postby frednet » Sun Jan 19, 2014 1:24 pm

Gaev wrote:Fred:

1) Thank you for the subroutine code ... but what I was asking in (a) was the code/logic that is used to (repeatedly) invoke this subroutine i.e. is there a Timer that invokes a GoSub every so often or is there another criteria/event that causes this to happen ? ... I don't need to see all of the "severe digestable spagetty like" code; just an explanation will do.


Sorry Geav, I missed your response because it was on the second page.
from one do while loop several gosubs are called, the last one is for the timing;
Code: Select all
:wacht
.test laatste digit voor 0
SetVar "[LastDigit]" "1"
While "[LastDigit]" "<>" "0"
SetVar "[myTime]" "[Time24]"
StrLen "[myTime]" "[myTimeLen]"
SubStr "[myTime]" "[myTimeLen]" "1" "[LastDigit]"
Delay "20"
endwhile
Return

the delay 20 is to reduce processor load.

2) Regarding the BATch file ...

a) not sure about the purpose of the ping localhost -n 10 >nul command ... is it some kind of delay mechanism ?

Yes it is
b) does the file meetgegevens.txt accumulate all the readings ? ... or does each iteration of the loop write over the previous reading ? ... note that if it does not, you can use the Type command to do so ...

Type meetgegevens.txt>>accumulated.txt

... so one resolution (workaround) would be to ...
- invoke the BATch file just once
- when required, fetch the accumulated contents
- enhance the BATch file to check for the presence of a stopnow.txt file ... and stop the loop when found

But still a useful exercise to find the root cause of the "memory loss" ... happy hunting.


The file meetgegevens.txt accumulates all readings I can even give the digitemp command an attribute to make it measure once every ten seconds. And if all else fails that is a method to go back to. But its not so practical because I want to synchronize the digitemp data with the velleman data. (see topic on velleman pcs10)

An other option would be the creation of another neobook plugin from the software here: http://www.digitemp.com/software/windows/dtreg17.zip
But i have no idea how hard that is.
Also there is an other hardware option to gather the data from digitemp sensors.
the usb device below is also capable of sensing other data sources.
http://www.ipcas.com/products/usb-input-device.html
Perhaps the drivers can be reprogrammed to become a neobook plugin.
regards Fred.
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue

Postby Neosoft Support » Mon Jan 20, 2014 12:09 pm

Using Fred's code, I created a simple pub with NeoBook 5.8.4 to check for memory loss. I ran the pub and monitored its stats in Task Manager, but I did not detect any change in memory usage while the pub was running.

I didn't have a copy of Fred's digitemp program so my example simply runs the DOS dir command.

Using a Timer instead of While loop would be much more efficient, so I also included an alternate Timer option in the pub.

You can download a copy of my test pub here: http://www.neosoftware.com/software/pubs/MemTest.zip

The only explanation I can think of for the memory loss would be if Fred's subroutine was somehow being executed multiple times concurrently which would cause the program's stack memory to be consumed. If this was happening, you would see commands being nested in NeoBook's debugger.
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5605
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

Re: memory issue

Postby frednet » Mon Jan 20, 2014 3:24 pm

Neosoft Support wrote:The only explanation I can think of for the memory loss would be if Fred's subroutine was somehow being executed multiple times concurrently which would cause the program's stack memory to be consumed. If this was happening, you would see commands being nested in NeoBook's debugger.

The digitemp hardware is not on my Neobook computer because it has no serial comports anymore.
So I don't know what the debugger does.
I will run the testpub you kindly made in both versions on the test system. And its also interesting to see what it does with the dir command you used.

Testing will take some time but i''l report a.s.a.p.

Thanks Fred.
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue test report

Postby frednet » Wed Jan 22, 2014 1:51 pm

In short: Digitemp or the dir command, run from within Neobook, loop or timer method. In about 6 to 8 hours it eats all memory. Ive put up some screens hots to illustrate. The testsystem is ten yer old laptop with half the memory removed, the system is running xp.

here is an image of the first test digitemp in the timer mode test
Image
the red line is Mbytes available, the green line is non paged.

this is loop mode
Image
timermode seems more efficient because the system tries to free up memory.

To my surprise the dir command showed the same effect.
Image

Because of the intensive processor usage of digitemp I will test what happens if the pause cycle becomes larger. Also the windows setting to use more horse power for the system instead of the programs running is something that remains to be tested.
A final workaround would be to restart the neobook pub by calling another pub.
exiting the pub gives back the lost memory.

Thanks Fred.
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue

Postby mishem » Thu Jan 23, 2014 4:06 am

You can try to identify memory leaks by using HeapMemView.
mishem
 
Posts: 575
Joined: Mon Oct 08, 2012 1:51 pm

Re: memory issue

Postby Neosoft Support » Thu Jan 23, 2014 11:28 am

I didn't run the test app for 6 to 8 hours. I'll try that here and see what happens.
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5605
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

Re: memory issue

Postby Neosoft Support » Fri Jan 24, 2014 11:56 am

I ran the sample pub for about eight hours and I was able to detect a very small, but measurable, memory leak. We'll need to look through the code used by NeoBook's Run action to identify the source of the leak. Once we find the source, we will need to release an update to correct the problem. There are a few other issues that have surfaced recently that we will want to take care at the same time. Unless something unexpected occurs, we should have a fix for you within the next two weeks.

Thank you for your patience in helping us understand and reproduce this problem, and I apologize for the inconvenience.
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5605
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

Re: memory issue

Postby frednet » Sun Jan 26, 2014 2:29 pm

I have tried heapmemview, but ive no idea what i'm looking at, its beyond my knowledge.
I'm glad you could reproduce the problem and look forward to an update. For the time being restarting the application works as a workaround.
Also thanks for your efforts, Fred
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue Solved

Postby frednet » Thu Mar 13, 2014 3:32 am

:)
A big smile on my face because after almost two days running the application there is absolutely no memory loss anymore!
The project now moves on to the next step which is controlling the complex central heating system it has been monitoring.
The digitemp sensors- and the velleman- interfaces work beautifully together.
The Neobook platform now offers me a system on which one can easily program several datalogging and system control apps.
Dont forget not only the the psc10 datalogger has al neobook plugin, but also the velleman vm 110 interface card has a neobook plugin. The Velleman and the Digitemp interfacing combined with Neobook offer a very versatile means of logging and controlling, that is easy to program.
Thanks very much to support and the others that helped point out and solve the problem.
Fred.
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue

Postby Neosoft Support » Fri Mar 14, 2014 12:21 pm

Thanks for the update. I'm glad that we were able to correct this problem.

Your description of using the psc10 datalogger and the velleman vm 110 interface card sounds very interesting. I wonder if the manufacturers of those devices would be interested in learning about NeoBook?
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5605
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

Re: memory issue

Postby frednet » Mon Mar 17, 2014 9:17 am

Neosoft Support wrote:Thanks for the update. I'm glad that we were able to correct this problem.

Your description of using the psc10 datalogger and the velleman vm 110 interface card sounds very interesting. I wonder if the manufacturers of those devices would be interested in learning about NeoBook?


Hard to say They have a responsive forum.
I have posted this topic there http://forum.velleman.be/viewtopic.php?uid=22851&f=1&t=11338&start=0&sid=a6247b6d89c873603ba6d4aaca3dc2a3

And of course it would be a good idea for Neobook to contact Velleman I see some Ipad apps appearing for some of the boards they make. Adding a more versatile windows based platform like neobook would certainly help them to sell more boards.

By the way the logger called the external digitemp routine for some 53.000 times in six days time and not a byte leaked away :)
User avatar
frednet
 
Posts: 83
Joined: Fri Sep 23, 2005 4:29 am
Location: netherlands

Re: memory issue

Postby dpayer » Mon Mar 17, 2014 11:03 am

Neosoft Support wrote:Thanks for the update. I'm glad that we were able to correct this problem.

Your description of using the psc10 datalogger and the velleman vm 110 interface card sounds very interesting. I wonder if the manufacturers of those devices would be interested in learning about NeoBook?



Dave,

As you are considering the future of NB, I would definitely consider a new section of the Neosoftware website that has a strong business/IT professional focus. Put examples of how the product can be used on that level. This use as a control application is perfect for that. Even consider getting a video interview with the author of this product and have him explain WHY he chose NB to do it.

David P.
David Payer
Des Moines, Iowa
USA
User avatar
dpayer
 
Posts: 1384
Joined: Mon Apr 11, 2005 5:55 am
Location: Iowa - USA

Re: memory issue

Postby Neosoft Support » Mon Mar 17, 2014 11:41 am

I would definitely consider a new section of the Neosoftware website that has a strong business/IT professional focus. Put examples of how the product can be used on that level. This use as a control application is perfect for that. Even consider getting a video interview with the author of this product and have him explain WHY he chose NB to do it.


That's a good idea.

Fred, would you be willing to send us a screen capture of your program for our website?
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5605
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

PreviousNext

Return to General NeoBook Discussions

Who is online

Users browsing this forum: No registered users and 3 guests