Source language: Translate to:

I wouldn't wish this situation on anyone

General questions about NeoBook

Moderator: Neosoft Support

I wouldn't wish this situation on anyone

Postby Enigman » Fri Jan 15, 2016 11:41 am

I had a situation develop two days ago that really let the air out of the balloon of my enthusiasm for developing software for the public in today's tech environment.

I am posting this thread for informational purposes for the benefit of anyone developing software so that you can watch out for anything like this. I honestly don't know what anyone could do about it, but it is what it is and I wish you all the best.

I should start by saying that I have worked with and loved NeoBook since it's early DOS days when it enabled me to create an elegant pictorial database of tropical fish, and distribute that software on my dialup BBS system called, "The Fish Tank BBS". Since then I have kept using NB to make utilities for companies I worked for in engineering and IT, and then to develop my own personal and commercial software products. NeoBook is a great system and I will continue to use it for my own software tools as long as I can see the screen and type on keyboards, which may not be long given what is happening with computing in general, but that's another story.

So here we go ...

I have had a software product, let's call it "XYZ", that I have produced as freeware for about 17 years. I offer that free software utility to promote my other software as well as let people know about my physical product stores. It has been relatively popular by single developer standards and over the years it has been picked up by hundreds of download sites around the world. All of those sites have pointed back to my domain and a download URL on my server that has been hosted with the same web hosting company for most of that time.

Two days ago I received a batch of e-mails with the same timestamp notifying me that my server had been shutdown. One of the e-mails sited the reason as "hosting malware". WHAT!!! No freakin way. I jumped on the phone to the hosting company, bounced through three departments and ended up in the "Terms of Service" department where I had the distinct pleasure of talking to a tech who had the attitude that I was some kind of criminal. I asked what was going on and he notified me that one of my files was identified as containing a "HEUR/QVM06.1.Malware.Gen" malware infection. He then e-mailed me a link to his "authority" for the situation, so while still on the phone I looked at the link.

It turned out to be a link to a data mill that stores essentially unsubstantiated reports of possible infected files and/or websites. For the sake of being generic, we'll call my file "XYZ_Setup.exe". The data mill listed 66 other reporting data mills. Out of that list, 2 of the 66 had branded my file URL as a "malicious website". When I clicked the link to find out why, I was given another list of 53 virus scanner development companies. That list contained all of the big names in virus scanning like Kaspersky, McAfee, Symantec, etc. out of the list of 53, only ONE company was listed as reporting an infection and that company was Qihoo-360. I should have known who it was by the 360 in the name, but I looked it up online just the same. Qihoo-360 is part of the Chinese 360.cn domains. In other words, it's a Chinese company making virus software for the 360 systems. (Don't get me started on Chinese businesses)

So back to the terms of service tech, I explained that what we have here is a false positive report from a podunk outfit of dubious skill, "... and the very thing that you just sent me as your reason for killing my server is the primary indicator that it is a false positive. Why? Because 52 of the 53 companies listed, mostly known big players, have determined that the file is clean. Next we have the fact that they picked on only XYZ_Setup.exe as positive when I also have a binary duplicate file called XYZ14.exe in the same directory. If the report had been the result of a physical file scan of my server, then they would have had a hit on XYZ14.exe as well. But they did not. XYZ14.exe is file name that is in many many really old download sites and I cannot find them to change the link, so any time I update XYZ_Setup.exe, I also copy the file to XYZ14.exe and upload that as well. Statistics show that both are still being downloaded. Last but not least, I scan everything before uploading so nothing went up to your server with an infection in the first place. So assuming that there even WAS an infection, which there wasn't, that leaves us with a situation where it got infected on YOUR server, so that would be ... let's see ... oh yeah ... YOUR problem. Do you even do physical scans of your servers?"

"Well, no ... we just rely on reporting agencies."

"Really? Does anyone even look at the report you sent me and apply any intelligence to the situation before wholesale disabling a server account with thousands of files?"

"Well, I won't debate this issue. You must delete the file before we can enable the server, and if you upload the file again we will delete your account."

"I see, and what is the real fix here. My file is referenced on hundreds of download sites so I cannot just rename the file to something else. The filename must be what it was on the URL that it was or I am just dead in the water."

"Like I said, I will not debate this. Even if clean, if the file is uploaded the reporting agency will hit on the filename and report it again and then we will delete your account without notice."

"So basically you are giving me no recovery option, all on the word of some unknown Chinese software outfit, when everyone else with a known reputation says the file is fine."

"Yes."

Click.

How special.

Here is the main problem with server farms relying on reporting agencies. Years ago virus scanner software started including a one-button option to report a file that was hit on a users computer as containing anything improper. The software defaults that report to a variety of reporting data mills. Personally, I always disable that option because I know that a percentage of scan hits will be false positives. I actually dropped McAfee years ago because their false positive percentage was so high. I didn't want to create a situation like I am experiencing right now. And yet other users would take that false positive as gospel and let it be reported.

My first call with Terms of Service left me with such a bad taste that yesterday I called the manager of the TOS department and ran down the whole story. After reviewing their records, he agreed that what we have is most likely a false report. But that doesn't really help me. Since the report exists out there in unchangeable data world, any reappearance of the same file will trigger another report, and if that happens then the hosting company is at risk for being blacklisted. My option is to reupload the file under another name or just encase it in a ZIP file which never seem to be reported with false positives. EXE files are often a problem in false reports. I explained why that is not useful and thanked him for climbing up the backside of the first guy over his "guilty until proven innocent" attitude.

So here's the Catch 22 reason's why this is unrecoverable for me "within reason":

1) There are hundreds of download sites looking for an exact URL and filename to download and I cannot fix those download links. I tried a while back, but in the course of 17 years of history, the sites have changed hands, been sold, moved to other URLs and so on and my login accounts no longer work or exist, yet the download listing still exists.

2) If I just reupload the file, it will be reported again simply on the filename and I risk losing my entire server for all software and image files as well as e-mail services.

3) If I move the domain to another hosting company and recreate the URL and file, then their report scanning operations will eventually find the false positive report and I'm back in the same boat.

4) If I just say "screw it" and move the file and rename it, then downloads drop to virtual zero and the exercise is pointless.

5) If I use an htaccess file to redirect the download to another location, then either download sites or personal firewalls or internet security programs will react negatively to the bait-n-switch appearance, and I risk more bad press with worse consequences.

6) The original reporting company is in China and unreachable for this kind of thing and even if I could they would not change their report on my say so. Even if I convinced them to scan the URL and verify that it is clean, I'd have to upload the file on the correct URL, and we're back to 2 and 3 above.

7) It's freeware, so exhaustive hocus pocus is not cost effective.

My solution at the moment is to post a message on the software page that the software is no longer publicly available, but still has tech support for existing installations. In other words, this product is boned, all over some yayhoo in gawd knows where that pushed a "report this" button or something equally ludicrous.

My caution to all of you is that if you haven't already, put your installation EXE files into a ZIP, tar or rar file and don't upload the raw EXE installer. I hope nothing like this happens to any of you.
User avatar
Enigman
 
Posts: 314
Joined: Tue Apr 12, 2005 3:57 pm
Location: Foothill Ranch, CA

Re: I wouldn't wish this situation on anyone

Postby stu » Fri Jan 15, 2016 12:23 pm

I dont get it, why not just move from hosting?
User avatar
stu
 
Posts: 318
Joined: Wed Aug 07, 2013 11:37 am

Re: I wouldn't wish this situation on anyone

Postby Enigman » Fri Jan 15, 2016 12:37 pm

stu wrote:I dont get it, why not just move from hosting?

See number 3 above.

Moving the hosting to another company does not solve the problem. That will only last until the new host's system finds the false positive report and then I'm back to square one.
User avatar
Enigman
 
Posts: 314
Joined: Tue Apr 12, 2005 3:57 pm
Location: Foothill Ranch, CA

Re: I wouldn't wish this situation on anyone

Postby Neosoft Support » Fri Jan 15, 2016 8:47 pm

That's infuriating!

Unfortunately, to the average user a false positive might as well be as a real virus. It give the impression that the anti-virus software is "protecting" you from something bad. Why should AV vendors spend time and money identifying real threats when they can just guess (heuristics) or outsource the problem to their users (reputation-based). Just throw up warning messages periodically, pat yourself on the back, and no one knows the difference?

The trend toward heuristic and reputation-based AV software is a disaster for small independent developers. With reputation-based AVs, any small vertical market product that hasn't been downloaded millions of times won't have a reputation! To the AV software, a program with NO reputation is no different than a program with a BAD reputation. Guilty until proven otherwise. Who would install a program after their AV displays a scary message warning them not to? "No one seems to be installing this software - why take the chance?" And no one ever will if you show them that message! Catch-22.

I think there's a class action lawsuit here somewhere. Millions of little software publishers unfairly slandered by lazy, careless AV vendors!
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5593
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA

Re: I wouldn't wish this situation on anyone

Postby Enigman » Fri Jan 15, 2016 10:53 pm

Neosoft Support wrote:That's infuriating!

Unfortunately, to the average user a false positive might as well be as a real virus. It give the impression that the anti-virus software is "protecting" you from something bad. Why should AV vendors spend time and money identifying real threats when they can just guess (heuristics) or outsource the problem to their users (reputation-based). Just throw up warning messages periodically, pat yourself on the back, and no one knows the difference? ....

I think there's a class action lawsuit here somewhere. Millions of little software publishers unfairly slandered by lazy, careless AV vendors!

Amen to that.

This is also like the e-mail SPAM reporting agencies. Any ninny can report any e-mail source as SPAM, right or wrong, and have that report silently relayed to blacklisting data mines. Once that happens you're boned. You cannot dispute the listing and the agencies do not verify the validity of it. They just sell the reports to their clients. Who cares if it is accurate?

The IT department of the company my wife works for did exactly that with one of my messages to her over a "language issue" after I relayed a funny story. It wasn't even a report from a human. An automated system did it. After that, their e-mail scanning system picked up their own report on the outside data service and then the same system banned all my e-mail accounts on my entire set of domains so that now I cannot get any e-mail through at all. And their IT department is so lame that they don't know how to undo it, nor do they know how to "white list" the domains to override their screwup.

All of the monetization of paranoia is really getting out of hand. It would be one thing if these companies knew how to handle things with any intelligence or grace at all, but they don't. It's just like you said. They make it look like they are on the job saving us all from harm, but they are not. By not being good or even reasonable at what they do they are doing as much harm as good, but at least they are not harming the people buying their services or scanners, they are just harming those that don't, so who cares.

The thing that really chaps my knickers is that it all started with a Chinese AV company. In 2015 I already had to change my entire approach to self employment through online sales due to market flooding by Chinese sellers with prices a factor of 10 below anything a US seller can do. And now this happens to my other side of self employment ... brought on by yet another Chinese "business". LOL ... like I said, don't get me started on them.
User avatar
Enigman
 
Posts: 314
Joined: Tue Apr 12, 2005 3:57 pm
Location: Foothill Ranch, CA

Re: I wouldn't wish this situation on anyone

Postby Wrangler » Sat Jan 16, 2016 10:00 am

3) If I move the domain to another hosting company and recreate the URL and file, then their report scanning operations will eventually find the false positive report and I'm back in the same boat.


I think I have to disagree with you here. The hosting company you choose can make a big difference. I have had several hosting companies over the years (20) promoting 19 software titles made with NB. Most of these were dedicated servers, a couple just cheap hosting. I've had my fair share of false positives over the years and have always managed to get things fixed. Your situation is a bit more complicated because of the Chinese (don't get me started either). However, the more well known hosting companies are more willing to work with you on this type of problem. The reaction you got from the tech just says that the hosting you use cuts loose their level 1 techs on security problems they aren't trained to handle. Since you can't choose which tech you get, I always call them as opposed to using their ticket system, and if they give me any bullsh*t, I ask them to transfer the problem to a higher level tech. If they DON'T, a simple email to a supervisor takes care of the problem.

Also, the bigger hosting companies don't rely on third party virus lists. They scan their own systems and deal with security problems in-house. You then get an email with their shutdown threats and they temporarily suspend the site, mostly to prevent the malware from spreading to the rest of the server (VPS). You then call them and address the problem. In one false positive incident I had when with Softlayer, they even contacted the blacklist sites for me and got the software delisted.

I guess my point is there are good, and bad, hosting companies. You may have to pay more for a good one, but the cost is well worth it in the long run. You have the most control over your server if you lease a dedicated box. Rob Cohen and I have been doing this for years. Ask Rob about some of the horror stories we endured during that time. But with a good tech support team, we got through it.

Moving your domain to a new server doesn't need to be a daunting task. Many hosting companies offer free migration to the new server, if both accounts use cPanel. I had Turnkey Internet move 22 sites for me this way, without problem. If your current hosting DOESN'T use cPanel, you are losing out. cPanel makes it very easy to maintain your account.

When searching for a new host, google the best hosting sites, talk to your friends, and then call them stipulating the problems you've had in the past with your previous host, and tell them unless they can do better, you don't need their services. If they don't have a tech phone number, just move on. The cheaper the host, the worse the tech is. You will also find some of your old php scripts won't run because they lock down the box so much for security reasons. They get paranoid because you share the server with a thousand others, and they don't want the server compromised. Some older scripts require ini_set be enabled, and this is generally disabled for security reasons.

I can see this is devastating to you, and I understand why. However, should your house ever burn to the ground, what do you do? BUILD A NEW HOUSE. My recommendation is to get yourself a quality hosting account and begin to rebuild your house. If you decide to do so, know that I am willing to help in any way I can. Just PM me.

There is a home for you out there. You just need to find it.
Wrangler
--------------
"You never know about a woman. Whether she'll laugh, cry or go for a gun." - Louis L'Amour

Windows 7 Ultimate SP1 64bit
16GB Ram
Asus GTX 950 OC Strix
Software made with NeoBook
http://highdesertsoftware.com
User avatar
Wrangler
 
Posts: 1505
Joined: Thu Mar 31, 2005 11:40 pm
Location: USA

Re: I wouldn't wish this situation on anyone

Postby Enigman » Sat Jan 16, 2016 12:00 pm

Wrangler,

Thanks for the words of encouragement. I don't disagree with any of your points. However, in this case, my point number 7 above sets the tone for attempts at recovery for this software.

A dedicated server is not warranted for me when the only thing I have on the server is image files and a few software installers. There are no websites on my server, just files. I used to code my own PHP websites years ago, but I have long since rebuilt my websites on Weebly for the ease of maintenance.

I have recently begun to question whether I even need the image file server at all. The images were originally served to my product listings on eBay, Amazon and Etsy. However, over the last 13 years these venues have moved away from promoting or allowing the use of user hosted images and now require sellers to upload images to the venue servers. Only eBay still allows user hosted images in the description block of a listing. So the usefulness of my thousands of hosted images is much less today and I can easily skip the posting of product images in the descriptions when they are already at the top of the listing pages anyway. That leaves just the few software installers and my e-mail services. I can actually host my installers on each Weebly website. I just won't get statistics on downloads so I haven't done that yet. That leaves only e-mail services, which I could move to my domain registrar and cut my 26 accounts down to 3 or 4. So I can conceivably drop my server altogether. Based on that, it certainly doesn't make sense for me to invest more in a new or dedicated server.

I mentioned that I called back and talked to the manager of the terms of service department in my hosting company. That is the kind of thing I normally do when I don't like what I am getting from first line support. It is very rare that anyone in first line support knows even a fraction of what I already know about their system, so I spend a lot of time talking to supervisors. The guy I talked to was definitely more professional, but in the end he only reiterated the kind of policies and options I had already heard. I could have recovered with some of his suggestions, but in my case the big stumbling block was all the aged download sites that I cannot update. I don't have that issue with my other software. Their installers can be moved or renamed at will.

2015 for me was all about reducing and removing unnecessary services. Why? Because of mutant slimey competition from Chinese sellers listing the same things I list for a 10th of the price. (Pteui on them) I had to rethink my product lines and decided to isolate down to the only things the Chinese sellers can't or don't sell, which is things that I make myself. So my inventory levels went from thousands of things that I buy and resell down to a couple of hundred that I make. That required dumping any efforts or services that cost money without offering any real return. That led me to examine my software products.

Early this year I thought about my freeware program and thought maybe it was time to drop that effort since it didn't net me anything. But before I made a move, this thing happened, and I resent having it go out that way at the hands of yet another freakin Chinese twit instead of by my choice. But I'm not going to spend money to resurrect it just so I can cancel it myself. There we are back to point 7 above.

I have also paused to consider my other software which at this point has been whittled down to a screen saver and an as yet unpublished password manager. Hmmmm. Although my screen saver is, I think, well designed and entertaining, it is after all a product for a prehistoric era. Ask anyone today what screen saver they use and they most likely will ask what a screen saver is. My password manager is designed for use on USB drives and apparently most people want password control on their phones and not desktop computers, so I find myself considering dumping a product that is not even released yet.

Such is life I guess. I also retired last year "officially" and that makes me a bit less tolerant of BS situations like the above.

Thanks again for your suggestions.
User avatar
Enigman
 
Posts: 314
Joined: Tue Apr 12, 2005 3:57 pm
Location: Foothill Ranch, CA

Re: I wouldn't wish this situation on anyone

Postby stevec » Mon Jan 25, 2016 10:36 am

I tried to sell an app several years ago, but it went anywhere. It worked, got great reviews, but no money.
I feel your pain.
Likewise, I have kept using NB to make utilities for making my job easier and from the DOS days, it has been a great asset.
Even today, when things break in our network controlled (group policy, you can't do that), my works created with NB works, from XP, 7, 8 and now 10.
A great example is our PC 'discovery' process, log into a website and push buttons, UUUGGGHHHH!!! :twisted: :twisted: :twisted:
What takes them an average of three minutes, I do in seconds, with a small app created in NB. I load software in minutes, not hours and have a zero defect history. :D :D :D
I can't thank the folks here for their input and help.
Steve Christensen
stevec
 
Posts: 223
Joined: Fri Apr 15, 2005 7:33 am
Location: Boise, Idaho

Re: I wouldn't wish this situation on anyone

Postby TMcD » Thu Feb 04, 2016 2:41 am

Is there a way to submit software to the freeware sites that is just a redirect URL?

http://your site.com/freeware1

That would then redirect (server-wise) to the actual file.

I.e. http://actual file.com/dl/real file.exe

Along those lines (and really for future projects), could you redirect to a drop-box URL?

Troy
TMcD
 
Posts: 237
Joined: Sun Apr 10, 2005 11:20 am

Re: I wouldn't wish this situation on anyone

Postby Enigman » Thu Feb 04, 2016 10:31 am

TMcD wrote:Is there a way to submit software to the freeware sites that is just a redirect URL?

http://your site.com/freeware1

That would then redirect (server-wise) to the actual file.

I.e. http://actual file.com/dl/real file.exe

Along those lines (and really for future projects), could you redirect to a drop-box URL?

Troy

What you are describing is called a "DNS Redirect" and that is normally done using a "301 redirect" record on your hosting domain. However, when it comes right down to it, most software download sites already are a redirect URL. Most sites have a link on THEIR domain that goes to the file on YOUR domain. That's already a caution flag for internet security programs, though they may not immediately react to it in a way that the user sees. But, ... if the link then hits a server that bounces the download request yet somewhere else again using a 301 redirect, then the security program rightfully should jump up and stop the process because that is the same behavior as a hijacked DNS record that redirects the user to a real malware or virus file. Therefore, anything like you suggest simply makes the problem worse. (See number 5 in my original post)

The average user today when confronted with a security program warning in a flashing red box uses ... shall we say ... "minimal reasoning skills" ... to put it kindly. They will simply "bahhh bahhh" like good little sheep and press the button to report the assumed attack attempt and thereby potentially destroy someones honest hard work without any reasonable proof of bad intent or a second thought. The makers of security programs have sown the seeds of fear to sell products, and they have done their jobs well. Simultaneously, the average user has given up the desire or responsibility for reasonable decision making, and there we are. We have the climate of ignorance and fear and "SS-like" reporting on ones neighbor that exists today.

As far as drop-box goes, I haven't looked into how that works, but in my current situation, making a change at all is still the problem for all the reasons I outlined.

As for future efforts, a download location that can be moved at will without changing the original link given to download sites, while still being only one "hop", might be useful. Then again, if that was possible, that method would also be employed by actual malware writers, learned by security programs, and then we're right back in warning flag land.

Other download sites insist on hosting the software file themselves instead of having an offsite link. Although those sites do offer a way to have distributed downloads, they are really more of a pain when you have an update and you have to go find them all. That is one thing that happened with this program. The sites and files have been out there so long that I no longer have access or locations for all of them. That's a whole other kind of bad press when the download is completed by a visitor, installed, and then doesn't work because the license is expired.

The real trick for future avoidance of this situation seems to be an old school method of placing your installation software into a ZIP or other archive file and not hosting EXE files directly.

Thanks for the input. :)
User avatar
Enigman
 
Posts: 314
Joined: Tue Apr 12, 2005 3:57 pm
Location: Foothill Ranch, CA

Re: I wouldn't wish this situation on anyone

Postby Neosoft Support » Sat Feb 06, 2016 5:52 pm

The makers of security programs have sown the seeds of fear to sell products, and they have done their jobs well.


Very true!

Large developers can buy, bully or sue their way out of this problem. Small developers are pretty much screwed.
NeoSoft Support
Neosoft Support
NeoSoft Team
 
Posts: 5593
Joined: Thu Mar 31, 2005 10:48 pm
Location: Oregon, USA


Return to General NeoBook Discussions

Who is online

Users browsing this forum: Bing [Bot] and 1 guest

cron