English
Chinese (Simplified)
French
German
Italian
Japanese
Russian
Spanish
Vietnamese
Antivirus update - Could not connect to any server - FAILED
-
Hello to all!
I will be brief: since two days ago (July 10th 2023, afternoon EET), on several of our customer's MDaemon servers (versions 21.x to 23.0) I've encountered lots of mail processing issues, with imposibility of updating ClamAV definitions.F
For example, on one of the installs (23.0.0), it started like this:
#Update session at 17:03:39 on 2023-07-10
Date Time Action Status Object
2023-07-10 17:03:39 -- SPUPDATER START --
2023-07-10 17:03:39 Version 23.0.0
2023-07-10 17:03:39 Subscription active: 189 days left
2023-07-10 17:03:41 Definition version: 202307101133
2023-07-10 17:03:42 Download file OK https://files.mdaemon.com/antivirus/ctav/updates/antivir-v2-i-202307101133.zip
2023-07-10 17:03:44 Install file OK E:\MDaemon\Queues\Temp\CTAVWORK\20230710170342\antiviri-v2.def
2023-07-10 17:03:45 New definition version: 202307101245
2023-07-10 17:03:45 Virus definitions updated successfully.
2023-07-10 17:03:48 -- SPUPDATER END --#Update session at 18:03:40 on 2023-07-10
Date Time Action Status Object
2023-07-10 18:03:40 -- SPUPDATER START --
2023-07-10 18:03:40 Version 23.0.0
2023-07-10 18:03:40 Subscription active: 189 days left
2023-07-10 18:03:41 Definition version: 202307101245
2023-07-10 18:03:42 Could not connect to any server. FAILED
2023-07-10 18:03:42 Update process failed!
2023-07-10 18:03:42 -- SPUPDATER END --And, two days later:
#Update session at 23:03:13 on 2023-07-12
Date Time Action Status Object
2023-07-12 23:03:13 -- SPUPDATER START --
2023-07-12 23:03:13 Version 23.0.0
2023-07-12 23:03:13 Subscription active: 187 days left
2023-07-12 23:03:14 Definition version: 202307101245
2023-07-12 23:03:15 Could not connect to any server. FAILED
2023-07-12 23:03:15 Update process failed!
2023-07-12 23:03:15 -- SPUPDATER END --It's happening with both ClamAV enabled and disabled.
Email processing was affected by this (CFilter crashing constantly, messages dissapearing from local queue as per routing log ...
Please advise!
PS: yes, I've seen that, since 23.0.1, Cyren was replaced, I will update one of the servers, and revert with info. But, still, this does not seem normal.
PPS: all installs running on Windows Server 2012 to 2022, with MDaemon and Security Plus licensing from 25 to 130 users (hope I am not mistaken). ALL installs are way in their licensing valid timings.
Thank you!
TM
-
Tyler Staff
@Tiberiu
Cyren's AV engine was replaced because they are no longer in business. Virus definitions are no longer available.
Please update MDaemon to a version that includes IKARUS.
https://mdaemon.com/pages/downloads-critical-updates
ClamAV virus updates can be found in updated versions of MDaemon by selecting Security > AntiVirus > AV Updater > ClamAV Updater > View Update report or in older versions by opening the lastupdate.log file in the \MDaemon\SecurityPlus\ClamAVPlugin\logs directory.
-
@Tyler
Thanks for your input.
Yes, I knew about this but, as I was mostly with work out of the country for the last months, I preferred not to do any remote updates.
The problem is not that the installs are not doing part of their AV updates, but that there are quite a lot of messages during previous two days or so that are just simply gone, they disappeared.
More specific: there are messages that are accepted for delivery, with error in AV inline scan, like below:
Wed 2023-07-12 xx:xx:xx.xxx: 12: * Message-ID: <message-id@id-tld>
Wed 2023-07-12 xx:xx:xx.xxx: 12: * xxxxxxxxxxxxxxx Size: 92467; <e:\mdaemon\queues\local\mdxxxxxxxxxxxxx.msg>
... and then nothing. They just vanish, no other refference about the same message-id, no mdxxxxxxxxxxxxx.msg anywhere (nor pdxxxxxxxxxx.msg), the message is simply lost, it was not even saved as an archive item. There is no physical way in recovering those messages for the customer apart of asking sender to re-send.
And this is probably because, at the same time (but almost every minute) there is a:
Wed 2023-07-12 xx:xx:xx.xxx: CFENGINE.EXE was not running. It was restarted.
The interesting part is that, although CFENGINE.EXE is throwing errors constantly, only for a specific period of time (like Wednesday afternoon) there is a bunch of messages lost on at least two of the servers (I did not manage yet to go deep in the logs of the other two, this will be a nice and relaxing week-end job for me 😲
The worst, apart from loosing quite few messages, is that there is no way in easily identifying which one is lost, and which one managed to get through. I already modified a parsing script so it will find all orphan message-id's, and will work from there.
If you have any other ideas, I would be grateful!
PS: Yes, I did update all servers to 23.0.2 last night, as wrote above, and this seemed to solve the issues (in fact, the CFilter saga was solved, apparently, after a simple MDaemon restart.
PPS: A big issue is also that, although I am running several customer's MDaemon installs, I did not receive any notification that something might be wrong. Or at least that Cyren is out of business, and it will be disabled / replaced etc. Nothing from MDaemon to warn me about this. Which is kinda ... unpleasant.
Thanks for input!
-
Tyler Staff
@Tiberiu
I'm sorry to hear about this. If the CFEngine process is not running, messages cannot be processed. If the CFEngine.exe process is working fine in MD 23.0.2, it may have been related to the fix applied to CFEngine in 23.0.2 related to the content filter.
https://files.mdaemon.com/mdaemon/release/RelNotes_en.HTML
[26997] fix to Content Filter - Possible crash in CFEngine.exeI would also suggest if you have the Windows Defneder or third party security application on the server to exclude the \MDaemon directory to rule out any future interference when CFEngine processes messages through the Spam/Antivirus filters.
As for reviewing messages, if you open the MDaemon GUI select Queues > Queue and Statistics Manager. Click on the Log page, click on Open Log, and open the MDaemon-YYYY-MM-DD-SMTP-(in).log file on the day you want to query. The Q&S tool will parse the SMTP inbound log and display all messages processed, successful and terminated sessions.
Perhaps it was overlooked, but we sent out the Critial Update detailing the changes to MDaemon and SecurityGateway on April 18th to our direct customers and channel partners.
-
@Tyler: good morning (at least in my part of the world! :) ).
One-by-one:
- CFilter: part of the machines were already on 23.0.0, one of them was on 25.x (lat x32 version I think). I'd go more for the AV (not being able to) process, thus interfering with CFilter, and crashing it. The problem is that there is no "alert" for postmaster that something is wrong. It would be nice to have it (let's say as a checkbox for: "Dude, CFilter just crashed on mail.xyz.tld").
- AV: defender / server AVs are excluded from MDaemon and MDaemon related folders / files. I am dancing with MDaemon since 2000 (version 3.x, I think), and I learned this the hard way long time ago. :)
- reviewing messages in Q&S, except when you have SMTP server monitoring from multiple distributed monitors / locations, and practically there are SMTP sessions every 30 seconds or less, this is when it becomes really hard to differentiate the good ones to the bad ones. That's why routing log is more helpful for this kind of situation.
- The worst part of it is that, MDaemon being a really admin-friendly by organizing everything in folders and files, and not through databases, we are still losing messages from time to time (they just disappear!), when something wrong is happening. And this should not happen: once a message is accepted after a SMTP session, and transfer complete is sent to the other party (thus the sending party is genuinely thinking that it's message got through), as the file is fed to local queue, I should be able to find a copy of that message, no matter what.
Look, a brief example from the recent crashes (I've obfuscated private data):
Wed 2023-07-12 12:38:16.423: 12: ----------
Wed 2023-07-12 12:43:07.388: 17: INBOUND message: md50001474481.msg
Wed 2023-07-12 12:43:07.388: 17: * From: EXTERNAL EMAIL ADDRESS
Wed 2023-07-12 12:43:07.388: 17: * To: INTERNAL USER
Wed 2023-07-12 12:43:07.388: 17: * Subject: SUBJECT
Wed 2023-07-12 12:43:07.388: 17: * Message-ID: <unique-message-id>
Wed 2023-07-12 12:43:07.388: 17: * Size: 6049138; <e:\mdaemon\queues\local\md50002786172.msg>
Wed 2023-07-12 12:43:07.388: 17: ----------There is no follow on that, the msg file is lost, I cannot identify it ANYWHERE (and, trust me, I've looked for it!). The **50002786172.msg is nowhere to be found, and the server is not even knowing that something is wrong with that.
And this is where I think (and hope) that something should (and can) be done. I am not a programmer (not a skilled one, at least), but I think that some kind of cross-checking (and a backup copy of the file, like - I don't know - a holding / Temporary queue copy) might be made before cfilter processing, so we can recover at least the original transmission, and re-route it manually to interested parties (or - at least - to ask the external party to re-send it).
This might add some overhead on specific servers (thus the possibility of disabling this by a checkbox / MDaemon.ini), but normally it should work with most of the installs.
I repeat, this is not a single time issue, it happens from time to time (once a month?! maybe more?) for each of theseveral completely separated installs of MDaemon I care of, so ...
I can work with you on this, also in private, if your are willing to get to the bottom of it.
Thanks,
TM
-
Tyler Staff
@Tiberiu
Messsages disappearring from directories without anything logged is an indication of something removing them, such as the Windows Defender or a third party security/monitoring solution grabbing files from MDaemon's directories. Or there could be issues with the disk MDaemon is writing to. Is there anything logged in the Windows Logs in the Windows Event Viewer when the message disappears? Anything related to MDaemon?
MDaemon will write something to its logs if the message was removed or re-routed or moved to a different queue due to an error. For example, when enabled, messages are sent to the holding queue if there are issues when processessing through the AntiSpam, AntiVirus, and/or Content filter engines. Is the Holding Queue enabled on your server?
https://help.mdaemon.com/mdaemon/en/queues_holding_queue.html
I would check the Content Filter, Routing, AntiSpam, and AntiVirus logs to verify the message is not being moved or deleted. Does your message appear in all of these logs? It's also worth noting, if the message is a mailing list message, when the message is broken up into individual messages for users, the Message-ID will not be the same.
If there's something you can reproduce or would like to send for us to inspect that you do not want posted in the forums, please feel free to submit a support request using our support portal.
