MDaemon Technologies, Ltd.

Matthew Staff

12/31/24, 4:22 PM

@John Hanks I have been off for the holidays, but will take a look at the script. SG stores the allow and block list entries in its database. I will test the script ASAP.

J

John

12/31/24, 4:38 PM

Hi Matthew, I hope you've been enjoying some time off!

Thanks so very much, I apprecaite it.

I'm just not sure how else to update the Allowlists for each user without doing it manually.

Matthew Staff

1/1/25, 9:40 PM

John,

The main change is that the folder name in MDaemon is now "Allowed Senders.IMAP". This updated script should import folders named "WhiteList" and "Allowed Senders". It also includes updated logic to import the "80_MDaemon_welcomelist_from.cf" file to the global allow list and any domain's "Allowed Senders" public contact folder in MDaemon to the matching domain's Allowlist in SG.

################################################################################################################################
################################################################################################################################
## SG Import MD Allowed Senders v 1.1.0                                                                                        ##
## January 1, 2025                                                                                                            ##
## Copyright MDaemon Technologies 2025                                                                                        ##
## This script is designed to retrieve contact email addresses from the user's whitelist and import them into the user's      ##
## SG Whitelist  This script is only to be used with  MDaemon Technologies products. The use of this script for any other     ##
## purpose or with any other products is not allowed.                                                                         ##
##                                                                                                                            ##
## MDaemon Technologies offers no warranty, provides no support, and is not responsible for any damages that may be arise     ##
## from the use of this script. Use at your own risk.                                                                         ##
##                                                                                                                            ##
################################################################################################################################
################################################################################################################################

# Configuration
$UserMailboxBasePath = "C:\MDaemon\Users"
$PublicFolderBasePath = "C:\MDaemon\Public Folders"
$SAFilePath = "C:\MDaemon\SpamAssassin\rules\80_MDaemon_welcomelist_from.cf"
$SGXMLRPCURL = "http://127.0.0.1:4000/SecurityGateway.dll?view=xmlrpc"
$Comment = "Imported from MDaemon"
$LogFile = "C:\MDaemon\Logs\SG_Import_Allowed_Senders.log"
$AllowedParentFolders = @("WhiteList.IMAP", "Allowed Senders.IMAP")  # Allowed parent folder names

# Ensure Log Directory and File Exist
function Ensure-LogFile {
    $LogDirectory = Split-Path -Path $LogFile -Parent

    if (-not (Test-Path -Path $LogDirectory)) {
        Write-Host "Log directory does not exist. Creating: $LogDirectory" -ForegroundColor Yellow
        New-Item -Path $LogDirectory -ItemType Directory -Force | Out-Null
    }

    if (-not (Test-Path -Path $LogFile)) {
        Write-Host "Log file does not exist. Creating: $LogFile" -ForegroundColor Yellow
        New-Item -Path $LogFile -ItemType File -Force | Out-Null
    }
}

# Function to log messages to file and console
function Log-Message {
    param(
        [Parameter(Mandatory=$true)]
        [string]$Message,

        [Parameter(Mandatory=$false)]
        [ValidateSet("DEBUG", "INFO", "WARNING", "ERROR")]
        [string]$Type = "INFO"
    )

    $Timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    $FormattedMessage = "$Timestamp - $Message"

    # Write to log file
    try {
        $FormattedMessage | Out-File -FilePath $LogFile -Append -Encoding UTF8
    }
    catch {
        Write-Error "Failed to write to log file: $_"
    }

    # Console output with color-coding
    switch ($Type) {
        "DEBUG"   { $Color = "Gray" }
        "INFO"    { $Color = "Green" }
        "WARNING" { $Color = "Yellow" }
        "ERROR"   { $Color = "Red" }
        default   { $Color = "White" }
    }
    try {
        Write-Host $FormattedMessage -ForegroundColor $Color
    }
    catch {
        Write-Error "Failed to write to console: $_"
    }
}

# Initialize Logging
Ensure-LogFile
Log-Message -Message "Log directory and file verified/created successfully." -Type "INFO"

# Check for SGAPI module
if (-not (Get-Module -ListAvailable -Name SGAPI)) {
    Log-Message -Message "SGAPI module is not installed. Exiting script." -Type "ERROR"
    exit 1
}

Import-Module SGAPI

# Prompt for credentials securely
try {
    $Credential = Get-Credential -Message "Enter SecurityGateway credentials"
    $OpenSession = Open-SGSession -URL $SGXMLRPCURL -UserName $Credential.UserName -Password $Credential.GetNetworkCredential().Password

    if($OpenSession -ne 0){
        throw "Failed to open SG session. Error code: $OpenSession"
    }
    Log-Message -Message "Successfully opened SG session." -Type "INFO"
}
catch {
    Log-Message -Message "Error opening SG session: $_" -Type "ERROR"
    exit 1
}

$SGUsers = Get-SGUsers -Domain $UserDomain

# Process User Mailboxes
try {
    ForEach($File in Get-ChildItem -Path $UserMailboxBasePath -Recurse -Filter "AddrBook.mrk") {
        # Extract Parent Folder Name
        $ParentFolder = Split-Path -Path $File.FullName -Parent | Split-Path -Leaf

        # Check if the parent folder is allowed
        if ($AllowedParentFolders -notcontains $ParentFolder) {
            Log-Message -Message "Skipping $($File.FullName) as its parent folder '$ParentFolder' is not in the allowed list." -Type "DEBUG"
            continue
        }

        $PathParts = $File.FullName -split '\\'
        $FolderName = ($PathParts[-2] -replace '\.IMAP$', '')

        # Calculate the number of path segments in $UserMailboxBasePath
        $UserMailboxPathDepth = ($UserMailboxBasePath -split '\\').Count

        # Extract UserDomain and UserMailbox dynamically
        $UserDomain = $PathParts[$UserMailboxPathDepth]
        $UserMailbox = $PathParts[$UserMailboxPathDepth + 1]
        $UserEmail = "$UserMailbox@$UserDomain"

        Log-Message -Message "Processing User Folder $UserEmail : $FolderName" -Type "INFO"

        $SGUsers = Get-SGUsers -Domain $UserDomain
        if ($SGUsers.MailBox -notcontains $UserMailbox) {
            Log-Message -Message "$UserEmail not found in SG. Skipping." -Type "WARNING"
            continue
        }

        [xml]$WhiteList = Get-Content $File.FullName
        $Emails = $WhiteList.SelectNodes('//addressBook/contact/email') | ForEach-Object { $_.InnerText.Trim() } | Where-Object { $_ -ne "" }

        foreach ($Email in $Emails) {
         $Result = 0
            #$Result = Add-SGToUserWhitelist -EmailToAdd $Email -UserEmail $UserEmail -Comment $Comment
            if ($Result -eq 0) {
                Log-Message -Message "Added $Email to $UserEmail's allowlist." -Type "INFO"
            } else {
                Log-Message -Message "Failed to add $Email to $UserEmail's allowlist. Error: $Result" -Type "ERROR"
            }
        }
    }
}
catch {
    Log-Message -Message "Error during user mailbox processing: $_" -Type "ERROR"
}

### **Process Public Folders**
try {
    ForEach ($File in Get-ChildItem -Path $PublicFolderBasePath -Recurse -Filter "AddrBook.mrk") {
        $ParentFolder = Split-Path -Path $File.FullName -Parent | Split-Path -Leaf

        if ($AllowedParentFolders -notcontains $ParentFolder) {
            Log-Message -Message "Skipping $($File.FullName) - invalid parent folder ($ParentFolder)." -Type "DEBUG"
            continue
        }

        $PathParts = $File.FullName -split '\\'
        $PublicFolderPathDepth = ($PublicFolderBasePath -split '\\').Count
        $UserDomain = ($PathParts[$PublicFolderPathDepth] -replace '\.IMAP$', '')
        $FolderName = ($PathParts[$PublicFolderPathDepth + 1] -replace '\.IMAP$', '')

        Log-Message -Message "Processing Public Folder $UserDomain : $FolderName" -Type "INFO"

        $SGDomains = Get-SGDomains
        if ($SGDomains.Name -notcontains $UserDomain) {
            Log-Message -Message "Domain $UserDomain not found in SG. Skipping." -Type "WARNING"
            continue
        }

        Log-Message -Message "Domain $UserDomain found in SG." -Type "INFO"

        [xml]$WhiteList = Get-Content $File.FullName
        $Emails = $WhiteList.SelectNodes('//addressBook/contact/email') | ForEach-Object { $_.InnerText.Trim() } | Where-Object { $_ -ne "" }

        foreach ($Email in $Emails) {
            $Result = Add-SGToDomainWhitelist -EntryToAdd $Email -Domain $UserDomain -Comment $Comment
            if ($Result -eq 0) {
                Log-Message -Message "Added $Email to $UserDomain's domain allowlist." -Type "INFO"
            } else {
                Log-Message -Message "Failed to add $Email to $UserDomain's domain allowlist. Error: $Result" -Type "ERROR"
            }
        }
    }
}
catch {
    Log-Message -Message "Error processing public folders: $_" -Type "ERROR"
}

### **Process SpamAssassin 80_MDaemon_welcomelist_from.cf **

# Check if the file exists
if (-Not (Test-Path -Path $SAFilePath)) {
    Write-Error "File not found: $SAFilePath"
    exit
}

# Read the file content
$FileContent = Get-Content -Path $SAFilePath

# Regex pattern to match email addresses
$EmailRegex = '(?<=^welcomelist_from\s+|^whitelist_from\s+|^wl_from\s+|^allowlist_from\s+)[\w\.\-]+@[\w\.\-]+\.\w+'

# Parse email addresses
$EmailAddresses = $FileContent | ForEach-Object {
    if ($_ -match $EmailRegex) {
        $Matches[0]
    }
}

# Remove duplicates and empty entries
$EmailAddresses = $EmailAddresses | Where-Object { $_ -ne $null } | Sort-Object -Unique

# Display the results
if ($EmailAddresses) {
    foreach ($Email in $EmailAddresses) {
        $Result = Add-SGToGlobalWhitelist -EntryToAdd $Email -Comment $Comment
        if ($Result -eq 0) {
            Log-Message -Message "Added $Email to global allowlist." -Type "INFO"
        } else {
            Log-Message -Message "Failed to add $Email to global allowlist. Error: $Result" -Type "ERROR"
        }
    }   
} else {
    Write-Host "No email addresses found in $SAFilePath."
}

J

John

1/1/25, 11:03 PM

@Matthew I will give this a try! Thanks so much for your efforts on this.

Just to be clear, this will import each users Contacts, right?

Matthew Staff

1/1/25, 11:27 PM

@John yes it should import the users contacts as well.

J

John

1/1/25, 11:32 PM

@Matthew

It generally runs well, just a couple of issues.

One, I want to run this weekly to update peoples "contacts" from MD into SG. Can I specify the username/password so it can be scheduled?

I did see some errors, for example:

2025-01-01 21:18:12 - Failed to add groupboard@notifications.pinterest.com to global allowlist. Error: @{ErrorCode=13; ErrorString=AddToGlobalAllowl
ist failed: }

2025-01-01 21:18:14 - Failed to add pinbot@notifications.pinterest.com to global allowlist. Error: @{ErrorCode=13; ErrorString=AddToGlobalAllowlist
failed: }

Below is the entries in context in the 80_MDaemon_welcomelist_from.cf

welcomelist_from pinbot@notifications.pinterest.com
welcomelist_from groupboard@notifications.pinterest.com

There were a few other failures too, but these are publicly known domains.

Matthew Staff

1/2/25, 10:38 AM

@John Hanks, below is an udpated script that will let you specify the username and password in the script.

SG does not allow the same address, including wildcard evaluation, to exist in a blocklist and the allowlist. I traced down a few instances of this error with my test, and this was the case.

I would check the global blocklist in SG for any potential matches? I will make a note to improve the error message returned from the API/Powershell Module.

#########################################################################################################
##                                                                                                     ##
##                             SG Import MD Allowed Senders v1.1.0                                     ##
##                                       January 1, 2025                                               ##
##                                   Copyright © 2025 MDaemon Technologies                             ##
##                                                                                                     ##
## Description:                                                                                        ##
## -----------                                                                                         ##
## This PowerShell script retrieves contact email addresses from Allowed Sender lists in MDaemon       ##
## and imports them into the SecurityGateway Global, Domain, and User allowlists using the SGAPI.      ##
##                                                                                                     ##
## Usage:                                                                                              ##
## ------                                                                                              ##
## - Must be excuted on the MDaemon Server                                                             ##
## - Customize the configuration section to match your environment.                                    ##
## - Run the script with appropriate SecurityGateway global administrator credentials                  ##
##                                                                                                     ##
## Requirements:                                                                                       ##
## -------------                                                                                       ##
## - MDaemon Email Server                                                                              ##
## - SecurityGateway for Email                                                                         ##
## - SGAPI PowerShell Module (if MDaemon is not installed on the same computer as SecurityGateway)     ##
##                                                                                                     ##
## Disclaimer:                                                                                         ##
## -----------                                                                                         ##
## MDaemon Technologies offers no warranty, provides no support, and is not responsible for any        ##
## damages arising from the use of this script. Use at your own risk.                                  ##
##                                                                                                     ##
#########################################################################################################

# Configuration - These values should be customized to match your environment
$SAFilePath = "C:\MDaemon\SpamAssassin\Rules\80_MDaemon_welcomelist_from.cf"
$PublicFolderBasePath = "C:\MDaemon\Public Folders"
$UserMailboxBasePath = "C:\MDaemon\Users"
$SGXMLRPCURL = "http://localhost:4000/SecurityGateway.dll?view=xmlrpc"
$Comment = "Imported from MDaemon"
$LogFile = "C:\MDaemon\Logs\SG_Import_Allowed_Senders.log"
$AllowedParentFolders = @("WhiteList.IMAP", "Allowed Senders.IMAP")  # Allowed parent folder names
#$DryRun = $true # Uncomment to only log actions without making changes

# Uncomment to specify a username and password
#$Username = "User"
#$Password = "Password"

# Ensure Log Directory and File Exist
function Ensure-LogFile {
    $LogDirectory = Split-Path -Path $LogFile -Parent

    if (-not (Test-Path -Path $LogDirectory)) {
        Write-Host "Log directory does not exist. Creating: $LogDirectory" -ForegroundColor Yellow
        New-Item -Path $LogDirectory -ItemType Directory -Force | Out-Null
    }

    if (-not (Test-Path -Path $LogFile)) {
        Write-Host "Log file does not exist. Creating: $LogFile" -ForegroundColor Yellow
        New-Item -Path $LogFile -ItemType File -Force | Out-Null
    }
}

# Function to log messages to file and console
function Log-Message {
    param(
        [Parameter(Mandatory=$true)]
        [string]$Message,

        [Parameter(Mandatory=$false)]
        [ValidateSet("DEBUG", "INFO", "WARNING", "ERROR")]
        [string]$Type = "INFO"
    )

    $Timestamp = Get-Date -Format "yyyy-MM-dd HH:mm:ss"
    $FormattedMessage = "$Timestamp - $Message"

    # Write to log file
    try {
        $FormattedMessage | Out-File -FilePath $LogFile -Append -Encoding UTF8
    }
    catch {
        Write-Error "Failed to write to log file: $_"
    }

    # Console output with color-coding
    switch ($Type) {
        "DEBUG"   { $Color = "Gray" }
        "INFO"    { $Color = "Green" }
        "WARNING" { $Color = "Yellow" }
        "ERROR"   { $Color = "Red" }
        default   { $Color = "White" }
    }
    try {
        Write-Host $FormattedMessage -ForegroundColor $Color
    }
    catch {
        Write-Error "Failed to write to console: $_"
    }
}

# Initialize Logging
Ensure-LogFile
Log-Message -Message "Log directory and file verified/created successfully." -Type "INFO"

# Check for SGAPI module
if (-not (Get-Module -ListAvailable -Name SGAPI)) {
    Log-Message -Message "SGAPI module is not installed. Exiting script." -Type "ERROR"
    exit 1
}

Import-Module SGAPI

# Prompt for credentials securely
try {

	if ($Username) {
		$OpenSession = Open-SGSession -URL $SGXMLRPCURL -UserName $Username -Password $Password
	}
	else {
		$Credential = Get-Credential -Message "Enter SecurityGateway credentials"
		$OpenSession = Open-SGSession -URL $SGXMLRPCURL -UserName $Credential.UserName -Password $Credential.GetNetworkCredential().Password
	}
    
	if($OpenSession -ne 0){
        throw "Failed to open SG session. Error code: $OpenSession"
    }
	Log-Message -Message "Successfully connected to SecurityGateway API." -Type "INFO"
}
catch {
    Log-Message -Message "Error initializing SG session: $_" -Type "ERROR"
    exit 1
}

# --- Process SpamAssassin Whitelist - Add to Global Allowlist ---

# Check if the file exists
if (-Not (Test-Path -Path $SAFilePath)) {
    Write-Error "File not found: $SAFilePath"
    exit
}

# Read the file content
$FileContent = Get-Content -Path $SAFilePath

# Regex pattern to match email addresses
$EmailRegex = '(?<=^welcomelist_from\s+|^whitelist_from\s+|^wl_from\s+|^allowlist_from\s+)[\w\.\-]+@[\w\.\-]+\.\w+'

# Parse email addresses
$EmailAddresses = $FileContent | ForEach-Object {
    if ($_ -match $EmailRegex) { $Matches[0] }
} | Where-Object { $_ } | Sort-Object -Unique

# Process the results
if ($EmailAddresses) {
    foreach ($Email in $EmailAddresses) {
		if ($DryRun) {
			Log-Message -Message "Dry Run: Would have added $Email to global allowlist." -Type "INFO"
			continue
		}  
		$Result = Add-SGToGlobalWhitelist -EntryToAdd $Email -Comment $Comment
        if ($Result -eq 0) {
            Log-Message -Message "Added $Email to global allowlist." -Type "INFO"
        } else {
            Log-Message -Message "Failed to add $Email to global allowlist. Error: $Result" -Type "ERROR"
        }
    }   
} else {
    Log-Message "No email addresses found in $SAFilePath." -Type "DEBUG"
}

# --- Process Public Folders - Add to Domain Allowlist ---
try {
    ForEach ($File in Get-ChildItem -Path $PublicFolderBasePath -Recurse -Filter "AddrBook.mrk") {
        $ParentFolder = Split-Path -Path $File.FullName -Parent | Split-Path -Leaf

        if ($AllowedParentFolders -notcontains $ParentFolder) {
            Log-Message -Message "Skipping $($File.FullName) - invalid parent folder ($ParentFolder)." -Type "DEBUG"
            continue
        }

        $PathParts = $File.FullName -split '\\'
        $PublicFolderPathDepth = ($PublicFolderBasePath -split '\\').Count
        $UserDomain = ($PathParts[$PublicFolderPathDepth] -replace '\.IMAP$', '')
        
        Log-Message -Message "Processing Public Folder $UserDomain : $ParentFolder" -Type "INFO"

        $SGDomains = Get-SGDomains
        if ($SGDomains.Name -notcontains $UserDomain) {
            Log-Message -Message "Domain $UserDomain not found in SG. Skipping." -Type "WARNING"
            continue
        }

        Log-Message -Message "Domain $UserDomain found in SG." -Type "INFO"

        try {
			[xml]$WhiteList = Get-Content $File.FullName
		} catch {
			Log-Message -Message "Failed to parse XML in file $($File.FullName). Error: $_" -Type "ERROR"
			continue
		}

        $Emails = $WhiteList.SelectNodes('//addressBook/contact/email') | ForEach-Object { $_.InnerText.Trim() } | Where-Object { $_ -ne "" }

        foreach ($Email in $Emails) {
			if ($DryRun) {
				Log-Message -Message "Dry Run: Would have added $Email to [$UserDomain] domain allowlist." -Type "INFO"
				continue
			}
            $Result = Add-SGToDomainWhitelist -EntryToAdd $Email -Domain $UserDomain -Comment $Comment
            if ($Result -eq 0) {
                Log-Message -Message "Added $Email to [$UserDomain] domain allowlist." -Type "INFO"
            } else {
                Log-Message -Message "Failed to add $Email to [$UserDomain] domain allowlist. Error: $Result" -Type "ERROR"
            }
        }
    }
}
catch {
    Log-Message -Message "Error processing public folders: $_" -Type "ERROR"
}

# --- Process User Mailboxes - Add to User's Allowlist ---
try {
	$SGUsers = Get-SGUsers -Domain $UserDomain

    ForEach($File in Get-ChildItem -Path $UserMailboxBasePath -Recurse -Filter "AddrBook.mrk") {
        # Extract Parent Folder Name
        $ParentFolder = Split-Path -Path $File.FullName -Parent | Split-Path -Leaf

        # Check if the parent folder is allowed
        if ($AllowedParentFolders -notcontains $ParentFolder) {
            Log-Message -Message "Skipping $($File.FullName) as its parent folder '$ParentFolder' is not in the allowed list." -Type "DEBUG"
            continue
        }

        $PathParts = $File.FullName -split '\\'
        
        # Calculate the number of path segments in $UserMailboxBasePath
        $UserMailboxPathDepth = ($UserMailboxBasePath -split '\\').Count

        # Extract UserDomain and UserMailbox dynamically
        $UserDomain = $PathParts[$UserMailboxPathDepth]
        $UserMailbox = $PathParts[$UserMailboxPathDepth + 1]
        $UserEmail = "$UserMailbox@$UserDomain"

        Log-Message -Message "Processing User Folder $UserEmail : $ParentFolder" -Type "INFO"

        $SGUsers = Get-SGUsers -Domain $UserDomain
        if ($SGUsers.MailBox -notcontains $UserMailbox) {
            Log-Message -Message "$UserEmail not found in SG. Skipping." -Type "WARNING"
            continue
        }

		try {
			[xml]$WhiteList = Get-Content $File.FullName
		} catch {
			Log-Message -Message "Failed to parse XML in file $($File.FullName). Error: $_" -Type "ERROR"
			continue
		}

        $Emails = $WhiteList.SelectNodes('//addressBook/contact/email') | ForEach-Object { $_.InnerText.Trim() } | Where-Object { $_ -ne "" }

        foreach ($Email in $Emails) {
			if ($DryRun) {
				Log-Message -Message "Dry Run: Would have added $Email to [$UserEmail] user allowlist." -Type "INFO"
				continue
			}
			$Result = Add-SGToUserWhitelist -EmailToAdd $Email -UserEmail $UserEmail -Comment $Comment
            if ($Result -eq 0) {
                Log-Message -Message "Added $Email to [$UserEmail] user allowlist." -Type "INFO"
            } else {
                Log-Message -Message "Failed to add $Email to [$UserEmail] user allowlist. Error: $Result" -Type "ERROR"
            }
        }
    }
}
catch {
    Log-Message -Message "Error during user mailbox processing: $_" -Type "ERROR"
}

J

John

1/2/25, 3:35 PM

@Matthew Thanks again, so MUCH!

This one works much better.

I'm still seeing some issues like below, but I have not looked into them and am not quite as concerned. You've addressed the main concerns I had.

2025-01-01 21:18:10 - Skipping C:\MDaemon\Users\domain.com\bill\Blocked Senders.IMAP\AddrBook.mrk as its parent folder 'Blocked Senders.IMAP' is
not in the allowed list.
2025-01-01 21:18:10 - Skipping C:\MDaemon\Users\domain.com\bill\Contacts.IMAP\AddrBook.mrk as its parent folder 'Contacts.IMAP' is not in the all
owed list.
2025-01-01 21:18:10 - Skipping C:\MDaemon\Users\domain.com\bill\Contacts.IMAP\xxxxx Contacts.IMAP\AddrBook.mrk as its parent folder '
xxxxx Contacts.IMAP' is not in the allowed list.

Matthew Staff

1/2/25, 3:42 PM

@John that is good to hear. Those messages are nothing to be concerned about. It is just debug logging. Unless the addrbook.mrk is found in a folder named "Whitelist" or "Allowed Senders" it will be skipped. The next step would be for me to extend the script to import contacts found in "Blocked Senders" to the SG blocklist.

Transfer Whitelist and other things from MD into SG