I presume that email address is in your address books.

Have you set up your Junk Mail Controls and made sure that you have selected the address books as whitelist ?

• http://kb.mozillazine.org/Junk_Mail_Controls

Message filters run in the order listed, so if a filter above that particular email filter captures those emails first it will perform an action on them, which means the filter selecting that email address cannot locate the emails as they were already moved by a previous filter. So I would suggest you move the specific email filter to the top of the list so it runs first.

then select filter and click on 'Edit' check that it really does 'Filter before Junk classification'

None of this worked.

1. My filter is and has been the first one in the filter list. 2. My filter is supposed to run BEFORE Junk Classification - but it's not. 3. I am on TB 31.3.0 and I do not see a way to get to the while list. I followed the instructions from the above link. I am on the Tools -> Account Settings -> Junk Settings and there is no link, window, or mention of a White List. Do do I get to this.

In Account Settings > Junk Settings for the mail account: See image below as aid.

This is where you set the Account specific Junk Mail settings. Select to enable adaptive junk mail controls for the mail account. There is a section: 'do not automatically mark mail as junk if the sender is in' It should show you a list of your address books. This is the whitelist. Any contacts in a selected address book will not be auto marked as Junk. Select the address books you want in that list.

Destination and Retention: Select 'Move new junk messages to' If Pop mail account choose: 'Junk folder on' and select the mail account name. If this pop mail account uses a Global Inbox (Local folders) then select 'Junk folder on' and select 'Local Folders'

If IMAP mail account choose: 'Other' and select the Junk or Spam folder (depends upon what that folder is called on the server) on that imap mail account.

The button 'Global Junk Preferences' is a shortcut to: 'Tools' > 'Options' > 'Security' > 'Junk' tab This is where you set your default junk mail settings. What to do when you manually mark messages as junk. You can also enable the junk filter log which I have found to be very useful especially when trying to work out what is being filtered.

If you want to clear the data collected and used for Junk Mail controls and reset the training data, click on button 'Reset Training Data'. Click on OK to close and save Options

Click on OK to close and save Account Settings.

Toad-Hall,

Ah - the picture was very helpful. I had not realize there was a Junk setting per account. Until your picture, then only Junk Setting I saw was in the Local Folders section.

I have done what you suggested. I have marked all of my Address books as being on the White List.

I have change my Junk Mail folder so that I now have one per e-mail account. And I have turned on Junk Filter logging so I can at least see what it is doing.

Thanks for you help. I shall see if this works.

Dennis

I have a similar problem:

There are a bunch of filters that seem to work fine and are protected in that they prevent further classification as junk. But this filter depends on the subject line and it has square brackets in the subject:

[Mycolleagues] Dubai International Conference ICSMET 2015, March 2015, Call For Papers - Reg

is an example. Such messages never get to my filter unless I do something like "run filters" on the junk window. They will also be treated correctly if I move them to the input window and then run filters. The problem seems to be present only while filters are run immediately after a mail download.

In accordance with the above ideas I have mucked with the junk filtering stuff. At least I turned the log on.

DougMcNutt Please post an image of the filter.

re :Such messages never get to my filter unless I do something like "run filters" on the junk window.

So the message is auto moved to Junk before your filters are applied. If the message is in Junk then your filter will not work because filters only work automatically on new messages in the Inbox.

Check your filter. It must say 'Filter before junk classification'.

Also, as in my previous response, you must set up your Junk Controls correctly, so that email addresses in selected address books are set as a whitelist. For this to apply to that particular email address, you must have that email address in a selected address book.

Wow. You're right on it. Thanks.

Extracted from file /home/doug/.thunderbird/********.default/Mail/mail.macnauchtan-1.com/msgFilterRules.dat

name="MyColleagues" enabled="yes" type="17" action="Move to folder" actionValue="mailbox://nobody@Local%20Folders/Pubs/MyColleagues" condition="AND (subject,begins with,[Mycolleagues])"

Filter before junk is set and applies properly for some 20 other filters in the same file.

When I have the junk file open and use the menbar to select "run filters" it works fine sending the [Mycolleagues] messages to "Local Folders/Pubs/MyColleagues"

There is no whitelist for the sources for these messages because they are all different. The group is not operating like a mailing list; it is simply forwarding announcements and the From: address is different for each source. That's why my filter is based on the subject header.

I have discovered another possibility. A surprising number of the [Mycolleagues] messages have been misidentified as spam by my arpanet provider, pair.com which supports macnauchtan.com. There are headers that claim spam and I do have something like "Honor Spamassassin headers" checked. I shall try to remember where the hell that button is and try turning it off. But. . . When I say "do the filtering first" it ought to do ALL of the filtering first.

Just for fun, my Eudora filter worked fine for years. I'm into tbird because pair.com wants encryption of passwords that Eudora doesn't support.

A bit more in the way of data:

It appears that one of these headers is the culprit. X-Spam-Status: Yes, hits=3.6 required=3.5 tests=DCC_CHECK,FREEMAIL_FORGED_REPLYTO,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=disabled version=3.004000 X-Spam-Flag: YES X-VERTICAL-IP-Spam-Score: -0.976

The From: header is useless because it shows the original sender of each message. These possible alternates are present.

X-BeenThere: mycolleagues@mailman.ufsc.br Return-Path: <mycolleagues-bounces@mailman.ufsc.br> X-Original-To: mycolleagues@mailman.ufsc.br Delivered-To: mycolleagues@mailman.ufsc.br

Would Thunderbird's junk finder be willing to look at any one of them if "mailman.ufsc.br" were to be added to a selected address book? Something tells me it would look only at the From: header. It appears that one of these headers is the culprit. X-Spam-Status: Yes, hits=3.6 required=3.5 tests=DCC_CHECK,FREEMAIL_FORGED_REPLYTO,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS autolearn=disabled version=3.004000 X-Spam-Flag: YES X-VERTICAL-IP-Spam-Score: -0.976

The From: header is useless because it shows the original sender of each message. These possible alternates are present.

X-BeenThere: mycolleagues@mailman.ufsc.br Return-Path: <mycolleagues-bounces@mailman.ufsc.br> X-Original-To: mycolleagues@mailman.ufsc.br Delivered-To: mycolleagues@mailman.ufsc.br

Would Thunderbird's junk finder be willing to look at any one of them if "mailman.ufsc.br" were to be added to a selected address book? Something tells me it would look only at the From: header.

Without using the results of Spamassasin it appears that the message is not set to junk which means that the filter is used first or that Thunderbird's checking is less critical with a learning process available.

But I would really like to have pair.com's spam results taken seriously for all of the other messages. As of now it's ignored for everything. Is it posible to declare junk status as a result of a filter hit? If so can I continue with the rest of the filters?

Is it a bug that should be reported as such? If so, is it a documentation bug or an execution bug?

re: I have discovered another possibility. A surprising number of the [Mycolleagues] messages have been misidentified as spam

this is set here: Tools > Account Settings > Junk Settings for the mail account just below the list of address books to use as whitelist - option: 'Trust junk mail headers set by:' name of program If this is enabled, TB auto considers any messages marked as 'Spam' by that program to be auto moved to Junk. i would leave this alone as you do want it to continue doing it's job.

When an email is marked as 'Spam', in my case at least, those words are added to the beginning of the subject usually in brackets. Check this is the case - is [Spam] or similar being put before [Mycolleagues] in the Subject header?

If it is, then your condition in the Filter will not work because you use 'Begins with' and the subject no longer 'begins with' [Mycolleagues].

I believe this may explains why those emails marked as 'spam' are being diverted to Junk. eg condition="AND (subject,begins with,[Mycolleagues])"

therefore, in filter, try these options:

changing the 'Begins with' to 'Contains' so it says Subject' 'contains' [Mycolleagues]

or If this line is always the same in all emails from that sender: X-Original-To: mycolleagues@mailman.ufsc.br

then create an option to select that header info.

• In the message filter
• Click on the 'Subject' to see drop down list
• at the bottom will be 'Customise' - click on this.
• a new small window opens.
• Type in the new header - X-Original-To - it must be exact
• click on 'Add'
• click on 'OK'

Now you should be ble to select that option in the drop down list.

• select: 'X-Original-To' 'is' mycolleagues@mailman.ufsc.br

This means that it is no longer searching on the 'Subject' but on the 'X-Original-To' header.

You're right, Todd. "Contains" is a better option.

These are all recorded: [Mycolleagues] [Suspeita] ISCC-SFCS 2015 CFP - Deadline approaching [Mycolleagues] [Spam] CFP UCAmI, IWAAL & AmIHEALTH 2015 Re: [Mycolleagues] CFP - Open Special Issue: Metaheuristics Optimization Re: [Mycolleagues] [Authors] Copyright Infringement

I have never seen [spam] added as the first item in a subject but I have seen "Junk***" used that way. I turned that feature off at pair.com where I am the owner of macnauchtan.com. Family members using that email address with their own usernames didn't like the Junk*** and I don't either.

I have considered using the "mycolleagues@mailman.ufsc.br" address which might allow me to use the sender and permit including it in a known address file. I'll try it after I see what the "contains" change does. I worry that the junk assignment during downloads might look only at the From: header for a match. Can the selected header be different for different filters in the filters file?

I'll use the Delivered-To: header though because RFC422 clearly says headers beginning with X- are specials that are not required of mail clients.

Is case sensitivity a problem? MyColleagues, mycolleagues, Mycolleagues are all used by the source but will not be happy when addressing a mailbox file in LINUX. What happens in the internals of Thunderbird?

Apparently Thunderbird defaults to Spamassassin as the "Trust junk mail headers set by" software. That is what pair uses and I suppose, but am not told, that the "X-Spam-Flag: YES" header line does the deed.

Is it legal to edit the msgFilterRules.dat file instead of using the limited capability of the filter tool available in the menu bar? What is type (17)? Can I perform other actions such as "set to junk". Will the filter process proceed to the next item if the current item does not transfer the message to another mailbox? That's a hangover from Eudora I know but are there really 16 lower types available? If so where are the docs?

I'll report on the change to using the address book in a couple of days.

My problem is that I have hundreds of junk mail that I can't get rid of. It all happened when I tried to add a filter at a last item in the list. The goal was to get my mail site's spam-flag YES to be honored as an item in the list of filters. Here's what the filter looked like:

name="PairSpamFlag" enabled="yes" type="48" **** Note that 48(10) = 00110000(2) action="Move to folder" actionValue="mailbox://dmcnutt@mail.macnauchtan.com/EXPERMENTING" condition="AND (\"x-spam-flag\",contains,YES)"

After a day I got frustrated because I became unable to move messages around or to delete them. It turns out that the filter was causing the problem but I upgraded from ubuntu 14:04 to 14:10 and some other things because I really thought thunderbird was not the whole probem. In desparation I changed the filter, using the filters menu and not by editing the filters file, selecting a run before the junk tests.

name="PairSpamFlag" enabled="yes" type="17" *** Note 17(10) = 00010001(2) action="Move to folder" actionValue="mailbox://dmcnutt@mail.macnauchtan.com/EXPERMENTING" condition="AND (\"x-spam-flag\",contains,YES)"

My problems disappeared and I could delete messages again. But with type 17 nothing ever shows up in my EXPERMENTING folder where I would like to honor the spam designations made by Spamassassin running on my mail server.

Changing filter item to "run after junk determination" changes type to 48

name="PairSpamFlag" enabled="yes" type="48" action="Move to folder" actionValue="mailbox://dmcnutt@mail.macnauchtan.com/EXPERMENTING" condition="AND (\"x-spam-flag\",contains,YES)"

This way I do get traffic sent to EXPERMENTING and the Junk folder gets populated. It appears that messages determined to be junk by thunderbird go to Junk even if they also contain a spam flag YES from the previous examination at Pair.com.

So different filters can be executed before and after junk checking occurs. Does that mean that the rules.dat file is read a second time for each message? Or if the file read once with other storage of the tests to be made? If so when does thunderbird read and store a changed .dat file?

Oh O! Four messages just arrived in EXPERMENTING with the junk flag set. How could that happen now after a couple of mail checks have occurred without getting such things.

Oh OO! I just received a Dr.Oz message in my regular Inbox. It has the Junk flag set in the Junk status column and the spam flag is YES as set by Perl.com. That one should have gone either to thunderbird's Junk or my EXPERMENTING mailbox.

Now, an hour later, I am getting everything that way. Nothing is ever sent to the Junk folder. And worse, the problem of not being able to move or delete messages has come back. Select a message advertised as junk; Right click and choose delete message; Nothing happens; Try dragging it to the trash; Nothing happens. I shall have to remove the filter that is now type 48 but at least it's not going to take a couple of days to figure it out. Thunderbird owes me some much more detailed description of just how filtering is supposed to work.

I am going to remove the filter type 48 and I'm pretty sure I will recover an ability to discard and move messages. I just will not be able to honor Spamassassin's findings.

Are the filter types really a set of binary flags that are used during the filtering process? If so what are the other flag bits used for? When does thunderbird read the filters' *.dat file? Restarting the application doesn't fix my problem but removing the type 48 filter fixes things immediately without reboot or anything else.

Yeah. I'll try Toad's suggestion to get the address book involved.

So I removed the filter type 48 but actually I removed the entire filter and I did it with a simple delete of the last few lines of the "filters".dat file. I now have proper access to the delete and move-to operations provided by the thunderbird application.

I would really like to know if this failure is a bug in thunderbird or a bug in the ubuntu version of Linux. Deep down I'm a UNIX user since the early 70's but only on big machines that I never could afford. I used Eudora in Apple's OS 9 but I found Apple's BSD UNIX in OSX unsatisfying, and Apple's iOS is something I refuse to learn. My Apple OS 9 box refuses to satisfy the demands for secure login for "modern" mail services.

Is ii possible that I need to dump ubuntu for a simpler version of Linux? Debian?

And Toad: I apologize for a previous mis-spelling. My son Todd is on the staff of John's Hopkins university and he probbly knows more than I about UNIX.

re: What is type (17)? Managed to locate this data about 'type' type="1" label="Checking Mail" type="16" label="Manually Run" type="17" label="Checking Mail or Manually Run" type="32" label="Checking Mail (after classification)" type="48" label="Checking Mail (after classification) or Manually Run"

It appears that messages determined to be junk by thunderbird go to Junk even if they also contain a spam flag YES. This is because you may have set the option in Account Settings > Junk Settings to trust Spamassassin. So thunderbird is trusting that spammassassin is correct and these messages are Junk.

http://mesquilla.com/2009/08/28/managing-spam-with-after-classification-filters/ * The order of message processing in Mozilla mailnews is:

• Run normal filters (on each message as it is received)
• Check whitelisting (on a message batch, this and subsequent steps)
• Run bayes classifier on non-whitelisted messages, and mark messages as junk or good.
• Apply “after classification” filters.
• Apply junk message moves using default junk processing.

So applying 'filter after junk classification' (type=48) should move those messages before the final stage where Junk message is moved to Junk. Hence why you can move specific spam messages to EXPERMENTING folder.

re: I am getting everything that way. Nothing is ever sent to the Junk folder. - all of them going to EXPERMENTING folder. That is because your filter name="PairSpamFlag" is looking for ANY email marked with condition="AND (\"x-spam-flag\",contains,YES)"

Using; 'Filter After Junk Classification' It is not looking for specific emails where two or more conditions must be true. Try: select 'Match all of the following' 'Subject','contains',[Mycolleagues]) 'x-spam-flag','contains',YES

Additional info: You could have the following conditions because it is set After classification: 'Subject','contains',[Mycolleagues]) 'x-spam-flag','contains',YES 'Junk Status' 'is' 'Junk'

I would also try resetting the Junk Status and then moving the message. If you also want the 'Junk flag removed try this set of actions: 'Set Junk Status to' 'Not Junk' 'Move Message to' 'EXPERMENTING'

Please report back on results. Are specific mycollegues emails move to the correct folder and not flagged as junk? Are they still not able to be moved or deleted etc?

Wow. That's a lot to try. I shall work on it and be back.

It's important that the type 48 test at the end was causing ALL messages from any of the four accounts being read to be undeletable and unmoveable. The entire thunderbird application was unusable. Even an application restart would run for a few POP3 downloads and then become unusable.

If I put the type 48 filter back in I create a far bigger problem than the original [Mycolleagues] bit because it affects the other three mailboxes being downloaded with POP3.

The option to trust Spamassassin for everything has been off for a long time. I was trying to do the same thing using a filter in what became a type 48 executed after the [mycolleagues] filter which works fine now.

If the original mycolleagues filter using before classification works and does what you require then I would not bother trying to work around it.

I was only putting in another suggestion to try to see if the trusted spam filter could be fixed by resetting the 'Marked as Junk' to not Junk. But this was assuming you were using the Trust Spamassassin in Junk controls.

Yes. I'm getting a bit off the track here and I still haven't tried your suggested test on one of the other headers in which the sender's address appears. That should be white listable even if it's not in the From: header. That will allow me to turn the Spamassassin tests on again for everything else.

mesquilla.com is interesting and tempting but I do have other things to do. I worry that I may have accidentally found some kind of bug that causes the complete failure of moving messages to trash or elsewhere. That's another topic and it might be ubuntu and not Thunderbird.

You have been most helpful.