My Almost-Perfect Spam Filter

Category: Spam

Researchers estimate that about 46 percent of all email traffic is spam. But I rarely see any spam in my inbox. That's because my spam filter blocks 99.9 percent of all spam, phishing and malware emails. Read on to learn how you can get near-perfect spam filtering, and declutter your inbox...

Try Blocking Spam With Gmail (it works for any inbox)

According to Statista, a global research firm that provides market and consumer data, the ratio of legit emails to spam is improving. In the beginning of 2011, spam accounted for 80 percent of all emails. Today, that number has decreased by almost half, but that's still not low enough. One factor in that decline is machine learning, an application of artificial intelligence, which has been part of Gmail’s spam filter program since its inception.

Every time a user clicks the “Report Spam” or “Not Spam” button on a message, Gmail learns something that helps it filter spam better. But Google is also using AI technology to help Gmail learn on its own, without the user’s clicks.

The Gmail spam filter uses an artificial neural network (ANN) that can detect and block the “especially sneaky” spam that sometimes slips past users’ radar. What’s an “artificial neural network,” you ask? The oversimplified answer is, a lot of computers connected to each other in an attempt to simulate the interconnections of human brain cells. It’s expensive to build an artificial neural network; most are tiny, with 1 to 10 million connections.

Gmail Spam Filter

Google has built an ANN with over one billion connections using the processors in its vast and far-flung empire of data centers. That sounds awesome, until you learn that the human brain contains several TRILLION connections! Nonetheless, Google’s ANN is capable of rudimentary human-like self-learning. That means you don’t have to teach it what spam is, because it's trained to understand the context and nuances of language, making it highly effective at distinguishing spam from legitimate messages.

In one experiment, Google’s ANN was fed millions of still images of cats culled from YouTube videos. The images were not labeled as “cats” and programmers did not tell the ANN what a “cat” is. The ANN figured it out for itself, learning to recognize cats in virtually any image. If it can do that, it can recognize spam with greater accuracy than most humans can. In 2019, Google announced that thanks to their machine learning framework, they were able to block an additional 100 million spam messages daily. With recent advances in AI, one might assume that number is higher (especially if the messages have anything to do cats).

Unidentified Frying Objects?

You don't have to do anything to use Gmail's spam filtering - it's automatic. But you can add your own filters to funnel messages into folders, forward to another address, and other actions. See my article [HOWTO] Tame Your Overflowing Email Inbox for help adding filters to Gmail, Yahoo, or webmail.

For more spam fighting tips, see my articles Defend Your Inbox With a Disposable Email Address, How to Trace a Spammy Email and Report a Spammer? (Read this FIRST...)

But the spam-eating technology is still not perfect. Just as humans see UFOs in pictures of streetlights, Gmail’s filter sometimes sees spam in legitimate messages and mistakenly consigns them to the spam folder. Google claims that only about 0.01 percent (1 in 10,000) of legitimate emails are falsely labeled as spam these days. As the system learns more about email, that figure should fall even further.

Take a look in your Gmail spam folder, and see how it's working. If you do find certain messages are being incorrectly flagged as spam, you can create a Gmail filter for them, with a "Never send to Spam" action, so they'll be delivered to your inbox.

On the flip side, occasionally, I do get a batch of spammy messages that recur over a number of days. Recently I was getting several emails daily emails about investing in bitcoin. It was clearly spam, but perhaps because of other tech-related stuff that I do read, Gmail thought otherwise. I created a filter to send them to the Trash, problem solved.

How does Gmail’s spam filter compare to other free Webmail services, such as Microsoft’s and Yahoo? I haven’t been able to find any independent, rigorous studies of this question. I’ll just note that Yahoo seems rather overzealous about spam filtering, and quite a few of my outgoing messages to Yahoo accounts end up wrongly labelled as spam, or just don't get delivered at all. If you have any relevant data on this, please do share.

In addition to flagging spam based on the content of the message, Gmail, Yahoo, and other popular webmail services assess the reputation of the sender's IP address and domain name. Emails from known spam sources and messages lacking proper authentication are flagged as potential spam. [Tech note: Techniques such as SPF (Sender Policy Framework), DKIM (DomainKeys Identified Mail), and DMARC (Domain-based Message Authentication) are used to verify the authenticity of the sender. Messages that fail these authentication checks are more likely to be considered spam or phishing attempts.]

Another spam-fighting technique that Gmail employs doesn’t really tackle spam, in my opinion. Gmail attempts to learn individual users’ reading preferences and filter out even legitimate mail that goes unread. The company gives the example of email newsletters. One user reads them assiduously while another just lets them pile up unread, or deletes them without reading. Why people don’t just unsubscribe from newsletters is one of life’s mysteries. Hopefully, Gmail can distinguish between legit opt-in email newsletters and the ones that just arrive without your consent.

Finally, Gmail is providing help for legitimate, large-volume email publishers whose communications are sometimes wrongly condemned as spam. The Gmail Postmaster Tools will help airlines, banks, credit card companies, and other well-known firms analyze the fate of their emails and improve their delivery and reading rates. That’s a good thing, because more and more large companies are encouraging customers to receive their monthly statements by email, rather than paper copies in the mail.

Even if you don't use or like Gmail, you can still use it to filter spam. Here's one technique that some people use to "pre-filter" their incoming emails. Instead of providing your actual email address when asked, give out a Gmail address that you've created. Configure that Gmail account to simply forward everything to your actual address. Gmail does spam-filtering BEFORE forwarding, so the messages that do get forwarded are virtually spam-free.

Gmail’s spam filter is so reliable and accurate that I hardly ever check my spam filter for false positives anymore. I get hundreds of emails daily, and I rarely find myself clicking on the “Report Spam” button. The spam filter just works.

Do you use Gmail? How is the spam filtering working for you? Your thoughts on this topic are welcome. Post your comment or question below.

Ask Your Computer or Internet Question

  (Enter your question in the box above.)

It's Guaranteed to Make You Smarter...

AskBob Updates: Boost your Internet IQ & solve computer problems.
Get your FREE Subscription!


Check out other articles in this category:

Link to this article from your site or blog. Just copy and paste from this box:

This article was posted by on 19 Jun 2024

For Fun: Buy Bob a Snickers.

Prev Article:
Can a Computer Virus Destroy Your Hard Drive?

The Top Twenty
Next Article:
SLOW Internet? Here's Why...

Most recent comments on "My Almost-Perfect Spam Filter"

Posted by:

20 Jun 2024

Yahoo WAS perhaps one of the best for filtering spam, although I had little trouble with any of the major providers. Within the last week, however, my daily Yahoo Inbox emails have quintupled. Yes, the "additional" ones are spam. Did I click on "something" odd just before that? I don't know. I just hope I can cut it down. (As an aside, seems very good at filtering spam.)

Posted by:

20 Jun 2024

Thanks! I completely agree. I formerly got quite a bit of spam but G-Mail spam filter does work, over time. However, I also use a mail forwarding service that I create new specific e-mail addresses using and if a sender ever sells or shares that e-mail address, with the click of a link it's dead. That way you only have given your real e-mail to the forwarding service and mine is a good one that never, ever sells or shares it (or spams me). You can do the same thing using G-Mail but that could be the topic of a future post by Bob as I'm sure he knows that trick too.

Posted by:

20 Jun 2024

So Google claim only one in ten thousand emails are wrongly treated as spam. BS. I receive about a thousand emails a month, of which at least one per month is wrongly sent to the spam box. I missed a work offer because of this. I have to look through my spam box regularly to check for real emails, so what is the point of having a spam box? I would sooner have all mail delivered and sort the spam out myself, but will Google let me choose to let everything through? No they are so convinced about their phoney statistics they won't lift their spam filters for me. There was also the dozens of emails that went missing and hours and hours wasted when Gmail decided, without warning me, that they would change their password authorisation. I am steadily migrating all my contacts towards using another email provider I use. I'm done with Google.

Posted by:

Brian B
20 Jun 2024

I have a mail filter on my Mozilla Thunderbird, that filters out almost 100% of spam and scam. In the From box in my filter editor, I just have "Isn't in my address book" and send it to trash. Obviously, any new contact you want to keep, you have to make sure you enter that email address in your address book. As a backup if needed (not needed so far) I have a list of 2 letter country codes that I never do business with.

Posted by:

20 Jun 2024

Quote: Why people don’t just unsubscribe from newsletters is one of life’s mysteries....
Doesn't doing this let the sender know that you truly exist ? I mean in order to unsubscribe you have to OPEN the email which is not only letting the sender know you are real, it also could contain a virus as stated by many experts. I'm lost to that line of thinking I guess.

Posted by:

Ernest N. Wilcox jr. (Oldster)
20 Jun 2024

I think the quote you reference

[Quote: Why people don’t just unsubscribe from newsletters is one of life’s mysteries....]

is referring to people who sign up for newsletters, but never unsubscribe when thy find that they have no interest in the content.

Ernie (Oldster)

Posted by:

20 Jun 2024

I prefer using a super fast email client (Thunderbird) rather than web based email, although this still gives me the option of accessing the web based email interface of my internet provider. Mozilla Thunderbird's spam filter is truly excellent at removing spam, and learns very quickly in response to my marking unwanted messages. The end result is that spam has not been a problem for me, thanks to Thunderbird, and I highly recommend it.

Posted by:

Steven Latus
20 Jun 2024

petefior, I also use Thunderbird, both for my Optimum account as well as my gmail account. One problem I had with Optimum within the past year is that they have their own spam filter that you cannot turn off (previously, you *could* do so and I had done so—I don't know when it changed). When I did not receive an email that I was expecting, I used Optimum's webmail interface to review the online spam filter. There were some legit messages in there. Now, despite disliking webmail, I feel compelled to log in every day to check that online spam folder and, yes, I've caught more legit messages in there. Annoying!

Posted by:

20 Jun 2024

I get very little spam because I'm always careful about giving out my primary email address.

When I sign up for something unimportant like an ezine or a newsletter I use a throw-away email account like instead of using my main email account.

Many people have one email address that they use for everything, this is a great way to increase your spam intake!

Posted by:

21 Jun 2024

Have Yahoo and hadn't checked my spam for a while. Wondered why I hadn't received several emails. So now will be checking spam folder daily. Frustrating.

Posted by:

Vicki Bunch
21 Jun 2024

Thanks for this article, Bob. For a couple of months I have marvelled at how much less spam I'm being delivered by Gmail. It has been a huge improvement, I have to say! I'm impressed that they have taken such effective steps, and although I don't like many aspects of Gmail, I will force myself to get used to them for the sake of having many fewer spam messages getting through.

Posted by:

21 Jun 2024

With some email providers you can add a bit extra to your email address when you subscribe to something, or when you send to anyone. For example, say your email address is "myemail@mysupplier" and you are subsscribing to "A New Deal". You can add anything between "myemail" and the "@" with a plus sign. In this case you might show it as "myemail+NewDeal@mysupplier" If you then get emails from anybody else using that email to you, you know it's not legit. But not all ISPs will accept that.

Posted by:

Brian B
21 Jun 2024

@petefior. I use Thunderbird too, but I have found it's too slow to learn, but in all fairness, this is probably due to the spammers continually changing their addresses and the wording and composition of the texts. I have found the best filter of all is the "Not in my address book" which is very hard for spammers to break through.

Posted by:

Sandy Jewell
22 Jun 2024

I have used my two gmail accounts for years. Filtering spam is my greatest love. The moment I purchase something from Amazon, unwanted emails appear. Not gmails fault. Amazon must sell our info.

Post your Comments, Questions or Suggestions

*     *     (* = Required field)

    (Your email address will not be published)
(you may use HTML tags for style)

YES... spelling, punctuation, grammar and proper use of UPPER/lower case are important! Comments of a political nature are discouraged. Please limit your remarks to 3-4 paragraphs. If you want to see your comment posted, pay attention to these items.

All comments are reviewed, and may be edited or removed at the discretion of the moderator.

NOTE: Please, post comments on this article ONLY.
If you want to ask a question click here.

Free Tech Support -- Ask Bob Rankin
Subscribe to AskBobRankin Updates: Free Newsletter

Copyright © 2005 - Bob Rankin - All Rights Reserved
About Us     Privacy Policy     RSS/XML

Article information: AskBobRankin -- My Almost-Perfect Spam Filter (Posted: 19 Jun 2024)
Copyright © 2005 - Bob Rankin - All Rights Reserved