My Near-Perfect Spam Filter

Category: Spam

Researchers estimate that about 45 percent of all email traffic is spam. But I rarely see any spam in my inbox. That's because Gmail blocks 99.9 percent of all spam, phishing and malware emails. Read on to learn how you can get near-perfect spam filtering, even if you don't use Gmail for your inbox...

Try Blocking Spam With Gmail (it works for any inbox)

According to Statista, a global research firm that provides market and consumer data, the ratio of legit emails to spam is improving. In the beginning of 2011, spam accounted for 80 percent of all emails. Today, that number has decreased significantly, but not enough. One factor in that decline is machine learning, an application of artificial intelligence, which has been part of Gmail’s spam filter program since its inception.

Every time a user clicks the “Report Spam” or “Not Spam” button on a message, Gmail learns something that helps it filter spam better. But Google is also using technology to help Gmail learn on its own, without the user’s clicks.

The Gmail spam filter uses an artificial neural network (ANN) that can detect and block the “especially sneaky” spam that sometimes slips past users’ radar. What’s an “artificial neural network,” you ask? The oversimplified answer is, a lot of computers connected to each other in an attempt to simulate the interconnections of human brain cells. It’s expensive to build an artificial neural network; most are tiny, with 1 to 10 million connections.

Gmail Spam FIlter

Google has built an ANN with over one billion connections using the processors in its vast and far-flung empire of data centers. That sounds awesome, until you learn that the human brain contains several TRILLION connections! Nonetheless, Google’s ANN is capable of rudimentary human-like self-learning. That means you don’t have to teach it what spam is.

In one experiment, Google’s ANN was fed millions of still images of cats culled from YouTube videos. The images were not labeled as “cats” and programmers did not tell the ANN what a “cat” is. The ANN figured it out for itself, learning to recognize cats in virtually any image. If it can do that, it can recognize spam with greater accuracy than most humans can. In 2019, Google announced that thanks to their machine learning framework known as TensorFlow, they are able to block an additional 100 million spam messages daily. Especially if they have anything to do cats, I presume.

Unidentified Frying Objects?

You don't have to do anything to use Gmail's spam filtering - it's automatic. But you can add your own filters to funnel messages into folders, forward to another address, and other actions. See my article Your Inbox Overfloweth? for help adding filters to Gmail, Yahoo, or Outlook.com webmail.

For more spam fighting tips, see my articles Fight Spam With a Disposable Email Address, Can You Trace an Email? Maybe, Here's how… and Report a Spammer? (Read this FIRST...)

But the ANN is still not perfect. Just as humans see UFOs in pictures of streetlights, Google’s ANN sometimes sees spam in legitimate messages and mistakenly consigns them to the spam folder. Google claims that only about 0.01 percent (1 in 10,000) of legitimate emails are falsely labeled as spam these days. As the ANN learns more about email, that figure should fall even further.

Take a look in your Gmail spam folder, and see how it's working. If you do find certain messages are being incorrectly flagged as spam, you can create a Gmail filter for them, with a "Never send to Spam" action, so they'll be delivered to your inbox.

On the flip side. occasionally, I do get a batch of spammy messages that recur over a number of days. Recently I was getting several emails daily emails about investing in bitcoin. It was clearly spam, but perhaps because of other tech-related stuff that I do read, Gmail thought otherwise. I created a filter to send them to the Trash, problem solved.

How does Gmail’s spam filter compare to other free Webmail services, such as Microsoft’s Outlook.com and Yahoo? I haven’t been able to find any independent, rigorous studies of this question. I’ll just note that Yahoo seems rather overzealous about spam filtering, and quite a few of my outgoing messages to Yahoo accounts end up wrongly labelled as spam, or just don't get delivered at all. If you have any relevant data on this, please do share.

Another spam-fighting technique that Gmail employs doesn’t really tackle spam, in my opinion. Gmail attempts to learn individual users’ reading preferences and filter out even legitimate mail that goes unread. The company gives the example of email newsletters. One user reads them assiduously while another just lets them pile up unread, or deletes them without reading. Why people don’t just unsubscribe from newsletters is one of life’s mysteries. Hopefully, Gmail can distinguish between legit opt-in email newsletters and the ones that just arrive without your consent.

Finally, Gmail is providing help for legitimate, large-volume email publishers whose communications are sometimes wrongly condemned as spam. The Gmail Postmaster Tools will help airlines, banks, credit card companies, and other well-known firms analyze the fate of their emails and improve their delivery and reading rates. That’s a good thing, because more and more large companies are encouraging customers to receive their monthly statements by email, rather than paper copies in the mail.

Even if you don't use or like Gmail, you can still use it to filter spam. Here's one technique that some people use to "pre-filter" their incoming emails. Instead of providing your actual email address when asked, give out a Gmail address that you've created. Configure that Gmail account to simply forward everything to your actual address. Gmail does spam-filtering BEFORE forwarding, so the messages that do get forwarded are virtually spam-free.

Gmail’s spam filter is so reliable and accurate that I hardly ever check my spam filter for false positives anymore. I get hundreds of emails daily, and I rarely find myself clicking on the “Report Spam” button. The spam filter just works.

Do you use Gmail? How is the spam filtering working for you? Your thoughts on this topic are welcome. Post your comment or question below…

Gmail antispam filter, gmail spam filter, spam blocker, Google machine learning, spam filter rules

 
Ask Your Computer or Internet Question

 
  (Enter your question in the box above.)

It's Guaranteed to Make You Smarter...

AskBob Updates: Boost your Internet IQ & solve computer problems.
Get your FREE Subscription!


Email:

Check out other articles in this category:



Link to this article from your site or blog. Just copy and paste from this box:

This article was posted by on 20 Sep 2022


For Fun: Buy Bob a Snickers.

Prev Article:
A Virus Destroyed Your Hard Drive?

The Top Twenty
Next Article:
Geekly Update - 21 September 2022

Most recent comments on "My Near-Perfect Spam Filter"

Posted by:

Hoophead
20 Sep 2022

I gotta agree with you, Bob, 100%


Posted by:

Bob K
20 Sep 2022

Actually, I disabled the SPAM filter in Gmail. While it is good, it was, at times, trapping valid emails.

By somewhat careful email practices I have gotten so not much SPAM comes my way. People that forward everything, and don't delete email addresses out of what they forward, are the cause of the spammers collecting email addresses.

I use Thunderbird here as an email client. When I do get spam, I look at the IP address where it came from, and add the leading part of that into the Thunderbird's filtering. More messages from that block of IP addresses get sent into another folder, not my inbox. I can review those when I have time, to make sure good things don't end up there;


Posted by:

jbakerjonathan
20 Sep 2022

An excellent article. Thank you. I assume that, if you use a Gmail account to screen your email before forwarding them to your preferred email address, the Gmail account gets filled up with both spam and legitimate email, thus requiring deletion of both legitimate and spam emails from the Gmail account. That action may require frequent attention if you receive 50 - 100 emails a day. Is there a work-around for that?


Posted by:

Craig
20 Sep 2022

Thanks for the information Bob. Recently my AOL spam box has been receiving about 40 messages daily(at least most of them are directed to spam not the inbox) so apparently I am being "spam bombed". This problem has worsened ever since AOL became part of Verizon


Posted by:

Jake O'
20 Sep 2022

To prevent spam from being sent to you, a disposable email address can be valuable. Read Bob's article on that topic which is linked above.


Posted by:

Paul
20 Sep 2022

While the spread of spam is reducing, the vast majority of cyber annoyances now stem from booming online advertising which makes some pages virtually unreadable.


Posted by:

Barry
20 Sep 2022

Sorry ... gmail has gotten horrid in recent years. The false positives have made the spam filter nearly unusable. Until I stopped using it, I had anywhere from 15 to 30 false positives a day. No matter how many times I tried to "teach" gmail by marking something as "not spam" ... it never learns. I'm down to about 2 to 3 false positives on email to my gmail account, and have almost no false positives from my Eudora mail reader.


Posted by:

Nat
20 Sep 2022

I started using the gmail spam filter a few years ago. As I recall I did this following an article in this newsletter. I just set up my various email accounts to automatically forward to my gmail account. Since I prefer a local email client, I just set it up to grab my emails from my gmail account. The only problem I have run into is that gmail no longer allows me to download emails via POP and I dislike the way IMAP works.


Posted by:

Allan
20 Sep 2022

Yahoo's spam filters are OK if the spammer only uses one IP address to send out spam, but as we all know they-spammers use hundreds, sometimes thousands of IP addresses to send out those ugly spam mails.

Yahoo also doesn't make it easy to block domains as well, IF one knows how to do that!


Posted by:

vcragain
20 Sep 2022

This could be dangerous unless you do go & check everything that gets diverted into your extraneous gmail account. I get several hundred spams a day, but maybe 2 or 3 times a week I have to rescue something that got labeled as spam that was not really so. Therefore I will never just delete all of the contents & have to scour the list carefully to make sure I am not deleting something valuable. So no advantage to creating yet another email box.


Posted by:

MartinW
20 Sep 2022

I use multiple email addresses with many different providers. I ALWAYS check the Spam folder. At the addresses where I get many emails daily (~600 daily at my most-used address), there's at least a 95% chance a valid email will be in Spam.Then in the Inbox there are usually at least a couple that are pure Spam. Also, about half (or more) of the "valid" emails are just junk about things in which I am not the least bit interested. I don't suppose there's much I can do about them except delete them. I don't want to mark them as Spam since, just once in a while, I find something truly interesting. (BTW, I delete 50-60% of these without reading, just checking the headers.) Then I go to all my OTHER email addresses. Fun, fun, fun!


Posted by:

RandiO
21 Sep 2022

Thank you Bob Rankin for all that you do for us.
:All Your data Are Belong to Us: takes on a whole new dimension. "The ANN figured it out for itself... If it can do that, ... In 2019, Google announced that thanks to their machine learning framework known as TensorFlow, they are able to ...{add}... an additional 100 million...messages daily..." to all the other meta-data.
Does the ANN have enough 'conscience' to know the limits of privacy versus surveillance?


Posted by:

Daniel Wiener
21 Sep 2022

My main email client is Gmail, but instead of using it as an intermediary I use one of several different email addresses (whose domain names I've purchased) and have them forward email to my Gmail account. (Then when I send emails from Gmail I use those other addresses as my sending aliases.)

This means I am technically independent of Gmail. If I ever wanted to I could instantly switch to another host (e.g., Yahoo or Outlook) and forward my emails there, without having to inform anyone about a new address.

But back to the point about spam. Gmail does a good job of filtering, but I still do a quick eyeball scan through the spam folder, just in case. And I sometimes (albeit very rarely) spot something important that I didn't want filtered.


Posted by:

Charley
21 Sep 2022

Bob, you are probably aware the Google has announced that they will be changing their spam filter so that it will no longer block political emails as spam. I will find that VERY annoying as I get tons of spam emails from both parties (and various groups on the extremes).

EDITOR'S NOTE: Yes, that's true. But at least you can still create filters to block them.


Posted by:

Karen
21 Sep 2022

I agree with the others who noted that Google puts legitimate emails in the spam folder. I check mine every day to retrieve desired emails. I admit that it has gotten better lately, but for several years, Google would randomly put an email from a sender I regularly receive and read, into the spam folder. I also observed that it would "hide" certain mailings and then I would have to do a search in order to see it. This is very annoying and irritating.


Posted by:

wts
21 Sep 2022

I control spam with a program called MailWasherPro, which gives complete control if used properly. (I have no relationship with the company behind it - I am just a satisfied user).


Posted by:

BaliRob
21 Sep 2022

Similar to today's subject.

Yahoo have been unable to open your Newsletter in my VIVO Android for quite a while now - what can be blocking you please?

Desktop as always ok


Posted by:

misterfish
21 Sep 2022

Hi Bob,
Yes, Gmail is improving but it still sends good messages to spam, which means I am obliged to visit the gmail website to check the spam box for incoming work contracts, especially annoying as I use an email client (T'bird) to manage all my email accounts in one screen. I reckon gmail do it deliberately to oblige folk with email clients to visit their site, no other logical explanation for why emails from clients safely received for years should suddenly be dumped in spam. Wotchafink?


Posted by:

Tom
22 Sep 2022

Email Forwarding ...does not always work...in using Spectrum email forwarding , I find about 3-5% of the emails are never forwarded.


Post your Comments, Questions or Suggestions

*     *     (* = Required field)

    (Your email address will not be published)
(you may use HTML tags for style)

YES... spelling, punctuation, grammar and proper use of UPPER/lower case are important! Comments of a political nature are discouraged. Please limit your remarks to 3-4 paragraphs. If you want to see your comment posted, pay attention to these items.

All comments are reviewed, and may be edited or removed at the discretion of the moderator.

NOTE: Please, post comments on this article ONLY.
If you want to ask a question click here.


Free Tech Support -- Ask Bob Rankin
Subscribe to AskBobRankin Updates: Free Newsletter

Copyright © 2005 - Bob Rankin - All Rights Reserved
Privacy Policy     RSS/XML


Article information: AskBobRankin -- My Near-Perfect Spam Filter (Posted: 20 Sep 2022)
Source: https://askbobrankin.com/my_nearperfect_spam_filter.html
Copyright © 2005 - Bob Rankin - All Rights Reserved