Beware The Deep, Dark Web!

Category: Search-Engines

Every so often, the mainstream media likes to tell us a scary story about the spooky, dangerous “Dark Web” where criminals, terrorists, hackers, and spammers conspire to victimize all of us. The reality is a bit more balanced and not so scary. There is a “Deep Web” where nobody wants to go and a “Dark Web” where people lurk anonymously but most of them are up to legitimate, even noble business.

What's Hiding in the Deep and Dark Webs?

Technically defined, the “Deep Web” is simply that vast portion of the Web that search engines don’t index. While Google, Yahoo, and other search engines can provide billions more Web pages than you can live to view, that still leaves over 90% of Internet destinations unsearchable. If you don’t know the URL (web page address), you can’t just “google” it. You have to find the “secret” URL another way, usually getting it from someone who knows it. So what’s out there in the deep, dark part of the Web?

Yes, there are bad places, people, and activities; they’re part of what’s called the “Dark Web” or “Darknet” for dramatic effect. “The Silk Road” was one infamous criminal site where drugs, weapons, data, hacking services and all manner of illicit things were traded until the FBI arrested its owner back in October, 2013. Some referred to Silk Road as the of the underworld, because it made shopping for illegal goods so easy.

Deep Web and Dark Web

Ross William Ulbricht, 29, known by his hacker handle "Dread Pirate Roberts," was nailed on charges of narcotics trafficking conspiracy, computer hacking conspiracy and money laundering conspiracy. Court documents allege that over $1.7 million in illegal money changed hands each month on The Silk Road. Other black market websites exist, but Silk Road was the best known.

Most of the rest of the Deep Web is unindexed simply because it’s uninteresting. Most of the Internet of Things fits into this category. (See What is the Internet of Things?) Who really wants to google the URL of a toaster? There are also password-protected sites that are accessible only to those with memberships or subscriptions to the content stored there. If there's a lock on the door, search engines can't get in to index the pages stored at that location.

And then there are legions of websites that just have no useful content. They may be spammy, scammy, ripoffs or duplicates that will never appear in search results, because search engines have gotten wise to many of the tricks that black-hats use to "game" the search results. There are also websites that have no inbound links, so search engines will never find them.

Sometimes You Need to Hide

But there are also oases of light in the Deep Web that can’t be called dark by any means. They’re where the struggle for freedom rages.

Dissidents, journalists, peace activists, and other good guys often need to hide their activities from oppressive governments and other institutions. Many citizens of China, Iran and other totalitarian nations cannot freely access uncensored news or trade opinions and facts about politics or corruption. Some of these people turn to the Deep Web, to hidden forums, sites, and servers of information that protect their secrets and identities. Law enforcement and the military also use Tor to protect their communications and intelligence gathering.

One of the most popular privacy tools is called Tor. Tor is, essentially, a network of Web proxy servers and browser software designed for them. When using the Tor browser, your identity and location are obscured and your connection to the Tor network is encrypted. Even your ISP doesn’t know where you’re really going because he can’t read the data stream that passes between you and the Tor proxy server. All anyone knows is that you accessed a Tor server.

Your requests for Web content go to a Tor server, which then reaches out to grab the requested content and relay it back to you over that encrypted connection. The destination site sees the Tor server’s location and ID but never yours. Theoretically, there is no way to tell what you accessed via a Tor server.

So the Deep Web is like any other phenomenon: a mixture of light and dark, good and evil, benefit and harm. It’s symbol might well be the Yin Yang which illustrates how opposite forces can be interconnected and intermingled.

Your thoughts on this topic are welcome. Post your comment or question below...

Ask Your Computer or Internet Question

  (Enter your question in the box above.)

It's Guaranteed to Make You Smarter...

AskBob Updates: Boost your Internet IQ & solve computer problems.
Get your FREE Subscription!


Check out other articles in this category:

Link to this article from your site or blog. Just copy and paste from this box:

This article was posted by on 29 May 2014

For Fun: Buy Bob a Snickers.

Prev Article:
Geekly Update - 28 May 2014

The Top Twenty
Next Article:
Facebook Tightens Privacy Protections

Most recent comments on "Beware The Deep, Dark Web!"

Posted by:

29 May 2014

This is a very interesting and enlightening article; thanks. Since there are only a finite (about 256^4) IP addresses, it would seem at least finding web sites that are not indexed should be possible (at least in theory). I wonder if some search providers will ever attempt to at least categorize (if not index) IP addresses (e.g., password protected or not).

EDITOR'S NOTE: There's a problem with your theory... Many websites can share an IP address. And each website can have an unlimted number of pages.

Posted by:

29 May 2014

You know Bob Rankin {of course, I know him!]?
I must admit that you actually put a lot of research and accuracy to your content and it is truly admirable. Even if I did not have ANY interest in anything you create every darn-tootin' day of the week, I would still read your daily inputs, just for the fresh and novel smell of it all!
Like I would have never known that the plural of 'oasis' would be 'oases'! Not that I have seen one, let alone many!
Thank you and kudos!
PS. Although I must interject (for those of the conspiracy-theorist inclined in the bunch) with the following link ragarding Tor+NSA >>

Posted by:

Mary A. Axford
29 May 2014

As a librarian, I depend on the deep web, as most of the databases, journals, and ebooks we use reside there. I also try to teach students about the deep web, as they tend to think they will find everything they need on Google and Wikipedia - and our professors are screaming that they are not finding the scholarly material. Anyway, I am always trying to find the right words to convey to them that Google and Wikipedia are good tools for certain things, but that I am showing them other tools and why they are useful. Anyway, your comments on what the deep web is may inform what I say to my students. Thanks! I find your articles so very useful.

Posted by:

Gloria Huffman
29 May 2014


This is the perfect place to note that I have personally double-checked the actual number of many Google results and I have repeatedly found that the stated number of results is inflated enormously, by 10 times or even more. Google says 10,000? It could be only 37. Google says 1,517,249? It could be only 732.

So don't let the huge number of reported results deter you from clicking through 70+ pages of results if you really need a piece of information. Seeing Google's claim for a huge number of results does two main things:

1. It lies and makes you think Google is so marvelous and has the "largest number of results of any search engine."

2. It probably makes you tired before you even begin to look through those results pages, keeping you tied to the first few pages (where people and businesses have paid to position themselves at the top of the results).

Posted by:

29 May 2014

Hey Bob, I just want to echo RandiO's and Mary's sentiments. The objective research you do and publish is not only prodigious but informative and thus wholly commendable. You're the type of bloke that really makes a difference to this world within (and probably without) the ambit of your expertise and I can only hope you are given the public recognition you deserve not that I think for a moment you seek it. Much respect.

Posted by:

Marc de Piolenc
29 May 2014

One of your better pieces - most articles on this topic fall in my "hysteria amplification" category, but yours is factual.

Posted by:

30 May 2014

There's also the Invisible Web of professional journals and research. I used to have a search engine just for that. Does anyone know if that's included in the Deep Dark Web?

Posted by:

Sandy Papavasiliou
31 May 2014

There are 'Ask Bob' Ad Choices popping up on this site. How do I get rid of them?

Posted by:

Frank Starr
31 May 2014

Mary A. Axford, do you have some internet sites that reference the material you pass on to students?

Gloria Huffman, thanks for your info. I use to overcome Google's tendency to pare results to market to you, and I use to catch what Google via might miss. Do you use other search engines regularly?

Posted by:

03 Jun 2014

Bob - It's a MAJOR omission on your part that you missed explaining how Tor was hacked by the FBI last year and only the older Tor versions are clean of this infestation.

If you REALLY plan to use Tor, as it was originally intended and to preserve your privacy, don't run out and get the new release. Use the version which is from Jan. 2013. Here is the last clean Tor release version number: 10.0.12

EDITOR'S NOTE: Sometime in 2013, the FBI hit the Tor network with malware that threatened to reveal the anonymous users' identities by exploiting a security flaw in the Firefox browser. A server in Ireland that was known for hosting child porn was compromised. That's MUCH different than "hacking" the TOR software itself. The FBI exploited a vulnerability in Firefox ESR 17, on which the Tor browser is based. They did not hack the Tor software. In addition, the Tor Browser Bundle 3.6, released in April 2014, uses Firefox 24.5. So the best solution is to download the LATEST version of the Tor Bundle, not an older one.

Post your Comments, Questions or Suggestions

*     *     (* = Required field)

    (Your email address will not be published)
(you may use HTML tags for style)

YES... spelling, punctuation, grammar and proper use of UPPER/lower case are important! And please limit your remarks to 3-4 paragraphs. If you want to see your comment posted, pay attention to these items.

All comments are previewed, and may be edited before posting.

NOTE: Please, post comments on this article ONLY.
If you want to ask a question click here.

Free Tech Support -- Ask Bob Rankin
RSS   Add to My Yahoo!   Feedburner Feed
Subscribe to AskBobRankin Updates: Free Newsletter
Copyright © 2005 - Bob Rankin - All Rights Reserved
Privacy Policy -- See my profile on Google.

Article information: AskBobRankin -- Beware The Deep, Dark Web! (Posted: 29 May 2014)
Copyright © 2005 - Bob Rankin - All Rights Reserved