Deep Web, Dark Web: What's Out There?

Category: Reference

Every so often, you'll hear a scary story in the news about the spooky, dangerous “Dark Web” where criminals, terrorists, hackers, and spammers conspire to victimize unwitting Internet users. The reality is a bit more balanced and not so scary. There is a “Deep Web” that you can't access with ordinary search engines, and a “Dark Web” where people lurk anonymously for both good and evil purposes. Read on...

What's Hiding in the Deep and Dark Webs?

Technically defined, the “Deep Web” is simply that vast portion of the Web that search engines don’t (or can't) index. While Google, Yahoo, and other search engines can provide billions more Web pages than you can live to view, that still leaves over 90% of Internet destinations unsearchable. (One source I found estimated that over 130 trillion individual Web pages exist.) If you don’t know the URL (web page address), you can’t just “google” it. You have to find the “secret” URL some other way.

The majority of the Deep Web is unindexed simply because it’s uninteresting. Most of the Internet of Things fits into this category. Who really wants to google the URL of a light bulb, doorbell, or toaster? (See my related article Things That Should NOT Be Connected To The Internet.)

There are also password-protected sites that are accessible only to those with memberships or subscriptions to the content stored there. If there's a lock on the door, search engines can't get in to index the pages stored at that location. Newspapers, professional journals, and research databases fall into this category. Many local government websites offer access to public records, such as real estate and legal filings. You can access them, often without a password, but these databases will not be indexed. That's because the search engine "spider" doesn't know what to do when it sees the search input box.

Deep Web and Dark Web

And then there are websites that perform dynamic, real-time queries for things like travel. You can go to Expedia and find out how much it will cost for a ticket from NYC to Miami on July 23rd, but the results won't be indexed by search engines, because they can change from one minute to the next.

And of course there are legions of websites that just have no useful content. They may be spammy, scammy, ripoffs or duplicates that will never appear in search results, because search engines have gotten wise to many of the tricks that black-hats use to "game" the search results. There are also websites that have no inbound links (links from pages on other sites) so search engines will never find them.

In my article Here's How to Search The Deep Web, I list several tools and tips that can help you explore the Deep Web.

So the Deep Web isn't scary -- it's just a part of the Internet that can't be (or hasn't been) indexed by search engines, or pages that require human interaction to continue on to the desired content or search query.

The Dark Side of the Internet

But what’s out there in the dark parts of the Web? Yes, there are bad places, people, and activities; they’re part of what’s called the “Dark Web” or “Darknet” for dramatic effect. “The Silk Road” was one infamous criminal site where drugs, weapons, data, hacking services and all manner of illicit things were traded until the FBI arrested its owner back in October, 2013. Some referred to Silk Road as the of the underworld, because it made shopping for illegal goods so easy.

Ross William Ulbricht, known by his hacker handle "Dread Pirate Roberts," was nailed on charges of narcotics trafficking conspiracy, computer hacking conspiracy and money laundering conspiracy. Court documents allege that over $1.7 million in illegal money changed hands each month on The Silk Road. Other black market websites exist, but Silk Road was the best known.

Private forums exist where cyber-criminals offer services such as hacking, denial of service attacks, ransomware and phishing scams. There are images on dark web pages that you would wish to forget after seeing them. And certainly, terrorists use the Internet to communicate and collaborate.

Sometimes You Need to Hide

But there are also oases of light in the Dark Web that can’t be called dark by any means. They’re where the struggle for freedom rages.

Dissidents, journalists, peace activists, and other good guys often need to hide their activities from oppressive governments and other institutions. Many citizens of totalitarian nations cannot freely access uncensored news or trade opinions and facts about politics or corruption. Some of these people turn to the Dark Web, to hidden forums, sites, and servers of information that protect their secrets and identities.

One of the most popular privacy tools is called Tor. Tor is, essentially, a network of Web proxy servers and browser software designed for them. When using the Tor browser, your identity and location are obscured and your connection to the Tor network is encrypted. Even your ISP doesn’t know where you’re really going because they can’t read the data stream that passes between you and the Tor proxy server. All anyone knows is that you accessed a Tor server.

Your requests for Web content go to a Tor server, which then reaches out to grab the requested content and relay it back to you over that encrypted connection. The destination site sees the Tor server’s location and ID but never yours. Theoretically, there is no way to tell what you accessed via a Tor server.

So the Dark Web is actually a mixture of light and dark, good and evil, benefit and harm. It’s symbol might well be the Yin Yang which illustrates how opposite forces can be interconnected and intermingled.

Your thoughts on this topic are welcome. Post your comment or question below...

Ask Your Computer or Internet Question

  (Enter your question in the box above.)

It's Guaranteed to Make You Smarter...

AskBob Updates: Boost your Internet IQ & solve computer problems.
Get your FREE Subscription!


Check out other articles in this category:

Link to this article from your site or blog. Just copy and paste from this box:

This article was posted by on 1 Oct 2019

For Fun: Buy Bob a Snickers.

Prev Article:
[SECURITY] Your Password Is Not Enough

The Top Twenty
Next Article:
Geekly Update - 02 October 2019

Most recent comments on "Deep Web, Dark Web: What's Out There?"

Posted by:

01 Oct 2019

"Who really wants to google the URL of a light bulb, doorbell, or toaster?" Funniest line ever!

Posted by:

Ryan James
01 Oct 2019

This is a topic that has tickled my curiosity for some time, but I have been leery about exploring.

Would this be valuable for finding clues for genealogy research, I wonder?

How about a follow-up article on the benefits of the dark side for the average legal Internet user?

Posted by:

Len Barnes
01 Oct 2019

I've made several attempts to find useful content using Tor and DuckDuckGo but I've not had much success - only finding pages I could reach through a standard Google search. An article giving some insight into how to use the Deep Web would be really helpful.

Posted by:

01 Oct 2019

Personally I feel that the average legal Internet user would never need to access the Dark Web.

As for the Deep Web, like the article above says, its not indexed because its uninteresting or because it requires you to be a member of that site to access. I'm sure there are some genealogy research sites out there that require membership (maybe free) to access.

Len: Hidden Tor sites (Dark Web) are only indexed manually. Their URLs look like a random string of characters and end in .onion There are several articles out there that list a few starting points, with the proviso that Tor sites come and go and may no longer exist. I've tried a few of the index sites and its sad that its seems that no matter what is searched for, the results included sites that no one should ever want to see.

Posted by:

01 Oct 2019

Bob: "So the Dark Web is actually a mixture of light and dark. . . " -

Me: SO, it's really more of a gray-web, right?

Posted by:

02 Oct 2019

I finally had cause to venture onto the Dark Web myself or the first time a few weeks ago. After taking precautions (i.e. using a bootable Linux tailored for TOR - TAILS, I think) (didn't need a VPN up front because 1) I'm not that paranoid, and 2) my ISP doesn't block TOR.) I poked around for a little bit. I found the ancient book I was looking for, but then I looked around at some of the illicit sites, just to see what they looked like. It sort of reminded me of the early days of the internet, where you had to keep your own log of sites (before Google), and the pages were strictly utilitarian with simple graphics. There was none of this force-fed advertising or auto-running video streams (mostly due to the bandwidth). I think the good it does vastly outweighs the bad. Personally I would never try to buy anything on the illicit sites because the term "Dark Web" has entered our everyday lives so much through news stories and even TV shows, that I would be afraid the people on the other end (trying to sell you something illicit) would either be scammers or cops. But, all-in-all, I'm glad its there.

Thanks to Bob for his excellent description of the differences between Dark Web and Deep Web.

Posted by:

Gerald Goldberg
02 Oct 2019

If you could see the BS that I get when I Google on Yahoo and Google! I get everything but what I searched for. I just typed in: Adolf Hitler Bolshevik and Zionist on a Proxy site. It said: About 220,000 results (0.51 seconds). So I went and clicked NEXT and this is what I got:HTTP Status 404 – Not Found

Description The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

Apache Tomcat/8.5.14 (Debian)

Big Brother doesn't want me to get the information I want & I must be on some list that even if a try use a Proxy site, they block me! Have you any advice for me?

Posted by:

11 Oct 2019

Gerald, the BS you see on the internet is merely a reflection of what's going on in your mind. Why don't you visit Israel and see the truth? It's a democratic country where Arab citizens get to vote. In all the surrounding countries which have attacked Israel several times in the last century, are there Jews who get to vote? Bob, could you please remove Gerald's hateful inaccuracies?

Posted by:

24 Oct 2019

There are search engines specializing in the Dark Web, and some of the stuff they turn up will turn your stomach. I haven't yet found anything useful through them.

Post your Comments, Questions or Suggestions

*     *     (* = Required field)

    (Your email address will not be published)
(you may use HTML tags for style)

YES... spelling, punctuation, grammar and proper use of UPPER/lower case are important! Comments of a political nature are discouraged. Please limit your remarks to 3-4 paragraphs. If you want to see your comment posted, pay attention to these items.

All comments are reviewed, and may be edited or removed at the discretion of the moderator.

NOTE: Please, post comments on this article ONLY.
If you want to ask a question click here.

Free Tech Support -- Ask Bob Rankin
Subscribe to AskBobRankin Updates: Free Newsletter

Copyright © 2005 - Bob Rankin - All Rights Reserved
Privacy Policy     RSS/XML

Article information: AskBobRankin -- Deep Web, Dark Web: What's Out There? (Posted: 1 Oct 2019)
Copyright © 2005 - Bob Rankin - All Rights Reserved