Pi-hole dot net and hardware ad blocking

[pyotr filipivich]

An Interesting thing to come up in my feed.

quote:

PRIVACY: Inside the Brotherhood of the Ad Blockers.

Anyone who works in the $200 billion digital advertising industry
should be scared of people like Mark Drobnak, because the ad blocker
he uses is way more powerful than yours. The college freshman says it
feels as though everyone at Rochester Institute of Technology, from
his roommate to his professors, has installed some way to ward off
online ads. Drobnak is one of the die-hards who goes further, working
with a handful of comrades to build what they call “a black hole for
advertisements.” His parents say the one he built them works great.

Pi-hole (as in “shut your?…”) is a free, open source software
package designed to run on a Raspberry Pi, a basic computer that’s
popular with DIYers, fits in the palm of your hand, and retails for
about $35. Most ad blockers have to be installed on individual devices
and work only in web browsers, but Pi-hole blocks ads across an entire
network, including in most apps.

Interesting. I get similar results taming the most obnoxious sites
using a customizable JavaScript blocker, but it can be tedious to set
up and the results aren’t always easy to predict. I’ve thought about
picking up a cheap PC and setting it up as a Ubuntu-based web server
with ad and tracking firewalls installed, but the effort-to-reward
ratio still breaks on my lazy side.

But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.

Endquote.

https://pi-hole.net/

Comments from the technoratti?

[Paul]

Post by pyotr filipivich
An Interesting thing to come up in my feed.
PRIVACY: Inside the Brotherhood of the Ad Blockers.
Anyone who works in the $200 billion digital advertising industry
should be scared of people like Mark Drobnak, because the ad blocker
he uses is way more powerful than yours. The college freshman says it
feels as though everyone at Rochester Institute of Technology, from
his roommate to his professors, has installed some way to ward off
online ads. Drobnak is one of the die-hards who goes further, working
with a handful of comrades to build what they call “a black hole for
advertisements.” His parents say the one he built them works great.
Pi-hole (as in “shut your?…”) is a free, open source software
package designed to run on a Raspberry Pi, a basic computer that’s
popular with DIYers, fits in the palm of your hand, and retails for
about $35. Most ad blockers have to be installed on individual devices
and work only in web browsers, but Pi-hole blocks ads across an entire
network, including in most apps.
Interesting. I get similar results taming the most obnoxious sites
using a customizable JavaScript blocker, but it can be tedious to set
up and the results aren’t always easy to predict. I’ve thought about
picking up a cheap PC and setting it up as a Ubuntu-based web server
with ad and tracking firewalls installed, but the effort-to-reward
ratio still breaks on my lazy side.
But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.
Endquote.
https://pi-hole.net/
Comments from the technoratti?

https://www.bloomberg.com/news/features/2018-05-10/inside-the-brotherhood-of-pi-hole-ad-blockers

I think you're saying, you want a "full time hobby" ?

Paul

[pyotr filipivich]

Post by Paul

Post by pyotr filipivich
An Interesting thing to come up in my feed.
PRIVACY: Inside the Brotherhood of the Ad Blockers.
Anyone who works in the $200 billion digital advertising industry
should be scared of people like Mark Drobnak, because the ad blocker
he uses is way more powerful than yours. The college freshman says it
feels as though everyone at Rochester Institute of Technology, from
his roommate to his professors, has installed some way to ward off
online ads. Drobnak is one of the die-hards who goes further, working
with a handful of comrades to build what they call “a black hole for
advertisements.” His parents say the one he built them works great.
Pi-hole (as in “shut your?…”) is a free, open source software
package designed to run on a Raspberry Pi, a basic computer that’s
popular with DIYers, fits in the palm of your hand, and retails for
about $35. Most ad blockers have to be installed on individual devices
and work only in web browsers, but Pi-hole blocks ads across an entire
network, including in most apps.
Interesting. I get similar results taming the most obnoxious sites
using a customizable JavaScript blocker, but it can be tedious to set
up and the results aren’t always easy to predict. I’ve thought about
picking up a cheap PC and setting it up as a Ubuntu-based web server
with ad and tracking firewalls installed, but the effort-to-reward
ratio still breaks on my lazy side.
But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.
Endquote.
https://pi-hole.net/
Comments from the technoratti?

https://www.bloomberg.com/news/features/2018-05-10/inside-the-brotherhood-of-pi-hole-ad-blockers
I think you're saying, you want a "full time hobby" ?

Well, yes I do.

But I do not think that is the one I really want.

Not yet anyway.

Post by Paul
Paul

[Mayayana]

"pyotr filipivich" <***@mindspring.com> wrote

|
| https://pi-hole.net/
|
| Comments from the technoratti?

Less than meets the eye. You need to set it up
on something Linux, and it's nothing really new.
It uses several HOSTS-type files, found in
adlists.default, in the download package:

https://github.com/pi-hole/pi-hole/archive/master.zip

A HOSTS file on Windows does similar. Windows
also has DNS proxy software available, such as
Acrylic. (pi-hole is a DNS proxy. A set of scripts,
with no finished interface as far as I can tell. You
just run the install command and then their list
decides what you block, without your intervention.)

Acrylic allows wildcards in its HOSTS file, so it's fairly
easy to set up a very efficient blocking list. And it's
easy to edit. No command line nonsense. Acrylic also
has the appeal of not having a crass name invented
by a teenager.

The pi-hole approach is highly inefficient. Example:

Here's the list of HOSTS-type files they use:

https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://mirror1.malwaredomains.com/files/justdomains
http://sysctl.org/cameleon/hosts
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://hosts-file.net/ad_servers.txt

That's 7 HOSTS-like files ll fed through their
script. The last one alone is 1.68 MB. It has
over 12,000 entries for Doubleclick! Over 3,500
entries for atdmt.com. Though most seem to
have an odd format ending with 302br.net. I've
never seen 302br.net before. Yet that block list
has some 15,000 versions of its URL. Similarly,
there are lots of 2o7.net, which is a Google
spyware/adware alias.

Many advertisers will have a separate URL,
or more than one, for each client. Things
like 32441.nyt-ads.liveclick.net (I just made
that up.) You only need that in your list if you
like to go to the NYT website. Then you'll probably
also need dozens of others. 32442, 3243k, and so
on. A new one every time they generate a new
randomly named subdomain. But with Acrylic you
can stop all that with just a few lines:

127.0.0.1 *.google-analytics.com
127.0.0.1 *.doubleclick.net
127.0.0.1 *.doubleclick.com
# There's all of doubleclick in 2 lines.
127.0.0.1 *.302br.net
127.0.0.1 *.2o7.net
127.0.0.1 *.atdmt.com

I occasionally download especially commercial
pages to check for new spyware companies, but
mostly it's the same culprits on every site. Google/
Doubleclick, scorecardresearch, etc.

What about fonts.googleapis.com? You may
want to allow web fonts and you may want
to allow that domain. If so then you're
giving Google enough data to track most of
your activity online. Does pi-hole block that?
I don't know, but probably not. Does it block
Facebook and Twitter tracking bugs? Probably
not, since most people want to be able to
reach those websites.

In other words, with a basic Windows DNS
proxy that blocks with wildcards, you can
easily block almost everything that matters.
And you don't need to buy a raspberry pi or
get into a mess of Linux scripts.

I don't remember the last time I saw an ad,
and all I use is Acrylic. But I never saw ads even
before Acrylic, with about 300 entries in my
HOSTS file. It's efficient to use that method
because the spyware and ads are extremely
centralized. (That's also what makes them so
pernicious. There are numerous domains that
can track almost everything you do because
they load something on almost every page you
visit.)

[pyotr filipivich]

Post by Mayayana
|
| https://pi-hole.net/
|
| Comments from the technoratti?
Less than meets the eye. You need to set it up
on something Linux, and it's nothing really new.
It uses several HOSTS-type files, found in

Thanks for the input. I will read it more thoughtfully.

For the nonce, let me say "Sounds cool, but sounds like more
'hobby' than I want.

[J. P. Gilliver (John)]

In message <pdnkg4$ib8$***@dont-email.me>, Mayayana
<***@invalid.nospam> writes:
[]

Post by Mayayana
Acrylic allows wildcards in its HOSTS file, so it's fairly

[]

Post by Mayayana
127.0.0.1 *.doubleclick.net
127.0.0.1 *.doubleclick.com
# There's all of doubleclick in 2 lines.

[]
I think I asked once before couldn't it could be just _one_ line,
0 *.doubleclick.*
(or possibly without the dots), but I don't remember the answer.

[Mayayana]

"J. P. Gilliver (John)" <G6JPG-***@255soft.uk> wrote

| >127.0.0.1 *.doubleclick.net
| >127.0.0.1 *.doubleclick.com
| ># There's all of doubleclick in 2 lines.
| []
| I think I asked once before couldn't it could be just _one_ line,
| 0 *.doubleclick.*
| (or possibly without the dots), but I don't remember the answer.

Yes, that sounds familiar.
If I remember correctly, it turned out that Acrylic
can handle RegExp, so you can do all kinds of things.
These two samples are included in its HOSTS file:

# 127.0.0.1 ad.*ads.*
# 127.0.0.1 /^ads?\..*$

In general I don't know how useful RegExp would
be here. Or even multiple wildcards. I find a subdomain
wildcard is adequate and I like to keep it simple.
Filtering on strings like "ads" or "banner" used to be
common, but that was back in the days when ads
were coming from the same URL as the webpage.

But, yes, there seems to be no limit to how much
you can customize and specialize with the Acrylic
HOSTS file.

[VanguardLH]

Post by J. P. Gilliver (John)

Post by Mayayana
Acrylic allows wildcards in its HOSTS file, so it's fairly
127.0.0.1 *.doubleclick.net
127.0.0.1 *.doubleclick.com
# There's all of doubleclick in 2 lines.

I think I asked once before couldn't it could be just _one_ line,
0 *.doubleclick.*
(or possibly without the dots), but I don't remember the answer.

The hosts file lists *hosts*, not domains. That is why it is called a
hosts file. There is no wildcarding. The hosts file was not created
for the purpose of adblocking.

https://en.wikipedia.org/wiki/Hosts_(file)

It was originally created to facilitate finding on intranet hosts
without having to access a DNS server, like having a short grocery
shopping list of what you want to buy rather than the entire grocery
store's inventory and checking off just the items you want to buy.

That Acrylic allows wildcarding is unique to that proxy in its
interpretation of the content of the hosts file. If you modify the
hosts file to add wildcarding, the DNS client or anything else reading
the hosts file won't know how to parse it because of the illegal syntax.

http://mayakron.altervista.org/wikibase/show.php?id=AcrylicFAQ#3
"Putting a large number of patterns or regular expressions inside the
AcrylicHosts.txt file may cause Acrylic to slow down significantly."

So it is not the hosts file where you can use wildcarding. It is
Acrylic's own hosts file (acrylichosts.txt) where you can use
wildcarding. This is confirmed also at the following page:

http://mayakron.altervista.org/wikibase/show.php?id=AcrylicHosts

Also mentioned in the first article:

"A domain name is free, a pattern is relatively cheap and a regular
expression is rather expensive."

I don't use Acrylic to know how it differentiates "pattern" from
"regular expression" (other than DOS wildcarding is *not* the same as
regex). "pattern" isn't mentioned in the 2nd article describing syntax
within the AcrylicHosts.txt file.

Maybe you could specify multiple wildcards but I've always found that
has side effects of matching on substrings you didn't intend.
Wildcarding tends to be sloppy. Instead you could use regex, as in
"^(\S+\.)?doubleclick\.(com|net)$". From the above wiki page, use of
regex is slower due to the parsing and substring functions. You can get
pretty damn complicated with regex.

[Mayayana]

"VanguardLH" <***@nguard.LH> wrote

| The hosts file lists *hosts*, not domains. That is why it is called a
| hosts file. There is no wildcarding. The hosts file was not created
| for the purpose of adblocking.
|
| https://en.wikipedia.org/wiki/Hosts_(file)
|
| That Acrylic allows wildcarding is unique to that proxy in its
| interpretation of the content of the hosts file.
|
| I don't use Acrylic...

Yet you're determined to explain it. John understands.
And probably everyone else here does, too, without
needing links to the history of HOSTS files.

If you want to understand then why not just read
the instructions in Acrylic HOSTS rather than posting
all this stuff that everyone already knows?

[VanguardLH]

Post by Mayayana
| The hosts file lists *hosts*, not domains. That is why it is called a
| hosts file. There is no wildcarding. The hosts file was not created
| for the purpose of adblocking.
|
| https://en.wikipedia.org/wiki/Hosts_(file)
|
| That Acrylic allows wildcarding is unique to that proxy in its
| interpretation of the content of the hosts file.
|
| I don't use Acrylic...
Yet you're determined to explain it. John understands.
And probably everyone else here does, too, without
needing links to the history of HOSTS files.

I was determined to find out WHICH file Acrylic actually used. You
misled by saying it was the 'hosts' file. Not true. Acrylic uses its
own and separate acrylichosts.txt file. It seemed dangerous to be
putzing with the standard 'hosts' file by putting entries within it that
nothing other than Acrylic would understand.

Post by Mayayana
If you want to understand then why not just read
the instructions in Acrylic HOSTS rather than posting
all this stuff that everyone already knows?

Provide evidence that "everyone one already knows" to circumvent your
misleading statement that the hosts file had wildcards. Um, so where
was that hidden help where you illustrated how to perform filtering on
Doubleclick in one line using regex? If pyotr knew about the standard
'hosts' file then why is using a DNS server such news to him? I see you
didn't bother to offer explanation of how Acrylic differentiates a
"pattern" from regex.

Oh, so someone that doesn't use Acrylic DNS cannot comment about it. Uh
huh. Well, gee, you said "If I remember correctly, it turned out that
Acrylic can handle RegExp." Doesn't sound like you are too intimate
with the product, either.

[Mayayana]

"VanguardLH" <***@nguard.LH> wrote

| Oh, so someone that doesn't use Acrylic DNS cannot comment about it.
|

Take a breath, V. Then if you want to know
how it works you can download Acrylic and try it.
It's free. The instructions, such as they are, are
in the HOSTS file.

(Yes, the Acrylic HOSTS file, named AcrylicHosts.txt.)

[Frank Slootweg]

Post by VanguardLH

Post by Mayayana
| The hosts file lists *hosts*, not domains. That is why it is called a
| hosts file. There is no wildcarding. The hosts file was not created
| for the purpose of adblocking.
|
| https://en.wikipedia.org/wiki/Hosts_(file)
|
| That Acrylic allows wildcarding is unique to that proxy in its
| interpretation of the content of the hosts file.
|
| I don't use Acrylic...
Yet you're determined to explain it. John understands.
And probably everyone else here does, too, without
needing links to the history of HOSTS files.

I was determined to find out WHICH file Acrylic actually used. You
misled by saying it was the 'hosts' file.

Nope, you didn't read (carefully enough) what you quoted:

<Mayayana>

Acrylic allows wildcards in its HOSTS file

</Mayayana>

Note 'its'.

Your error was probably caused by replying to J. P. Gilliver's response
to Mayayana's response, instead of directly to Mayayana's response.

Anyway, Mayayana was clear, you made an error, acknowledge it and move
on.

[Rest of straw men / red herrings deleted.]

[VanguardLH]

Post by Frank Slootweg

Post by VanguardLH

Post by Mayayana
| The hosts file lists *hosts*, not domains. That is why it is called a
| hosts file. There is no wildcarding. The hosts file was not created
| for the purpose of adblocking.
|
| https://en.wikipedia.org/wiki/Hosts_(file)
|
| That Acrylic allows wildcarding is unique to that proxy in its
| interpretation of the content of the hosts file.
|
| I don't use Acrylic...
Yet you're determined to explain it. John understands.
And probably everyone else here does, too, without
needing links to the history of HOSTS files.

I was determined to find out WHICH file Acrylic actually used. You
misled by saying it was the 'hosts' file.

<Mayayana>
Acrylic allows wildcards in its HOSTS file
</Mayayana>
Note 'its'.
Your error was probably caused by replying to J. P. Gilliver's response
to Mayayana's response, instead of directly to Mayayana's response.
Anyway, Mayayana was clear, you made an error, acknowledge it and move
on.

Then why say:

Acrylic allows wildcards in its HOSTS file

when what was meant was:

Acrylic allows wildcards in its AcrylicHosts.txt file

?

Acknowledge the misleading statement and move on.

[VanguardLH]

Post by pyotr filipivich
An Interesting thing to come up in my feed.
PRIVACY: Inside the Brotherhood of the Ad Blockers.
Anyone who works in the $200 billion digital advertising industry
should be scared of people like Mark Drobnak, because the ad blocker
he uses is way more powerful than yours. The college freshman says it
feels as though everyone at Rochester Institute of Technology, from
his roommate to his professors, has installed some way to ward off
online ads. Drobnak is one of the die-hards who goes further, working
with a handful of comrades to build what they call “a black hole for
advertisements.” His parents say the one he built them works great.
Pi-hole (as in “shut your?…”) is a free, open source software
package designed to run on a Raspberry Pi, a basic computer that’s
popular with DIYers, fits in the palm of your hand, and retails for
about $35. Most ad blockers have to be installed on individual devices
and work only in web browsers, but Pi-hole blocks ads across an entire
network, including in most apps.
Interesting. I get similar results taming the most obnoxious sites
using a customizable JavaScript blocker, but it can be tedious to set
up and the results aren’t always easy to predict. I’ve thought about
picking up a cheap PC and setting it up as a Ubuntu-based web server
with ad and tracking firewalls installed, but the effort-to-reward
ratio still breaks on my lazy side.
But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.
Endquote.
https://pi-hole.net/
Comments from the technoratti?

While most adblockers work as extensions to web browsers (and some web
browsers have them built-in) because that is obviously the most prone
vector for presenting ads from the web, it is not the only way to block
unwanted content. Adblockers can work as local proxies through which
your network (mostly web) traffic will get filtered. For example, there
are proxy adblockers for Android phones. Alas, you have to root them to
get them to work. For those that don't want to root their phones, there
is DNS66 which has your phone use an ad-filtering DNS server: DNS
requests to ad sources get blocked. Alas, while active, the Play Store
app won't work (can't get apps, can't update them). For obvious
reasons, DNS66 is not available in Google's Play Store because it
thwarts Google's Analytics revenue.

The claim to block ads across an entire network obviously means the
device with the adblocking as a filtering proxy must be upstream of all
other devices. That is, that host with the adblock proxy is the gateway
for all other hosts. Again, nothing new there.

"No client-side software required". Well, that just means the filtering
is done upstream, like with DNS66 where you merely reconfigure your
host(s) to use a different DNS server. OpenDNS, DNS66, and plenty of
other DNS providers already supply filtering. OpenDNS lets you choose
by categories of what you want to block (but I don't remember if ad
filtering are specifically supported). DNS66 simply uses the same
DNSBLs (DNS blocklists) that the web browser extensions use locally.

"Install by running one command". Huh? The site just said no
client-side software gets installed. So what does install.pi-hole.net
have for a script? The command gets piped into bash. That's is because
that web page presents a bash script, so the online script gets download
and piped into the bash interpreter. It's been about 20 years since I
last did any bash scripting and it probably wasn't as complicated as
this one, but my guess from the following line is that it changes the
DNS server assignment in your OS:

# We need to know the IPv4 information so we can effectively setup the
DNS server
# Without this information, we won't know where to Pi-hole will be found

So it looks like the same old DNS filtering that has been available for
a l-o-n-g time either via DNS providers, similar to DNS66 with their own
DNS server for the DNSBLs they choose to use. I suspect you could roll
your own using the Acrylic DNS proxy on your own gateway host on your
intranet rather than relying on the filtering definitions of a 3rd party
DNS provider, especially one that really isn't a commercial enterprise
but instead a hobby project.

[Char Jackson]

Post by VanguardLH

Post by pyotr filipivich
An Interesting thing to come up in my feed.
PRIVACY: Inside the Brotherhood of the Ad Blockers.
Anyone who works in the $200 billion digital advertising industry
should be scared of people like Mark Drobnak, because the ad blocker
he uses is way more powerful than yours. The college freshman says it
feels as though everyone at Rochester Institute of Technology, from
his roommate to his professors, has installed some way to ward off
online ads. Drobnak is one of the die-hards who goes further, working
with a handful of comrades to build what they call “a black hole for
advertisements.” His parents say the one he built them works great.
Pi-hole (as in “shut your?…”) is a free, open source software
package designed to run on a Raspberry Pi, a basic computer that’s
popular with DIYers, fits in the palm of your hand, and retails for
about $35. Most ad blockers have to be installed on individual devices
and work only in web browsers, but Pi-hole blocks ads across an entire
network, including in most apps.
Interesting. I get similar results taming the most obnoxious sites
using a customizable JavaScript blocker, but it can be tedious to set
up and the results aren’t always easy to predict. I’ve thought about
picking up a cheap PC and setting it up as a Ubuntu-based web server
with ad and tracking firewalls installed, but the effort-to-reward
ratio still breaks on my lazy side.
But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.
Endquote.
https://pi-hole.net/
Comments from the technoratti?

While most adblockers work as extensions to web browsers (and some web
browsers have them built-in) because that is obviously the most prone
vector for presenting ads from the web, it is not the only way to block
unwanted content. Adblockers can work as local proxies through which
your network (mostly web) traffic will get filtered. For example, there
are proxy adblockers for Android phones. Alas, you have to root them to
get them to work. For those that don't want to root their phones, there
is DNS66 which has your phone use an ad-filtering DNS server: DNS
requests to ad sources get blocked. Alas, while active, the Play Store
app won't work (can't get apps, can't update them). For obvious
reasons, DNS66 is not available in Google's Play Store because it
thwarts Google's Analytics revenue.
The claim to block ads across an entire network obviously means the
device with the adblocking as a filtering proxy must be upstream of all
other devices. That is, that host with the adblock proxy is the gateway
for all other hosts. Again, nothing new there.

It's a DNS proxy, so it doesn't have to be upstream. It just has to be
reachable. It doesn't even have to be on your LAN. No actual content
traffic runs through it. It only sees DNS traffic.

Post by VanguardLH
"No client-side software required". Well, that just means the filtering
is done upstream, like with DNS66 where you merely reconfigure your
host(s) to use a different DNS server.

Right, as a DNS proxy, its role is to filter/block/deny any requests for
what it thinks are ads, apparently using DNSBLs.

Post by VanguardLH
"Install by running one command". Huh? The site just said no
client-side software gets installed.

That's to install it on your shiny pi, not on any of your hosts. All
your hosts need is to swing the DNS server over, or you could do that in
your gateway device if you want a one-and-done solution.

[Char Jackson]

On Fri, 18 May 2018 12:59:08 -0700, pyotr filipivich

Post by pyotr filipivich
But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.
Endquote.
https://pi-hole.net/
Comments from the technoratti?

Others have provided some of the cons, with which I don't disagree. One
of the pros, however, is that it's positioned as a single solution that
works with every web-enabled device on your LAN, including smartphones
and smart TVs, and none of your devices has to know how to use it other
than pointing their DNS entry at it. No hosts files to play with, no
multiple configurations for multiple browsers, etc.

The one thing I didn't see mentioned is the increasingly common and
obnoxious behavior of sites that are configured to notice, in a big way,
that you're downloading their content but not their ads. Some of them
replace the content with a big message that says, in effect, "Hey, we
see that you're using an ad blocker. Cut it out!" I'm not sure how this
solution addresses that, but I didn't read very far into it. If it was a
full proxy, it could download ads and drop them on the floor rather than
passing them on to you, but it's only a DNS proxy so it can't do that.

[Mayayana]

"Char Jackson" <***@none.invalid> wrote

| The one thing I didn't see mentioned is the increasingly common and
| obnoxious behavior of sites that are configured to notice, in a big way,
| that you're downloading their content but not their ads. Some of them
| replace the content with a big message that says, in effect, "Hey, we
| see that you're using an ad blocker. Cut it out!"

I've yet to see that. Maybe it requires javascript
to work. (Which I generally don't enable.) That seems
like poetic justice: They need javascript to spy on
you but they also need javascript to check whether
you're letting them spy. :)

But I do come across an increasing number of sites
that are broken without script. Some are not usable
at all but many work fine if I also disable CSS. They
do things like cover the page with a gray rectangle,
or pile things on top of each other, or make some of
the content non-visible. Then those deliberate
screw-ups are fixed by script when the page loads.
It's a rather odd strategy. Those sites never tell me
I need to enable script. They just try to make sure
the page is broken without script.

[VanguardLH]

Mayayana wrote:

: Char Jackson wrote
:
:: The one thing I didn't see mentioned is the increasingly common and
:: obnoxious behavior of sites that are configured to notice, in a big way,
:: that you're downloading their content but not their ads. Some of them
:: replace the content with a big message that says, in effect, "Hey, we
:: see that you're using an ad blocker. Cut it out!"
:
: I've yet to see that. Maybe it requires javascript to work. (Which I
: generally don't enable.) That seems like poetic justice: They need
: javascript to spy on you but they also need javascript to check
: whether you're letting them spy. :)

No, you have a connection to them so your IP address is known during
your session with the site. If your client refuses to get their ad
content using that IP address during your session with them, they can
detect your client is not retrieving all content and only some of it.
The site will cooperate with the off-domain site (theirs or someone
else's) to see if you went there to get that content. They can do that
whether or not Javascript is enabled/available in your client or not.

A problem with the server-side detection is the assumption that CDNs
(content delivery networks) seldom go down. If the CDN doesn't notify
the target site that you visited the CDN then the target site assumes
you blocked that content and pukes out its alarm. If the CDN is down,
the site might not yet know and puke out the same alarm despite you are
not using an adblocker. Also, Javascript is lightweight versus server-
side cooperative detection requires more effort to setup, maintain, and
to operate - but ad content is big business representing lots of money.

The backend detection requires more effort and cooperation than using
Javascript within the delivered web page. For example, the page may
encapsulate an ad within a <div> where the ad content then specifies the
size of the div element. If the Javascript sees the <div> has zero
height then the Javascript knows the ad content did not get retrieved.

https://www.bleepingcomputer.com/news/technology/9-percent-of-popular-websites-use-anti-adblock-scripts/

Of course, you could modify the page's Javascript hoping not to break
it. You could also disable Javascript but we all know the effect of
that: the page is empty or nearly worthless. Sites have moved to
dynamic page content which replies on scripting to decide what the page
will contain.

For sites that rely on only Javascript, there are adblocker-blocker
DNSBLs and extensions that modify the page to remove the checker script
(since often the sites are investing in a 3rd party to give them the
adblocker detection). I've tried a couple of the adblocker-blocker
DNSBLs but they don't seem very effective probably due to the server-
side detection and cooperation with the CDNs to see if you retrieved
that off-domain content.

Example of an adblocker-blocker (aka anti-adblocker) DNSBL:
uBlock Filters - Unbreak

Example of an adblocker-blocker extension:
https://github.com/reek/anti-adblock-killer
Greasemonkey or Tampermonkey (modify the scripts in the delivered page)

Neither of which will help with server-side cooperative detection. If
you don't retrieve it, they can know.

[Mayayana]

"VanguardLH" <***@nguard.LH> wrote

| : I've yet to see that. Maybe it requires javascript to work. (Which I
| : generally don't enable.) That seems like poetic justice: They need
| : javascript to spy on you but they also need javascript to check
| : whether you're letting them spy. :)
|
| No, you have a connection to them so your IP address is known during
| your session with the site. If your client refuses to get their ad
| content using that IP address during your session with them, they can
| detect your client is not retrieving all content and only some of it.
| The site will cooperate with the off-domain site (theirs or someone
| else's) to see if you went there to get that content. They can do that
| whether or not Javascript is enabled/available in your client or not.
|

In theory. But that's unlikely. Sending data requests
back and forth to 3rd parties with every page request
would put a significant load on their traffic and processing.
It makes much more sense to just write script that puts
all the load on the visitor. Something like a heavily obfuscated
version of...

"Go to xyz, send them this data, and load the ad from
xyz.com. If that fails then show a nasty message."

It's all built into the javascript in that case. And as
I said, I've never had a site show me with a message
about blocking ads. I know you hate to have your
beliefs contradicted by other peoples' experience.
You find your beliefs so devilishly delicious. Second
only to your opinions in their ability to transfix you
with pleasure. But thems the facts.

Think about what you're claiming. I visit a site. They
contact their 6 ad servers and spyware partners to
see if I'm loading the ads. But I can only load ads
after I've loaded the page. Without script, how are
they going to block the page they just gave me while
they wait 3 seconds for the other servers to respond?
And who's going to wait 3 seconds for a page to load
these days? (The webmaster rule of thumb is that if your
page takes more than 1/4 second, people will start
to leave.)

I guess they could track me to take revenge on
the next page. But I've also never seen a blank
page that says, "AHA! Caught ya! You script-blocking,
ad-evading son of a gun!"


|
https://www.bleepingcomputer.com/news/technology/9-percent-of-popular-websites-use-anti-adblock-scripts/
|

This contradicts your original farfetched claim
that websites will get around my disabling script
by contacting the ad server from the backend
and asking whether they see my IP address. This
article is talking about adblock blocking scripts.
(This is hilarious. There's adblock, anti-adblock,
and anti-anti-adblock.)

|
| Of course, you could modify the page's Javascript hoping not to break
| it. You could also disable Javascript but we all know the effect of
| that: the page is empty or nearly worthless.

Now you're off on another of your huffy
pronouncements and not paying attention to
my original statement: I don't get blocked for
blocking ads because I don't enable script.
It's all in the script.

You live in a world of escalating arms war, using
ublock origin. I just use a HOSTS file. You may
insist that's not possible, but I'm doing it. I just
visited blockadblock.com and downloaded their
adblock blocker script. (The compound obfuscation
is stunning.) They're using script to block ad
blockers but it's all dependent on CSS. Without
CSS their pages are perfectly functional. You may
not want to view pages with No Style, but I
increasingly find it's often better.

Some examples of my browsing:
WashPo, npr.org, TheRegister, Slashdot, Wired,
alternet.org, infoworld, duckduckgo, Google,
stackoverflow and most other online programming
info sites....
They all currently work fine without script.

Theatlantic.com is a mess. Headlines on top
of each other. But it's fine with no style.
Similarly, Ars Technica redesigned their site
and I only see 2-3 headlines unless I disable
CSS. But then it's fine.

Some sites I visit are blank. Then I disable CSS.
That makes for a plain page, but actually I
find that I'm increasingly disabling CSS even
when I don't need to. Example: I go to WashPo
and see a normal homepage, with article links.
I click a link and see a normal article. But there's
one problem. The font is serif, 18px high, with triple
line spacing! It's like reading a billboard from
6 feet away. Why? I don't know. Maybe they're
catering to phones? In any case, I often find it
easier to switch to no style and read the article
in simple, 12px verdana.

A few sites are actually completely broken. I used
to sometimes read business articles at forbes.com.
Their site is now broken. The webpage content itself
is embedded in script! But Forbes was never a great
news source, anyway. Nothing lost there. WSJ takes
another approach. They let you read a teaser, maybe
2 paragraphs. Then they want you to sign up. So I
don't go to WSJ.

What this boils down to is that they're refusing to
allow access to their website unless you allow them
to run a rather large software program on your computer.
It's an end-run version of a push webpage. I'm not
going to allow push webpages. Good riddance to them.
It's one thing to pay for a newspaper. It's another thing
entirely to be recorded while you read the paper and
to have the article dynamically change in order to
get me to look at ads. Why would anyone put up with
that once they realize it's happening? (Well, OK,
millions of Facebookie addicts put up with it. :)

| Sites have moved to
| dynamic page content which replies on scripting to decide what the page
| will contain.
|
Bingo. So they decide what news you'll read, what
price you'll pay when you shop, what search results
you'll see.... Apparently you don't care if their page
is bullshit as long as you don't have to see ads? I do
care, and I'm not accepting this push-webpage
sleight of hand. I suggest that anyone who doesn't
want to live on a push-based, spyware Web might
want to consciously consider what their response
should be.
If you allow script then you're handing your browser
(and security, and privacy) over to the sites you
visit, along with "every Tom, Dick and Harry"
business partner they have.

[VanguardLH]

|: I've yet to see that. Maybe it requires javascript to work. (Which I
|: generally don't enable.) That seems like poetic justice: They need
|: javascript to spy on you but they also need javascript to check
|: whether you're letting them spy. :)
|
| No, you have a connection to them so your IP address is known during
| your session with the site. If your client refuses to get their ad
| content using that IP address during your session with them, they can
| detect your client is not retrieving all content and only some of it.
| The site will cooperate with the off-domain site (theirs or someone
| else's) to see if you went there to get that content. They can do that
| whether or not Javascript is enabled/available in your client or not.
|
In theory. But that's unlikely. Sending data requests
back and forth to 3rd parties with every page request
would put a significant load on their traffic and processing.

Oh, as, um, versus using analytics. Uh huh. Bury your head.
Cloudflare and other CDNs provide feedback to their customers plus the
customers can institute their own handlers. Is the expense of paying
for more bandwidth with a CDN by a site worth the loss of revenue
generating by adblockers eliminating their advertising content? Why
would sites strive to prevent adblocking if it wasn't worth it to them?

You think Javascript is the only means of thwarting adblockers. Nope.
A page could be written to incorporate ad-looking tags into HTML tags,
like <body id="banner_ad">. Do the same in every <div> tag. The result
is the adblocker filters out the tag hence the document has no body or
divs are no longer delineated so the client has nothing to show. Every
URL to a resource, like the file in href for an <img> tag could include
a substring that looks ad-like, so all images would disappear when the
adblocker was active in the client. Those old methods just uses HTML.
No Javascript is involved at all.

It's all built into the javascript in that case. And as
I said, I've never had a site show me with a message
about blocking ads. I know you hate to have your
beliefs contradicted by other peoples' experience.

You've lived a very charmed life or are highly selective and narrow on
the number of sites you visit and never roam outside that list.

https://www.google.com/search?q=blocking%20adblocker
https://www.google.com/search?q=site+asks+me+to+disable+adblocker

Think about what you're claiming. I visit a site. They contact their 6
ad servers and spyware partners to see if I'm loading the ads. But I
can only load ads after I've loaded the page. Without script, how are
they going to block the page they just gave me while they wait 3
seconds for the other servers to respond?

Wow, you really don't know how external resources are delivered. That
you get a web page delivered to you does NOT mean you yet have the ad
content from the external resource. Obviously the externally resourced
content cannot be determined until AFTER the document has been delivered
to your client.

https://www.bleepingcomputer.com/news/technology/9-percent-of-popular-websites-use-anti-adblock-scripts/
This contradicts your original farfetched claim that websites will get
around my disabling script by contacting the ad server from the
backend and asking whether they see my IP address.

You must be employed as an editor of a history book that likes to
rewrite history to bend to how you want it to have happened. I never
server-side detection is the only method. However, you seem hell bent
on claiming Javascript is the only detection method. Geez, even when
someone agrees with you in part you just cannot stand it. You want to
deny server-side detection is employed. Okay, bury your head. Above I
showed that Javascript is not needed to thwart adblockers by just using
HTML. However, I *was* agreeing with you that Javascript present a
low-effort method of thwarting adblockers. Get it now, I agree with you
that Javascript is very much used. I disagree with you that Javascript
is the ONLY means available to thwart adblockers.

Now you're off on another of your huffy
pronouncements and not paying attention to
my original statement: I don't get blocked for
blocking ads because I don't enable script.
It's all in the script.
You live in a world of escalating arms war, using
ublock origin. I just use a HOSTS file. You may
insist that's not possible, but I'm doing it.

I never said it wasn't not possible. I've claimed that it is clumsy.
There is no quick means of disabling the hosts file during a web session
as there is with an adblocker extension. When content you want is
getting blocked at a site you choose to visit, just how to you get the
hosts file out of the way and ONLY to allow some content and not all of
it? You would have to rename the hosts file (so it cannot be found) and
likely have to restart your web browser (with it configured to purge its
local data) to load the page again but without interference from a hosts
file that can longer be found.

Meanwhile the rest of us can use an extension that disables Javascript
by default and we can enable it at will, plus the rest of us use
adblocker extensions that let use either globally enable/disable them or
selectively allow only some content at a site to make it usable.

That you claim to leave Javascript disabled means you only visit a few
sites that don't have dynamic content. Those are becoming rare.

WashPo, npr.org, TheRegister, Slashdot, Wired,
alternet.org, infoworld, duckduckgo, Google,
stackoverflow and most other online programming
info sites....
They all currently work fine without script.

Fine for you. The rest of use are not restricting our use of the Web to
just non-scripted sites or those where scripts do not limit or control
content.

A few sites are actually completely broken. I used
to sometimes read business articles at forbes.com.
Their site is now broken. The webpage content itself
is embedded in script!

They aren't unique. Not by far at all.

What this boils down to is that they're refusing to
allow access to their website unless you allow them
to run a rather large software program on your computer.
It's an end-run version of a push webpage. I'm not
going to allow push webpages. Good riddance to them.

Again your choice. Your opinions and choices do not reflect how others
want to use the Web as a resource.

It's one thing to pay for a newspaper. It's another thing
entirely to be recorded while you read the paper and
to have the article dynamically change in order to
get me to look at ads. Why would anyone put up with
that once they realize it's happening? (Well, OK,
millions of Facebookie addicts put up with it. :)

Wow, someone who compares many centuries old technology that contained
static content on clay tablets, papyrus, or paper to the Web. Hmm,
wonder why that old communications venue had static content. Oh, I know
why, their medium was static. Duh! Only in movies due to special
effects do you see writing on paper morph into something else. So,
let's see, we have multple editions of a book to make corrections and
add material that was unavailable or not ready at the time of printing.
So buy a whole new book to get the updated version. Doesn't seem to be
any "cheaper" than viewing web pages with dynamic content.

[J. P. Gilliver (John)]

In message <pdpbpd$c8u$***@dont-email.me>, Mayayana
<***@invalid.nospam> writes:
[]

Post by Mayayana
I click a link and see a normal article. But there's
one problem. The font is serif, 18px high, with triple
line spacing! It's like reading a billboard from
6 feet away. Why? I don't know. Maybe they're
catering to phones? In any case, I often find it
easier to switch to no style and read the article
in simple, 12px verdana.

[]
Did you really mean px, or pt?

In general - VanguardLH and Mayayana: can you please kiss and make up?
Both of you know far more about the mechanisms involved in ads,
adblocking, adblocker-blocking, and so on than I do, and I _suspect_ you
both know most of what the other does too. This thread looks like each
not wanting to give way, and I don't like to see people I consider
friends arguing, at least not heatedly rather than good-naturedly.

[Mayayana]

"J. P. Gilliver (John)" <G6JPG-***@255soft.uk> wrote

| >I click a link and see a normal article. But there's
| >one problem. The font is serif, 18px high, with triple
| >line spacing! It's like reading a billboard from
| >6 feet away. Why? I don't know. Maybe they're
| >catering to phones? In any case, I often find it
| >easier to switch to no style and read the article
| >in simple, 12px verdana.
| []
| Did you really mean px, or pt?
|

px. Here's a typical example:

https://www.washingtonpost.com/national/the-earth-cracked-open-walls-of-lava-encroached-thats-just-life-on-a-hawaiian-volcano/2018/05/19/b54128ca-5ac7-11e8-858f-12becb4d6067_story.html

I looked at the P tag for this text in Firefox
Inspector:
"Live on the black rock and amid the skinny palms."

I see this:

font-size: 18px;
line-height: 1.8em;
margin-bottom: 18px;

There's so much whitespace in the text that I find
it easier to read it without CSS.

[J. P. Gilliver (John)]

Post by Mayayana
| >I click a link and see a normal article. But there's
| >one problem. The font is serif, 18px high, with triple
| >line spacing! It's like reading a billboard from
| >6 feet away. Why? I don't know. Maybe they're
| >catering to phones? In any case, I often find it
| >easier to switch to no style and read the article
| >in simple, 12px verdana.
| []
| Did you really mean px, or pt?
|
https://www.washingtonpost.com/national/the-earth-cracked-open-walls-of-
lava-encroached-thats-just-life-on-a-hawaiian-volcano/2018/05/19/b54128c
a-5ac7-11e8-858f-12becb4d6067_story.html
I looked at the P tag for this text in Firefox
"Live on the black rock and amid the skinny palms."
font-size: 18px;
line-height: 1.8em;
margin-bottom: 18px;
There's so much whitespace in the text that I find
it easier to read it without CSS.

Oh. OK. (I'd assumed px was pixels and pt was point.)

Seems a poor design decision to measure font size in one unit (px) and
line height in another (em). [Whether that decision is made by the page
designer, or whoever defined the "language" s/he is using.]

[Mayayana]

"J. P. Gilliver (John)" <G6JPG-***@255soft.uk> wrote

| Oh. OK. (I'd assumed px was pixels and pt was point.)
|
Yes.

| Seems a poor design decision to measure font size in one unit (px) and
| line height in another (em).

It's not really different, but it is superfluous.
Em is the measure of the font. If the font is 18px
then em is 18px. In theory it's possible that em
and line height are different, but it doesn't seem
to work that way in practice. The px spec seems
to be the line height, rather than, say, the measure
of capital A.

So line-height: 1.8
and line-height: 1.8em
are actually the same thing. They both cause a
line, including white space, to measure 32+ px
vertically with 18px font.

I think of 18 as the biggest size for the biggest header.
But WashPo's headlin on that page is 44px! Weird.

I suspect the use of em may be due to automated,
WYSIWYG CSS code. Most people don't actually
write webpage code anymore. You can see the bloat
that results in almost all commercial webpages.
That page seems to have close to 100 class names
just to assign styles to text sections! No one
writing the code could ever keep track of such
a mess. There are at least 2 CSS files pulled in, plus
an unknown number linked from the voluminous,
obfuscated script.

[Mayayana]

"Mayayana" <***@invalid.nospam> wrote

| There are at least 2 CSS files pulled in, plus
| an unknown number linked from the voluminous,
| obfuscated script.
|

That page has some interesting glop that's apparently
connected to trying to block ad blockers. The word
"doubleclick" doesn't appear in the page. But there is
a long section of this kind of thing:

\x65\x45\x6c\x65\x6d\x65\x6e\x74','\x70\x75\x73\x68','\x63\x68\x69\x6c\x64\x4e\x6f\x64\x65\x73','\x70\x61\x72\x65\x6e\x74\x4e\x6f\x64\x65','\x69\x6e\x73\x65\x72\x74\x42\x65\x66\x6f\x72\x

That's ASCII codes. \x65 means hex 65, which is decimal
101, which is "e". Running the obfuscation through
a translator script turns up, among other things, this:

'https://securepubads.g.doubleclick.net',
'http://s0.2mdn.net/instream/video/',
'https://securepubads.g.doubleclick.net/gpt/pubads_impl_108.js'

It turns out 2mdn.net is yet another Google alias.
I just added it to my Acrylic HOSTS file. :)

So there's no place in the webpage to find a
doubleclick link, but it does come out once the
script is run. Not only an ad but also another
script from doubleclick itself. (I'd go check out
what's in that script, but it's too much trouble
to undo my doubleclick blocks.)

[J. P. Gilliver (John)]

In message <pdqkc5$d02$***@dont-email.me>, Mayayana
<***@invalid.nospam> writes:
[]

Post by Mayayana
That's ASCII codes. \x65 means hex 65, which is decimal
101, which is "e". Running the obfuscation through
'https://securepubads.g.doubleclick.net',
'http://s0.2mdn.net/instream/video/',
'https://securepubads.g.doubleclick.net/gpt/pubads_impl_108.js'
It turns out 2mdn.net is yet another Google alias.
I just added it to my Acrylic HOSTS file. :)

I've just added the securepubads bit to my hosts file (haven't got round
to implementing acrylic yet), so thanks for that, but as for 2mdn, I
found this in my hosts file:

#Youtube
...
#127.0.0.1 s0.2mdn.net #} comment out everything in this
segment
#127.0.0.1 static.2mdn.net #} from this point down if you need
to use
#127.0.0.1 www.youtube-nocookie.com #} YT, since these next few servers
are its
#127.0.0.1 youtube-nocookie.com #} dependencies and can cause major
#127.0.0.1 youtube-noscript.com #} functionality problems if blocked
#127.0.0.1 www.youtube-noscript.com #}

since I _do_ use YouTube, I've left that bit be. (I can't remember where
I got that information from - possibly you!)
[]

[Mayayana]

"J. P. Gilliver (John)" <G6JPG-***@255soft.uk> wrote

| since I _do_ use YouTube, I've left that bit be. (I can't remember where
| I got that information from - possibly you!)

It's news to me. I use DownloadHelper in Firefox.
If that can't get a video I don't see the video. I've
never streamed anything and don't plan to start.
So I don't need script, cookies, etc to see videos.

I've noticed that some videos are blocked with
DH, but in general I don't care about those. For
example, I can usually get homemade handyman
videos, or obscure, 20-year-old lectures, but I
probably can't get most pop music videos. I don't
generally listen to music by choice (much less
top-40 pop) so I don't care about those.

This recalls the discussion with VanguardLH. People
vary a lot in terms of what they do online. Someone
who does interactive things like shopping, watching
music videos, and using Facebook, for instance,
probably can't afford to improve their security and
privacy. Everything will break. They're already using
the Internet as interactive TV. So they've given up
most options aside from making entertainment choices.

There are trade-offs.

[pyotr filipivich]

Post by Mayayana
| since I _do_ use YouTube, I've left that bit be. (I can't remember where
| I got that information from - possibly you!)
It's news to me. I use DownloadHelper in Firefox.
If that can't get a video I don't see the video. I've
never streamed anything and don't plan to start.
So I don't need script, cookies, etc to see videos.

I keep having trouble with that - found vdyoutube.com which will
download Youtube Videos.

"works for me."

Post by Mayayana
This recalls the discussion with VanguardLH. People
vary a lot in terms of what they do online. Someone
who does interactive things like shopping, watching
music videos, and using Facebook, for instance,
probably can't afford to improve their security and
privacy. Everything will break. They're already using
the Internet as interactive TV. So they've given up
most options aside from making entertainment choices.
There are trade-offs.

Always.

[Mayayana]

"pyotr filipivich" <***@mindspring.com> wrote

| I keep having trouble with that - found vdyoutube.com which will
| download Youtube Videos.
|

They must be too young to know that "vd" stands
for venereal disease. :)

I gave it a quick try, but it fails without referrer.
I'm guessing it probably also requires script. I think
a lot of methods will work if script is enabled and/or
if you play the video first.

But I agree that DH is limited. And it often needs
to have the latest version. As noted, I don't try
to download commercial stuff, like music videos.

Youtube-dl worked for me recently on something,
but I don't remember what it was.

[pyotr filipivich]

Post by Mayayana
| I keep having trouble with that - found vdyoutube.com which will
| download Youtube Videos.
|
They must be too young to know that "vd" stands
for venereal disease. :)
I gave it a quick try, but it fails without referrer.
I'm guessing it probably also requires script. I think
a lot of methods will work if script is enabled and/or
if you play the video first.

What I find works is to look up the vid on Youtube. then insert
"vd" before youtube so that

becomes
http://youtu.be/E5rGFZWQfzk

As for scripts - I know zilch.

Post by Mayayana
But I agree that DH is limited. And it often needs
to have the latest version. As noted, I don't try
to download commercial stuff, like music videos.

it is always something. I usually avoid music videos. Back in
college, back in the stoner age when MTV showed music videos I
realized that I'd effectively "watched my radio" in the dorm lounge
for two hours, without having got anything else done.

[Mayayana]

"pyotr filipivich" <***@mindspring.com> wrote

| What I find works is to look up the vid on Youtube. then insert
| "vd" before youtube so that
| http://youtu.be/E5rGFZWQfzk
| becomes
| http://youtu.be/E5rGFZWQfzk
|
I just tried your link. It downloaded very slowly
and then didn't work. The same link at YT worked.
Looking at the two in HxD, the VD one is about
3 MB bigger and has a different header. I don't
know anything about mp4 haders, but it's different
right from the start:

VD: ....ftypdash....iso6avc1mp41...¯moov...lmvhd

YT: ....ftypmp42....isommp42..åsmoov...lmvhd

[Paul]

Post by Mayayana
| What I find works is to look up the vid on Youtube. then insert
| "vd" before youtube so that
| http://youtu.be/E5rGFZWQfzk
| becomes
| http://youtu.be/E5rGFZWQfzk
|
I just tried your link. It downloaded very slowly
and then didn't work. The same link at YT worked.
Looking at the two in HxD, the VD one is about
3 MB bigger and has a different header. I don't
know anything about mp4 haders, but it's different
VD: ....ftypdash....iso6avc1mp41...¯moov...lmvhd
YT: ....ftypmp42....isommp42..åsmoov...lmvhd

Dash gets me this article.

https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP

But that doesn't tell me "how to play it".

What if you step through the file a bit, and look
for some other 4CC codes ?

Paul

[Mayayana]

"Paul" <***@needed.invalid> wrote

| > VD: ....ftypdash....iso6avc1mp41...¯moov...lmvhd
| >
| > YT: ....ftypmp42....isommp42..åsmoov...lmvhd
|
| Dash gets me this article.
|
| https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
|
| But that doesn't tell me "how to play it".
|
| What if you step through the file a bit, and look
| for some other 4CC codes ?
|
I don't understand what you're saying. I'm not
familiar with mp4 format, don't know what "4CC"
means, and don't know what streaming has to
do with it. I understood the site offered downloads.
Are you saying they stream it and send the stream?
I guess that's OK if it gets otherwise unavailable
videos, but since I've never used streaming I have
no idea what to do with the file.

[Paul]

Post by Mayayana
| > VD: ....ftypdash....iso6avc1mp41...¯moov...lmvhd
| >
| > YT: ....ftypmp42....isommp42..åsmoov...lmvhd
|
| Dash gets me this article.
|
| https://en.wikipedia.org/wiki/Dynamic_Adaptive_Streaming_over_HTTP
|
| But that doesn't tell me "how to play it".
|
| What if you step through the file a bit, and look
| for some other 4CC codes ?
|
I don't understand what you're saying. I'm not
familiar with mp4 format, don't know what "4CC"
means, and don't know what streaming has to
do with it. I understood the site offered downloads.
Are you saying they stream it and send the stream?
I guess that's OK if it gets otherwise unavailable
videos, but since I've never used streaming I have
no idea what to do with the file.

That header, the only thing it suggests to me,
is they have recorded all the packets sent from
a stream and squashed them together.

The more intelligent containers for video and images
use four character sequences, which denote the beginning
of a section. A player selects which ones of these it
understands and tries to play the content. It skips
the ones it doesn't understand, using a length field.

The 4CC codes not recognized, may typically contain metadata
added by some other tool. The idea would be, to examine
the video in a hex editor. You know the file is 3MB larger
than the other file, and that 3MB extra could be split between
header and trailer. So you really don't need to examine more than
about 3MB to decide whether there's anything interesting in there.

I don't know what's in there, or know whether this
can be played or not in the conventional sense of
passing a file to a player application.

Paul

[pyotr filipivich]

Post by Mayayana
| What I find works is to look up the vid on Youtube. then insert
| "vd" before youtube so that
| http://youtu.be/E5rGFZWQfzk
| becomes
| http://youtu.be/E5rGFZWQfzk
|
I just tried your link. It downloaded very slowly
and then didn't work. The same link at YT worked.
Looking at the two in HxD, the VD one is about
3 MB bigger and has a different header. I don't
know anything about mp4 haders, but it's different
VD: ....ftypdash....iso6avc1mp41...¯moov...lmvhd
YT: ....ftypmp42....isommp42..åsmoov...lmvhd

Same here. I have hacked my way around it to get it to work "for
me". Which is to say , more often than not I have to first find my
notes to remember what the steps are.

[J. P. Gilliver (John)]

Post by pyotr filipivich

Post by Mayayana
| since I _do_ use YouTube, I've left that bit be. (I can't remember where
| I got that information from - possibly you!)
It's news to me. I use DownloadHelper in Firefox.

Me too - in Firefox 27.0.1. Maybe I'll try removing those #s and see if
things still work. I've noticed I sometimes can't save "commercial" type
things from YouTube - but I don't often want to: this is things like
music videos (official ones from the record companies I mean), films and
trailers for them, and such like, and I rarely want these.

Post by pyotr filipivich

Post by Mayayana
If that can't get a video I don't see the video. I've
never streamed anything and don't plan to start.
So I don't need script, cookies, etc to see videos.

I keep having trouble with that - found vdyoutube.com which will
download Youtube Videos.
"works for me."

I like DownloadHelper because it's not YouTube-specific (and does seem
to work on a lot of sites).

Post by pyotr filipivich

Post by Mayayana
This recalls the discussion with VanguardLH. People
vary a lot in terms of what they do online. Someone
who does interactive things like shopping, watching
music videos, and using Facebook, for instance,
probably can't afford to improve their security and
privacy. Everything will break. They're already using
the Internet as interactive TV. So they've given up
most options aside from making entertainment choices.
There are trade-offs.

Always.

Indeed. I buy (mostly from ebay), but I don't watch music videos or use
twitbook. Or bank. (I use telephone banking and find that's excellent.)

[J. P. Gilliver (John)]

Post by Mayayana
| Oh. OK. (I'd assumed px was pixels and pt was point.)
|
Yes.
| Seems a poor design decision to measure font size in one unit (px) and
| line height in another (em).
It's not really different, but it is superfluous.
Em is the measure of the font. If the font is 18px
then em is 18px. In theory it's possible that em
and line height are different, but it doesn't seem
to work that way in practice. The px spec seems
to be the line height, rather than, say, the measure
of capital A.

Yes, in points, it's the height of the printing block, or something like
that - IIRR, 72 point is an inch.

Post by Mayayana
So line-height: 1.8
and line-height: 1.8em

"If the font is 18px then em is 18px." as you said, then I'd assume
1.8em would be 1.8×18=32.4px?

Post by Mayayana
are actually the same thing. They both cause a
line, including white space, to measure 32+ px
vertically with 18px font.
I think of 18 as the biggest size for the biggest header.
But WashPo's headlin on that page is 44px! Weird.
I suspect the use of em may be due to automated,
WYSIWYG CSS code. Most people don't actually
write webpage code anymore. You can see the bloat
that results in almost all commercial webpages.

Autogenerated HTML is huge and sloppy. Multiple nested DIVs and TABLEs -
with little or, frequently, nothing inside one and not inside the other
(apart from sometimes a single NBSP) to justify them.

Post by Mayayana
That page seems to have close to 100 class names
just to assign styles to text sections! No one
writing the code could ever keep track of such
a mess. There are at least 2 CSS files pulled in, plus
an unknown number linked from the voluminous,
obfuscated script.

Yes, it's like the .PDF (or similar) some OCR software produces - might
generate something that looks the same, but - with each letter or maybe
group of two or three letters, or at least word, on the page having its
own XY co-ordinates, no chance of the text actually being usable. (To be
fair, OCR has improved a lot lately, perhaps due to the needs of blind
people.)

[Wolf K]

Post by Char Jackson
On Fri, 18 May 2018 12:59:08 -0700, pyotr filipivich

Post by pyotr filipivich
But Pi-Hole installed on a $35 Raspberry Pi is awfully tempting.
Endquote.
https://pi-hole.net/
Comments from the technoratti?

Others have provided some of the cons, with which I don't disagree. One
of the pros, however, is that it's positioned as a single solution that
works with every web-enabled device on your LAN, including smartphones
and smart TVs, and none of your devices has to know how to use it other
than pointing their DNS entry at it. No hosts files to play with, no
multiple configurations for multiple browsers, etc.
The one thing I didn't see mentioned is the increasingly common and
obnoxious behavior of sites that are configured to notice, in a big way,
that you're downloading their content but not their ads. Some of them
replace the content with a big message that says, in effect, "Hey, we
see that you're using an ad blocker. Cut it out!"

And I suppose the advertisers think that the ads are actually viewed?
Poor sods, they're wasting their money. The most effective ad-blocker is
the one in my head. Eg, I don't see ads when I view a news source on the
PC, but I do see them when I view that source on the phone. Guess what?
I haven't a clue what most of those ads are about. Racking my brains,
OK, I vaguely remember some colours. Pink. Green... I do recall that
Subaru advertised on one of those news sources, but I was primed to
notice it, as I was musing about trading my car for a newer one, and
Subaru topped my list. But I thought better of it. My car is good for at
least another 100K km. :-)

Post by Char Jackson
I'm not sure how this
solution addresses that, but I didn't read very far into it. If it was a
full proxy, it could download ads and drop them on the floor rather than
passing them on to you, but it's only a DNS proxy so it can't do that.

Agreed, that would be the best solution.

Fact is, most content isn't worth that much, so if the providers insist
I let them dump ads on me, I just go somewhere else.

Best,

[pyotr filipivich]

Post by Wolf K

Post by Char Jackson
The one thing I didn't see mentioned is the increasingly common and
obnoxious behavior of sites that are configured to notice, in a big way,
that you're downloading their content but not their ads. Some of them
replace the content with a big message that says, in effect, "Hey, we
see that you're using an ad blocker. Cut it out!"

And I suppose the advertisers think that the ads are actually viewed?
Poor sods, they're wasting their money.

They are wasting their money - on you. As the saying goes "Half
of your advertising budget is wasted; the problem is you can't tell
which half." Likely they get enough responses as to make it worth the
effort. How many is that? Enough that income remains > expenses (the
numerical answer is left as an exercise for the student.)

Post by Wolf K
The most effective ad-blocker is the one in my head.

Fifty years later, I still refuse to buy General Tires, due to
their advertising slogan when I was I was ten.
Dentine and Trident have a negative Q score for me, in large part
due to some obnoxious ads back in the 80's, unfortunately reinforced
due to recorded radio programs I hear now and then.

Post by Wolf K
Eg, I don't see ads when I view a news source on the
PC, but I do see them when I view that source on the phone. Guess what?
I haven't a clue what most of those ads are about. Racking my brains,
OK, I vaguely remember some colours. Pink. Green... I do recall that
Subaru advertised on one of those news sources, but I was primed to
notice it, as I was musing about trading my car for a newer one, and
Subaru topped my list. But I thought better of it. My car is good for at
least another 100K km. :-)

Neighbors had a gas guzzler, and couldn't afford the gas. So they
traded that one in for a new car, rolled what they owed on the gas
guzzler into the new car payments ... You know, for what you're still
paying for the old car, you could have bought a lot of gasoline.
They had other issues, too.

[Wolf K]

Post by pyotr filipivich

Post by Wolf K

Post by Char Jackson
The one thing I didn't see mentioned is the increasingly common and
obnoxious behavior of sites that are configured to notice, in a big way,
that you're downloading their content but not their ads. Some of them
replace the content with a big message that says, in effect, "Hey, we
see that you're using an ad blocker. Cut it out!"

And I suppose the advertisers think that the ads are actually viewed?
Poor sods, they're wasting their money.

They are wasting their money - on you. As the saying goes "Half
of your advertising budget is wasted; the problem is you can't tell
which half." Likely they get enough responses as to make it worth the
effort. How many is that? Enough that income remains > expenses (the
numerical answer is left as an exercise for the student.)

Post by Wolf K
The most effective ad-blocker is the one in my head.

Fifty years later, I still refuse to buy General Tires, due to
their advertising slogan when I was I was ten.
Dentine and Trident have a negative Q score for me, in large part
due to some obnoxious ads back in the 80's, unfortunately reinforced
due to recorded radio programs I hear now and then.

Post by Wolf K
Eg, I don't see ads when I view a news source on the
PC, but I do see them when I view that source on the phone. Guess what?
I haven't a clue what most of those ads are about. Racking my brains,
OK, I vaguely remember some colours. Pink. Green... I do recall that
Subaru advertised on one of those news sources, but I was primed to
notice it, as I was musing about trading my car for a newer one, and
Subaru topped my list. But I thought better of it. My car is good for at
least another 100K km. :-)

Neighbors had a gas guzzler, and couldn't afford the gas. So they
traded that one in for a new car, rolled what they owed on the gas
guzzler into the new car payments ... You know, for what you're still
paying for the old car, you could have bought a lot of gasoline.
They had other issues, too.

I've done the arithmetic. At a 25% saving in fuel, I'd save about
$1320/year in fuel. A newer car (2 or 3 years old) would cost me about
$15k net. At the rate I drive (about 12,000km/year) that's between 13
and 14 years to break even on fuel savings. The car would then be 15 to
17 years old. 25% fuel savings over what I have is unlikely. (My car
averages 8 to 9l/100km overall.)

Conclusion: Buying a newer vehicle works for fuel savings only for very
high annual mileage situations.

[VanguardLH]

As the saying goes "Half of your advertising budget is wasted; the
problem is you can't tell which half." Likely they get enough
responses as to make it worth the effort. How many is that? Enough
that income remains > expenses (the numerical answer is left as an
exercise for the student.)

The same reason spam (e-mail, Usenet) continues to survive. It only
take one or few boobs to pay the spammer to make it worth their efforts
to continue spamming some more. If spamming generated absolutely no
revenue, it would've died long ago. That advertising survives attests
that is *is* effective.

Advertising is big business. Businesses don't survive without
customers. So blame the customers on keeping advertising (and spamming)
alive. Yet, just how would we consumers know about any products without
being told about them? Just take the car out for a drive (and how did
you determine which car to buy) and wander around to see what happens to
be out there? I'm sure you don't work for free. You like to get paid.
So do all those people working at the sites providing "free" content to
you. So it's a bit of pleasure and a bit of pain: you want something
but they want something, too.

So we consumers started a war. We block the ads that we detest (well,
actually ALL of them, including those we might want to see at the time)
to focus on the other content in a web page. That product costs money
to deliver to you. As with any business, they aren't looking to merely
sustain their existence. Businesses want to grow. Mom and pop stores
would be incapable of providing the world-wide news and information that
you seek. To survive, they fight back. Gee, that's not really a
surprise or it shouldn't be. They need positive cash flow, not
negative. So they decide not to deliver some or all of their content to
you if you block other content. Hey, it's still THEIR site, their
property, their service. Yes, you could go elsewhere but then that
elsewhere still wants to survive and grow, too.

Tis too bad the sites didn't form a legal coalition to force advertisers
to comply with a known set of rules (adiquette) regarding good behavior.
Bad behavior is what instigated the use of adblockers in the first
place. They shot themselves in their own foot. Instead of fixing their
gun, they bandaged their foot and got a different gun.

[J. P. Gilliver (John)]

In message <kl7c5w73rejr$***@v.nguard.lh>, VanguardLH <***@nguard.LH>
writes:
[]

Post by VanguardLH
you seek. To survive, they fight back. Gee, that's not really a
surprise or it shouldn't be. They need positive cash flow, not
negative. So they decide not to deliver some or all of their content to
you if you block other content. Hey, it's still THEIR site, their
property, their service. Yes, you could go elsewhere but then that
elsewhere still wants to survive and grow, too.

Sometimes, I get "we see you are using an ad-blocker. We rely on
advertising to fund this site, so would you consider turning it off, at
least for us?", or similar wording. (Yes, whichever of you said they
never see these - I _do_. Often on smaller sites, such as a regional
newspaper, or specialist site.) When such a request does pop up, I often
_do_ turn off my blocker (I still use ABP; that can be set to turn off
just for a given site, or even just for a page); if they've asked
nicely, and I think they are a site I'd like to see continue, I will do
so. (Equally, I occasionally make donations to other sites, for example
Wikipedia and Gravestone Photographic Resource - especially if they
specifically say they don't take advertising, such as Wikipedia do.)

However, when - and it's usually the bigger sites - they just
malfunction, I tend to just go elsewhere. I probably _would_ do so on
principle, being just an ornery guy over such matters, but mainly, I go
elsewhere _because_ the page is malfunctioning.

Post by VanguardLH
Tis too bad the sites didn't form a legal coalition to force advertisers
to comply with a known set of rules (adiquette) regarding good behavior.
Bad behavior is what instigated the use of adblockers in the first
place. They shot themselves in their own foot. Instead of fixing their
gun, they bandaged their foot and got a different gun.

Nice analogy (-:. Though realistically I can't see many advertisers
complying with a code of practice (I like your "adiquette") - probably
only the companies who already were anyway, who are probably the ones
whose ad.s we didn't mind seeing anyway.

[Mayayana]

"J. P. Gilliver (John)" <G6JPG-***@255soft.uk> wrote

| Sometimes, I get "we see you are using an ad-blocker. We rely on
| advertising to fund this site, so would you consider turning it off, at
| least for us?", or similar wording. (Yes, whichever of you said they
| never see these - I _do_. Often on smaller sites, such as a regional
| newspaper, or specialist site.) When such a request does pop up, I often
| _do_ turn off my blocker

That's become a kind of noble ideal among
some techies. I see people at Slashdot at least
claim that they allow ads on some sites. But
the whole approach is not workable.

I don't actually block any ads. Not one. I only
block adware/spyware domains that I never chose
to visit. I block Google analytics. I block Doubleclick.
But if a website has an ad I'll see it. I've never
blocked images that are actually on the site I'm
visiting.

The problem is that they want to make more
money, and that means targetted ads, and that
means being tracked online. The website asking
you to allow ads is not asking you to allow their
ads. They're asking you to allow tracking and ads
injected by a dozen spyware/ad companies.
That means a system of dishonesty and
misrepresentation, as they sneak around trying to
hide what they're doing.

And as I described elsewhere, it's now leading to
a sneaky kind of push webpage, where the website
you visit expects to take over your browser, ID you,
then customize the actual content of the page. Not
just the ads but also articles, prices for retail goods
or airline tickets, search results, etc.

I don't accept the idea that they have a right to
rig pages with spyware simply because it's possible.
No more than the company selling me a newspaper
has a right to film me as I read the paper. The only
difference between the two is that one is easy and
can be done transparently, while the other would be
an unwieldy, obvious and possibly illegal undertaking.

In the interest of civility, the whole system will
eventually have to be reworked. In the interest of
relevant content as well. A site that operates that
kind of spyware-based content doesn't have content
worth seeing. Their business model becomes like
Facebook, doing whatever it takes to keep you
on their site. The whole thing becomes an aggressive,
push-based ad. The website, at that point, doesn't
really have any content at all. They only have
strategic advertising machinery.

In short, the Web is being turned into one big
sleazy racket. The solution can't be to adjust how
the racket works. A new model is required. A model
that doesn't reject honesty and dignity.