www.fabiankeil.de/sourcecode/privoxy/
patch-current.patch contains all my Privoxy patches, except the status code patch which wasn't a good idea in the first place and is obsolete anyway.
The patch is integrated in Privoxy 3.0.5 beta. Some parts were modified but the modifications aren't reflected on this page. Check the CVS history if you care about the details. Don't use this patch, get the latest official Privoxy version instead.
No longer adds crunch-header{string}. This action has been split into
crunch-client-header and
crunch-server-header.
If you used this action, clean your action files before updating the patch,
otherwise Privoxy will fail to start.
Additionally sets
Pragma: no-cache
for Privoxy's error messages. While it isn't specified as a response header, most browsers respect it.
Replaces crunch-if-modified-since with
hide-if-modified-since
which can do more than just blocking. If you used crunch-if-modified-since
in your action files, you will have to replace it with hide-if-modified-since{block},
otherwise Privoxy will fail to start.
Adds overwrite-last-modified.
Internal changes to sed() to only parse all headers in the first run
and only show the parsed headers once. The performance gains aren't measurable,
but running overwrite-last-modified with the random option twice
would weaken the randomness, as the random range gets smaller which each run.
crunch-client-header and
crunch-server-header
justify their activity by displaying the crunch pattern.
Fixed spelling in hide-if-modified-since's
log message (s/substracted/subtracted/).
Added
redirect and
handle-as-empty-document.
Added fast-redirects{check-decoded-url}.
Randomize statistics for overwrite-last-modified
and hide-if-modified-since are only
calculated if they get logged. No, you will not notice any performance impacts.
Privoxy answers forbidden requests for encrypted connections with a short description page (black and white only, not yet in Privoxy style™), instead of showing an empty page with a easy to overlook message inside the headers.
Privoxy answers forbidden requests for encrypted connections again without body, but the
added treat-forbidden-connects-like-blocks
can change the default behaviour.
Replacement images are now send with status code 403 instead of 200.
Answering with 200 to a request with Connect is only allowed
if the remote server was reached. Some clients act weird if the proxy sends 200
together with an unencrypted document.
Logging of the randomized If-Modified-Since header is working again.
It was broken since the optimization attempt two releases ago.
Minor code restructuring to let the patch apply against Privoxy 3.0.3 again.
(There was a failed chunk in filter.c.)
sed no longer adds a second Connection: close
header for HEAD responses. This bug was introduced with the
sed changes in release 2006-05-29.
Minor template fixes.
Added filter-headers and a memory leak.
Template spelling fix.
Action files are patched to use +fast-redirects{check-decoded-url}
by default.
Replaced Fixed memory leak in
filter-headers with another bug.
Failed DNS resolution attempts are now also
retried if the six argument version of gethostbyname_r is used
(recently added to FreeBSD RELENG_6 but also available on several other
operating systems), or if gethostbyname needs
pthread mutex locks (MacOS and OpenBSD).
Fixed memory leak fix
and added pcrs_execute
failure logging in filter-headers.
Changed default value for hide-if-modified-since's
randomize option from 1
to -1
.
crunch-server-header{string}Blocks or replaces headers set by the server if they contain the supplied string.
crunch-client-header{string}
Blocks or replaces headers set by the client if they contain the supplied string.
Can be combined with add-header to build a custom header filter.
hide-accept-language{}
Blocks or replaces the Accept-Language header. Useful if you fake a foreign
user agent.
content-type-overwrite{}
Replaces the Content-Type header.
Useful to let the browser render broken XHTML
as broken HTML. By default it only applies to
text documents, but if you know what you're doing you can enable
force-text-mode to modify binary content types as well.
Enables filtering of documents whose Content-Type wasn't recognized as text. Use with care.
hide-content-disposition{}
Removes or replaces annoying content-disposition headers.
Useful to view as much files inside the browser as possible, instead of downloading them first
and using an additional program just to view them.
Replacing the suggested file name is possible but less useful.
overwrite-last-modified{}
Removes or overwrites the server header Last-Modified.
Removing the headers is useful if you are testing a filter and don't want the browser to set the
If-Modified-Since header for the next request of the same document.
The randomize option overwrites the value of the Last-Modified header with a
time between the original value and the current time. This way the server can't detect the time of your
last visit, but the browser can still revalidate cached documents. In theory the server could send
each document with a different Last-Modified header to track your steps without using
cookies. I don't think anyone is using this tracking method yet, but it doesn't hurt to be prepared.
The reset-to-request-time option overwrites the value of the Last-Modified
header with the current time. You could use this option together with crunch-if-none-match
to customize your random range, but you should rather stick with the randomize option.
hide-if-modified-since
Removes or overwrites the If-Modified-Since header.
Removing is useful for filter testing, where you want to force a real reload instead of getting status
code 304 which would cause the browser to use a cached copy of the page. Use together with
crunch-if-none-match.
Instead of removing the header hide-if-modified-since can also add or substract
a random amount of time to/from the headers value. You specify a range of hours were the
random factor is chosen from and Privoxy does the rest. Negative value means subtracting,
a positive value adding.
If your If-Modified-Since header's value is higher than the original value of
the server's Last-Modified header was, you could miss updates, if it is lower
the browser can no longer revalidate cached documents and the server will always send
a new one.
Therefore it makes sense to only use a small negative value and let
overwrite-last-modified{randomize}
make the greater changes.
Use this action together with crunch-if-none-match.
crunch-if-none-match
Removes the If-None-Match header. Useful for filter testing, where you want to force
a real reload instead of getting status code 304 which would cause the browser to use
a cached copy of the page, but it also prevents user tracking.
Use this action together with hide-if-modified-since.
hide-referrer{conditional-block}
New option for hide-referrer. Only hides the referrer if the host has changed.
Less suspicious than the other options without giving away any information the website
owner couldn't get out of the log file anyway.
redirect{URL}
Redirect the request to a given URL.
Note that Privoxy was already able to redirect without this patch, but you had to combine
block, handle-as-image and set-image-blocker{URL}.
handle-as-empty-document
New blocking flavour. If blocks occur, Privoxy sends a replacement document containing just one space.
Useful to block documents which are neither HTML
nor images, without getting syntax complains from the browser.
The Content-Type for the empty document is taken from
content-type-overwrite{}.
fast-redirects{check-decoded-url}
New option for fast-redirects.
Privoxy's old fast-redirect action only checks if a the
URL contains http://
more than once and redirects to the last matching string. If you request
http://www.example.org/foo?http://www.example.net/bar
you get a redirect
to http://www.example.net/bar
, without contacting http://www.example.org/
first.
RFC 2396
rules that some characters have to be escaped, and some site follow the advice.
Yahoo's redirects for example look like
http://rds.yahoo.com/lots-of-tracking-garbage/**http%3a//www.privoxy.org/
and
Privoxy's old fast-redirect action misses the escaped
URL because it doesn't match http://
.
fast-redirects{check-decoded-url} decreases the false negatives by decoding
the URL before checking for http://
.
The old behaviour can still be used with +fast-redirects{simple-check}.
With the patch enabled, Privoxy no longer understands +fast-redirects.
If you have old action files, you can fix them by running:
sed -i '' -e 's/+fast-redirects/+fast-redirects{check-decoded-url}/g' *.action
or:
sed -i '' -e 's/+fast-redirects/+fast-redirects{simple-check}/g' *.action
in Privoxy's configuration directory.
treat-forbidden-connects-like-blocks
By default Privoxy answers forbidden attempts to open an encrypted connection
with status code 403 and a message inside the header. Unless the browser shows the
headers as well, you only get a white page.
treat-forbidden-connects-like-blocks lets Privoxy answer with the usual
blocking message, which brings you the See why
and Go there anyway
links
unless you change the default blocking behaviour.
filter-headersRegular expressions can now be used to filter headers as well. Check your filters before activating this action, as it can easily lead to broken requests.
At the moment the filters are applied to each header on its own, not to all at once. It makes it easier to diagnose problems, but on the downside you can't write filters that only change header x if header y's value is z.
The filters are used after the other header actions have finished and can use their output as input.
Whenever possible one should specify ^
, $
, the whole header name
and the colon, to make sure the filter doesn't cause havoc to other headers or the
page itself.
For example if you want to transform Galeon User-Agents to Firefox User-Agents you shouldn't use:
s@Galeon/\d\.\d\.\d @@
but:
s@^(User-Agent:.*) Galeon/\d\.\d\.\d (Firefox/\d\.\d\.\d\.\d)$@$1 $2@
With the patch, Privoxy ...
Content-Type
containing the string xml as text.
Privoxy checks the Content-Type to see if the document can be safely filtered.
Filtering binary data with regular expressions could destroy parts of it. By default
Privoxy only filters documents with a Content-Type beginning with text/
(but not text/plain which is often used for unknown document types) or with
Content-Type: application/x-javascript.
If you ever wondered why Privoxy wouldn't filter that stupid atom feed or this XHTML document, that's why.
The feed was probably transferred with Content-Type: atom/xml, and the
XHTML document with
Content-Type: application/xhtml+xml.
Tired of getting cached error messages, if the real problem is gone? Me too.
Privoxy sends its error messages with Cache-Control: no-cache
and sets the Expires and Last-Modified headers
to the current time. One might think this would be enough to prevent caching, but it isn't.
It only makes sure the browser revalidates its cached copy, but doesn't prevent the following problem:
If a temporary problem like a DNS timeout occurs
and the user tries again after getting Privoxy's error message, a compliant browser may set
the If-Modified-Since header with the content of the error page's
Last-Modified header.
If the temporary problem is gone, the request reaches the original server and
more often than not, the document on the server is older than Privoxy's error message.
The server sends status code 304 and the browser displays Privoxy's outdated
error message again and again.
There is no way to forbid caching with revalidation, but one can set Last-Modified
to a date in the past. For example to Tim Berners-Lee's birthday, which
predates the age of any document on the web and could be safely used to revalidate
without getting a status code 304.
There is no point to let a useless If-Modified-Since header reach the server,
it is therefore removed by Privoxy.
If you use Privoxy together with Tor (smart move by the way), you will notice a few DNS time outs per day. (If you see more than a few, you should update Tor, there were lots of improvements.)
With this patch, Privoxy will retry three times if the (socks) server could be reached
but the connection failed. It will log every retry, but only show the 404 No such domain
message if the last attempt still failed.
Note that with a socks4a connection Privoxy can't see a difference between
DNS
time outs and hard DNS errors like a non existing domain.
The socks server's response is the same and Privoxy will therefore retry for hard errors
too, even though it makes no sense at all.
Tor itself already makes several retries and if tripleing them wasn't enough, the problem probably wasn't time out related. In general Privoxy's first retry will be successful.
If you let Privoxy itself do the DNS resolution
(don't, the DNS traffic will be neither
encrypted nor anonymous) and if your operating system uses a gethostbyname(3)
or gethostbyname_r(3) implementation similar to FreeBSD's, Privoxy will retry ten
times if temporary DNS problems occur, but show its
error message for hard errors right away.
Privoxy no longer only logs crunch!
, but mentions the crunched header as well.
If it doesn't begin with http or https Privoxy will log a warning
instead of just blocking the referrer.
I'm currently writing a small script for Privoxy filter testing. It shows the difference between an original page and the page after it was filtered by Privoxy. To do that, it gets a local copy of the untouched document and then loads the copy through Privoxy and a local server.
I wanted the request for the original file to use Tor, and the easiest way to do this is to go through Privoxy. Of course I didn't want to have two versions of Privoxy running, one with filtering enabled and one without it.
The solution was to let the script set the custom header
while getting the untouched copy. The header will be honored by Privoxy for the ongoing
request only.
X-Filter: No
Interested how the most important features of the patch work? Read on.
hide-referrer{conditional-block} and hide-accept-language{}
This part of my Privoxy log file shows hide-accept-language{} and
hide-referrer{conditional-block} in action. (I edited the output to
delete requests which didn't trigger new behaviour):
May 21 20:00:48 Privoxy(08117200) Header: New HTTP Request-Line: GET /links.html HTTP/1.1 May 21 20:00:48 Privoxy(08117200) Header: scan: GET /links.html HTTP/1.1 May 21 20:00:48 Privoxy(08117200) Header: scan: Host: www.fabiankeil.de May 21 20:00:48 Privoxy(08117200) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 21 20:00:48 Privoxy(08117200) Header: scan: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 May 21 20:00:48 Privoxy(08117200) Header: scan: Accept-Language: en May 21 20:00:48 Privoxy(08117200) Header: scan: Accept-Encoding: gzip,deflate May 21 20:00:48 Privoxy(08117200) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 21 20:00:48 Privoxy(08117200) Header: scan: Keep-Alive: 600 May 21 20:00:48 Privoxy(08117200) Header: scan: Proxy-Connection: keep-alive May 21 20:00:48 Privoxy(08117200) Header: scan: Referer: http://www.fabiankeil.de/ May 21 20:00:48 Privoxy(08117200) Header: Referer: http://www.fabiankeil.de/ (not modified, still on www.fabiankeil.de) May 21 20:00:48 Privoxy(08117200) Header: Modified: User-Agent: Ich bin's wirklich May 21 20:00:48 Privoxy(08117200) Header: Suppressed offer to compress content May 21 20:00:48 Privoxy(08117200) Header: crumble crunched: Keep-Alive: 600! May 21 20:00:48 Privoxy(08117200) Header: crumble crunched: Proxy-Connection: keep-alive! May 21 20:00:48 Privoxy(08117200) Header: addh-unique: Host: www.fabiankeil.de May 21 20:00:48 Privoxy(08117200) Header: Adding: Connection: close May 21 20:00:48 Privoxy(08117200) Request: www.fabiankeil.de/links.html May 21 20:00:53 Privoxy(08117200) Header: scan: HTTP/1.1 200 OK May 21 20:00:53 Privoxy(08117200) Header: scan: Date: Sun, 21 May 2006 18:00:52 GMT May 21 20:00:53 Privoxy(08117200) Header: scan: Server: Apache/df-exts 1.2 (Unix) mod_ssl/2.8.22 OpenSSL/0.9.7d AuthPG/1.3 May 21 20:00:53 Privoxy(08117200) Header: scan: Last-Modified: Sun, 19 Mar 2006 15:43:01 GMT May 21 20:00:53 Privoxy(08117200) Header: scan: ETag: "25831d-15ae-441d7c05" May 21 20:00:53 Privoxy(08117200) Header: scan: Accept-Ranges: bytes May 21 20:00:53 Privoxy(08117200) Header: scan: Content-Length: 5550 May 21 20:00:53 Privoxy(08117200) Header: scan: Connection: close May 21 20:00:53 Privoxy(08117200) Header: scan: Content-Type: text/html May 21 20:00:53 Privoxy(08117200) Header: crumble crunched: Connection: close! May 21 20:00:53 Privoxy(08117200) Header: Adding: Connection: close May 21 20:00:54 Privoxy(08117200) Header: scan: HTTP/1.1 200 OK May 21 20:00:54 Privoxy(08117200) Header: scan: Date: Sun, 21 May 2006 18:00:52 GMT May 21 20:00:54 Privoxy(08117200) Header: scan: Server: Apache/df-exts 1.2 (Unix) mod_ssl/2.8.22 OpenSSL/0.9.7d AuthPG/1.3 May 21 20:00:54 Privoxy(08117200) Header: scan: Last-Modified: Sun, 19 Mar 2006 15:43:01 GMT May 21 20:00:54 Privoxy(08117200) Header: scan: ETag: "25831d-15ae-441d7c05" May 21 20:00:54 Privoxy(08117200) Header: scan: Accept-Ranges: bytes May 21 20:00:54 Privoxy(08117200) Header: scan: Content-Length: 5550 May 21 20:00:54 Privoxy(08117200) Header: scan: Content-Type: text/html May 21 20:00:54 Privoxy(08117200) Header: scan: Connection: close May 21 20:00:54 Privoxy(08117200) Header: crumble crunched: Connection: close! May 21 20:00:54 Privoxy(08117200) Header: Adjust Content-Length to 5550 May 21 20:00:54 Privoxy(08117200) Header: Adding: Connection: close
This is me visiting my Link page.
The Referer shows that I came through my Homepage. The website owner could get this information by
sorting his server log file after the IP addresses, modifying
the Referer would look suspicious.
May 21 20:00:54 Privoxy(0812d600) Header: New HTTP Request-Line: GET /bilder/icons/ausgang.ico HTTP/1.1 May 21 20:00:54 Privoxy(0812d600) Header: scan: GET /bilder/icons/ausgang.ico HTTP/1.1 May 21 20:00:54 Privoxy(0812d600) Header: scan: Host: www.fabiankeil.de May 21 20:00:54 Privoxy(0812d600) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 21 20:00:54 Privoxy(0812d600) Header: scan: Accept: image/png,*/*;q=0.5 May 21 20:00:54 Privoxy(0812d600) Header: scan: Accept-Language: en May 21 20:00:54 Privoxy(0812d600) Header: scan: Accept-Encoding: gzip,deflate May 21 20:00:54 Privoxy(0812d600) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 21 20:00:54 Privoxy(0812d600) Header: scan: Keep-Alive: 600 May 21 20:00:54 Privoxy(0812d600) Header: scan: Proxy-Connection: keep-alive May 21 20:00:54 Privoxy(0812d600) Header: Modified: User-Agent: Ich bin's wirklich May 21 20:00:54 Privoxy(0812d600) Header: Suppressed offer to compress content May 21 20:00:54 Privoxy(0812d600) Header: crumble crunched: Keep-Alive: 600! May 21 20:00:54 Privoxy(0812d600) Header: crumble crunched: Proxy-Connection: keep-alive! May 21 20:00:54 Privoxy(0812d600) Header: addh-unique: Host: www.fabiankeil.de May 21 20:00:54 Privoxy(0812d600) Header: Adding: Connection: close May 21 20:00:54 Privoxy(0812d600) Request: www.fabiankeil.de/bilder/icons/ausgang.ico May 21 20:00:59 Privoxy(0812d600) Header: scan: HTTP/1.1 200 OK May 21 20:00:59 Privoxy(0812d600) Header: scan: Date: Sun, 21 May 2006 18:00:58 GMT May 21 20:00:59 Privoxy(0812d600) Header: scan: Server: Apache/df-exts 1.2 (Unix) mod_ssl/2.8.22 OpenSSL/0.9.7d AuthPG/1.3 May 21 20:00:59 Privoxy(0812d600) Header: scan: Last-Modified: Wed, 12 Nov 2003 15:33:35 GMT May 21 20:00:59 Privoxy(0812d600) Header: scan: ETag: "9182c4-2fe-3fb252cf" May 21 20:00:59 Privoxy(0812d600) Header: scan: Accept-Ranges: bytes May 21 20:00:59 Privoxy(0812d600) Header: scan: Content-Length: 766 May 21 20:00:59 Privoxy(0812d600) Header: scan: Connection: close May 21 20:00:59 Privoxy(0812d600) Header: scan: Content-Type: image/x-icon May 21 20:00:59 Privoxy(0812d600) Header: crumble crunched: Connection: close! May 21 20:00:59 Privoxy(0812d600) Header: Adding: Connection: close
This is the request for the link page's favicon. Firefox doesn't set the Referer header,
hide-referrer{conditional-block} stays inactive.
May 21 20:01:09 Privoxy(08113a00) Header: New HTTP Request-Line: GET / HTTP/1.1 May 21 20:01:09 Privoxy(08113a00) Header: scan: GET / HTTP/1.1 May 21 20:01:09 Privoxy(08113a00) Header: scan: Host: www.eff.org May 21 20:01:09 Privoxy(08113a00) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 21 20:01:09 Privoxy(08113a00) Header: scan: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 May 21 20:01:09 Privoxy(08113a00) Header: scan: Accept-Language: en May 21 20:01:09 Privoxy(08113a00) Header: scan: Accept-Encoding: gzip,deflate May 21 20:01:09 Privoxy(08113a00) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 21 20:01:09 Privoxy(08113a00) Header: scan: Keep-Alive: 600 May 21 20:01:09 Privoxy(08113a00) Header: scan: Proxy-Connection: keep-alive May 21 20:01:09 Privoxy(08113a00) Header: scan: Referer: http://www.fabiankeil.de/links.html May 21 20:01:09 Privoxy(08113a00) Header: New host is: www.eff.org. Crunching Referer: http://www.fabiankeil.de/links.html! May 21 20:01:09 Privoxy(08113a00) Header: Modified: User-Agent: Mozilla/5.0 (X11; U; NetBSD alpha; en-AU; rv:1.8.0.3) Gecko/20060512 Firefox/1.5.0.3 May 21 20:01:09 Privoxy(08113a00) Header: Suppressed offer to compress content May 21 20:01:09 Privoxy(08113a00) Header: crumble crunched: Keep-Alive: 600! May 21 20:01:09 Privoxy(08113a00) Header: crumble crunched: Proxy-Connection: keep-alive! May 21 20:01:09 Privoxy(08113a00) Header: Accept-Language header crunched and replaced with: Accept-Language: en-au May 21 20:01:09 Privoxy(08113a00) Header: addh-unique: Host: www.eff.org May 21 20:01:09 Privoxy(08113a00) Header: Adding: Connection: close May 21 20:01:09 Privoxy(08113a00) Request: www.eff.org/ May 21 20:01:20 Privoxy(08113a00) Header: scan: HTTP/1.1 200 OK May 21 20:01:20 Privoxy(08113a00) Header: scan: Date: Sun, 21 May 2006 17:57:58 GMT May 21 20:01:20 Privoxy(08113a00) Header: scan: Server: Apache May 21 20:01:20 Privoxy(08113a00) Header: scan: X-Powered-By: PHP/4.4.2 May 21 20:01:20 Privoxy(08113a00) Header: scan: Vary: Accept-Encoding May 21 20:01:20 Privoxy(08113a00) Header: scan: Connection: close May 21 20:01:20 Privoxy(08113a00) Header: scan: Transfer-Encoding: chunked May 21 20:01:20 Privoxy(08113a00) Header: scan: Content-Type: text/html; charset=ISO-8859-1 May 21 20:01:20 Privoxy(08113a00) Header: crumble crunched: Connection: close! May 21 20:01:20 Privoxy(08113a00) Header: Adding: Connection: close May 21 20:01:21 Privoxy(08113a00) Header: scan: HTTP/1.1 200 OK May 21 20:01:21 Privoxy(08113a00) Header: scan: Date: Sun, 21 May 2006 17:57:58 GMT May 21 20:01:21 Privoxy(08113a00) Header: scan: Server: Apache May 21 20:01:21 Privoxy(08113a00) Header: scan: X-Powered-By: PHP/4.4.2 May 21 20:01:21 Privoxy(08113a00) Header: scan: Vary: Accept-Encoding May 21 20:01:21 Privoxy(08113a00) Header: scan: Transfer-Encoding: chunked May 21 20:01:21 Privoxy(08113a00) Header: scan: Content-Type: text/html; charset=ISO-8859-1 May 21 20:01:21 Privoxy(08113a00) Header: scan: Connection: close May 21 20:01:21 Privoxy(08113a00) Header: crumble crunched: Connection: close! May 21 20:01:21 Privoxy(08113a00) Header: Adding: Connection: close
Now I'm leaving my own web site, using the link to the EFF.
hide-referrer{conditional-block} detects the change of hosts and blocks
the Referer header. On the EFF's server it looks as if I used a bookmark or
typed in the web site's address myself.
You can also see that I'm faking a foreign User-Agent (it is generated automatically
by uagen.pl). The Accept-Language header is overwritten
to match the locale inside the User-Agent.
May 21 20:01:21 Privoxy(08117600) Header: New HTTP Request-Line: GET /stylesheets/eff-fluid.css HTTP/1.1 May 21 20:01:21 Privoxy(08117600) Header: scan: GET /stylesheets/eff-fluid.css HTTP/1.1 May 21 20:01:21 Privoxy(08117600) Header: scan: Host: www.eff.org May 21 20:01:21 Privoxy(08117600) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 21 20:01:21 Privoxy(08117600) Header: scan: Accept: text/css,*/*;q=0.1 May 21 20:01:21 Privoxy(08117600) Header: scan: Accept-Language: en May 21 20:01:21 Privoxy(08117600) Header: scan: Accept-Encoding: gzip,deflate May 21 20:01:21 Privoxy(08117600) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 21 20:01:21 Privoxy(08117600) Header: scan: Keep-Alive: 600 May 21 20:01:21 Privoxy(08117600) Header: scan: Proxy-Connection: keep-alive May 21 20:01:21 Privoxy(08117600) Header: scan: Referer: http://www.eff.org/ May 21 20:01:21 Privoxy(08117600) Header: Referer: http://www.eff.org/ (not modified, still on www.eff.org) May 21 20:01:21 Privoxy(08117600) Header: Modified: User-Agent: Mozilla/5.0 (X11; U; NetBSD alpha; en-AU; rv:1.8.0.3) Gecko/20060512 Firefox/1.5.0.3 May 21 20:01:21 Privoxy(08117600) Header: Suppressed offer to compress content May 21 20:01:21 Privoxy(08117600) Header: crumble crunched: Keep-Alive: 600! May 21 20:01:21 Privoxy(08117600) Header: crumble crunched: Proxy-Connection: keep-alive! May 21 20:01:21 Privoxy(08117600) Header: Accept-Language header crunched and replaced with: Accept-Language: en-au May 21 20:01:21 Privoxy(08117600) Header: addh-unique: Host: www.eff.org May 21 20:01:21 Privoxy(08117600) Header: Adding: Connection: close May 21 20:01:21 Privoxy(08117600) Request: www.eff.org/stylesheets/eff-fluid.css May 21 20:01:25 Privoxy(08117600) Header: scan: HTTP/1.1 200 OK May 21 20:01:25 Privoxy(08117600) Header: scan: Date: Sun, 21 May 2006 17:58:04 GMT May 21 20:01:25 Privoxy(08117600) Header: scan: Server: Apache May 21 20:01:25 Privoxy(08117600) Header: scan: Last-Modified: Thu, 11 May 2006 21:17:48 GMT May 21 20:01:25 Privoxy(08117600) Header: scan: ETag: "be158-3c92-c0817700" May 21 20:01:25 Privoxy(08117600) Header: scan: Accept-Ranges: bytes May 21 20:01:25 Privoxy(08117600) Header: scan: Content-Length: 15506 May 21 20:01:25 Privoxy(08117600) Header: scan: Connection: close May 21 20:01:25 Privoxy(08117600) Header: scan: Content-Type: text/css May 21 20:01:25 Privoxy(08117600) Header: crumble crunched: Connection: close! May 21 20:01:25 Privoxy(08117600) Header: Adding: Connection: close May 21 20:01:26 Privoxy(08117600) Header: scan: HTTP/1.1 200 OK May 21 20:01:26 Privoxy(08117600) Header: scan: Date: Sun, 21 May 2006 17:58:04 GMT May 21 20:01:26 Privoxy(08117600) Header: scan: Server: Apache May 21 20:01:26 Privoxy(08117600) Header: scan: Last-Modified: Thu, 11 May 2006 21:17:48 GMT May 21 20:01:26 Privoxy(08117600) Header: scan: ETag: "be158-3c92-c0817700" May 21 20:01:26 Privoxy(08117600) Header: scan: Accept-Ranges: bytes May 21 20:01:26 Privoxy(08117600) Header: scan: Content-Length: 15506 May 21 20:01:26 Privoxy(08117600) Header: scan: Content-Type: text/css May 21 20:01:26 Privoxy(08117600) Header: scan: Connection: close May 21 20:01:26 Privoxy(08117600) Header: crumble crunched: Connection: close! May 21 20:01:26 Privoxy(08117600) Header: Adjust Content-Length to 15506 May 21 20:01:26 Privoxy(08117600) Header: Adding: Connection: close
This request is still on the same host. The Referrer
isn't touched.
For this example I told Privoxy to additionally log information about the connection status, stopped Tor and requested The Blog of Ben Rockwood:
May 21 21:00:32 Privoxy(08086000) Connect: OK May 21 21:00:32 Privoxy(08086000) Connect: accept connection ... May 21 21:00:32 Privoxy(0812d600) Header: New HTTP Request-Line: GET /blog/ HTTP/1.1 May 21 21:00:32 Privoxy(0812d600) Header: scan: GET /blog/ HTTP/1.1 May 21 21:00:32 Privoxy(0812d600) Header: scan: Host: cuddletech.com May 21 21:00:32 Privoxy(0812d600) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 21 21:00:32 Privoxy(0812d600) Header: scan: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 May 21 21:00:32 Privoxy(0812d600) Header: scan: Accept-Language: en May 21 21:00:32 Privoxy(0812d600) Header: scan: Accept-Encoding: gzip,deflate May 21 21:00:32 Privoxy(0812d600) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 21 21:00:32 Privoxy(0812d600) Header: scan: Keep-Alive: 600 May 21 21:00:32 Privoxy(0812d600) Header: scan: Proxy-Connection: keep-alive May 21 21:00:32 Privoxy(0812d600) Header: scan: Referer: http://www.fabiankeil.de/links.html May 21 21:00:32 Privoxy(0812d600) Header: New host is: cuddletech.com. Crunching Referer: http://www.fabiankeil.de/links.html! May 21 21:00:32 Privoxy(0812d600) Header: Modified: User-Agent: Mozilla/5.0 (X11; U; NetBSD alpha; no-NO; rv:1.8.0.3) Gecko/20060518 Firefox/1.5.0.3 May 21 21:00:32 Privoxy(0812d600) Header: Suppressed offer to compress content May 21 21:00:32 Privoxy(0812d600) Header: crumble crunched: Keep-Alive: 600! May 21 21:00:32 Privoxy(0812d600) Header: crumble crunched: Proxy-Connection: keep-alive! May 21 21:00:32 Privoxy(0812d600) Header: Accept-Language header crunched and replaced with: Accept-Language: no-no May 21 21:00:32 Privoxy(0812d600) Header: addh-unique: Host: cuddletech.com May 21 21:00:32 Privoxy(0812d600) Header: Adding: Connection: close May 21 21:00:32 Privoxy(0812d600) Request: cuddletech.com/blog/ May 21 21:00:32 Privoxy(0812d600) Connect: to cuddletech.com May 21 21:00:32 Privoxy(0812d600) Connect: SOCKS4 negotiation write failed... May 21 21:00:32 Privoxy(0812d600) Connect: connect to: cuddletech.com failed: Broken pipe
Without Tor running, Privoxy obviously can't fulfill my wishes and sends an error message instead. This is the same request from Firefox' point of view:
http://cuddletech.com/blog/ GET /blog/ HTTP/1.1 Host: cuddletech.com User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 600 Proxy-Connection: keep-alive Referer: http://www.fabiankeil.de/links.html HTTP/1.x 503 Connect failed Content-Length: 4109 Content-Type: text/html Cache-Control: no-cache Date: Sun, 21 May 2006 19:00:32 GMT Last-Modified: Wed, 08 Jun 1955 12:00:00 GMT Expires: Sat, 17 Jun 2000 12:00:00 GMT
I restarted Tor and tried again:
May 21 21:00:32 Privoxy(08086000) Connect: OK May 21 21:00:32 Privoxy(08086000) Connect: accept connection ... May 21 21:00:32 Privoxy(0811ae00) Header: New HTTP Request-Line: GET /send-stylesheet HTTP/1.1 May 21 21:00:32 Privoxy(0811ae00) Request: config.privoxy.org/send-stylesheet cgi call May 21 21:00:32 Privoxy(0811ae00) Request: config.privoxy.org/send-stylesheet crunch! May 21 21:00:43 Privoxy(08086000) Connect: OK May 21 21:00:43 Privoxy(08086000) Connect: accept connection ... May 21 21:00:43 Privoxy(0812d200) Header: New HTTP Request-Line: GET /blog/ HTTP/1.1 May 21 21:00:43 Privoxy(0812d200) Header: scan: GET /blog/ HTTP/1.1 May 21 21:00:43 Privoxy(0812d200) Header: scan: Host: cuddletech.com May 21 21:00:43 Privoxy(0812d200) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 21 21:00:43 Privoxy(0812d200) Header: scan: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 May 21 21:00:43 Privoxy(0812d200) Header: scan: Accept-Language: en May 21 21:00:43 Privoxy(0812d200) Header: scan: Accept-Encoding: gzip,deflate May 21 21:00:43 Privoxy(0812d200) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 21 21:00:43 Privoxy(0812d200) Header: scan: Keep-Alive: 600 May 21 21:00:43 Privoxy(0812d200) Header: scan: Proxy-Connection: keep-alive May 21 21:00:43 Privoxy(0812d200) Header: scan: If-Modified-Since: Wed, 08 Jun 1955 12:00:00 GMT May 21 21:00:43 Privoxy(0812d200) Header: Modified: User-Agent: Mozilla/5.0 (X11; U; NetBSD alpha; no-NO; rv:1.8.0.3) Gecko/20060518 Firefox/1.5.0.3 May 21 21:00:43 Privoxy(0812d200) Header: Suppressed offer to compress content May 21 21:00:43 Privoxy(0812d200) Header: Crunching useless If-Modified-Since header. May 21 21:00:43 Privoxy(0812d200) Header: crumble crunched: Keep-Alive: 600! May 21 21:00:43 Privoxy(0812d200) Header: crumble crunched: Proxy-Connection: keep-alive! May 21 21:00:43 Privoxy(0812d200) Header: Accept-Language header crunched and replaced with: Accept-Language: no-no May 21 21:00:43 Privoxy(0812d200) Header: addh-unique: Host: cuddletech.com May 21 21:00:43 Privoxy(0812d200) Header: Adding: Connection: close May 21 21:00:43 Privoxy(0812d200) Request: cuddletech.com/blog/ May 21 21:00:43 Privoxy(0812d200) Connect: to cuddletech.com May 21 21:00:43 Privoxy(0812d200) Connect: OK May 21 21:00:44 Privoxy(0812d200) Header: scan: HTTP/1.1 200 OK May 21 21:00:44 Privoxy(0812d200) Header: scan: Date: Sun, 21 May 2006 19:00:44 GMT May 21 21:00:44 Privoxy(0812d200) Header: scan: Server: Apache May 21 21:00:44 Privoxy(0812d200) Header: scan: X-Powered-By: PHP/4.3.4 May 21 21:00:44 Privoxy(0812d200) Header: scan: Connection: close May 21 21:00:44 Privoxy(0812d200) Header: scan: Transfer-Encoding: chunked May 21 21:00:44 Privoxy(0812d200) Header: scan: Content-Type: text/html May 21 21:00:44 Privoxy(0812d200) Header: crumble crunched: Connection: close! May 21 21:00:44 Privoxy(0812d200) Header: Adding: Connection: close May 21 21:00:46 Privoxy(0812d200) Header: scan: HTTP/1.1 200 OK May 21 21:00:46 Privoxy(0812d200) Header: scan: Date: Sun, 21 May 2006 19:00:44 GMT May 21 21:00:46 Privoxy(0812d200) Header: scan: Server: Apache May 21 21:00:46 Privoxy(0812d200) Header: scan: X-Powered-By: PHP/4.3.4 May 21 21:00:46 Privoxy(0812d200) Header: scan: Transfer-Encoding: chunked May 21 21:00:46 Privoxy(0812d200) Header: scan: Content-Type: text/html May 21 21:00:46 Privoxy(0812d200) Header: scan: Connection: close May 21 21:00:46 Privoxy(0812d200) Header: crumble crunched: Connection: close! May 21 21:00:46 Privoxy(0812d200) Header: Adding: Connection: close
Firefox is trying to revalidate Privoxy's error message on the original server. Yes, that's stupid.
Without the patch the error message's Last-Modifed header would be
the actual time of date which means it's would be more recent than the original
page. The original server would answer 304 Not modified and Firefox
would show Privoxy's error message again.
With the patch Privoxy detects and prevents such nonsense. The useless
If-Modified-Since is stripped and the server delivers the whole
page.
It works the same for 404 error messages caused by Tor related
DNS timeouts, but they don't happen
on request and with this patch applied, they are more often than not resolved
by retrying (as shown in the next section).
fk@TP51 ~ $grep failed /var/log/privoxy/privoxy.log.5 | grep sslMay 16 13:55:56 Privoxy(0810ec00) Error: failed the 1. time to connect to images-ssl.sourceforge.net:443. Trying again.
As this grep output shows, Privoxy detected that Tor had trouble to connect to
images-ssl.sourceforge.net
.
Seeing only one line mentioning the problem means Privoxy's first retry attempt was successful and I never
saw an error message (and couldn't use the request to prove that the cache usage prevention actually works).
Last Modified and If-Modified-Since headers useless for user trackingThis is a clean request for the feed of the German blog Behindertenparkplatz:
May 30 11:46:18 Privoxy(0810e200) Header: New HTTP Request-Line: GET /blog/feed/ HTTP/1.1 May 30 11:46:18 Privoxy(0810e200) Header: scan: GET /blog/feed/ HTTP/1.1 May 30 11:46:18 Privoxy(0810e200) Header: scan: Host: www.ortegalink.com May 30 11:46:18 Privoxy(0810e200) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 30 11:46:18 Privoxy(0810e200) Header: scan: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 May 30 11:46:18 Privoxy(0810e200) Header: scan: Accept-Language: en May 30 11:46:18 Privoxy(0810e200) Header: scan: Accept-Encoding: gzip,deflate May 30 11:46:18 Privoxy(0810e200) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 30 11:46:18 Privoxy(0810e200) Header: scan: Keep-Alive: 600 May 30 11:46:18 Privoxy(0810e200) Header: scan: Proxy-Connection: keep-alive May 30 11:46:18 Privoxy(0810e200) Header: Modified: User-Agent: Mozilla/5.0 (X11; U; NetBSD alpha; de-DE; rv:1.8.0.3) Gecko/20060528 Firefox/1.5.0.3 May 30 11:46:18 Privoxy(0810e200) Header: Suppressed offer to compress content May 30 11:46:18 Privoxy(0810e200) Header: crumble crunched: Keep-Alive: 600! May 30 11:46:18 Privoxy(0810e200) Header: crumble crunched: Proxy-Connection: keep-alive! May 30 11:46:18 Privoxy(0810e200) Header: Accept-Language header crunched and replaced with: Accept-Language: de-de May 30 11:46:18 Privoxy(0810e200) Header: addh-unique: Host: www.ortegalink.com May 30 11:46:18 Privoxy(0810e200) Header: Adding: Connection: close May 30 11:46:18 Privoxy(0810e200) Request: www.ortegalink.com/blog/feed/ May 30 11:46:21 Privoxy(0810e200) Header: scan: HTTP/1.1 200 OK May 30 11:46:21 Privoxy(0810e200) Header: scan: Date: Tue, 30 May 2006 09:46:20 GMT May 30 11:46:21 Privoxy(0810e200) Header: scan: Server: Apache/1.3.33 (Unix) FrontPage/5.0.2.2634 May 30 11:46:21 Privoxy(0810e200) Header: scan: Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 May 30 11:46:21 Privoxy(0810e200) Header: scan: ETag: "50a978a3b306abd06c94aebb755c8c6f" May 30 11:46:21 Privoxy(0810e200) Header: scan: Expires: Thu, 19 Nov 1981 08:52:00 GMT May 30 11:46:21 Privoxy(0810e200) Header: scan: Pragma: no-cache May 30 11:46:21 Privoxy(0810e200) Header: scan: X-Pingback: http://www.ortegalink.com/blog/xmlrpc.php May 30 11:46:21 Privoxy(0810e200) Header: scan: X-Powered-By: PHP/4.3.11 May 30 11:46:21 Privoxy(0810e200) Header: scan: Set-Cookie: PHPSESSID=6f70d0be15c11c3b52637ef1d678ccc2; path=/ May 30 11:46:21 Privoxy(0810e200) Header: scan: Last-Modified: Mon, 29 May 2006 15:36:54 GMT May 30 11:46:21 Privoxy(0810e200) Header: scan: Connection: close May 30 11:46:21 Privoxy(0810e200) Header: scan: Transfer-Encoding: chunked May 30 11:46:21 Privoxy(0810e200) Header: scan: Content-Type: text/xml; charset=utf-8 May 30 11:46:21 Privoxy(0810e200) Header: Crunched incoming cookie -- yum! May 30 11:46:21 Privoxy(0810e200) Header: crumble crunched: Set-Cookie: PHPSESSID=6f70d0be15c11c3b52637ef1d678ccc2; path=/! May 30 11:46:21 Privoxy(0810e200) Header: crumble crunched: Connection: close! May 30 11:46:21 Privoxy(0810e200) Header: Randomizing: Last-Modified: Mon, 29 May 2006 15:36:54 GMT May 30 11:46:21 Privoxy(0810e200) Header: Randomized: Last-Modified: Tue, 30 May 2006 08:39:10 GMT (added 0 days 17 hours 2 minutes 16 seconds) May 30 11:46:21 Privoxy(0810e200) Header: Crunching server header: Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 (contains: no-cache) May 30 11:46:21 Privoxy(0810e200) Header: Crunching server header: Pragma: no-cache (contains: no-cache) May 30 11:46:21 Privoxy(0810e200) Header: Adding: Connection: close May 30 11:46:27 Privoxy(0810e200) Header: Set: Transfer-Encoding: identity
Headers containing no-cache
are crunched by +crunch-server-header{no-cache} to allow
Firefox to cache the document. The value of the Last-Modified header is randomized by
+overwrite-last-modified{randomize}.
This a tainted request for the same document. Firefox is trying to revalidate its
cached version by setting the If-Modified-Since and If-None-Match
headers. The value for the If-Modified-Since header is taken from the
Last-Modified header which was previously randomized by Privoxy, the value
for the If-None-Match header is taken from the Etag header:
May 30 11:46:34 Privoxy(0810e000) Header: New HTTP Request-Line: GET /blog/feed/ HTTP/1.1 May 30 11:46:34 Privoxy(0810e000) Header: scan: GET /blog/feed/ HTTP/1.1 May 30 11:46:34 Privoxy(0810e000) Header: scan: Host: www.ortegalink.com May 30 11:46:34 Privoxy(0810e000) Header: scan: User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8.0.3) Gecko/20060506 Firefox/1.5.0.3 May 30 11:46:34 Privoxy(0810e000) Header: scan: Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 May 30 11:46:34 Privoxy(0810e000) Header: scan: Accept-Language: en May 30 11:46:34 Privoxy(0810e000) Header: scan: Accept-Encoding: gzip,deflate May 30 11:46:34 Privoxy(0810e000) Header: scan: Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 May 30 11:46:34 Privoxy(0810e000) Header: scan: Keep-Alive: 600 May 30 11:46:34 Privoxy(0810e000) Header: scan: Proxy-Connection: keep-alive May 30 11:46:34 Privoxy(0810e000) Header: scan: If-Modified-Since: Tue, 30 May 2006 08:39:10 GMT May 30 11:46:34 Privoxy(0810e000) Header: scan: If-None-Match: "50a978a3b306abd06c94aebb755c8c6f" May 30 11:46:34 Privoxy(0810e000) Header: scan: Cache-Control: max-age=0 May 30 11:46:34 Privoxy(0810e000) Header: Modified: User-Agent: Mozilla/5.0 (X11; U; NetBSD alpha; de-DE; rv:1.8.0.3) Gecko/20060528 Firefox/1.5.0.3 May 30 11:46:34 Privoxy(0810e000) Header: Suppressed offer to compress content May 30 11:46:34 Privoxy(0810e000) Header: Randomizing: If-Modified-Since: Tue, 30 May 2006 08:39:10 GMT (random range: -1 hour) May 30 11:46:34 Privoxy(0810e000) Header: Randomized: If-Modified-Since: Tue, 30 May 2006 08:37:15 GMT (subtracted 0 hours 1 minute 55 seconds) May 30 11:46:34 Privoxy(0810e000) Header: crumble crunched: Keep-Alive: 600! May 30 11:46:34 Privoxy(0810e000) Header: crumble crunched: Proxy-Connection: keep-alive! May 30 11:46:34 Privoxy(0810e000) Header: Accept-Language header crunched and replaced with: Accept-Language: de-de May 30 11:46:34 Privoxy(0810e000) Header: Crunching If-None-Match: "50a978a3b306abd06c94aebb755c8c6f" May 30 11:46:34 Privoxy(0810e000) Header: addh-unique: Host: www.ortegalink.com May 30 11:46:34 Privoxy(0810e000) Header: Adding: Connection: close May 30 11:46:34 Privoxy(0810e000) Request: www.ortegalink.com/blog/feed/ May 30 11:46:36 Privoxy(0810e000) Header: scan: HTTP/1.1 304 Not Modified May 30 11:46:36 Privoxy(0810e000) Header: scan: Date: Tue, 30 May 2006 09:46:35 GMT May 30 11:46:36 Privoxy(0810e000) Header: scan: Server: Apache/1.3.33 (Unix) FrontPage/5.0.2.2634 May 30 11:46:36 Privoxy(0810e000) Header: scan: Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 May 30 11:46:36 Privoxy(0810e000) Header: scan: ETag: "50a978a3b306abd06c94aebb755c8c6f" May 30 11:46:36 Privoxy(0810e000) Header: scan: Expires: Thu, 19 Nov 1981 08:52:00 GMT May 30 11:46:36 Privoxy(0810e000) Header: scan: Pragma: no-cache May 30 11:46:36 Privoxy(0810e000) Header: scan: X-Pingback: http://www.ortegalink.com/blog/xmlrpc.php May 30 11:46:36 Privoxy(0810e000) Header: scan: X-Powered-By: PHP/4.3.11 May 30 11:46:36 Privoxy(0810e000) Header: scan: Set-Cookie: PHPSESSID=fbfae2d083ff0d717c6d1a77a667d205; path=/ May 30 11:46:36 Privoxy(0810e000) Header: scan: Last-Modified: Mon, 29 May 2006 15:36:54 GMT May 30 11:46:36 Privoxy(0810e000) Header: scan: Connection: close May 30 11:46:36 Privoxy(0810e000) Header: scan: Content-Type: text/html May 30 11:46:36 Privoxy(0810e000) Header: Crunched incoming cookie -- yum! May 30 11:46:36 Privoxy(0810e000) Header: crumble crunched: Set-Cookie: PHPSESSID=fbfae2d083ff0d717c6d1a77a667d205; path=/! May 30 11:46:36 Privoxy(0810e000) Header: crumble crunched: Connection: close! May 30 11:46:36 Privoxy(0810e000) Header: Randomizing: Last-Modified: Mon, 29 May 2006 15:36:54 GMT May 30 11:46:36 Privoxy(0810e000) Header: Randomized: Last-Modified: Mon, 29 May 2006 19:39:42 GMT (added 0 days 4 hours 2 minutes 48 seconds) May 30 11:46:36 Privoxy(0810e000) Header: Crunching server header: Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 (contains: no-cache) May 30 11:46:36 Privoxy(0810e000) Header: Crunching server header: Pragma: no-cache (contains: no-cache) May 30 11:46:36 Privoxy(0810e000) Header: Adding: Connection: close
Privoxy blocks the If-None-Match header because it could have been misused as
a cookie. The If-Modified-Since header gives the server enough information to
revalidate the document anyway.
Privoxy further randomizes the value of the If-Modified-Since header,
but uses a smaller range to pick the random value from. Most of the time this isn't
necessary, but it prevents the server from outsmarting +overwrite-last-modified{randomize}
by setting the Last-Modified header to the time of the request (If the time of the request
and the Last-Modified header's value are the same, the random range is zero and the
Last-Modified header stays the same).
Note that both server responses have the same Last-Modified and Etag
values. In other words: they were not used for user tracking purposes.
The patch applies against Privoxy 3.0.3 with offset warnings you can safely ignore. There shouldn't be any failed chunks.
Since release 2006-05-29, the patch requires (but currently doesn't check) the availability of the
C functions strptime and strtol. Both are new dependencies for Privoxy,
but on most operating systems they are already present as part of the standard C library.
If you are a FreeBSD user, you can still use the benefits of the ports collection:
cd /usr/ports/www/privoxy/files/ fetch http://www.fabiankeil.de/sourcecode/privoxy/patch-current.patch cd .. make install clean
OpenBSD users need a few extra steps to prevent conflicts with the port's own patches:
cd /usr/ports/www/privoxy/ ftp http://www.fabiankeil.de/sourcecode/privoxy/patch-current.patch sed -e 's@#ifdef OSX_DARWIN@#if defined(OSX_DARWIN) || defined(__OpenBSD__)@;' \ -e 's@#elif defined(OSX_DARWIN)@#elif defined(OSX_DARWIN) || defined(__OpenBSD__)@;' \ -e 's@strcpy(buf, CFORBIDDEN);@strlcpy(buf, CFORBIDDEN, sizeof(buf));@' \ patch-current.patch > patches/patch-zcurrent.patch rm patch-current.patch make install clean
If you use the NetBSD Packages Collection the steps are:
cd /usr/pkgsrc/www/privoxy/ make patch ftp -o - http://www.fabiankeil.de/sourcecode/privoxy/patch-current.patch | patch -d work/privoxy-3.0.3-stable/ make install clean
If your operating system uses a different package system, try to find out how to incorporate foreign patches. If you can't, follow the Privoxy handbook section: 2.2. Building from Source but run:
fetch -o - http://www.fabiankeil.de/sourcecode/privoxy/patch-current.patch | patch
before make.
If it doesn't work, please make sure you are able to compile the unmodified Privoxy sources without problems, before you send any complains to me. I only tested the patch on FreeBSD, NetBSD and OpenBSD, but unless stated otherwise its features should be operating system independent.
If your Russian is better than your English and if you're using an operating system which can deal with RPM files, you have another choice: the Russian Privoxy Friends included an older version of this patch in their Privoxy release 3.0.3-8b2.
While this patch certainly contains nice features, it probably contains not-so-nice bugs as well.
One minor problem is that the web interface's edit menu automatically creates links
to the documentation of every action dis- or enabled. It will also create
links for actions introduced by this patch, even though there is no documentation
available on the official Privoxy website. This could be prevented with lots of
exceptions, but I'm hoping the patch will one day be included in the official Privoxy
version and don't think writing exceptions now is worth the time.
Privoxy's documentation got updated and this is no longer an issue.
If you encountered any problems with this patch, please let me know. I'm also interested
in success stories on operating systems not mentioned on this page.
I no longer care, unless you can reproduce the problem with an
official Privoxy version.