D90 Tools & Techniques » Linux

Custom 404 for Apache (using PHP!)

Matt — Thu, 04 Nov 2010 23:03:44 +0000

Images, css, js just get a simple 404 Not Found page.

Everything not in the list of items we’re checking get redirected to the homepage. So a simple typo will get the redirect, but a missing PNG file that’s called by one of our pages won’t send a copy of the homepage to the client telling it it is an image file!

/* This is a custom 404 handler.


It does makes a decision -- if it's a small, auxiliary file like an image, code sheet, etc

that is being requested and not found then we'll send a standard Apache 404 page.
However, anything that looks like a website request (i.e. everything NOT in the list)

will be given a 301 redirect to our homepage.
Invoke by:

ErrorDocument 404 /404.php
4 November 2010

Matt Kivela

*/
/* Currently set to filter:

   aspx

   asp

   cgi

   css

   gif

   ico

   jpg

   ogg

   png

*/
if (preg_match('/\.[Aa][Ss][Hh][Xx]|[Aa][Ss][Pp]|[Cc][Gg][Ii]|[Cc][Ss][Ss]|[Gg][Ii][Ff]|[Ii][Cc][Oo]|[Jj][Pp][Gg]|[Jj][Ss]|[Oo][Gg][Gg]|[Pp][Nn][Gg]$/', $_SERVER[REQUEST_URI], $foo))

  {

     echo Header("HTTP/1.1 404 Not Found");

     echo Header("Status: 404 Not Found");

     echo "404 Error:
File: http://$_SERVER[SERVER_NAME]$_SERVER[REQUEST_URI] not found.
";

     echo "If this is causing a problem, you may contact admin@yourdomain.org or
";

     echo "submit a ticket at http://bugzilla.yourdomain.org/";

   }

else

  {

    $new_url = "http://$_SERVER[SERVER_NAME]/";

    echo Header( "HTTP/1.1 301 Moved Permanently" );

    echo Header( "Location: $new_url" );

  }

?>

~

Rootkit Hunters

Matt — Wed, 10 Feb 2010 18:53:23 +0000

A few resources for finding rootkits:

Linux:
Rootkit Hunter
Chkrootkit

Windows:
RootkitRevealer

Cool one-line report…

Matt — Tue, 14 Jul 2009 01:46:23 +0000

sudo grep “GET https://www.miroguide.com/ HTTP/1.1″ /var/log/squid/access.log | grep “\”Miro/” | sed ‘s/^.*”-” “//g’ | sed ‘s/ .*$//g’ | grep Miro | sort | uniq -c

The sort | uniq -c being the good part. It parsed 625,000 entries in the log to make this useful (to the person who asked for it) report:

      2 Miro/0.9.8
      7 Miro/0.9.8.1
      8 Miro/0.9.9
     86 Miro/0.9.9.1
     42 Miro/0.9.9.1a
     12 Miro/0.9.9.9
     18 Miro/0.9.9.9.1
    383 Miro/1.0
     10 Miro/1.0-svn
    362 Miro/1.1
      1 Miro/1.1.1
    255 Miro/1.1.2
     56 Miro/1.2
     92 Miro/1.2.1
    119 Miro/1.2.2
    600 Miro/1.2.3
    415 Miro/1.2.4
      4 Miro/1.2.5
    217 Miro/1.2.6
    825 Miro/1.2.7
   1784 Miro/1.2.8
     66 Miro/1.2-svn
    237 Miro/2.0
   1119 Miro/2.0.1
    292 Miro/2.0.2
   3772 Miro/2.0.3
   7171 Miro/2.0.4
32008 Miro/2.0.5
     33 Miro/2.0-svn
    119 Miro/2.5-rc1
    125 Miro/2.5-svn

The somewhat frightening part is that line took all of 90 seconds for me to whip out…

Squid handling http –> https redirects

Matt — Sat, 30 May 2009 01:35:48 +0000

In configuring Squid to handle both our port 80 and 443 traffic, we have the issue that we can use redirects at the webserver level to redirect certain pages to https:// .

So this is handled in Squid.

First, make a simple script. There’s a possibility another redirector like Squirm might do a better job, but I haven’t played with them.

!/usr/bin/perl
$|=1;
while (<>) {
s@http://www7.getmiro.(com|net|org)/adopt(.*)$@301:https://www7.getmiro.com/adopt$2@;
print;
}

Saved at /etc/squid3/squid_redirector.pl and chown/chmod so the user “proxy” that squid runs under can run it. Your path, of course, may vary.

The key part for what we need is that we pre-pend “301″ before https: in the rewrite. When this is returned to the user’s browser it redirects them to the secure page. This script also takes anything at com, net, or org and forces them to a tld of .com as well.

It’s easy to test this perl script. Simply type ./squid_redirector.pl which launches it interactively.

# ./squid_redirector.pl
http://www7.getmiro.com/foo

http://www7.getmiro.com/foo

http://www7.getmiro.com/adopt/test
301:https://www7.getmiro.com/adopt/test
http://www7.getmiro.net/adopt/matt/is/an/evil/genius
301:https://www7.getmiro.com/adopt/matt/is/an/evil/genius

Next, tell Squid to use it. We need to enable these lines in the squid.conf file:

url_rewrite_program /etc/squid3/squid_redirector.pl
url_rewrite_children 10
url_rewrite_host_header off
url_rewrite_bypass on

The first line tells Squid what to use to rewrite URLs, the second tells it to spawn 10 instances on startup. I’m not sure, in the end, if host_header needs to be off. url_rewrite_bypass on allows Squid to skip the re-writing step if all the redirectors are busy. That’s a decision knowing our security risks, users, and needs — and I’m going with more reliability over absolute security. We’ll should see skips showing up in the logs and adjust settings from there if necessary.

Restart Squid, give it a test. Famous last words — it should work now.

References:

http://wiki.squid-cache.org/Features/Redirectors

http://brainextender.blogspot.com/2009/01/simple-squid-redirector-perl-script.html

Lighttpd, virtual hosts, alternative ports

Matt — Fri, 29 May 2009 20:25:50 +0000

In the configuration of our new server, all ports 80 and 443 traffic is handled by Squid as a reverse proxy. 8080 is the “backdoor” that bypasses Squid and hits Lighttpd directly.

But the standard format of a Lighttpd virtual host entry doesn’t recognize alternate ports appended after the tld. Not a big deal, this does the trick:

$HTTP["host"] =~ “(^|\.)getmiro\.(com|net|org)($|:8080$)” {

Translated:
(^|\.) Any hostname
getmiro\ Going to the gemtiro domain
.(com|net|org) with a top level domain of com, net, or org
($|:8080$){ and ending with the tld or :8080 will be processed by the rules that follow.

Lighttpd, virtual hosts, and wildcard domains

Matt — Fri, 29 May 2009 20:01:53 +0000

So we’re setting up mirocommunity.com, and I don’t want to be hassled continously to create new hostnames in DNS.

To avoid that, it’s a simple wildcard entry like this in the appropriate named database:

*.mirocommunity.com. IN CNAME mirocommunity.com.

Which directs everything to our server.

Now our server hosts multiple sites via host entries, so we can’t use a simple negation like this:

$HTTP["host"] !~ “^(www|medfield)\.mirocommunity\.(com|net|org)($|:8080$)” {
url.redirect = (
“^(.*)$” => “http://www.mirocommunity.com$1″,
)
}

Note the negation by using !~ instead of =~. That would work if all we had was mirocommunity sites to host, but when hitting another site on the server like www7.getmiro.com it would read it as not being www or medfield dot mirocommunity, and thus drop you to www.mirocommunity.com. For the curious, the 8080 part of the url parsing is a bypass of the Squid proxies on ports 80 and 443.

Anything that doesn’t match a virtual host or alias on our server gets dropped by default to /var/www.

There lies the simple solution — put an index.php file there that does the redirect work:

// Hostnames that aren’t matched in Lighttpd get dropped here
// by default.
// This script removes the hostname(s) and drops them to
// www.[domain].[tld]
// 29 May 2009 MRK
$split_host = split(“\.”, $_SERVER[HTTP_HOST]);
$domain = count($split_host) – 2;
$tld = count($split_host) – 1;
$new_host = “http://www.$split_host[$domain].$split_host[$tld]“;
// echo “$new_host”;
header(“Location: $new_host”);
exit;
?>

The split command splits the $_SERVER[HTTP_HOST] variable at each period, and put it’s contents less the periods into an array called $split_host.

The count($split_host) determines how many members we have in the $split_host array. We know we always want the last (the top level domain — .com, etc) and second to last (the domain — mirocommunity, etc). Since arrays start at 0, we simply count -1 for the tld and -2 for the domain.

By adding the count logic, we can handle domains like brooklyn.newyork.mirocommunity.com which have more then one hostname before the domain and tld.

$new_host then forms the URL we want to catch wildcard hostnames that haven’t been configured yet. It’s simply the www.domain.tld form. That’s fed to a http header which causes the user’s browser to redirect to the default website we want.

So as of today, while newyork.mirocommunity.com and brooklyn.newyork.mirocommunity.com have no virtual hosts, you do arrive successfully at www.mirocommunity.com.

Our developers can activate those hostnames simply by adding an entry in the appropriate lighttpd conf file and reload lighttpd — no need to contact the sysadmin to go make an entry in our DNS system for each new city added.

Installing Squid to handle both 80 and 443

Matt — Tue, 26 May 2009 20:54:41 +0000

This outlines configuring Squid, running two instances, to handle both port 80 and 443 traffic on an Amazon EC2 instance running Ubuntu Jaunty. We can bypass Squid by going directly to Lighttpd on port 8080.

To answer a couple questions off the top, you should also read my post on how to configure http –> https redirects at the Squid level since the web server won’t be able to handle that in this configuration, and this post documents a little bit of magic that needs to be done to support 8080 with virtual hosts.

In configuring our new servers, the choice of Squid was pretty easy — it can handle SSL traffic, Varnish can’t by itself. We already use Squid to do ssl traffic on some of our physical servers being replaced, so I’d like to continue using that feature. In a future post, we’ll talk about configuring Squid to use a Universal Certificate that can handle multiple domains on one IP (it looks doable in theory, but I haven’t purchased that yet).

Normally installation is a simple

apt-get install squid

to install Squid. However, Ubuntu doesn’t package OpenSSL with Squid and for license reasons has no intention of doing so. So you’re better off following these directions and modifying a package to include ssl support, then installing that.

Modify /etc/squid/squid.conf

This is the port 80 traffic. Note — we actually had a large number of “acl valid_dst dstdomain,” which block attempts to use Squid as a pass-thru proxy at the proxy level instead of having the webserver reject the traffic.

http_port 80 accel vhost

cache_peer 127.0.0.1 parent 8080 0 no-query originserver login=PASS

logformat combined %>a %ui %un [%{%d/%b/%Y:%H:%M:%S +0500}tl] “%rm %ru HTTP/%rv” %Hs %h” “%{User-Agent}>h” %Ss:%Shaccess_log /var/log/squid/access.log combined

acl SSL_ports port 8080

acl Safe_ports port 8080

acl valid_dst dstdomain .somedomain.com

http_access allow valid_dst

Copy squid.conf to squid_ssl.conf, comment out http_port and make the following changes:

https_port 443 accel vhost cert=/(cert location) key=/(key location)
cache_log /var/log/squid3/cache_ssl.log
cache_store_log /var/log/squid3/store_ssl.log

We have seperate cache and store logs for troubleshooting, but both configurations use access.log to record traffic. This simplifies using AWStats to analyze the logs; if we run into performance problems in the future we may need a tool like logmerge.pl to consolidate seperate access logs. While I can think of a few things that could go wrong, I don’t know they will go wrong till we try, so let’s see if the simple way works first.

Now, let’s configure and initialize a seperate spool for ssl traffic:

mkdir /var/spool/squid3_ssl
chown -R proxy:proxy /var/spool/squid3_ssl/
squid3 -z -f /etc/squid3/squid_ssl.conf

Copy /etc/init.d/squid3 to /etc/init.d/squid3_ssl and make the following changes:

NAME=squid3_ssl
SQUID_ARGS=”-D -YC -f /etc/squid3/squid_ssl.conf”
CONFIG=/etc/squid3/squid_ssl.conf
$DAEMON -z -f $CONFIG

And do a ln -s /etc/init.d/squid3_ssl /etc/rc2.d/S30squid3_ssl to make it start automatically.

Optimizing Website integration with Amazon’s S3 Service

Matt — Sat, 28 Feb 2009 06:15:26 +0000

At Participatory Culture Foundation we use Amazon’s S3 Service to host our static content — css, js, and images.

This accomplishes two things — it improves the performance for our visitors since Amazon has faster performance and reliability then we can afford on our own servers, and it does so at a lower cost.

In this post we’ll look at how much bandwidth and files/redirect we use without S3, then with various combination of local and redirected files, up to code as optimized as I have been able to make it — fully optimized we our servers only transfer 6.7% of the bytes that the “unoptimized” site would. Optimizing this single popular page to use S3 efficiently saves PCF about $1,000 a year in hosting costs.

1)
Let’s look at a redacted Squid log after the February, 2009 redesign of www.getmiro.com when not using S3 at all:

"GET http://www.getmiro.com/ HTTP/1.1" 200 21781 "-" "Mozilla/5.0
 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6"
"GET http://www.getmiro.com//css/nav.css HTTP/1.1" 200 5004 "http://www.getmiro.com/" "Mozilla/5.0
"GET http://www.getmiro.com//css/styles.css HTTP/1.1" 200 21515 "http://www.getmiro.com/" "Mozilla/5.0
"GET http://www.getmiro.com/css/index.css HTTP/1.1" 200 7709 "http://www.getmiro.com/" "Mozilla/5.0
"GET http://www.getmiro.com//i/blue_bg.png HTTP/1.1" 200 1198 "http://www.getmiro.com//css/styles.css" "Mozilla/5.0
"GET http://www.getmiro.com//i/nav_back.gif HTTP/1.1" 200 530 "http://www.getmiro.com//css/nav.css" "Mozilla/5.0
[blah blah blah...]

That’s 37 files, for a total of 338,359 Bytes.

2)
Now let’s look if we load CSS from our server, but use Apache to re-write images and js to the S3 service:

"GET http://www.getmiro.com/ HTTP/1.1" 200 21781 "http://www.getmiro.com/" "Mozilla/5.0
 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.6) Gecko/2009011913 Firefox/3.0.6"
"GET http://www.getmiro.com//css/nav.css HTTP/1.1" 200 5004 "http://www.getmiro.com/" "Mozilla/5.0
"GET http://www.getmiro.com//css/styles.css HTTP/1.1" 200 21515 "http://www.getmiro.com/" "Mozilla/5.0
"GET http://www.getmiro.com/css/index.css HTTP/1.1" 200 7709 "http://www.getmiro.com/" "Mozilla/5.0
"GET http://www.getmiro.com//i/blue_bg.png HTTP/1.1" 302 688 "http://www.getmiro.com//css/styles.css" "Mozilla/5.0
"GET http://www.getmiro.com//i/nav_back.gif HTTP/1.1" 302 690 "http://www.getmiro.com//css/nav.css" "Mozilla/5.0
[blah blah blah...]

Now it’s four files, plus 33 redirects — and only 78,705 bytes.

3)
Now let’s use Apache to redirect the CSS to be pulled from Amazon S3.

"GET http://www.getmiro.com/ HTTP/1.1" 200 21781 "-" "Mozilla/4.0
 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
"GET http://www.getmiro.com//css/nav.css HTTP/1.1" 302 684 "http://www.getmiro.com/" "Mozilla/4.0
"GET http://www.getmiro.com//css/styles.css HTTP/1.1" 302 690 "http://www.getmiro.com/" "Mozilla/4.0
"GET http://www.getmiro.com/css/index.css HTTP/1.1" 302 688 "http://www.getmiro.com/" "Mozilla/4.0
"GET http://www.getmiro.com//i/blue_bg.png HTTP/1.1" 302 687 "http://www.getmiro.com/" "Mozilla/4.0
"GET http://www.getmiro.com//i/nav_back.gif HTTP/1.1" 302 689 "http://www.getmiro.com/" "Mozilla/4.0
[blah blah blah...]

Still four files and 33 redirects, but down to 46,506 bytes.

So far it’s all been pretty standard stuff in Apache using mod_rewrite redirects. Apache sees a CSS sheet being called, it redirects it to S3.

RewriteRule ^/css/(.*) http://s3.getmiro.3.0.com.s3.amazonaws.com/css/$1

And then a css sheet may have a line like this:

background: url(../i/screen_dropshadow.png) -20px -36px no-repeat;

Now an observant user may note in the logs above that I switched from using Firefox to IE. Why? The browsers interpret the CSS differently.

Firefox interprets “../i/” relative to where the CSS style sheet is LOADED from — in our case http://s3.getmiro.3.0.com.s3.amazonaws.com/css/.
Internet Explorer interprets “../i/” relative to where the CSS style sheet is CALLED from — in our case http://www.getmiro.com/css/.

Those familiar with unix notation know that “../i/” from “getmiro.com/css/” gets you to “getmiro.com/i/”.

4)
Now we get fancy.

In implementing S3, we have a bash script which handles the synchronization between our servers and S3. So in that script, let’s intercept the CSS sheets, do a simple SED, and upload the modified files to a special location:

# getmiro css
# This substitutes ../i with http://s3.getmiro.3.0.com.s3.amazonaws.com/ in the getmiro css code
# and uploads them to a special directory in amazon.  This is in turn re-written by Apache to point there.
# Having the full url hard coded saves tens of thousands of redirects and gigs of bandwidth.
# It's also more efficient then "php-ifying" css to do the url substitution.

  # First, copy they css to a working directory:
    cp /data/getmiro/css/*.css /scripts/getmiro_css

  # It's safer to just modify files we know about, rather then automate finding and modifying without foreknowledge:
    sed -i 's/\.\.\/i/http:\/\/s3.getmiro.3.0.com.s3.amazonaws.com\/i/g' /scripts/getmiro_css/download-features.css
    sed -i 's/\.\.\/i/http:\/\/s3.getmiro.3.0.com.s3.amazonaws.com\/i/g' /scripts/getmiro_css/index.css
    sed -i 's/\.\.\/i/http:\/\/s3.getmiro.3.0.com.s3.amazonaws.com\/i/g' /scripts/getmiro_css/nav.css
    sed -i 's/\.\.\/i/http:\/\/s3.getmiro.3.0.com.s3.amazonaws.com\/i/g' /scripts/getmiro_css/styles.css

  # And let's upload them:
    /usr/local/s3sync/s3sync.rb -r -p -v /scripts/getmiro_css/ s3.getmiro.3.0.com:css/s3_coded/

In the background so the web developers don’t have to worry about modifying the CSS sheets to include the hard link, transforming lines like:

background: url(../i/screen_dropshadow.png) -20px -36px no-repeat;

into

background: url(s3.getmiro.3.0.com.s3.amazonaws.com/i/screen_dropshadow.png) -20px -36px no-repeat;

It’s necessary to use the pattern \.\./i/ in sed in case a developer does hard code the amazon link. The \. means literally a period; regexes like this otherwise use a . as a wildcard to match one character, and just ../i/ would match any two characters before /i/.

In Apache, we change the redirect to this:

RewriteRule ^/css/(.*) http://s3.getmiro.3.0.com.s3.amazonaws.com/css/S3_coded/$1

With this change implemented:

"GET http://www.getmiro.com/ HTTP/1.1" 200 21781 "-" "Mozilla/4.0
 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)"
"GET http://www.getmiro.com//css/styles.css HTTP/1.1" 302 708 "http://www.getmiro.com/" "Mozilla/4.0
"GET http://www.getmiro.com//css/nav.css HTTP/1.1" 302 702 "http://www.getmiro.com/" "Mozilla/4.0
"GET http://www.getmiro.com/css/index.css HTTP/1.1" 302 706 "http://www.getmiro.com/" "Mozilla/4.0

Much better! One file, three redirects for 23,897 bytes. That one change represents nearly a 50% reduction in bandwidth usage on our physical servers from just the example above, and only about 1/3rd the bandwidth if we used CSS sheets being served locally even if they had the hard links to S3 on them.

5)
Finally one more tweak.

The default Apache redirect includes HTML code saying where a file has moved. But this isn’t necessary — a web browser just needs the correct headers to tell it where to go.

So replacing the redirect to S3 we used before, we use this:

RewriteRule ^/css/(.*) /custom_messages/css_rewrite.php

This is css_rewrite.php:

Invoke by: RewriteRule ^/css/(.*) /custom_messages/css_rewrite.php */

$new_server = "http://s3.getmiro.3.0.com.s3.amazonaws.com/css/s3_coded/"; $new_url = preg_replace('/^.*\//', $new_server, $_SERVER[REQUEST_URI]);

echo Header( "HTTP/1.1 301 Moved Permanently" ); echo Header( "Location: $new_url" ); ?>

This produces a very minimal redirect — under 400 bytes rather then over 700 bytes.

I haven’t done a complete analysis to know if this significantly slower then a native Apache redirect; initial review shows it is not slower for any given page load. This step would need a very, very busy site however to make a meaningful performance or cost impact. It’s something I’m noting though, because there could be other situations this type of optimization could be useful. Most importantly the variables Apache (or IIS) can pass to other programs like PHP. See this link for a list of them.

"GET http://www.getmiro.com/ HTTP/1.1" 200 21781 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)" "GET http://www.getmiro.com//css/styles.css HTTP/1.1" 301 397 "http://www.getmiro.com/" "Mozilla/4.0 "GET http://www.getmiro.com//css/nav.css HTTP/1.1" 301 394 "http://www.getmiro.com/" "Mozilla/4.0 "GET http://www.getmiro.com/css/index.css HTTP/1.1" 301 396 "http://www.getmiro.com/" "Mozilla/4.0

Now just one file, three redirects and 22,968.

Bottom line?

Let’s take a typical day when the getmiro.com homepage is called 20,000 times.

Scenario Size Total Daily Estimated Bandwidth Daily Cost** 1 338,359 6.5GB $4.73 2 78,705 1.5GB 1.09 3 46,506 887MB* .65 4 23,897 455MB .33 5 22,968 438MB .32 * Unadjusted for Firefox's interpretation of CSS paths ** This estimate is based on purchasing enough fixed bandwidth (Mbps) to cover our peak daily usage. Our communication costs for our physical servers is approximately times as much as Amazon S3 based on actual transfers.

So without S3 or any optimization, we’d be looking at a monthly cost around $142.00.

With S3 and with all our optimization, we’re looking at a monthly cost around $54.00.

For a small non-profit, that’s a nice savings over time.

Passwords and Data Mining

Matt — Sat, 28 Feb 2009 03:43:04 +0000

I believe the working assumption must be we’re under a persistent, long term attack by organized groups.

This is not just organized crime, but I suspect organized criminal groups that are tolerated by states like Russia and China if not outright state sponsored. It’s not just those old “boogey men” either — there are many nations from Poland to South Korea to Israel that have long histories of industrial espionage, and we must assume at the very least Western European and the descendents of the British Empire (U.S.A. included) at the very least collect and preserve usernames and passwords found passing the internet unencrypted.

When you read in the news about security breach after security breach, you have to assume organizations out there are selling, buying, collecting, and storing in data warehouses usernames and passwords. If people sell credit card information and known good email addresses, this data has value too.

Passwords don’t even have to be in cleartext to have value in such a market. As computing power comes down the ability to attack them through methods such as rainbow tables become more and more viable — by my calculations, you could build a highly efficient rainbow table of all possible Windows passwords on Amazon’s Elastic Computing Cloud for about $60,000/month. While that takes funding and organization, we’re not exactly talking about needing the resources of SPECTRE to pull it off. How many other databases out there store passwords also without salts, like Windows, making them very susceptible to rainbow tables? Or how many databases, if breached, are likely to have the salt also revealed?

Most people use variations on the same password. So once you start to data mine 15+ years of data collected in security breaches you start to have a big collection associating usernames, email addresses, and passwords. This now gives you a fairly small number of passwords to try in focused attacks.

For an example, let’s consider this data set:

Jdoe password Jdoe passw0rd Jdoe password1 Jdoe Jdoe@aol.com passw0rd4 JohnDoe Jdoe@aol.com

Either using data mining or simply people as “mechanical turks” we can make a good guess is we see the username and email “JohnDoe Jdoe@aol.com” that his password will be something like password, passw0rd, password3. So given just the username and email, we can try setup a low intensity attack — have a botnet try each one password a day so it doesn’t raise alarm bells for bad login attempts. The “holy grail” for hacker would be to get into Jdoe@aol.com itself and now can get account resets sent to themselves…and given that set of data it’s a darn good chance he’s using some sort of “password” variant for it.

We know single factor authentication is fundamentally insecure. There is no doubt today it’s wholely inadequate for logging into systems you expect to trust. So what do we do until there is universal two factor authentication schemes in place?

First, divide your accounts into ones you care about and ones you don’t.

For sites like newspaper comments or your favorite web forum use a simple password because you don’t care who knows it. This is analagous to a simple padlock — while it doesn’t stop a determined attacker, it stops the casual attacks and that is good enough.

For finances, work computers, and the like…use a highly secure password.

How do you generate them in a way that is highly resistant to data mining efforts?

My criteria is you can re-create them at will from something you know (in your mind), and can even leave them sitting in plainview and still not be compromised.

How?

Let’s first think up a pattern. That can be a standard word or phrase you use in conjunction with how you identify each resource you need to access. Then run that compound phrase (base phrase + resource) through a hash generator

So in Linux, we can issue a series of commands like this:

$ echo mybasicpassword@www.yahoo.com | sha256sum 9a7c3ff19da0207cae4c4c7f820d38397f672a47500795c4f56d6b45fe578603 $ echo mybasicpassword@www.d90.us | sha256sum f4d0ccb1eb6b8e40472132cd44efc5b6b9bc976a4f951205e9e1bb96a12a1fda $ echo mybasicpassword@bankofamerica | sha256sum 857a0d7ed6b510f7b7ab615072446552291429ba3c7ca40fe91553520b2f56a3 $ unset HISTFILE

The unset HISTFILE removes the history of the commands you just typed so they’re not stored after you log off, revealing to a hacker your secret “mybasicpassword” as well as the secret way you identify the resources. What you pipe into the hash generator can change — maybe you have to reset a password quarterly and make it a habit of adding the month and year when you generated it. The only place that pattern should be is in your head, plus maybe a note that helps you remember when you last generated it.

Now simply write them down…

yahoo 9a7c 3ff1 9da0 207c ae4c 4c7f 820d 3839 7f67 2a47 5007 95c4 f56d 6b45 fe57 8603 d90 f4d0 ccb1 eb6b 8e40 4721 32cd 44ef c5b6 b9bc 976a 4f95 1205 e9e1 bb96 a12a 1fda bankofamerica 857a 0d7e d6b5 10f7 b7ab 6150 7244 6552 2914 29ba 3c7c a40f e915 5352 0b2f 56a3

Now also pick a pattern of what part of the hash to use. Maybe it’s the 64th, 62nd, 60th, 1st, 3rd, and 7th characters in that order, so for “bankofamerica” you’d use 36f877.

The nice part being is anywhere you are you can re-create the password at will, yet it’s secure from other people unless they’re intercepting unencrypted signals or torture it out of you.

Depending on how you make them, you may need to write a note to yourself — like the date you made the password.

Let’s take a slight variation on this theme for another example:

$ echo mybasicpassword@mybank_022709 | sha256sum 15218a3a5bed25963213e9b558f62d36dffc916dcc874ff307a37b26e62b6257

So in a secure place, like a TrueCrypt encrypted volume you write a note like:

mybank 022709

That really doesn’t reveal much at all, since you still now the algorithm (in this case ‘base password’@'resource’_'date’) in your head.

Now maybe the bank requires special letters and characters. So on your cheat sheet you write:

mybank 1521 8a3a 5bed 2596 3213 e9b5 58f6 2d36 dffc 916d cc87 4ff3 07a3 7b26 e62b 6257 +A-

Using the same choice in characters I stated above, you look at that and realize you’ve set your password to be 72b123A- .

If you’re using a system you trust you can use a tool like Password Safe to keep your website passwords without having to type each one in each time.

Of course you *should* be using two factor authentication whenever you can. For the times you can’t, I believe the system I laid out here is almost as strong — and most importantly prevents the breach of one or any combination of resources from exposing many other resources where you have an account.

NTP Configuration

Matt — Thu, 24 Jan 2008 19:10:46 +0000

1) Install ntpd if not already there. Link it to your default startup directory.

ln -s /etc/init.d/ntpd /etc/rc3.d/S99ntpd

2) /etc/ntp/step-tickers:

time.nist.gov
pool.ntp.org

3) In /etc/sysconfig/iptables open up the port:

# NTP is only accessible via the Gig Backbone:
-A RH-Firewall-1-INPUT -m state –state NEW -m udp -p udp –dport 123 -s 192.168.1.0/0 -j ACCEPT

4) Now (re)start iptables & ntpd

5) On internal-only servers, install ntpd.

6) Set step-ticker to point to internal server. If using hostname, verify /etc/hosts and/or the /etc/resolv.conf is set properly to allow the hostname to resolve.

7) Link it to your default startup directory.

Start it

D90 Tools & Techniques » Linux

Custom 404 for Apache (using PHP!)

Rootkit Hunters

Cool one-line report…

Squid handling http –> https redirects

Lighttpd, virtual hosts, alternative ports

Lighttpd, virtual hosts, and wildcard domains

Installing Squid to handle both 80 and 443

Optimizing Website integration with Amazon’s S3 Service

1) Let’s look at a redacted Squid log after the February, 2009 redesign of www.getmiro.com when not using S3 at all:

2) Now let’s look if we load CSS from our server, but use Apache to re-write images and js to the S3 service:

3) Now let’s use Apache to redirect the CSS to be pulled from Amazon S3.

4) Now we get fancy.

5) Finally one more tweak.

Bottom line?

Passwords and Data Mining

NTP Configuration

1)
Let’s look at a redacted Squid log after the February, 2009 redesign of www.getmiro.com when not using S3 at all:

2)
Now let’s look if we load CSS from our server, but use Apache to re-write images and js to the S3 service:

3)
Now let’s use Apache to redirect the CSS to be pulled from Amazon S3.

4)
Now we get fancy.

5)
Finally one more tweak.