SSL using website -- Setting up the server

First of all you need to realize that SSL doesn't work for name based virtual hosts so it needs to be an ip. Technically 1 SSL using host does work it does but the SSL cert is shared for all sites and this is a serious issue from a business/search engine/customer point of view.

1. Check that you have completed the basic ssl module set up
   

   sudo a2enmod ssl
   

   
   And check that the server is listening on 443. either netstat antp for it or grep for the Listen line in the apache2 config files.
   

   Listen 443
   

2. 
   Choose a new ip address for the ssl version of the server. Lets say we pick the IP: 192.168.1.200. Once you have gotten the crt back from the provider move it into place and remember to chown and chmod it for root only

   Lets assume we placed it at; /etc/apache2/ss/www.mysite.com.crt

   
   

3. Now to multi-home the server (ie give it the new ip address to play with). You do this by editing and appending the following to /etc/network/interfaces
   

   #this is mysite's ip for its ssl
   auto eth0:1
     iface eth0:1 inet static
     address 192.168.1.200
     netmask 255.255.255.0
     network 192.168.1.0
     broadcast 192.168.1.255
   

   Remember to update your DNS server if needed
   
   
4. Then add a new virtual host for the SSL version of the site.
   
   

   <VirtualHost 192.168.1.200:443>
     ... COPY OF NON-SSL VERSIONS SETTINGS ...
   
     SSLEngine On
     SSLCertificateKeyFile   /etc/apache2/ssl/www.mysite.com.key
     SSLCertificateFile      /etc/apache2/ssl/www.mysite.com.crt
   </VirtualHost>
   

I may have missed a few things since my servers have been serving SSL for a long time now. Refer:
https://help.ubuntu.com/8.04/serverguide/C/httpd.html https://help.ubuntu.com/8.04/serverguide/C/certificates-and-security.html

SSL using website - Generating an SSL cert for apache2

1. Generate a the ssl key pair
   

   openssl genrsa -out www.mysite.com.key 2048
   

   • using a pass-phrase is problematic since the apache2 server cant boot with the having the pass-phrase input.
     
   • if you need to use a passphrase then add on -des3 param
     
2. Generate a code signing request (without a pass-phrase)
   

   openssl req -new -key www.mysite.com.key -out www.mysite.com.csr
   

   • do not enter an email address, challenge password or an optional company name when generating the CSR.
     
   • Enter the info which MATCHES THE WHOIS for the domain or your request is likely to get rejected.
     
     • Country Name:
       
     • State or Province: (the capitalized two letter code)
       
     • Locality or City: (without abbreviations)
       
     • Company: (without &, @, or any other symbol)
       
     • Organizational Unit: (optional; to skip hit enter)
       
     • Common Name: the host name ie "www.mysite.com" (make certain it matches the main one used by end customers, to avoid ssl mismatch warnings.)

3. Send the code code signing request to the certificate authority and wait for them to send the signed certificate back (the crt file).
   
4. The files should be stored at the following location with the following permissions/owner. Remember to do it or the key can be viewed and copied.
   

   /etc/apache2/ssl$ ls -al
   drwxr-xr-x 2 root root 4096 2010-01-08 16:38 .
   drwxr-xr-x 9 root root 4096 2010-01-08 09:42 ..
   -r-------- 1 root root 1354 2010-01-08 09:17 www.mysite.com.crt
   -r-------- 1 root root 1354 2010-01-08 09:17 www.mysite.com.csr
   -r-------- 1 root root 1675 2010-01-08 16:38 www.mysite.com.key
   

Refer https://knowledge.verisign.com/support/ssl-certificates-support/index?page=content&actp=CROSSLINK&id=AR198

rails - Forcing a certain encoding type for the page

response.headers["Content-Type"] = "text/html; charset=shift_jis"

tcpdump to debug the an encoding problem

This is kind of over kill but the HTTP headers plugin for firefox wasn't telling me the truth. I dev web apps using a locale apache2 server. And in this case I needed to deal out an sjis page, of course rails is utf-8 inside and somewhere/somehow the encoding is getting forced to utf8. So to get the truth tcpdump it.

sudo tcpdump -i lo  -Xx -s1500

After this I refreshed the page in question and read the log. Its was clear that rails is always writing utf-8 into the HTTP header which overrode the weaker meta tag setting I was trying to us.

fixing "svn: Malformed file" and other broken svn file problems...

This happens when you get to smart for yourself and alter the contents of the .svn dir accidentally...

To fix it;
Basically move the whole working dir over and recheck out and then restore the new/edited/deleted files. Sounds hard but it is quite easy(especially if you have had to do it a few times).... Run the following commands on the console. And note the commands with "| sh" allow you to first confirm the exact action before executing it so use it to double check before you make a total mess

#move broken trunk out of the way
mv trunk broken
mkdir trunk 
cd trunk
svn checkout https://server.svn/repos/project/trunk
cd ../broken

# make the new dirs -- confirm first...
find ./ -type d | grep -v svn | egrep "^./(app|test|lib|db)" | sed "s|\(.*\)|mkdir -p ../trunk/\1|"
find ./ -type d | grep -v svn | egrep "^./(app|test|lib|db)" | sed "s|\(.*\)|mkdir -p ../trunk/\1|" | sh
# move all the normal non-svn files that are relevant back in (im my case its a rails app) -- confirm first...
find ./ -type f | grep -v svn | egrep "^./(app|test|lib|db)" | sed "s|\(.*\)|cp \1 ../trunk/\1|"
find ./ -type f | grep -v svn | egrep "^./(app|test|lib|db)" | sed "s|\(.*\)|cp \1 ../trunk/\1|" | sh
# remove all deleted files. -- confirm first...
svn status | grep "^D" | sed "s|^D *|svn del ../trunk/|"
svn status | grep "^D" | sed "s|^D *|svn del ../trunk/|" | sh

rails - using truncate from a the lib dir ie a module

Here is how to get the helpers into the libs area, Sometimes the helpers use/set data on the controller etc so this doesn't work.

module MyModule
  include ActionView::Helpers::TextHelper
  module_function :truncate
end

Here is how to get it on the console

helper.truncate(string, :length => length)

A quick javascript console web page.

If you have problematic javascript on a website then its often a pain to locate the exact cause of the problem in the particular browser (excluding firefox due to firebug). This site gives you a nice in browser console to work out your issues on.

http://www.jconsole.com/

Hopefully some hack doesn't find it and use it for blackhat purposes that results in it getting taken down.

/proc/: The linux kernels live state

/proc/ is a virtual (ie in memory) part of the linux file-system that displays the current status and configuration of the kernel.

The amount of information about the system in this area is amazing.

For example try out these commands

cat /proc/cpuinfo
cat /proc/meminfo
cat /proc/version
cat /proc/uptime
cat /proc/loadavg
cat /proc/net/dev

When scripting you will probably want to use the info in a more digested form. The command for a script could be something like

cat cpuinfo  | grep vend |  cut -d' ' -f2

A useful debug trick is to determine what the environment settings where for a process after it has been started. To do so you would cat the processes environ file like so;

cat /proc/<process_id>/environ

The /proc/sys directory can also be used to alter certain files in a live system to adjust the kernel system wide. To tell which files can be altered just ls -al and look for the writable bit. Keep in mind that changes made in the area are temporary, and some changes can easily kill the system. To make the changes permanent the /etc/sysctl.conf needs to be edited.

Refer to
http://www.linuxjournal.com/article/8381

http://linuxhelp.blogspot.com/2005/04/proc-filesystem.html

crontab for every X minutes excluding a period Y hours

Crontab isnt really hard its just limited.

To execute for every 5 minutes except between 1:00 to 2:00

*/5 0-1,2-23 * * * /bin/hello.sh

If you need finer control than an hour you might need to consider using 2 lines;

*/5 0-1,2-23 * * * /bin/hello.sh
0,5,10,55 1-2 * * * /bin/hello.sh

mass replacement of text in files

UPDATE: Phil pointed out there was an easier way to get mass replacement of text, Kudos mate.

find ./ -type f | grep -v svn  | xargs perl -pi -e "s/partA/partB/g;s/partC/partD/g;s/partE/partF/g"

Here is a crazy bit of script that uses a pair of seds to replace a several pieces of text across all files in the directory and below.

#example script that converts the following
#  _address => _address1
#  _address_street => _address2
#  _address_building => _address3
find ./ -type f | grep -v svn | sed "s/\(.*\)/sed 's|_address_building|_address3|g' \1 > \1.script; mv \1.script \1/" | sh
find ./ -type f | grep -v svn | sed "s/\(.*\)/sed 's|_address_street|_address2|g' \1 > \1.script; mv \1.script \1/" | sh
find ./ -type f | grep -v svn | sed "s/\(.*\)/sed 's|_address|_address1|g' \1 > \1.script; mv \1.script \1/" | sh
find ./ -type f | grep -v svn | sed "s/\(.*\)/sed 's|_address12|_address2|g' \1 > \1.script; mv \1.script \1/" | sh
find ./ -type f | grep -v svn | sed "s/\(.*\)/sed 's|_address13|_address3|g' \1 > \1.script; mv \1.script \1/" | sh

Free OCR that is worth it

Wow OCR of Japanese text really is hard but this system just did a great job for me (>80% accurate). It worked best with large pictures

http://weocr.ocrgrid.org/cgi-bin/weocr/search.cgi?lang=jpn&fmt=html

PS i tried the iPhone version "C'est What!!" of the app it doesn't work so dont purchase it.

rails testing - undefined method `request='

Problem:

test_some_functional_test_case(SomeFunctionalControllerTest):
NoMethodError: undefined method `request=' for #<SomeMispeltFunctionalController:0x7f5b55bd0a58>

Solution:
Basically it means you haven't required the controllers file. Or you have a typo in the name that is instantiating the a non controller for the @controller variable in the test.

rails - DEFAULT_RAILS_LOGGER not working in libs

I just noticed that "DEFAULT_RAILS_LOGGER" is not working in libs on some of the newer rails versions.. The new alternative is "Rails.logger" it seems to work perfectly fine.

OWASP - Web site security documents of worth

The owasp site is a great resource for web site security guide lines

http://www.owasp.org/

http://www.owasp.org/index.php/XSS_%28Cross_Site_Scripting%29_Prevention_Cheat_Sheet

http://code.google.com/p/owasp-development-guide/wiki/WebAppSecDesignGuide_D6

Script injection attacks to be aware of

The following is an xss attack that few people release is possible;

<script>
alert('</script><script>alert(2);</script>');
alert('1');
</script>

How it works:
Its simple really the HTML parser of browsers first parse the HTML structure of the page, they have no knowledge of the structure of javascript and contents contained in the tag body. _ANY_ HTML tag, even one present in correctly escaped javascript takes precedence.

Interview puzzle questions

This site has a good list of interview questions and puzzles;

http://www.mytechinterviews.com/category/puzzles

If I bother to work out a problem that is on that blog myself that ill still post it here but.. for now its a good resource for interview prep.

cleaning up locales in ubuntu

if you notice your ubuntu install wasting space in locales here is how to kill it quickly

sudo apt-get install localepurge
sudo localepurge

Rails DateTime converting into your time zone

Quick and dirty way to get a time into your locale

DateTime.parse(some_other_time_string).in_time_zone(::Time.zone)

Confirming and ip/email/web sites owner

Scammers will often try to make themselves look more presentable by hiring a web designer/programmer to make a professional site for them. Then they simply walk away with the code and content without paying and worst still use your work to scam a bigger fish, most likely with info in it that clearly identifies it as YOUR work.

Here are a couple of nslookup/dig/whois trace examples

nslookup 66.249.89.104
dig -x 66.249.89.104
dig 104.89.249.66.in-addr.arpa
whois  66.249.89.104

Normally you can trace digs actions to see what servers are being talked with however my DSL modems dns server doesnt seem to not support it. So I cant use it. Here it is anyway.

dig -x 66.249.89.104 +trace

Here is a manually worked trace

dig @a.root-servers.net 104.89.249.66.in-addr.arpa +norecurse
dig @V.ARIN.NET 104.89.249.66.in-addr.arpa +norecurse
dig @ns2.google.com 104.89.249.66.in-addr.arpa +norecurse
dig @ns1.google.com -x 66.249.89.104 +norecurse

For this trace the path ends at an SOA record

dig @a.root-servers.net www.google.com +norecurse
dig @g.gtld-servers.net  www.google.com  +norecurse
dig @ns1.google.com  www.google.com  +norecurse

Now what to do with that raw info. Here are some ways to figure out if its legit

• Grab domain name with a dig or nslookup on the real ips/urls that find in the emails
  
• Grab the who-is info off the urls and find the registered owners,
  
• Check them against an scammer list like http://www.autosurfinfo.net/badiplist.html
  

Then Since your dealing with a company;

• Google search the company and its affiliates, Using the info from the whois and what they claimed to be in there emails.
  
• Then First confirm the basics about the company web site.
  
  • Confirm that your looking at the real companies site. Not some fake site that is trying to trick you into believing its part of the really company. ie microsoft.com vs microsoft.jack.com
    
  • Are they claiming to be a massive company and yet they have a website made by some crud web publishing.
    
  • Is the content all pictures or stolen text (possibly modified when taken from a 3rd party place)
    
  • Is the web site hosted in a free host somewhere
    
• Then you want to look at the companies location and employer info;
  
  • Google map the location and look at the building with street view
    
  • Does it have the companies names/sign logo etc
    
  • Can the companies number of employees actually fit in the building/floor they claim to be on?
    
  • Does the office space even appear to be used?
    
  • Is the building/location appropriate for there type of business?
    
• Then you want to look at the companies finance info;
  
  • How much operating cash do they have and how many employees, figure out the ratio of cash to employees then match that to what that employee would be payed yearly if the number is too far off then how are they paying employees?
    
  • Is size/wealth of company reflected in the web site. Why would a multi-million dollar company have a crub website?
    
  • And look over the products/services that they sell and search around for 3rd party info on
    it. Confirm that they really can make the cash they are talking about.
    

Rails - 422 error code

The Rail 422 error code is a bit of an odd ball.

Most likely cause of this is the CSRF is firing and killing whatever is posting to your site from outside in my case its PAYPAL posting an IPN request to a site im working on.

To fix it add this line of code to the controller

If entire controller can be posted to from out side then this is ok

skip_before_filter :verify_authenticity_token

If you need to open access to a single page then try this:

protect_from_forgery :execpt => :pages_that_are_posted_to_from_out_side