So, Landmark, a famous book/music/movies store in India is on the net at LandmarkOnTheNet.com. I was there to check whether they sell 3D glasses, even though there are some cheap paper glasses on ebay.in, but I want better quality. As usual, the bad habit of checking for SQL Injections and XSS tingled in my brain.

Apparently, LandmarkOnTheNet has both the problems.

and for SQL Injection, I didn’t try much, just give a single quote as input in search, the server choked with an error in the SQL Query (it actually told what the error is and where the error is; what a away to configure the production server)

So, there are many things you can have fun with above two, these are the keywords - My Account, Session Stealing, Gift Cards, SQL Injection, PIN Numbers, Free Shipping.

If I were a landmark website user, I would ensure that my browser is Firefox with NoScript addon installed. If I were a landmark webmaster, I would buy a book on Web Security right now, ah nevermind, I would just search in Google for tips of fixing XSS and SQL Injection vulnerabilities and fix them ASAP. I might also feel very bad for using so many tables in my markup.

And … Good Luck for that.

Etherpad is a lovely app which is bought by google and was opensourced 10 days before. Now, just to add a drop of water to the ocean.

A small how-to on deploying Etherpad

The package

• Etherpad source from http://code.google.com/p/etherpad
• Some minor fixes and changes, here and there
• File Imports [gist]
• File exports to DOC/PDF/ODT using Oo.org [gist]

How to compile, run and test

1. Grab the code from my github repository
   $ git clone git://github.com/karteek/etherpad.git
   You can optionally do a diff -r to above created clone with original clone to see the changes I’ve made to the code.

2. Set up the environment to compile the code. Open the provided env.sh, and look at the needed variables. Existing values are values from my MBP, and they most probably won’t work for you
   $ source env.sh

3. Create a database “etherpad” on your MySQL database. Same time, create a user “etherpad” with password “password” and grant him all privileges on the database “etherpad”

4. Update etherpad.SQL_JDBC_URL, etherpad.SQL_USERNAME, etherpad.SQL_PASSWORD and etherpad.adminPass in the file etherpad/etc/etherpad.localdev-default.properties

   • etherpad.SQL_JDBC_URL is the jdbc URL of your database (in our case, its jdbc:mysql://localhost:3306/etherpad )
   • etherpad.SQL_USERNAME is the username to connect to database (in our case, its etherpad)
   • etherpad.SQL_PASSWORD is the password to connect to database (in our case, its password)
   • etherpad.adminPass is the password needed to login to admin panel of Etherpad instance
5. Change directory to etherpad, compile the jar, and start the server
   $ cd etherpad
   $ bin/rebuildjar.sh
   $ bin/run-local.sh
6. Now, fire up your browser, and visit http://localhost:9000 to play with your instance

File Imports

1. File Imports depend on com.oreilly.servlet (cos.jar found at http://www.servlets.com/cos/)
2. Check their license @ http://www.servlets.com/cos/license.html. If you are fine with their license, download cos.jar and copy it to infrastructure/lib folder
3. Search for locations where file upload related code is commented, and uncomment the code.
   $ grep -r “REMOVED_COS_OF_COS” *
4. Recompile the package
   $ cd etherpad
   $ bin/rebuildjar.sh

File Exports

1. File exports depend on Openoffice.org for conversion
2. You need to start Oo.org as a service
   $ /path/to/openoffice/soffice.bin -headless -nofirststartwizard               -accept=”socket,host=localhost,port=8100;urp;StarOffice.Service”
3. Just make sure that etherpad.soffice has some value in etherpad/etc/etherpad.localdev-default.properties
4. Check infrastructure/com.etherpad.openofficeservice/importexport.scala for more information

How to deploy the same on example.com

1. Wild card dns for your domain. Point *.example.com to same server where example.com resides. This is the server where you are deploying this instance.
2. Make sure that your SMTP is working. Activation of Pro accounts happen over email, so its important.
3. Open etherpad/src/main.js, and update domain in line #273
4. Open etherpad/src/etherpad/globals.js and change the domain in variable  SUPERDOMAINS found at line #30
5. Open etherpad/src/static/crossdomain.xml and add your domain to crossdomains.xml
6. Look into etherpad/src folder and search for etherpad.com. You will find many locations, where your instance will be coded to mail from an @etherpad.com account. Update all of them to your domain.
   $ grep -ir “etherpad.com” *
7. Open etherpad/etc/etherpad.localdev-default.properties and update
   • devMode to false
   • etherpad.isProduction to true
   • listen to example.com:80
   • etherpad.isPNE to true (only if you want to run in PNE mode)
   • And ofcourse, change etherpad.adminPass
8. As long as you din’t change anything in infrastructure folder, you need not recompile the jar. So, just go ahead and start the server. For starting a server on port numbers less than 1000, you will need to have root access on OS X.

Running the instance behind Apache using mod_proxy

1. Do the same as above from steps #1 to #7.
2. In step #7, rather than changing listen to example.com:80, change it to localhost:9000 or just 9000
3. Start the server using bin/run-local.sh
4. Your apache configuration might look like -
   

   <VirtualHost *:80>
   ServerName      www.example.com
   ServerAlias       *.example.com
   ServerAdmin     admin@example.com
   ErrorLog        /var/log/apache2/etherpad.error.log
   LogLevel warn
   CustomLog /var/log/apache2/etherpad.access.log combined
   ServerSignature Off
   ProxyPass / http://localhost:9000/ 
   ProxyPassReverse / http://localhost:9000/
   ProxyPreserveHost on
     <Proxy *> 
       Options FollowSymLinks MultiViews
       AllowOverride All
       Order allow,deny
       Allow from all
     </Proxy>
   </VirtualHost>
   

5. This might or might not work for you. Anyway, I guess you got the idea. Check manual for more information

Well, the patch I wrote for enabling file exports so as this tutorial come with no warranty. Don’t ask me even if it breaks your bones. This is what I did, following it is completely your choice. Code from Etherpad is under the license they specified. The file import part depends on MultipartWrapper and MultipartFilter. If you don’t like their license, its better to write your own code for doing their work.

Feel free to ask any doubts and/or fix my bugs. There is a great tutorial on installing Etherpad by Nuba Princigalli. Do check it, if you find any problems with my post.

Zimbra offers an awesome Open Source collaboration suite. EC2 is an awesome compute cloud service. The points I learned and did while deploying Zimbra on EC2.

1. Zimbra on Ubuntu is supported on Hardy (8.04), but not newer. Install only if you want an adventure. I didn’t want an adventure, and I love Hardy.
2. Hardy EBS backed image was not available on Amazon (by today) So, I did this, and got things ready. (If you want the AMI, let me know. I can share the same with you.)
3. Download Zimbra Ubuntu package, unpack stuff and … wait.
4. Zimbra uses LDAP for maintaining user registry, so, you need to know that Zimbra is going to install LDAP for you.
5. But, LDAP has a weird habit of checking for host over DNS even if you install the LDAP server on same machine AND editing /etc/hosts file do not suffice.
6. So, you need few entries in your DNS.
   1. MX Entry for your website. (MX entry tells which server will handle mails for that particular domain)
   2. A Entry for your MX server. (that particular mail server should be resolved to an IP)
7. This should solve your problems if your servers are connected to internet, but most of time, they do not. They will be behind a firewall, and they will have LAN IPs which won’t be same as your public IPs, and your server might not even know that it has a public IP (server will just know natted IP). This is just the case with EC2.
8. So, now you need to configure a DNS server on your LAN to do the work for you. Follow this tutorial on how to create zones. It will help you to create proper entries on your Bind server.
9. Confirm that your Bind server is working by doing. Replace example.com and mail.example.com with appropriate domain names.
   • dig @localhost example.com MX  - to see your server’s MX record
   • dig @localhost mail.example.com A - to see your mail server’s A record
10. Once, local DNS is configured. Add an entry to your /etc/resolv.conf and also add mail.example.com xxx.xxx.xxx.xxx to your /etc/hosts. Replace domain and IP appropriately.
11. Now, you can continue with installation of Zimbra. Just start the installation. And follow the steps. Zimbra installation, by itself is a fairly simple process.
12. During the installation process, Zimbra will think that mail.example.com is the domain for which you are installing, and try to check MX record for it. As, we do not want email for our mail server (we dont want mail@mail.example.com, rather we want mail@example.com), we didn’t not create any MX entries for for mail.example.com. So, Zimbra will fail to check for MX record of mail.example.com and ask whether you want to change it. Now, change it to example.com
13. And, finish installation.
14. Last and important, goto your AWS console, Security Groups, and enable SMTP, SMTPS (IMAP, IMAPS, POP3, POP3S ports if needed)

Now, as you are on EC2, there are some issues, you are going to face.

1. EC2 doesn’t support reverse DNS, which means, your example.com will resolve to Amazon IP, but that amazon IP when nslookup’ed on, will return something like ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com rather than example.com. There is no way of changing this as of now. LittleScrewed.
2. That means, some mail servers might not accept your mails, thinking that your spamming. Screwed.
3. EC2 has very bad limit on sending outgoing emails from its SMTP ports. I got an email from EC2 abuse for sending 10 emails. Totally Screwed.

But, as light at end of the tunnel, there are few things you can do.

• You can use an external SMTP service to send email. For that you have to configure your MTA ie., Postfix here, to relay emails using that external SMTP service. Of course, the a reliable external SMTP service will add some weight to your bill.
• Configure a proper SPF on your DNS and pray that all mail servers accept your email.
• Rather than going for external SMTP, you can configure SPF and request Amazon to increate your email limit and hope they approve.

But, because bad things always happen, few mail servers like my employers, throw you a 554 code when reverse DNS fail for your IP. Logically speaking, things should work when proper SPF is configured, but, this is practical world. Things are always beyond the control of standards. If you are an IE user, you would never know this.

After Amazon launched EC2 back in 2006, they’ve been adding new features in regular intervals. On EC2, for starters, one thing that confuses a lot is the lack of persistent storage - in simple words, you lose all changes made and data once you reboot. There were options like S3 and EBS though to over come this problem. But, much configuration is needed, need for regular backups, booting was slower etc., etc.

One of their recent addition to features list is EBS-backed AMI. It allows an EBS volume to be root device for an instance, in other words, one can boot an instance from EBS volume. Property of EBS is being persistent, which automatically makes your instance’s root persistant, which means, one can reboot their instance as much as they want without having to worry about losing changes they made to the settings or their data. So, conclusion is EBS backed EC2 is … cool.

But, if you are an Ubuntu guy like me, you would be disappointed to see that there are only 4 (as of now) public EBS based Ubuntu Images on EC2 right now, and you would be even more disappointed to see that none of them is Hardy (8.04.3 LTS) image. No LTS image. But, Everything’s shiny captain, nothing to fret. One very nice tutorial and a kick-ass Python API for EC2 come to rescue.

This tutorial on Elastician which is based on Boto comes to rescue and help you to create an EBS-backed AMI. Before going through and following that tutorial, just make sure that you have latest version of Boto (>1.8d). Just make sure that the AMI which you are using to create an EBS-backed AMI is a reliable one.

By the way, you can see an awesome list of available images at The Cloud Market.