Etherpad is a lovely app which is bought by google and was opensourced 10 days before. Now, just to add a drop of water to the ocean.

A small how-to on deploying Etherpad

The package

How to compile, run and test

  1. Grab the code from my github repository
    $ git clone git://github.com/karteek/etherpad.git
    You can optionally do a diff -r to above created clone with original clone to see the changes I’ve made to the code.
  2. Set up the environment to compile the code. Open the provided env.sh, and look at the needed variables. Existing values are values from my MBP, and they most probably won’t work for you
    $ source env.sh

  3. Create a database “etherpad” on your MySQL database. Same time, create a user “etherpad” with password “password” and grant him all privileges on the database “etherpad”

  4. Update etherpad.SQL_JDBC_URL, etherpad.SQL_USERNAME, etherpad.SQL_PASSWORD and etherpad.adminPass in the file etherpad/etc/etherpad.localdev-default.properties

    • etherpad.SQL_JDBC_URL is the jdbc URL of your database (in our case, its jdbc:mysql://localhost:3306/etherpad )
    • etherpad.SQL_USERNAME is the username to connect to database (in our case, its etherpad)
    • etherpad.SQL_PASSWORD is the password to connect to database (in our case, its password)
    • etherpad.adminPass is the password needed to login to admin panel of Etherpad instance
  5. Change directory to etherpad, compile the jar, and start the server
    $ cd etherpad
    $ bin/rebuildjar.sh
    $ bin/run-local.sh
  6. Now, fire up your browser, and visit http://localhost:9000 to play with your instance

File Imports

  1. File Imports depend on com.oreilly.servlet (cos.jar found at http://www.servlets.com/cos/)
  2. Check their license @ http://www.servlets.com/cos/license.html. If you are fine with their license, download cos.jar and copy it to infrastructure/lib folder
  3. Search for locations where file upload related code is commented, and uncomment the code.
    $ grep -r “REMOVED_COS_OF_COS” *
  4. Recompile the package
    $ cd etherpad
    $ bin/rebuildjar.sh

File Exports

  1. File exports depend on Openoffice.org for conversion
  2. You need to start Oo.org as a service
    $ /path/to/openoffice/soffice.bin -headless -nofirststartwizard               -accept=”socket,host=localhost,port=8100;urp;StarOffice.Service”
  3. Just make sure that etherpad.soffice has some value in etherpad/etc/etherpad.localdev-default.properties
  4. Check infrastructure/com.etherpad.openofficeservice/importexport.scala for more information

How to deploy the same on example.com

  1. Wild card dns for your domain. Point *.example.com to same server where example.com resides. This is the server where you are deploying this instance.
  2. Make sure that your SMTP is working. Activation of Pro accounts happen over email, so its important.
  3. Open etherpad/src/main.js, and update domain in line #273
  4. Open etherpad/src/etherpad/globals.js and change the domain in variable  SUPERDOMAINS found at line #30
  5. Open etherpad/src/static/crossdomain.xml and add your domain to crossdomains.xml
  6. Look into etherpad/src folder and search for etherpad.com. You will find many locations, where your instance will be coded to mail from an @etherpad.com account. Update all of them to your domain.
    $ grep -ir “etherpad.com” *
  7. Open etherpad/etc/etherpad.localdev-default.properties and update
    • devMode to false
    • etherpad.isProduction to true
    • listen to example.com:80
    • etherpad.isPNE to true (only if you want to run in PNE mode)
    • And ofcourse, change etherpad.adminPass
  8. As long as you din’t change anything in infrastructure folder, you need not recompile the jar. So, just go ahead and start the server. For starting a server on port numbers less than 1000, you will need to have root access on OS X.

Running the instance behind Apache using mod_proxy

  1. Do the same as above from steps #1 to #7.
  2. In step #7, rather than changing listen to example.com:80, change it to localhost:9000 or just 9000
  3. Start the server using bin/run-local.sh
  4. Your apache configuration might look like -
    <VirtualHost *:80>
    ServerName www.example.com
    ServerAlias *.example.com
    ServerAdmin admin@example.com
    ErrorLog /var/log/apache2/etherpad.error.log
    LogLevel warn
    CustomLog /var/log/apache2/etherpad.access.log combined
    ServerSignature Off
    ProxyPass / http://localhost:9000/
    ProxyPassReverse / http://localhost:9000/
    ProxyPreserveHost on
    <Proxy *>
    Options FollowSymLinks MultiViews
    AllowOverride All
    Order allow,deny
    Allow from all
    </Proxy>
    </VirtualHost>
  5. This might or might not work for you. Anyway, I guess you got the idea. Check manual for more information

Well, the patch I wrote for enabling file exports so as this tutorial come with no warranty. Don’t ask me even if it breaks your bones. This is what I did, following it is completely your choice. Code from Etherpad is under the license they specified. The file import part depends on MultipartWrapper and MultipartFilter. If you don’t like their license, its better to write your own code for doing their work.

Feel free to ask any doubts and/or fix my bugs. There is a great tutorial on installing Etherpad by Nuba Princigalli. Do check it, if you find any problems with my post.

Zimbra offers an awesome Open Source collaboration suite. EC2 is an awesome compute cloud service. The points I learned and did while deploying Zimbra on EC2.

  1. Zimbra on Ubuntu is supported on Hardy (8.04), but not newer. Install only if you want an adventure. I didn’t want an adventure, and I love Hardy.
  2. Hardy EBS backed image was not available on Amazon (by today) So, I did this, and got things ready. (If you want the AMI, let me know. I can share the same with you.)
  3. Download Zimbra Ubuntu package, unpack stuff and … wait.
  4. Zimbra uses LDAP for maintaining user registry, so, you need to know that Zimbra is going to install LDAP for you.
  5. But, LDAP has a weird habit of checking for host over DNS even if you install the LDAP server on same machine AND editing /etc/hosts file do not suffice.
  6. So, you need few entries in your DNS.
    1. MX Entry for your website. (MX entry tells which server will handle mails for that particular domain)
    2. A Entry for your MX server. (that particular mail server should be resolved to an IP)
  7. This should solve your problems if your servers are connected to internet, but most of time, they do not. They will be behind a firewall, and they will have LAN IPs which won’t be same as your public IPs, and your server might not even know that it has a public IP (server will just know natted IP). This is just the case with EC2.
  8. So, now you need to configure a DNS server on your LAN to do the work for you. Follow this tutorial on how to create zones. It will help you to create proper entries on your Bind server.
  9. Confirm that your Bind server is working by doing. Replace example.com and mail.example.com with appropriate domain names.
    • dig @localhost example.com MX  - to see your server’s MX record
    • dig @localhost mail.example.com A - to see your mail server’s A record
  10. Once, local DNS is configured. Add an entry to your /etc/resolv.conf and also add mail.example.com xxx.xxx.xxx.xxx to your /etc/hosts. Replace domain and IP appropriately.
  11. Now, you can continue with installation of Zimbra. Just start the installation. And follow the steps. Zimbra installation, by itself is a fairly simple process.
  12. During the installation process, Zimbra will think that mail.example.com is the domain for which you are installing, and try to check MX record for it. As, we do not want email for our mail server (we dont want mail@mail.example.com, rather we want mail@example.com), we didn’t not create any MX entries for for mail.example.com. So, Zimbra will fail to check for MX record of mail.example.com and ask whether you want to change it. Now, change it to example.com
  13. And, finish installation.
  14. Last and important, goto your AWS console, Security Groups, and enable SMTP, SMTPS (IMAP, IMAPS, POP3, POP3S ports if needed)

Now, as you are on EC2, there are some issues, you are going to face.

  1. EC2 doesn’t support reverse DNS, which means, your example.com will resolve to Amazon IP, but that amazon IP when nslookup’ed on, will return something like ec2-xxx-xxx-xxx-xxx.compute-1.amazonaws.com rather than example.com. There is no way of changing this as of now. LittleScrewed.
  2. That means, some mail servers might not accept your mails, thinking that your spamming. Screwed.
  3. EC2 has very bad limit on sending outgoing emails from its SMTP ports. I got an email from EC2 abuse for sending 10 emails. Totally Screwed.

But, as light at end of the tunnel, there are few things you can do.

  • You can use an external SMTP service to send email. For that you have to configure your MTA ie., Postfix here, to relay emails using that external SMTP service. Of course, the a reliable external SMTP service will add some weight to your bill.
  • Configure a proper SPF on your DNS and pray that all mail servers accept your email.
  • Rather than going for external SMTP, you can configure SPF and request Amazon to increate your email limit and hope they approve.

But, because bad things always happen, few mail servers like my employers, throw you a 554 code when reverse DNS fail for your IP. Logically speaking, things should work when proper SPF is configured, but, this is practical world. Things are always beyond the control of standards. If you are an IE user, you would never know this.

Filed ↓ ZimbraEC2TechTips

After Amazon launched EC2 back in 2006, they’ve been adding new features in regular intervals. On EC2, for starters, one thing that confuses a lot is the lack of persistent storage - in simple words, you lose all changes made and data once you reboot. There were options like S3 and EBS though to over come this problem. But, much configuration is needed, need for regular backups, booting was slower etc., etc.

One of their recent addition to features list is EBS-backed AMI. It allows an EBS volume to be root device for an instance, in other words, one can boot an instance from EBS volume. Property of EBS is being persistent, which automatically makes your instance’s root persistant, which means, one can reboot their instance as much as they want without having to worry about losing changes they made to the settings or their data. So, conclusion is EBS backed EC2 is … cool.

But, if you are an Ubuntu guy like me, you would be disappointed to see that there are only 4 (as of now) public EBS based Ubuntu Images on EC2 right now, and you would be even more disappointed to see that none of them is Hardy (8.04.3 LTS) image. No LTS image. But, Everything’s shiny captain, nothing to fret. One very nice tutorial and a kick-ass Python API for EC2 come to rescue.

This tutorial on Elastician which is based on Boto comes to rescue and help you to create an EBS-backed AMI. Before going through and following that tutorial, just make sure that you have latest version of Boto (>1.8d). Just make sure that the AMI which you are using to create an EBS-backed AMI is a reliable one.

By the way, you can see an awesome list of available images at The Cloud Market.

Filed ↓ EC2EBSBoto

After two years of my last migration from blogspot to wordpress, Here I am, after 3 years of blogging, migrated to third service. This time, moving out of self-hosted wordpress to a nice hosted solution Tumblr. You might haven’t seen many differences other than theme becoming extremely minimalistic.

Well, there are the changes.

  1. Now, the service is no longer wordpress, its Tumblr :D
  2. All posts are migrated
  3. All comments are migrated to Disqus, a very good global comment system
  4. URLs of all posts are changed, but any time you can you use the search
  5. All bookmarks, external links, google indexed pages till re-indexing are broken. Boo Hoo.

Migration went smooth, but there was a small problem for doing the same. There were no softwares available to do the same. “Migration from Wordpress to Tumblr”. If any existed, they will certainly miss migrating the comments or the attachments you’ve done in your previous wordpress blog. And, it of course is a tedious task to migrate everything if you do that manually.

So, on my way here, I wrote a small script for doing the same - Wordpress to Tumblr migration. I called it Wp2Tumblr.Py

Well, it does the following things -

  1. Takes your wordpress extended rss archive and parses it
  2. Post all your wordpress posts to Tumblr
  3. While doing above step, it posts all your image uploads to Tumblr as Private post, extract the url of the image from Tumblr post, change the same in  your wordpress post.
  4. Migrate all the comments of the post to your disqus account and link them to your post

Well, as I promised, I’m going to release all my personal code as opensource under WTFPL. So, if you’ve a similar requirement, you can clone the repo from http://github.com/karteek/wp2tumblr.py

By the way, I didn’t tell you whats better in Tumblr than Wordpress yet. These are few I found as new

  • Totally customizable themes. Of course, you can do same on self-hosted versions of wordpress. Paid feature on wordpress.com, I guess.
  • Multiple of types of posts. Unnecessary in my opinion, but what the hell.
  • Custom domain support. Self hosted wordpress, obvious. wordpress.com, its a paid feature.

Above all, the main reason, I wanted to move to a hosted service. Blogger is too messy. Comment system sucks there. Wordpress is nice, but I couldn’t get username ‘karteek’ and custom domains are not part of standard feature set and I was feeling all excited for an adventure :p

I still think that Wordpress is the best blogging software available, but updating the codebase, payment to the hosting provider etc etc are few things you really are not supposed to be worrying about when you blog for fun. Tumblr made life easy, without losing my identity - karteek.selfdabba.com. All I had to do is point my subdomain’s A record to Tumblr’s server.

Its been a year that Bombay was attacked by terrorists and killed a 166 innocent people. Well, whats happening after that.

- India has got a new guest. Terrorist Kasab. Yeah, Government has spent 31 crores for his safety
- Pakistan is still playing blame game that India is not giving enough evidence to arrest Saeed, founder of a terrorist group Lashkar-e-Taiba
- US is giving more and more aid to Pakistan to fight terrorism
- India is expecting US to be headmaster and detain Pakistan
- Pakistan is expecting US to be headmaster and force India to give Kashmir to Pakistan
- US doesn’t care if Pakistan is sending terrorists to India as long as it kills the terrorists which are attacking the US.
- Headmaster as well as both the kids know that things are not moving anywhere.

So, after all the crap that has happened in last one year, troubled remained troubled, terrorists remained terrorists, victims remained victims, and every one’s shitty life just remained shitty life. Just to find out what Google thinks about India and Pakistan, I took couple of screenshots -

So, people who search Google think that India is the richest country in the world, in asia, a third world country, great, Afghanistan’s largest trading partner. They want to know what India is known for and which continent it is in.

And about Pakistan, they think that its failed state, dangerous, doomed, a problem and a mess and few think its better than India. Few want to know where it is located.

Well, as an Indian, I think Pakistan’s leaders are stupid, immature at the same time, Indian leaders are just stupid other than a few like Raj Thackeray. They are stupid as well as immature.

TL;DR - One year over after attacks. No progress, none punished. Pakistan leaders are stupid.


Page 2 of 33