Thursday, November 13, 2014

Enough of this functionality!!!!

We have built an app. Next comes the difficult part of making it a delight to use the app.
This is where non functional aspects comes into play.


  1. Defects
  2. User experience
  3. performance
  4. analytics (metrics collection)
  5. logging
  6. error tracking

These are some of the things that come to my mind when we think of making an app usable.
We want the user to use the app as effortlessly as possible.
At the same time we want feedback so that we can improve the experience for the user.
Most of the above points fall under these 2 buckets. (feel free to add more buckets and topics under them or challenge the priority of these items)

Some of you may have worked on some of these topics before.
My ask from everyone is to select a topic from above and work on it for the coming 1 month in order to make the app rigorous.

Wednesday, October 29, 2014

Delegation to improve performance of your app

We have a need to send multiple emails to coursemates instantaneously whenever someone posts a takeaway in their course.

This is usually a time taking task since we have to iterate over the coursemates and keep sending individual email to each one of them who has the instantaneous email activated.

We have achieved it using the below approach
  1. use celery to start a worker on a different heroku dyno
  2. make the worker and your web server use the same AMQP url and DB url (heroku config will give you all the details)
  3. use a different Procfile to start your celery worker on worker dyno
  4. we are using sparse checkout in git so that we dont have merge conflicts whenever we push the latest to the worker dyno.

This can be used for pretty much any heavy weight tasks which need not be handled in the current thread.

Here is how you push code to the worker.

  1. git remote add worker git@heroku.com:worker-takeaway.git
  2. git pull worker master (to get the content from worker dyno git repo)
  3. add your Procfile to the sparse checkout list in your .git config to avoid getting it overwritten by the Procfile from origin.
  4. Procfile differs from worker dyno and app server dyno since the former is a worker and latter is web type of dyno.
  5. once you have added the Procfile to sparse checkout list get the latest from origin to include any changes to the tasks and push it to the worker.

Monday, September 22, 2014

What's needed for the future???

What features do we need by Dec??


  1. ability to Scale
  2. ability to provision new instance for a school
    1. decide on the approach
  3. aws vs heroku
  4. performance
  5. monitoring
  6. analytics
  7. error reporting
  8. db performance
  9. which db ???

Saturday, September 20, 2014

One Shot Dev Environment Setup using Vagrant


This env setup works with pretty much any OS and uses Vagrant.
Given below are the steps
Note: You will see some errors while running "vagrant up". no need to worry as long as you are able to do "vagrant ssh".
  1. Download Vagrant 1.5.4 here (https://www.vagrantup.com/downloads-archive.html)
  2. Download VirtualBox latest (https://www.virtualbox.org/wiki/Downloads)
  3. Make sure VirtualManage from program files/oracle/virtualbox dir is included in the path.
  4. Download the virtual box image of Django on Ubuntu from here. https://www.dropbox.com/s/aevgxvcwuozn1pj/django-base-v2.1.box?dl=0
  5. On Windows, Install  putty or Gitshell or some shell with ssh provision.
  6. once you have installed all three run the below commands in cygwin or gitshell
    1. vagrant box add django-base-v2 location-to-django-base-v2.box
    2. get the latest from github bolg repo to get the Vagrant file
    3. run "vagrant up" from bolg directory once you have the latest
    4. run "vagrant provision" once thats done
    5. run "vagrant ssh" to get into the vm.
    6. Once in the vm, run "export DJANGO_SETTINGS_MODULE=settings" and "export DJANGO_ENVIRONMENT=local"
    7. run "python manage.py runserver 0.0.0.0:8000" to start the server.
  7. Once the above is done, type "http://localhost:8000/" in your browser and you should see the app.
    1. this is happening because of port forwarding between VM and your machine.
  8. Now, since Django apps are loaded during runtime. you can use the editor of your choice and edit the files in the project and the app running in VM will pick up the changes.
    1. this is because of shared folders between your host machine and VM.
    2. now when you refresh the browser you see the changes made by you on host machine.
  9. Once you have the above run below commands to seed the db 
    1. python manage.py syncdb
    2. create admin user so that you can log into admin screen
    3. python manage.py migrate takeaway
    4. python manage.py migrate notifications
    5. go to this link localhost:8000/takeaway/initload to load data
    6. now login with ravi/abc123 or atluri/abc123 to enter the app
    7. have fun

Monday, September 1, 2014

Push code to Heroku

Checklist for developers to push code to Heroku

  1. Developer needs to be added as collaborator for the heroku app.
  2. Developer needs to add the right heroku git repo in his project.
    1. git remote add heroku git@heroku.com:<app-name>.git
  3. git push heroku master (to push code to heroku git repo)
  4. heroku logs (to check the log entries, verify if the app crashed or running)
  5. To run python commands shell or syncdb use heroku run.
    1. heroku run python manage.py shell
    2. heroku run python manage.py syncdb
  6. To run SQL on Heroku
    1. heroku pg:psql
    2. cat magic.sql | heroku pg:psql (to run the SQL statements from magic.sql file)
  7. To capture url of backed up database (https://devcenter.heroku.com/articles/pgbackups)
    1. heroku pgbackups:url  
    2. refer to the link above for import/export of data.
  8. heroku's documentation is very good and saves us lot of effort 
    1. https://devcenter.heroku.com/


Dreaded Git installation by RaviKiran

If you are on Ubuntu

1. $ sudo apt-get install git
  This step will install git on your machine

2. https://help.github.com/articles/set-up-git#setting-up-git

3. Use this page for ssh key generation. Blindly follow all steps
   https://help.github.com/articles/generating-ssh-keys

4. git clone git@github.com:ravikirandutta/BOLG.git   ( where you want to get the code)

This should be all the steps to get the codebase on to your system.

Saturday, August 30, 2014

Settings split

We are splitting the settings between live and local environments for takeaway app.
Followed the instructions given here to achieve that.
https://coderwall.com/p/v1sjsw

With this new change you need some modifications on your local env for this to work


  1. Set DJANGO_SETTINGS_MODULE env variable on your local to "settings".
  2. set DJANGO_ENVIRONMENT to "local"
this will load the local.py file from the settings folder.
So any settings you want to  change between local and prod, the entry needs to be put in both local.py and live.py with corresponding value.
For example, DEBUG is True in local.py and False in prod.
same goes for SESSION_TIMEOUT, if you want a longer timeout set it in local.py but dont check it in.


Sunday, August 24, 2014

Custom registration in Django

The Django-registration is a powerful plugin but you need to be careful as to how you handle it.
If you add custom fields to it make sure you do it the right way.
Sometimes the registration can fail silently when the validation of its fields fail and you will not be explicitly notified about it.
So whenever you see POST on registration yielding 200OK and simply returning the form again
you can check the below

  1. that all user specific fields are being sent (username, email, pwd1, pwd2)
  2. that none of the custom fields validation is failing (i tested it by extending ChoiceField with a NoValidationChoiceField and my registration seems to be passing)

Friday, August 8, 2014

Django Related setup


Ubuntu comes with default python 2.7

install pip using sudo apt-get install python-pip 


Create a virtualenv so that all you various versions of installations are localized to just that env.

For example : If you are working on 2 applications , one requiring 1.4 DJango and another one 1.6 Django
You do not want both of them to overlap. This is where virtualenv will come in handy.

use linux command sudo pip install virtualenv

Once you have the virtual env

create a virtual env  $virtualenv renv (renv can be any name you want)

before activating the virtual env set these variables up

edit your activate file inside renv/bin/ to include these exports
export DJANGO_SETTINGS_MODULE=settings
 export DJANGO_ENVIRONMENT=local

activate this virtualenv by issuing the command $ cd renv

followed by $ source/bin/activate  

This would save you the time in setting these values every time.

Once inside the virtual env and activated start installing the required software by using PIP.

Get the requirements.txt for your project.

But before installing the packages install psycopg libraries to avoid failure.
sudo apt-get install libpq-dev python-dev

now run

$ sudo pip install -r requirements.txt

This will install all the softwares listed in the file. At this point all of these softwares are only visible if the virtualenv (in this case renv) is activated.

You should have a clean build. If not document the errors here so that future Bolgians can seek help.

In order to install psycopg libraries use this
http://stackoverflow.com/questions/5420789/how-to-install-psycopg2-with-pip-on-python


Once you have the code from GIT and have installed all the python dependencies run the below commands to set up the data needed for logging in


  1. python manage.py syncdb
  2. create admin user so that you can log into admin screen
  3. python manage.py migrate takeaway
  4. python manage.py migrate notifications
  5. go to this link localhost:8000/takeaway/initload to load data
  6. now login with ravi/abc123 or atluri/abc123 to enter the app
  7. have fun


Getting started

This is an internal document for setting up development environment. May not necessarily make sense to everyone.

Brief explanation of the setup to get up and running
  1. Use a linux environment to start with. Windows works too as long as you can make it work.
  2. Install GIT, python 2.7.2, Django 1.6.5 and postgres (if possible)
  3. We use Django rest api for backend and backbone + HTML5 + CSS3 for front-end.
  4. We currently use sublime text for our development and chrome developer tools for debugging front end code. 
  5. We deploy to Heroku using git push. 

Thursday, July 3, 2014

Git revert changes

git reset <filename> to unstage the file

Once unstaged,
git checkout -- <filename> to get rid of the changes and go back to normal

Database heroku changes

Heroku add-on pgbackups provides ability to import export database.
link here: https://devcenter.heroku.com/articles/heroku-postgresql

https://devcenter.heroku.com/articles/heroku-postgres-import-export

 heroku pgbackups:capture
$ curl -o latest.dump `heroku pgbackups:url`

$ heroku pgbackups:restore DATABASE 'https://s3.amazonaws.com/me/items/3H0q/mydb.dump'


Access to heroku postgres DB prompt
heroku pg:psql

Djano South plugin which enables migrations
http://www.mikeball.us/blog/using-south-on-heroku/



Wednesday, July 2, 2014

Git

Start with this..

git init - to initialize an empty repository

git add -A - add all files changed as well as newly created files

git commit -m "description here"

git push origin master

more here

https://www.atlassian.com/git/tutorial/git-basics


Setting up SSH keys
https://help.github.com/articles/generating-ssh-keys