TAB completion for fabric tasks [on OS X]

Found an excellent bash script on foobar.lu that enables tab completion on the fabric tasks defined in fabfile.py script (in case you don’t know what Fabric is, try Yuji Tomita’s article to get started with it). I just had to modify a little bit to make it work on OS X. I actually wanted to post it in their comments section, but it seems the comments are blocked on that site. So here it goes:

Install bash-completion with Homebrew

brew install bash-completion

You’ll need to add some lines in your ~/.bash_profile file to finish the installation. This will create an etc/bash_completion file under /usr/local directory. Now open the ~/.bash_profile file and add the following lines at the end:

function _fab_complete() {
    local cur
    if [ -f "fabfile.py" ]; then
        cur="${COMP_WORDS[COMP_CWORD]}"
        COMPREPLY=( $(compgen -W "$(fab -F short -l)" -- ${cur}) )
        return 0
    else
        # no fabfile.py found. Don't do anything.        
        return 1
    fi
}
if [ -f /usr/local/etc/bash_completion ] && ! shopt -oq posix; then
    . /usr/local/etc/bash_completion
    complete -o nospace -F _fab_complete fab
fi

Reload the file with source ~/.bash_profile command, navigate to your project directory where your Fabric script is located and try it out yourself.

Learn core Python from Stackoverflow

The other day I needed to brush up my knowledge on Python decorators, for implementing a profiling hook for a Django REST API I’ve been working on. I googled around and landed on a question titled Understanding Python Decorators on Stackoverflow.com. The answer given by the user e-satis is an excellent introduction to decorators in Python, although it has not been selected as the official answer. Go ahead an give it read if you have done some Python coding.

This got me thinking. This guy does not care about whether or not his answer gets selected before others (so that he can bag some reputation). He took his time and explained what decorators are in a well thought-out, step-by-step manner. The end result is an answer worthy of being placed in a beginner level Python book as a chapter on decorators! So there must be other answers as awesome as this one, given by users as knowledgable as e-satis! I set out to find few more great StackOverflow threads on core Python programming, had them bookmarked for future reads, and now sharing them on this blog post. You should read each of them at least once to make your knowledge on core Python more solid.

Understanding Python decorators

As I mentioned above, great introduction to decorators in Python.

What is a meta-class in Python

Another great answer from e-satis (later turned into a community wiki). Explains what meta-classes are in Python and why are they so useful.

How to sort a list of objects in Python, based on an attribute of the objects?

The answer to this question often comes in handy!

Understanding Python super() and init methods

A short yet useful answer explaining what good is super() for. You should also read the article Pythons’s super() considered super! along with this.

What is the relationship between __getattr__ and getattr?

I always wondered; and Kimvais provided the answer! Also checkout Alex’s answer in the same thread.

The Python yield keyword explained

Another gem from e-satis. Introduced the concept of iterators and generators in Python and goes on to explain the use of yield statement.

Single quotes vs. double quotes in Python

The stylistic reasons for choosing double quotes or single quotes when coding in Python. Worth a read.

Python Compilation/Interpretation process

Very detailed answer explaining Python compilation/interpretation process to programmers not coming comp-sci background; something every Pythonist should know.

Python progression path- from apprentice to guru

Last but not the least! A great thread showing what road a Python beginner should take to achieve mastery.

If you know about any other great StackOverflow posts, please do share them on the comment section. I’ll update my list. The posts must be about Python (core Python more specifically).

Getting started with pip/virtualenv

If you find yourself in a situation where you have to work with multiple Django projects on the same system, where each one requires a specific version of Django(and some other libraries), or even specific version of Python(2.5/2.6..), the combination of virtualenv and pip will make your life easier.

virtualenv is tool to create isolated Python environments.

pip is a Python package installer, much like easy_install. It’s complementary with virtualenv; you’ll use pip to install, upgrade, or uninstall packages in isolated virtualenvs.

In this post, I’ll show a simple workflow that will give you the basics on how to use these tools to manage multiple Django projects.

Installation

First, get these tools installed on your machine. If you have easy_install set up, you can install pip by just issuing the following command

sudo easy_intsall pip

Now install virtualenv, using pip itself

sudo pip install virtualenv

Creating virtualenvs

Now that you have them installed, let’s try creating some virtualenvs. It’s good to have a specific location on your machine where you would have all the virtualenvs. I personally like to have them under ~/.virtualenvs directory.

Create the directory

mkdir -p ~/.virtualenvs

Navigate into it

cd ~/.virtualenvs

Create a virtualenv

virtualenv -p python2.5 --no-site-packages projectA

The ‘-p’ option specifies which version of the Python interpreter you want to use for the virtualenv(In case you have multiple in your system). If not specified, the default interpreter will be used.

The ‘–no-site-packages’ option means do NOT inherit any packages from /usr/lib/python2.5/site-packages (or wherever your global site-packages directory is). Use this if you don’t want depend on the global packages and want more isolation.

Finally ‘projectA’ is the name of the virtualenv, named after the project itself(not a rule, just a good convention).

You’ll see the directory projectA created once the command prompt returns. This is the virtualenv, feel free to explore it.

Activating/Deactivating the virtualenv

Once pip have finished downloading and installing the packages, hopefully without any glitches, activate the virtualenv

source ~/virtualenvs/projectA/bin/activate

You should see “(projectA)” in your command prompt.

To deactivate it type ‘deactivate’ in your prompt and hit enter. The “(projectA)” should disappear.

So now you have an isolated and functional environment for your Django project. You should go ahead and create a new virtualenv with more recent versions of Python/Django as your homework.

I hope you’ve got the basics properly. Check out the online docs to learn more advanced usages of these tools.

In the next post, I’ll talk about virtualenvwrapper, a tool that makes managing multiple virtualenvs a little easier.

Django: How to make a variable available in all templates

Sometimes when building a web application with Django, you have a common piece of information that is available at all the templates. Foe example, you may have a dynamically built tree menu appearing in multiple templates.  It’s possible to achieve so by adding the data to the context of each template. But that goes against Django’s policy on code reuse. The right way to do that, is to use template context processors and ReuestContext objects.

The sections on Django official documentation are either too short or under misleading headings. So I’m trying to write this in a paint-by-numbers sort of way.

First, the template

Let’s suppose you have the following silly piece of HTML in each of your templates

Hello World! My name is {{name}}

Next, create a custom context processor

Under the directory of your Django application create a new file. Let’s name it ‘custom_context_processors.py’. This is your custom context processor, that will list a number of methods, each of which will have a HttpRequest object as parameter. Suppose you want to have a variable named ‘domain’ available at all template. Let’s add a method to the processor by that name:

define name(request):
   return {'name': 'Django Guru'}

Install the Context Processor

Open settings.py. Add the following line in the TEMPLATE_CONTEXT_PROCESSORS

'myapp.custom_context_processors.domain,'

Add the RequestContext

In your views.py, import the RequestContext module

from django.templates import RequestContext

Now when you render the template using render_to_response() method, a RequestContext object has to be added as the optional context_instance argument. The RequestContext object takes a HttpRequest object as a parameter

def my_view(request):
   #view code
   return redner_to_response('my_template.html', {'foo:bar'}, \
                              context_instance=RequestContext(request))

The variable {{name}} will now be available in my_template.html, without explicitly adding it in the template context. If you need to make it available in some_other_template.html, all you need to do is to pass the RequestContext object as the third parameter to render_to_reponse().

Python equivalent of PHP’s ip2long()

I’ve been spoiled by PHP! It sometimes makes your job too easy with its large collection of library functions. Python, on the other hand, has a lot of powerful tools for doing low-level stuff. To get something done, which would take a single function call in PHP, you may have to meld together a number of those tools and build a new one.

I was looking for a Python equivalent of PHP’s ip2long() function. which converts an IPv4 address from dotted decimal notation(for example 208.69.34.231) to a 32 bit integer(3494191847). After spending few minutes with Google, I realized that Python has no such function in it’s core or standard library. To achieve this, I had to make use of Python’s socket interface and struct library. Here’s what I did:

from socket import inet_aton
from sturct import unpack

def ip2long(ip_addr):
...ip_packed = inet_aton(ip_addr)
...ip = unpack("!L", ip_packed)[0]
...return ip

Note: Dots(.) in the above code represents indentation. WordPress is eating up the whitespace mysteriously!

The first line inside the ip2long() method, the inet_aton() function, converts the dotted-quad IP address to a 32-bit packed binary format, which is string of four characters in length. To make it an integer, you have to unpack it with the unpack() method, which takes the format as the first argument(which is “!L” in this case, for big-endian unsigned integer) and the packed string as the second. It returns a tuple with  the 32-bit integer as the first element.

To make a long2ip() function you can just reverse engineer the above process with struct.pack and and socket.inet_ntoa.

Deploying Django application on mod_python and Mini Mac

Server  : Apache 2.2
OS      : Mac OS X 10.5.4 
Python  : Version 2.5.1
Django  : Version 1.0.2 final
Machine : Mac Mini

mac finder

Mod_python is a loadable Apache module which embeds the Python interpreter, so Python code can be run in-process by Apache. To deploy and run your Django application in an Apache web server you have to

  1. Download and install mod_python
  2. Load it in Apache
  3. Configure Apache to run the Django app.

In this post I’m going to share my experience on how to do all that on a Mac Mini that runs on OS X 10.5.

The Software & Hardware

Here was my system configuration

  • Server  : Apache 2.2
  • OS      : Mac OS X 10.5.4 
  • Python  : Version 2.5.1
  • Django  : Version 1.0.2 final
  • Machine : Mac Mini running on 2.0 GHz Intel Core 2 Duo processor with 2GB RAM

1. Get the mod_python source

Check out the mod_python source code from the remote SVN repository*

$ svn co https://svn.apache.org/repos/asf/quetzalcoatl/mod_python/trunk/ mod_python_src/

*The reason I’m downloading the source from an SVN trunk and not from the Apache website, is because the source available at the later one isn’t properly patched for Mac Mini’s architecture. In my earlier attempts, when I downloaded and built mod_python from stable releases, Apache was failing to start whenever the mod_python module was being loaded using the LoadModule Directive.

2. Build the source

$ cd mod_python_src

$ ./configure --with-apxs=/usr/sbin/apxs

$ make

$ sudo make install

3. Configure Apache to load mod_python when it starts

Open the Apache configuration file

$ sudo vi /etc/apache2/httpd.conf

Add the following line to the file 

LoadModule python_module libexec/apache2/mod_python.so

Now restart Apache

$ sudo /usr/sbin/apachectl restart

4. Configure Apache for Django 

Let’s suppose you have a Django app callled “mysite”. Copy you the directory to the Apache DocumentRoot

$ cp -r /path/to/mysite /Library/WebServer/Document/

Add the following lines to the httpd.conf file

<Location "/mysite">

    SetHandler python-program

PythonPath "['/Library/WebServer/Documents'] + sys.path"

     PythonHandler django.core.handlers.modpython

     SetEnv DJANGO_SETTINGS_MODULE mysite.settings

PythonAutoReload On

PythonDebug On

</Location>

This’ll make Apache to route any requests under URL “/mysite” to the mod_python handler. It passes the value of DJANGO_SETTINGS_MODULE so mod_python knows which settings to use.

Now restart Apache and go to the site http://localhost/mysite

If it works then congratulations! You’ve just deployed your app on Apache!

Troubleshooting tip(s)

If Apache fails to launch after you’ve installed and loaded mod_python, run it in debugging mode

$ sudo /usr/sbin/httpd -X

Look for any error message printed in the terminal, copy it and google it. You may find a solution online.

You should also check out

Installing Python on Nokia N73 phone

p1000137

Python for S60(PyS60 for short) is a general port of the Python programming language on the S60 software platform of Symbian OS based mobile phones. It is based on Python 2.2.2 and contains many of the standard library modules of the language, including mobile-platform specific modules such as native GUI elements, Bluetooth, SMS messaging etc.

In this article I’ll explain how I installed PyS6o on my Nokia N73 phone, which runs on Symbian 9.1(3rd Edition) operating system and supports S60 software platform.

So here is the plan:

  • Download necessary files
  • Transfer them to the phone
  • Install PyS60
  • Write a test script and run it.

1. Download necessary files

You need to download two python installers (.sis extension) from Sourceforge.net. The first is the PyS60 interpreter and the second one is the Python Script Shell(i.e. user interface). Here are the links for the files that I downloaded:

PythonForS60_1_4_5_3rdEd.sis
PythonScriptShell_1_4_5_3rdEd.SIS

Notice that the second part of the filename represents the version number(1.4.5) and thrid part represents the edition of the OS(in this case 3rd).

2. Transfer them to the phone

Transfer the downloaded files to the phone by USB cable or Bluetooth or whatever medium that you use.

3. Install PyS60

Go to Menu > Tools > File Manager. Browse to the directory where you have placed the installers and run them. After installation go to Menu > Applications and look for the Python logo that appears on the list.

4. Write a test script and run it

You would notice that a directory name ‘Python’ has been created on the phone file system, depending on whether you have installed PyS60 on the phone memory or on the expandable memory. Write a single line script in your computer:

print "Hello World";

Save it as hello.py and transfer it to the Python directory of your phone. Now launch the Python application. Select Run script from Options, choose hello.py from the list and run it by clicking OK. It will print the message on the console. Alternatively you can run the demo scripts provided in the Python directory.

Update: PyS60 1.9.0 has recently been released(thanks to cb22 for informing me). Download it from here.