Kafka Streams Topology Testing: Mocked Streams 3.3 is out.

Simple Todo API with Django and OAuth2

Tutorial of a simple Todo REST API with Django, Django REST Framework and OAuth2.

Today, we live in a multi-platform world. We own at least one computer or tablet and a smartphone. Our little gadgets store and retrieve data. What a great opportunity of reaching billions of people with cool applications, even your mother or grandmother can handle. Whatever frontend you intend, on which platform, a way to store data is often neccessary. They are are lots of tutorials on mobile applications on the net, but a simple tutorial on storing data remotely via a REST API is hard to find.

The Hello World of REST APIs

A simple Todo application seems like the Hello World example of client/server communication and storing data. In this article, I want to give a tutorial how to create a simple todo API with SQLite which communicates over JSON and REST. I will use the Django web framework and the Django REST framework. Authentication is important. Here we use the standardized OAuth2 authentication, which is pretty common among top mobile applications and APIs. For the authentication we use the optional library django-oauth2-provider. It is intended to keep the example application as simple as possible, which excludes steps like e.g. version control, test-driven development, deployment, cover special use cases or error handling, respectively. The full source code is available as django-todo. Grab a coffee or tea, get into learning mode, ten steps await to satisfy your curiosity.

1. Virtual Environment and Requirments

First of all, it is a good practice to set up a new Python virtual environment to isolate the requirements from the rest of the system. You can install virtualenv and all the other requirements over PyPI. So make sure the PyPI package installer pip is installed. I assume you installed virtualenv. Create a project directory and create a virtual environment directoy inside it.

$ mkdir /home/jpzk/work/todoapi; cd /home/jpzk/work/todoapi
$ virtualenv venv
$ source env/bin/activate
(venv) $

Now you have immersed in your virtual environment and every requirement you install will be placed inside the venv folder. This is pretty useful for integration tests as well as for deployment.


Subsequently, we install the necessary requirements in specific versions. These are just the latest versions by the time I am writing this. You can store them as a list in the requirements.txt file in the top level of your folder. And install them via pip install -r requirements.txt. Let us move over to the world of Django.

2. Create a Django Project and Application

This is standard use case when starting a Django project. A Django project can manage multiple Django applications. In our case, we start a new project, called todosite, and start one new Django application, named todo.

$ (env) django-admin.py startproject todosite
$ (env) cd todosite
$ (env) django-admin.py startapp todo

3. Adjust Project Settings

Let us start with adjusting the project settings. The settings are defined in the file /todosite/settings.py. First, we have to add the installed apps, the Django apps we will use in this project. By convention, the installed apps are listed in the INSTALLED_APPS constant. We add the mandatory Django application for our application, the Django Rest Framework rest_framework, and the Django OAuth2 provider provider, provider.oauth2. Additionally, we have to list our own Django application todo.


Afterwards, we have to set the settings of the Django REST Framework. We want to use the browseable API, hence we need a session-based authentication next to the OAuth2 we intend to employ. Furthermore, we set the standard serializer class to the simple ModelSerializer, which does not render one-to-many or many-to-many relationships. The permission class is set to IsAdminUser. This implies that admin-rights are required if the definition of rights on a logical endpoint are omitted. Add the following constant to your settings.py file. If you want to compare, see the complete file.


4. Create Models

Now, we switch to the Todo App in the folder todo and create the models we need in the file /todo/models.py. Do not be afraid, we are doing baby steps here. In order to define a Todo model, we need to derive from the Model class and reference the User class of the standard authentication as a foreign key in the owner attribute. We define an owner, a description of the task, a state done and an attribute which stores the last time a model gets updated. For a detailed description for the field types, see the documentation.

from django.db import models 
from django.contrib.auth.models import User

class Todo(models.Model):
    owner = models.ForeignKey(User)
    description = models.CharField(max_length=30)
    done = models.BooleanField()
    updated = models.DateTimeField(auto_now_add=True)

5. Create Serializers

Still in the Todo App folder, we will define Django REST framework serializers in order to validate incoming JSON requests and to automagically create HTML responses of JSON objects in our browseable API. In the file /todo/serializers.py just derive from the model serializers classes in the following way. The registration serializer is used to validate the incoming registration request. The fields attribute projects the attribute of the model to the given tuple. For more information on serializers, see the documentation

from rest_framework import serializers
from todo.models import Todo
from django.contrib.auth.models import User

class UserSerializer(serializers.HyperlinkedModelSerializer):
    class Meta:
        model = User

class RegistrationSerializer(serializers.ModelSerializer):
    class Meta:
        model = User
        fields = ('username', 'password')

class TodoSerializer(serializers.ModelSerializer):
    class Meta:
        model = Todo
        fields = ('id', 'description','done')

6. Adjust URL Dispatcher

Now it is time to set up the URL dispatcher of the Django project in the file /todosite/urls.py. The URL dispatcher sets the URL routes to specific views in Django. A view handles logic and sends HTTP responses. The views will be implemented in step eight and nine. In other web frameworks, this component is often called controller. The URL routes are specified with regular expressions, see the following source code. For a detailed understanding, see the URL dispatcher documentation.

from django.conf.urls import patterns, include, url
from todo import views

urlpatterns = patterns('',
    # Registration of new users
    url(r'^register/$', views.RegistrationView.as_view()),

    # Todos endpoint
    url(r'^todos/$', views.TodosView.as_view()),
    url(r'^todos/(?P<todo_id>[0-9]*)$', views.TodosView.as_view()),

    # API authentication
    url(r'^oauth2/', include('provider.oauth2.urls', namespace='oauth2')),
    url(r'^api-auth/', include('rest_framework.urls',\

We will implement two views, a RegistrationView and a TodosView. The first is for registering new users and the second manages the creation, listing and updating of todos. As you can see in the code, there are two routes for the todos endpoint. The first is a normal routing and the second uses a URL parameter which is passed as a method parameter named todo_id to the methods of the TodosView. For OAuth2 authentication we have to set up special routes.

7. Initialize the Database

Okay, if you are new to Django or REST Framework you might have learned a lot, recapitulate the last steps before proceeding. We are almost done, take a deep breath here comes an easy one. We have to set up the database for storing data. In the default settings a SQLite database with the required schema is automatically created with the following command.

$ (env) python manage.py syncdb

8. Adding the Views: Registration View

It is best practice to write the tests at this moment and proceed with test-driven-development (TDD). As I already mentioned we skip the TDD part to keep the tutorial as simple as possible. If you are interested in how to test this API, I applied TDD and the code for testing is available. To understand it, the REST Framework testing guide might be an interesting read. Back to the Django views. First, we implement the view, which lets unregistered users register. The following belongs to the file /todo/views.py, for the sake of presentation I omitted the module imports. For imports see the complete file.

class RegistrationView(APIView):
    permission_classes = ()

    def post(self, request):
        serializer = RegistrationSerializer(data=request.DATA)

        # Check format and unique constraint
        if not serializer.is_valid():
            return Response(serializer.errors,\
        data = serializer.data
        u = User.objects.create(username=data['username'])

        # Create OAuth2 client
        name = u.username
        client = Client(user=u, name=name, url='' + name,\
                client_id=name, client_secret='', client_type=1)
        return Response(serializer.data, status=status.HTTP_201_CREATED)

We derive the RegistrationView from the APIView from the REST Framework. Everyone, logged in or not, should be able to use this view, therefore we set the permission_classes to an empty tuple. Remember, in the setttings.py we set the default permission to is authenticated. Okay, let us write some code for the POST verb on this endpoint. First, we validate the incoming data with the RegistrationSerializer. Serializers are handy and provide useful error messages, see documentation. Because we set the model attribute of the RegistrationSerializer to the User class, even the unique constraint of a user is respected. After validation we create the user, and create an OAuth2 client with the crendentials of the user.

9. Adding the Views: TodosView

Okay, now we arrived the core part of our Todo application. We are still in the file /todo/views.py. We will define the HTTP verbs GET, POST and PUT. Every user needs to be authenticated in order to access this endpoint. The get method filters all todos by the logged-in user and just responds with the serialized data. In the post method, we validate the incoming data with the TodoSerializer. If the incoming data is valid, we create a Todo object and save it. The method replies with the incoming data and the primary key id.

class TodosView(APIView):
    permission_classes = (IsAuthenticated,) # explicit

    def get(self, request):
        todos = Todo.objects.filter(owner=request.user.id)
        serializer = TodoSerializer(todos, many=True)
        return Response(serializer.data)

    def post(self, request):
        serializer = TodoSerializer(data=request.DATA)
        if not serializer.is_valid():
            return Response(serializer.errors, status=
            data = serializer.data
            owner = request.user
            t = Todo(owner=owner, description=data['description'], done=False)
            request.DATA['id'] = t.pk # return id
            return Response(request.DATA, status=status.HTTP_201_CREATED)

    def put(self, request, todo_id):
        serializer = TodoSerializer(data=request.DATA)
        if not serializer.is_valid():
            return Response(serializer.errors, status=
            data = serializer.data
            desc = data['description']
            done = data['done']
            t = Todo(id=todo_id, owner=request.user, description=desc,\
                     done=done, updated=datetime.now())
            return Response(status=status.HTTP_200_OK)

Additionally, we will implement the put method to modify existing todos. In this case we use URL dispatcher parameters like http://localhost:8000/todos/1. Analogous to the post methods introduced, we validate the incoming data. If it valid, we create a todo with the todo_id given an overwrite the old todo. As you might assume here is a potential for code improvement, consider it as a small homework exercise. Okay, it is finally time to interact with our API :).

10. Interact with the Browsable API and with cURL

Start the development server of your Todo project. By default this starts a HTTP server on port 8000. Make sure to use HTTPS when using this API in production mode, because user credentials are transmitted in plaintext.

$ (env) python manage.py runserver

Because the REST Framework provides a browseable API, feel free to interact with the API through the web browser at http://localhost:8000/, like this example. You can send requests and retrieve responses analogous to the interaction in the following with cURL. Here is a screenshot of how it looks like.

Example of Browseable API

In the following we will use cURL to interact with the API. Make sure you have installed it. Let us start with creating a user and login. Both requests use the application/x-www-form-urlencoded encoding for data. For whatever reason, the OAuth2 provider does not work with application/json encoding. After registration we can get the authentication token using the second request. The authentication token is only valid for a certain time span.

// Register a new user. 
$ curl -X POST -d 'username=jpzk&password=yourguess'\

// Get authentication token.
$ curl -X POST -d

// The response: Authentication token.
{"access_token": "41b59e8238bb418c1fc98cfc6f523dd1a7839a03", "token_type":
"Bearer", "expires_in": 2591999, "scope": "read"}

Okay, lets create a todo and get a list of all todos of the logged-in user. In both requests we have to provide the authentication token we previously retrieved to identify the user. Furthermore, in both requests the application/json encoding of data is used. // Create a todo. $ curl -X POST -H 'Content-Type: application/json' -H 'Authorization: bearer 41b59e8238bb418c1fc98cfc6f523dd1a7839a03' -d '{"description":"write tutorial"}' http://localhost:8000/todos/

// Get a list of all todos of the user.
$ curl -X GET -H 'Content-Type: application/json' 
-H 'Authorization: bearer 00db9a38ea3f86d04dd3eeded9128620f11158eb' 

// The response: The list of todos.
[{"id":1, "description":"write tutorial", "done":"False"}]

// Update a todo.
$ curl -X PUT -H 
-H 'Content-Type: application/json' 
-H 'Authorization: bearer 41b59e8238bb418c1fc98cfc6f523dd1a7839a03' 
-d '{"description":"write tutorial", "done":"True"}'

Final Thoughts

Take a deep breath. You implemented a simple Todo API with Django and Django REST Framework and OAuth2 authentication and I wrote my first tutorial on this blog. I picked the topic because in our interconnected multi-platform world APIs are everwhere and are getting more important due to the rise of mobile devices. I hope your curiosity got satisfied. If you have questions or additional comments, please use the comment section below.

One or two mails a month about the latest technology I'm hacking on.