Sample WikiTool Creation Tutorial

In this document we will build a very simple (not to say crude) Wiki tool for the Toolserver Framework For Python. The full source is available in CVS. To run it, you need at least Version 0.3.1 of the Toolserver Framework For Python.

We start with normal definitions and imports. These is mostly Toolserver.Utils stuff.

import os

import re

from Toolserver.Tool import registerTool, StandardTool

from Toolserver.Utils import logWarning, logError

from Toolserver.Utils import NotFoundError, ForbiddenError, RedirectError

from Toolserver.Utils import parseQueryFromRequest, parseQueryFromString

Now we need two templates. One is used for rendered Wiki pages, the other is used for editing Wiki pages.

pageTemplate = """

<html>

<head>

<title>%(title)s</title>

</head>

<body>

<p align="right"><a href="%(edit)s">edit</a>

<h1>%(title)s</h1>

<p>

%(body)s

</p>

</body>

</html>

"""

pageEditTemplate = """

<html>

<head>

<title>%(title)s</title>

</head>

<body>

<p align="right"><a href="%(view)s">view</a>

<form action="%(update)s" method="POST">

<input type="hidden" name="name" value="%(name)s">

<p><input name="title" value="%(title)s">

<p><textarea name="text">%(text)s</textarea>

<p align="right"><input name="submit" type="submit" value="Submit">

</form>

</body>

</html>

"""

And last some predefined regular expressions we need. LINK is to recognize links in pages. They are written as [[page title]] and are automatically mapped to pagenames using the regular expression ILLEGAL to recognize illegal characters for page names.

LINK = re.compile(r'\[\[([^\]]*)\]\]')

ILLEGAL = re.compile(r'[^a-zA-Z0-9\-\_\+]+')

Now we will build a very simple and crude data model for the Wiki tool. Pages are textfiles that have a very simple format, much like RFC messages: there are several headers, than one empty line, then paragraphs of text, each separated by an empty line. The needed data model methods make use of the tools standard methods for file storage access: _store, _load and _exists to access pages in the webspace of the toolserver. Pages and their rendered output are stored both in the webspace, so you can access pages, can access their source and can access methods to work with those pages.

class WikiTool(StandardTool):

      # all content is HTML with the tool

      _content_type = 'text/html'

      # this private method finds a page and returns it's link

      # (or it's creation link, if the page doesn't exist)

      def _findPage(self, title):

              name = ILLEGAL.sub(lambda m: '', title)

              if self._exists('%s.txt' % name):

                      return self._getNameAsPath() + name + '.html'

              else:

                      return self._getMethodAsPath('getPage', name=name)

      # this private method rewrites page links

      def _rewriteLinks(self, page):

              for i in range(0, len(page['paras'])):

                      page['paras'][i] = LINK.sub(

                              lambda m: '<a href="%(view)s">%(name)s</a>' % {

                                      'view': self._findPage(m.group(1)),

                                      'name': m.group(1)

                              }, page['paras'][i]

                      )

      # this private method loads a wiki page from a file

      def _loadPage(self, pagename):

              content = self._load('%s.txt' % pagename)

              paragraphs = content.split('\n\n')

              header = paragraphs[0].split('\n')

              del paragraphs[0]

              res = {'paras': paragraphs, 'name':pagename}

              for h in header:

                      (var, val) = h.split(': ', 1)

                      res[var] = val

              return res

      # this private method stores a wiki page to a file

      def _storePage(self, page):

              content = ''

              for h in ('title',):

                      content += '%s: %s\n' % (h, page.get(h,''))

              content += '\n'

              content += '\n\n'.join(page['paras'])

              self._store('%s.txt' % page['name'], content)

      # this private method renders a wiki page

      def _renderPage(self, page):

              self._rewriteLinks(page)

              body = '<p>'.join(page.get('paras',()))

              content = pageTemplate % {

                      'edit': self._getMethodAsPath('getPage', name=page['name']),

                      'title': page['title'],

                      'body': body

              }

              self._store('%s.html' % page['name'],content)

registerTool(WikiTool, 'wiki')

This is really very crude. You can now _renderPage a page, this creates the HTML represenation. You can _storePage and _loadPage pages directly. Internally pages are just dictionaries. You can _findPage a page and get back an URL to either view or edit the page. And you can _rewriteLinks a page - change the [[page title]] links to standard HTML links.

This is just the internal model. You can't do much from the outside, as all methods are marked as private. So next you need to add API methods to work with Wiki pages. We have two API methods, one to updatePage a page and one to getPage a page.

# this method updates an existing page. It can be used via REST

# to update a page. It will redirect to the page editor if used

# via REST.

def updatePage(self, pagename, title, text):

        page = {

                'name': pagename,

                'title': title,

                'paras': text.split('\n\n')

        }

        self._storePage(page)

        return self._async(self._renderPage, page)

# this method returns a page as structured element

# ready for editing. If used via REST it will produce

# a edit form.

def getPage(self, pagename):

        if self._exists('%s.txt' % pagename):

                return self._loadPage(pagename)

        else:

                return {

                        'name': pagename,

                        'title': 'Put your Title here',

                        'paras': []

                }

So now we can make external RPC access to our system and create pages, update pages and get pages from the file storage. This still doesn't do much - but the updatePage already renders the page to HTML in the background (the _async call in the end of the method).

Now we need wrappers to make Browser access to these APIs available. We don't want to write complete new stuff, we want to directly expose the RPC API via REST. This is quite easy with the Toolserver Framework For Python. We start by providing wrappers for the getPage call. These wrappers allow to fetch an edit form for a given page. We only handle GET requests, updates are handled by updatePage.

def getPage_parser_GET(self, request, data):

        form = parseQueryFromRequest(request)

        return ((form['name'][0],), {})

def getPage_generator_GET(self, request, result):

        return pageEditTemplate % {

                'update': self._getMethodAsPath('updatePage'),

                'view': self._getNameAsPath() + result['name'] + '.html',

                'name': result['name'],

                'title': result['title'],

                'text': '\n\n'.join(result['paras'])

        }

This now allows us to access an edit form for a page with an URL of the form http://localhost:4334/wiki/getPage?name=start - but that only gives us the edit form (based on the pageEditTemplate from above). Now we need to provide REST wrappers for the updatePage API, too:

def updatePage_parser(self, request, data):

        if request.command in ('GET', 'HEAD'):

                form = parseQueryFromRequest(request)

        elif request.command in ('POST', 'PUT'):

                form = parseQueryFromString(data)

        else:

                raise NotImplementedError()

        request._myForm = form

        return ((form['name'][0], form['title'][0],form['text'][0]), {})

def updatePage_generator(self, request, result):

        raise RedirectError(self._getMethodAsPath('getPage', name=request._myForm['name'][0]))

These are a bit more complicated, as we want to be able to use GET, POST or PUT to update pages (actually we even can use HEAD in this sample). And since the generator needs to know parameters from the request, we pass the parsed form on to the generator via a request private variable.

Now only one thing is missing: we need to write an index method that dispatches us to either the edit form for the start page or to the start page itself. We don't want to access that method via RPC, so we forbid RPC access:

# this method checks wether the start page exists or doesn't

# exist. If it doesn't exist, the user is redirect to the

# edit page for the start page, otherwise he is redirected to the

# start page itself. This method is only called via REST style API

def index(self, request, data):

        if self._exists('start.txt'):

                raise RedirectError(self._getNameAsPath()+'start.html')

        else:

                raise RedirectError(self._getMethodAsPath('getPage', name='start'))

def index_validate_RPC(self, *args, **kw):

        raise ForbiddenError()

That's all there is to do. Now we have a nice and shiny Wiki running in our Toolserver Framework For Python accessible as http://localhost:4334/wiki/ - cool, isn't it?

Ok, it's a very crude Wiki - for example pages are rendered to HTML when edited. But if you create a new page by referencing one that isn't already there, you will only edit that new page, but not the old page - that one will still link to the edit form and not to the HTML output. New pages are not marked as such, so you might not see wether a page is new or not. There are several more problems with this code - for example there is no history of pages (you might want to hook up RCS to this tool to get history). But it's just a sample to show how to do application development with the Toolserver Framework For Python.