Katiba-Info Issue: Unicode characters

Email and chat output is unable to display unicode characters as shown below. I am reading an XML file into a cacheable byte array and thereafter wrapping that array in a ByteArrayInputStream which is in turn wrapped in a InputStreamReader with UTF-8 encoding for use in the XSLT transformation. Suggestions are welcome.

Kenya Constitution Lookup via Email, Chat

After several weeks of coding and hacking, I hosted katiba info on google app engine on Sept 1st. The applications is envisioned as a mobile friendly site that will allow articles and clauses in the Kenya constitution via email and chat.

Sept 1st was long after my self imposed deadline to have it online before the promulgation ceremony. The idea had started as a desire to have the constitution online in a way that people could access in a simple way. Katiba.mobi was already online but I wanted to utilize the app engine platform for its scaling capabilities. On an earlier project (Kura Info), I had used python and django and had been impressed by that combination which greatly simplified web application development.

Having decided that the user interface part will be a walkover, I turned to the data where I converted the PDF file to text using pdftotext. The task that faced me then was how to delimit the various sections of the document - chapters, articles, clauses, parts and so on. The articles are numbering is not restarted at the chapter level for the body of the document, so I extracted that portion for further analysis, leaving out the TOC and appendices. The complexity of breaking down the text file was dawning on me so I turned to a compiler construction book for ideas on how to automate the process of building parsers. The book led me to use flex for my first stage where chapter headings, article headings, article numbers and clauses were sorrounded by XML tags.

The next step according to compiler theory would have been to create a grammer, tokenize the file and feed it into bison. At that point, I decided to take a short cut by using regular expressions in python to finish the task of surrounding chapters, part and article sections with new XML tags. With an XML version of the constitution available, my original plan had been to use it to create another XML file in a format suitable for uploading to the datastore. This approach was abandoned due to the additional complexity and time pressure. I decided to use XPATH to mine the desired information. I quickly found out that XPATH support on google app engine (python) was only available as a third party library and decided to switch to Java due to its XML processing facilities.

I did not expect a repeat of the ease I had experienced while using django for the user interface so I went for a user interaction medium that demanded the least in terms of UI development - email. When email started was ready and the application hosted, adding chat was a straightforward process.

Referendum Reflections

The referendum results are long out and the new constitution promulgated. I felt discouraged that Kenyans did not heed the call of the church as communicated by the clergy. Before the referendum, a radio host on Hope FM had asked for what listeners thought would happen after the vote. The text below is what I wrote in.

Am expecting after the referendum:

- That many 'Sauls' will preach the gospel they persecuted,

- That those who plot for the downfall of the church will be used for its promotion,

- That the evil meant against our country will work for good,

- That where the enemy planned defeat there be victory.

- Eric.

In my state of discouragement, I remembered my response above and decided to focus on its prophetic perspective which is based on the biblical accounts of Joseph and Esther. Joseph faced opposition and ultimately rejection from his brothers, yet God made it work out for his good and his family. For Esther, it was the entire community of Jews that were endangered. Her intervention, prompted by Mordecai's counsel, left the Jews stronger and elevated them higher.

In Esther's scenario, there is a gem that is especially shines in the current constitution dispensation. It is written that Mordecai was remembered and honored by the King because he had once uncovered a plot against the kings life. In the same way, I believe God will remember and honor those who voted to save the lives of our unborn children even if their action not appreciated or recognized right away.

Postgres DB Management Server

While trying out Postgresql (EnterpriseDB 8.4) database, I wanted to see what the management server has. Looking through the enterprise DB documentation did not turn up the port number. I used lsof to list the ports on listen mode. This action turned up several ports between 9011 and 9019. At this point, I tried some of them on the browser but none worked possibly because of the local firewall.

After several unsuccessful searches, I navigated through the installation folders and realized that the management server run on jboss. I then made a specific search for Jboss http port configuration as I my experience was with tomcat only. It was a pleasant surprise that Jboss uses tomcat for its web apps. Looking through $EDB_HOME/mgmtsvr/server/default/deploy/jboss-web.deployer/server.xml revealed that the HTTP port is 9011. I added this to the firewall exclusion list and I was ready to explore.

At first sight, the management server does not appear to have many features. This is especially since I have interacted with the Oracle enterprise console. With that ended another step in my journey with Postgresql.

Getting Started with Amanda

Having initially considered Bacula, I chose amanda for use in backup due to it's better organized documentation. Being a newbie, I selected the quick start getting started tutorial as the first step through the online documents.

In that process, I consulted the bsd authentication howto for more information on authentication. I also found the vtape howto very helpful since I did not have a physical tape device. Having gone followed the tutorial and branching into the howtos at the relevant sections, I was able to do a successful test run.

Using another article in the howtos collection, I made further changes to the configuration to allow for dumps to be split across multiple tapes (vtapes in my case). It was then time to follow another howto to duplicate that configuration to the production environment where the virtual tapes needed to be configured for use on a non-Linux partition.

Creating a Pre-printed Report Template

One way to speed up printing receipts on a dot-matrix printer is to print the desired (dynamic) information on a pre-printed template that contains the labels, content placeholders, logos and other static information.

The first thing I did was to adjust the report to display the values only as it has previously been printing all the dynamic and static information. I deleted all the static information and left the bare dynamic info that changes with every print out.

I then printed out the bare report on the target dot matrix printer. In the process, I made adjustments to paper guides on the printer so as to get an optimum placement. I repeated this process several times as the draft print outs revealed changes that needed to be made to the report writer design.

When the bare printout had an optimum placement, I took the printout and scanned it. I then opened the resulting image with Gimp Image Manipulation Program. Using gimp, I added layers which contained the static information e.g. text layers for labels and a transparent layer for content placeholders. I even added a layer with a white color that I would hide/show to see how the report template would look like on printed paper.

All this time, the original scanned layer was acting as a background which helped with the positioning. Thereafter, I hid this original layer (leaving the all white layer as the background) and printed the resulting report template on paper. Now it was time to test it by printing out the bare information report on the preprinted template. After a few minor adjustments and refinements made from observing the data placement on the template, the final version was ready for mass production.

Workaround for uploading data to google app engine dev server

Google app engine's dev server will ask for a google account to upload data as described in the uploading data section of the online docs.
I found this requirement restrictive since I did not have a internet connection at the time. My workaround was to comment out a section of the file that handles upload requests i.e. $PYTHON_LIB/google/appengine/ext/remote_api/handler.py.

The following is the result of running diff handler.py.dist handler.py where the first file is the original file and the second is the modified one.

295,296c295,296
<     if not self.CheckIsAdmin():
<       return
---
>     #if not self.CheckIsAdmin():
>     #  return

The appcfg.py script will still ask for an email and password after the change but any fictitious value will be accepted.