Trouble with Windows patch files and projects on Linux

I was having a rough time patching a project in Debian Linux using a patch file generated in Windows using svn diff. There were many errors of the type Hunk #1 FAILED.

From this forum thread, I got the idea that it might be the windows style line endings that were causing the errors. I proceeded to run dos2unix on my patch file but the errors persisted when running patch -p0 < path/to/patchfile.

On further investigation, it emerged that the project I was patching was also using Windows style line endings. The following command changed all the files that are listed in the patch file to Unix line endings.

cat file.patch | grep ^Index | cut -d: -f2 | xargs dos2unix

New files in the patch are gracefully skipped by dos2unix. After this, the patch succeeded.

getting started with request tracker and gmail

I had used request tracker at my former work place as a user. This was my first time installing it. Ubuntu 12.04 provided a smooth install experience. The slight surprise here is that the main package is not named rt4 but request-tracker4.

The documents that guided me on this are:

• Installing Request Tracker 4 on Debian
  This blog gives a good account of installing request tracker on debian for a using a custom email domain.
• GmailAndExim4 on the debian wiki
  It has a good breakdown of how to setup exim4 to send emails to a smarthost. I adapted the instructions to work with gmail.
• Gmail POP3 with Fetchmail
  The fetchmail configuration given by the first source does not show how to connect fetchmail via SSL. This document shows a configuration that can be used for gmail.

In my setup, an email is sent to a gmail account (mfano@gmail.com) which fetchmail logs on to and delivers the email to the request tracker queue. Once request tracker has processed the email, it sends it out using a local account address (rt4@localhost). Exim4 takes over at this stage and delivers the email to gmail using smtp.

Still trying to figure out how the comment email will be sent via exim4 as it appears that one account will be used to send all emails that match the google dns.

Default Queue Settings
These were provided in the debconf wizard during the rt installation.

Queue settings (Local email account)

/etc/email-addresses
rt4:mfano@gmail.com
rt4@localhost:mfano@gmail.com
rt4@app-server:mfano@gmail.com

/etc/fetchmailrc

poll pop.gmail.com protocol pop3
   username "mfano@gmail.com" password "siri" options ssl
   mda "/usr/bin/rt-mailgate --queue general --action correspond --url http://localhost/rt/"
   no keep
;

/etc/exim4/passwd.client

*.google.com:mfano@gmail.com:siri

Perl script to remove stale PID file

Here's a script for removing a stale PID file. It is my first real world perl script. This also happens to be my first blog this year, following the renaming of this blog from locait.blogspot.com. I had to create this script when apacheds refused to start due to a stale PID file.

#!/usr/bin/perl
# perl script to remove state pid file
# adapted from example at http://search.cpan.org/~cwest/File-Pid-1.01/lib/File/Pid.pm
# it expects the pid file as the sole argument

use File::Pid;
  if ($#ARGV < 0) {
        die "Expects the path to the pid file but received $#ARGV args";
}
  my $pidfile = File::Pid->new({
    file => "$ARGV[0]",
  });

  if (! $pidfile->running ) {
      $pidfile->remove;
  }

Mounting a webdav url with davfs2

I found a very useful article online and my points of departure from the author's directions are summarised as a comment on the article and also quoted below.

Many thanks for the helpful information.
I placed the secrets file in ~/.davfs2/secrets and added an entry to /etc/fstab and it worked well.
Being on a 64bit system, I needed to install the 32 bit version of libneon27 for davfs2 to work.

Adding new vtapes to amanda and troubleshooting

I needed to add new vtapes to amanda after successfully adding new space via LVM. Using a one line script included in the vtape how-to on the amanda wiki, I created the additional tape directories - slot[m] to slot[n].

After this, I changed the configuration file's tapecycle parameter to reflect the advice by martineau in a zmanda forum post - that tapecycle should be at least runspercycle * runtapes.

On executing amcheck on one of the amanda configurations, I ran into an error - WARNING taper changer error: slot : Slot  is already in use by driveN. This was resolved by deleting the driveN directory as suggested by dustin in another zmanda forum post.

Earlier, an all estimate timed out error had been resolved by adding the directive estimate calcsize to the configuration's dumptype as suggested by paddy in a zmanda forum post.

Screen Capture/Recording with ffmpeg

My attempt to record a video tutorial was failing with the error 'x11grab AVParameters don't have video size and/or rate. Use -s and -r'. This was in response to typing in the following command as based on an example in the ffmpeg man page.

ffmpeg -f x11grab -s 1280x1024 -i :0.0 -r 25 -sameq /tmp/test.mpg

After several failed tries, I determined that the error could be coming from the -r option. I moved it to immediately after the -s option so that both -s and -r options followed the x11grab specification. This time, it worked great. The final command is shown below.

ffmpeg -f x11grab -r 25 -s 1280x1024 -i :0.0 -sameq /tmp/test.mpg

Setting up the first LVM partition

Most of the LVM setups I have created have been done by the OS installer. I wanted to add more disk space to a server that did not have any LVM partitions setup. Below are the steps that I took:

• added the hard disk (it was a virtual hard disk in this case)
• created a partition and assigned it the lvm partition type id using fdisk
  
• created the physical volume
  pvcreate /dev/sdb1
• created a volume group called system with the earlier created physical volume:
  vgcreate system /dev/sdb1
• created a logical volume to take up all the free space
  lvcreate -n storage -l 100%FREE system
• created an ext3 file system on the logical volume
  mkfs.ext3 /dev/system/storage
• Made an entry in /etc/fstab
  /dev/system/storage /storage ext3 defaults 1 2
  

Creating a patch file with Eclipse 3.5 Git plugin

This process assumes the changes to the file to create a patch from have been committed. The process may not be very different for working changes yet to be committed.

• Display the Git repository view if it is not visible:
  Window > Show view > Other ... > Git > Git Repositories.
• Navigate to the desired file. This can be done quickly by turning on the link with editor behavior of this view if the file is open.
• Right click on the file in the Git repository view and select 'Show in History'.
• In the history view, right click on the commit that has the desired changes and select 'Create Patch...' then provide a destination for the file in the subsequent dialogs.

Using OJB Xdoclet in Eclipse

First of all, I copied the example Ant build file on the ojb xdoclet online reference (also available with the source distribution). Several attempts at running the build failed since ant produced an error to the effect that ojbdoclet was not a recognized sub-element of taskdef.

I therefore decided to comment out the outer taskdef element and left objdoclet as the a sub element of target. In Window > Preferences > Ant > Runtime > classpath, I added the xdoclet-ojb-module jar under Global Entries. I then clicked on the Tasks tab, and clicked on add new. In the resulting dialog, I gave the name of the task as ojbdoclet, selected the jar I had just added then selected the class file implementing the ant task - OjbDocletTask.


I then brought up the run dialog by clicking on the build file and selecting Run as > Ant build ... on the context menu. In the classpath tab, I added all the jars in the ojb binary distribution lib folder using the button add labelled external jars to the user entries section.

The build file, which also shows a task to delete the schema file during a build, is shown below:

<?xml version="1.0" encoding="UTF-8"?>

<project name="martinlaw-build" default="ojb-xdoclet">
<property name="destdir" value="./repository"/>
<property name="schemaFile" value="schema/schema.xml"/>

<target name="ojb-xdoclet" depends="delete-old-schema">
<!--
<taskdef name="ojbdoclet"
classname="xdoclet.modules.ojb.OjbDocletTask"
> --><!-- classpathref="build-classpath" -->
<!-- defined in eclipse ant > global entries > add jar > define task, add supporting jars to
build classpath under external tools configuration -->
<ojbdoclet destdir="${destdir}">
<fileset dir="../../main/java/edu/su/cv/ojb">
<!-- restrict to files with xdoclet markup only -->
<filename name="CourseTraining.java"/>
</fileset>
<ojbrepository destinationFile="repository_user.xml"/>
<torqueschema databaseName="placement_bundled" destinationFile="${schemaFile}" dtdUrl="database.dtd"/>
</ojbdoclet>
<!-- </taskdef> -->
</target>

<target name="delete-old-schema">
<delete>
<fileset dir="${destdir}">
<filename name="${schemaFile}" />
</fileset>
</delete>
</target>

</project>

Using yum behind a proxy

I was recently attempting to install a software on centos 5 and ran into an error with the yum install command as shown below:

Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=5&arch=x86_64&repo=os error was [Errno 4] IOError: <urlopen error (111, 'Connection refused')>

Setting the http_proxy did not help matters. I found documentation on forums that suggested adding the line proxy_http=http://proxy:port in /etc/yum.conf. That still did not help. I finally came across a post on centos forums that suggested that the setting to be made in /etc/yum.conf is actually proxy=... and not http_proxy=...

Image Locations for Pentaho Reports 3.0

My experience with pentaho's report designer is that at design time, images used in the report can be at any arbitrary location. On publishing them to the XML format, each image used in the report design (.report file) is placed in the same directory as the output file but with a new name in the format - Report_staticImage[N].[ext]. N is a number from zero depending on the number of images while ext is the original file extension e.g png.

This behaviour is bound to cause trouble for someone using different locations for the development and production environment. Ideally, the production environment should be setup to reflect the local or vice versa but sometimes this is not possible. To get round this, the production path should be created on the development environment and the desired solution folder linked to the repository location.

For example, let the production path be /path/to/prod/pentaho-solutions/solution1. In this case, the solution1 is the directory containing .xaction files and report templates (.xml). Let the development path be /path/to/dev/pentaho-solutions. A link should be created from the duplicated/replicated production path to the actual repository location on the development environment.

This means that the report designer and pentaho design studio on development will manipulate the files in /path/to/prod/pentaho-solutions/solution1, resulting in paths that will match the production environment on deployment. The local pentaho server, which in my case knows the repository location as /path/to/dev/pentaho-solutions, can access the solution directory as a link - /path/to/dev/pentaho-solutions/soft-link-to-solution1.

Katiba-Info Issues

Please report your issue (bug, error, feature request) as a comment on this blog post and I will endevour to create a separate blog post where a more detailed discussion can be held and updates posted. You may want to browse through the list of issues first before adding a comment. Thank you for the feedback.

Katiba-Info Issue: Text Wrapping

Text wrapping is not handled gracefully when the chat/email window is narrow. The solution for now is to resize the chat window so that the embedded line breaks can be moved to the end of the line. I am working on a removing the embedded line breaks and white space and the direction I am leaning to is using a regular expression. Suggestions are welcome.

Narrow chat window

Wide chat window

Katiba-Info Issue: Unicode characters

Email and chat output is unable to display unicode characters as shown below. I am reading an XML file into a cacheable byte array and thereafter wrapping that array in a ByteArrayInputStream which is in turn wrapped in a InputStreamReader with UTF-8 encoding for use in the XSLT transformation. Suggestions are welcome.

Kenya Constitution Lookup via Email, Chat

After several weeks of coding and hacking, I hosted katiba info on google app engine on Sept 1st. The applications is envisioned as a mobile friendly site that will allow articles and clauses in the Kenya constitution via email and chat.

Sept 1st was long after my self imposed deadline to have it online before the promulgation ceremony. The idea had started as a desire to have the constitution online in a way that people could access in a simple way. Katiba.mobi was already online but I wanted to utilize the app engine platform for its scaling capabilities. On an earlier project (Kura Info), I had used python and django and had been impressed by that combination which greatly simplified web application development.

Having decided that the user interface part will be a walkover, I turned to the data where I converted the PDF file to text using pdftotext. The task that faced me then was how to delimit the various sections of the document - chapters, articles, clauses, parts and so on. The articles are numbering is not restarted at the chapter level for the body of the document, so I extracted that portion for further analysis, leaving out the TOC and appendices. The complexity of breaking down the text file was dawning on me so I turned to a compiler construction book for ideas on how to automate the process of building parsers. The book led me to use flex for my first stage where chapter headings, article headings, article numbers and clauses were sorrounded by XML tags.

The next step according to compiler theory would have been to create a grammer, tokenize the file and feed it into bison. At that point, I decided to take a short cut by using regular expressions in python to finish the task of surrounding chapters, part and article sections with new XML tags. With an XML version of the constitution available, my original plan had been to use it to create another XML file in a format suitable for uploading to the datastore. This approach was abandoned due to the additional complexity and time pressure. I decided to use XPATH to mine the desired information. I quickly found out that XPATH support on google app engine (python) was only available as a third party library and decided to switch to Java due to its XML processing facilities.

I did not expect a repeat of the ease I had experienced while using django for the user interface so I went for a user interaction medium that demanded the least in terms of UI development - email. When email started was ready and the application hosted, adding chat was a straightforward process.

Referendum Reflections

The referendum results are long out and the new constitution promulgated. I felt discouraged that Kenyans did not heed the call of the church as communicated by the clergy. Before the referendum, a radio host on Hope FM had asked for what listeners thought would happen after the vote. The text below is what I wrote in.

Am expecting after the referendum:

- That many 'Sauls' will preach the gospel they persecuted,

- That those who plot for the downfall of the church will be used for its promotion,

- That the evil meant against our country will work for good,

- That where the enemy planned defeat there be victory.

- Eric.

In my state of discouragement, I remembered my response above and decided to focus on its prophetic perspective which is based on the biblical accounts of Joseph and Esther. Joseph faced opposition and ultimately rejection from his brothers, yet God made it work out for his good and his family. For Esther, it was the entire community of Jews that were endangered. Her intervention, prompted by Mordecai's counsel, left the Jews stronger and elevated them higher.

In Esther's scenario, there is a gem that is especially shines in the current constitution dispensation. It is written that Mordecai was remembered and honored by the King because he had once uncovered a plot against the kings life. In the same way, I believe God will remember and honor those who voted to save the lives of our unborn children even if their action not appreciated or recognized right away.

Postgres DB Management Server

While trying out Postgresql (EnterpriseDB 8.4) database, I wanted to see what the management server has. Looking through the enterprise DB documentation did not turn up the port number. I used lsof to list the ports on listen mode. This action turned up several ports between 9011 and 9019. At this point, I tried some of them on the browser but none worked possibly because of the local firewall.

After several unsuccessful searches, I navigated through the installation folders and realized that the management server run on jboss. I then made a specific search for Jboss http port configuration as I my experience was with tomcat only. It was a pleasant surprise that Jboss uses tomcat for its web apps. Looking through $EDB_HOME/mgmtsvr/server/default/deploy/jboss-web.deployer/server.xml revealed that the HTTP port is 9011. I added this to the firewall exclusion list and I was ready to explore.

At first sight, the management server does not appear to have many features. This is especially since I have interacted with the Oracle enterprise console. With that ended another step in my journey with Postgresql.

Getting Started with Amanda

Having initially considered Bacula, I chose amanda for use in backup due to it's better organized documentation. Being a newbie, I selected the quick start getting started tutorial as the first step through the online documents.

In that process, I consulted the bsd authentication howto for more information on authentication. I also found the vtape howto very helpful since I did not have a physical tape device. Having gone followed the tutorial and branching into the howtos at the relevant sections, I was able to do a successful test run.

Using another article in the howtos collection, I made further changes to the configuration to allow for dumps to be split across multiple tapes (vtapes in my case). It was then time to follow another howto to duplicate that configuration to the production environment where the virtual tapes needed to be configured for use on a non-Linux partition.

Creating a Pre-printed Report Template

One way to speed up printing receipts on a dot-matrix printer is to print the desired (dynamic) information on a pre-printed template that contains the labels, content placeholders, logos and other static information.

The first thing I did was to adjust the report to display the values only as it has previously been printing all the dynamic and static information. I deleted all the static information and left the bare dynamic info that changes with every print out.

I then printed out the bare report on the target dot matrix printer. In the process, I made adjustments to paper guides on the printer so as to get an optimum placement. I repeated this process several times as the draft print outs revealed changes that needed to be made to the report writer design.

When the bare printout had an optimum placement, I took the printout and scanned it. I then opened the resulting image with Gimp Image Manipulation Program. Using gimp, I added layers which contained the static information e.g. text layers for labels and a transparent layer for content placeholders. I even added a layer with a white color that I would hide/show to see how the report template would look like on printed paper.

All this time, the original scanned layer was acting as a background which helped with the positioning. Thereafter, I hid this original layer (leaving the all white layer as the background) and printed the resulting report template on paper. Now it was time to test it by printing out the bare information report on the preprinted template. After a few minor adjustments and refinements made from observing the data placement on the template, the final version was ready for mass production.

Workaround for uploading data to google app engine dev server

Google app engine's dev server will ask for a google account to upload data as described in the uploading data section of the online docs.
I found this requirement restrictive since I did not have a internet connection at the time. My workaround was to comment out a section of the file that handles upload requests i.e. $PYTHON_LIB/google/appengine/ext/remote_api/handler.py.

The following is the result of running diff handler.py.dist handler.py where the first file is the original file and the second is the modified one.

295,296c295,296
<     if not self.CheckIsAdmin():
<       return
---
>     #if not self.CheckIsAdmin():
>     #  return

The appcfg.py script will still ask for an email and password after the change but any fictitious value will be accepted.