Ins and Outs of Inputs

Yesterday I promised I'll talk about why I hate mobile phones. Of course I didn't mean all of them. Just the ones I have to deal with. Why? Well my mobile phone kind of died few days ago. I have a Nokia N73 and it's really quite good phone even if it's a bit old by today's standards. You control the phone by using "joystick" kind of thing in the upper part of keyboard. I decided to include image so you don't have to look for it :-)

So this joystick stopped working (even slightest touch would be evaluated as pushing it, therefore it was unusable). I didn't have my backup phone with me, but one friend gave me her battered Siemens S55. So what was the problem? Well I have the same sim card for almost 10 years now. Back then only 100 contacts would fit on it. I have almost 300 contacts in my N73. So how do I get all contacts from one phone to the other? Normally I could just send them through bluetooth, but since I couldn't really control my N73 this was out of question. I was barely able to turn on the bluetooth. I thought that I'll use SyncML interface to get vCards from N73 to my computer and then sync them again to the S55. In the end I kind of did, but boy was that an unpleasant experience!

So what exactly happened? I installed OpenSync libraries and tools and using multisyncgui I created sync group with one side being file-sync plugin and other was syncml-obex-client plugin. Configuraion of file-sync plugin was mostly just changing path to directory where I wanted to sync. Final version looked like this:

<?xml version="1.0"?>
<config>
  <directory>
    <path>/tmp/sync</path>
    <objtype>contact</objtype>
    <objformat>vcard30</objformat>
  </directory>
</config>

Configuration for syncml-obex-client appeared to be much more challenging. It appears that Nokia N73 has two quirks:

• It only talks to SyncML client if it says its name is "PC Suite"
  
• It contains a bug that causes it to freeze after certain amount of data if configuration is not correct
  

First of these quirks is mentioned in almost every tutorial on data synchronization in Linux. However the second one caused me to lose quite some time. My Nokia N73 would freeze after synchronizing approximately 220-240 contacts. To continue working I had to restart the whole phone.In the end I found out that I need to set parameter recvLimit to 10000 in order to synchronize everything. Final setting for syncml-obex-client looks like this:

<?xml version="1.0"?>
<config>
  <type>2</type>
  <bluetooth_address>00:1B:33:3A:D1:37</bluetooth_address>
  <!-- Nokia Obex server channel -->
  <bluetooth_channel>13</bluetooth_channel>
  <interface>0</interface>
  <identifier>PC Suite</identifier>
  <version>1</version>
  <wbxml>1</wbxml>
  <username></username>
  <password></password>
  <usestringtable>1</usestringtable>
  <onlyreplace>0</onlyreplace>
  <onlyLocaltime>0</onlyLocaltime>
  <recvLimit>10000</recvLimit>
  <maxObjSize>0</maxObjSize>
  <database>
     <name>Contacts</name>
     <objtype>contact</objtype>
     <objformat>vcard21</objformat>
  </database>  
</config>

So after all that I was able to get vCards from my N73 to my notebook. For every vCard OpenSync created file in directory /tmp/sync. Now came the interesting part. How to get these vCards to Siemens S55?

Simple Google search on Siemens S55 and synchronization in Linux seemed to suggest that tool most suited to do the job was scmxx. This little app is specialized on certain Siemens phones. According to some manuals it was supposed to be able to upload vCards themselves, however I couldn't get it to work as scmxx was complaining about invalid command line arguments.After some testing I found out that it could access and change sim card phone numbers.

Unfortunately for me, my sim card has limit of 100 phone numbers, each with 14 character identifier (name). This meant I needed to convert vCards from N73 to special format that scmxx used. Mentioned format looked something like this:

1,"09116532168","Jones Rob"
2,"09223344567","Moore John"
...

First column being number of slot that will be overwritten by new information, second column is number and third one name of contact (less than 15 characters).

So I fired up vim and started coding conversion script. It didn't take long and I had my contact in the old-new phone. There are a lot of hard-coded things in that script since I don't plan to ever use it again but you can download it from my dropbox. Consider it public domain, and if anyone asks I didn't have anything to do with it :-)

import os
import re

MAX_CONTACTS=100

class PbEntry(object):

    def __init__(self, name, tel, year, month, day):
        self.name = name
        self.tel = tel
        self.year = year
        self.month = month
        self.day = day

def cmp_pb(e1, e2):
    if e1.year > e2.year:
        return -1
    elif e1.year < e2.year:
        return 1
    else:
        if e1.month > e2.month:
            return -1
        elif e1.month < e2.month:
            return 1
        return 0


telRe = re.compile('TEL(;TYPE=\w+)*:([*#+0-9]+)', re.M)
revRe = re.compile('REV:(\d{4})(\d{2})(\d{2}).*', re.M)
nameRe = re.compile('^N:(.*);(.*);;;', re.M)
def get_entry_from_text(text):
    ret = nameRe.search(text)
    surname = None
    name = None
    tel = None
    rev = None
    if ret:
        surname =  ret.group(1)
        name = ret.group(2)

    ret = telRe.search(text)
    if ret:
        tel =  ret.group(len(ret.groups()))
    
    if surname and name:
        fn = "%s %s" % (surname,name)
    elif surname:
        fn = surname
    else:
       fn = name
    
    if fn:
        ret = re.search('(.{0,14}).*', fn)
        fn = ret.group(1)

    
    ret = revRe.search(text)
    year = ret.group(1)
    month = ret.group(2)
    day = ret.group(3)

    return PbEntry(fn, tel, year, month, day)


entries = []

files = os.listdir('/tmp/sync')
for file in files:
    fh = open('/tmp/sync/%s' % file, 'r')
    content = fh.read()
    entry = get_entry_from_text(content)
    entries.append(entry)

entries = sorted(entries, cmp=cmp_pb)

i = 1
for entry in entries:
    print '%d,"%s","%s"' % (i, entry.tel, entry.name)
    i = i + 1
    if i > MAX_CONTACTS:
        break

I had my share of incompatibilities between mobile phones, computers and other devices. Fortunately most of devices being sold today use open communication protocols for sharing of data (and other stuff). Too bad people had to put so much energy into reverse engineering proprietary solutions in the past. Just ranting about this vendor lock-in could be spread on quite a few pages. Imagine having 300+ contacts and calendar information in your phone of brand X. When you are buying your new phone, you would be able to synchronize your data only if you bought new phone also from brand X. Would that affect your decision? It sure would affect mine.

Now I have a choice. After fixing my old N73 I will start looking into new phone. So far HTC Hero looks pretty cool and reviews are not half bad.

So this year's Google Summer of Code is officially over. Today 19:00 UTC was deadline for sending in evaluations for both mentors and students. Therefore I think some kind of summary what was happening and what I was doing is in order.

I was working on implementing neat idea that would allow previously impossible things for Gentoo users. Original name for the idea was "Tree-wide collision checking and provided files database". You can find it on Gentoo wiki still. I later named the project collagen (as in collision generator). Of course implemented system is quite a bit different from original wiki idea. Some things were added, some were removed. If you want to relive how I worked on my project, you can read my weekly reports at gentoo-soc mailing list (I will not repeat them here). Some information was aggregated also on soc.gentooexperimental.org. As final "pencils down" date approached I created final bugreports of features not present in delivered release (and bugs there were present for that matter). Neither missing features, nor present bugs are a real show-stopper, they mostly affect performance. And more importantly I plan to continue my work on this project and perhaps other Gentoo projects. I guess some research what those projects are is in order :-)

Before GSoC I kind of had an idea how open-source projects work since I've participated with some to a degree. However I underestimated a lot of things, and now I would do them differently. But that's a good thing. I kind of like the idea that no project is a failed one as long as you learn something from it. It reminds me of recent Jeff Atwood's post about Microsoft Bob and other disasters of software engineering. To quote him:

The only truly failed project is the one where you didn't learn anything along the way.

I believe I have learned a lot. I believe that if I started collagen now, it would be much better in the end. And the best thing is that I can still do that. I get to continue my project and learn some more. If I learned anything during my work on collagen it's this:

If you develop something in language without strong type checking CREATE THE DAMN UNIT TESTS! It will make you life later on much easier.