:::: MENU ::::

Pure Data workshop with MFS at Madlab

On Tuesday I attended a workshop on Pure Data (PD) hosted by Manchester Free Software (MFS) at Madlab hackerspace.

By the end of the evening, most attendees had used purely graphical drag-and-drop tools to create mouse controlled synthesised instruments. The room was filled with cacophony of 50’s style beeps and twangs. Clanger fans were well catered for.

I learned that PD is used as a back end for many games and audio applications, including the massively successful Spore. I saw great potential for the use of PD by performance artists and interactive exhibitions.

A variety of people took part, and I was glad to see a comparatively high percentage of women getting involved.

Pure Data workshops are held in Manchester each week.

Building an instrument at the Pure Data workshop

Filling in the bank’s blanks with regular expressions

I’ve recently had to do a lot of work on a set of data relating to my bank account transactions, which required a great deal of text manipulation, and working with several regular expressions. My bank doesn’t believe that giving their customers access to digital copies of their account and transaction history is important, and they only make available images (stored in PDFs) of past statements which have been posted. Because of this, I had to use Optical Character Recognition software to extract text from the images, fix by hand all the errors in the resulting output, and then manually structure the data into columns and rows by using regular expressions (as my OCR software didn’t detect them). To make matters worse, the images provided by my bank had a large text watermark on each one, written diagonally across the page stating “duplicate”. All contents of the spreadsheet which came into contact with this text was unreadable during OCR, and had to fixed by hand.

Once the data was successfully extracted and structured, I still had a problem however – my bank chooses to list each date only once, and so most of the listed transactions for my account had no date associated with them. The date is critical for the data to be useful to me, and I couldn’t sort or further analyse the accounts without knowing the date that each transaction had taken place. After some thought, I decided use a regular expression to fix the problem.


A spreadsheet with thousands transactions, one per row, where the date of each transaction is listed only once, with all following dates of that transaction missing a date (transactions were listed in date order). Before further processing the data (sorting, analysing) I needed to ensure that each transaction had its own date listed in the correct column.


By using regular expressions in KDE’s Kate text editor, I was able to quickly associate all rows with the correct date. I searched for empty cells which should have housed a date, and for each one took the date from the transaction above. I repeated this until each row had its own date.

The regex that I used (note that Kate wont accept pattern delimiters, switches, or greediness operators):

find: n([0-9]|[0-9][0-9]) (jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec);(.*)n;

replace: n1 2;3n1 2;

The pattern finds all dates in the correct format and which begin the row, are contained within a cell (I used semicolon ; cell delimiters to avoid confusion with decimals and thousands in this CSV), and are followed by a row which does not contain a transaction date (ie the first cell is empty, and the newline starts with a cell delimiter ; ).

Each time the pattern is found, the matching text is replaced with the same line and an altered version of the first cell of the following line. The replace pattern inserts a duplicate of the date found in the first line into the first cell of the following line, thereby copying the date from one to the next.

Having entered these patters into Kate’s find and replace fields I clicked on “replace all” several times, until no more matches were found, meaning that all transactions now had a date associated with them, listed in the correct table column at the start of each new line.


My spreadsheet now contains a complete dataset, with each transaction correctly dated. I can sort the data as I wish (by price, incoming or outgoing, location, business, etc.) and easily manipulate it in preparation for submitting my tax return.

GSM: global social menace?

Last night I heard Ali Gündüz speak about Free Software GSM. During the talk, and in the following discussion, many interesting points were made:

  • The GSM standard is controlled by only four or five international corporations
  • The standard is very complicated for even experts and seasoned developers to grasp
  • Mobile phone manufacturers have no access to the source code of the GSM modems that they use in their devices – they don’t know how they work
  • Many phones have two processors – one dedicated solely to the GSM modem, meaning that even Free Software and open hardware phones do not have access to the way that GSM is used (even Openmoko phones)
  • GSM is incapable of meaningfully encrypting your calls, and its common practice for carriers to allow no encryption at all (in some countries encrypting calls is actually illegal)
  • With only a few thousand euros, and the motive to do so, it is possible to eavesdrop on any GSM based conversation
  • Several countries’ law enforcement agencies use GSM eavesdropping without warrants
  • Network licenses issued by governments are required to set up your own GSM network, are prohibitively expensive, and often already all sold to large carriers.
  • Network licenses are are throwback to early 20th century radio and are unnecessarily applied to protocols like GSM – Internet like packet based communication methods could do away with the need for costly government controlled broadcasting
  • Because GSM is so tightly controlled and little understood, we can only guess at its privacy problems. Location may be recordable by third parties, chips may always be on and tracking location and call information, law enforcement may be able to gather data without permission
  • Free Software GSM systems are used extensively on cruise ships operating in international waters
  • GSM is very insecure and vulnerable to trivial attacks – it trusts members of the network to identify themselves correctly and obey guidelines and restrictions, making it easy to bring down a GSM network, lie about who and what your device and network is, and intercept and re-route calls and data meant for somebody else

What will be the outcome of 68% of all human beings relying on a communication system that is insecure, unconfidential and closed to inspection and understanding? GSM looks like a disaster waiting to happen; watch this space!

malignant pocket dwellers abound

Ali speaking about GSM at the Berlin Fellowship meeting

Open Respect: about time

I recently became aware of the new initiative from Jono Bacon called “open respect”. As you may know, this campaign and accompanying website aims to improve the standard of communications between members of the Free Software community, particularly where it takes place in informal environments, such as online forums.

Although this initiative resulted from some potentially just community criticism of Canonical’s recent actions relating to Gnome, I’d like to stress the importance of this project’s message.

The importance of maintaining a fair and civil tone in online FS exchanges may seem obvious to many. However, I see this issue as more than merely lipservice or sour grapes. In fact, I consider the tone and spirit of a community and its communication to be critical to the continued growth of Free Software’s user base.

Around the end of July a crisis developed at the British Computer Society (a chartered institute with headquarters in London’s Covent Garden). The BCS’ published an article by the director of a British FS company which made several contraversial and arguably unsubstantiated claims regarding the relartive insecurirty of GNU/Linux in comparison to proprietary operating systems. Bedlam ensued in the form of hundreds of comments (several streching to hundreds of words) submitted to the article, and tens of others on FSFE’s UK mailing list. Multiple complaints were filed with BCS.

The tone of responses turned personal, expletive, and enraged by comment number two. Diatribes were exchanged. And even on FSFE’s list, verbal attacks came thick and fast, and were at times just as unsubstantiated as the article which they rebuked. Some contributions sounded positively rabid. The venom was palpable.

For me, the fact that the BCS had been foolish enough to publish the poorly thought out piece was not particularly newsworthy. Biased stories concerning Linux security are published every day of the year, by more reputable agencies than BCS. The disproportionate reaction of the British Free Software community was, however, unwelcome news. When I voiced my concerns about the escalation and degeneration of debate to those concerned, I found myself in an apparent minority of one.

“I don’t think that this kind of reaction does Free Software in the UK any favours…from reading these comments many people would think that Free Software advocates are extremists” – from my post to FSFE’s UK list.

Managing the atmosphere of online communities takes work. Creating an environment where newcomers feel confident when asking questions, and where knowledgeable members stay around long enough to correct and inform those less experienced requires skill and patience. When its done correctly however, a productive and resilient community can develp, and provide the necessary catalysis for knowledge and development to thrive. Here I would point to two examples of groups that have achieved this balance: the PCLinuxOS community forums, and the Maemo.org community website. Both are routinely cited as their respective project’s ‘killer feature’, and both have successfully retained users and visitors over a period of several years.

For Windows users coming to a GNU/Linux desktop for the first time there is a huge amount to learn. Much of this information isn’t written down, and that which is takes skill to find for those who are uninitiated. Whether or not these details get communicated clearly and quickly, with patience and understanding, can be a deciding factor in whether an intrepid Free OS adventurer becomes a habitual user, and potential advocate. It is therefore of vital importance that users are routinely treated with respect. If this is overlooked then new users will never graduate from asking irritating obvious questions, and existing users will become disillusioned and cease to provide the necessary framework of support that makes a voluntary community.

Respect, tolerance and dignity are necessary components of effective communication and group cohesion in a medium which doesn’t allow for tempering body-language or verbal inflection. For these reasons I warmly welcome the Open Respect initiative. This inter-community effort to address such a critical issue is long overdue.

FSFE in Sweden at Free Society Conference

Last weekend Matthias (Fellowship Co-ordinator) and I joined other FSFE fellows and flew to Gothenburg for FSCONS (Free Society Conference and Nordic Summit). We arrived on Friday afternoon and went straight to the venue at Gothenburg IT University.


Arrival in Gothenburg on Friday

FSFE hosted a track this year called Divide and Reconquer, which focussed on the problem of the trend towards centralised non-free Internet services, and possible solutions.

All of our five talks went well, each generating extensive discussion in the question and answer sessions. I was too busy hosting the talks and fielding questions to photograph each talk, but here are a few that I managed to snap.

Benjamin Bayart talking about the structure of the Net

Benjamin Bayart talking about the structure of the Net

Michael Christen Talking about YaCy

Michael Christen Talking about YaCy

Matt Lee talking about GNU FM and GNU social

Matt Lee talking about GNU FM and GNU social

As it turned out, I was responsible for hosting all talks in our conference room, for both days. Matthias helped out and hosted two out of fourteen. This gave me a chance to try my salesmanship at our booth, which was located on the fourth floor.

The FSFE booth

The FSFE booth

As I had hoped, the conference provided a unique forum for debate and discussion. Our talk “Socially responsible social networks” was particularly intense. Whilst Michael Chisari from Applseed Project discussed possibilities for implementing browser level encryption in his software, I was pleased to be able to introduce Werner Koch; founder and lead developer of GNU Privacy Guard (GPG). Werner stood up and commented on the merits of using GPG with appleseed.

Jon Philips from Status.net was also present in the front row and answered some questions relating to privacy and standards of the emerging Free Software social networking ecosphere. Two hours after this talk ended, I noticed that heated discussion continued in a group of about 15 outside the room where it had taken place.

Another highlight for me was witnessing Michael Christen demonstrating the seemingly endless features of YaCy, the complete Free Software search engine. I was literally on the edge of my seat, barely able to believe my eyes, as Michael demonstraed YaCy’s capabilities and optimisations. At one point a member of the audience asked “this is kind of a strange idea, but would it eventually be possible for me to use YaCy to replace grep and use it to search my desktop?”. Michael silently turned around, and in ten seconds had demonstrated how this functionality was already alive and well within the latest stable release. Audience enthusiasm was clear from the first question of the Q&A: “why isn’t this in the Debian repositories already?!”.

I received new, very positive, feedback on the upcoming replacement fsfe.org website design, and discovered FSFE’s Google Adwords account with the aid of Finnish Fellowship Coordinator Otto Kekäläinen.

Many old friends were present at the conference, and the numerous social events provided opportunity to update each other on subjects from artificial intelligence, to reverse proxies, to Swedish leftist politics, to new UK hackerspaces. On Sunday night several of us ended up in Gothenburg’s unique GNU/Linux shop and cafe Gnutiken, where we stayed until the early hours, benefiting from the company of some undrinkable Swedish liqueur.



After stopping for Swedish specialities at a supermarket in Nordstan, we headed home on a flight to Berlin. I took this as we approached Central Station for the last time.

Central Station, on the route home

Central Station, on the route home

Progress of new fsfe.org website design

At the FSFE web sprint on the weekend before last I focused on implementing a new design for fsfe.org. The design will provide a fresh new appearance for all fsfe.org pages (it wont affect other FSFE sites like fellowship or planet; that will come later), and is the result of more than a year of discussion and planning.

Since I began working full time at FSFE’s Berlin office three months ago, I have been working towards getting a fresh look for the site, and replacing the current design which has desperately needed attention since before I first joined the organisation in 2009.

In September and early October I created several detailed vector based design proposals, based upon Andreas Tolf Tolfsen’s inspired ‘ribbon’ themed mock-ups. New site wide menus, a theme-integrated identi.ca feed, and interactive rotation-animated campaign banners were some of the innovations that I showcased in my submissions.

Together, Andreas and I worked out solutions to usability problems arising from new site features, and responded to reactions from community members. Aided by astute and encouraging feedback from Reinhard Muller, I worked feverishly in the run up to the web sprint to create a design that was as progressive as possible, whilst remaining sufficiently conservative to maintain consensus amongst web team members.

Whilst other team members worked on fixing existing bugs with the site and the new build system (which handles generation of our thousands of translations, amongst other things), I worked on turning the new design into the code of fsfe.org for the first time. The schedule, which I published two days before the sprint, stated the implementation of the new design as the number one priority.

Although I had hoped to have the design implemented and approved for launch by the end of the week following the sprint, it turned out to be more work than I had anticipated. I had also not expected to be the only one working on this task, which, aside from a few appreciated commits made by Andreas, turned out to be the case. As a result, nearly two weeks after the sprint fsfe.org remains its old ugly self. But not for long.

Approximately 85% of the front page, 95% of generic content pages, and 70% of specialised pages (such as ‘about’, ‘contribute’, ‘shop’ etc.) are complete. Since the sprint I have been squeezing in a few rushed minutes here and there between other tasks, and the site is finally nearing completion.

Once my implementation is ready on the ‘design’ branch of the web SVN. I’ll send it to the FSFE team mailing-list for approval, which I trust it shall speedily receive. Then will come the task of merging it with the current ‘trunk’ version of the website, at which point all my changes and improvements will, finally, be publicly visible.

Web sprint begins

This morning the FSFE web sprint began. Last night we met and went to dinner at a very traditional (and remarkably cheap) Turkish restaurant, where we sat cross legged in the back room on patterned cushions and carpets, and ate a variety of stuffed flat breads, vine leaves, white cheese and salad. The only item not to my taste was a popular Turkish yoghurt drink; its salty, milk taste wasn’t to my liking.

After a meeting this morning, and a recap of the evolution of the FSFE website through the ages (since 2000) up to this point, work got under way. The room is now abuzz with merry typing.

I’m on freenode’s #fsfe-web channel, and signed in to jabber (samtuke@jabber.fsfe.org) if you want to talk web or contribute 🙂

Web Fridays

As part of an increased focus on improving FSFE’s websites, each Friday afternoon from this point on will be dedicated to working through issues and improvements to these services.

FSFE’s Berlin office staff, in addition to key members of the web team, will be available on the fellowship jabber channel for live discussion about web issues.

Web Fridays are a great opportunity to work along side other FSFE staff and fellows, and are a good time to learn about how to contribute code and improvements to our websites.

To get involved:

join fellowship@conference.jabber.fsfe.org (fellowship chat channel)

join https://mail.fsfeurope.org/mailman/listinfo/web (web team mailing list)

register at https://trac.fsfe.org/fsfe-web (technical issue tracker)

PDF Readers: Getting judgemental

During FSFE’s recent campaign to stop governments advertising proprietary PDF readers, a common question was raised by the public in emails and comments. “Can Free Software PDF readers match up to Adobe Acrobat Reader?” they ask. The implication of this question is that promoting alternatives to Adobe, whether they be Free Software (FS) or not, is only reasonable if they are ‘as good as’ (e.g. file compatible and feature feature identical with) Acrobat Reader.

This view undermines the PDF standard however, and falsely defines what PDF is in terms Adobe’s implementation.

Although PDF is an ISO standard, how PDF is used is not standardised. Thanks to non-conforming proprietary PDF editing software, the Internet is awash with PDF files which do not conform to any version of the ISO PDF standard. When FS PDF readers cannot open these non-standard files, they are perceived as inferior.

This is particularly so in the case of PDF documents supplied by government organisations. Citizens are understandably frustrated when they cannot properly view important information from their state, submit tax, or registration information using the PDF forms that they have been provided with. If switching to Adobe’s reader solves their problems, then FS readers are blamed. Nobody asks about whether the file is valid PDF.

So how can FSFE encourage wider use of FS readers in good faith when the expectation is that PDF documents will work more reliably with Acrobat Reader?

The acronym PDF means something specific. It is a file format, and it is amongst a tiny number that has been formally standardised. PDF means the PDF standard. When companies release PDF software that intentionally generates files that do not conform to that standard, they are distancing themselves from PDF. This is exactly what Adobe is doing each time it releases a new version of Acrobat with extended functionality not supported by the PDF standard.

Corrupting standards is a long established practice, and popular amongst big software players. It is a shame however when governments unintentionally support the corruption of standards by providing documents with a .pdf extension which do not conform to PDF standards. When a state employee creates a new file in Acrobat Pro, no warning messages will appear to alert them to the fact that they are about to break the internationally agreed definition of PDF.

When a user downloads such a file, there is no immediate way of telling whether it conforms to standards either. Many naturally assume that because the file is .pdf, it is a valid document in PDF format, and that any reader that does not open it correctly must therefore not be a good PDF reader.

A far more accurate way to assess the success or failure of a PDF reader however would be based on the extent its implementation of PDF standards. It isn’t reasonable to judge an application based upon how well it works with something that it wasn’t designed to accommodate – namely a non-standards compliant PDF file.

Even Adobe’s reader fails when measured by such unfair criteria – Acrobat can’t open non-compliant files any better than FS readers can when they don’t have the advantage of having generated them themselves. In such cases the company ironically becomes the victim of their own game: on more than one occasion I’ve had to assist a friend in opening a PDF document using Free Software because Acrobat was unable to do so.

What is less obvious is that we should take a similar approach to PDF files. When FS readers are criticised because of incompatibility with non-compliant PDF documents, the fault of the file should be highlighted, both to the user, and to the organisation from which the publication originated.

Therefore the question should not be “Can Free Software PDF readers match up to Adobe Acrobat Reader?” but rather “do any Free Software PDF readers fully implement ISO PDF standards?”, and in cases of specific documents: “does this file conform to ISO PDF standards? If not, why not?”.

Adobe’s implementation of PDF is far from perfect. If we judge FS implementations of PDF according to their similarity to Acrobat Reader then they will always come up short. Instead of chasing Adobe’s definition, we should all be chasing ISO’s definition, and this includes the governments who publish documents in PDF.