No, but seriously. #claimtheclimate #climatechange #bruxelles #brussels #brussel

I made an iPad game. I then made it work on iPhone too. It took me 5 years to finish. I liked it. People who played said they liked it, but it only reached a tiny audience even if I did some paid advertising and used all the traffic of populationpyramid.net to promote it.

Now, it’s not on the App Store anymore, because it did not make sense to pay an Apple account for less than 10 downloads/year and the notch of the new iPhones felt like too much pain to handle for a symmetrical gameplay like this. 🤷🏻‍♂️

In software engineering, remote working makes a lot of sense since, most of the time, you only need a computer and an internet connection to perform your duties. So there are less reasons to force people to sit in a predefined office everyday. Consequently, it has become an important feature of a lot of IT jobs, even in Belgium, while it is definitely not the most forward looking job market. That said, most of the time, the remote working offer only apply to a part of the week (1 or 2 days a week seems to be the most frequent proposition here). Nevertheless, it definitely is getting a hold in most companies.

There is a lot of good to say about remote working, and I see a lot of rabid defence of the practice. I agree with most of it. That said, I have been working remotely for a little more than 5 years now, and I now must acknowledge that it does not come without stress. This might come as a surprise for some, but in the end, I think that remote working has taken some toll on me over the last two years, especially when I went almost fully remote for a year, from June 2016 to June 2017.

So, what can make remote working stressful?

Dehumanisation

If you are working remotely, communication tends to stick to structured channels: the chats, the daily standup, maybe a few global meeting (retrospective, company status) every other week, Jira for the tasks and bug reports and lots and lots of emails. This works well to accomplish structured tasks, but you might definitely feel disconnected from the company sometimes. The fact that most of this communication happens in written form, or in front of groups makes them unsuitable for small talk or more informal information. This can hamper your work, as just chatting about the atmosphere at work can deliver very important information about the smooth progress of projects, but most importantly, it can prevent you to feel as part of a community.

Also, written exchanges are more prone to misinterpretation, even with people you know very well. Not to mention that, if you already spend your day typing on a keyboard to accomplish your technical programming tasks, it might become annoying to have your communication done in written form too: you might end up feeling like a text processing machine.

So, after some time working remotely, it happened multiple times to me to miss the coffee chats, previously felt as unproductive wastes of time. I felt detached from the team, especially when the teams I worked with were made of multiple people working in the same office/place, and seeming to have fun.

Interruptions and multitasking

When working remotely as a developer, the chat (usually Slack or Hipchat) quickly becomes your lifeline to the company: that is the way most people will try to contact you. And to me, being responsive on the chat accomplishes the same as being on time at work in an office: it gives an image of reliability. It also implies that if you do not really want to give the impression that you are taking a lot of breaks, you might find yourself checking your notifications a lot while taking lunch for example, while if people had seen you working the whole morning, or if you were just talking with colleagues at that point, you would not feel the need to be so responsive. I actually often realized that other colleagues working remotely were criticized because they were not answering very quickly on the chat.

A part of the problem is that on a chat, people do not see you physically, so they cannot really estimate if you are at a good moment to be interrupted. So, you are interrupted a lot, and if you are a bit like me, you feel forced to answer quickly. So, you interrupt your work a lot. And in case you do not know it: interruptions are loathed by programmers, since it is really bad for their productivity as it breaks their focus.

The other problem with this remote chat setup is that people do not know if you are already speaking with somebody else. I cannot count the number of times where I have been juggling three different conversations at the same time, which, to me, can become stressful, especially when I do have tasks to finish by the end of the day.

Also, I must mention that there are also often “leisure” chats, created to host non-work stuff (usually, a lot of memes) that can become very, very chatty. At least, I feel that it is really sane to mute these chats most of the time, but when you come back, catching on everything that was said can be a daunting task, while it might be your only opportunity to have some “office spirit” that you are missing by being a remote worker.

Overworking

There are two types of obligations you get when you are taking a job:

Obligations of results, where you commit to give a certain result by a given date. Typically, for a developer, accomplishing a sprint with a given set of bugs/features to develop.

Obligations of means, where you mainly commit to spend some of your time everyday on your work, and you just give the results you managed to produce over that time.

I am not naive, and I know that in the end, in software engineering, most jobs are really about results (you will get fired in the end if you produce nothing), and not means, but in remote working, since people are not seeing you work, you might feel more obliged to show results everyday, even if it forces you to work way past 8 hours a day. As an example, I cannot count the number of times where a configuration problem or a customer call took me a few hours of my day, but I still felt forced to finish the task I committed to do on that day, so that nobody could assume that I was slacking off instead of working. I feel that if people had seen me in front of my computer all day, I would have felt more relaxed about deciding to finish the task the next days.

This leads to two things for me: being really appreciated for the reliability of my output, and … being seriously overworked. According to Jason Fried from Basecamp, this is the true challenge of managing remote workers: people who work too hard.

In the end, this all amounts for me to the question of trust: your employer trusted you a lot, allowing you to work on your own terms , and in exchange, I have always felt compelled to actually work a lot more than if I was in an office.

Being a stay at home dad

When you spend a good part of your time at home, your family sees you as more available than they should. Even if you have places dedicated to work that should be forbidden to your kids, it is still tempting to come ask you “just a little something”. Especially for kids, it is hard to compartimentalize their home. Actually, it is hard for me too …

This can make video calls a bit stressing, when you are talking with a customer for example, and your kids end up just appearing behind you on video. Here is the most well known example of this:

Farewell 2017. Thank you for giving us this. 😂 pic.twitter.com/tIUAzNnIUU

— BBC (@BBC) December 31, 2017

I also know some people have problems resisting the need to perform home duties, like cleaning the kitchen, for example. This has never been too much an issue for me, but on the other hand, this created tension with my wife from time to time, since it was difficult for her to understand how I could have left a dirty dish the whole day on the dining room table while I was actually at home. Answer: I was working and avoiding interruptions…

Loneliness.

Working at home can mean a lot of loneliness. I do enjoy being alone quite a lot, but even for me, after two weeks of only seeing colleagues through my screen, and then my family at night, I end up feeling quite sad. I miss feeling integrated in a community of pairs.

Social networks might help you fight that loneliness a little, but on the other hand, pausing only with social networks is not different enough physically from working on your computer. Moreover, it is also well known that spending more time on social networks rather tends to make you less happy than the opposite.

In the end, I really started to hate being alone most of the time and have felt it to be quite bad for my mental health and my mood. This is a well documented phenomenon.

One of the most recommended way to fight this is to work in coworking spaces. To me, they are a mixed bag: they cost some good money (that your employer might agree to pay, or not), and often ask for commitment (you usually need to register for at least a month). They can create social link (and for me, they created a lot of work opportunity, especially the Betacowork), but at the risk of feeling a bit too much like a vacation camp, with activities everyday (cooking, massages, meet ups, …), to force people to socialise.

Notice that I have been to coworking spaces where there was no socialisation, and gave up rather quickly, since it seemed rather pointless to go somewhere to avoid loneliness, and then not talk to anybody all day. So, I am ambiguous on the subject of forced socialisation.

Another problem with coworking spaces is that you might introduce long commute times in your life again, then work with headphones all day to avoid being distracted, barely taking a break because you lost time commuting, and feeling awkward for not having spoken with anybody while being there for the community. Not to mention that video calls are more difficult to do in these settings, since you usually lack a place to be alone, there is always a bit of noise, and people might be annoyed by your call. Moreover, sometimes, there are things that you do not want to say in front of strangers…

Stress to decide where you work everyday.

This might be just me, but not knowing where I will be working everyday, and having to think about which hardware I need to take with me (example: keyboards, DVI adapters, chargers) has led to a lot of problems, by removing the predictability of being all the time in the same office.

Also, notice that I think that working in coffee shops is a bad idea, at least for full days. There is usually too much noise, and I do not like feeling obligated to take something to eat/drink every hour (your mileage may vary), and the chairs are bad for your back…

You never leave “work”.

When you work remotely, you do not leave your working place at night. On top of that, if you work with people in different timezones, you might end up communicating a lot with people while your day is already over (I did that for months when working with people based in New York or SF). It often makes a lot of sense, since otherwise, you might have too few time to communicate with your team, and this can really hamper the progress of a project, but it means that there is no time in the day free of working concerns, which again, is bad for your mental health over the long run.

Also, working at home does not leave you time to cool off while coming back home from work. For me, the ideal duration of a commute is 15 to 20 minutes. That leaves you some time to walk, and so, do at least a bit of physical exercise, and to change your thoughts a little bit. There were a lot of evenings where I ended up going from a video meeting to a family dinner in 30 seconds, and I must says that it is not always easy to give a good attention to my kids in such occasions.

Career risk

Working remotely makes you less visible in your company. If you intend to have more responsibility in your job over time, this can end up being a problem. I worked for example for a company where I felt that people “in the office” were preferred for promotions. So, you have to consider that too when working remotely.

On a more humorous note, you have to consider than working in your sweat pants for years, unshaved and without too much time constraints might make you unfit to go back to work in an office. You might suffer from some degradation of your social skills:

See the whole comic about working at home here: http://theoatmeal.com/comics/working_home

Conclusion

Globally, I have enjoyed working remotely over the last few years, and it has been a boon to my family and my wife while my kids were small. I do think it made possible for my wife and me to pursue our careers without too much hassle, as I was more available to take care of the kids when they were sick, which happens a lot in the infancy years. This flexibility came to me at the price of catching up on work in the evenings and weekends, but it was still very much appreciated.

Remote working also allowed me to join teams of great quality that I would not have found in my local job market and to work on ambitious projects. So, all in all, I am still a great fan.

On the other hand, after more than 5 years of remote working, I have to say that it really took a toll on my mental comfort sometimes, and that had some unwanted impact on my family relationships, mainly irritability on my side. Before you experience it, it might seem as the perfect solution for your life, but as I tried to explain, there are a lots of unforeseen ways in which it can create stress.

To summarize, the main problem for me is to feel like a text processing machine, receiving mails, Jira tickets and chat messages as input and writing code as output, without the human interactions needed to make it more meaningful. I do not like becoming a kind of a remote developer black box.

#BXLdemandscleanair

I like taking pictures of tramways 🚋. ¯_(ツ)_/¯ #bruxelles #tramway #visitbrussels #brussels #brussel #ixelles

You will very rarely hear me say that something is a “must-read”, because I hate the phrase, but this is it.

A test of the waterlogue app. Lovely.

1, Infinite Loop.

I have been working on the iOS platform for two years now. I really enjoy it. I like the focus on fluidity and good user experience championed by Apple but I also like the engineering challenges that programming apps offers. Here is a whirlwind tour of what difficulties you could expect when beginning to program on iOS.

Fluidity

The first constraint that you have to take into account is this famous focus on fluidity. As I once saw on Twitter, the rule is "60fps or GTFO" (I think this was from @flyosity). Apple has set a standard of smoothness through all their apps (OK, almost all). The scrolling speed and responsiveness of the iOS platform is still globally a notch above everything else. This is seen as a given and you will see most of your users tap impatiently on their device if your app freezes more than half a second. And freezing while scrolling will most probably decrease your average number of stars on the app store.

Concurrency

To get fluidity, the recipe is well known: there is one main thread in charge of updating the UI of your apps. You just need to avoid doing input/output (disk or network) and expensive computations on this thread that would cause UI freezes.

Put differently, you have to make your program concurrent: launching the expensive tasks in background and waiting for them to callback. Problem: concurrency is hard, PhD thesis level hard. It is very difficult to think about a concurrent program because you lose the strict sequentiality, and you can easily get in troubles when modifying the same memory space through different threads.

The main problem here is that if you forget to go back to the main thread to update your UI at some point, you will experience crashes in your app (because UIKit, the graphical framework of Apple, is not thread-safe), but as always with concurrency, these crashes might not always happen, and be hard to reproduce in the debugging environment, because setting a breakpoint for example will change the sequence of events.

The matter is further complicated by the fact that if you use Core Data (and if you need structured data storage, this is the way), it brings its own constraints. Basically, you cannot manipulate a ManagedObject (the base data class of Core Data) on another thread than the one where it was created without risks of crashes. Happily, iOS offers some nice ways of distributing your work over multiple threads, but this brings me to the next difficulty of the iOS platform.

More than one way to do it

The Perl motto applies very well to iOS: "There's more than one way to do it". The technologies used in iOS date back to NeXTSTEP in 1989 and it shows, because you still find in the libraries the traces of this long history, with lots of variations in the way to do things over time. You often find very similar implementations of the same concepts, one in Objective-C and the other in a pure C API (UIFont vs CTFont, for example), with naturally some discrepancies. To get back to threads for example, you have at least 4 different ways to launch a task on a background thread:

1. using NSObject's performSelectorInBackground:withObject:
2. using NSThread
3. using the Grand Central Dispatch C API
4. using NSOperation introduced in iOS4

How do you know which one you should use? As a rule of thumb, you should use the highest level of abstraction that suits your needs, but this brings us to the next difficulty of iOS.

The doc

You will have to navigate the docs to find out which way is the right one in your case. Unfortunately, Apple's documentation is not easy for the beginners. As I read somewhere that I cannot find back now: the documentation is quite clear once you have understood the subject matter, meaning that everything is explained, but not always in the most straightforward manner, and sometimes, information is hidden in suprising spots. Notably, when lost, you should hunt first example Xcode projects provided by Apple to illustrate the use of some APIs, but beware, some of them are seriously outdated, and then you should look at WWWDC videos. Honestly, it was quite infurating for me to discover that some information is only available in video format (about Core Data contexts merging for example) which is obviously not very googlable (especially since this material is in fact behind a login wall).

Network

Another thing that makes iOS programming difficult is that you cannot count on stable network connections (but this applies to all mobile platforms, obviously). Every network request can fail due to the user movement. This is quite different from what happens when you do front end development or server development, where most connections are quite stable. You have to provide graceful behaviours for so much more types of problems.

What makes this even more important is that, compared to web apps, where users have been long taught to do a refresh when something seems to freeze, on mobile, users are not used to relaunch their apps in case of problems (and happily so). If your app is stuck in a bad state with no way for the user to get back in a better situation, there is a good chance that he will simply uninstall the app.

Other difficulties arise when you need to offer some functionalities offline (think about people using your app on the plane). This brings synchronization problems, that as a general rule, are as hard as concurrency problems. Nothing strictly related to iOS, but still a major cause of headaches.

Resources

Constrained resources must be cited too as difficulties. Over 70MB of RAM, your app will probably be killed by the system (but not for sure, it depends on other processes and the memory available on the device). Lots of javascript web apps consume more than that these days! Aside of memory, CPU is not that powerful, a lot less than your laptop computer obviously. This has to be taken into account too.

Memory management used to be a major pain point in objective-c, but it has gotten a lot better since the introduction of automatic reference counting in 2011. You still have to understand what will be kept in memory (everything for which your keep a strong pointer), and you will probably have to understand at some point how ARC behaves when using blocks, but over is the time where you spent hours chasing the superfluous release in your app.

Fighting the system

The degree of liberty offered in iOS to developers is way smaller than what is offered on Android. For example, you are not allowed to keep a daemon running in background when your app is minimized (to download files for example). If you have something like that to do, you have at most 10 minutes to execute them after the user minimized your app. Similarly, you are not allowed to keep a connection open to a server passively either, unless you are a VOIP app. There are also constraints to access the address book of users, keeping the GPS active, and so on, and so forth.

Most of these constraints stems from Apple's intent to keep the user experience pristine from bad developers experimentations, but that does not mean that these constraints cannot be frustrating and that finding workarounds is easy.

App Layout and Text manipulation

There is nothing similar to the flowing layout of HTML in iOS. If you want for example a UI with a paragraph of text and a button just underneath, you will have to compute the height of the text, and put the button at the right place. This can seem particulary brutal for web developers, and be hard to explain to customers and managers accustomed to the relative plasticity of html and css.

Furthermore, this may not apply to all types of apps, but if you need pixel perfect text manipulation, like I did for checkthis, you are in for lots of troubles. Either you have text editing using UITextview but without too much control on things like line height and stylings used for links, either, you have the full control offered by CoreText but without any editing capacities. You have to program all editing by yourself. I must say that there is hope on this front, with this (paying) component, but I did not test it. If you do not need text edition, but just more control over the layout and styling, you can use TTTAttributedLabel.

For layouting text, lots of people resort to HTML and CSS in UIWebView but this can get a bit unholy and offers challenge on its own due to the asynchronous rendering of webviews. I will not dive too much into the controversy of Native vs Web Apps, but let me just state that most popular apps on the App Store are mainly native.

Closed source

This might be obvious to some, but the fact that you cannot access the sources of the UIKit framework, or other important parts of the iOS platform can turn into a real problem when you hit a hard to understand bug. I lost count of the number of time I read on Twitter something along the lines of : "Apple's component x does not do expected behaviour Y, please give me back the days lost trying to understand." and I have been a victim numerous times too. The platform is not especially buggy, but not having access to the source complexifies debugging when needed.

As a side note, the quality of iOS open source varies wildly and there are big surprising holes that are not covered, like for example having a good Twitter connection library that would take into account the latest additions of the Social Framework, while fallbacking to classic in-browser-oauth when on iOS4-5 or when the user did not setup his accounts. I am not complaining about anyone's work here, just observing.

Conclusion

In conclusion, iOS is at the same time one of the most rewarding and challenging platform that I had to work with. There are lots of constraints to take into account but happily, I find the tooling extremely good. Instruments and Xcode are first rate tools. They obviously have their own quirks, but compared to Eclipse or Visual Studio, they fare very well.

On the rewarding side, Apple's customers are using tons of apps that they will take wherever they go, and this is the most personnal computing that I know of. Furthermore, there is a lot of demand for competent mobile developers these days and mobile is where the double digit growth is. So, if you are on the fence, I would encourage you to take the plunge into iOS development.

A gift for my sister in law. She’s a shrink. This thing makes me giddy.

Réservoir de Central Park

Near the betacowork

This one was of my best concert experience. An attendance around 40 people, in an old Brussels house. A super tight set by the BNRS band

The last 11 months have been quite hectic. Mainly, I bought a house and moved into it with my family, I got a new job as the senior programmer in a small web agency and finally, I sneaked in some side projects that were important to me.

This blog post is the first part, from January to July, of an almost exhaustive recap of what I have been programming this year.

First, in January and already detailed on this site, there has been populationpyramid.net, a website allowing you to browse the population pyramids of all the countries in the world, based on the United States data. This was my first incursion in the open data world, and a first attempt at making data visualization in HTML5.

Then, at the beginning of February, I became the IT manager of a small web agency in Brussels: "my media is rich". My duties include system administration, technical architecture, lots of development and globally, technical lead in all software matters. The first thing they had me doing was an iPad application aimed at the Brussels car show. A 6 days deadline for a commercial project on a platform I had never programmed on before, that is the kind of challenge I like to take on (at a bit of expenses to my sleep and family, I must admit). I finally got that one off the ground and learned quite a lot in the process about how web agencies are working (in a very industrialized way, with well defined position: the designer, the motion designer, the project manager, the programmer, the commercial). It was quite interesting. You can see the app here.

The one point that has been obvious from the start there, was that I enjoyed the change to small, well defined projects that those companies churn out, by opposition to the never ending projects I have been working on before, either at the university for my research or the projects I had in my previous company. I had the feeling of getting things done, without too much meetings or email exchanges.

Next up, I had to realize some websites for big clients of ours like http://www.bfoskodarally.be (my first use of Django in a team, with a customized admin aimed to be used by a client) and http://www.generousdays.be, which is now offline. I learnt quite a lot about javascript with those projects.

Then came a very nice challenge. I had to realize one of the fist viral marketing campaign on LinkedIn for Audi. This gave the Audi A6 Challenge. The concept was quite simple: we had to create a "Hot or Not" on LinkedIn, meaning that any user would have to vote repeatedly on two of his/her LinkedIn connection to decide which one they thought as the best professional.
There is a lot to be said about this project. We got quite a bit of attention, even from the people at LinkedIn (if only to inform us about some rule trespassing that we had unwillingly been doing). I liked this project because it was the first time that I got to really work with an API of a social network (the kind your keep hearing about all the time). I was happy that we managed to create a system where the contest was still winnable by lots of people by the end. There is indeed a risk in such contests that a few people take the lead in the beginning and then get unreachable scores, preventing other people to take part. I used a variation of the ELO score used in chess, where people lose more points when losing against somebody with a lower score. This prevented people to get away too fast. Furthermore, we managed quite efficiently to shut down cheating, which is a tough problem for every contest on the web as I have discovered. There are lots of sites for exchanging cheats in such contests (like this one). Finally, the user interface and design were quite polished, which again was a big change of focus from my previous jobs.

Following that, I had to implement a small iOS application for stands on the Carrefour Running Tour Except for the tight deadline again and the fact that I learned to hate iTunes Connect, not much to say about this.

During those months, from february to july, there were three side projects

• In April, I helped a bit to enhance the Django development dashboard, by proposing to add some sparklines (the yellow lines on the site), and implementing them using raphael.js. This forced me to delve into Jacob Kaplan-Moss's code (one of the creators of Django), where I could pick a few interesting tricks. In the end, I submitted a pull request that got accepted, which was important to me, as my first participation to a preexisting open source project, even if in the end, Jacob did not keep much of my code anyway (for good reasons). I have to participate in such projects again. I was very enthusiastic about this.

• At end of May, I was contacted by Jonathan Van Parijs, who organizes the Hack Democracy meetups in Brussels to take part, on the computer front, to a participatory democracy experiment, the g1000. I will not delve into the ideas of the initiative, which are well explained on the site. They asked me to help for the creation of their website using a yet to be defined Photoshop design… In two weeks… In my spare time…
  Let just say that I did not get enough sleep again during those weeks, but I fortunately had help from Jonathan and my awesome colleague, Christophe Gérard (he's freelance now, hire him). This project, not excessively complex, was executed using Django again, but in the process, I have been forced to enhance my Photoshop slicing and css skill a lot.
  From this experience, I keep lots of good memories.
  I will especially remember the long hacking nights with a sense of purpose and in the end the launch of the site at midnight on the 10th of june, from my nephew's room (we were invited to his mother's birthday), with loud basses coming from the party downstair, using my android phone tethering connection because nothing else was available. The opening of the site at midnight had been announced in newspapers and through a press conference and so, we had an expecting audience that brought us some hundreds of visits in the first two hours. A nice, memorable launch.
  Later on, the site has been updated a lot, and in the end, after the main event, it has already got around 150 000 visits. Not too shabby for such a subject, but I must say that the event got a lot of press coverage, even internationally. Technically, the site has been hosted on a 10€/month cloud server at Rackspace, using nginx, mysql and memcached and we never had any performance hiccups at anytime.
  

• The third one is not a programming project. At the end of june, I took two evenings to write down what happened to my significant other when her gmail account had been hacked. This gave the previous blog post If I wrote it, it is obviously because I thought this could have some interest, but I was definitely overwhelmed by the reaction. I submitted it to Hacker News before going to sleep and it raised to the top spot on the front page, where it remained for almost a day bringing around 35000 visits in one day. There were lots of interesting comments (notably, the top one, from the well-known Matt Cutts). On the technical side, this blog is hosted on a shared server at djangohosting.ch and it never got unresponsive. This leads me to wonder which kind of traffic you need on a rather static site to have performance issues. I still do not have answer for this…

In part two of this recap, I will talk about the rather big projects that happened in the second part of the year, and about what's next (with a surprise!).

(To Be Continued...)

On may 17th, in the evening, I received an email from the Gmail account of Charlotte, my significant other. It was written in french (which is normal for her) and looked like this :

How are you ? Would you have time to spend by email on a peculiar situation about me ? I am in deep problems and couldn’t cope with your support.

Hoping to hear from you really soon.

Best, Charlotte

You’ll find the french original text here under (so that people can find it on Google).

I was quite busy and so immediately dismissed this as spam, and did not bother to check where this email had been sent from. Faking email addresses is way to easy to bother for each suspect email. As many people with a public email address, I often receive fake emails from myself.

But this time, the problem was deeper, as I learnt when Charlotte, the real one, called me to warn me that she could not access her Gmail account anymore and that her phone was constantly ringing because of people worried about her. She also told me about a popup that she had seen in the morning about suspect access to her account from the Ivory Coast. At the time, she was quite busy, clicked on some option that looked reassuring and went on with her day. Damn, that was bad.

I immediately tried to access the account (I know her password), to discover that her password had been changed and much worse, that the recovery email address had been changed too (it read something like xxxx1@live.fr where it should have read something like xxxx@ucl.ac.be). I also soon realized that the security question had been changed too. Really, really bad.

Without much hope, I began to fill the “last chance form” of Google. Let it be clear that if you use a free Google account (and even a paying one, in my opinion), you have very few chances to get real people at Google looking at your problem. It is quite logical if you think of the very high ratio of users versus google members : to keep things manageable, they automate administration tasks as much as possible and so, you are in fact only interacting with their programs, never with them directly.

So, when you fill the “last chance form” of Google, you know that you have to fill it as precisely as possible, so that you request pass their automated test. And to examine your request, Google is asking as much information as possible, notably : - the date of your google account registration and the verification code you received at this moment (happily, both information were available in my Gmail account) - the name of the people to whom you write emails the most - the name of the tags you employ and so on and so forth.

With one phone call to Charlotte, I was able to get almost all these information and finally submitted the form, hoping to get an answer for the next day, if I ever got one.

In fact, I got an answer under 15 minutes. I was really happy about this. I immediately logged into Charlotte’s account and went at the bottom of Gmail home page to view the little snippet of text showing if the account was open anywhere else (more info here). By clicking on details on the right of this snippet, you can also sign out all other sessions. There was indeed a session opened in the Ivory Coast (while we are living in Belgium….) so I did close the sessions and changed the password, security question and backup email address. I finally examined all settings of the Gmail account to find out that a forwarding of all emails had been put in place to the following address : tatianalabelle1@live.fr (I don’t see any reason to keep this secret….). Now I did feel almost safe, but still a bit nervous.

Time now for some damage evaluation. I immediately saw that all contacts had been deleted (annoying but not too bad) and that apparently all emails had been put in the trash (but I immediately saw that some were missing too, from the last few days) except for the responses to the fake “emergency” email. Already around 10 responses. I immediately responded to people not to take this into account. This was a bit embarassing for Charlotte, since lots of his colleagues and her whole familly and friends had been contacted. This took me around 10 minutes.

That’s when I lost the connection to the account again.

I immediately tried to login again, to find out that passwords, security questions and backup email addresses had been changed again. It was time for heavy swearing on my part, with some punching of the table.

I then did everything again : filling the form, waiting 15 minutes to get an answer, signing out sessions, changing passwords, security questions and checking again if no other session had been opened in the mean time. According to Google, it was not the case. As a side not, I was very glad that the “last chance form” did work twice. I was really thinking that it could be blocked after a first attempt, to allow further investigation. But I guess that since a human investigation is not really an option, Google chose to let the system run as much as needed. I will probably never know… So I began responding to emails again.

That’s when I lost the connection again…. Password, security question and backup email changed again.

At this stage, I was getting a bit desperate. I called Charlotte, asking her if any of our computers were open with a session (this may not sound very rational, but you never know). It turns out that the windows XP machine that I keep for gaming was on. I told her to turn it off (I did not had tim to inspect it since). Charlotte also told me to delete all her emails if I could get into the account again, to decrease privacy intrusions (even if at this stage, it could have been too late). In despair, I wrote an email to tatianalabelle1@live.fr asking to stop pirating my girlfriend email account.

I tried the “last chance form” again and it did work a third time. I did everything to try to secure the account and began to respond to emails again. It turns out that this time, I did not lose the connection again. I have been checking this regularly for than a month now, and there was no more suspect activity.

This whole story did left a sour taste though. Charlotte lost all her contacts and past emails. This is not too bad for her, she is not an heavy Gmail user (she has a professional email address too) but I definitely am. I use it for all my emails, my calendar, and lots of other google tools (Analytics for websites for example). If I ever lose my Google account, I could lose a great deal of time and value. This is accentuated by the fact that I have used my Gmail address for registering in lots of other systems too (Ebay, PayPal, Facebook, Twitter, Apple Dev Center, Itunes, some server hosting and so on). For almost all those services, if you have access to my email, you can get access to the service (by filling the “lost password” form). They will send a link to allow you to change the password and login again.

In the precise case of Charlotte, using this kind of accounts did not seem to be the plan. Some people who responded to the fake email got responses where they were asked to send money using Western Union to help Charlotte in Africa, where she had allegedly almost been raped and had not access to a phone. Not very subtle but I still felt the bullet passing much too close…

To mitigate the risk, Google recently launched two-factor authentication, a mechanism that requires you to input, on top of your password, a code generated by an application installed on your phone (iPhone, Android and maybe some others). I have activated this today. You can find more information about this here : advanced sign in security for your google account This indeed increases security, but tends to be a bit cumbersome (I often have a depleted battery, for example, which could prevent access to my emails from a computer) and does not solve other case (like somebody stealing my laptop and using an already opened session).

In the end, I am still feeling a tad insecure about using Gmail as may main account. I think it’s way too difficult to get somebody on the line to help you in case of problem. I am nevertheless quite addicted to their interface, and consequently does not want to give it up, but I will probably switch my main account to a domain I own, so that at least I could shut down the email address in case of need, but still use the Gmail tools through Google apps.

I decided to write this all down so that it could serve as a cautionary tale. You should keep in mind the limitation of your email provider and if you decide to use Gmail, you should keep as much information about your Gmail registration as possible (try to find back your first registration confirmation), and if the option is offered to you, which seems to be the case only for heavy users (probably the people using some paying services from Google), you should activate two-factor authentication. As a side note, I would be definitely interested in buying from Google a gadget similar to the Blizzard Battle net authenticators to use two-factor authentication. I would then be able to keep my phone as a backup option only.

Finally, I am still wondering how those guys in the Ivory Coast got access to Charlotte’s email account. My main hypothesis is that she must have accessed her account on a pc infected by a key logger (To excuse her, she’s working in a big structure, she not in IT and could not possibly control every machine). That said, I found the action quite thoroughly organized (notably, forwarding emails to an external email address is a step that could easily be missed during the recovery of an account), and the most distressing to me is that I am still unable to explain how those guys were able to get access to the account twice after I changed the password, security questions and backup email address from my Mac that does not seem to be compromised.

If you have plausible explanations, I am definitely interested in hearing them.

Original French Text of the fake email :

Comment vas tu ? Aurais tu du temps à consacrer à une situation particulière me concernant discrètement et par mail ? Je suis dans des difficultés telles que je ne saurai que faire sans ton soutien et apport.

Je reste dans l’attente urgente de te lire.

Cordialement,

Charlotte

For some months, I have been interested in data visualization. I have been playing with Processing, and Raphaël and read one of Edward Tufte's books.

Besides, as any motivated web developer, I am interested by the new crop of features appearing in our browsers under the banner HTML5, especially SVG and canvas, which allows to draw vectorial graphics in your browser more easily and to manipulate them using Javascript.

Finally, I am fascinated by the idea of open data, that is, the public availability of data in an easily processable format. For an example of projects made possible by open data, you could check www.wheresmyvillo.be which was made possible by the availability (not official in this case ...) of raw data about the bike sharing scheme of Brussels on the web. My interest about this subject was initially sparked by Adrian Holovaty, one of the creator of django, who created www.everyblock.com, a site allowing you to be informed about every public bit of information in you neighborood (from meetups about programming to burglaries).

With all this in mind, in November last year, I went to the first Hacks/Hackers event in Brussels. It is an event aimed at bringing programmers and journalists together to find out interesting ways to exploit open data. It was a very nice meeting, very inspiring, and it crystalized my interests. I had to do something combining it all.

So, in December, I began to look for interesting public data sets. I found out, thanks to a Wikipedia article reference, that the United Nations have made available the content of their World Population Prospects at http://esa.un.org/unpp/. After some thinking and googling, I came to the conclusion that something as basic as the set of population pyramids of all countries of the world were not directly available online. I settled to publish it, using Raphaël.js to obtain a beautiful visualization. Although not very original, I think those graphs are highly thought-provoking.

Here are a few interesting points:

• One can easily spot the problem for future public retirement plans problem in Europe (with more and more elderly people ). See here for example : population pyramid for Belgium in 2050
• I am still wondering what is happening in countries like Qatar in 2010 (population pyramid for Qatar in 2010) How come there are so many more men than women ? According to a Hacker News comment, this could be caused by migrant workers
• Near 1950 in Europe, you can spot the "dent" made by WW1 in the population, plus the baby boom. Example : population pyramid for France in 1950
• It is already interesting to compare the shape of the pyramid for countries like Afghanistan in 2010 (population pyramid for Afghanistan in 2010) VS the US in 2010 ( population pyramid for the U.S.A. in 2010). One can easily see the difference in development, to say the least.

It took me approximatively two months to finish the project (in fact approximatively five long evenings, but it took me two months to find those five evenings....). From the start, I had envisioned a project with no backend, only Javascript goodness. So, yes, no Django this time. This made possible to host the project on GitHub pages (from the homepage : "The GitHub Pages feature allows you to publish content to the web by simply pushing content to one of your GitHub hosted repositories.") which meant free hosting, and furthermore was a good fit, since I wanted to publish all code under an open source license.

About the code

All code is available on GitHub at : https://github.com/madewulf/populationpyramid.net. I do not think this code would be very good for reuse, but it could be used as an inspiration. The resulting html page is unfortunately completely dependent on the availability of JavaScript to render anything useful and will consequently probably not be referenced very efficiently by Google. I hope to fix that later on (if I can find one or two more evenings ;-) )

In the process, I learnt to use a few things:

• Raphaël.js support for SVG paths. This allows you to draw complex shapes in a quite straightforward way using the SVG path syntax and there are nice animation effects available (the "morphing" effect from one population pyramid to another is only one line of code). As a side note, Raphaël is really a great tool because it fulfills today lots of the promises of HTML5 about vector graphics, even in Internet Explorer (from version 6) by using Mircrosoft VML where SVG is not available. The only downside is that it does not work on Android machines (but works very well on your iThings).
• the 960 grid system a CSS framework that allowed me to finally fix some of my gripes about CSS (the other ones should be fixed by the use of something like SASS).
• the use of cURL to scrape the content of web sites. I shoud definitely have known this before.
• the use of Google Web Fonts. I think a fair part of the reasonable visual appeal of the site is due to the typography;
• the set up of a custom domain name on top of GitHub pages. Super duper easy. By the way, thanks to the GitHub people for offering such awesome tools!

Publication

Once I got a sufficiently good version of the site, I published it at http://populationpyramid.net. I wanted some people to see it and maybe give me some feedback. After one Facebook wall post and a tweet that did not bring much traffic, I posted the link to Hacker News. I am a daily reader of this site about tech news and the startup world, and I thought that by insisting on the technological side of my endeavour, people there would be interested. I consequently chose the title "Population Pyramids of the World in SVG using Raphaël.js" and it worked quite well. Two hours after publication, PopulationPyramid.net was featured on the front page of Hacker News (here is the comment page). It reached around the 15th spot, which was much more than what I hoped for. Over two days, Hacker News brought around 1200 direct referred visits and around 1000 more through tweets of the different Hacker News feeds.

Next, although I am not a reader of this site, I posted the link to the programming section of Reddit thinking that it could interest people there (I realized later on that I maybe did not pick the right section, which could explain some downvoting there, although it seems to be a common practice on Reddit). There again, PopulationPyramid.net reached the front page, where it remained for around 24 hours, which brought around 1800 direct visits (here is the comment page ). I am quite happy with those numbers, even if honestly, I thought that those sites were able to bring more visits (but I only have a single data point here). That said, even if the traffic spike was nice, I mainly hope that the site will be used as a reference later on. My dream is to have a school using it as material for some homework for teenagers. If you know somebody that could be interested...

Conclusion

I loved to do this side project and I definitely recommend to any programmer wanting to improve to do this kind of one shot project to pick up a bunch of new technologies. Furthermore, I hope that this project can bring some value to people around the world to better understand the social problems we are currently facing. I chose not to add any comments to the figures because I lacked expertise and time, but also because I hope that people will take time to think about those images and express their own interpretations.

New Feature Ideas

As a final note, here is a list of features I would like to add someday:

• offering the ability to compare two pyramids;
• allowing to embed a pyramid in a distant site (so that people could blog about some pyramids);
• adding a curve showing the population size evolution;
• exploiting the wealth of other data available on the UN website. There is for sure something interesting to do with the available migration rates for example;
• add coordinate information when hovering over a data point;
• allow to download the different images as separate files (currently, they are generated in your browser on the fly). This would probably involve using a Raphaël serializer and would allow to create a site allowing progressive enhancements, solving the dependency on Javascript.

One point of the Django documentation that I did not get right at first is : how to get the ORM to generate a bit complex join queries. Yeah, sorry, I learnt SQL before ORMs, as many people, and I definitely think in terms of relational models and SQL joins.

For example, given the following models involving deliveries of products in a driver schedule, how do you get the list of products appearing in a schedule ? (I know that this sounds a lot like a Data Base 101 course).

class Product(models.Model):
name = models.CharField(max_length=150)
class Schedule(models.Model):
name = models.CharField(max_length=150)
class Delivery(models.Model):
.... some data fields here
product=models.ForeignKey(Product)
schedule=models.ForeignKey(Schedule)

The answer is deceptively simple, just write :

s = Schedule.objects.get(....)
Product.objects.filter(delivery__schedule = s).distinct()

which roughly will translate into the following SQL :

select distinct(product.*) from product
inner join delivery on product.id = delivery.product_id
inner join schedule on schedule.id = delivery.schedule_id
where schedule.id = some_id

The important thing to notice is that in the line

Product.objects.filter(delivery__schedule = s).distinct()

you can use delivery as the beginning of the lookup parameters, while there is no delivery field in the Product class. Django will understand.

What is maybe a bit misleading (even if it is quite logical if you think about it), is that there actually is a field named delivery_set created on the Product class (it is a RelatedManager), but you can not use it in the lookup parameters. It took me a long time to figure this out, so it might be the case for other people too....

That said, everything is explained in the Django doc here even if for once, I find the explanation a bit perfunctory

Disclaimer

This blog post is not very original, it is essentially a mix of two previous blog posts, one from Andrew Turner and one from Derek Willis.

And now, the real content

For my pet project, ZoFa, I wanted to allow my users to publish their schedule, computed on the site, to their favourite calendar application, particulary to Google Calendar, which is the calendar application I am using myself.

To do this, there are two steps to accomplish : First,create a view generating an ical file (with extension .ics) I used the vobject library to do this as showed here.

By the way, vobject is a part of the doomed Chandler project described in excellent book. Highly recommended.

Having battled quite some time with this, I think that the following cod snippet could be of interest to some people. So, here is the code of my view :

def ical(request, worker_id):
worker=Worker.objects.get(pk=int(worker_id))
cal = vobject.iCalendar()
cal.add(‘method’).value = ‘PUBLISH’ # IE/Outlook needs this
for shift in worker.shift_set.filter(published=True):
vevent = cal.add(‘vevent’)
vevent.add(‘dtstart’).value=shift.begin #begin and end are python datetime objects
vevent.add(‘dtend’).value=shift.end
vevent.add(‘summary’).value=shift.name
vevent.add(‘uid’).value=str(shift.id)
icalstream = cal.serialize()
response = HttpResponse(icalstream, mimetype=‘text/calendar’)
response[‘Filename’] = ‘shifts.ics’ # IE needs this
response[‘Content-Disposition’] = ‘attachment; filename=shifts.ics’
return response

Finally, just add the url to your urls.py :

url(r’^ical/(?P<worker_id>\d+)/shifts.ics$’,‘ical’,name=“shifts_ical”)

It seemed necessary for Google to pick it up to add a file name at the end of the url, but I am not sure about this.

The second step needs that your user import the url of your calendar into Google Calendar. To do this in the current interface :

1. open the “other calendars” little frame
2. click on “add” (at the bottom)
3. click “add by url” and paste the url.

Some concluding remarks :

• It can take some time before Google really requests your calendar and displays it
• The .ics file generated can also be used in Outlook, ICal or Thunderbird.
• Your url needs to be publicly available : there is no way to provide to Google Calendar a login and a password to your site.

The next step for me would be to automatize the addition to google calendar (with a nice “Add your schedule to Google Calendar” button) but I still have no idea about the feasability of this.