Digital Freedom

Offline Android OpenStreetMap mapping

Last year I did my first mapping during my holiday. I guess any reason would be a valid reason to contribute spare time to OpenStreetMap (OSM), but in this particular case the bird zoo we were going to was hardly mapped on OSM. For OSM mappers this can only mean one thing: a great opportunity to bring value with a mapping contribution. And so I did.

Last year

Without an internet connection at the time (last year), it was hard to get a GPS fix, and even then my battery would run out of juicy pretty quickly. So I finally ended up writing some notes, taking some pictures, and do the final OSM mapping with the Bing sattelite imagery and the zoo’s map I got at the entrance. I was able to crank out the zoo roughly, but I know things could have been better. Well, a good initial step.

This year

We went to the same place, and decided to visit the same zoo. That again meant a clear opportunity to contribute to OSM. Looking at the current state of the mapping, little to nothing had changed since my mapping last year, so every improvement would be a clear win. And I already knew there was still some low-hanging fruit.

Preparation

I still lack a mobile internet connection internationally, and from experience I know my Fairphone 2 without network assistance or WIFI maps (which I assumed would be of no use at the park) would take about 15 minutes to get a proper fix. So unless I would get a hold of some powerbank solution, using GPS and mapping with OSMTracker for Android would be a no-go. I went looking for a solution in my solution provider F-Droid.

Tabulae

Most notably I came across Tabulae which should be able to download OSM map tiles locally for offline viewing, and enable some point of interest commits. I was able to get an initial preview, but downloading the tiles failed on multiple occasions. My best bet is that the shipped version of Tabulae is no longer compatible with the latest OSM tile URL’s.

OsmAnd~

Eventually I came back to my trusted OSM client OsmAnd~. Lucally my changes of last year were already included in the map version I had loaded, so I could use it as a base.

Mapping with OsmAnd~

Although OsmAnd~ offers an explicit plugin feature to take video and audio notes, my approach was much more makeshift. I created a seperate category in my favorites, dedicated for mapping notes. To add a note on a particular place on the map, I would touch the location and add it as a favorite in the mapping category. The name of the favorite, which can be quite long, would hint me to what was going on. Mapping the final outcome was also supported by pictures I took, to provide a better reference.

Steps summarized

1. Install OsmAnd~, download the latest map of what you are about to map (as a reference).
2. Create a seperate favorite category in OsmAnd~ to prevent interference with your normal favorites.
3. Navigate in the app towards the location you want to put a note, select it, and add a favorite.
4. Name the favorite to give the most useful description.
5. Take additional pictures, notes, video, and sound recordings seperately to provide more information if desired.
6. Use your favorites and additional notes to make your contribution to OpenStreetMap.

Batch YouTube-dl task

YouTube music

There is just so many music available to people on YouTube, it’s amazing. And with the advanced screening and copyright infringement reporting policies, as a user you can rest assured that the music on YouTube is available for you on purpose.

To utilize this music availability, I use YouTube-dl. YouTube-dl is a tool that allows you to (batch) search, select, download and convert video’s. The tool itself is not constrained to the use-case of YouTube, but that is just the provider this posts focuses on.

Commandline power

This is the command I’ve been using thus far for youtube-dl to download multiple tracks based on the tracksearch.txt file.

youtube-dl command

1

$ youtube-dl --default-search ytsearch2: -a tracksearch.txt --max-filesize 10M --min-views 1000 -f bestaudio -x --audio-format mp3 --audio-quality 192

tracksearch.txt

1
2

lee rosevere - earthmover
brad sucks - making me nervous

Breaking it down

Search

Selection

Download

Conversion

Room for improvement

Now if multiple video’s (2 for this case) pass the viewcount and filesize selectors, all videos will be downloaded and converted. Ideally only 'the best' video will be downloaded. Likewise, appending the --get-duration option to the command will show the duration for the video, to ensure the downloaded song will not be too short or too long of a duration.

A free music alternative

Although this solution works well for several use-cases, I want to use this post to direct you to the Free Music Archive, a curated selection of free (as in freedom) music. A lot of music is actually CC-NC, so NonCommercial usage allowed, but let this not ruin your party. There are a lot of great artists on this platform, and there are several ways the platform supports discoverability of this great music.

My C720P upgrade to freedom

A couple of years ago I started using my own laptop at work under the policy of bring your own device although to me it wasn’t about the device, it was about bringing a free computing platform I can trust, a GNU/Linux distribution, and many free and powerful applications, all to improve my short-term and long-term effectivenes as an engineer. This however changed my needs regarding battery life, screen brightness and form-factor. And with the recent dying of my Lenovo Thinkpad T60p, I desired an upgrade, to be able to destine my current Lenovo Thinkpad T61 to become my backup.

Currently there are plenty of interesting developments regarding more free laptop projects, which are even destined to pass the FSF Respects Your Freedom certification. Specifically the Novena laptop, the Libreboot X200, other corebooted Thinkpads, the Librem laptop, and the EOMA68 15 inch laptop. Of this set only the Libreboot X200 and the Librem seem to provide the desired technical upgrade relative to my T61. The Librem seems more ideal, but import taxes would drastically increase the already hefty price. Eventually the Libreboot X200 seemed to be the best deal around, and that’s probably why it is used by many FSFE contributors.

However eventually I decided to go another route, in retrospect mainly driven by technical aspects of an even longer battery-life, an even brighter screen, and an even smaller form-factor. I decided to use a converted Acer C720 Chromebook, more specifically the C720P model bringing more RAM, a larger SSD, a touchscreen and a white housing. Also there are quite a number of copies available second-hand, reducing the cost. Having an X86-architecture supported by SeaBIOS, the level of freedom on the C720P can be increased rather easily. Removing an internal screw allows the BIOS to be reflashed, and thanks to John Lewis amongst others a free SeaBIOS payload is available to flash the laptop using coreboot. I must admit this laptop wouldn’t pass the FSF’s Respects Your Freedom certification, due to the Intel Management Engine and VGA BIOS, which are on the coreboot tasklist.

C720P buffed by Kevin

C720 BIOS reflash being enabled

Although I’m somewhat of a tinkerer, I left the freedomification to my Dutch FSFE Fellow Kevin Keijzer. He has flashed his own Acer C720 with coreboot, having used it since early 2014. We agreed on a fair price, as free software isn’t about free as in gratis, it’s about free as in freedom. I must admit I was pleasantly surprised by the level of service I was given. The laptop was buffed to remove scratches, reflashed, pre-installed with Ubuntu GNOME according to my specifications, configured with all the right shortcuts and device-specific configurations, and subjected to a test run to make sure everything was working correctly. As a finishing touch, to remind me about practices by other vendors preloading unwanted media, I was given my best preloaded media yet. I donated my defect T60p to Kevin’s effort on creating freedom respecting laptops from discarded Thinkpads.

Ubuntu GNOME on Acer C720P

Personal laptop sticker on Acer C720P

I strongly encourage anyone to contact Kevin for his Quiet Life Linux Services to experience his level of expertise and great level of service.

Freedom respecting Thinkpad T60

Now having two operational laptops with two slightly different use-cases, I’m even more encouraged to finish my syncing setup. So far my synchronization is done using Syncthing, Mozilla Sync, my own Freenas build, and a remote OwnCloud server, but more on that later.

Fairphone back to the drawing board

Previously I’ve shared my thoughts and concerns on freedom in mobile operating systems. The Fairphone project unfortunately has a bad reputation in this area. Not because they don’t care, but because they failed to deliver on this promise in their first version. Other people involved in open hardware design for mobile devices saw it coming as they’ve been struggling with exactly the same issue for many years already. Especially for them it shouldn’t have been a surprise that a perfectly fine hardware platform would be kept from future firmware updates.

But as in any process of innovation, a new version allows for improvements. And so will a new upcoming version of the Fairphone. For months the Fairphone has featured several lengthy threads discussing alternative, generally more free operating systems. As I tried to state with a lengthy forum post there are multiple interest in strong conflict with each other. So even whether there will be multiple OS flavours or one for all customers is not yet decided. The great news however is that the Fairphone team have taken on this challenge, big time! I’ve had some email conversations with Kees Jongenburger and Joe Mier regarding further plans and options. But more importantly, they went looking for alternatives at the Mobile World Congress. Over the course of the next months the plans will be finalized, so I’d like to encourage anybody with relevant information to contribute to the discussions. Let’s make the next Fairphone far more fair.

Confronting your digital self

July last year I’ve deleted my Facebook account, but not just by deactivating my account, but by removing each and every post, tag and like. This was a head on confrontation with my digital self. The little information I believed to have submitted proved to be an overwhelming pile of data with serious privacy implications. This wasn’t just a rigorous action, it was a treatment teaching me about privacy. Being convinced I would be one of the few crazy enough to make the effort of deleting the individual scraps of information, I was surprised to find out that some of my friends did exactly the same thing. And more importantly, they had a similar mind-boggling experience. What if there would be a tools continually reminding you about the information you have shared, in statistics as well as by highlighting some of that information? Giving this feedback would certainly help to make people more privacy-aware, the initial step towards a better common practice.

Ubuntu calling freedom

So far the flash sales of the first Ubuntu phone by Bq has been sold out, and certainly not without a reason: the Ubuntu phone holds great promises for both the users and the development community. On the FSFE Discussion mailinglist I already gave a quick and general overview mostly based on a recent Linux Unplugged podcast, and so in this post I’d like to revisit my comments with a focus on freedom, as this is lacking in other articles. One word of caution though, I haven’t yet read formal documents or code, so all listed info is second-hand.

First off, embedded devices are difficult, and phones in particular are hard, like Fairphone for instance has come to find. The problem with phone-hardware in general, is the fact that a build is needed for a specific phone since auto-discovery of peripherals like on a regular computer is missing. Add to that the fact that electronics are developed more rapidly than free drivers an be developed, as was the case for the Vivaldi tablet. So unless you have a say in the electronics, and allow a non-signed bootloader, it is very hard and especially time-consuming to develop this lowest layer as free software. and that is also why project like Neo900 and GTA04 exist. One of the added benefits the GTA04 offers, is that the modem is physically separated from other processors, as the modem implementation is locked-down by law. This is about as free software supporting as free hardware designs can get, but this freedom comes at a cost in performance and money, thus requiring plenty of commitment to become a reality.

So in order to actually ship a product, using non-free designs and chips will be the default option, like Ubuntu did in this instance. In order to get a kernel running the device-specific board support package offers the prerequisites needed to boot the Linux kernel. But rather than modifying the Linux kernel and building a tightly integrated software stack for a particular device, as is the case for Android, Ubuntu Phone separates the software stack in two separate layers: a device-specific part and a Ubuntu-part. This separation is ingenious and brings great benefits.

By having a separate Ubuntu-part, this can be updated in the future, without having to do revisions on the device-specific part, thus allowing all models to stay up to date with the newest Ubuntu, and thus avoiding both the platform segregation of Android and the limited number of firmware updates like on iOS. Users can thus still get security fixes and the features newer applications might rely on. Also regarding this part, it would be possible to run a different top layer for a specific mobile operator, or run a different interface on top of this Ubuntu separation layer. I haven’t looked into this layer, but ideally it should be a clean and stable in order to allow others to adopt it.

Likewise the bottom part can be swapped. For instance an Ubuntu Phone port was made to the Nexus 5, which was done by building the necessary but limited hardware support and offering the separation layer. Due to the additional separation, this port will be able to keep up with firmware updates, and so all additional development efforts can go towards improving the device-specific part rather than keeping up with firmware versions. Depending on the required complexity of this device-specific layer, porting additional devices is relatively easy and particularly fruitful as it can remain nearly a one-time effort.

I’m not aware how free the Ubuntu-part is, although I assume this would be in line with other Ubuntu distributions where it mostly adheres to your needs for freedom. The interface is based on Qt5 and is very supportive of HTML5 applications. In this way mobile applications would be able to run on the Ubuntu desktop in the same matter, offering a great convergence solution. Also it is supportive of efforts being made to put forward HTML5 applications for a run-everywhere solution. There is no policy which requires applications to be free, so you can install all kinds of applications, of which a long list is already available. Users are able to sideload applications, avoiding the dependence on an appstore, which is probably the reason why no appstore was launched by Ubuntu just yet. Of no less importance, it seems to be well designed and offer great usability.

One somewhat overlooked part, is the availability of scopes. They aren’t as much overlooked in functionality, but rather in philosophy. Android and iOS have recently realized that apps can be complementary and it is up to the firmware to provide the integration. This can be news and weather, but more recently health and home automation seem relevant as well. The fact that scopes can either work with local data or on the internet but not both, respects the capabilities of the device and prevents unwanted data transmission. More importantly by offering aggregated scopes, you can create a locally generated view. This adheres to the vision of a web which is decentralized rather than centralized and in which each computer has many outgoing connections.

Of course the big elephant in the room is that the phone ties into the Ubuntu ecosystem and so convergence would be best between the Ubuntu phone and the Ubuntu desktop, and likewise it would bring a boost to the Ubuntu store, Ubuntu Snappy Core and presumably to cloud services. So what if Ubuntu would be the next big platform? Well, it would bring a very free firmware which is very friendly to porting devices, it would encourage development in HTML5 and Qt, it would encourage more decentralized applications, it would enable development of the Ubunthu phone itself, and it would put a great alternative next to the Google-ized Android and other systems.

Either way, I nearly bought one but I just missed out by the flash-sale. I’d strongly consider ordering one, because I believe this stack is much more freedom-respecting than Android. More frustrating my perfectly fine phone is still on Android 2.2 with a lack of application support and a whole load of known bugs. I haven’t looked deep enough into Jolla or Tizen to judge them. There are many known improvements available to be adopted, both in hardware, firmware and the available applications. Currently however this seems to be a great phone, with a great software platform, which is another stepping stone in the right direction.

Software isn’t magic

Last month the news landed that the recent Microsoft Outlook app for Android and iOS was leaking and exploiting login credentials. Because of this leak the European Parliament and some universities have blocked the use of this app. Although Microsoft promises double-encryption of the credentials, this specification is an optimistic representation of the actual practice:

What I saw was breathtaking. A frequent scanning from an AWS IP to my mail account. Means Microsoft stores my personal credentials and server data (luckily I’ve used my private test account and not my company account) somewhere in the cloud! They haven’t asked me. They just scan. So they have in theory full access to my PIM data.

— Rene Winkelmeyer

From an engineering perspective this seems to be a straightforward way of offering push messages when the original synchronization interface wasn’t suitable to. But something is of course totally off in the interface of the app. Asking whether or not you’d like to receive push-messages only covers part of the deal. The real result of switching on push-messages can be read in the privacy statement:

We provide a service that indexes and accelerates delivery of your email to your device. That means that our service retrieves your incoming and outgoing email messages and securely pushes them to the app on your device. Similarly, the service retrieves the calendar data and address book contacts associated with your email account and securely pushes those to the app on your device. Those messages, calendar events, and contacts, along with their associated metadata, may be temporarily stored and indexed securely both in our servers and locally on the app on your device. If your emails have attachments and you request to open them in our app, the service retrieves them from the mail server, securely stores them temporarily on our servers, and delivers them to the app.

— Microsoft

It is a unfortunate combination of a lack of security with an unclear presentation to the user. Likewise I’m curious who actually knows for that Google is storing all WiFi credentials of users having enabled the ‘backup’ option. In fact, these misconceptions of the inner working aren’t an exception, it’s more the usual case. Arne Padmos spoke at the last CCC and referred to a research into public perception of email. The over-simplistic drawings on page 15 clearly shows peoples lack of understanding about parties involved. Likewise 29% of U.S. citizens believe the cloud has something to do with the weather, and 95% are using cloud services whilst they thought they weren’t.

Software isn’t magic, but unfortunately it isn’t easy to understand for most people either. I’m certain we can, and should, do better job in educating the general public on these topics. It feels like a big secret waiting to come out, that so many parties and services are involved in getting a service to work. A secret we’d rather not bother a customer with, because the engineers have taken care of it and weighed the pro’s and con’s for the customer. But wouldn’t the customer be better of knowing what decisions underlie a system, to allow an educated choice?

In the Netherlands we have standardized obligatory layouts for energy bills so that customers have a better chance of understanding the product. Likewise there is a standard specification describing more complex financial products for a similar goal. In this regard it seems odd that digital services, which are often times highly complex, can get away with obfuscating instead of explaining. If more people would know their emails are like postcards, and would know that many parties handle those emails, I’m certain the demand for encryption would increase.

Optional rights

Our societies are built on rights which correspond to social norms; fundamental rights correspond to fundamental norms and local rights correspond to local norms. These rights can either be written down like laws, or they can merely be the practical manifestation of the informal norm. This collection of rights is a product of many, many years of progress, but this unfortunately doesn’t mean we can take them for granted. Every single day our rights are subject to discussion and shifting norms.

In recent history it seems that our established rights are no longer taken for granted but are repeatedly being offered as an option. The choice is offered between either keeping your rights or either having some increased convenience or financial benefit. Whilst this does not directly attack our rights, it still does so by way of shifting our norms. If some majority of people aren’t aware of this ‘trap’ and consequently give up their rights, this decreased level of rights becomes the new norm. In these cases the option of choice is hurtful to society, unlike the choice in the marketplace. This choice analogy is however used as an argument to justify the optional rights.

Recently in the Netherlands the right to choose your doctor was subject to debate in parliament, as the liberal party wanted to offer it as a choice, rather than as a right. Giving up this right would enable a financial benefit to the health insurances resulting from their increased negotiation position. In principle consumers should be able to still have this right available to them, but this market principle only holds if some insurances are offering this freedom of choice and the consumers are in fact aware of this consideration and care enough to defend this right. Erecting a new insurance company that adheres to these norms would be the way of the market, but unfortunately this is easier said than done. This market principle thereby undermines the stack of rights we have built over the years as a society via our democratic process.

This grim future of unavailable rights is already a fact in the Dutch educational system, as explained at this Dutch page. Whilst the Dutch parliament has agreed on the right for people to strictly use open standards and free software during education, there is no single Dutch school offering such an educational program. The reason for this unavailability is that in practice schools can choose their IT-systems and the student in market for education respecting open standards and free software is apparently too small or to distributed. So despite our democratic parliament agreeing on this right, in practice this right is subject to the market offering and as a consequence this right isn’t defended anywhere.

Another example is the infamous Facebook, which uses their social lock-in principle to trap users into accepting new terms which violate social norms on privacy, intellectual property and copyright. So rather than offering any benefit in return, it leaves not using the service as the only alternative. In order to defend our established rights, we must stand against this violation both as users and as a society. In this regard we can be glad the Dutch Data Protection Authority is at least investigating Facebook’s new terms.

Considering established levels of privacy, security, freedom or any other kind or right as a marketable feature is hurtful to society, because it erodes our values, our norms and therefore our rights.

This insight was triggered partially by the presentation on Privacy in Context by Helen Nissenbaum and the presentation by Richard Stallman at 31c3.

Why EOMA68 will advance both free software and free hardware

If you’re not familiar with EOMA68, it’s an open electronic interface standard specifically designed to support the development of small computing devices built-up of free hardware and free software. It is mostly known for it’s involvement in the third attempt for creating the KDE-tablet, known as the Spark tablet and the Vivaldi tablet. In this project it was found that it is impossible to rely on the continuity of hardware specifications by Asian electronic vendors. If your goal is to develop a software stack, targeting changing hardware will consume most of the development resources, rendering the project useless. So it became clear that control of the hardware is very important in the fast-paced world of embedded and mobile computing. The EOMA68 standard is an important stepping stone in this regard, because it defines a strict interface between the processing board which includes the main component drivers and the board it is inserted into to provide all the necessary interfaces for the final use-case. This means that the processing boards can be produced at sufficient volumes to enable the desired control over the internal components and thus the free software support. The devices interfacing with the processing boards might be subject to electronic changes, but due to the EOMA68 abstraction, the impact on the software stack will at the very least exclude the basic working of the operating system.

So in this way EOMA68 enables the development of free software for this kind of hardware, but it also increases the ability to design free hardware. If a more free option for chips becomes available, the only step involved for freeing the end-user devices is to develop and build new processing boards. This is far easier of a task than incorporating all the interfaces (like screen drivers) and also the production count can be higher since it is more widely applicable. Also in the process of development the new processing board, it could be tested on the existing EOMA68 platforms without having to develop specific setups. For instance new processing boards can be beta-tested by swapping new cards around between people having EOMA68 compatible devices. Likewise new EOMA68 platforms can be developed and tested by comparing the performance between different processing platforms. So say a driver is functional on a general 64-bit architecture, the driver on the other architecture can be tested to produce the same results, all without creating specific setups for each hardware component.

Then in addition the standard brings the advantage of upgradeable hardware and even shared hardware to the table. The PCMCIA-based boards can be handled by consumers without risking ESD-issues and the interface allows repeated plugging and unplugging without deprecating the contacts. So if your laptop gets slower you just buy a new board for it. And by switching your boards around like a domino-game you can consequently upgrade your netbook, tablet, router or even your smartphone as well. You can leave the now spare processing board on the shelf as a back-up or buy an additional platform to fill another need. This type of upgrading reduces cost and e-waste. Another option would be to have true continuity by carrying a processing board and changing its interfaces depending on the need. You could even change to device with another screen type if you would like to work out in the sun or you could use the built-in connectors of the processing board to watch your holiday pictures at a friend’s place.

So how can you get on board with this? Well, there is a crowdfunding campaign about to launch in order to bootstrap this new paradigm. And just as this system enables, a new and more free processing board is already in development.

IEEE Open Source Software Task Force

Sometimes an open initiative just ‘clicks’, because it fills a growing need and does so in the right way. Great non-software examples I have come across in recent history are Wikipedia OpenStreetMap RepRap DIY Book Scanner WikiHouse OpenDesk and EOMA68. Just yesterday I experienced another such a ‘click’ initiative: the IEEE Task Force on Open Source Software for Power Systems. This initiative has a clear mission in encouraging free software adoption in this rather conservative field:

This Task Force explores the potential for open source software (OSS) in the Power Engineering Society (PES). The mission of the Task Force is twofold:

1. diffuse the philosophy of OSS in the power systems community
2. promote OSS for the benefit of the PES ranging all the way from simple pedagogical OSS to commercial-grade OSS.

— IEEE Open Source Software Task Force

Having a power system background, ever since I’ve become aware of free software I’ve wondered about why so little free software is being developed and used in the field of power systems. This concerns both software for calculations and simulations, but also operational systems like SCADA which could certainly benefit from having more eyes on the code. Also the calculation and simulation software is entering the operation domain now that the increased number of measurements and the available computation power allow for real-time grid analysis.

In any case power system software is becoming an ever more important part of the core business of power system development and management. Some vendor-independence and collaboration in development therefore seems to be important and sensible. Current practice however seems to indicate a low level of adoption by the industry, probably because free software has only recently come to the attention of the industry, and because of the lack of companies offering support. The latter has proven to work for the software industry, with Red Hat as a great example.

Two listed presentations at the panel sessions of 2009 because it shows the task force cares about software integration. An interchangeable data format was discussed which expands upon existing standards to better allow software programs to tie in with each other. Likewise GIS integration has been discussed, which is an important development in bridging the gap between the real-world and the simulation model, since the scope of a power system is greater than its individual components.

The software list published by the task force certainly lists some projects I’ll look further into. I certainly hope the efforts of this task force and the listed projects will contribute to a bright power system future.