Yet another little corner of the web

Random thoughts and experience, committed to electrons

Management Hacking

In an ideal world, everyone would implicitly understand that it just makes good business sense to upstream some of the modifications made when creating your Linux powered devices. However this is a long way from being common knowledge and is still something that a lot of managers that will need convincing that this infact in their best interests. Just so that we are clear, I’m not suggesting here that your next Linux powered device should be an entirely open design. We live in the real world and unless your explicit aim is to produce a completely open platform, doing so is unlikely to be good for your companies profitabilty. What does make sense is to protect those parts of your product that drive your value proposition and look to reduce your costs in places which don’t drive the value add or unique selling point that your market to your customers. This is where upstreaming and the open source can offer you a massive advantage, if done right.

So, you have your new product in production, you have a number of cool features to implement that you hope will drive customers to your door. You also have a new hardware design, the hardware guys have discovered some funky new devices that optimise and improve the new design. You’ve picked up the SoC vendors slightly out-dated kernel tree and discovered that some of these devices already have some support in the kernel, awesome. For others there is no support, either in the vendors tree or in the mainline tree, so backporting isn’t an option and your looking at writing some drivers. You’ve heard something about upstreaming and would like to give it a go, but is this going to be a good idea? Is this going to help my company? The answer is generally “Yes”.

Uptreaming is the process of submitting the changes that you have made to, typically, existing open source projects so that they become part of the main (or upstream) codebase. This may be changes to support specific hardware (typically kernel level changes), changes to fix bugs that you’ve exposed via your specific use case or additional features that may extend existing libraries that you use in your project.

Upstreaming provides you with a number of tangible advantages which can be used as rationale to help convince your mangement: - You gain at least one 3rd party review of your changes, by a domain expert, giving you confidence in the quality of the changes. - You reduce your delta with the upstream codebase, reducing your maintenace burden when maintaining your product (you do security updates, right?), providing product updates and potentialy when creating the next version of your product. - Community suggested improvements providing you with ways to reduce your code size whilst simultanously increasing available features.

I will use the kernel as an example (one which many product developers are likely to need to modify) of how these benefits manifest as this is the project that I am familiar with.

Changes submitted to opensource projects are not blindly accepted. Projects need to take some care that the changes are not going to negatively impact other users of the project that may have other use cases and ensure that the changes are both sensible and done in a way that ensures that the project can be maintained in the future. As a result changes may need to be modifiied before being accepted, but such changes are likely to have a positive impact on your modifications - the reviewer (who is very likely to be an expert in the area in which you are making changes) may be able to point out existing infrastructure that can be used to reduce code length and increase code use, or recommend changes that may remove potential race conditions or fix bugs that may not have been triggered during your testing. As the kernel, like most projects, expect a specific code style, there may be requests to change code to meet these as a consitent code style makes maintenance of the code easier. Once merged, the maintainer will be taking on the burden of maintaining this code, so will want to ensure this can be done efficiently.

The upstream kernel code base is being modified at a fast pace, with a change being merged at a rate of one every 7 minutes. Different parts of the kernel develop at different rates, some places see little change, others see quite fast paced change, though there are limited places in the kernel that undergo no changes at all. Over time there is an increasingly likely chance that such changes will stop any local patches that you have. Such changes require developer time to make modifications to the changes when updating the software stack on the existing product or when attempting to re-use such changes on a new product. Conversely, when such changes are applied upstream and existing code will be changed when APIs change, generally resulting in the modifications continuing to function as required in your use case without any engineering effort on your behalf. Additionaly once, for example, a driver is added to the kernel, others may add features to the driver that weren’t of immediate use to you, however as your requirements change and grow for updates and subsequent revisions, such changes may prove very useful to you and available with minimal work. A well documented example of this is Microsoft’s submission of hyper-V support. This large body of work was initially added to the “staging” area, an area where drivers that aren’t ready for full inclusion in the kernel can be put to enable them to be modified and improved with the help of the community. Whilst in the staging area the drivers were greatly improved, the drivers were modified to comply with the Linux Driver Model, reducing the code line count by 60% whilst simultaneously significantly improving performance and stability.

There are also less tangible reasons for contributing upstream. As a company, if you are planning to utilise Linux and other free software in your products, it is likely that you will have a wish to hire talented, experienced developers to help you create your products. Contributions made to open source projects relevant to you are likely to be noticed by the very developers that you hope to atract to your company and will reflect well on your company should they be looking for new opportunies or should these developers be asked if they have any recommendations for good places to work.

Submitting upstream can also be a very rewarding experience for existing employees. This can help with staff retention, providing those interested with the very rewarding experience of contributing to open source projects. As well as being rewarding, the experience of making contributions to open source projects is likely to build the experience and unserstanding of the projects on which your products are based, providing them with knowledge and contacts that are likely to be highly beneficial to your continuing use of open source and enabling them to more efficiently and cofidently build products in the future.

15 Nov 2018 #linux #open source