| Documentation/compat |
| ==================== |
| |
| This is now deprecated, use :doc:`hacking on backports <backports/hacking>`. |
| |
| This page is dedicated to compat, a generic Linux kernel backport |
| compatibility module that has been brewing since 2009. The compat module |
| provides functionality introduced in newer kernels to older kernels |
| through a set of header files and exported symbols. The compat module |
| allows code from newer kernel releases to be used on older kernels |
| without modifications with a few exceptions. Exceptions to achieve |
| automatically backporting kernel code through the compat module are |
| dealt with through specific mechanisms documented in the :doc:`compat-drivers |
| project page <compat-drivers>` and most recently through the `slides |
| used at the 2011 Linux Plumbers presentation on automatically |
| backporting the kernel |
| <http://www.linuxplumbersconf.com/2011/ocw/system/presentations/771/original/kernel-backport-for-good.odp>`__ |
| and its `follow up presentation |
| <https://docs.google.com/presentation/d/1axVNEGwKZjnzG1ocdd289WMqPxzJ3qfMv70ghGcnUKc/edit>`__ |
| at the 2012 Linux Collaboration summit in San Francisco. |
| |
| By using compat and frameworks such as :doc:`compat-drivers |
| <compat-drivers>`, and techniques to integrate `additional patches |
| <http://wireless.kernel.org/en/users/Download/stable/#Additional_patches_to_stable_releases>`__ |
| into upstream releases that prefer and push for upstream, we hope to |
| ensure developers can prioritize working directly upstream on the Linux |
| kernel and provide deliverables to end users and customers for any |
| target kernel. |
| |
| compat-drivers today automatically backports the `Wireless |
| <http://wireless.kernel.org/>`__, `Bluetooth <http://www.bluez.org/>`__ |
| and `Ethernet |
| <https://www.linuxfoundation.org/collaborate/workgroups/networking/ethernet>`__ |
| drivers directly from the Linux kernel on a daily basis by using |
| `linux-next.git |
| <http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. |
| We also make `stable backport releases |
| <http://wireless.kernel.org/en/users/Download/stable/>`__ by using the |
| `linux-stable.git |
| <http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__ |
| tree. We are considering expanding this framework to automatically |
| backport more drivers -- patches are welcomed, otherwise, you'll have to |
| wait until someone decides to integrate other subsystems. |
| |
| Supported kernels |
| ----------------- |
| |
| The goal is to backport new kernel functionality at least down to the |
| oldest stable supported kernel listed on `kernel.org |
| <https://kernel.org>`__, but backporting code further is certainly |
| welcomed. Currently this means supporting backporting down to the 2.6.27 |
| Linux kernel release. Although our goal is to at least support backport |
| efforts down to 2.6.27 the compat module currently has code to support |
| even older kernels, and we welcome code to support older kernels as |
| well. |
| |
| Mailing list |
| ------------ |
| |
| See the :doc:`Backports mailing list <../mailing_list>` page. |
| |
| Presentations |
| ------------- |
| |
| Metrics explaining the amount of code saved by automatically backporting |
| the kernel using compat and current research are explained. |
| |
| - `2011 Linux plumbers kernel backport presentation <http://linuxplumbersconf.org/2011/ocw/proposals/771>`__ (`slides <http://www.linuxplumbersconf.com/2011/ocw/system/presentations/771/original/kernel-backport-for-good.odp>`__) |
| - `follow up presentation Automatically backporting the Linux kernel - 2012 Linux Collaboration summit <https://docs.google.com/presentation/d/1axVNEGwKZjnzG1ocdd289WMqPxzJ3qfMv70ghGcnUKc/edit>`__ |
| |
| Estimating deliverables |
| ----------------------- |
| |
| Once you use compat and kernel framework integrators such as |
| :doc:`compat-drivers <compat-drivers>` you should have no excuse to work |
| outside of the upstream Linux kernel. In order to help guestimate when |
| your upstream code will be available through a stable release such as |
| the :doc:`compat-drivers stable releases <../releases>` you can rely on |
| the `phb-crystall-ball <http://phb-crystal-ball.org/>`__. |
| |
| Tracking down when a feature went upstream |
| ------------------------------------------ |
| |
| The compat module expect you to put functionality into either header |
| files for defines and static inlines or into c files. This functionality |
| must be tucked away under a file for the kernel release that introduced |
| the functionality. To help aid you with your scavenger hunt on tracing |
| down when a specific functionality was introduced into the upstream |
| Linux kernel you can use several techniques. This section documents |
| those techniques. |
| |
| Git tree setup for backporting |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| In order to understand how to track a feature in the Linux kernel you |
| must first understand how the development of the Linux kernel happens. |
| Below we provide a basic description of how this works. |
| |
| Greg Kroah-Hartman maintained the stable series of the Linux kernel. |
| Linus hands off kernel maintainership to Greg after he blesses the first |
| release of a stable kernel. Greg maintains the stable kernel via through |
| the `linux-stable.git |
| <http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__ |
| tree. Each stable release of the kernel has a respective |
| origin/linux-3.x.y branch, where x is the current release number. The |
| "y" here is literal, so for the 3.2 release there is an |
| origin/linux-3.2.y branch that keeps track of the 3.2.1, 3.2.2 and |
| future extra version releases of the 3.2 release. You can use the |
| `linux-stable.git |
| <http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=summary>`__ |
| tree to verify if code is present on a stable release of a kernel. |
| |
| Linus maintains the release candidates of the Linux kernel (RC) up to |
| the point he blesses away the first release of a kernel release. So for |
| example Linus maintained the 3.2 kernel from the inception of 3.2-r1 |
| down to 3.2-rc7. During the 3.2-rc1 to 3.2-rc7 release Linus is supposed |
| to only accept regression fixes. After that he releases the first |
| blessed release of 3.2 and handed off maintainership to Greg. All the |
| magic that went into 3.2-rc1 though happens in what we call the |
| development cycle of the Linux kernel. The development cycle of the |
| Linux kernel varies subsystem to subsystem but the idea can be best |
| described in terms of the usage of two subsystem git trees. Subsystem |
| maintainers typically have to maintain two git trees, a subsystem.git |
| tree and a subsystem-next.git tree. The subsystem.git tree contains code |
| that should go into Linus' tree during the regression testing of the RC |
| release of the kernel. The subsystem-next.git contains code is being |
| queued up for the next release of the Linux kernel, hence the "-next" |
| postfix. All code in the subsystem-next.git trees is ephemeral, that is, |
| what you see there today does not mean that it will be present tomorrow, |
| it is code that is simply accepted on that day by the subsystem |
| maintainer, but a lot of it can simply be dropped, and in fact Linus can |
| reject merging it eventually. Linus will start merging all |
| subsystem-next.git trees into his own tree during what we call, the |
| merge window. This typically consists of a 2-3 week window period. For |
| example the merge window for the 3.2 kernel started after Linus released |
| the v3.1 kernel release on Mon Oct 24, 2011, and ended when Linus |
| released the v3.2-rc1 release on Mon Nov 7, 2011, the merge window in |
| this case lasted exactly 2 weeks. |
| |
| The merge window can often be painful though and to aid with this |
| Stephen Rothwell has decided to take up merging all subsystem-next.git |
| trees into one tree, `linux-next.git |
| <http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__ |
| on a daily basis. He rants to each subsystem maintainer on any issues |
| during his merge, including build breakage and general insanity. |
| |
| The implications here are that we can not only track code that is |
| already released, but we can also predict what likely will go into the |
| next release of the Linux kernel by using `linux-next.git |
| <http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. |
| This means we can backport the Linux kernel proactively through every |
| subsystem development cycle by using `linux-next.git |
| <http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. |
| |
| If this is still a bit fuzzy you can review the `wireless development |
| process <http://wireless.kernel.org/en/developers/process>`__ for a |
| review of how that specific subsystem development works. The development |
| process there even includes a shiny diagram. |
| |
| Using git grep to identify what file adds new functionality |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| First you want to identify what file added the functionality you seek to |
| backport. Suppose you want to identify what file adds a new function, |
| foobar(). To do this you can use:: |
| |
| git grep foobar\( |
| |
| to identify where foobar() is used in the entire Linux kernel. git grep |
| will use the indexed data and at times can often run faster than a |
| regular grep, as it will also skip object file in a regular grep search. |
| Given that the point of compat is to backport functionality and that |
| this functionality is typically shared between modules and subsystems, |
| you can rest assured most functionality has a declaration in the header |
| files of the Linux kernel. You can typically save yourself git grep time |
| by starting to search through the include/ directory of the Linux |
| kernel. |
| |
| :: |
| |
| git grep foobar\( include/ |
| |
| Using git blame to identify introduction of a commit |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Typically compat will have a lot of features already backported for |
| older kernels, so to help backport new functionality you should be using |
| `linux-next.git |
| <http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=summary>`__. |
| To identify at what point a specific feature was added into the Linux |
| kernel you can use git blame to identify the sha1sum of the commit that |
| added the functionality you want to backport. At times though such |
| functionality may have not been introduced via that commit but instead |
| it may have simply been moved from one place to another. You can use:: |
| |
| git blame sha1sum~1 |
| |
| to review if the functionality was present before the last commit you identified added that functionality to a file. |
| |
| Using git describe to identify where an commit was introduced |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Once you have identified the commit that introduced a specific |
| functionality you can use git describe --contains sha1sum to give you |
| the first commit tag that contains the commit. You can use the base of |
| the tag to help you figure out the first kernel release that added the |
| commit. |
| |
| For example, commit 600715dcdf567c86f8b2c6173fcfb4b873e25a19, or short, |
| 600715dc added to the Linux kernel the CONFIG_PHYS_ADDR_T_64BIT feature. |
| We can determine that it was added on the v2.6.28 release by reviewing |
| the results of using:: |
| |
| git describe --contains 600715dc |
| |
| which yields: v2.6.28-rc1~271^2^4~6. This means that in order to |
| backport 600715dc we can edit two files on compat: |
| |
| - include/linux/compat-2.6.28.h |
| - compat/compat-2.6.28.c |
| |
| Since this functionality can be dealt with header files the c file is |
| not touched to backport this functionality. You can review the |
| respective `compat CONFIG_PHYS_ADDR_T_64BIT backport via gitweb |
| <https://github.com/mcgrof/compat/commit/4ec0edbd1cc2e6165e29f8ba466324e975e1cd73#include>`__. |
| |
| Testing across kernels |
| ---------------------- |
| |
| The goal behind compat is to add features to provide backport but to |
| older kernels but to do this successfully we must also ensure that each |
| new patch added does not break building against a supported kernel. It |
| may be a daunting task to get all supported kernels to and also to build |
| against then, which is why we have started adding support to do all of |
| this for you. Under the bin/ directory you will find two scripts which |
| we document below: |
| |
| - get-compat-kernels |
| - ckmake |
| |
| get-compat-kernels |
| ~~~~~~~~~~~~~~~~~~ |
| |
| You can run this script to install all known supported kernels. |
| Currently this will only work for Ubuntu, but `work is being considered |
| <https://lkml.org/lkml/2012/2/6/85>`__ to standardize on the same |
| kernels for any Linux distribution. |
| |
| ckmake |
| ~~~~~~ |
| |
| You can run this to test compiling compat against supported kernels |
| available. Although initially designed to support Ubuntu only the goal |
| is to make this Linux distribution agnostic. |
| |
| compat modules |
| -------------- |
| |
| compat.git provides a few modules and headers to help with general |
| kernel compatibility. |
| |
| compat module |
| ~~~~~~~~~~~~~ |
| |
| Provides all exported symbols implemented in each respective kernel |
| compat-2.6.xy.c files. Upon module load it just initializes the Linux |
| kernel's *power management Quality Of Service* (aka **pm-qos**) |
| Interface interface added as of the 2.6.24 kernel. No other things are |
| initialized, the rest of the compat module just acts as a library of |
| exported symbols. |
| |
| compat_firmware_class module |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Another module which compat.git provides is a backport of the |
| firmware_class module which got updated recently newer with a new |
| request_firmware_nowait() to allow better asynchronous firmware |
| uploading. This was added as of the 2.6.33 kernel. The firmware_class |
| module has been backported into a new module called |
| compat_firmware_class. A separate module has been defined instead of a |
| direct replacement for firmware_class since your system may have old |
| drivers which use the old request_firmware_nowait() and would bust if |
| they used the new request_firmware_nowait(). The compat_firmware_class |
| module registers its own sysfs subsystem and as such also gets udev |
| events sent through a separate subsystem. Because of this a new udev |
| rules file is required and provided. |
| |
| Research |
| -------- |
| |
| Active research is ongoing which could enhance automatically backporting |
| the Linux kernel further. For details please read up on: |
| |
| - `Jesper Andersen & Julia L. Lawall - Generic Patch Inference <http://www.diku.dk/~julia/andersen_ase08.pdf>`__ |
| - `Coccinelle spatch <http://coccinelle.lip6.fr/>`__ |
| |
| Consider one set of diffs that can be read by spdiff to produce an |
| spatch. This could in theory allow us to backport all functionality on a |
| kernel if only two or three sets of files were patched and interpreted |
| by spdiff. |
| |
| Contributions |
| ------------- |
| |
| Contributions to compat follow the same mechanisms as used in the Linux |
| kernel, this means you should provide as Singed-off-by tag as documented |
| on the `Developer's Certificate of Origin 1.1 |
| <http://gerrit.googlecode.com/svn/documentation/2.0/user-signedoffby.html>`__. |
| |
| Submitting patches |
| ------------------ |
| |
| compat and compat-drivers contributions follow the contribution model |
| implemented by the Linux kernel. Patches or pull requests for compat and |
| compat-drivers must have be signed-offed. If you don't sign off on them |
| they will not accepted. This means adding a line that says |
| "Signed-off-by: Name email" at the end of each commit, indicating that |
| you wrote the code and have the right to pass it on as an open source |
| patch. For exact definition of what the Signed-off-by tag is you can |
| read the definition of the "Developer's Certificate of Origin 1.1", |
| which you can read here: |
| |
| http://gerrit.googlecode.com/svn/documentation/2.0/user-signedoffby.html |
| |
| You can send patches as follows:: |
| |
| To: mcgrof@kernel.org |
| Cc: backports@vger.kernel.org |
| Subject: compat: foo |
| |
| License |
| ------- |
| |
| The compat module and code is all licensed under the same license as the |
| Linux kernel and is considered a complete derivative work of the Linux |
| kernel. For more information refer to the COPYRIGHT file. |
