With the release of Android Oreo and many devices such as the Xiaomi Redmi Note 3, Google Nexus 5 and others unofficially receiving it, it's probably fair to wonder why the same features (mostly the camera) tend to be broken when developers port an Android Open Source Project (AOSP) based ROM. You've probably seen XDA forum threads of ROMs with a long list of broken features up top. "What works" followed by a list of working features, then down below that the iconic "What doesn't work? You tell me!" are two popular refrains on our forums that have practically become a meme on places like Reddit and Twitter.
Why is so much functionality broken whenever a developer tries to port an AOSP ROM onto their device? The basic answer is that because functions change across different versions of Android, old device drivers packaged as BLOBs won't work with newer versions of Android, or even just with stock AOSP. To overcome that, developers make use of what's called a "shim," but the process involved is tricky, time intensive, and sometimes very difficult to debug.
In this article, we will outline how shims work, especially with regards to getting the camera working properly on AOSP-based ROMs. We will be using the OnePlus 3T as an example. Note that the difficulty involved in getting these features working is highly device specific.
OnePlus 3T Running OxygenOS. Although OnePlus phones are known for their custom development friendliness, there is a lot of work that developers do behind the scenes to make stable ports of AOSP.
What is a shim or a BLOB?
To even begin to understand part of what developers are doing, we first need to explain a few things. Although the Android OS is open source (it's called the Android Open Source Project for a reason), the software (sans the kernel) shipped on thousands of Android devices is not. Developers do not have access to the source code of Samsung Experience, EMUI, OxygenOS, or any of the other third-party flavors of Android.
Now, developers porting stock AOSP to a non-Google device probably don't care about the source code of these Android skins as they aren't going to be modifying and building these ROMs. That would be true, if not for one big, big reason: the parts necessary to get the camera working properly, mainly the camera HAL (Hardware Abstraction Layer), are also closed source.
The problem with having not only the camera HAL but also the ROM closed source is that developers working on porting AOSP onto their device will be working blind. The closed source OEM ROM is able to interface with the camera HAL just fine because the OEM has access to the camera HAL source. The camera HAL is what allows the ROM to "talk to" the camera hardware—without it, the camera would be non-functional. Think of the camera HAL as the steering wheel and pedals of the car. The steering wheel/pedals allow for control of the internal components of the vehicle by providing an external interface for the driver (the ROM) to make use of the internal components.
Graphic showing the camera architecture. Source: Google
As camera hardware becomes more and more complex (the advent of dual cameras, for instance), having access to the camera HAL source would make porting an AOSP ROM with a functional camera a much easier endeavor.
However, OEMs do not provide access to the camera HAL source for various reasons. First, if they do not have all of the ownership rights to the camera HAL (such as when they incorporate intellectual property from other companies), then they cannot distribute the source. Second, releasing the camera HAL source may jeopardize their own intellectual property. Finally, companies are under no legal obligation to provide this source code (unlike the kernel source code which they are obligated to release under the GPL), thus they have no incentive to release it. So without access to the camera HAL source, how exactly do developers get the camera working on AOSP ROMs? The answer is a BLOB, shim, and lots and lots of debugging.
A device BLOB (Binary Large OBject) contains pre-packaged binaries which are the compiled form of software. In this case, the camera HAL source is compiled by the OEM and shipped on devices as binaries. When developers talk about BLOBs, they refer to those binaries that ship on live devices that they are able to extract. Now, the topic of "camera BLOBs" has long plagued OnePlus for many months, but the truth of the matter is that developers have always had access to the camera BLOBs. The camera HAL source code is the golden ticket for developers here, though, but that will never, ever be released due to the legal jeopardy it would put companies like OnePlus in.
Thus, developers looking to bring AOSP onto a device are left only with BLOBs of the camera HAL for which they do not have access to the source code. Rarely if ever can a developer pair their AOSP ROM code with the camera HAL BLOB and expect it to work, so in order to bridge the gap between the two, developers create what is called a "shim."
To "shim" is to "wedge (something) or fill up a space." This is effectively what a developer does when writing a shim—they add code to allow for the BLOB to interface with the AOSP source code they're working with. Shims are used to make BLOBs of all different sorts work with AOSP, but usually, it's the camera BLOB that requires the most shimming. As we mentioned before, shimming is required not only for porting newer versions of Android onto a device (such as all of those unofficial Android Oreo ROMs) but also needed when porting AOSP of the same Android version onto that device.
Recommended Reading: From Store to Shelf: An In-Depth Capitulation of Why MSM8974 Devices Are Excluded from Nougat
The OnePlus 2, for example, received its last official major OS update in the form of Android 6.0 Marshmallow. The device, however, actually has fully working custom AOSP-based ROMs based on Android Nougat, and that is thanks to the hard work of developers and their shims. We will be breaking down some examples of shims, but first, we need to talk about how exactly shims work.
How does shimming work?
Since developers do not have access to the camera HAL or OEM ROM source (and only the precompiled binaries), they can't know what functions the camera HAL expects. Because of this, there's often a mismatch between the name of the function that the camera HAL is looking for and the actual name of the function in the AOSP code the developer is working with.
To solve this issue, the developer simply creates a new function that uses the same name of the function that the camera HAL BLOB expects, but this new function just executes what the developer wants it to. This new function that acts as a middleman between the BLOB and AOSP is the shim. This particular scenario wherein the BLOB fails to find the function it is looking for is one of the most common ones where a shim is needed.
Very simple MS paint diagram showing where a shim is needed.
Perhaps things will make a bit more sense with a hypothetical example involving the OnePlus 3T. We will create an example using OxygenOS and the OnePlus camera. If we use camera BLOBs taken from OxygenOS Nougat for the OnePlus 3T to build an AOSP based Nougat ROM, we may run into issues. This is because the camera BLOBs (which were originally compiled by the OEM) will be able to reference all of the functions it needs within OxygenOS, but since the compiled AOSP ROM may not have those functions or may have compiled them under a different name (thus leading to a mismatch between function symbols), there will be an error. This can be fixed by creating a new function within the AOSP ROM with the name the BLOB expects—our shim.
Symbols in a programming context are used to refer to specific functions in code. Symbols are necessary because the position of a function can change when the code is edited, and so in order to avoid hardcoding references to functions, the compiler creates a table of symbols that other functions can use to always refer to the right function. When you change the name of a function before compiling, its symbol changes too, so basically any changes that the OEM makes to the camera HAL source before compilation will require developers to create a new shim.
Viewing a Symbol Table with Hopper. Source: Apriorit
The explanation we've offered thus far makes it sound like creating shims is easy. Changing a few function names here and there doesn't sound too hard, right? If only it were that easy. The reality of shims involves more than just function renames. We spoke with XDA Recognized Developer Sultanxda who was able to provide us an example of one of the more difficult shims he has worked on.
Shimming – Not as Easy as it Sounds
For those unfamiliar with the OnePlus 3T, the front-facing camera was rather broken initially on AOSP-based custom ROMs. To start with, attempting to take any picture over 8MP would result in crashing. In his attempt to solve this issue, Sultanxda made several shims to allow the OnePlus 3T front-facing camera to work properly.
Shim #1 – Changing the Camera Package Name
In order to stop the front-facing camera from crashing whenever the user took a picture over 8MP, Sultanxda forced the camera HAL to identify all cameras as the OnePlus camera. This is done because OnePlus decided to dedicate a helper function to certain applications (isOnePlusCamera, isFacebookCamera, etc.) for some reason. Sultanxda fixed this by shimming the camera HAL so it points to a new function which always returns "true" as if the user is using the OnePlus camera—even when they are not.
Shim #2 – Disable QuadraCfa
For his next shim, he had to disable QuadraCfa which is presumably a proprietary Qualcomm technology relating to the camera. We say presumably because neither myself nor Sultanxda is exactly sure what QuadraCfa is, but Sultanxda does know that it broke the front-facing camera whenever it was enabled.
He observed that QuadraCfa would somehow enable itself, but he wasn't sure why or how it was doing so. Solving this required a rather unconventional modification on his part. In a conventional shim, the shim function, when compiled, provides the missing symbol that the BLOB is looking for. In this case, the BLOB already had the symbols it needed—the ones which presumably represented the functions which were starting QuadraCfa.
Bless Hex Editor. The program Sultanxda used.
Thus, he needed to override the symbols used by the camera HAL and in essence, make them "missing" so his shims would provide those "missing" symbols. The only way to do that is via hex editing the camera HAL itself. Hex editing is basically looking through a bunch of unorganized gibberish in the form of binary data in order to find a needle in the haystack—either a function or a string you are looking to edit.
Hex editing a function is substantially more difficult than hex editing a string, but fortunately, Sultanxda was able to avoid having to hex edit the functions behind QuadraCfa by instead hex editing the symbol names to void those symbols.
Shim #3 – Bright Light Crash Fix
Next up, Sultanxda identified that taking a picture from the front-facing camera when under bright lighting conditions would cause the camera to crash. In order to reproduce this bug on his own device, Sultanxda actually turned on the flashlight function of his OnePlus One and shined the light in front of the OnePlus 3T's front-facing camera in order to make it crash and produce usable logs! Once he discovered what function was causing the crash, he created a shim to force the device to use low light mode all the time for the front-facing camera.
Shim #4 – Low-Resolution Front-Facing Camera Pictures
After fixing the bright light crash with the previous shim, Sultanxda discovered another bug which actually arose as a direct result of that shim: low-resolution front-facing camera pictures. Rather than taking pictures at the user requested resolution (eg. 16MP), the resulting picture would be taken at 4MP.
Solving this required that he shim the functions handleSuperResolution and isSuperResolution to always return true, but ONLY when the front-facing camera is active (because otherwise, the camera would crash when taking pictures from the rear sensor).
Lesson Learned – Shimming can be Hard
Sultanxda admits that the shims he had to create to get the OnePlus 3T front-facing camera working don't represent your typical example of a shim. He's rather proud of his shim given its complexity and the rare necessity to hex edit the BLOB itself. But this example just goes to show how difficult it can be to get the camera hardware working on certain devices.
May your camera shim adventures be less painful than mine were. -Sultanxda
Logs, logs, and more logs. Without a consistent way to reproduce a crash and without logs, developers have little hope of finding the source of the issue. Even if they do find what causes the problem, it's not always a straightforward fix. The whole process of finding and squashing these bugs can take days or weeks and is the reason why fixing the camera on AOSP ROMs is one of the more difficult tasks.
If your device has an AOSP ROM ported to it with fully functioning hardware, hopefully, you can begin to appreciate the struggle that those developers might have gone through in order to bring you those features. Appreciate them for their work, because it's not easy. It's a lot of work which the vast majority of users won't even notice, as talented developers on our forums are taking care of the many unseen parts of Android.
We would like to give special thanks to Sultanxda for the many contributions he suggested in the making of this article.
from xda-developers http://ift.tt/2hs6l6b
via IFTTT
No comments:
Post a Comment