tdm
-
Content Count
801 -
Joined
-
Last visited
-
Days Won
84
Posts posted by tdm
-
-
59 minutes ago, okayphoneme said:
Verified Boot does concern me. It's not absolutely critical, but I'd like to have it running if possible and didn't, in fact, realise that LineageOS was missing this functionality at first. Ideally I would like to have built and signed releases myself using my own keys.
That is not possible. If it were, anyone could do the same and it would defeat the purpose of verification. Only the OEM has the signing key and only the OEM can create signed images to pass verified boot.
QuoteAre the idealte bits not used at all for Lineage?
As mentioned, yes, some of the IdeaLTE bits/blobs are used to build Lineage. But only those necessary to support the hardware. In general, OEMs that do stupid and/or sneaky things place them into the upper layers of Android which Lineage does not use.
Briefly, code running on Qualcomm devices (and most/all Android devices) can be divided into four broad categories:
1. The modem co-processor
This is entirely closed source. You have no choice but to trust it if you want to use phone.
2. The trusted execution environment (TEE)
Same, entirely closed source.
3. The boot loader
This is the "BIOS" of the phone. It is mostly closed source, but I have access to some of it via the BSP. In particular, the abl code (though, of course, it is not absolutely guaranteed that idealte published the same code that they used to build the release bits). You mostly have to trust it to use your phone.
4. The upper layers
This is the code that we can control, for the most part. It can further be divided into boot/kernel, vendor, and Android/AOSP.
4a. Boot/Kernel
Lineage builds this entirely from source. OEMs can introduce security issues here, but the code is open and you can be assured that Lineage built kernels will not have any such issues.
4b. Vendor
The vendor code consists of both lower layers to support hardware and upper layers to support ... other things. Lineage needs the hardware bits -- things like camera, fingerprint, gps, and such. Lineage does not need or build the upper layers -- things like vendor-specific apps (including adups OTA support) and such. These upper layers are where vendors will typically place malware, backdoors, and such.
4c. Android/AOSP
Lineage builds this entirely from source.
So the takeaways are (1) you need to trust a certain amount of closed source bits on your device for it to function, and (2) Lineage takes the only the closed source bits that are necessary for hardware support and nothing else.
As always, remember that security and trust are never absolutes. You decide what you trust and what are acceptable risks and tradeoffs. Lineage attempts to minimize the amount you need to trust by using the most open source code possible. But in order to do this, it needs to have verified boot disabled.
EDIT: Regarding the vendor hardware bits, there are some 1000 files that are needed for full hardware support on msm8998. Qualcomm has different levels of OEM licenses that afford different levels of access to the sources for these files. IdeaLTE appears to have a rather low level (read: cheap) license. The BSP that they published has roughly half of the files closed source and prebuilt straight from Qualcomm. They have access to the source and do build roughly the other half.
-
2
-
4
-
-
10 minutes ago, silversolver said:
Can't disagree, BUT on the other hand, if you have that concern, you can always flash the image Chen provided over the top as soon as you get the device to verify that the manufacturer didn't do anything sneaky. Or are you saying they built the image and Chen may not even know what's in it?
That's not to say I'm not eagerly awaiting your success. 🙂
FxTec does not build anything. Chen only distributes what FxTec gets from IdeaLTE.
EDIT: And further, I know that FxTec does not have the device signing keys. So if they did build something, it wouldn't boot with a locked boot loader.
-
3
-
-
10 minutes ago, okayphoneme said:
There are some concerns raised here, for example - I don't know if they still apply, one year later:
Uhhhmmmm, you do realize that the Copperhead team is rather antagonistic to Lineage, right? It's not surprising that they would slam Lineage as being insecure. They do raise some valid points (particularly about verified boot), but most are bogus and/or outdated.
Verified boot requires a locked boot loader. You cannot run any custom software with it enabled, including this ROM. That may be fine at the time of release (though with a BSP that is already 4 months out of date, probably not). But as soon as the OEM stops updating, you have a choice of running with verified boot and known security issues or running with an unlocked boot loader and getting security fixes with custom software. Personally, I'll choose the latter every time.
Some responses to the other issues:
- ffmpeg, is long since gone, it has not been present since CM 13.x (that's before Cyanogen died and Lineage was created).
- CAF code is OEM code that gets released. There are no alpha/experimental commits, but there are many security fixes. You want the newest code to be up to date and have the latest security patches.
- For firmware, Lineage cannot ship proprietary bits. Only your OEM can do that. But there are different levels of firmware. Much of the stuff in vendor will be freshly built and up to date in Lineage long after the OEM stops updates.
- Lineage does not, in general, disable or weaken any selinux rules that affect device security. That would be silly.
I realize that everyone has different opinions on security. If you feel that Lineage is not secure enough for you -- perhaps you value verified boot -- then by all means, run stock. But as for me and quite a few other folks, we value the up to date security fixes that Lineage offers long after the OEM has stopped updating.
I'll also mention that I personally don't trust a no-name Chinese OEM (sorry idealte, it's true) to not do something stupid and/or sneaky. There are plenty of examples of smaller OEMs putting back doors, malware, and other things into stock software. Here's a quick example that popped up on the top of my duckduckgo search:
-
1
-
4
-
2 hours ago, okayphoneme said:
I appreciate the explanation, thank you.
I've been reading up on Lineage and I'm not super keen on it now - the project seems to put an emphasis on features over security - which is fine, but I think I'm actually interested in using something like the Ungoogled ROM you've posted here. The thing is, I'd like to make some modifications to it (entirely for my own purposes). I can see the stuff in the Android source that I would change but my knowledge of how the whole thing slots together is rather limited.
My question is: is there a way of merging in the proprietary (binary) bits of this BSP into AOSP so that I can obtain something similar to what you have here, or is not having access to the source going to scupper idea that completely?
That's an interesting perspective. May I ask how you came to that conclusion? My feeling is exactly opposite.
The OEM may or may not provide security updates for a period of time after release. My copy of the BSP is on tag LA.UM.7.4.r1-05500-8x98.0 which was released August 23, 2019. Incremental updates may have newer code, I'm not sure because I don't do OTAs. But either way, IdeaLTE will stop publishing updates at some point.
As a member of Lineage and former Cyanogen employee, I can assure you that many/most of the Lineage developers care a great deal about security. You can see the evidence -- Lineage faithfully merges security patches every month, even for older releases. So at some point Lineage is assuredly going to be more secure than stock.
The direct answer to your question is yes, but not easily. You can create your own source tree based on the publicly available qcom BSP, merge in any missing security patches, and then add in the vendor blobs just like Lineage does. But it's quite a lot of work. I did exactly this some years ago for my devices using jellybean and kitkat. If you are interested in going this route, I would suggest looking on XDA for one of the "pure" Android ROMs and use that as a starting point. And even after you do all that, you have a set of vendor blobs that are frozen in time and may have security issues which cannot be patched at all.
-
2
-
3
-
-
8 hours ago, Polaris said:
Edit: Since OTAs are out (which I like), can it be repaved to axe A/B support and gain the extra internal 4 Gb as well?
I can certainly look into this. I also hate to waste 4gb of storage for no benefit to me (because I never use OTAs). But do understand that it's pretty low priority.
Also note that it will require repartitioning the device, which makes most folks nervous. And I'll need to make a non-AB version of lineage.
In other words it's a fair bit of work and will take some time.
-
3
-
-
3 minutes ago, silversolver said:
@tdm do you expect to have VoLTE working in your initial phone-working release? I will be unable to use LOS on my device otherwise, so may have to wait to install it otherwise.
I don't use VoLTE, so I'll need others to test. But I'll do my best to make it work.
-
2
-
-
Just a quick update...
I had two weeks off for the holidays and I am back this week.
I've experimented with building a vendor image but that looks like it will take a fair amount of time. So next I looked at a method of overlaying changes over the vendor files instead of building the entire vendor image. This is easier and significantly reduces the amount of data transfer (both from github to me and from my website to you) so I will probably continue down that path for the official Lineage release.
But for now, I'm pretty tired of playing with vendor files so I'm going to start fixing issues that have been raised on github. First up is voice calls (hey, it's a phone, so it should do phone stuff, right?) I hope to get that fixed tomorrow.
-
10
-
-
6 minutes ago, Ralf said:
Preface: I am sorry if this is a stupid question, please point me to any helpful information sources in that case.
What would be the advantage of using this ROM instead of a LineageOS build without GoogleApps?
Something along the lines of better hardware support comes to mind, but then it was my understanding that the proprietary blobs needed to support the hardware are fully available already. Otherwise LineageOS would not run on the device I'd imagine. Or are there maybe specific android software (functional) additions in the stock build that are only available that way?
I was answering this very question as you posted. Lineage isn't quite ready yet so there's that. But after it's working well, I don't think there will be much of any reason to use ungoogled stock. Some folks have expressed that they like the landscape lock in the stock launcher, so there's that. But I imagine that some sort of landscape lock will probably be added to Lineage at some point.
-
3
-
-
1 minute ago, okayphoneme said:
I see. So Lineage in theory does everything that the BSP does, but is an open source implementation?
Well...... yes but not exactly.
Lineage is built on the open source portion of Android (AOSP) and the portions of the Qualcomm code that are released as open source (the CodeAurora project, or CAF).
The portions that Qualcomm does not release as open source are taken as binary blobs from the stock firmware and integrated into the Lineage build to make a complete package. Because this is a sort of legal gray area, these blobs are kept strictly partitioned from the Lineage project and made available under the github account TheMuppets (really!)
The portions that Google does not release open source are known as Google Apps or gapps. These are released as a separate flashable package. This is done for historical reasons, because several years ago Google sent a nasty cease-and-desist letter to the CyanogenMod project (which is the precursor to Lineage).
So if you flash Lineage plus gapps, you will get something that does everything the stock ROM does plus a ton of fixes and enhancements made by the Lineage team.
You can also use Lineage without gapps which will result in something like the ROM posted here plus the Lineage fixes and enhancements. Given this, I don't really know why someone would want to use the ungoogled stock ROM. But since there have been requests and it's rather easy to do, I've gone ahead and made it available.
-
3
-
5
-
-
6 minutes ago, elvissteinjr said:
I take the EDL tool is not something that exists for the Pro1 yet, right? Having something like that could come handy, but I don't want to be the one to burden you with that stuff as I'd probably wouldn't end up using it much.
I have not made one for the Pro1 yet. Not out of complexity or anything, just because it's not needed. The original stock files are made available by FxTec, which is a much easier way to retrieve them than messing with EDL.
-
2 minutes ago, david said:
Good idea. I have a stupid question about A/B updates that I guess I've never thought about. Consider this scenario:
1) You have an original version in A and B.
2) You are running on A.
3) You get an OTA that does an incremental update on B, B gets marked as active, and you reboot into B.
4) You now get an incremental OTA. This incremental OTA can't be applied to A, because A doesn't have the previous update yet. What does the update process do in this case?
I'm not really an OTA expert, but I believe the OTA takes the active slot data plus the patches and applies them to the inactive slot while the device is in use. When complete, it switches the active slot and asks to reboot. There is no such thing as a slot-specific update.
-
1
-
-
14 minutes ago, elvissteinjr said:
Magisk root modifies the boot image and does not change the system partition in any way. This leaves us with the issue of not being able to pull the new boot image from the device as root is gone after the OTA... and in order to restore root we need to flash a patched boot image... which can only be the old one in this case.
Unless some other way to grab the partition data surfaces or someone succeeds stringing together the intercepted incremental OTA packages, I suppose.
In any case, doesn't seem like something you can help us with after all, unfortunately. Just hope Fx will provide updates eventually then.
Thanks anyway.This is one of the nice things about A/B devices. You can root one slot and extract images from the other slot. In the case of an OTA, you can root the older inactive slot and extract the new one.
And in any case, if you want a way to read and write the boot partition, I can do that with EDL. I've done it with several devices in the past. Example:
EDIT: Also, I forgot to mention that FxTec has made fastboot packages for stock available, which contain the boot image and much more. There is no reason to believe they could not or would not continue to do this with future updates.
-
4
-
-
15 minutes ago, okayphoneme said:
Is the source available for this so we can build it ourselves?
Unfortunately, no. The BSP is proprietary. FxTec has shared the BSP code with a few developers, but it is not generally available. This is one of the reasons that Lineage exists.
And please don't get upset at FxTec and IdeaLTE. The bulk of the BSP is licensed to OEMs by Qualcomm. It is only through the generosity and trust in developers that FxTec has made the BSP available to developers at all. They did not need to do this and, in fact, are not really supposed to do it. Also note FxTec and IdeaLTE could release their device code if they chose, but it wouldn't do much good without the proprietary Qualcomm code.
-
2
-
5
-
-
11 minutes ago, elvissteinjr said:
What was asking was if the boot image can be used with the official Android builds as Fx themselves appear to only provide updates through OTA so far.
For Magisk however, we need a boot image file to patch, so keeping up to date with the kernel while keeping root doesn't appear possible right now. And I personally want to root right away when I get my phone.Not asking for direct assistance on that, but knowing if it could work that way is useful. Perhaps Fx does eventually update their flashable files, but it could take a while, knowing them.
With the lack of a device I'm on in immediate need of this yet, but I think there would be a few people here who would appreciate if you could also provide an up to date boot.img as a side effect of this rom.
Thanks for your efforts again.I'm not going to try right now, but the short answer is no because the kernel binary is not signed with the device keys, which means the stock kernel modules in /vendor would not load and WiFi would not work. You could work around this by putting the stock kernel binary into my boot image. However, at that point, you may as well use a stock boot image. Additionally, the following things come to mind:
- The adups properties are not present. This will likely prevent OTA updates from downloading. Which is fine, because:
- The build is signed with test keys, so recovery will refuse to flash stock packages and updates.
- GMS property ro.com.google.gmsversion is missing which may make GMS fail to work, I don't know.
- Property ro.build.description shows test-keys. I think Play Store uses ro.build.fingerprint so this may be okay.
Note the stock boot image is available in another thread on this forum so you should probably use that instead. And once you get root, you can pull the latest boot image off your device anyway.
So overall, I really don't see the point in trying to do this.
-
1
-
1 minute ago, elvissteinjr said:
That's neat. Thank you!
Not sure if I'll use this (lack of device makes it hard so far), but does this "stock" mean plain Android without any extras or like the official Fx images but without Google and OTA?
Not quite the point of this ROM and since the sources are from November it doesn't actually help, but I was wondering if they were more current, would the boot.img be pretty much the same as the official one and be usable for Magisk patching?I'm not quite sure what you are asking.
No, this is not "plain" Android -- it is the full stock BSP build minus adups and gms.
The sources are relatively stable at this point. I just did a sync a few minutes ago and there are a couple of recent changes: (1) a fix to the sensors config to avoid a crash when referencing the motion sensor, (2) a fix to the keyboard kernel driver to enable suspend/resume, and (3) a fix to allow storing camera pictures on the sdcard. I'll probably incorporate these into a new build at some point, but they are relatively minor.
The boot image (and everything else) is indeed the same as official stock. I don't have access to the device signing key used by stock so it is signed using "test keys". But other than that, and the aforementioned changes, it is exactly stock.
I don't use Magisk. You can try using it if you like but you'll be on your own.
-
2
-
-
I am pleased to announce a new member of the Pro1 ROM family: "Ungoogled Stock".
What is it?
This is the stock ROM as built from the BSP directly from IdeaLTE with the following changes, and only these changes:
- Removed AdUps OTA update code.
- Removed GMS (Google Apps).
Where is it?
The current version is:
http://files.nwwn.com/android/pro1/QX1000-user-nogms-noadups-20191208.zip
MD5: 01c90007a843328b869bb294c8f88330
SHA1: 3aff440c50ec2464437c5698bc1c7740ed827c34
Source synced to BSP as of Nov 29, 2019.
How do I flash it?
Unzip the package. You will find 5 img files. Flash each of these, set the active slot, and reboot.
Note:
- It is recommended to flash both boot slots, but this is not technically necessary.
- Data wipe may be necessary if the device does not boot.
Example:
adb reboot bootloader fastboot flash boot_a boot.img fastboot flash dtbo_a dtbo.img fastboot flash system_a system.img fastboot flash vbmeta_a vbmeta.img fastboot flash vendor_a vendor.img fastboot --set-active=a fastboot reboot
Can I add Google Apps later?
It would be theoretically possible but it would require wiping data and disabling some security measures (verity and avb). I am not prepared to support this. If you want Google Apps, use official stock or Lineage.
-
6
-
9
-
24 minutes ago, Polaris said:
The link that @Craig posed above comes up as 404 Not Found. The URL is: http://files.nwwn.com/android/pro1/QX1000_user_20191028_oldfirehose_base.qfp
It's to the 'oldfirehose' file, that's the link he said isn't working. The 20191028 file that you are referencing (and is downloadable) is at: http://files.nwwn.com/android/pro1/QX1000_user_20191028.qfp
Yes, I removed the oldfirehose_base file. That was just for testing. Both of the qfp files linked from the web page have the old firehose now.
-
1
-
-
4 hours ago, Polaris said:
The file permissions aren't set for us to be able to see/download the file. The only visible files are two .html files.
I'm a bit confused. I just downloaded the 20191028 file successfully.
The index links to the LineageOS page and the Factory Restore page. But if you followed the link in the OP of this thread you should land on the Factory Restore page directly. The links for all the restore files are there and should be working.
-
1
-
-
3 hours ago, Craig said:
The link is dead, at least for me.
Yes the old firehose was verified to work. I updated the main package. Please download a fresh copy and it should work.
-
2
-
-
15 minutes ago, netman said:
About that... Can we always get into EDL mode without disassembling the device and if not, how deep do we have to dig in trainwreck scenarios? Or do you just need a special cable in such cases?
My web page answers that, mostly. There are situations where the device must be disassembled to enter EDL, but they are exceedingly rare. Almost always either the PBL will fail to load XBL and panic to EDL (this is what devyl 's device did), or XBL will load and check for the data short on the USB connector that the EDL cable provides.
So really you only need to disassemble the device if XBL is functional enough for PBL to load but not functional enough to make it to the USB check.
-
3
-
1
-
-
Oh, and just so everyone knows...
This tool and the linked factory packages are meant to do a "normal" factory restore. It does not, by design, reset every single bit that the factory programs. For example, it will not reset the frp partition which contains the boot loader lock flag and the Android factory reset allowed flag. Nor will it reset the radio parameters. Etc.
I wrote this tool and I have full access to the underlying capabilities of the programmer. If a normal factory reset does not work, I can generate packages to read and write every single bit on the flash chip.
With EDL, you quite literally cannot brick or otherwise break this device in a way that cannot be fixed.
-
5
-
2
-
-
20 hours ago, Polaris said:
Wait, what?!? What shop? Where you bought it? As in you returned it and are done with it for good, or is this just temporary? I hope it's temporary as @tdm was/is working diligently to get this corrected for you, and the solution might very well help everyone.
@tdm I have been watching and reading this issue with interest, as always thanks for all your support!
I'd like to expand on this a bit.
If you screw up your device then you are responsible for fixing it. Returning the device to the retailer is bad for several reasons: (1) you screwed up and you are making someone else eat the cost of your mistake which is not cool. (2) the retailer has less incentive to stock the device (and any non-mainstream device generally) because they are seen as less reliable and thus less profitable. (3) the reputation of FxTec suffers.
So resist the temptation to take the device back and claim that it "just stopped working". Not only is it untruthful, we have a unique tool here that can restore your device to a factory clean state. The vast majority of people will never have this type of tool for their devices.
Consider yourself fortunate and don't screw it up for everyone.
[Soapbox mode off]
-
4
-
2
-
-
@devyl I have added the 2019-12-03 package. You can try it to see if it works.
-
2
-
-
27 minutes ago, devyl said:
The Flashing works, but the devices still doesn't make a real reboot - it just stays in EDL-Mode (or something like that; it just gets detected again in the flash tool). Is there any way to reset the state of the device?
Hard reboot is done by holding the power button down for about 10 to 15 seconds. But I suspect that won't work either.
I can make a package for the 20191203 build, perhaps that might work?
-
2
-
ROM: Ungoogled Stock
in Support
Posted
This is news to me. I'm not terribly familiar with qcom devices on this level, but in the past when I was working at Cyanogen, I had a co-worker sign a new device with a freshly generated key. He told me that it was a one-time operation that could not be reversed or changed. I'd be interested in hearing more about this. But even if it is possible, I have strong doubts that any mainstream device would be setup to do this.