The N9/N950 Kernel Upstreaming Force
 

tl;dr: Do you want to help upstreaming the existing N9 drivers?

=======================================
Specification of the issue

Bibliography - in post #2

As you may know, there was an initiative to update the N9/N950 kernel. [1] While the project succeeded [2] (only the GPS driver was missing), the patches were never sent upstream. This happened due to various reasons. [2]

This means that we're stuck with a 3-year-old kernel on our devices. Some of the drivers depend on closed source blobs. (which of them are needed is not documented very well)

This is a blocker for some projects. Such an example is DebiaN900, an initiative to bring native Debian to N900. If the drivers were upstreamed, this initiative would've extended to N9/N950 a long time ago. [3] In fact, wicket himself said he would support the Harmattan devices if the kernel were properly upstreamed. [4]

Would you like to help us upstream the kernel? Some cases are easy (it's just the coding style that needs fixing), some cases may be challenging. If you don't have a physical device but would like to help anyway, let us know. Maybe we'll find something for you too :)

Tasks pending [stage 1]:

• create a table on elinux.org indicating which areas are possibly not covered by the upstream kernel [free for taking]
• contact N900 kernel hackers about what has already been done by the N900 folks [free for taking]
• add bibliography about switching to DeviceTree [free for taking]
• developing a testing procedure [community]

Future tasks [stage 2]:

• real kernel development [on hold]
• real testing (N9) - peterleinchen [ for taking, multiple people]
• real testing (N950) - marmistrz [for taking, multiple people]

Current team:

• marmistrz [working on the BH1770 driver]
• filip.pz [working on the N9 display]
• hedayat (will confirm once the work has started)
• danpio

Re: The N9/N950 Kernel Upstreaming Force
 

Bibliography:

[1] the N9 kernel update project: http://talk.maemo.org/showthread.php?t=86971
[2] information from the N9 kernel update developer: http://talk.maemo.org/showpost.php?p...&postcount=133
[3] wicket's post about N9/N950 support: http://talk.maemo.org/showpost.php?p...&postcount=129
[4] wicket about upstream kernels (N9, N900): http://talk.maemo.org/showpost.php?p...&postcount=131
[5] pali about upstreaming kernels: http://talk.maemo.org/showpost.php?p...&postcount=134
[6] support for N9/N950 in current mainline kernel (DeviceTree): http://talk.maemo.org/showpost.php?p...64&postcount=5

-----------
DeviceTree:
[7] DeviceTree syntax example: http://elinux.org/Device_Tree_Usage

Mods are welcome to append useful information.

Re: The N9/N950 Kernel Upstreaming Force
 

Currently, I cannot take any responsibility (might be able to do so later). But, if the effort started, I'll try to help in some areas.

Re: The N9/N950 Kernel Upstreaming Force
 

Yes. Put me on testing for N9.

Re: The N9/N950 Kernel Upstreaming Force
 

Some device tree work has been mainlined within the last couple of years:

https://git.kernel.org/cgit/linux/ke...s/omap3-n9.dts
https://git.kernel.org/cgit/linux/ke...omap3-n950.dts
https://git.kernel.org/cgit/linux/ke...3-n950-n9.dtsi

This commit seems to imply that the mainline kernel is actually bootable:

https://git.kernel.org/cgit/linux/ke...737887fea9d46d

I think a good starting point would be to build a kernel from mainline and see if it boots on an N9 or N950. I would test it against either Nemo or Sailfish as it's more likely to be compatible with one of those rather than Harmattan. Of course you could also bootstrap a Debian system and try that. ;)

Anyone got a spare N9 for filippz?

Re: The N9/N950 Kernel Upstreaming Force
 

No need for that. One of my N9s is functional - I jus't cant use CMT (ie no call, SMS, 3G). For some reason it just reboots after entering SIM pin (tried in Harmattan & Nemo), but everything else is fully operational.

Thanks anyway!

Re: The N9/N950 Kernel Upstreaming Force
 

Would realky love to see this developed further.
Please send me your details, and I will ship fully working N9/64G/black device to you.

Re: The N9/N950 Kernel Upstreaming Force
 

Still, it's a shame that your interest has been reduced and you're having to use Android (far from a pleasant experience :D). I think you deserve a fully working N9!

Re: The N9/N950 Kernel Upstreaming Force
 

Thanks nieldk, but as mentioned earlier - there is no real need for that. I can use my current N9 for development (and it's has already everything set up nicely - ubiboot/easyboot paired with Harmattan/ Nemo/Sailfish).

Fully working N9 would probably be used as daily driver, and not for development :) What can I say - I'm weak :o

Re: The N9/N950 Kernel Upstreaming Force
 

Feel free to use it that way :)
My offer stands still

Re: The N9/N950 Kernel Upstreaming Force
 

Back to business...

Does anybody know Aaro? Seems that he would be an excellent addition to this effort.

Re: The N9/N950 Kernel Upstreaming Force
 

Perhaps catch his attention through linkedin ?
https://fi.linkedin.com/pub/aaro-koskinen/48/73b/ab8

Re: The N9/N950 Kernel Upstreaming Force
 

This means good news! :)

I'll update my scripts to install Debian onto N950 to support N9 too. I'll push them for review here. If they're correct, I'll test it on my N950 and ask peterleinchen to do it on his N9.

But, which config will we use for building mainline kernel for the N9/N950? Maybe we could copy the config from the 3.5.3?

But, what about kbd on N9 & Debian?

Added post #5 to bibliography. Besides, I'd love to read some good introduction to DeviceTree. As for know I know that it exists.

I have another idea to boost this project: a donation pool. Those who can't help directly could pledge a donation for this project. It would be disposed when the project succeeds. But I have several apprehensions:
1. Won't this deprive us of any possible helping hands?
2. Who should be eligible to get a part of the donations?

I created a template but not pushing anything to the public yet.

Re: The N9/N950 Kernel Upstreaming Force
 

Or maybe mail him directly? The address is available here: https://git.kernel.org/cgit/linux/ke...737887fea9d46d

Re: The N9/N950 Kernel Upstreaming Force
 

As first step, I tried building 4.3.0-rc1 kernel and boot Nemomobile (with Easy Boot) on it - but a few second into the boot, N9 just turns off (the same thing happened with 3.14.0). I didin't expect much (working USB, maybe) but this is still disappointing.

I used old "mer" defconfig for N9 updated with make olddefconfig. A few manual tweaks/bits were required as described here: http://elinux.org/N900#DT_Kernel_Compilation, and compilation was done using sb2.

I'm unsure what can be used to debug this kind of behaviour apart from using working serial console.

Has anybody else tried building/booting it?

Re: The N9/N950 Kernel Upstreaming Force
 

Hmmm that reminds that I need to create that serial connection to N9 :(
No, unfortunately, at this early stage, I think serial console is only option.
I tried to enable USB serial, in kernel, but obviously that failed.

Re: The N9/N950 Kernel Upstreaming Force
 

Well, I haven't build anything yet. I was waiting for some replies about the config to use. (see #13). And I won't be able to try anything for a week.

I'd see one theoretical scenario. Can we stop the device from rebooting if boot fails? Can we get some boot log on screen (on N900 one did this by adding "modprobe fbcon" at the beginning of /sbin/preinit)

If yes, then we could get some information where the boot fails.

/edit: Filip, did you append the device tree (.dtb) and enable the option in the kernel config? See "Device tree for dummies" for basic information. I still don't know how to enable building the .dts and .dtsi to .dtb, though.

Re: The N9/N950 Kernel Upstreaming Force
 

That's the usual way to go. Having said that, Fremantle (and I think Harmattan too) provides a mechanism for logging kernel messages to one of the mtd partitions on the NAND flash. I haven't looked into how this works but it might be possible to use this to log the kernel messages during boot up in order to debug the boot process without a serial console.

Unless Filip has a better suggestion, I would just take the config file from 3.5.3 as you suggested. The config options have changed since then but it should serve as a starting point. When you build the kernel it will prompt you set any new options.

The N900 DT Compilation instructions should help you with that.

Re: The N9/N950 Kernel Upstreaming Force
 

I managed to get logs http://pastie.org/10433050, but I'm AFK right now. If anyone sees something obvious just let me know. I'll be able to continue later today.

Re: The N9/N950 Kernel Upstreaming Force
 

It looks like switch_root is being called with incompatible arguments, probably due to the previous mount errors.

Make sure you have CONFIG_BLK_DEV_LOOP enabled. That should hopefully fix the /dev/loop0 errors.

Re: The N9/N950 Kernel Upstreaming Force
 

CONFIG_BLK_DEV_LOOP is set to y.

Here's boot with 4.3.0-rc1: http://pastie.org/pastes/10433203
Seems that mmc driver is having some DMA trouble:
omap_hsmmc 480b4000.mmc: unable to obtain RX DMA engine channel 3230369304

I'm guessing that needs to be taken care of first of all.

Re: The N9/N950 Kernel Upstreaming Force
 

OK, mmc issue sorted out by adding
CONFIG_DMADEVICES=y
CONFIG_DMA_OMAP=y

This time Nemo does get started, but dies soon after (http://pastie.org/10433263). I guess watchdog shuts it off before DSME is able to start and prevent that.

Anyway - I now have a working serial console cable, and we have kernel that kinda boots. I hope to make some more progress in the following days (time permitting).

Re: The N9/N950 Kernel Upstreaming Force
 

The first error I noticed was

It seems as though the kernel didn't find the right partition.
Would it be possible to dump the partition table before booting? I can't really do it myself now, but you may post a question on unix.stackexchange.com

Re: The N9/N950 Kernel Upstreaming Force
 

That problem has been solved, as described in my last post:

Seems that MMC driver works only with DMA enabled.

I've also captured boot of 3.5.3 kernel for reference (http://pastie.org/10433952).

I think that this is the one to solve next:

Re: The N9/N950 Kernel Upstreaming Force
 

This is just a warning, that some service didn't stop within a given timeout. You will find that even in normal log iam sure.

Re: The N9/N950 Kernel Upstreaming Force
 

It's not present when booting 3.5.3. I'd expect for kernel to kick the watchdog just before init, so it doesn't time out. Seems to me that's just the case here.

Re: The N9/N950 Kernel Upstreaming Force
 

Possibly its related to the systemd errors. While annoying, I don't think its a stopper.

But, what about the unknown partition table listed?

Re: The N9/N950 Kernel Upstreaming Force
 

This seems to be related:
https://github.com/filippz/kernel-ad...030_wdt.c#L235
I'll try to implement this into new kernel and see

Re: The N9/N950 Kernel Upstreaming Force
 

The N900 which also uses twl4030_wdt, doesn't need that hack. Check that you are not doing anything differently. Here's the latest N900 config file which you might find useful for reference given the hardware similarities:

https://github.com/pali/linux-n900/b...rx51_defconfig

Re: The N9/N950 Kernel Upstreaming Force
 

In fact N9 uses TWL5031, but the driver is the same. Both 2.6 and 3.5 had some kind of WD hacks:
https://github.com/filippz/kernel-ad...030_wdt_hack.c
https://github.com/filippz/kernel-ad...514acf4db038b0

The one from 3.5.3 seem to help a bit, ie boots a bit further.

Only difference I see is that CONFIG_OMAP_WATCHDOG and CONFIG_TWL4030_WATCHDOG are set to m whereas I used y. I'll retest with that, but not until tomorrow.

Re: The N9/N950 Kernel Upstreaming Force
 

wicket, how did you know that it was switch_root and why did the config option fix it?

filip.pz: what about disabling watchdog altogether? In the N900 world one did it with a flasher.

Re: The N9/N950 Kernel Upstreaming Force
 

The kernel log displayed the usage of BusyBox's switch_root command. Command usage is often displayed when a command is executed with invalid arguments.

The "losetup: /dev/loop0: No such file or directory" error suggested that CONFIG_BLK_DEV_LOOP might not be enabled/loaded (see the help text for this config option for a description of what it does). It turned out that this wasn't the problem as Filip already had this enabled. He fixed it by enabling DMA.

Re: The N9/N950 Kernel Upstreaming Force
 

Just an update on progress so far (or lack of it).

I've ported hack from 3.5.3 that deals with broken watchdogs, but it hasn't made any difference. To be sure that watchdogs are not to blame, I've used flasher to disable them (good idea marmistrz):
but that didn't help.
Also wehen 3.5.3 gets compiled with: I also get warnings about watchdogs (thanks nieldk), twice about two independent watchdogs:Funny thing is that in 4.3.0-rc2: I get two warnings of single watchdog:I'll rechek if kernel knows about both watchdogs (omap_wdt and twl4030_wdt), and why I get warning for just one of them as opposed to 3.5.3 where both warn about Unexpected close

If nothing helps I'll try to disable systemd services that can't work at this stage, but can make problems (loading kernel modules, starting SGX driver) - still I would expect to see somthing in console, if that's the case

Re: The N9/N950 Kernel Upstreaming Force
 

I don't have any computer access right now, but I'll try the same with Debian when I have it. This way we'll make sure it's not Nemo-related.

Re: The N9/N950 Kernel Upstreaming Force
 

That would be awesome!
I just tried disabling the wd hack in 3.5.3, and it boots, so event without that hack boot shouldn't fail (If I got it right that hack is just for suspend state when watchdogs can't be disabled, but userspace is suspended and doesn't kick them from time to time)

Re: The N9/N950 Kernel Upstreaming Force
 

Filip, did you put up your sources of the 4.x kernel on GH? If no, the please just list all changes from mainline 4.x / the old config

Re: The N9/N950 Kernel Upstreaming Force
 

Nothing public yet. I'll send you and e-mail containing two patches (one for building, and one with wd hack) in a few minutes.

Re: The N9/N950 Kernel Upstreaming Force
 

OK, new theory:

In 3.5.3 watchdogs were miscdevices and accessible trough filesystem. When filesystem got re/u/mounted drivers were getting release file operation called, and kicked the watchdogs, just before init - leaving enough time for userspace to take over.

Now watchdog drivers are in their own kernel class, and only first registered watchdog is linked to /dev/watchdog miscdevice for legacy reasons. So, in our case with 2 watchdogs, only omap_wdt gets kicked and twl4030_wdt doesn't. That doesn't leave enough time for userspace to start DSME that does the kicking in Mer based distros.

Maybe krenel should kick all watchdogs one last time before calling init. I'll test that tomorrow.

Re: The N9/N950 Kernel Upstreaming Force
 

I got Nemo to boot (kinda: http://pastie.org/10440946). Since OBS for Nemo is in a poor state, a lot of packages are not being updated to current versions so I added Jolla repo to update them. This prevents N9 from shutting down during the boot process (DSME gets loaded earlier preventing watchdogs to kick in).

Obviously, graphics are broken and that also prevents MCE from running (even if MCE complains about "config subsystem".) For now I'll try to build new SailfishOS image and continue developing on it, but I imagine that Debian would also be able to boot (probably to nothing usable, but still...)

Re: The N9/N950 Kernel Upstreaming Force
 

With respect to Debian, I have started to package some of the core components from Nemo (dsme, mce-dev. mce, statefs) as part of my Gtk3 work. There not definately not perfect yet and untested on arm as I've been running in a x86_64 container.

dsme I know needs some manual editing after installation as the packaged systemd service file is looking in the wrong place (ie /usr/lib instead of /usr/lib/x86_64-linux-gnu).

Need to change container at the moment as statefs requires fuse (fine with qemu but aiming to get working via lxc) and to work out why mce is playing up (fails to start, causes boot to be delayed by 3min then starts fine)

If they are of any use to anyone, there on my github page...once I remember to upload a wip oneshot package (nemo one is full of bashisms I need to learn to fix).