• Intro
• Downloads
• Community
• Development
• News
• TALK

Maemo Reconstructed

Revision as of 17:20, 3 November 2008

This wiki page has the purpose of bringing together developers both internally in Nokia and community developers, who have an interest in the Maemo platform. The goal is trying to reach a vision of how a future Maemo system architecture could be like, that's satisfying to both Nokia and community. This is meant as the actual low level -technical- system goals (use of upstart, busybox specifics, initfs or not) not high level items such as "ogg in media players" or "open source everything".

This discussion was sparked by:

• http://wiki.maemo.org/Mainstream_Linux_Alignment
• http://qgil.jaiku.com/presence/47635104
• http://lists.maemo.org/pipermail/maemo-developers/2008-October/035409.html .

The Maemo "Reconstructed" Platform

My idea of Maemo "Reconstructed" is to make Maemo a general OS for tablet devices, including Nokia devices, and to make it more developer-friendly and be more in alignment with standard Linux distributions to ease upstream and downstream adoption. It is also within the philosophy that the platform should be "hackable", a main selling point with OSS friendly hardware.

My belief is that we should stop seeing the tablets as strictly embedded low-cpu systems - they are at least as powerful as those 166mhz machines we had in the end of the 90s, - and were fully capable of running unix base systems without extreme slow downs. -- stskeeps 17:15, 3 November 2008 (UTC)

Distribution and archives

stskeeps 20:21, 1 November 2008 (UTC)

I believe the methods/categories/distributions mentioned in https://stage.maemo.org/svn/maemo/projects/haf/doc/mvo/system-model-2.txt could come to good use in Maemo "R". Maybe a part about Maemo vs Maemo + Nokia closed source components and seperation in archives?

There should exist scripts/documented ways how to create a Maemo "R" image than can be adapted to any setup or device.

Development of the Maemo "R" platform (non-device-specific and UI differentiation) should be done in the open, with public SCM, bugtrackers and blueprints. This way developers can adjust their applications to later 'stable' releases and know what is coming regarding the Maemo API.

Kernel

Current N8x0 partition is at 2Mb. For the Maemo "R" platform specific kernels does not matter, but it may be relevant to discuss certain features that would be nice to have for the platform to work correctly.

Initfs / Boot process / Rescue menu

stskeeps 16:06, 31 October 2008 (UTC)

I believe that a initfs is a useful thing for a tablet. It can help you in many cases when the main system has crashed for whatever reason, and help you fix the problem or rescue your data. For Maemo "R" I believe it should contain the following things:

• Boot menu alike fanoush's bootmenu , with a /etc/bootmenu.d like setting (inspiration from grub menu.lst, http://internettablettalk.com/forums/showthread.php?t=23051)
  • Motivation: Cloning to SD is a popular activity amongst community users, to gain more application space. A bootmenu item system should be brought to Maemo to avoid a lot of hacks laying about for the same thing, which makes it difficult for SSU to deal with.
  • Note: Bootmenu should be unintrusive and invisible to the "normal" user (but may be activated to be more verbose in control panel)
• Rescue menu, possibly alike http://www.youtube.com/watch?v=h24f2YjzWBE (or in text form). That can both activate certain services from initfs or set boot settings such as framebuffer console or other settings for the system (think F8-while-boot in windows.)
  • Motivation: Many community users often run into the dreaded reboot loops. This can be solved by having various things in initfs to solve the general problems. One can be usbnet/g_serial (for emergency shell - or PC accesory programs for dumping the files from NAND /home/user/ through usbnet telnet) , acting like a file storage (expose LUN of MMC cards), dump NAND contents to files on MMC, - the possibilities are endless. These abilities should be modelled with a "plugin" structure, to encourage innovation in this area.
• Contain closed-source hardware specific firmware, drivers (cx3110x, etc), daemons (BME, hw watchdog kicker), but should not try to load wifi drivers and such initially except for FS drivers needed to boot, leaving it up to the post-initfs OS to do this.
  • Motivation: Fact is, there as certain trade secrets and we're not going to get around them easily. Let's have them in initfs as a kind of "Hardware Abstraction Layer" for each kind of device. Kernel drivers should ideally be open source but this may not always be possible. Recompilation services for newer kernels could be considered - maybe easier access for community versions? (automated "I would like this driver for kernel X, and I hope that the API didn't change inside the kernel so it breaks..", output .ko)
• Busybox+uclibc to ensure minimal initfs. Toolkit for the right combination should be as easy to get as Maemo SDK. Busybox shouldn't be stuck at 1.0 like it is in initfs right now.
  • Motivation: This will definately aid in the development of rescue menu abilities.

Behaviour and documentation:

• Communication protocols with closed source daemons should ideally be documented (it should be possible to get remaining battery time and bars to show and such from BME.).
  • Motivation: This will satisfy most OSS OS developers, but the problem about closed source daemons is that there should optimally be updates, even for old devices (backwards compatible daemons maybe, maybe automated build services again/community/head contributors access to recompile, so Nokia wouldn't have to invest time.)
• The initfs should mount the rootfs at /mnt/rootfs, pass on control to a /linuxrc (which runs in initfs namespace), which will then deal with the typical items such as pivot_root and passing control to init/upstart.
  • Motivation: Many of the items in current init/linuxrc in traditional initfs is very Maemo specific. If someone wants to run Mamona, Angstrom or Poky, why not let them without having to do Maemo-specific tricks to unwrangle things to look like they were just booted from kernel?
• Daemon communication from post-initfs should rely on daemon unix sockets being in /mnt/initfs/tmp/, not /tmp as many typical
  • Motivation: Other OS'es than Maemo has a tendancy to clean out /tmp when booting.
• Initfs should be updated with apt-get in Maemo "R", but updated (/mnt/initfs paths in packages?), not reflashed with a static image.
  • Motivation: This will ease work on rescue menu abilities, work on pre-OS processes.
• Basic device/Mobile service DSME should be open sourced, but may load modules of closed source modules (CAL access, watchdog kicker, etc), and DSME protocol should be documented and the modules associated with it (CAL access seems to be a touchy subject though).
  • Motivation: DSME is a good part of a mobile platform (process watchdog, etc), but needs documentation of the communication protocol (even though most of it has been reverse engineered) for community OS developers to work properly with it. Can it really be true things like Advanced Backlight (http://adv-backlight.garage.maemo.org/) has to use undocumented DSME protocol to do what it should?

Initfs should be a initial step towards booting, a rescue possibility and a way to get the system in a sane state to get Maemo "R" or anything else started. Initfs should fit in 4mb flash with space to spare. Initfs can also be a alternate root device to boot from when a key combination is pressed.

Initfs may also (with larger size) contain kernels to boot (think bootloader with path to kernel to load)

Maemo Base System

stskeeps 09:50, 1 November 2008 (UTC)

I believe there is a need for change - bringing Maemo more in alignment with "major" distributions. The Maemo Base System is a good place to start this change. Maemo is it's own, it is targeted towards embedded systems, but on the other hand Maemo runs on systems that are as powerful as the computers these distributions were initially developed on, and many of the typical embedded tricks may simply not be needed and it could help seeing Maemo as system targeting power-saving mobile devices where it's fine to be able to run a traditional "desktop" *nix base - which would make porting much easier.

I agree with mvo's statement (see jaiku discussion), quote "As to what should happen: Nokia should embrace the Debian heritage of Maemo fully and put processes and practices into place that make Maemo a Debian derivative like Ubuntu, only better." I have for this proposal tried to see Maemo as a deriative of Ubuntu. The reasons I've chosen this direction is:

• The existence of the research project of http://mojo.handhelds.org, sponsored by Nokia
• Debian, for now, targets armv4t, and mojo has already done the hard work to target typical Maemo device processors.
• Taking Ubuntu over Debian (for now), gives us upstart along (part of Task:Maemo_roadmap/Fremantle ), which has not been adapted properly (except in Debian experimental) in Debian just yet.

Maemo base system includes:

• 'minbase' Mojo handhelds Ubuntu 'Hasty' (based on Hardy, or later)
  • Motivation: This is a proof of concept. How functional and how well performing is a standard minimal ubuntu base image on a N800? Our restrictions is about 250mb flash availiable to us, with ideally space to spare.
  • Details:
    • # cp /usr/share/debootstrap/scripts/hardy /usr/share/debootstrap/scripts/hasty
    • # debootstrap --arch=arm --variant=minbase hasty /target http://repository.handhelds.org/hasty-armv5el-vfp/
    • # chroot /target mount -t proc proc /proc
    • # chroot /target mount -t devpts devpts /dev/pts
    • # apt-get install apt-utils udev
    • # cd /dev; /sbin/MAKEDEV fb0 fb1 and such.
    • # apt-get autoclean; apt-get clean
    • add in a couple of boot scripts from and framebuffer console kernel drivers from Deblet, and docpurge-1.6 from Maemo.
    • Now you have a bootable minimal ubuntu image (tested on N800 with bootmenu with linuxrc ability), with full coreutils and bsdutils, etc.
  • Performance: From selecting in boot menu to boot, 12 seconds to get to framebuffer console login, with ext3 mount, udev and all.
  • Size: uncompressed 89.5mb but, mkfs.jffs2 -r /target -o maemo-r.jffs2 -e 128 -l -n , 37.0mb in JFFS2.
  • Contents: http://bsd.tspre.org/~stskeeps/minboot.txt + docpurge
  • Other examples, http://bsd.tspre.org/~stskeeps/ubuntumin.txt (69m jffs2, may have forgotten docpurge in this one.)
  • Advantages: We now have a full ubuntu base system on our tablet. We've got Upstart for for free without any development. We have a base we can easily work with upstream on, as we base on typical packages. The mojo project also gets to have instant value to the Nokia/Maemo development and Nokia gets a return from the research investment in it.
  • Discussion: Is busybox really needed on a tablet? In initfs, sure, but in the actual system? Having sane coreutils and bsdutils (and debconf) -really- helps porting. Now that dash exists, is there a reason for busybox as sh? This avoids any conflicts with coreutils, bsdutils etc if they are going to be used.
    • A simple (which may be unfair and inaccurate) experiment with dpkg-deb -x of procps, coreutils, util-linux, bsdutils into a path, and then mkjffs'ing it after stripping man pages, locales and info pages, 2.2mb flash, and with busybox getting same treatment, 300k flash.
    • Is 1.9mb flash so much to sacrifice for making porting for Maemo a matter of hildonization and platform-centric things, instead of hunting debconf, busybox incompatibilities and such for weird libraries you never heard of?
    • Are there any direct and noticable differences in performance that justifies the use of busybox, and do they justify the lessened ability of being able to port to Maemo without hassle?
  • Problems: For some reason the mojo people decided to divert from Debian arch notation and call their ARM EL distribution arch type 'arm', not 'armel', which is in wide use with Maemo.
• Open Hardware Manager / OHM
  • Motivation: Part of Task:Maemo_roadmap/Fremantle. http://ohm.freedesktop.org/wiki/UseCases , - they're obvious. OHM may apply device-specific plugins but these are not included in the Maemo "R" base. Developers should strive towards not using sysvinit init scripts, but go for upstart job descriptions instead, which can be controlled by OHM (disable RSS feed reader download attempts while obviously offline, for instance).
• GUPnP
  • Motivation: Part of Task:Maemo_roadmap/Fremantle
• Meta Tracker (trackerd)
  • Motivation: Part of Task:Maemo_roadmap/Fremantle
• D-Bus
  • Motivation: Part of the Maemo platform as of now, - is ideal for embedded systems as it works with signals and events.
• HAL
  • Motivation: Part of the Maemo platform as of now, needed for OHM. HAL should be targeted to embedded systems but should not contain device specific things. These should be published as HAL addons (and may be closed source, example could be hald-addon-bme as it done right, and hald-addon-omap-gpio not), as to keep as close to HAL upstream as possible.
• Generic API for connectivity (libconic comes to mind), using D-Bus. It should not have to include a ICD as some vendors may vary on that (Nokia), but libconic should be generic and usable in both Nokia ICD and NetworkManager based setups. Ideally a NM-based ICD or a stub ICD should be provided along with platform, but should be replaceable. Under this things like iptables, udhcpc, dnsmasq would be.
  • Motivation: Many Maemo applications depend on libconic and this could be a showstopper for portability.
• Udev
  • Motivation: Part of Maemo platform already, and useful for desktop and embedded systems.
• Ke-recv & Alarmd
  • Motivation: Some daemons representing these (Kernel event handling + scripts, maybe OHM?), and a API for setting device wakeups. Maybe even a cron-like interface?
• GConf
  • Motivation: Central repository for system-wide configuration.

Other things like supporting libraries such as openssl and such are not directly mentioned but obviously needed as dependancies.

Maemo "Reconstructed" User Experience Base

stskeeps 20:53, 1 November 2008 (UTC)

• Xorg: Part of Fremantle roadmap, I believe.
  • Motivation: Device-specific Xorg drivers (omap for N8x0's, other devices has other drivers, etc..). It should not be a requirement which Xorg driver is in use, but the capabilities of this.
• GStreamer
  • Motivation: Part of Maemo platform as of now, ideal for codecs, include vendor-specific propertiary ones.
• PulseAudio
  • Motivation: Part of Task:Maemo_roadmap/Fremantle. Optional part of platform? What does Maemo really target of device types?

(not done yet)

Interesting fact: Ubuntu minbase as mentioned earlier, plus apt-get --no-install-recommends hildon-desktop (that is Ubuntu Mobile), though without an X server which should be trivial to install. 271.5m uncompressed, 110.4M JFFS. dpkg -l at http://bsd.tspre.org/~stskeeps/ubuntumobile.txt - Obviously some things should be stripped down, as UM focuses on a flash-like interface, as far as I recall, in this distro version.

Maemo "Reconstructed" SDK (developer kit for the platform)

stskeeps 17:31, 31 October 2008 (UTC)

The SDK is a basic part of developing for Maemo. But there are occasional horrorstories. Based on some of the stories I've heard and other trends in the mobile device world, I've tried to come up with a vision of a proper SDK for Maemo "R".

Maemo "R" SDK in my mind, should consist of four parts:

• Scratchbox (cross-compilation environment) - build programs for another architecture than on the target one.
• Maemo "R" rootstrap - Maemo base OS with libraries and dev packages to cross-compile any applications for Maemo "R", not to run them.
  • Motivation: Cross-compiliation certainly speeds up compilation and allows a proper developer environment for developing for Maemo "R". "Not to run them" is meant as, CPU transparency is okay, but the rootstrap should not try to be a fullly deployed Maemo "R" system. The rootstrap shouldn't try to act as an tablet - we have the emulators for that.
• Qemu appliances with target device(s) emulated, with a device-specific Maemo "R" installed on it. This is where built applications should be run, not inside the SDK (no Xephyr, not full hildon desktop, etc. in Scratchbox). These should be configured with sbrsh for cpu transparency from initial setup.
  • Motivation: N8x0 emulation is already seen (http://blog.haerwu.biz/2008/04/11/nokia-n800-emulation/). Qemu appliances will allow the user to instantly test how their applications work on the actual device or an emulated version.
• Scripts to ease deployment of and debugging built applications inside Scratchbox+rootstrap to emulated and actual devices.
  • Motivation: It's much better when you can test on an emulated device instead of dealing with a rootstrap trying to act like it's a device.
  • Motivation: Easy development ability, and it's a question that's often seen when people start developing for Maemo (how do we transfer to the device? Try see Visual Studio's deployment ways)
  • Idea: work on easy deployment of developed packages (# sb-copy-to-tablet -i usb, -i emu, etc) or running in similar manner to sbrsh with NFS and the likes.

Contents of Maemo "R" rootstrap:

• A Maemo "R" SDK may have Nokia/device specific dev packages but developers should be encouraged to develop for the defined Maemo "R" platform. Nokia/other hardware vendors specific dev packages should strive towards non-device-specific standardization of the interfacing if possible (a BME dev package may not be relevant on other devices).
  • Motivation: Maemo has great possibility to become a general platform for tablet like devices. The possibility of additional device platforms which also embrace the Maemo minimal base + build-essential + dev packages direction will cause Nokia devices to have additional applications as well.

Good development behaviour:

• Packages for Maemo should not expect Scratchbox to be where they're compiled (I've seen examples of this amongst Nokia code.)
  • Motivation: Increases portability of Maemo platform. A package that builds in Scratchbox should also build on a tablet.

Distribution of SDK:

• Maemo "R" SDK should be easy to install. It should be a matter of apt-get'ing a package or installing a VMware appliance for non-Linux users. It's alright for device-specific dev packages & emulator images to require a license to download, but the base Maemo "R" SDK should be easily installable and allow you to start developing for the base Maemo "R" platform immediately.
  • Motivation: You shouldn't have to go through hoops to get Scratchbox + Rootstrap + Generic Maemo device emulator image and such, - this attracts developers.
• Embrace VMware Applicances/VirtualBox with SDK + Emu inside, for windows / other OS users to get developing for Maemo "R".
  • Motivation: Linux isn't the only platform people use for development.