| Latest revision |
Your text |
| Line 1: |
Line 1: |
| | = The /opt Problem = | | = The /opt Problem = |
| | | | |
| - | There is not enough space on the [[rootfs]] to fit all applications, therefore an additional partition has to be used. But how? | + | There is not enough space on root to fit all applications, therefore additional partition has to be used. But just how? |
| | | | |
| - | There is a section in the Maemo 5 Developer Guide explaining the basics: | + | There is a section in the Maemo 5 Developer Guide explaining the basics: |
| - | [[Documentation/Maemo_5_Developer_Guide/Packaging,_Deploying_and_Distributing/Installing_under_opt_and_MyDocs|Installing under /opt and /MyDocs]]
| + | http://wiki.maemo.org/Documentation/Maemo_5_Developer_Guide/Packaging%2C_Deploying_and_Distributing/Installing_under_opt_and_MyDocs |
| | | | |
| | = Current situation = | | = Current situation = |
| Line 11: |
Line 11: |
| | * 32GB eMMC: /home ~2GB (ext3), /home/user/MyDocs ~29GB (vfat), swap 0.7GB | | * 32GB eMMC: /home ~2GB (ext3), /home/user/MyDocs ~29GB (vfat), swap 0.7GB |
| | * symlink: /opt/ -> /home/opt/ | | * symlink: /opt/ -> /home/opt/ |
| - | * OneNAND is probably "much" faster than the eMMC (hard numbers? read? write? small blobs? big blobs?) | + | * OneNAND is faster (hard numbers? read? write? small blobs? big blobs?) |
| - | * file system [http://talk.maemo.org/showpost.php?p=504332&postcount=76 benchmark results] for different media
| + | |
| - | * OneNAND performance figures: Sustained read performance: 108MB/s. Sustained write performance: up to 17MB/s. See [http://www.samsung.com/global/business/semiconductor/products/fusionmemory/downloads/onenand_brochure_200609.pdf Samsung OneNAND specs].
| + | |
| - | * eMMC performance figures: read performance 37MB/s. Write performance 20MB/s. Write speed of 17MB/s has been achieved in this test at [http://symbianbriefs.blogspot.com/2010/01/n900-review-part-10-transferring-data.html symbianbriefs.blogspot.com]. My own read tests however show ~11MB/s (370MB file).
| + | |
| | * Decisions were taken in a "what shall we do about /opt?" BOF at the [[Maemo Summit 2009]]. Minutes: http://lists.maemo.org/pipermail/maemo-developers/2009-October/021289.html | | * Decisions were taken in a "what shall we do about /opt?" BOF at the [[Maemo Summit 2009]]. Minutes: http://lists.maemo.org/pipermail/maemo-developers/2009-October/021289.html |
| | | | |
| Line 21: |
Line 18: |
| | * Firmware should be flashable. | | * Firmware should be flashable. |
| | * Should not break applications. | | * Should not break applications. |
| - | * One should be able to install more than 256mb worth of software. | + | * One should be able to install more that 256mb worth of software. |
| - | * Standards compliance, less need to make incompatible quirks. | + | * Standards compliance, less need to make uncompatible quirks. |
| | * A fix should be doable after shipping (before shipping, probably not?) | | * A fix should be doable after shipping (before shipping, probably not?) |
| | | | |
| - | = Current problems / developer and packaging issues = | + | == Current problems / developer and packaging issues == |
| | | | |
| | This should be a list of current problems developers are experiencing related to optification: | | This should be a list of current problems developers are experiencing related to optification: |
| Line 32: |
Line 29: |
| | | | |
| | * Symlinks take up space in the rootfs. This may become an issue if there are lots of them. | | * Symlinks take up space in the rootfs. This may become an issue if there are lots of them. |
| - |
| |
| - | * [[/Non-Optified_packages|List of non-optified packages and applications]] - Please feel free to add every piece of software which is using too much space in rootfs
| |
| - |
| |
| - | = Tool for detection of non-optified packages =
| |
| - | [[Image:Storageusage.png|thumb|300px|<br>[http://code.google.com/p/storageusage/ Storage Usage] provides a great visual way to detect non-optified packages.
| |
| - | ]]
| |
| | | | |
| | = Possible solutions, even the bad ones = | | = Possible solutions, even the bad ones = |
| - |
| |
| - | ==== Applications are made relocatable and installed to /opt ====
| |
| - | * Relocatable applications can move everywhere without breaking.
| |
| - | * No symlinks get scattered over the rootfs (symlinks take up space, too).
| |
| - | * Not all applications can easily be made relocatable.
| |
| - | * Requires more work by the programmer.
| |
| - | * May not be feasible for ports from an upstream source.
| |
| - | * Putting a startup script into /usr/bin setting up environment variables such as LD_LIBRARY_PATH can help.
| |
| - | * autotools-based packages can use --prefix=/opt/<packagename>.
| |
| - | * Python apps can make use of the [http://docs.python.org/reference/datamodel.html#index-821 __file__] module-scope variable to determine where they are installed (e.g. installdir = os.path.dirname(__file__)).
| |
| - | * Could be taken into account as an alternative to maemo-optify wherever feasible.
| |
| - |
| |
| - | ==== Have a 2-5gb partition on the emmc for addon packages mounted to /usr/local or /opt ====
| |
| - | * Most apps by default usually go there
| |
| - | * for /opt store according to the FHS standard /opt/<package> and symlink to /opt/maemo/{bin,lib,share,...} (actually the standard is /opt/<vendor>/whatever In this case using /opt/maemo directly would work).
| |
| - | * Having ld.so.conf and PATH set properly avoids all the symlink madness
| |
| - | * Don't do symlinks for /usr/local|/opt - make it a real partition
| |
| - | * Explain to people why they don't have full 32GB of storage - or use LVM on the eMMC to dynamicaly change the size as things get installed through some dpkg hooks or such - we do know how much it will take ahead of time so if there isn't enough storage do a resize(VFAT should be resizable with something like gparted maybe?) ext3/2 are both online resizeable (growth only!)
| |
| - | * Requires a bit more work when packaging - make some defaults pre-set in SDK
| |
| - | * All apps should use a normal usr layout underneath any path. Example: /opt/maemo/{bin,sbin,lib, etc...} same if it would be in /usr/local - works like GNU stow
| |
| - |
| |
| - | ==== store addon packages completely in /home/opt hierarchy ====
| |
| - | * [http://wiki.maemo.org/User:Tanner#full_optification see this page]
| |
| - | * like previous solution but on the standard /home partition
| |
| - | * very clean solution as no files (except for /var/lib/dpkg which may be moved as well) are created or modified on the NAND.
| |
| - | * /etc and /var could be copied there as well and mounted to root using bind. This way OneNAND would only be changed be reflashes and if /home is not mounted (e.g., by a keypress during boot) a fallback system could boot from unmodified OneNAND.
| |
| - | * if OneNAND is significantly faster, mark some often used shared libraries as candidates for a /usr/lib/cache directory in which they are copied during installation until a certain cache limit is reached.
| |
| - |
| |
| - | ==== During packaging move all files >500kb to /opt and symlink to root ====
| |
| - | * Optify is basically this.
| |
| - | * Install is not slowed down, work is done when packaging.
| |
| - | * Some packages break when replaced with symlinks. How to prevent - hack on hack?
| |
| - | * A buildbot could force this to debian/rules automatically.
| |
| | | | |
| | ==== Use aufs/unionfs to overlay OneNAND root with eMMC filesystem. ==== | | ==== Use aufs/unionfs to overlay OneNAND root with eMMC filesystem. ==== |
| Line 81: |
Line 39: |
| | * Might be difficult to have new programs to benefit from faster OneNAND. | | * Might be difficult to have new programs to benefit from faster OneNAND. |
| | | | |
| - | ==== Hack with --datadir, --libdir etc. through debian/rules ==== | + | ==== Move root from OneNAND to eMMC and combine with /home to create a new root ==== |
| - | * Could be forced by a buildbot. | + | * How much faster OneNAND is? Numbers? |
| - | * Should move most of the big lumps on larger media if there are some. | + | * Can flasher flash on eMMC? |
| - | * Works at least with packages with configure.
| + | |
| - | * Requires at least additional $LIBDIR setup?
| + | |
| - | * Programs which have hardcoded paths (ugly code) break.
| + | |
| - | * Could additionally symlink files from opt to root.
| + | |
| | | | |
| | ==== Require that applications have --prefix /opt ==== | | ==== Require that applications have --prefix /opt ==== |
| Line 97: |
Line 51: |
| | * Could additionally symlink files from opt to root. | | * Could additionally symlink files from opt to root. |
| | * Just reminds me of 770's /var/lib/install.. | | * Just reminds me of 770's /var/lib/install.. |
| - | * Against standard
| |
| | | | |
| - | ==== Move root from OneNAND to eMMC and combine with /home to create a new root ==== | + | ==== Hack with --datadir, --libdir etc. through debian/rules ==== |
| - | * How much faster OneNAND is? Numbers? | + | * Could be forced by a buildbot. |
| - | * Can flasher flash on eMMC? | + | * Should move most of the big lumps on larger media if there are some. |
| - | * Leave only the bootable and rescue stuff in the rootfs | + | * Works at least with packages with configure. |
| - | * Maybe use it uncompressed for speed and add a /fast/usr folder to OneNAND to store some system files that would benefit from the speed (example: move /usr/something to /fast/usr and symlink it or just place /fast/usr in the path) | + | * Requires at least additional $LIBDIR setup? |
| | + | * Programs which have hardcoded paths (ugly code) break. |
| | + | * Could additionally symlink files from opt to root. |
| | | | |
| - | ==== Use mount -o bind (or -o rbind) ==== | + | ==== During packaging move all files >500kb to /opt and symlink to root ==== |
| - | * Method: Do not use NAND for any additional data. Boot into standard image. Any additional changes made are only available after -o bind or -o rbind which are defined in /etc/fstab . Setup provides potential fail-over; if -o bind or -o rbind fails user gets into 'rescue shell' providing 1) recovery method 2) quick install to vanilla firmware without 3rd party utilities, networking, or computer (think: no network available, expensive roaming, no computer with fwflasher or Nokia Software Downloader available, ...)
| + | * Optify is basically this. |
| - | | + | * Install is not slowed down, work is done when packaging. |
| - | ==== Turn the problem around, pivot the root with /opt ====
| + | * Some packages break when replaced with symlinks. How to prevent - hack on hack? |
| - | * After pivot /opt would be root and old root would be /opt. Symlink the old root contents to new root would make those available. | + | * A buildbot could force this to debian/rules automatically. |
| - | * Can the base system dependencies handle this? | + | |
| - | * Can Nokia created software handle this? | + | |
| - | * Could this work? If so, then all additional packages would install directly on eMMC. | + | |
| | | | |
| | ==== After install move all files >500kb to /opt and symlink to root ==== | | ==== After install move all files >500kb to /opt and symlink to root ==== |
| Line 120: |
Line 72: |
| | * Some packages break when replaced with symlinks. How to prevent - hack on hack? | | * Some packages break when replaced with symlinks. How to prevent - hack on hack? |
| | * Uninstall might be messy, some post-remove magic needed. | | * Uninstall might be messy, some post-remove magic needed. |
| | + | |
| | + | ==== Turn the problem around, pivot the root with /opt ==== |
| | + | * After pivot /opt would be root and old root would be /opt. Symlink the old root contents to new root would make those available. |
| | + | * Can the base system dependencies handle this? |
| | + | * Can Nokia created software handle this? |
| | + | * Could this work? If so, then all additional packages would install directly on eMMC. |
| | | | |
| | ==== Chroot install in /opt, symlink to where it belongs ==== | | ==== Chroot install in /opt, symlink to where it belongs ==== |
| Line 126: |
Line 84: |
| | * Difficult hack.. would need all the tools etc to be visible in the chroot. | | * Difficult hack.. would need all the tools etc to be visible in the chroot. |
| | | | |
| - | ==== Make /usr another partition ==== | + | ==== Applications are made relocatable and installed to /opt ==== |
| - | You know, historically /usr exists entirely for the purpose of separating the application partition from the underlying system, which is exactly our problem.
| + | * Relocatable applications can move everywhere without breaking. |
| - | * All the core nokia software and libraries should be relocated to / or maybe a /maemo. | + | * No symlinks get scattered over the rootfs (symlinks take up space, too). |
| - | * /usr and /opt should become symlinks to /home/usr and /home/opt. | + | * Not all applications can easily be made relocatable. |
| - | * Maemo targeted GUI software would be encouraged to use /opt/[application]. | + | * Requires more work by the programmer. |
| - | * Linux libraries and CLI software could safely use /usr, minimizing the required modifications. | + | * May not be feasible for ports from an upstream source. |
| - | * Any frequently used accessed files in /home but not /home/user/MyDocs could be cached in /cache, which then gets overlaid with /home using unionfs or something more specialized.
| + | * Putting a startup script into /usr/bin setting up environment variables such as LD_LIBRARY_PATH can help. |
| - | * Another location like /alt could be used by packages that know they need the NAND's speed, but this is discouraged. | + | * autotools-based packages can use --prefix=/opt/<packagename>. |
| | + | * Python apps can make use of the [http://docs.python.org/reference/datamodel.html#index-821 __file__] module-scope variable to determine where they are installed (e.g. installdir = os.path.dirname(__file__)). |
| | + | * Could be taken into account as an alternative to maemo-optify wherever feasible. |
| | | | |
| | [[Category:Development]] | | [[Category:Development]] |
