Uploading to Extras-devel

(Change to drop here too)
("non-free" packages)
 
(34 intermediate revisions not shown)
Line 1: Line 1:
-
[[Extras]] is the recommended home for all Maemo software. It provides an easy mechanism for users to discover new applications, and can help ensure quality and continued support for applications and their authors.
+
[[Extras]] is the recommended home for all Maemo software. It provides an easy mechanism for users to discover new applications, and can help ensure quality and continued support for applications and their authors. Before an application can get to Extras, it must be uploaded to [[Extras-devel]] and promoted to [[Extras-testing]]. Community testing will begin, and finally the package can be promoted to Extras.
== Pre-requisites ==
== Pre-requisites ==
Line 11: Line 11:
<small>''This step is unnecessary if you are only going to use the web-based [https://garage.maemo.org/extras-assistant/index.php Extras Assistant] wizard to upload.''</small>
<small>''This step is unnecessary if you are only going to use the web-based [https://garage.maemo.org/extras-assistant/index.php Extras Assistant] wizard to upload.''</small>
-
Put your SSH key<ref>[http://kimmo.suominen.com/docs/ssh/#ssh-keygen Generating SSH keys with ssh-keygen]</ref> online:
+
Put your [http://kimmo.suominen.com/docs/ssh/#ssh-keygen SSH key] online:
* If you need to create an RSA SSH key (make sure it is an RSA key):
* If you need to create an RSA SSH key (make sure it is an RSA key):
-
*# <code>ssh-keygen -t rsa</code> (enter, enter, passphrase (if desired), enter).
+
*# <code>ssh-keygen -t rsa</code> (enter, enter, passphrase (if desired), enter). This can be done from within MADDE as well.
* Copy the contents of <code>~/.ssh/id_rsa.pub</code> in the "Public SSH keys" section of your [https://garage.maemo.org/account/ garage account maintenance] page. This will be available once your upload-invitation request has been accepted
* Copy the contents of <code>~/.ssh/id_rsa.pub</code> in the "Public SSH keys" section of your [https://garage.maemo.org/account/ garage account maintenance] page. This will be available once your upload-invitation request has been accepted
 +
 +
'''NOTE: If you modify public key in garage account, server will need some time to refresh that information (it should be done in few hours) otherwise you might get message "''Server refused our key''" (when you try to upload package).'''
=== Setting up Scratchbox to use the Extras repository ===
=== Setting up Scratchbox to use the Extras repository ===
Line 34: Line 36:
== Preparing a source package ==
== Preparing a source package ==
 +
 +
{{main|Packaging}}
 +
There are three main routes to creating source packages:
There are three main routes to creating source packages:
# Using Debian tooling directly.
# Using Debian tooling directly.
Line 40: Line 45:
=== Debian tooling ===
=== Debian tooling ===
-
Once your source tree contains a <code>debian</code> directory, create your package with dpkg-buildpackage in the project source directory:
 
-
dpkg-buildpackage -rfakeroot -sa -S -i
+
{{main|Documentation/Maemo_5_Developer_Guide/Packaging%2C_Deploying_and_Distributing#Creating_Debian_Packages}}
-
 
+
-
If you use git then you may not want to include the entire git repo in your source bundle. With dpkg-source version 1.13.25 the -i option isn't git-aware so you can do:
+
-
 
+
-
  dpkg-buildpackage -rfakeroot -sa -S -i -I.git
+
-
 
+
-
You may verify that your <code>Build-Depends</code> field in debian/control is complete by running
+
-
 
+
-
dpkg-depcheck -m dpkg-buildpackage -rfakeroot -b
+
-
 
+
-
in the source tree.  (You'll need to <code>fakeroot apt-get install devscripts</code> for this to work).
+
=== mud-builder ===
=== mud-builder ===
Line 62: Line 56:
=== py2deb ===
=== py2deb ===
Python applications can be developed inside or outside of Scratchbox. [http://wiki.maemo.org/Py2deb py2deb] makes it easy to build packages ready for upload from a Python script.
Python applications can be developed inside or outside of Scratchbox. [http://wiki.maemo.org/Py2deb py2deb] makes it easy to build packages ready for upload from a Python script.
 +
 +
=== PyPackager ===
 +
[http://wiki.maemo.org/PyPackager PyPackager] essentially replaces py2deb and provides the capabilities to build both a finished Debian Package (.deb) file as well as the source tarball and control files required by the garage build queues as py2deb does.
== Upload ==
== Upload ==
=== "free" packages ===
=== "free" packages ===
There are three mechanisms for uploading packages:
There are three mechanisms for uploading packages:
-
# The [https://garage.maemo.org/extras-assistant/index.php Extras Assistant] is a web-based wizard, which guides you through the process.
+
# The [https://garage.maemo.org/extras-assistant/index.php Extras Assistant] is a web-based wizard, which guides you through the process
-
# Debian's <code>dput</code> tool<ref>[http://extras-cauldron.garage.maemo.org/HOWTO.html#how-to-prepare-and-upload-source-packages-for-build Upload to autobuilder with dput]</ref> which uses SSH.
+
# Debian's <code>[http://extras-cauldron.garage.maemo.org/HOWTO.html#how-to-prepare-and-upload-source-packages-for-build dput]</code> tool
-
# SSH directly (using <code>scp</code><ref>[http://lists.maemo.org/pipermail/maemo-users/2006-September/002080.html Use scp to upload a package]</ref>).
+
# SSH directly (using <code>[http://lists.maemo.org/pipermail/maemo-users/2006-September/002080.html scp]</code>
==== dput ====
==== dput ====
Line 74: Line 71:
  [DEFAULT]
  [DEFAULT]
-
  default_host_main = diablo-extras-builder
+
  default_host_main = fremantle-extras-builder
   
   
-
  [chinook-extras-builder]
+
  [fremantle-extras-builder]
  login = <your_garage_login_name>
  login = <your_garage_login_name>
  fqdn = drop.maemo.org
  fqdn = drop.maemo.org
Line 82: Line 79:
  hash = md5
  hash = md5
  allow_unsigned_uploads = 1
  allow_unsigned_uploads = 1
-
  incoming = /var/www/extras-devel/incoming-builder/chinook
+
  incoming = /var/www/extras-devel/incoming-builder/fremantle
   
   
  [diablo-extras-builder]
  [diablo-extras-builder]
Line 92: Line 89:
  incoming = /var/www/extras-devel/incoming-builder/diablo
  incoming = /var/www/extras-devel/incoming-builder/diablo
   
   
-
  [fremantle-extras-builder]
+
  [chinook-extras-builder]
  login = <your_garage_login_name>
  login = <your_garage_login_name>
  fqdn = drop.maemo.org
  fqdn = drop.maemo.org
Line 98: Line 95:
  hash = md5
  hash = md5
  allow_unsigned_uploads = 1
  allow_unsigned_uploads = 1
-
  incoming = /var/www/extras-devel/incoming-builder/fremantle
+
  incoming = /var/www/extras-devel/incoming-builder/chinook
-
 
+
 +
[gregale-extras-direct-binary-upload]
 +
login = <your_garage_login_name>
 +
fqdn = drop.maemo.org
 +
method = scp
 +
hash = md5
 +
allow_unsigned_uploads = 1
 +
incoming = /var/www/extras/incoming/gregale
Then run:
Then run:
Line 110: Line 114:
  dput PACKAGE_VERSION_source.changes
  dput PACKAGE_VERSION_source.changes
-
and it should go to diablo-extras-builder (because of the DEFAULT:default_host_main setting.
+
and it should go to fremantle-extras-builder (because of the DEFAULT:default_host_main setting.
==== scp ====
==== scp ====
Line 130: Line 134:
  [fremantle-extras-devel-non-free]
  [fremantle-extras-devel-non-free]
  login = <your_garage_login_name>
  login = <your_garage_login_name>
-
  fqdn = garage.maemo.org
+
  fqdn = drop.maemo.org
  method = scp
  method = scp
  hash = md5
  hash = md5
Line 143: Line 147:
Please make sure your dependencies are correct. Should there be a dependency problem between a free and a non-free package, the non-free package will most likely be removed first. This is because we can recompile free packages, but cannot recompile non-free packages since we do not have the source.
Please make sure your dependencies are correct. Should there be a dependency problem between a free and a non-free package, the non-free package will most likely be removed first. This is because we can recompile free packages, but cannot recompile non-free packages since we do not have the source.
 +
 +
 +
 +
 +
'''For Windows 7'''
 +
 +
You will need these programs:
 +
 +
1) [http://the.earth.li/~sgtatham/putty/latest/x86/puttygen.exe puttygen.exe]
 +
 +
2) [http://the.earth.li/~sgtatham/putty/latest/x86/pageant.exe pageant.exe]
 +
 +
3) [http://the.earth.li/~sgtatham/putty/latest/x86/pscp.exe pscp.exe]
 +
 +
 +
First run puttygen.exe, and make private and public keys ('''RSA'''). Save public and '''private key''' as files somewhere (and remember location!).
 +
 +
Open public key file with some text editor (notepad will do job), and now copy key.
 +
 +
This is how public key file should look like:
 +
---- BEGIN SSH2 PUBLIC KEY ----
 +
Comment: "rsa-key-20120905"
 +
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdas
 +
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdas
 +
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdas
 +
asdasdas
 +
---- END SSH2 PUBLIC KEY ----
 +
You only need:
 +
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdas
 +
Just make sure that doesn't contain blank characters - eg. new line or space.
 +
Now you can put that in garage account with prefix "ssh-rsa", so at the end it should look like this:
 +
'''ssh-rsa''' asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdas
 +
 +
 +
Now start pageant.exe and add that private key.
 +
 +
Open console (click on windows logo, and in search filed type "cmd" - without quotes - and press enter). Now go to folder where did you downloaded/putted pscp.exe file.
 +
 +
 +
To upload file, run this command:
 +
 +
pscp -v -scp '''<deb_full_path>''' '''<username>'''@drop.maemo.org:/var/www/extras-devel/incoming-nonfree/fremantle/
 +
 +
Where you need to replace:
 +
 +
- '''<deb_full_path>''' - with absolute file path (example: ''C:\MyWork\my_app.deb'')
 +
 +
- '''<username>''' - your maemo garage username
 +
 +
 +
After that, wait for package to be uploaded and thats it :)
== Autobuilder special considerations ==
== Autobuilder special considerations ==
Line 152: Line 207:
Also make sure the <code>Section:</code> in the control file is '''all lowercase''', e.g. "user/navigation" not "user/Navigation", otherwise after building it will silently fail and NOT be moved to extras-devel.
Also make sure the <code>Section:</code> in the control file is '''all lowercase''', e.g. "user/navigation" not "user/Navigation", otherwise after building it will silently fail and NOT be moved to extras-devel.
 +
 +
The Autobuilder supports automatic [[Documentation/Maemo_5_Developer_Guide/Packaging%2C_Deploying_and_Distributing/Installing_under_opt_and_MyDocs|Optification]] of packages, using the tool [http://maemo.org/packages/view/maemo-optify/ maemo-optify]. You just need to create the file "debian/optify" containing the word "auto" to trigger maemo-optify during build.
 +
To predetermine the results of enabling maemo-optify, you can install this package from the Extras-devel repository inside your scratchbox and run maemo-optify-deb on a precompiled package.
 +
 +
If you have problems with the autobuilder, ask for help on the [[Community mailing lists|maemo-developers mailing list]], and possibly contact the [[debmaster]] if you are sure that the problem is with the autobuilder and not your package.
''In addition, one additional idea to increase package quality is to introduce [[Maemian]], a [http://en.wikipedia.org/wiki/Lintian lintian]-like tool for Maemo . This would help in doing additional checks for common inconsistencies and errors.''
''In addition, one additional idea to increase package quality is to introduce [[Maemian]], a [http://en.wikipedia.org/wiki/Lintian lintian]-like tool for Maemo . This would help in doing additional checks for common inconsistencies and errors.''
Line 168: Line 228:
* [https://garage.maemo.org/promoter/diablo diablo] - Maemo 4.1
* [https://garage.maemo.org/promoter/diablo diablo] - Maemo 4.1
-
Promotion from Fremantle Extras-devel to Extras-testing and Extras-testing to Extras is done through the [http://maemo.org/packages/ maemo.org package interface] and the process is described at [http://wiki.maemo.org/Extras-testing#Promoting_packages_to_extras-testing Promoting packages to extras-testing].
+
Promotion from Fremantle Extras-devel to Extras-testing and Extras-testing to Extras is done through the [http://maemo.org/packages/ maemo.org package interface] and the process is described in the [[Extras-testing#Promotion_.2F_Demotion_NOT_FINAL_FIXME|Extras-testing documentation]].
-
== Resources ==
+
== Additional information ==
* [https://garage.maemo.org/extras-assistant/index.php Extras assistant]
* [https://garage.maemo.org/extras-assistant/index.php Extras assistant]
* [[Extras repository process definition]]
* [[Extras repository process definition]]
* [https://garage.maemo.org/account/ Garage SSH key]
* [https://garage.maemo.org/account/ Garage SSH key]
* Build logs: [http://garage.maemo.org/builder/chinook/ chinook], [http://garage.maemo.org/builder/diablo/ diablo], [http://garage.maemo.org/builder/fremantle/ fremantle]
* Build logs: [http://garage.maemo.org/builder/chinook/ chinook], [http://garage.maemo.org/builder/diablo/ diablo], [http://garage.maemo.org/builder/fremantle/ fremantle]
 +
* [[Debmaster|The debmaster]] is responsible for any problems with the autobuilder
-
== Reference ==
+
[[Category:Packaging]]
-
{{refs}}
+
-
 
+
-
[[Category:Development]]
+

Latest revision as of 21:20, 9 September 2012

Extras is the recommended home for all Maemo software. It provides an easy mechanism for users to discover new applications, and can help ensure quality and continued support for applications and their authors. Before an application can get to Extras, it must be uploaded to Extras-devel and promoted to Extras-testing. Community testing will begin, and finally the package can be promoted to Extras.

Contents

Pre-requisites

Three things are necessary to upload to maemo.org Extras:

  • A Garage account: [signup]
  • An "invitation" to upload: [request]
  • ssh-keygen (optional: not needed if only going to use the web-based upload)

Setup

Setting up SSH to upload to extras

This step is unnecessary if you are only going to use the web-based Extras Assistant wizard to upload.

Put your SSH key online:

  • If you need to create an RSA SSH key (make sure it is an RSA key):
    1. ssh-keygen -t rsa (enter, enter, passphrase (if desired), enter). This can be done from within MADDE as well.
  • Copy the contents of ~/.ssh/id_rsa.pub in the "Public SSH keys" section of your garage account maintenance page. This will be available once your upload-invitation request has been accepted

NOTE: If you modify public key in garage account, server will need some time to refresh that information (it should be done in few hours) otherwise you might get message "Server refused our key" (when you try to upload package).

Setting up Scratchbox to use the Extras repository

Add the extras repository to your Scratchbox install:

  1. Login to Scratchbox
  2. Create /etc/apt/sources.list.d/extras-repo.list
  3. Update apt's information.

Note: this should be done in both your ARMEL and x86 targets.

/scratchbox/login
cat >/etc/apt/sources.list.d/extras-repo.list <<EOT
#Maemo extras
deb http://repository.maemo.org/extras/ fremantle free non-free
deb-src http://repository.maemo.org/extras/ fremantle free
EOT
fakeroot apt-get update

Preparing a source package

Main article: Packaging


There are three main routes to creating source packages:

  1. Using Debian tooling directly.
  2. Using mud-builder to help repackage existing upstream sources.
  3. Using py2deb for Python applications.

Debian tooling

Main article: Documentation/Maemo_5_Developer_Guide/Packaging,_Deploying_and_Distributing#Creating_Debian_Packages


mud-builder

mud-builder uses a simple XML file to define an upstream source location (such as an svn location, web-based tarball or existing Debian package), it then handles the Maemo packaging specifics, builds a source package ready to upload and can even upload directly to Extras using a few simple commands:

mud build vim
mud upload vim

py2deb

Python applications can be developed inside or outside of Scratchbox. py2deb makes it easy to build packages ready for upload from a Python script.

PyPackager

PyPackager essentially replaces py2deb and provides the capabilities to build both a finished Debian Package (.deb) file as well as the source tarball and control files required by the garage build queues as py2deb does.

Upload

"free" packages

There are three mechanisms for uploading packages:

  1. The Extras Assistant is a web-based wizard, which guides you through the process
  2. Debian's dput tool
  3. SSH directly (using scp

dput

The /etc/dput.cf (or ~/.dput.cf) sections for free packages are:

[DEFAULT]
default_host_main = fremantle-extras-builder

[fremantle-extras-builder]
login = <your_garage_login_name>
fqdn = drop.maemo.org
method = scp
hash = md5
allow_unsigned_uploads = 1
incoming = /var/www/extras-devel/incoming-builder/fremantle

[diablo-extras-builder]
login = <your_garage_login_name>
fqdn = drop.maemo.org
method = scp
hash = md5
allow_unsigned_uploads = 1
incoming = /var/www/extras-devel/incoming-builder/diablo

[chinook-extras-builder]
login = <your_garage_login_name>
fqdn = drop.maemo.org
method = scp
hash = md5
allow_unsigned_uploads = 1
incoming = /var/www/extras-devel/incoming-builder/chinook

[gregale-extras-direct-binary-upload]
login = <your_garage_login_name>
fqdn = drop.maemo.org
method = scp
hash = md5
allow_unsigned_uploads = 1
incoming = /var/www/extras/incoming/gregale

Then run:

dput -f fremantle-extras-builder PACKAGE_VERSION_*.changes
dput -f diablo-extras-builder PACKAGE_VERSION_*.changes
dput -f chinook-extras-builder PACKAGE_VERSION_*.changes


For the single default case you should be able to do:

dput PACKAGE_VERSION_source.changes

and it should go to fremantle-extras-builder (because of the DEFAULT:default_host_main setting.

scp

Using scp directly is trivial:

scp *.tar.gz *.diff.gz *.changes *.dsc <your_garage_login_name>@drop.maemo.org:/var/www/extras-devel/incoming-builder/chinook/
scp *.tar.gz *.diff.gz *.changes *.dsc <your_garage_login_name>@drop.maemo.org:/var/www/extras-devel/incoming-builder/diablo/
scp *.tar.gz *.diff.gz *.changes *.dsc <your_garage_login_name>@drop.maemo.org:/var/www/extras-devel/incoming-builder/fremantle/


Note: Make sure you upload your .dsc files last. The .dsc file triggers a build and the autobuider might reject the package because of missing files.

"non-free" packages

Publishing non-free packages in the Maemo Extras Devel repository for Fremantle is done via dput. Direct upload of binary packages is allowed for non-free packages only.

Settings for dput.cf for Fremantle Extras-Devel non-free:

[fremantle-extras-devel-non-free]
login = <your_garage_login_name>
fqdn = drop.maemo.org
method = scp
hash = md5
allow_unsigned_uploads = 1
incoming = /var/www/extras-devel/incoming-nonfree/fremantle

Upload with dput:

dput -f fremantle-extras-devel-non-free PACKAGE_VERSION_*.changes

The packages are imported into the repository once per hour.

Please make sure your dependencies are correct. Should there be a dependency problem between a free and a non-free package, the non-free package will most likely be removed first. This is because we can recompile free packages, but cannot recompile non-free packages since we do not have the source.



For Windows 7

You will need these programs:

1) puttygen.exe

2) pageant.exe

3) pscp.exe


First run puttygen.exe, and make private and public keys (RSA). Save public and private key as files somewhere (and remember location!).

Open public key file with some text editor (notepad will do job), and now copy key.

This is how public key file should look like:

---- BEGIN SSH2 PUBLIC KEY ----
Comment: "rsa-key-20120905"
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdas
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdas
asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdas
asdasdas
---- END SSH2 PUBLIC KEY ----

You only need:

asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdas

Just make sure that doesn't contain blank characters - eg. new line or space. Now you can put that in garage account with prefix "ssh-rsa", so at the end it should look like this:

ssh-rsa asdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdasda+dasdasdasdasasda3dasdasdasdasdasda/dasd=sdasd9asdasdasasdasdas


Now start pageant.exe and add that private key.

Open console (click on windows logo, and in search filed type "cmd" - without quotes - and press enter). Now go to folder where did you downloaded/putted pscp.exe file.


To upload file, run this command:

pscp -v -scp <deb_full_path> <username>@drop.maemo.org:/var/www/extras-devel/incoming-nonfree/fremantle/

Where you need to replace:

- <deb_full_path> - with absolute file path (example: C:\MyWork\my_app.deb)

- <username> - your maemo garage username


After that, wait for package to be uploaded and thats it :)

Autobuilder special considerations

If it builds under your local Scratchbox but doesn't on the autobuilder, you need to do the above dependency check (dpkg-depcheck) and/or otherwise correct the Build-Depends field: the builder uses the minimal rootstrap and only adds extra packages if they are mentioned in this field. This includes most -dev versions which include the special libraries and includes, and the pkg-config metadata; so applies to almost everything.

See Building packages with sbdmock for information on using the sbdmock tool to test your package build in a clean scratchbox environment before submitting to the autobuilder.

If you do not know what to write in the Build-Depends field, then in your scratchbox, install devscripts (apt-get install devscripts) and then run dpkg-genbuilddeps -rfakeroot in your source directory. The package will build in scratchbox and at the end, you will see a list of Build-Depends. Use common sense to figure out which Build-Depends you need to add. -dev packages are usually a good bet.

Also make sure the Section: in the control file is all lowercase, e.g. "user/navigation" not "user/Navigation", otherwise after building it will silently fail and NOT be moved to extras-devel.

The Autobuilder supports automatic Optification of packages, using the tool maemo-optify. You just need to create the file "debian/optify" containing the word "auto" to trigger maemo-optify during build. To predetermine the results of enabling maemo-optify, you can install this package from the Extras-devel repository inside your scratchbox and run maemo-optify-deb on a precompiled package.

If you have problems with the autobuilder, ask for help on the maemo-developers mailing list, and possibly contact the debmaster if you are sure that the problem is with the autobuilder and not your package.

In addition, one additional idea to increase package quality is to introduce Maemian, a lintian-like tool for Maemo . This would help in doing additional checks for common inconsistencies and errors.

Promotion

After the autobuilder builds a package, it will be available in Extras-Devel. This is the experimental repository.

Please make sure your package is ready for users by testing:

  • if it installs without problems
  • if it uninstalls without problems
  • if it doesn't introduce dependency problems for other packages.

You can promote your package to Extras with the promotion interface.

Promotion from Fremantle Extras-devel to Extras-testing and Extras-testing to Extras is done through the maemo.org package interface and the process is described in the Extras-testing documentation.

Additional information