Editing Migrating to Community-driven Infrastructure
Warning: You are not logged in.
Your IP address will be recorded in this page's edit history.
The edit can be undone.
Please check the comparison below to verify that this is what you want to do, and then save the changes below to finish undoing the edit.
Latest revision | Your text | ||
Line 1: | Line 1: | ||
- | ' | + | == Introduction == |
- | + | [up to date as of 2013-02-08] | |
+ | Albeit Nokia's plans about discontinuation of maemo support been known since spring 2012, Nokia gave "Go" to Nemein (service provider on behalf of Nokia) for the real migration work not earlier than 2 weeks before christmas 2012. | ||
- | + | As of January, 18th 2013 the *.maemo.org infrastructure has been consolidated from a 20+ physical servers (aka "irons") to current config and completely migrated to new locations independant of Nokia servers. This task been accomplished by Nemein. Talk.maemo.org forum been integrated with the other infra, many thanks to Nemein for donating the VM for that. Also many thanks to Nemein for this incredible piece of work done during times when others (as well as the guys there) usually are already away for winter holidays. | |
- | + | The current setup (see below) consists of around 10 Virtual Machines hosted by Nemein on their xen-grid. This is an interim solution. Nokia paid Nemein for this consolidation/migration and hosting until end of February. | |
- | + | Handing over control of servers still pending, right now (2013-01-30) it's still Nemein and affiliates to control that infra. | |
- | + | Transfer of control over (*.)maemo.org DNS entries ("the domain") is still being negotiated between Nokia and HiFo, all DNS changes done so far been done by Nokia dnsmaster on Nemein's request | |
+ | |||
+ | The plans of council and HiFo board so far are: kindly ask Nemein to have *.maemo.org nicely bundled. We hope for this setup to be free of major known bugs (I.E. autobuilder working, repository working albeit maybe slow) when Nemein hands us the package. | ||
+ | |||
+ | [2013-02-08] Negotiations about direct migration to one of our 3 options (see below) are ongoing. | ||
+ | |||
+ | ===further plans, state of migration=== | ||
+ | Further plans are to migrate again to some hosted root servers, either on a sponsor like http://osuosl.org/about-osuosl or to our own stuff we may rent from e.g. Hetzner. | ||
+ | |||
+ | [2013-02-08] currently we're in negotiations about 3 possible ways into future hosting: | ||
+ | * osuosl (could provide VM or rootservers or CoLo [UPS server shipping: 48h:1200EUR, 7d:630EUR, +customs]) | ||
+ | * IPHH, a ISP in Hamburg. Falk contacted them and they are willing to offer CoLo basically free of charge. Of course we will put their name on our maemo.org frontpage to give due credit. HW service will be done by Falk. (costs ~300EUR for setup and HW upgrade, plus 50..300EUR for shipping the iron to Hamburg) | ||
+ | * get own paid rootservers, like 2 of http://www.hetzner.de/en/hosting/produkte_rootserver/ex10 (costs ~300EUR/month, 400EUR setup) | ||
+ | Depending on option chosen, we might or might not keep the SuperMicro. | ||
+ | |||
+ | [2013-02-17] Hildon Foundation board has agreed on following IPHH and keeping OSUOSL open as an alternative if something with our "plan-A" goes awry. Tech staff wholheartedly agrees and will act occordingly, sending iron to IPHH/Falk on 2013-02-18/19, and also negotiating with OSUOSL about how a possible migration to them would look like so we get a decent checklist in case we need it. | ||
+ | |||
+ | Falk's mail forwarded form IPHH to HiFo: | ||
<pre> | <pre> | ||
- | + | Hi everyone, | |
- | + | ||
- | + | ||
- | + | these are the details, what IPHH is willing to offer us. | |
- | + | Best regards, | |
- | + | Falk | |
- | + | Begin forwarded message: | |
- | + | ||
- | [ | + | > From: Rene Sasse <support@iphh.xxx> |
+ | > Subject: [IPHH #442659] Re: maemo.org | ||
+ | > Date: 18. Februar 2013 11:08:33 MEZ | ||
+ | > To: falk@fourecks.xxx | ||
+ | > Cc: joerg@openmoko.xxx | ||
+ | > Reply-To: support@iphh.xxx | ||
+ | > | ||
+ | > Falk, | ||
+ | > | ||
+ | > IPHH offers the following services to Hildon Foundation for one year free of | ||
+ | > charge: | ||
+ | > | ||
+ | > * Colocation/electricity for the following devices: | ||
+ | > - 1 Server (2RU) | ||
+ | > - 1 Switch (1RU) | ||
+ | > * 1 100MBit/s Uplink Port | ||
+ | > * A /27 IPv4 Network | ||
+ | > | ||
+ | > This offer is valid for one year and has to be discussed for renewal after 11 | ||
+ | > month. | ||
+ | > | ||
+ | > Legal Contact will be: | ||
+ | > | ||
+ | > Hildon Foundation | ||
+ | > 120 West 10th Street, Erie, PA, 16501, USA | ||
+ | > | ||
+ | > Technical Contact will be: | ||
+ | > | ||
+ | > Falk Stern (FS7182-RIPE) | ||
+ | > Rathmann-Cohrs-Straße 12, 21357 Bardowick, Germany | ||
+ | > Mobile: +49-160-71560xx | ||
+ | > | ||
+ | > | ||
+ | > best regards | ||
+ | > Rene | ||
+ | > | ||
+ | > -- | ||
+ | > Rene Sasse E-Mail: support@iphh.xxx | ||
+ | > Technical Consultant Tel: +49 (0)40 374919-xx | ||
+ | > IPHH Internet Port Hamburg GmbH Fax: +49 (0)40 374919-xx | ||
+ | > Wendenstrasse 408 AG Hamburg, HRB 76071 | ||
+ | > D-20537 Hamburg Geschaeftsfuehrung: Axel G. Kroeger--7E94C7404EC25FD69CC85C3653348297 | ||
+ | > | ||
+ | </pre> | ||
- | + | Iron to move form: ( http://nemein.com/fi/ ) | |
+ | Nemein Oy | ||
+ | tel. +358 20-198 6030 | ||
+ | Vilhonvuorenkatu 11 D, 8 krs | ||
+ | 00500 Helsinki, FINLAND | ||
+ | FIN-1647219-2 | ||
+ | support AT nemein.com | ||
- | + | to ( http://www.iphh.net/en/contact.html ) | |
+ | IPHH Internet Port Hamburg GmbH | ||
+ | #444615 | ||
+ | Wendenstrasse 408 | ||
+ | 20537 Hamburg | ||
+ | Germany | ||
+ | T : +49 40 37 49 19-0 | ||
+ | F : +49 40 37 49 19-29 | ||
+ | E : info@iphh.net | ||
- | + | Package details: | |
+ | size | ||
+ | x: 100cm | ||
+ | y: 66cm | ||
+ | z: 28cm | ||
+ | weight: ~40kg | ||
- | |||
- | + | Shipping accomplished: | |
+ | Shipment number 1139212793 | ||
+ | Status from Wed, 20.02.2013 10:57 hours Delivered - signed for by Herr POLROK* | ||
+ | Recipient TPHH | ||
+ | Delivered on Herr POLROK* | ||
+ | via DHL account provided by Nokia/Pekka (many thanks!) on 2013-02-19. | ||
+ | Courtesy Aslan and Eero of Nemein. | ||
- | + | ||
+ | Hosting migration timing plan: | ||
+ | https://docs.google.com/spreadsheet/ccc?key=0AuQnrvWRbTtzdFhERzlDbEp0eVNQQTdfNWpQbzdIT0E&usp=sharing | ||
+ | |||
+ | |||
+ | Alternatives - however obvious - for the above plans have been discussed with Nemein and HiFo and are not feasible. E.G. there was no way we could get the money instead of the server iron hardware. Sustaining the current xen-grid based VM hosting would be ~1500EUR per month plus a basically not evadable 2200EUR on top for maintenace. We want to switch away from that by all means, thus the 2nd migration. | ||
- | |||
- | |||
- | |||
This page is intended as a central place where status and other operational information can be gathered. | This page is intended as a central place where status and other operational information can be gathered. | ||
- | === Plan for migration / Timeline | + | === Plan for migration / Timeline === |
* Friday, 22.2. (falk) | * Friday, 22.2. (falk) | ||
Line 56: | Line 140: | ||
* ... (hidden DNS master set up) | * ... (hidden DNS master set up) | ||
** sync databases, switch DNS entries | ** sync databases, switch DNS entries | ||
- | |||
VMs we need to migrate: | VMs we need to migrate: | ||
{| | {| | ||
- | ! Name !! Disk Size !! Location | + | ! Name !! Disk Size !! Location !! _migrated? !! _Comments |
|+ | |+ | ||
- | | static || 30G || nemein || | + | | static || 30G || nemein || copied || (copied!) |
|+ | |+ | ||
- | | wiki || 20G || nemein || | + | | wiki || 20G || nemein || copied || also bugs. (copied!) |
|+ | |+ | ||
- | | repository || 900G || nemein || | + | | repository || 900G || nemein || copying || We need to check the disk size, this might be too big for current hw, maybe split tablets-dev off. (copying) |
|+ | |+ | ||
- | | mail || 20G || nemein || | + | | mail || 20G || nemein || no || also has lists |
|+ | |+ | ||
- | | scratchbox || | + | | scratchbox || 20G || nemein || no || has an NFS mount from db! |
|+ | |+ | ||
- | | vcs || 50G || nemein || | + | | vcs || 50G || nemein || copying || has NFS mounts from garage and repository (copying) |
|+ | |+ | ||
- | | garage || 100G || nemein || | + | | garage || 100G || nemein || copied || has NFS mounts from stage and vcs (copied!) |
|+ | |+ | ||
- | | db || 100G || nemein || | + | | db || 100G || nemein || copied || 73G used, maybe disk needs to be bigger (copied!) |
|+ | |+ | ||
- | | builder || 50G || nemein || | + | | builder || 50G || nemein || no || |
|+ | |+ | ||
- | | talk || 20G || nemein || | + | | talk || 20G || nemein || copied || |
+ | |} | ||
+ | |||
+ | === Setup with IPHH === | ||
+ | |||
+ | ==== Networks ==== | ||
+ | |||
+ | We have 2 /28 Subnets (213.128.137.0/28 and 213.128.137.16/28) | ||
+ | |||
+ | Networks are configured as follows: | ||
+ | |||
+ | {| | ||
+ | ! IPv4 !! IPv6 !! VLAN !! Xen Bridge !! default GW | ||
|+ | |+ | ||
- | | | + | | 213.128.137.0/28 || not yet || 1 || xenbr0 || 213.128.137.14 |
+ | |+ | ||
+ | | 213.128.137.16/28 || not yet || 2 || xenbr1 || 213.128.137.17 | ||
+ | |+ | ||
+ | | 10.0.1.0/24 || not yet || 3 || xenbr2 || 10.0.1.1 | ||
|} | |} | ||
- | === | + | IP Plan for vlan 1 |
- | + | ||
+ | {| | ||
+ | ! IPv4 !! IPv6 !! Hostname | ||
+ | |+ | ||
+ | | 213.128.137.1 || n/a || firewall-carp | ||
+ | |+ | ||
+ | | 213.128.137.2 || n/a || firewall-a | ||
+ | |+ | ||
+ | | 213.128.137.3 || n/a || firewall-b | ||
+ | |+ | ||
+ | | 213.128.137.4 || n/a || blade-a | ||
+ | |+ | ||
+ | | 213.128.137.5 || n/a || blade-b | ||
+ | |+ | ||
+ | | 213.128.137.6 || n/a || portforwarding for monitor | ||
+ | |+ | ||
+ | | 213.128.137.7 || n/a || | ||
+ | |+ | ||
+ | | 213.128.137.8 || n/a || | ||
+ | |+ | ||
+ | | 213.128.137.9 || n/a || | ||
+ | |+ | ||
+ | | 213.128.137.10 || n/a || | ||
+ | |+ | ||
+ | | 213.128.137.11 || n/a || | ||
+ | |+ | ||
+ | | 213.128.137.12 || n/a || IPHH Router 1 | ||
+ | |+ | ||
+ | | 213.128.137.13 || n/a || IPHH Router 2 | ||
+ | |+ | ||
+ | | 213.128.137.14 || n/a || IPHH-VRRP | ||
+ | |} | ||
+ | |||
+ | IP Plan for vlan 2 | ||
+ | |||
+ | {| | ||
+ | ! IPv4 !! IPv6 !! Hostname | ||
+ | |+ | ||
+ | | 213.128.137.17 || n/a || firewall-carp | ||
+ | |+ | ||
+ | | 213.128.137.18 || n/a || firewall-a | ||
+ | |+ | ||
+ | | 213.128.137.19 || n/a || firewall-b | ||
+ | |+ | ||
+ | | 213.128.137.20 || n/a || static/www | ||
+ | |+ | ||
+ | | 213.128.137.21 || n/a || wiki | ||
+ | |+ | ||
+ | | 213.128.137.22 || n/a || repository | ||
+ | |+ | ||
+ | | 213.128.137.23 || n/a || mail | ||
+ | |+ | ||
+ | | 213.128.137.24 || n/a || scratchbox | ||
+ | |+ | ||
+ | | 213.128.137.25 || n/a || vcs | ||
+ | |+ | ||
+ | | 213.128.137.26 || n/a || garage | ||
+ | |+ | ||
+ | | 213.128.137.27 || n/a || builder | ||
+ | |+ | ||
+ | | 213.128.137.28 || n/a || talk | ||
+ | |+ | ||
+ | | 213.128.137.29 || n/a || DNS | ||
+ | |+ | ||
+ | | 213.128.137.30 || n/a || | ||
+ | |} | ||
+ | |||
+ | IP Plan for vlan 3 | ||
+ | |||
+ | {| | ||
+ | ! IPv4 !! IPv6 !! Hostname | ||
+ | |+ | ||
+ | | 10.0.1.1 || n/a || firewall-carp | ||
+ | |+ | ||
+ | | 10.0.1.2 || n/a || firewall-a | ||
+ | |+ | ||
+ | | 10.0.1.3 || n/a || firewall-b | ||
+ | |+ | ||
+ | | 10.0.1.10 || n/a || db | ||
+ | |+ | ||
+ | | 10.0.1.11 || n/a || monitor | ||
+ | |+ | ||
+ | | 10.0.1.200 || n/a || blade-a/IPMI | ||
+ | |+ | ||
+ | | 10.0.1.201 || n/a || blade-b/IPMI | ||
+ | |+ | ||
+ | | 10.0.1.202 || n/a || maemo-switch | ||
+ | |} | ||
+ | |||
+ | ==== Disk Layout of blade-[ab] ==== | ||
+ | |||
+ | Both disks have the following partitioning: | ||
+ | RAID1 Volume for /boot (/dev/md0), consisting of /dev/sda1 and /dev/sdb1 (200M) | ||
+ | RAID1 Volume /dev/md1 consisting of /dev/sda2 and /dev/sdb2 (around 970G) | ||
+ | The RAID1 Volume contains a physical LVM volume. | ||
+ | We only have one VolumeGroup (vg_blade[ab]), which has LogVol00 with 20G as root volume, LogVol01 with 2 Gig as swap and vmstore with the rest as VM Storage mounted on /vmstore. | ||
==== Tips & Tricks for migration ==== | ==== Tips & Tricks for migration ==== | ||
Line 107: | Line 301: | ||
<pre>cd / ; rsync -arvSxz . root@host:/mount/point</pre> | <pre>cd / ; rsync -arvSxz . root@host:/mount/point</pre> | ||
- | == | + | == Inventory (obsolete, please update) == |
- | * | + | As a first step we try to gather information about the present infrastructure at *.maemo.org. This "inventory" is intended to provide an overview about all components of the infrastructure as well as to provide information that will later on aid during the actual migration. |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | Currently the following topics are considered important for the migration: | |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
* Legal Issues (Names, Trademarks, Domain Names, etc.) | * Legal Issues (Names, Trademarks, Domain Names, etc.) | ||
* Infrastructure (Web Site, Forum, Wiki, Autobuilder, Mailinglists, Garage, etc.) | * Infrastructure (Web Site, Forum, Wiki, Autobuilder, Mailinglists, Garage, etc.) | ||
+ | |||
== Legal Issues == | == Legal Issues == | ||
Line 205: | Line 354: | ||
== Operational Platform == | == Operational Platform == | ||
- | + | For now (2013-01,02) all services are implemented as VM running on Nemein's xen-grid. Plan is to migrate those VM to a virtualization installed on the serevrs (see below) that Nokia donates to the community. This hardware is located at Nemein office right now (2013-02-16) and '''should''' get shipped to iphh.net in Hamburg on Monday 2013-02-18. | |
List of hardware Nokia will donate to HiFo, according to Nemein's plans. [2013-02-08] | List of hardware Nokia will donate to HiFo, according to Nemein's plans. [2013-02-08] | ||
Line 229: | Line 378: | ||
|- | |- | ||
| 01 | | 01 | ||
- | | | + | | |
| | | | ||
| | | | ||
Line 237: | Line 386: | ||
| http://www.supermicro.nl/products/system/2u/2027/SYS-2027TR-HTRF.cfm?parts=SHOW | | http://www.supermicro.nl/products/system/2u/2027/SYS-2027TR-HTRF.cfm?parts=SHOW | ||
| 2U 19" Rackmount | | 2U 19" Rackmount | ||
- | | Intel® Xeon® processor E5-2620 | + | | 2 * Intel® Xeon® processor E5-2620 |
- | | 32GB | + | | 2 * 32GB |
- | | (raid1 | + | | 2 * (raid1 2 * 1TB) |
| | | | ||
| 3 years | | 3 years | ||
Line 246: | Line 395: | ||
|- | |- | ||
| 02 | | 02 | ||
- | | | + | | |
| | | | ||
| | | | ||
Line 254: | Line 403: | ||
| | | | ||
| | | | ||
- | | | + | | |
- | | | + | | |
- | | | + | | |
| | | | ||
| | | | ||
Line 262: | Line 411: | ||
|} | |} | ||
- | |||
=== OS and virtulization on community iron (planning, discussion) === | === OS and virtulization on community iron (planning, discussion) === | ||
Please don't forget to tag your contributions with your nick! | Please don't forget to tag your contributions with your nick! | ||
Line 272: | Line 420: | ||
- | ==== | + | ====Virtulization==== |
===== alternative A ===== | ===== alternative A ===== | ||
XEN (with OS blabla of above) | XEN (with OS blabla of above) | ||
Line 473: | Line 621: | ||
=== More Detailed Information === | === More Detailed Information === | ||
- | |||
In this sub section more detailed information about the entries in the table can be placed. The intent is to keep the table concise while still being able to have all relevant information at hand. | In this sub section more detailed information about the entries in the table can be placed. The intent is to keep the table concise while still being able to have all relevant information at hand. | ||
Line 488: | Line 635: | ||
188.117.59.202 wiki.maemo.org | 188.117.59.202 wiki.maemo.org | ||
188.117.59.212 bugs.maemo.org | 188.117.59.212 bugs.maemo.org | ||
- | + | 188.117.59.203 repository.maemo.org | |
- | 188.117.59.205 stage.maemo.org | + | 188.117.59.205 stage.maemo.org |
188.117.59.206 vcs.maemo.org | 188.117.59.206 vcs.maemo.org | ||
Line 567: | Line 714: | ||
| maemo || warfare || Falk Stern || falk<at>fourecks.de || (maemo master sysop) || || | | maemo || warfare || Falk Stern || falk<at>fourecks.de || (maemo master sysop) || || | ||
|+ | |+ | ||
- | | maemo || chemist || Ruediger Schiller || | + | | maemo || chemist || Ruediger Schiller || chemist<at><to-be-disclosed-by-owner> || Talk || || || |
|+ | |+ | ||
| maemo || merlin1991 || Christian Ratzenhofer || <at> || Repos || || [preliminary accepted] || | | maemo || merlin1991 || Christian Ratzenhofer || <at> || Repos || || [preliminary accepted] || | ||
Line 576: | Line 723: | ||
|+ | |+ | ||
| || || || || (planet???) || || | | || || || || (planet???) || || | ||
- | |} | + | |} |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
== Steering == | == Steering == | ||
- | |||
council is in charge of any steering. | council is in charge of any steering. | ||
- | |||
Joerg Reisenweber got appointed for "maemo.org infra administration coordinator" and thus is the single point of coordination for any detail questions. | Joerg Reisenweber got appointed for "maemo.org infra administration coordinator" and thus is the single point of coordination for any detail questions. | ||
Line 605: | Line 734: | ||
* OBS @ TiZen or SuSe : https://bugs.tizen.org/jira/browse/TINF-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel | * OBS @ TiZen or SuSe : https://bugs.tizen.org/jira/browse/TINF-48?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel | ||
- | |||
- | |||
- | |||
- | |||
- |
Learn more about Contributing to the wiki.