User:Jebba/Espejo

Latest revision as of 02:57, 17 January 2010

1 Overview
2 Usage
3 Server Details
4 TODO

[edit] Overview

Espejo is a temporary virtual computer set up to mirror the Maemo 5 (Fremantle) repository.

Hostname: espejo.freemoe.org

Admins:
- jebba <moe@blagblagblag.org>
- jeremiah

Location: USA @ NetDepot

Services:
- http://espejo.freemoe.org
- rsync://espejo.freemoe.org

[edit] Usage

If you would like to use this mirror, you can add any of these lines to your /etc/apt/sources.list or add them via tap tap tap with the Hildon Application Manager:

# Extras
deb http://espejo.freemoe.org/repository.maemo.org/extras/ fremantle free
# Extras source
deb-src http://espejo.freemoe.org/repository.maemo.org/extras/ fremantle free

# Extras Testing
deb http://espejo.freemoe.org/repository.maemo.org/extras-testing/ fremantle free
# Extras Testing source
deb-src http://espejo.freemoe.org/repository.maemo.org/extras-testing/ fremantle free

# Extras Development
deb http://espejo.freemoe.org/repository.maemo.org/extras-devel/ fremantle free
# Extras Development Source
deb-src http://espejo.freemoe.org/repository.maemo.org/extras-devel/fremantle free


# SDK
deb http://espejo.freemoe.org/repository.maemo.org/ fremantle/sdk free
# SDK Source
deb-src http://espejo.freemoe.org/repository.maemo.org/ fremantle/sdk free

# Tools
deb http://espejo.freemoe.org/repository.maemo.org/ fremantle/tools free
# Tools Source
deb-src http://espejo.freemoe.org/repository.maemo.org/ fremantle/tools free

[edit] Server Details

The server is a Debian Lenny KVM (QEMU) within a Debian Squeeze/Sid host server.

[edit] Host Hardware

(8) 2GHz Processors (E5405)
4 Gig RAM
750G SATA (ST3750330AS)
(3) 30G SCSI RAID-0

[edit] Guest Hardware

Virtual, of course. :)

4 CPU
1 Gig RAM
Disk:

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/vda1             4.0G  758M  3.0G  20% /
/dev/vde1              50G  183M   50G   1% /var/www/repository.maemo.org
/dev/vdb1              50G  951M   49G   2% /var/www/repository.maemo.org/extras
/dev/vdc1              50G  2.5G   47G   5% /var/www/repository.maemo.org/extras-testing
/dev/vdd1              99G   16G   84G  16% /var/www/repository.maemo.org/extras-devel
/dev/vdf1              50G  2.3G   48G   5% /var/www/repository.maemo.org/pool

[edit] Daemons

[edit] apache2

No modperl, modphp, etc. installed. Other than that, total stock install.

[edit] rsync

Stock Debian Lenny.

$ cat /etc/rsyncd.conf

[repository.maemo.org]
  path = /var/www/repository.maemo.org
  comment = Extras Repositories for Maemo 5 (Fremantle)
  read only = yes
  use chroot = yes
  max connections=256
  list = yes
  uid = nobody
  gid = rsync
  dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

cat /etc/default/rsync  # relevent tidbit:
RSYNC_ENABLE=true

I added group "rsync" as Debian doesn't come stock with a group nobody. Don't think it matters "too" much.

[edit] Guest inital setup

This is quite crude, but gives you an idea:

After install, log in as root via VNC

apt-get install git-core
apt-get install openssh-server openssh-client

then ssh in remotely as jebba

# faster way to do this but wtf
jebba@espejo:~$ ssh localhost
jebba@espejo:~$ logout

jebba@donald:~$ scp -p .ssh/id_rsa.pub espejo.freemoe.org:.ssh/authorized_keys

jebba@donald:~$ ssh jebba@espejo.freemoe.org

topo:~$ cat /etc/.gitignore 
ld.so.cache
prelink.cache
*.swp
ld.so.cache
adjtime
blkid.tab
blkid.tab.old
mtab
adjtime
sv/dnscache/log
sv/dnscache/seed
ld.so.cache


espejo:~# cd /etc/
espejo:/etc# git init
Initialized empty Git repository in /etc/.git/
espejo:/etc# chmod og-rwx .git
espejo:/etc# git add .
espejo:/etc# EDITOR=vim git commit -a

blow out unwanted apache shit, just reinstall to get rid of too much crap
espejo:/etc# /etc/init.d/apache2 stop

espejo:/etc# apt-get remove libapache2-mod-perl2 libapache2-mod-php5 libapache2-mod-python ; apt-get autoremove

espejo:/etc# apt-get remove apache2-doc apache2-mpm-prefork apache2-utils apache2.2-common

gah, it installs mysql shit too. Should have started with a bare install
apt-get autoremove

gah, clean up poo
dpkg -l|grep ^rc

espejo:/etc# dpkg --purge apache2-doc apache2.2-common libapache2-mod-perl2 libapache2-mod-php5 libapache2-mod-python libapr1 libaprutil1 libmysqlclient15off libpq5 mysql-common php5-common

rm -rf /etc/php5

update-rc.d -f exim4  remove
update-rc.d -f rsync remove
update-rc.d -f nfs-common remove
update-rc.d -f openbsd-inetd remove

apt-get install apache2
gah, drags in libmysqlclient15off libpq5 mysql-common
reboot


espejo:~# cd /etc/
espejo:/etc# git add .
espejo:/etc# git c
espejo:/etc# EDITOR=vim git commit -a

apt-get install sudo

# shoulda set up gitconfig earlier
$ cat ~/.gitconfig 
[user]
	name = Jeff Moe
	email = moe@blagblagblag.org
[color]
  branch = auto
  diff = auto
  status = auto


mkdir /var/www/repository.maemo.org


setup  new drives in startup script and POWEROFF (not reboot--need to restart script to read in new drives)


sudo fdisk /dev/vdb
sudo fdisk /dev/vdc
sudo fdisk /dev/vdd

and give them one fat partition
# format
sudo /sbin/mkfs.ext3 -j -m0 /dev/vdb1
sudo /sbin/mkfs.ext3 -j -m0 /dev/vdc1
sudo /sbin/mkfs.ext3 -j -m0 /dev/vdd1

mkdir -p /var/www/repository.maemo.org/extras /var/www/repository.maemo.org/extras-testing /var/www/repository.maemo.org/extras-devel

add them to fstab
sudo mount -a

chown them to jebba:jebba
sudo chown jebba:jebba /var/www/repository.maemo.org/extras /var/www/repository.maemo.org/extras-testing /var/www/repository.maemo.org/extras-deveel

rsync over from topo what you already downloaded:
rsync -ault --rsh=ssh /HUGE/mirrors/repository.maemo.org/ espejo.freemoe.org:/var/www/repository.maemo.org/


sudo apt-get install reprepro

add rsyncd conf
turn of fucking portmap ? why that starting.
sudo /usr/sbin/update-rc.d -f portmap remove
sudo /usr/sbin/update-rc.d -f mountnfs.sh remove
sudo /usr/sbin/update-rc.d -f mountnfs-bootclean.sh remove
sudo /usr/sbin/update-rc.d -f x11-common remove


turn on rsyncd

set up vsftpd

sudo apt-get install reprepro gpgsm inoticoming

 reprepro -vv --basedir=/var/www/repository.maemo.org/

[edit] Host Server

[edit] Initial setup

Latest kernel with virtio etc. (Include relevant kernel .config).

Create disk images:

Example:

qemu-img create -f qcow2 -o cluster_size=2097152,preallocation=metadata /HUGE/kvm/espejo/espejo-FOO.qcow2 50G

[edit] Daily Use

Start kvm on boot via good ol' rc.local:

/usr/local/bin/kvm-start-espejo >> /tmp/espejo-kvm-err 2>&1 &

Startup script:

$ cat /usr/local/bin/kvm-start-espejo 
#!/bin/sh

/usr/local/bin/qemu-system-x86_64 \
       -drive \
        file=/HUGE/kvm/espejo/espejo.qcow2,index=0,if=virtio,cache=writeback,boot=on \
       -drive \
        file=/HUGE/kvm/espejo/espejo-extras.qcow2,index=1,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-extras-testing.qcow2,index=2,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-extras-devel.qcow2,index=3,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-pool.qcow2,index=4,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-maemo.qcow2,index=5,if=virtio,cache=writeback \
       -cpu kvm64 \
       -m 1024 \
       -vnc :6 \
       -daemonize \
       -enable-kvm \
       -name espejo \
       -net nic,vlan=0,model=virtio \
       -net user,hostfwd=tcp:63.247.92.156:22-:22,hostfwd=tcp:63.247.92.156:80-:80,hostfwd=tcp:63.247.92.156:873-:873,hostfwd=tcp:63.247.92.156:20-:20,hostfwd=tcp:63.247.92.156:21-:21 \
       -smp 4 \
       -runas jebba &

[edit] TODO

Could set up separate partitions for /tmp and such, but probably not worth it.

Build latest/greatest kernel on guest to take advantage of latest virtio code to improve disk performance.

Set up puppet for configs (worthwhile?).

Retrieved from "http://wiki.maemo.org/User:Jebba/Espejo"

This page was last modified on 17 January 2010, at 02:57.
This page has been accessed 20,229 times.

@@ Line 96: / Line 96: @@
 * I added group "rsync" as Debian doesn't come stock with a group nobody. Don't think it matters "too" much.
-===vsftpd===
-I'm not sure what the latest/greatest ftp server is nowadays, but this one should work.
-Added these two lines to stock /etc/vsftpd.conf:
- userlist_enable=NO
- anon_root=/var/www
 ==Guest inital setup==

Personal tools

Navigation

Views