User:Jebba/Espejo

(→Daily Use)
(→Overview: ditched ftp. http for downloads, rsync for mirroring.)
Line 12: Line 12:
* Services:
* Services:
** [http://espejo.freemoe.org http://espejo.freemoe.org]
** [http://espejo.freemoe.org http://espejo.freemoe.org]
-
** [ftp://espejo.freemoe.org ftp://espejo.freemoe.org]
 
** rsync://espejo.freemoe.org
** rsync://espejo.freemoe.org

Revision as of 02:56, 17 January 2010

Contents

Overview

Espejo is a temporary virtual computer set up to mirror the Maemo 5 (Fremantle) repository.

  • Hostname: espejo.freemoe.org

Usage

If you would like to use this mirror, you can add any of these lines to your /etc/apt/sources.list or add them via tap tap tap with the Hildon Application Manager: 

# Extras
deb http://espejo.freemoe.org/repository.maemo.org/extras/ fremantle free
# Extras source
deb-src http://espejo.freemoe.org/repository.maemo.org/extras/ fremantle free

# Extras Testing
deb http://espejo.freemoe.org/repository.maemo.org/extras-testing/ fremantle free
# Extras Testing source
deb-src http://espejo.freemoe.org/repository.maemo.org/extras-testing/ fremantle free

# Extras Development
deb http://espejo.freemoe.org/repository.maemo.org/extras-devel/ fremantle free
# Extras Development Source
deb-src http://espejo.freemoe.org/repository.maemo.org/extras-devel/fremantle free


# SDK
deb http://espejo.freemoe.org/repository.maemo.org/ fremantle/sdk free
# SDK Source
deb-src http://espejo.freemoe.org/repository.maemo.org/ fremantle/sdk free

# Tools
deb http://espejo.freemoe.org/repository.maemo.org/ fremantle/tools free
# Tools Source
deb-src http://espejo.freemoe.org/repository.maemo.org/ fremantle/tools free


Server Details

The server is a Debian Lenny KVM (QEMU) within a Debian Squeeze/Sid host server.


Host Hardware

  • (8) 2GHz Processors (E5405)
  • 4 Gig RAM
  • 750G SATA (ST3750330AS)
  • (3) 30G SCSI RAID-0


Guest Hardware

Virtual, of course. :)

  • 4 CPU
  • 1 Gig RAM
  • Disk: 
$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/vda1             4.0G  758M  3.0G  20% /
/dev/vde1              50G  183M   50G   1% /var/www/repository.maemo.org
/dev/vdb1              50G  951M   49G   2% /var/www/repository.maemo.org/extras
/dev/vdc1              50G  2.5G   47G   5% /var/www/repository.maemo.org/extras-testing
/dev/vdd1              99G   16G   84G  16% /var/www/repository.maemo.org/extras-devel
/dev/vdf1              50G  2.3G   48G   5% /var/www/repository.maemo.org/pool

Daemons

apache2

No modperl, modphp, etc. installed. Other than that, total stock install.


rsync

Stock Debian Lenny. 

$ cat /etc/rsyncd.conf

[repository.maemo.org]
  path = /var/www/repository.maemo.org
  comment = Extras Repositories for Maemo 5 (Fremantle)
  read only = yes
  use chroot = yes
  max connections=256
  list = yes
  uid = nobody
  gid = rsync
  dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz

cat /etc/default/rsync  # relevent tidbit:
RSYNC_ENABLE=true
  • I added group "rsync" as Debian doesn't come stock with a group nobody. Don't think it matters "too" much.

vsftpd

I'm not sure what the latest/greatest ftp server is nowadays, but this one should work.

Added these two lines to stock /etc/vsftpd.conf: 

userlist_enable=NO
anon_root=/var/www

Guest inital setup

This is quite crude, but gives you an idea: 

After install, log in as root via VNC

apt-get install git-core
apt-get install openssh-server openssh-client

then ssh in remotely as jebba

# faster way to do this but wtf
jebba@espejo:~$ ssh localhost
jebba@espejo:~$ logout

jebba@donald:~$ scp -p .ssh/id_rsa.pub espejo.freemoe.org:.ssh/authorized_keys

jebba@donald:~$ ssh jebba@espejo.freemoe.org

topo:~$ cat /etc/.gitignore 
ld.so.cache
prelink.cache
*.swp
ld.so.cache
adjtime
blkid.tab
blkid.tab.old
mtab
adjtime
sv/dnscache/log
sv/dnscache/seed
ld.so.cache


espejo:~# cd /etc/
espejo:/etc# git init
Initialized empty Git repository in /etc/.git/
espejo:/etc# chmod og-rwx .git
espejo:/etc# git add .
espejo:/etc# EDITOR=vim git commit -a

blow out unwanted apache shit, just reinstall to get rid of too much crap
espejo:/etc# /etc/init.d/apache2 stop

espejo:/etc# apt-get remove libapache2-mod-perl2 libapache2-mod-php5 libapache2-mod-python ; apt-get autoremove

espejo:/etc# apt-get remove apache2-doc apache2-mpm-prefork apache2-utils apache2.2-common

gah, it installs mysql shit too. Should have started with a bare install
apt-get autoremove

gah, clean up poo
dpkg -l|grep ^rc

espejo:/etc# dpkg --purge apache2-doc apache2.2-common libapache2-mod-perl2 libapache2-mod-php5 libapache2-mod-python libapr1 libaprutil1 libmysqlclient15off libpq5 mysql-common php5-common

rm -rf /etc/php5

update-rc.d -f exim4  remove
update-rc.d -f rsync remove
update-rc.d -f nfs-common remove
update-rc.d -f openbsd-inetd remove

apt-get install apache2
gah, drags in libmysqlclient15off libpq5 mysql-common
reboot


espejo:~# cd /etc/
espejo:/etc# git add .
espejo:/etc# git c
espejo:/etc# EDITOR=vim git commit -a

apt-get install sudo

# shoulda set up gitconfig earlier
$ cat ~/.gitconfig 
[user]
	name = Jeff Moe
	email = moe@blagblagblag.org
[color]
  branch = auto
  diff = auto
  status = auto


mkdir /var/www/repository.maemo.org


setup  new drives in startup script and POWEROFF (not reboot--need to restart script to read in new drives)


sudo fdisk /dev/vdb
sudo fdisk /dev/vdc
sudo fdisk /dev/vdd

and give them one fat partition
# format
sudo /sbin/mkfs.ext3 -j -m0 /dev/vdb1
sudo /sbin/mkfs.ext3 -j -m0 /dev/vdc1
sudo /sbin/mkfs.ext3 -j -m0 /dev/vdd1

mkdir -p /var/www/repository.maemo.org/extras /var/www/repository.maemo.org/extras-testing /var/www/repository.maemo.org/extras-devel

add them to fstab
sudo mount -a

chown them to jebba:jebba
sudo chown jebba:jebba /var/www/repository.maemo.org/extras /var/www/repository.maemo.org/extras-testing /var/www/repository.maemo.org/extras-deveel

rsync over from topo what you already downloaded:
rsync -ault --rsh=ssh /HUGE/mirrors/repository.maemo.org/ espejo.freemoe.org:/var/www/repository.maemo.org/


sudo apt-get install reprepro

add rsyncd conf
turn of fucking portmap ? why that starting.
sudo /usr/sbin/update-rc.d -f portmap remove
sudo /usr/sbin/update-rc.d -f mountnfs.sh remove
sudo /usr/sbin/update-rc.d -f mountnfs-bootclean.sh remove
sudo /usr/sbin/update-rc.d -f x11-common remove


turn on rsyncd

set up vsftpd

sudo apt-get install reprepro gpgsm inoticoming

 reprepro -vv --basedir=/var/www/repository.maemo.org/


Host Server

Initial setup

  • Latest kernel with virtio etc. (Include relevant kernel .config).
  • Create disk images:

Example: 

qemu-img create -f qcow2 -o cluster_size=2097152,preallocation=metadata /HUGE/kvm/espejo/espejo-FOO.qcow2 50G


Daily Use

  • Start kvm on boot via good ol' rc.local: 
/usr/local/bin/kvm-start-espejo >> /tmp/espejo-kvm-err 2>&1 &

Startup script: 

$ cat /usr/local/bin/kvm-start-espejo 
#!/bin/sh

/usr/local/bin/qemu-system-x86_64 \
       -drive \
        file=/HUGE/kvm/espejo/espejo.qcow2,index=0,if=virtio,cache=writeback,boot=on \
       -drive \
        file=/HUGE/kvm/espejo/espejo-extras.qcow2,index=1,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-extras-testing.qcow2,index=2,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-extras-devel.qcow2,index=3,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-pool.qcow2,index=4,if=virtio,cache=writeback \
       -drive \
        file=/HUGE/kvm/espejo/espejo-maemo.qcow2,index=5,if=virtio,cache=writeback \
       -cpu kvm64 \
       -m 1024 \
       -vnc :6 \
       -daemonize \
       -enable-kvm \
       -name espejo \
       -net nic,vlan=0,model=virtio \
       -net user,hostfwd=tcp:63.247.92.156:22-:22,hostfwd=tcp:63.247.92.156:80-:80,hostfwd=tcp:63.247.92.156:873-:873,hostfwd=tcp:63.247.92.156:20-:20,hostfwd=tcp:63.247.92.156:21-:21 \
       -smp 4 \
       -runas jebba &

TODO

  • Could set up separate partitions for /tmp and such, but probably not worth it.
  • Build latest/greatest kernel on guest to take advantage of latest virtio code to improve disk performance.
  • Set up puppet for configs (worthwhile?).
Retrieved from "http://wiki.maemo.org/User:Jebba/Espejo"