Device management using Bcfg2

m (TODO)
(Getting Bcfg2 up and running)
 
(9 intermediate revisions not shown)
Line 21: Line 21:
|-
|-
| Enterprise
| Enterprise
-
| A large company ot other organization that wants employees to ba able to acces company IT systems using Maemo based devices
+
| A large company or other organization that wants employees to ba able to acces company IT systems using Maemo based devices
|-
|-
| Desktop computer
| Desktop computer
Line 40: Line 40:
*[http://www.linuxlinks.com/article/20100222155850632/ConfigurationManagement.html 7 of the Best Free Linux Configuration Management Tools]
*[http://www.linuxlinks.com/article/20100222155850632/ConfigurationManagement.html 7 of the Best Free Linux Configuration Management Tools]
-
* [http://trac.mcs.anl.gov/projects/bcfg2/wiki/Bcfg2Doc Bcfg2 documentation]
+
* [http://docs.bcfg2.org Bcfg2 documentation]
== Device management? Why bother? ==
== Device management? Why bother? ==
Line 72: Line 72:
== Getting Bcfg2 up and running ==
== Getting Bcfg2 up and running ==
-
<font color="red">A word of warning</font>: Configuration management is complicated task. Learning Bcfg2 and the concepts behind it does take some time and effort. Do NOT by try to manage Maemo clients as your first Bcfg2 rehearsal. Instead, familiarize yourself with Bcfg2 first using "ordinary" computers  as clients.
+
<font color="red">A word of warning</font>: Configuration management is complicated task. Learning Bcfg2 and the concepts behind it does take some time and effort. Do NOT try to manage Maemo clients as your first Bcfg2 rehearsal. Instead, familiarize yourself with Bcfg2 first using "ordinary" computers  as clients.
=== Problems with Bcfg2 in Maemo ===
=== Problems with Bcfg2 in Maemo ===
Line 192: Line 192:
  This is a simple file
  This is a simple file
-
Edit file <code>/var/lib/bcfg2/Cfg/etc/simple/simple</code> to contain following:
+
Edit file <code>/var/lib/bcfg2/Cfg/etc/simple/info.xml</code> to contain following:
  <FileInfo>
  <FileInfo>
Line 257: Line 257:
=== Autogroup probe ===
=== Autogroup probe ===
-
Bcfg2 is able to automatically determine group memberships using probes. [http://wiki.maemo.org/Image:Autogroup.sh Here] is a sample probe code which recognizes Maemo devices among many other computers. Just put the code into a file at Probes directory, for example, <code>/var/lib/bcfg2/Probes/autogroup</code>
+
Bcfg2 is able to automatically determine group memberships using probes. Aa sample probe code which recognizes Maemo devices among many other computers is shown below:
 +
 
 +
<source lang="bash">
 +
#! /bin/sh
 +
 
 +
opi=/usr/bin/osso-product-info
 +
swv=/usr/bin/sw_vers
 +
 
 +
# Are we Linux?
 +
if uname 2>/dev/null | grep -q "Linux"
 +
then
 +
    # LSB saves the day
 +
    if lsb_release -a > /dev/null 2>&1
 +
    then
 +
        lsb_release -ci 2>/dev/null |\
 +
        awk -F: '{gsub(/^[ \t]+/, "", $2);print "group:" tolower($2)}'
 +
    # It did not? We might be maemo
 +
    elif [ -x $opi ]
 +
    then
 +
        if $opi 2>/dev/null | grep "OSSO_PRODUCT_RELEASE_NAME" | grep -q 'Maemo 5'
 +
        then
 +
            echo "group:fremantle"
 +
        elif $opi 2>/dev/null | grep "OSSO_PRODUCT_RELEASE_NAME" | grep -q 'Maemo 6'
 +
        then
 +
            echo "group:harmattan"
 +
        fi
 +
        echo "group:maemo"
 +
    # We were not? Get desperate
 +
    elif uname -v 2>/dev/null | grep -q "Ubuntu"
 +
    then
 +
        echo "group:ubuntu"
 +
    elif [ -f /etc/fedora-release ]
 +
    then
 +
        echo "group:fedora"
 +
    elif [ -f /etc/redhat-release ]
 +
    then
 +
        echo "group:redhat"
 +
    else
 +
        echo "group:linux"
 +
    fi
 +
fi
 +
 
 +
# Are we Mac?
 +
if uname 2>/dev/null | grep -q "Darwin"
 +
then
 +
    if [ -x $swv ]
 +
    then
 +
        if  $swv 2>/dev/null | grep "ProductVersion:" | grep -q '10.6'
 +
        then
 +
            echo "group:snowleopard"
 +
        elif $swv 2>/dev/null | grep "ProductVersion:" | grep -q '10.5'
 +
        then
 +
            echo "group:leopard"
 +
        elif $swv 2>/dev/null | grep "ProductVersion:" | grep -q '10.4'
 +
        then
 +
            echo "group:tiger"
 +
        else
 +
            echo "group:macosx"
 +
        fi
 +
    else
 +
        echo "group:freebsd"
 +
    fi
 +
fi
 +
 
 +
# What arch?
 +
if uname -m | grep -E '^arm' > /dev/null 2>&1
 +
then
 +
    echo "group:armel"
 +
elif uname -m | grep -E '^i.?86'> /dev/null 2>&1
 +
then
 +
    echo "group:i386"
 +
elif uname -m | grep -E '^x86_64'> /dev/null 2>&1
 +
then
 +
    echo "group:amd64"
 +
else
 +
    echo "group:`uname -m`"
 +
fi
 +
</source>
 +
 
 +
Just put the code into a file at Probes directory, for example, <code>/var/lib/bcfg2/Probes/autogroup</code>
At Bcfg2 web site is another [http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Probes/examples/group example] of autogroup probe code. This code does not recognize Maemo but might do better work with some other platforms.
At Bcfg2 web site is another [http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Probes/examples/group example] of autogroup probe code. This code does not recognize Maemo but might do better work with some other platforms.
Line 280: Line 359:
#* Suitable approach for servers but brings tiny device at it's knees
#* Suitable approach for servers but brings tiny device at it's knees
#* We probably only want to control that package X is installed and let package manager take care of the rest (updates & such)
#* We probably only want to control that package X is installed and let package manager take care of the rest (updates & such)
-
#* No any form of proxy handling (odds being inside corporate firewall are good)
+
# No any form of proxy handling (odds being inside corporate firewall are good)
==== Opportunities ====
==== Opportunities ====
Line 290: Line 369:
We need to write a "APTlite" client driver. Probably modifying an simple existing driver like <code>FreeBSDPackage.py</code> would do. Some notes about the task:
We need to write a "APTlite" client driver. Probably modifying an simple existing driver like <code>FreeBSDPackage.py</code> would do. Some notes about the task:
-
* RefreshPackages needs to be modified to include only packages received from server and ignore the rest. Looks like following piece of code generates the list we need
+
* <code>RefreshPackages</code> method needs to be modified to include only packages received from server and ignore the rest. Looks like following piece of code generates the list we need
  desired = [entry.get('name') for struct in self.config for entry in struct if entry.tag == 'Package']
  desired = [entry.get('name') for struct in self.config for entry in struct if entry.tag == 'Package']
-
* Then, we could run something like this for each pkg in desired list:
+
* Then, we could run something like this for each pkg in desired list to fill up the <code>self.installed</code> dictionary:
  /usr/bin/dpkg-query -W --showformat='${Version}\t${Status}\n' <pkg>
  /usr/bin/dpkg-query -W --showformat='${Version}\t${Status}\n' <pkg>
-
* Package verification (VerifyPackage) needs probably to be made even more liberal, like returning true if entry.attrib['version'] is 'auto' or 'any'
+
* <code>VerifyPackage</code> method needs probably to be made a bit more liberal, like returning true if <code>entry.attrib['version']</code> equals <code>'auto'</code>  or <code>'any'</code>
 +
 
 +
* <code>pkgtool</code> could probably be copied from <code>APT.py</code>
 +
 
 +
* Proxies could be copied from shell environment by adding them into <code>pkgtool</code>. Something like
 +
 
 +
proxies = ""
 +
for p in ('http','https','ftp'):
 +
    if os.environ.has_key('%s_proxy' % p):
 +
        proxies += '%s=%s ' % ('%s_proxy' % p, os.environ.get('%s_proxy' % p))
 +
pkgtool = proxies + ' ' + pkgtool
 +
 
 +
And how to get proxies into shell variables? One option is to use Bcfg2 to manage some file which is sourced by shell
[[Category:Power users]]
[[Category:Power users]]

Latest revision as of 07:49, 20 February 2011

This article contains information about using Bcfg2, an open source configuration management system in management of Maemo based devices.

Note! At this phase, although the title says otherwise, instructions given in this article do NOT constitute a device management solution. At the moment these instructions only guide to experiment device management using Bcfg2.

Contents

[edit] Scope and terminology

Scope of the article is using Bcfg2 to manage Maemo devices of employees at a fairly large company, where the number of devices is counted on hundreds or thousands. In a private use, or in small companies things covered here hardly make any sense.

In the remainder if this document, following terminology is used

Term Description
Device Maemo based handset, such as Nokia N900
Device management Generally used term for configuration management which takes place on Devices
Enterprise A large company or other organization that wants employees to ba able to acces company IT systems using Maemo based devices
Desktop computer A full-size computer (traditional desktop or laptop) used to access corporate IT systems
Enterprise configuration A set of applications and configuration values which the Enterprise wants to deploy into the Device as a prerequisite for accessing corporate IT systems. Usually includes hardening the device security.
Provisioning The process which equips the Device with Enterprise configuration

Instructions given in this article are tested on Fremantle. They may work on other releases as well, but probably not.

[edit] Recommended reading

Basic use and concepts of Bcfg2 are not in the scope of this article. To get familiar with Bcfg2, following reading is recommended:

[edit] Device management? Why bother?

Provisioning Devices without management is like standing on the waterfront, throwing rocks in to a lake trying to get them land into a bucket at a bottom. The throwing movement can be very controlled and calculated, but once the rock is released from hand, all control and traceability is lost. As a result, we do not really know if it ever hit the bucket. And if it did not, there is no way to get it there. Not to mention ability to move it into another bucket if so desired. Over time, we also inevitably lose track of how many rocks have we have thrown.

Hence, Provisioning-only approach leads at least following shortcomings

  • No statistics about Devices Provisioned
  • No information about success of the Provisioning
  • No method to fix failed Provisioning
  • No method for managing changes at Enterprise configuration (other than publish a new release hoping users will pick it up)

Using the rock analogy, in Device management approach we never completely release the rock. We tie a thin nylon line into each rock before throwing it. Now we can follow the line and track whether the rock did hit the bucket or not. We can use the line to lift misses to the bucket. We can also use the line to move them to another bucket if so desired. We can also easily have statistics about the rocks thrown; just count the lines.

It should be also noted that there are legislations such as Sarbanes-Oxley which mandate keeping track of computers able to access and store corporate data.

[edit] Why Bcfg2?

Openness is at heart of the Maemo philosophy. Thus, using an Open Source configuration management system seems a logical choice to try out.

Bcfg2 was chosen as the first candidate to try-out because

  • Architecture is server-centric. Processing is performed at the server end as much as possible. This makes the client lightweight. It makes it also simpler and less frequently changing.
  • Device management is a special use case for a software like this. They are all geared more for server and desktop management. Thus, fair amount of customization is anticipated. Bcfg2 has very flexible plugin architecture where most of it's core functionality implemented as plugins. This makes it very customization-friendly, nearly all components are replaceable
  • Anticipation of customization puts lot of weight to implementation language. Bcfg2 is written in Python, which suits the author best.

That said, there is no reason why other configuration management systems such as CFEngine or Puppet wouldn't work as well. (Actually, Puppet was tried out, got successfully running at the Device. Only it did not communicate with the server. This is possibly caused by the SSL problem discussed later. That's was the point where lack of author's Ruby skills kicked in :)


[edit] Getting Bcfg2 up and running

A word of warning: Configuration management is complicated task. Learning Bcfg2 and the concepts behind it does take some time and effort. Do NOT try to manage Maemo clients as your first Bcfg2 rehearsal. Instead, familiarize yourself with Bcfg2 first using "ordinary" computers as clients.

[edit] Problems with Bcfg2 in Maemo

At the moment there are some problems we need to work around in order to install Bcfg2 client into Device.

  1. Lack of proper SSL support
    • Bcfg2 prior to 1.0 used Python implementation of SSL called tlslite. At 1.0 tlslite war replaced with Python 2.6 built-in SSL module
    • Bcfg2 has internal fallback to use M2crypto module if SSL module fails
    • Maemo (Fremantle), however, has Python 2.5, which has no SSL module and no M2crypto module either
      • There is actually already bug report filed about the problem. However, the original problem (importing SSL module) was never solved, the problem the reporter faces seems to be worked around other way
    • Possible workarounds:
      • Compile SSL 1.15 module into Python 2.5
      • Compile M2crypto module into Python 2.5
        • Both fail into lack of complete set of OpenSSL development headers
      • Re-include tlslib into Bcfg2
        • Possible, however there will be no server identity validation
      • Use Bcfg2 prior to 1.0 in Device
        • Possible, however there will be no server identity validation
    • Contributions welcome
  2. Bcfg2 is not packaged for Maemo
    • Must be installed from source
  3. There is no good way for bootstrapping right now
    • Installation must be performed from Device command line

[edit] Server installation

Download and install version 1.0.1 following instructions found at Bcfg2 web site

Note that the server should reside in a network the Device is able to access.

[edit] Client installation

[edit] Option 1: Install old version

Download bcfg2-0.9.6 in to the Device. Open terminal window and install Bcfg2 by entering following commands

apt-get install python
tar zxvf bcfg2-0.9.6.tar.gz
cd bcfg2-0.9.6
python setup.py install --install-layout deb --record /root/bcfg2files

[edit] Option 2: Install current version

This option is somewhat more complicated since we need to re-include tlslib into Bcfg2

Download bcfg2-0.9.6 and bcfg2-1.0.1 both. Extract them:

tar zxvf bcfg2-0.9.6.tar.gz 
tar zxvf bcfg2-1.0.1.tar.gz

Get tlslib and from older version

cp -r bcfg2-0.9.6/src/lib/tlslite bcfg2-1.0.1/src/lib

Download this version of Proxy.py from Bcfg2 site and save it as bcfg2-1.0.1/src/lib/Proxy.py

Edit bcfg2-1.0.1/src/lib/Proxy.py. Add dummy placeholders ca and allowedServerCNs into ComponentProxy definition so it looks like below:

def ComponentProxy (url, user=None, password=None, fingerprint=None,
                    key=None, ca=None, allowedServerCNs=None, cert=None):

Edit bcfg2-1.0.1/setup.py Add the packages Bcfg2.tlslite, Bcfg2.tlslite.integration, and Bcfg2.tlslite.utils back into the packages list in setup.py, as seen in here

Repackage the source directory

tar zcvf bcfg2-1.0.1-mod.tar.gz bcfg2-1.0.1

Transfer bcfg2-1.0.1-mod.tar.gz to the Device. Open (at Device) terminal window and install Bcfg2 by entering following commands

apt-get install python
tar zxvf bcfg2-1.0.1-mod.tar.gz
cd bcfg2-1.0.1
python setup.py install --install-layout deb --record /root/bcfg2files

[edit] Installation notes

Reinstalling Bcfg2: Remove bcfg2-1.0.1/build directory before re-run of setup.py

Removing Bcfg2: remove files listed at /root/bcfg2files

[edit] Simple sample configuration

In the following we create a simple configuration at the Bcfg2 server, just to make you get on board quicker (empty configurations are not very illustrative). In the sample configuration we do three things:

  • Manage the content of a simple file /etc/simple
  • Manage the content of a file /etc/bcfg2.info using template
  • Run a simple action, a shell command ls / > /tmp/foobar

Throughout this article we assume the configuration repository is at default location /var/lib/bcfg2.

Edit file /var/lib/bcfg2/Metadata/groups.xml to contain following:

<Groups>
    <Group name='armel'/>
    <Group name='linux'/>
    <Group name='deb'>
        <Group name='linux'/>
    </Group>
    <Group name='maemo'>
        <Group name='deb'/>
        <Bundle name='sample'/>
    </Group>
    <Group name='fremantle'>
        <Group name='maemo'/>
    </Group>
</Groups>

Edit file /var/lib/bcfg2/Bundler/sample.xml to contain following:

<Bundle name='sample'>
    <ConfigFile name='/etc/simple'/>
    <ConfigFile name='/etc/bcfg2.info'/>
    <BoundAction name="simple" timing='post' when='always' status='check' command="ls / > /tmp/foobar"/>
</Bundle>

Edit file /var/lib/bcfg2/Cfg/etc/simple/simple to contain following (Create directories as needed):

This is a simple file

Edit file /var/lib/bcfg2/Cfg/etc/simple/info.xml to contain following:

<FileInfo>
    <Info owner='root' group='root' perms='0644' encoding='ascii'/>
</FileInfo>


Edit file /var/lib/bcfg2/TCheetah/etc/bcfg2.info/template to contain following (Create directories as needed):

Hostname: $self.metadata.hostname
Uuid: $self.metadata.uuid
Password: $self.metadata.password
Profile: $self.metadata.profile
Groups: #echo ','.join($self.metadata.groups)#
Bundles: #echo ','.join($self.metadata.bundles)#

Edit file /var/lib/bcfg2/TCheetah/etc/bcfg2.info/info.xml to contain following:

<FileInfo>
    <Info owner='root' group='root' perms='0644' encoding='ascii'/>
</FileInfo>


[edit] Some explanation:

Bcfg2 builds the configuration using layered approach:

  • Metadata (roughly: "which kind of configuration should be where")
  • Abstract (roughly: what should be configured)
  • Literal (roughly: how exactly that "what" should be achieved)

In this example, we have two instances of the same abstract configuration item (ConfigFile). They are however handled by two different literal configuration generators, Cfg, which handles simple files and TCheetah which handles more complicated files using built-in Cheetah templating engine.

One thing to like in Bcfg2 is that it does not mandate things too much. Third item is an example of that. Bcfg2 allows short-circuiting the Literal layer processing altogether by adding "Bound" in front of the keyword. In a simple cases where what we want exactly is already known already at abstract level this can simplify things.

[edit] Hooking the Device and Bcfg2 server together

At server, add following line into /var/lib/bcfg2/Metadata/clients.xml

<Client uuid="foo" name="bar" profile="maemo" password="xyzzy" pingable="N" location="floating" auth="cert+password"/>

No need to restart the Bcfg2 server, it picks the changes on the fly. Next, at the Device, edit the file /etc/bcfg2.conf to look like following:

[communication]
protocol = xmlrpc/ssl
user = foo
password = xyzzy

[components]
bcfg2 = https://bcfg2server.example.com:6789

Now you should be able to invoke Bcfg2 client and make first connection to the server using command

bcfg2 -I

You should now be prompted a confirmation for three configuration item defined in the sample configuration, due to use of -I option.


[edit] Appendix: Hints on Bcfg2 usage

[edit] Autogroup probe

Bcfg2 is able to automatically determine group memberships using probes. Aa sample probe code which recognizes Maemo devices among many other computers is shown below:

#! /bin/sh
 
opi=/usr/bin/osso-product-info
swv=/usr/bin/sw_vers
 
# Are we Linux?
if uname 2>/dev/null | grep -q "Linux"
then
    # LSB saves the day
    if lsb_release -a > /dev/null 2>&1
    then
        lsb_release -ci 2>/dev/null |\
        awk -F: '{gsub(/^[ \t]+/, "", $2);print "group:" tolower($2)}'
    # It did not? We might be maemo
    elif [ -x $opi ]
    then
        if $opi 2>/dev/null | grep "OSSO_PRODUCT_RELEASE_NAME" | grep -q 'Maemo 5'
        then
            echo "group:fremantle"
        elif $opi 2>/dev/null | grep "OSSO_PRODUCT_RELEASE_NAME" | grep -q 'Maemo 6'
        then
            echo "group:harmattan"
        fi
        echo "group:maemo"
    # We were not? Get desperate
    elif uname -v 2>/dev/null | grep -q "Ubuntu"
    then 
        echo "group:ubuntu"
    elif [ -f /etc/fedora-release ]
    then
        echo "group:fedora"
    elif [ -f /etc/redhat-release ]
    then
        echo "group:redhat"
    else
        echo "group:linux"
    fi
fi
 
# Are we Mac?
if uname 2>/dev/null | grep -q "Darwin"
then
    if [ -x $swv ]
    then
        if   $swv 2>/dev/null | grep "ProductVersion:" | grep -q '10.6'
        then
            echo "group:snowleopard"
        elif $swv 2>/dev/null | grep "ProductVersion:" | grep -q '10.5'
        then
            echo "group:leopard"
        elif $swv 2>/dev/null | grep "ProductVersion:" | grep -q '10.4'
        then
            echo "group:tiger"
        else
            echo "group:macosx"
        fi
    else
        echo "group:freebsd"
    fi
fi
 
# What arch?
if uname -m | grep -E '^arm' > /dev/null 2>&1
then
    echo "group:armel"
elif uname -m | grep -E '^i.?86'> /dev/null 2>&1
then
    echo "group:i386"
elif uname -m | grep -E '^x86_64'> /dev/null 2>&1
then
    echo "group:amd64"
else
    echo "group:`uname -m`" 
fi

Just put the code into a file at Probes directory, for example, /var/lib/bcfg2/Probes/autogroup

At Bcfg2 web site is another example of autogroup probe code. This code does not recognize Maemo but might do better work with some other platforms.


[edit] TODO

This is the "dont look here" part, stuff under construction


[edit] APT package driver

[edit] Problems

  1. Maemo does not have debsums utility
    • Causes APT package driver not to load
  2. Maemo has no Python bindings to APT cache
    • Causes APT package driver not to load/work
  3. APT package driver is too "control freak" for the purpose
    • Wants to check package versions and even verify all the files
    • Suitable approach for servers but brings tiny device at it's knees
    • We probably only want to control that package X is installed and let package manager take care of the rest (updates & such)
  4. No any form of proxy handling (odds being inside corporate firewall are good)

[edit] Opportunities

Hildon application manager is not able to go through proxies, but APT is

[edit] Proposed solution

We need to write a "APTlite" client driver. Probably modifying an simple existing driver like FreeBSDPackage.py would do. Some notes about the task:

  • RefreshPackages method needs to be modified to include only packages received from server and ignore the rest. Looks like following piece of code generates the list we need
desired = [entry.get('name') for struct in self.config for entry in struct if entry.tag == 'Package']
  • Then, we could run something like this for each pkg in desired list to fill up the self.installed dictionary:
/usr/bin/dpkg-query -W --showformat='${Version}\t${Status}\n' <pkg>
  • VerifyPackage method needs probably to be made a bit more liberal, like returning true if entry.attrib['version'] equals 'auto' or 'any'
  • pkgtool could probably be copied from APT.py
  • Proxies could be copied from shell environment by adding them into pkgtool. Something like
proxies = ""
for p in ('http','https','ftp'):
    if os.environ.has_key('%s_proxy' % p):
        proxies += '%s=%s ' % ('%s_proxy' % p, os.environ.get('%s_proxy' % p))
pkgtool = proxies + ' ' + pkgtool

And how to get proxies into shell variables? One option is to use Bcfg2 to manage some file which is sourced by shell