wtogami (wtogami) wrote,

Report: LTSP Hackfest 2007

LTSP is Linux Terminal Server Project, a collection of tools and scripts to enable Linux desktops to networks of thin clients.  Through K12LTSP, literally thousands of schools and hundreds of thousands of students worldwide have been exposed to Red Hat and Fedora Linux over the past 6+ years.  This report is on my trip to the latest LTSP hackfest, where I am working toward integration of LTSP directly into the Fedora Project.

Every year LTSP has a hackfest near the ocean, which they call LTSP BTS or By-The-Sea.  Saturday November 3rd began this year's hackfest at the Seawall Motel in Southwest Harbor, ME right next to Acadia National Park.  The motel is located next to some unique and beautiful scenery.  This being well outside of peak season, rates are lower, and we basically take over the motel and its small conference center.  Almost no cell phone reception there, but I was getting 1MB/sec download speeds on their wireless network.

Reorganizing LTSP Upstream
The main goal for both me and Eric Harrison for this hackfest is to transform LTSP-5 currently used by Ubuntu to be in suitable shape for Fedora.  The LTSP-5 as-is today is exclusively on Debian and Ubuntu, and it has a number of problems:
  • no upstream versioned tarballs and no semblance of a typical "upstream" project release methodology.
  • almost zero development communication on a mailing list, they do everything on IRC, difficult to enable others to join.
  • Poorly organized bzr source repository.
I showed LTSP lead developer Scott Balneaves, Canonical developer Oliver Gawert and Debian LTSP maintainer Vagrant Cascadian my example ltsp-server bzr repository.  Worked together to reorganize the upstream source repository in a few ways:
  • split pieces that could be independent software into separate repositories and packages.
  • reorganize the repository layout: use Makefile's to automate release tagging and tarball creation.
  • standardize on Makefile modes to allow installation of distribution specific plugins.
Convincing upstream LTSP to go through this source repo reorg was a crucial first step to allow upstream LTSP-5 to support other distributions.  These upstream BZR repositories are now located at:
  • bzr get http://www.ltsp.org/~sbalneav/jetpipe
  • bzr get http://www.ltsp.org/~sbalneav/ldm
  • bzr get http://www.ltsp.org/~sbalneav/ltsp
  • bzr get http://www.ltsp.org/~sbalneav/ltspfs
Currently the code contained is not yet functional for anyone.  We will be working on refactoring and merging of Fedora-specific code next.

Thin Client Hardware Problems
AMD Geode based thin clients have severe problems with X.  Currently X.org driver support is currently very bad for the integrated video in these boxes.  One reportedly common SIS based thin client is almost fully working, except the SIS-based sound currently has no available Linux driver.  I am working with two separate vendors (both leading LTSP hackers) on getting hardware to the Red Hat Westford office for development/testing.

Various Things People Were Working On...
  • Scott Balneaves worked on reworking the upstream bzr repositories to use my suggested repository structure.  He then auto-toolified each repo to allow configure, make, make install that will work easily on each target distribution.
  • Warren got NBD boot to work instead of NFS boot.  NBD is a lot faster to boot, especially for slower thin client hardware.  Ubuntu uses unionfs on top of either NFS or NBD root in order to achieve the convenience of a read-write filesystem on their thin clients.  I tried to use the devicemapper trick similar to Fedora's LiveCD, but ran into some challenges.  Firstly, our NBD image would need to be ext3 (possibly within squashfs) instead of squashfs directly.  Secondly, LiveCD achieves the devicemapper block-level read-write only through a very complicated mkinitrd replacement called mayflower, which has no relation to mkinitrd.  Quite a bit of further work will be necessary to get this working in a non-hackish fashion.  Although perhaps Fedora-LTSP doesn't need read-write root, because the /etc/sysconfig/readonly-root mode works fairly well.  In any case optional NBD boot is a necessary performance win.
  • Warren booted Eric's Intel 950 video laptop as a thin client.  compiz running on Warren's laptop worked on Eric's laptop with spinning cube and wobbly windows with seemingly no performance impact.  compiz used less than 1% of CPU on Warren's laptop.  Pretty cool.
  • Warren attempted to get bootcharts of thin-client boot, but failed.  bootchartd seems to have problems with readonly-root mode.  Even with /sbin/bootchartd edited to use /dev/somewhere for its tmpfs, it fails to stop in the presence of certain configured processes (like Xorg) and "bootchartd stop" exhibits race conditions as it attempts to tar up the results.  Some help here would be appreciated.  You can easily test this on any Fedora system by installing bootchart and enabling readonly-root mode.
  • We tried to boot Fedora with dash as /bin/sh instead of /bin/bash.  Reportedly, while Upstart is credited with the boot speed improvements of Ubuntu, most boot performance gains were achieved due to dash.  A simple fix to /etc/rc.d/init.d/functions:163 allows most of Fedora's initscripts to work.  The thin client boot scripts are simple enough to work without any problems, while my laptop had a few more issues.  Somebody else might want to look into making permanent corrections to initscripts to allow dash compatibility.  #!/bin/bash can be used in initscripts that explicitly need bash scripting, like iptables.
  • Gideon Romm worked on local-apps support, which allows usage of some apps running on the thin client hardware along-side desktop and other apps running on the remote server.
Current Status [Bug Tracker]
Much progress was made during the hackfest.  Lots of work remains to have Fedora working to Ubuntu parity, and Fedora's LTSP needs fully integrated into the new upstream.
  • 359621: EASY: Correct several error messages that appear during readonly-root boot.
  • 369361: EASY: Bootchart failures with readonly-root mode
  • 369371: EASY: SELinux denies vncts and nbd-server from xinetd
  • 207341: DIFFICULT: Auto-Detection and loading of network drivers during initrd.  This is the most urgent thing needing to be fixed that cannot be done at LTSP upstream.
  • Integration of Fedora-specific ways of doing things (like anaconda or mock chroot install) into ltsp-build-client.  Split this out into upstream's plugin structure.
  • Make sure truly common upstream bits are pulled out of the debian directory, while Debian specific parts are moved into their own plugins.
  • Package the new upstream ltsp, which has sub-packages ltsp-client and ltsp-server.
  • Package jetpipe.
  • Package ltspfs.
  • Package ldm.  LDM is an alternate display manager that allows ssh authentication and (optional) session encryption.
If we get the above done, we can spin a new K12LTSP distribution based on Fedora 8.  I hope we can do this within the next month.

Development Communication
Please discuss all LTSP (even Fedora-specific) development on upstream's list.  You can also find LTSP in irc.freenode.net channel #ltsp.

Other Note: Ubuntu's External Community Relations
Jorge Castro was recently hired by Canonical to work on Ubuntu's relationships with upstream projects.  He described his role as both getting Ubuntu's patches into upstream, and also forming relationships with other distributions to work on common problems upstream.  Perhaps there are potential opportunities in working with Jorge (pronounced George) in the future.  In particular he had interest in the future direction of smolt, perhaps as an "upstream" distribution agnostic hardware compatibility database project.
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded