Introduction
This weekend I build myself a FreeBSD desktop. Notably, it runs Chrome (or rather, Chromium) flawlessly and supports Flash video (HD, full screen, full audio, no stuttering). Furthermore, it turns out KDE 4 is a beautiful desktop environment which I really enjoy using. I totally didn't expect this to turn out as well as it did.
Motivation
I originally had a coding project in mind for this weekend. However, when I sat down on Saturday to begin work, I was immediately confronted by an annoying fact: I hate coding on Mac. This is mostly for little reasons: the home and end keys uselessly scroll to the top or bottom of the document rather than move to the beginning or end of the line; ctrl+left/right does what home and end are supposed to do, rather than skipping forward or back by one word; pgup/pgdn move the view but not the cursor; GUI hotkeys all use "command" instead of "control" but terminal apps still revolve around "control"; etc. Some of these things can be partially mitigated by configuration, but not completely -- the command/control inconsistency is a stubborn one that isn't fixed by merely swapping the keys.
Of course, if I were a devoted Mac user I could get used to all this, but I do the vast majority of my coding at work, on Linux. Also, given Apple's anti-competitive behavior with the iPhone platform, I've begun to feel dirty using even their desktop products.
Meanwhile, playing with hardware last weekend was fun and I was itching to do it again. So, the natural thing to do was to go out and buy the pieces for a new machine.
Hardware
Requirements:
- Must run FreeBSD.
- Optimized for compiling code -- a task that is easily parallelized.
- Must drive two 30-inch dual-link DVI monitors.
- Avoid excessive power consumption.
Non-requirements:
- Not a gaming machine.
- Does not need top-of-the-line (read: exponentially expensive) hardware.
- Does not need much hard drive space.
I ended up settling on:
- Intel Core i7-930 processor: Four cores, hyperthreaded. Costs a quarter of the six-core i7 extreme.
- Intel DX58SO "smackover" motherboard: If it's Intel's processor and Intel's chipset anyway, might as well let them make the board too. More likely to work right. There does not appear to be any nForce chipset for the i7 currently.
- GeForce GTS 250: nVidia actually offers FreeBSD drivers based on the same codebase as their Windows drivers, including the whole OpenGL implementation. The 250 appeared to be the cheapest modern card that met the dual-DVI requirement.
- 3x2GB RAM: The motherboard has three RAM channels so you want to put three sticks in it.
- 128GB SSD: Yay for fast seeks and cool, quiet operation. Mass storage belongs on the network anyway.
Operating System
I chose FreeBSD for two reasons:
- When writing Evlan in 2004-ish, I learned that the FreeBSD kernel offered many features that I wanted which Linux sorely lacked. Things like kqueue and AIO are key to implementing a framework based on event-loop concurrency, as Evlan was. I believe Linux has since gained equivalent features, but I was left with an impression that FreeBSD was more thoughtfully designed whereas Linux was more ad hoc.
- I explicitly wanted a challenge. If I wanted something that "just worked", I'd be installing Windows. But I want a project where I learn something, and feel some sense of satisfaction in the end. FreeBSD is more difficult to set up, but in the process you learn how the system works, and once configured it can run essentially all the same software as Linux.
I installed FreeBSD 8.0 (UPDATE: 8.1 -- see end of post).
Details
Make no mistake: FreeBSD is not ready for the desktop. It works great once you get it set up, but it requires a lot of work to get there. As I said above, I explicitly wanted to tinker, because it is fun and educational. But anyone who wants a system that "just works" should look elsewhere.
Below are things I did, what worked, what didn't, and how I solved those problems. Hopefully someone will find these useful.
Installer
FreeBSD can be installed from a USB memory stick. They even provide a disk image for this exact purpose. Yay! I was able to write the disk image from within Mac OSX using the "dd" command as described in the FreeBSD release notes. The System Profiler utility reported that my USB drive was at /dev/disk1. However, before OSX would let me write, I had to unmount the flash drive -- NOT by "ejecting" it, which made it disappear entirely, but by using the "umount" command.
The FreeBSD installer is not very user-friendly. Being difficult to understand is bad, but being difficult to understand *and* not having any way to go back if you made a mistake is much worse. It turns out the standard installation mode has this property -- you usually cannot go back if you chose the wrong thing. However, the "custom install" mode -- which is labeled "for experts" -- actually presents you with a list of all the steps, from which you can choose which ones to perform. You should still go through them in order, but this allows you to repeat steps or go backwards when you make a mistake.
The installer initially could not detect my memory stick, even though it was installing from said stick. To fix this I had to choose "options" from the top-level menu, and then choose "rescan devices" (or something like that) from the options menu. This appeared to do nothing, but when I then went back and told the installer to install from USB again, it worked. (UPDATE: Fixed in FreeBSD 8.1.)
Installing packages
The memory stick installer only provides basic packages. Any interesting programs have to be installed separately, through the "packages" system (pre-compiled binaries downloaded from freebsd.org) or the "ports" system (scripts which automatically download original source code and compile/install it for you). I prefer packages whenever they are available, since they're faster. The "sysinstall" command can be run after installation in order to select additional packages to download, which is good for going through and getting the basics together. "pkg_add -r" can be used to install specific packages by name, on-demand (e.g. "pkg_add -r protobuf").
With this I installed X.org and kde4, among other things.
Graphics driver
The nVidia driver is not included with FreeBSD because it is not open source, and requires agreeing to a license. Whatever. There is a "port" for it, but it is out of date. I downloaded the most recent version from the nVidia web site and followed their installation instructions. This turned out to be surprisingly easy -- they actually provide a script which updates all the necessary config files for you! What a crazy idea.
For a long time, I could not convince Xorg to display at a reasonable resolution; it kept giving me something like 1280x800 (though I swear it looked more like 320x240; I must be spoiled). Turns out I needed this line in the "Screen" section of my xorg.conf:
Option "AllowDualLinkModes" "True"
WTF? Why on Earth would an option like this not be on by default? (This was actually before I installed the nVidia driver; I was still using the open source "nv" driver. Not sure if it makes a difference.)
Graphical login
To enable graphical login I had to edit /etc/ttys and change the "ttyv8" line to:
ttyv8 "/usr/local/kde4/bin/kdm -nodaemon" xterm on secure
This gave me a nice KDE-style login prompt on startup, but with a major problem: the keyboard layout was Qwerty, while I use Dvorak. Changing my keyboard layout in the KDE settings UI did not fix it, since those are user-specific and don't apply to the login prompt. Changing xorg.conf *also* did not fix it: under recent versions of FreeBSD, InputDevices in xorg.conf are ignored in favor of letting the USB daemons "automatically" figure it out. Of course, said daemons don't know what keyboard layout I use. The FreeBSD manual contained instructions on how to tell it (via /usr/local/etc/hal/fdi/policy/x11-input.fdi), but they didn't work -- the file seemed to have no effect. I banged my head against this for a long time. Nothing I did could even make Xorg print an error; it just ignored everything.
Eventually I figured out that kdm runs /usr/local/kde4/share/config/kdm/Xsetup before displaying the login dialog. So I inserted "setxkbmap dvorak" into it, which fixed the problem. It's a hack but gets the job done.
Audio
Enabling audio was as simple as loading the right sound driver. In my case:
kldload snd_hda
However, this created four separate /dev/dsps. It turned out /dev/dsp0 referred to the line-out jacks on the back of my machine, while /dev/dsp1 referred to the front headphone jack (the other two were digital outputs). Most apps only play to /dev/dsp0; I'd prefer for this to go to all jacks.
It turns that the snd_hda driver supports rather complex configuration. Annoyingly, it requires you to know details that are different for every chipset. Although the driver knows these details, there is no simple tool to query it; you must reboot and pass -v as a kernel boot flag, then examine the results with dmesg. For me, the relevant output was:
hdac0: Processing audio FG cad=2 nid=1...
hdac0: GPIO: 0xc0000002 NumGPIO=2 NumGPO=0 NumGPI=0 GPIWake=1 GPIUnsol=1
hdac0: nid 17 0x01451140 as 4 seq 0 SPDIF-out Jack jack 5 loc 1 color Black misc 1
hdac0: nid 18 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1
hdac0: nid 20 0x01014410 as 1 seq 0 Line-out Jack jack 1 loc 1 color Green misc 4
hdac0: nid 21 0x02214420 as 2 seq 0 Headphones Jack jack 1 loc 2 color Green misc 4
hdac0: nid 22 0x01016011 as 1 seq 1 Line-out Jack jack 1 loc 1 color Orange misc 0
hdac0: nid 23 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1
hdac0: nid 24 0x02a19860 as 6 seq 0 Mic Jack jack 1 loc 2 color Pink misc 8
hdac0: nid 25 0x01011012 as 1 seq 2 Line-out Jack jack 1 loc 1 color Black misc 0
hdac0: nid 26 0x01813450 as 5 seq 0 Line-in Jack jack 1 loc 1 color Blue misc 4
hdac0: nid 27 0x01a1985f as 5 seq 15 Mic Jack jack 1 loc 1 color Pink misc 8
hdac0: nid 28 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1
This shows all the inputs and outputs the hardware has. The number after as indicates to which /dev/dsp the line will be connected (off by one). The seq indicates which speaker pair this is -- e.g. front, back, etc. Following the instructions, I added a line to /boot/device.hints:
hint.hdac.0.cad2.nid21.config="as=1 seq=15"
This says that to move nid 21 (the headphone jack) to as 1 (the first output device, which includes the back line-out jacks). Setting seq to 15 has a special meaning for headphones: when connected, all other speakers on the same as should be muted.
So, not only does my headphone jack work, but as an added bonus, plugging in headphones automatically mutes the other speakers. I completely didn't expect FreeBSD to support that. I wish it would have been set up that way by default, though. But I guess it's neat that I could actually set up each jack (three in the back, one in the front) to be an independent stereo output, with different programs playing to each one. I could use that for my house, to power the speakers that I'll have around the place.
Chrome
Following the instructions on the FreeBSD Chromium wiki page, I built Chromium. There were a few difficulties:
- Pulling the Git repo takes forever -- it's over 1GB! Next time, I'll try subversion.
- The "Sync source" step fails with an error, but this error appears to occur after everything has been synced. It looks like it tries to start some of the build process, but fails because you haven't applied the FreeBSD patch yet.
- The wiki fails to mention bison among the dependencies.
pkg_add -r bison
- The wiki lists ALSA as a dependency, but this package doesn't exist in FreeBSD 8.0. I had to grab the ALSA packages from a FreeBSD 8.1 release candidate instead. I got a warning about dependency version mismatches when I tried to
pkg_add them, but it seems to have worked fine.
With all that done, Chromium built and ran flawlessly. It's just like using Chrome on any other system. I was expecting some stuff to be semi-functional, but so far I've found nothing. It's fast, too.
Flash
I totally didn't expect to be able to use Flash at all on FreeBSD. On my Linux machine at work, Flash animations of any sort seem to blow away the CPU, and video in particular stutters horribly with no audio.
It turns out that FreeBSD has a port which sets up the Flash plugin to run in the Linux emulation layer. Amazingly, the plugin can run on the Linux ABI even when the rest of the browser is running natively.
The port installer ran into trouble when the Linux Flash installer downloaded from Adobe did not match the expected size and hash. It looks like they had released a new version since the port was written -- the file was not versioned. I had to modify the port to tell it the new hash so it would accept the file (I didn't want to find the old version because it may have had security problems). This did not seem to create any problems.
The port installer had several dependencies on Linux stuff which was pulled from Fedora Core 10. It seemed to have a lot of trouble finding a working Fedora mirror for this stuff. I had to download several packages manually after finding them on Google. (Fortunately all the hashes matched.)
Once installed properly, Chrome picked up the plugin automatically. Amazingly, video played flawlessly, with sound, at HD resolution, full screen.
Fonts
On initially installing Chrome, I noticed that some fonts (especially the debug console) looked bad or even unreadable (due to under-sampling). Also, most unicode characters were replaced with boxes -- these turn out to be common even in English text. The solution was to install two ports:
- webfonts: This pulls a bunch of Windows truetype fonts off some site that seems to be hosting them and has nothing to do with FreeBSD. Technically since these are part of Windows, you must have a Windows license to use them. I, of course, have zillions of Windows licenses. (UPDATE: A commenter notes that actually Microsoft allows gratis redistribution of these fonts on the condition that they be distributed only in the form of the original exe files. So you have to run a script that extracts the fonts from the exes. I was confused by the pkg-descr which mentioned that a Windows license was needed only for the Tahoma font.)
- code2000: A "shareware" unicode font. Installing this got rid of the boxes. Apparently I'm supposed to pay someone $5 if I continue to use it, though.
Suspend/Resume
After digging around for awhile, I figured out that the proper way to make FreeBSD go to sleep is with the zzz command. So far I've tested this once, and it failed to wake up -- it partially awoke (responded to pings), but then spontaneously rebooted. Will have to play with this more.
iTunes replacement
I was sad to learn that Songbird -- a popular open source iTunes clone -- had discontinued Linux support (wat?), and certainly did not support FreeBSD. Doh.
However, Amarok looks like another decent alternative. I'm currently in the process of importing my iTunes library into it.
Update: So far Amarok failed at importing my iTunes library. The importer appeared to be O(n2) somehow -- it started out fine, but scanning each file became progressively slower, and after running overnight I came back to find Amarok using 100% of one CPU and making no progress. Grr. I'll have to investigate this next weekend. (UPDATE: A newer version of Amarok fixed this problem.)
Amarok's name and icon keep making me think of Three Wolf Moon.
Conclusion
Back in 1999 I used to be a huge Linux desktop fan, but in 2000 I gave it up in favor of Windows 2000 (which, unlike previous versions, did not crash daily). I had become tired of all the work it took to configure and maintain a Linux machine. Ten years later, configuring FreeBSD is about as much work as Linux was at the time. But, I think these days I'm more interested in this hands-on approach, and the ability to customize every aspect of my OS is appealing.
Meanwhile, I'm delighted to find that KDE seems to have kept up with the UI innovations made in Windows and Mac OSX over the last ten years. KDE can even do Mac-style Expose -- perhaps the thing I miss most whenever I work on a non-Mac system.
In general I've been pleasantly surprised by what FreeBSD/KDE has to offer as a desktop system. I had originally intended to use this machine for coding projects only, but now it looks like I may as well make it my primary desktop. Now that everything's configured, I don't see anything significant that I'll miss from OSX, and using a fully open source system feels so much nicer.
Dénouement
So, as it turns out, a mere two days after I installed FreeBSD 8.0, they released version 8.1. Argh. So this weekend I had to update, leading to a new series of issues...
Hard drive has a new address
Apparently the new kernel version decided that that it really preferred to put my hard drive at /dev/ad10 instead of /dev/ad6 like 8.0 did. As a result, when I first tried to boot the new kernel, it couldn't find the boot volume. Amazingly, though, instead of just crashing, it brought me to a prompt where it told me what hard drives and options it knew about, and asked me what to do. I was able to figure out what happened and tell it where to boot from. It then dumped me into a single-user shell from which I was able to edit /etc/fstab to fix the boot config, then reboot happily.
portupgrade
The update instructions say that you need to re-compile all of your software after updating the system, as the new base libraries may not be binary-compatible with the old. Ugh. It suggested running portupgrade -af for this purpose. I decided instead to run portupgrade -afP to tell it to use pre-compiled packages where available. I also updated my ports tree before starting so that I'd get the latest versions of everything.
Alas, one of these two changes (or maybe both?) seems to have been a deadly mistake. portupgrade took forever, and quite a few packages failed to install because of new dependencies that weren't present. KDE was completely broken after this. I tried to fix it by running portupgrade -RP kde4 to make it re-try installing kde4 and all its new dependencies. This again took ridiculously long, and repeatedly complained that it couldn't install required dependency Perl 5.10 because it conflicted with Perl 5.8. After manually forcing an uninstall of Perl 5.8 and install of Perl 5.10, I got to the point where I could start up KDE, but other things seemed broken. Chrome, in particular, was now SIGBUSing regularly, and continued to do so even after re-compiling it from scratch.
Screw it, start over
I downloaded the FreeBSD 8.1 memstick image, backed up a couple config files that I didn't want to rewrite, and set off to re-install from scratch. Sigh.
ZFS
Since I had to start over, I decided to do something I had forgotten on my first time through: use ZFS. This is a new, radically different filesystem which is, IMO, the way filesystems should be -- read the Wikipedia entry for details. Linux does not support ZFS because the ZFS code is under a GPL-incompatible license. FreeBSD, however, has no problem using ZFS. Hah!
There is still no support for ZFS in the FreeBSD installer. Lots of guides on the internet suggest a variety of ways to get FreeBSD on ZFS. I ended up using this one. The basic idea is to install a minimal FreeBSD system on a small partition, then create the ZFS volumes and migrate all data to them. The small original partition must stick around as the boot kernel cannot yet be read from a ZFS volume. But, once the kernel is up, it can map in the ZFS drives, so everything else can live there.
Eclipse
After setting up the rest of the system as before, I wanted to install Eclipse to do some coding. There were a couple bumps in the road.
First, you really need to install the eclipse-devel package. Plain old eclipse is way out-of-date.
Second, the package depends on jdk-1.6, but I actually installed OpenJDK instead. Luckily jdk-1.6 cannot be installed automatically since you have to accept a Sun license agreement -- this means that installing Eclipse will not accidentally install a whole second JDK. I just had to tell the installer to ignore the dependency problem, and then set my JAVA_HOME to OpenJDK and everything worked fine. (UPDATE: This lead to some crashes, unfortunately. So I gave in and installed JDK 1.6, which worked better. The annoying part about this is that you must build JDK from sources (licensing restrictions), and since it is itself written in Java, the port actually installs yet another JDK called "diablo" first. So now I have three whole Java implementations installed. Urgh.)
Third, when I went to Eclipse's built-in plugin installer to install the plugins I need, it didn't have any download sites registered. I had to copy them over from the Eclipse install on my Mac.
Fourth, Eclipse refused to talk to any of these sites, giving a cryptic error message about an invalid argument (to what, it didn't say). Googling revealed that the problem was that Java tries to use IPv6 by default, and if that doesn't work, throws an exception. Brilliant. The fix is to pass -vmargs -Djava.net.preferIPv4Stack=true to Eclipse to force IPv4.
Finally, when I installed the C++ development plugin (CDT) through the updater, it appeared to install fine, but seemed to have no effect -- none of the C++ development features showed up in Eclipse. After lots of head-scratching, I figured out that the CDT has a couple of native-code parts which have to be compiled explicitly for FreeBSD. I didn't want to download the full CDT source code to compile, but I figured out that I could pull the necessary bits out of the eclipse-cdt FreeBSD package despite it being way out-of-date -- apparently these parts don't change much. Here's what I did:
mkdir temp
cd temp
fetch ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8.1-release/Latest/eclipse-cdt.tbz
tar xf eclipse-cdt.tbz
cp -rv eclipse/plugins/*freebsd* ~/.eclipse/org.eclipse.platform_3.5.0_1216342082/plugins
Boot-up Splash Screen
It turns out that FreeBSD supports setting a boot-up splash screen, so you don't have to look at all that ugly text flying by. I followed these instructions to set it up -- he even provides a pretty nice image to use.
On the bright side
- The installer now detects the USB drive without a rescan.
- The newer version of Amarok successfully imported my iTunes collection.
- ALSA libs (needed by Chrome) are now part of the released distro.
- KDE looks a little prettier than before.
- OpenJDK 7 wasn't available in 8.0.
At this point, everything appears to be working. Hopefully now I can write some code!
Starcraft 2!
This is amazing. I can run Starcraft 2 on FreeBSD!
Kernel update
I had to upgrade to 8-STABLE. 8.1-RELEASE, even though it is only a couple weeks old, is too old: apparently a patch to fix Starcraft was just submitted recently. Note that the symptom of not doing this is that the game crashes when you try to log in.
Install Wine
I followed these instructions to install Wine. This is a little tricky: Only the 32-bit build of Wine currently works on FreeBSD, but of course I'm running the 64-bit kernel. Luckily, FreeBSD has a compatibility layer for running 32-bit binaries. Unfortunately, the ports system is not very well set up for this. I basically had to build/unpack a 32-bit FreeBSD base image into /compat/i386, chroot into it, and install Wine there. I also had to install the 32-bit version of the nvidia drivers, for the GL libraries -- note that it's important to use the exact same driver version.
winetricks
On the advice of this page I used winetricks to install several packages:
winetricks droid fontfix fontsmooth-rgb gdiplus gecko vcrun2008 vcrun2005 allfonts d3dx9 win7
I'm not sure all of these were actually necessary, though. Dan Kegel, winetricks author (and former Google employee whom I've met), replied to that post saying he didn't think all of those packages were really needed. I also had to install IE6 using winetricks as the Gecko-based HTML widget seemed to have troubles with the SC2 installer.
Get the installer
My machine does not have an optical disk drive, so I copied the files off the DVD from my Mac. This was a little tricky because by default my mac mounted the Mac partition of the DVD, hiding the Windows installer. I had to manually mount the correct partition:
umount /dev/disk1s2
mkdir mountpoint
mount_udf -o rdonly /dev/disk1s1 mountpoint
I could then copy all the files from mountpoint. I later couldn't get the disk to eject (even after mounting it normally) and had to reboot. Hrm.
TURN OFF COMPOSITING
If you are using a compositing window manager, disable it! For KDE, go to System Settings -> Desktop -> Desktop Effects and click the "Suspend Compositing" button. Otherwise 3D games are going to render slower than usual. Turn it back on when you're done playing. :)
Install
The installer ran fine. The patching process did not. It kept popping up something like "Microsoft C Runtime error". However, I noticed that even when this error popped up, the download progress would continue to go up for awhile, until finally getting stuck. If I then killed Starcraft II.exe and started it again, it would continue where it left off, crash again, but get further. After several iterations of this I had managed to download and install all the patches. Note: This problem may have been fixed by the kernel update, which I did not actually install until later. (Update: Nope, still happens.)
Sound
The internets claimed that if I built Wine with OpenAL support, sound would work. I did this, but the game was silent when I first ran. I think I fixed this by going into winecfg, under the "Audio" tab, and setting "Hardware Acceleration" to "Emulation". On the advice of another site, I also went under the "Libraries" tab and added an override for "mmdevapi", which I set to "disabled". I'm not sure if the latter fix was actually needed.
That's it!
I think that's everything I did. The game runs pretty smoothly at 2560x1600 with all graphics settings on "High" (although it defaulted everything to "Ultra", which was too much for my GeForce GTS 250).
So far I've played one round against the computer. Everything went fine until I accidentally activated KDE's "Desktop Grid" view which causes it to zoom out and display all windows from all desktops such that they don't overlap (like OSX's "Expose"). Amazingly, Starcraft kept on rendering through this, in its little window, only finally crashing when I tried to restore it. I will avoid activating that in the future. :)
