Proceeding with Arch Linux: UMTS, clbuild, cups, booting through VBox

Sat, 17 Apr 2010 23:40:03 +0000

Even Though I had some problems in the past, now UMTS works through NetworkManager. I still need Windows sometimes, when I have to put new money on my umts-account, since I dont know how to do this without the GUI my provider provides.

Well, then due to the lack of Common Lisp Packages, I installed clbuild. Unfortunately, also clbuild is not very well maintained anymore, some packages dont even build anymore without modifications. I had in mind to create an AUR-Package for clbuild, but that is a problem since there are already guidelines for CL-Packages and I dont see how to include clbuild into them. Under Debian, there was the Common-Lisp-Controller, but as far as I understood, something like the CLC is not the philosophy of Arch Linux.

Then I installed CUPS and it sort of works with both of my printers. Not perfectly, but well.

Then finally one thing I really wanted for a long time now is being able to boot my Arch Linux through VirtualBox under Windows (well, actually I wanted to boot it through Mac OS X, but I almost dont use Mac OS X anymore). This is a bit tricky, since I didnt get the MBR to boot it right. So what I was basically doing is booting a Grub-CD and manually chainloading into the rootpartition of arch linux. I now unfortunately have the problem that sometimes my X-Server crashes, but I actually dont think that this is caused by VBox at all, I had a major system-upgrade short before. I dont even use the VBox Guest Additions, since I just login through my NX Server.

Ok, besides all the good things, there is one drawback: zfs-fuse doesnt work anymore. I tried to recompile it but it somehow always crashes. A pity, but I am sure this will be solved soon by the developers of zfs-fuse (or maybe Linus Torvalds just allows publishing a Linux-Kernel with ZFS).

I am sure there comes a time when I will reinstall my whole System. But for now I am just glad that everything works. Maybe I will wait until btrfs or zfs with deduplication gets stable – or some other deduplicating filesystem.

Advertisements

Datendeduplikation mit ZFS-Fuse

Fri, 02 Apr 2010 22:42:04 +0000

Ich schrub ja bereits, dass ZFS-Fuse einige Ladungsspitzen hatte bei der Benutzung. Somit ist es für mich momentan nicht zu gebrauchen als Root-Dateisystem – was schade ist. Dennoch habe ich mir heute mal die Zeit genommen, die git-version von zfs-fuse auszuprobieren. Dafür gibt es im AUR das tolle Paket zfs-fuse-git. Im Gegensatz zur Release-Version unterstützt diese nämlich bereits deduplikation.

Also USB-Stick gezückt, und zfs erzeugt.

# zpool create -O dedup=verify dedup /dev/sdc1
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                 85M   21K   85M   1% /dedup
# dd if=/dev/urandom of=/tmp/random bs=1024 count=$((40*1024))
# cp /tmp/random /dedup/
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                 85M   41M   45M  48% /dedup
# cp /tmp/random /dedup/random_b
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                124M   79M   45M  64% /dedup
# cp /tmp/random /dedup/random_c
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                165M  121M   45M  73% /dedup
# cp /tmp/random /dedup/random_d
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                204M  159M   45M  79% /dedup
# cp /tmp/random /dedup/random_e
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                244M  199M   45M  82% /dedup
# cp /tmp/random /dedup/random_f
# cp /tmp/random /dedup/random_g
# df -h
Dateisystem           Size  Used Avail Use% Eingehängt auf
/dedup                324M  279M   45M  87% /dedup

Bemerkenswert.

Update: Ich verwende jetzt mal temporär auf einer meiner Backup-Platten ZFS-Fuse-Git. Mal sehen, ob es sich bewährt.


Proceeding with Arch Linux: Xdmcp, FreeNX, TwinView, no more zfs, etc.

Thu, 01 Apr 2010 20:33:53 +0000

First, the unfortunate news: I dont use zfs-fuse anymore. I never had data loss. But I ran into some problems with zfs-fuse running wild and almost freezing the system. I created a second pool /home/usr, because my /usr-partition was too small. And copied everything in it, and symlinked it to /usr. Shortly after that I had strange load peaks and partial freezes, with zfs-fuse running at up to 90% cpu-speed. I found out that the reason for this was man-db. So I copied /usr/share/man back to the primary harddisk and symlinked. This made the problem occur less often, but it randomly occured. I noticed that whenever it occurs I ran something like pacman or git or darcs – which are all creating lockfiles – which also man-db does. So my theory is that zfs-fuse has some problems with lockfiles. But I cannot verify that. Since the stable version of zfs-fuse has no inline-deduplication anyway, well, I maybe create a little backup-server with OpenSolaris and deduplicating ZFS in the kernel. Or I’ll wait until btrfs becomes stable – or zfs-fuse gets into the linux-kernel. Or maybe I really just write my own deduplication format which runs fully in usermode and just creates a bunch of archive files which point to blocks. I wonder why there are softlinks and hardlinks, but not yet „copy-on-write“-links in the common filesystems, at least for files (for directories I can imagine it could get a lot more complicated, but directory-entries shouldnt consume that much space). Of course, making it block-wise like btrfs and zfs is better, but of course harder.

Well, for incremental backups, there should be a lot of „userspace“-Solutions which run on any normal unix-filesystem – of course, you cannot have your snapshots on your working-partition without loss of a lot of space, but for most backup-purposes this should be sufficient. For my purposes it of course isnt quite, I have a lot of old „snapshots“ which can hardly be synchronized (except maybe with fdupes …). Maybe I will really write my own little backup-archive-format which just pushes anything into a huge file which save their blocks and anything. Shouldnt be too hard. But – of course  – I will first search for other solutions.

So well, enough of that then. Since I always forget my Screen-Adapter-Cable, but dont want to be always bound to the 13“-Screen on my mbp, I thought of setting up my thinclient. I activated xdmcp, adding the entry

[xdmcp]
Enable=true

to /etc/gdm/custom.conf. Xdmcp may not be the most flexible solution, but when combining it with Xephyr and Xpra, I am sure it can be made a lot more flexible. And on the other hand, inside the LAN, it is a lot better than VNC and RDP, it even gets comparably good results when using GLX. Unfortunately, Xdmcp is not very secure – but using it inside my home network (and the maths dept’s local network) I dont care about that because I simply trust these networks. Setting up an Xvnc-Server also is not hard, but mostly I am doing this manually only, by running Xvnc in a remote shell and forwarding the ports (and of course sometimes I use gnome’s desktop-sharing-feature). I still couldnt manage to set up an Xrdp-Server (except by tunneling X11 through VirtualBox, of course).

My favourite solution for this is FreeNX now, since it also performs well through DSL-Connections (and is usable through EDGE). Setting it up is not hard, using the tutorial in the Arch Linux Wiki. It uses SSH and should therefore be secure. It works well with other computers, unfortunately, apparently I cant get my thinclient booting properly -.- but this time I didnt forget my Screen Adapter Cable.

Which brings me to … TwinView … Which I use to manage the two screens now. It can be configured via the UI-Tool nvidia-settings – actually I was surprised not having to dig through the xorg.conf myself.

Furthermore, I removed a lot of stuff I didnt need anymore (which should be a good idea when /usr tended to be full …) – beginning with a lot of crap kde installed. The only KDE-Application I really use is Konversation now – simply because I am too lazy to switch to Xchat.

The next thing I am about to configure is trying to get the „Mobile Partner“ from my Huawei-UMTS-Stick working on Linux through Wine. The configuration through umtsmon is a mess, but at least it sometimes works – which doesnt apply to the setup through networkmanager. In theory, it should be possible to access the stick through Wine. The Software itself installs well, it just cant find the device. But there are hacks for Wine which can make Wine-Applications access USB-Devices. So well, its worth a try – and I am interested in what this software does when it found its stick – through which way will it provide network access? And will other processes inside Wine get access, too? If so, I can just cross-compile a little socks-proxy and be happy. If not … well, it gets harder, and maybe I’ll lose my interest to that topic.

At least this is not the most stupid way I was yet trying to get that stick work: I already forwarded it into a virtualized Windows … which is what I will maybe do again if anything else fails. Which reminds me of an older xkcd-comic:


Proceeding with Arch Linux – btrfs, s2disk, umtsmon

Mon, 15 Mar 2010 00:49:11 +0000

Just a little Info of how it goes on with Arch Linux on my MacBook.

I switchen my Home-Partition to btrfs. It is experimental – that is, I know that I shouldnt save anything of value only on this – as you should never save anything of value only in one place. But the snapshotting-facilities are extremely useful, especially when experimenting with software. In the long term I want to switch my root-partition completely to it. At the moment, I will keep it that way – there will certainly come a time when I completely damage my system and have to reinstall it, and then I can redesign it.

Then – as you might notice from my former rant about KDE – I switched to gnome. Its nice, it works, it didnt crash so far, and I could customize it in a way I like. Meanwhile, Networking, also Wifi, works. Thats nice. Also the german keyboard layout and my customized settings for the touchpad work under Gnome. What doesnt work is compiz – but I can live without it for now.

Then for my GSM-Stick I used umtsmon. It uses ppp, but the default version of ppp doesnt support removing the default route. Therefore, one needs the package ppp-rdr in the AUR directory – which had a bug in the PKGBUILD-File (the file doesnt state correctly that it does provide ppp), so you have to fix that yourself. I am sure they already fixed it.

I had to write an udev-rule for to allow a user to access /dev/ttyUSB0. Though, still, I have to use umtsmon as root, since somehow ppp doesnt allow the -noauth-Option without being root. Therefore, I use sux to switch. I will have to investigate on this. Maybe, since now NetworkManager seems to work, I can connect through it, instead of umtsmon.

A problem with umtsmon was resolv.conf – it doesnt set it properly. I just added 8.8.8.8 (the google-nameserver) to it, then it worked. Sort of. Well, mobile internet is a pain – but thats not umtsmon’s fault.

For suspending to disk I use s2disk – I couldnt manage to make the Suspend-Facilities of Gnome and KDE work, but s2disk works. So I wrote an own script and added an application launcher to my panel. The content of the script is:

#! /bin/bash

kquitapp konversation;
purple-remote setstatus?status=offline;
gnome-screensaver-command -l && sleep 5 && sudo s2disk

This quits Konversation if online, sets the Pidgin-Status to offline if it runs, Locks the screen, then waits 5 seconds (this is a hack, because the gnome-screensaver fades out the screen – but it also fades after hibernation, which is annoying (and I couldnt find a gconf-entry to turn off this fading), and then suspends (of course, I had to add s2disk to the /etc/sudoers). There may be more elegant ways to do this, but well, it works that way, and it fits my needs.

And well, I did something epic: I installed debootstrap and bootstrapped a debian-system on an USB-Stick. Bootstrapping Debian from Arch Linux. Isnt it nice? Well, the AUR-Package didnt have a current version which is downloadable, so I had to modify the md5-checksum and the package version to make it work.

I wonder if there is something like debootstrap for arch linux.


Proceeding with Arch – GRUB works, KDE works, Wifi piddles around

Tue, 09 Mar 2010 10:54:20 +0000

So, just giving a little update on my proceedings with Arch Linux. Well, I finally managed to get a proper triple boot system with OS X, Windows and Arch Linux – thanks to Kompottkin btw. The problem is that Windows needs to be the last partition on the disk to boot, and OS X and the EFI Bootloader has the two partitions in the beginning of the disk – i.e., only one Partition remains, until Windows gets confused – but thats just for the MBR Table. In the GPT Table, there can be partitions with a higher number than 4 between the third and the fourth partition. Linux recognizes GPT Disklabels and can handle these – the only problem is that GRUB (in a reasonable simple setup) so far only understands MBR Disklabels, which is why you have to install your /boot-Partition into the third MBR-Partition. But for anything else, it doesnt matter for Linux and Windows. So the basic installation procedure that worked for me was (this is just a compendium, not a guide – dont know if it always works):

  • Perform a fresh install of OS X (this shouldnt be necessary, but well, it prevents from any problems).
  • Use BootCamp-Assistant to partition disks (I gave Windows 32 Gig) but dont install.
  • Use Disk Utility and add an additional partition (This sometimes fails – I actually dont get why. Well, as far as I saw, you have to select the BootCamp-Partition and then press „+“ and add the partition.
  • Install Windows completely
  • Boot OS X, install rEFIt. Run „enable-always.sh“ to make the system always boot into rEFIt.
  • Boot the Arch Linux LiveCD, and use parted to create partitions as shown below.
  • Reboot into rEFIt. Let it Sync your Disklabels (in the partition tool). If you have a QUERTZ-Keyboard, remember to press „z“ instead of „y“. Otherwise, rEFIt wont do anything, since the default is not to change the disk.
  • Reboot again, boot Windows – now Windows should be confused and tells to boot from the Install Disk and run the Disk Repair Utility – well, I have done so. After reboot, Windows booted again.
  • Now boot again from the Arch Linux CD and install Arch. Grub wont install, because the Partition Type of /dev/sda3 is set to something different than 83. Use fdisk (not parted!) to change this. Now anything should be OK and after several reboots you should be able to boot all the 3 Systems.

At least this procedure worked for me. Here are the outputs of fdisk and parted (showing Dos and GPT Disklabel):

# fdisk -cu /dev/sda

WARNING: GPT (GUID Partition Table) detected on '/dev/sda'! The util fdisk doesn't support GPT. Use GNU Parted.

Command (m for help): p

Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders, total 312581808 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x37ff4f62

Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1      409639      204819+  ee  GPT
/dev/sda2          409640    63264575    31427468   af  HFS / HFS+
/dev/sda3   *    63526720    65011712      742496+  83  Linux
/dev/sda4       245778432   312580095    33400832    7  HPFS/NTFS
# parted
GNU Parted 2.2
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: ATA Hitachi HTS54501 (scsi)
Disk /dev/sda: 160GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number  Start   End     Size    File system     Name                  Flags
1      20.5kB  210MB   210MB   fat32           EFI System Partition  boot
2      210MB   32.4GB  32.2GB  hfs+            Macintosh HD
3      32.5GB  33.3GB  760MB   ext2            linux-boot
5      33.3GB  41.9GB  8590MB  linux-swap(v1)  linux-swap
6      41.9GB  50.5GB  8590MB  ext4            linux-root
7      50.5GB  126GB   75.4GB  ext3            linux-home
4      126GB   160GB   34.2GB  ntfs            BOOTCAMP

rEFIt claims that /dev/sda3 has a wrong filesystem label in the Dos Table. I dont know why, but I didnt let it resync the partitions, since this seemed not to produce any problems so far – and at least the partition boundaries are equal. So who cares.

For KDE4, there are the kdemod-Packages which are recommended by the Arch Linux Wiki. However, these packages didnt work for me (random crashes of plasma), so I preferred to install the supported packages. These also crashed after adding the NetworkManager-Applet to plasma, but after reconfiguring anything, it didnt crash anymore so far. A problem which remains is that it seems like my xorg.conf is just ignored – at least the XkbLayout-Option, and the options for my synaptics touchpad. I always have to use setxkbmap and synclient manually to set it to proper values as a workaround. But I am sure to be able to find out the reason soon.

Actually, I dont want to use KDE anyway – not that its not a good system, but its … huge. The main reason for using it is that there is basically no equivalent Frontend for NetworkManager, except under Gnome, and I dont like Gnome. Under KDE3, there was KNetworkManager, but they have replaced it by a „Plasma Applet“ now. There is still KNetworkManager in the AUR-Archives, but it fails to compile. And there is cnetworkmanager, but I dont really get how to set up eduroam connections with it. Well, there are other network management tools than networkmanager, and I will try these.

The main problem is not the frontend but the backend: I need the official broadcom-driver for my card, which fails hard! It seems like the Wifi is randomly working perfectly, but randomly just refuses to do anything. This is strange, because on Ubuntu, Wifi worked well, as well as on a similar arch linux installation on a MacBook Pro of some other person. The firmware should be the latest. So I dont really have any clue on how to investigate on this. I think the next thing I will try is ndiswrapper, but I couldnt find any 64 Bit Windows Driver for downloading yet. Well, we’ll see.


My first expieriences with Arch Linux

Thu, 04 Mar 2010 02:43:55 +0000

As I already wrote, I want to switch to Arch Linux. Well, I am new to this distribution. Since back to the days when I used SuSE, I only used debian-derivatives on my PCs. Its not that I dont like debian anymore, its a great distribution, but I am interested in learning more about the underbrushes of Linux.

Seems like Arch Linux has a great community, but so far, I cannot contribute anything, since I am completely new to this distribution. But at least, well, I can write a few things here. About some problems I ran into, about tutorials that I read, etc., in the hope that this will be usefull to other people.

The first time I ran Arch inside a VirtualBox. And about the first thing that happened was, that it crashed when trying to start X11. I forgot to install and run hal. Thats nice. Setting DAEMONS in /etc/rc.conf properly solved that problem quickly.

Well, installing on my MacBook Pro was not that easy, though. I chose the most complicated setup on a MacBook Pro – a Triple Boot System with OS X, Windows and Arch Linux. Well, the Tutorial for this was not very helpfull at that point (later it was). Well, since I already installed Ubuntu in the past, I knew a bit of what to do.

Firstly, I performed a clean installation of Mac OS X. Then I ran BootCamp, but only let it repartition the hard disk, not reboot and install Windows. I chose to give Windows 32 Gig. Then, with the Disk Utility, I added a third partition, between the bootcamp-partition and the partition of OS X. I chose to give OS X 32 gig, since I want to mainly run Arch Linux in the future.

Then I installed Windows. I wanted to have Windows 7 64, but unfortunately, this cannot be installed since it lacks of some CD-Driver. So I first installed Windows Vista 64 and then upgraded to Win 7 (thankfully, I can obtain Windows from the MSDNAA). After doing the whole setup and stuff, I rebooted and booted OS X. Then I installed rEFIt.

At this point, I rebooted into the Arch Linux CD-Rom. I followed the usual /arch/setup procedure. I didnt repartition, I set up /dev/sda3 as ext3 to be mounted on /. I ignored the warning that there is no swap and no separate /boot-Partition. The problem is: I cannot have so much partitions – 4 is the Limit. A helpfull source is the Documentation for Ubuntu.

Well, everything worked so far, except that the installation of the bootloader claims that it could not read stage1. Trying to fix it manually didnt work. I downloaded the newest Grub-Sources and compiled them in the chroot on /mnt (where /dev/sda3 is mounted through the installation process) myself, after performing a pacman -Sy and installing some dependencies (I dont know all of them anymore, but the configure-script will tell them). Calling this grub-install gave me some errors, that installing Grub on /dev/sda3 only works with blocklists, which are not reliable. Well, even though it was not recommended, I tried to install it on /dev/sda – but it also didnt work. Somehow it cannot install to the MBR.

I cant tell why, but now at least I have some sort of Grub to boot. Grub just drops a shell now, but I can boot the System manually using

root (hd0,3)
linux /boot/vmlinuz26 root=/dev/sda3
initrd /boot/kernel26.img
boot

in the Grub-shell. Unfortunately, I can only boot Windows 7 from this GRUB now – using the manual chainloader. I dont know why it doesnt recognize my menu.lst, but well, I dont want to mess around with my bootloader-configuration right now, since I dont want to break it – at least it works so far.

A comparably easy task was then installing the X-Server. I installed hal and dbus, and put them into the DAEMONS-Variable. Then I installed xorg and the nvidia-drivers (pacman -S xorg nvidia nvidia-utils), according to the Wiki-Documentation of Arch Linux. I also installed xdm and icewm, since this is the environment I want to use. Afterwards, I ran nvidia-xconfig. I also edited .xinitrc to start icewm-session on X startup.

Then I rebooted and started xdm. It worked. Then I adapted the Settings of the nvidia-driver according to the Wiki-Documentation, and again restarted.

For the Synaptics Touchpad, there is an excellent documentation. The customized part of my hal-settings-file so far are:

<merge key="input.x11_options.VertEdgeScroll" type="string">false</merge>
<merge key="input.x11_options.HorizEdgeScroll" type="string">false</merge>
<merge key="input.x11_options.TapButton1" type="string">0</merge>
<merge key="input.x11_options.TapButton2" type="string">0</merge>
<merge key="input.x11_options.TapButton3" type="string">0</merge>
<merge key="input.x11_options.MaxTapTime" type="string">0</merge>
<merge key="input.x11_options.TapClickFinger1" type="string">1</merge>
<merge key="input.x11_options.TapClickFinger2" type="string">2</merge>
<merge key="input.x11_options.TapClickFinger3" type="string">3</merge>
<merge key="input.x11_options.VertTwoFingerScroll" type="string">true</merge>
<merge key="input.x11_options.HorizTwoFingerScroll" type="string">true</merge>
<merge key="input.x11_options.CircularScrolling" type="string">true</merge>

For Sound, I use the Jack Sound Server – basically, because the performance I got with pure alsa was bad, and because I like jackd. I installed qjackctl, and adjusted the settings. The content of the created .jackdrc with the Settings that work for me are:

/usr/bin/jackd -R -P30 -p512 -m -dalsa -dhw:0 -r44100 -p256 -n3 -S

However, when using jackd, there are some additional problems. Namely, under Debian, I can remember never getting the Flash Plugin to work. Well, this appears not to be a problem using the software flashsupport-jack, which is an AUR-Package, but works perfectly for me. I installed it as stated in the Wiki.

Of course, there are still a few problems remaining. Didnt get jackd-support for all of the packages. The GRUB doesnt work right at the moment. But well, it is a System I can already work with.