Bioinformatics News and Reviews

Monday, July 12, 2010

Thursday, July 1, 2010

some more links from iEvoBio

Continuing with my twitter voyeurism of iEvoBio, here are some web pages (softwares and servers) mentioned today at the meeting:

There is also a Nature Preceedings collection about the event that is certainly worthwhile to take a look.

(I might update this entry later if other links appear. Stay tuned!)

Tuesday, June 29, 2010

some links from iEvoBio - the poor guy's bookmarking

OK, let me just note down these interesting links discussed at iEvoBio - I'm not there but the tubes are - while I prepare for SMBE next week.

via @rdmpage and @phylogenomics (this blogpost will be updated a few times...)

Intellectual Proporty around the tubes

Helpless Mainstreamers Grappling with Intellectual Property (Stephan Kinsella):

A recent CNET video on “Intellectual property rights vs. journalism” shows a Stanford University’s Innovation Journalism conference on June 7, with a panel discussion by various mainstreamers discussing the quesion “Is intellectual property protection a threat to journalism?” The lack of libertarian principle and sound economics has these commentators floundering as they discuss various cases where IP infringes free speech and freedom of the press. Lacking any principled approach they retreat to legal positivism, talking about how the Constitution protects both freedom of the press and speech as well as IP rights, so some “balance” must be made. Without Austrian economics and libertarian principle, even well-intentioned people, who sense that something is wrong, are helpless before the state’s propaganda and onslaught of legal positivism.

You can tell that intellectual property rights are not real property rights just from this very interesting item on BoingBoing, a report on round one of the Google vs. Viacom case. With real property rights, people do not usually go around actively violating their own rights in order to collect from innocent defendants. I’m not even sure I understand how that would work in a case of real property rights. But in this Viacom case:
Filings in the case reveal that Viacom paid dozens of marketing companies to clandestinely upload its videos to YouTube (sometimes “roughing them up” to make them look like pirate-chic leaks). Viacom uploaded so much of its content to YouTube that it actually lost track of which videos were “really” pirated, and which ones it had put there, and sent legal threats to Google over videos it had placed itself.
Why would Viacom have done that if the company really believed that these postings were hurting its business? Perhaps they believed they would help the business: promoting its product and creating an opportunity for legal blackmail.

Was Napster Right?: Is this what they mean by analog hole? (David K. Levine):

Having trouble with DRM on your ebooks? Try this site. The problem with DRM is it encourages piracy. It can always be removed - but it can be a hassle. So: if you are going to distribute it widely it is worth the effort - and if you take the trouble to do it yourself you are so pissed off that you feel a strong temptation to share it. Irritating your customers hasn't proven a winning business model in the past.

Ideas vs. Execution Shows Why Competition Is A Good Thing (Mike Masnick):

Similarly, just because there are so many companies in a market, it doesn't mean any of them are really executing well (...) In fact, this succinctly reiterates a whole bunch of the points that we've discussed repeatedly over time. First, there's a big difference between ideas and execution. Just because others are in the market (and even well established), it doesn't mean you can't do a better job. It also highlights the difference between invention and innovation, where invention is just coming up with something new, but innovation is really bringing it to market successfully. Facebook and Google are both great examples of innovative companies who didn't really "invent" their initial markets.

Sunday, June 20, 2010

64-bit Linux Flash Player discontinued

It seems that at least temporarily the 64bit version of Flash Player will not be available for Linux. I found out when upgrading my system, where at some point I got the following error message:

[myPC:/home/leo] update-flashplugin-nonfree --install
ERROR: 64 bit player temporarily not supported
More information might be available at:
http://wiki.debian.org/FlashPlayer

The solution is to install the 32bit version of it, as explained here at this forum (another option is to install gnash, but I haven't tried). The details are described on this post, specifically.

Sunday, June 13, 2010

Recovering an encrypted hard disk and creating encrypted partitions

I recently wrote about how to make a secure connection to a remote machine. Now I'll make a few comments on how to create and use an encrypted partition, including DVDs. I apologize for skipping comments and technical details, these are just personal notes in case I must repeat the procedure in the future. I started to think about this when my not-so-old laptop broke, and I had to recover my file system, which was automagically set up by the Debian installation using the LVM+LUKS encryption. That is, what I was using as partitions were in fact "logical volumes" whose physical volume was an encrypted partition of the hard disk.

Creating an encrypted partition

First, a simpler version of the problem, which is to create an encrypted partition from a removable disk using cryptsetup with LUKS extensions. There is nothing special about being a removable disk, the procedure is exactly the same for a local disk. If we assume that the disk /dev/sdc has a partition /dev/sdc1, we can encrypt it by:

[thisPC:~] cryptsetup luksFormat /dev/sdc1
WARNING!
========
This will overwrite data on /dev/sdc1 irrevocably.

Are you sure? (Type uppercase yes): YES
Enter LUKS passphrase:
Verify passphrase:

You can change the cipher algorithm with e.g. the parameters "-c aes -s 256 -h sha256". I haven't mentioned but the partition should be initialized with garbage data using the command dd. If the system complains, try loading some modules:

[thisPC:~] apt-get install loop-aes-utils cryptsetup
[thisPC:~] modprobe loop
[thisPC:~] modprobe aes
[thisPC:~] modprobe cryptoloop
[thisPC:~] modprobe dm-crypt

(those with "loop" in the name are actually for creating a loopback partition, see below). This partition is still unusable, we must map it to a accessible device by decrypting it:

[thisPC:~] cryptsetup luksOpen /dev/sdc1 backup
Enter passphrase for /dev/sdc1:
Key slot 0 unlocked.
[thisPC:~] dmsetup ls
backup (254, 5)
thisPC-swap_1 (254, 2)
thisPC-usr     (254, 3)
sda2_crypt     (254, 0)
thisPC-home    (254, 4)
thisPC-root    (254, 1)

The first command "opens" a device called "backup", accessible through /dev/mapper/backup. The name "backup" is arbitrary, and could be any name distinct from the existing mapped devices. The second command shows us the list of these devices - we can see that besides the newly-created "backup" I have other "logical" partitions set up by LVM. The device "backup" can now be formatted as an ext3 partition:

[thisPC:~] mkfs.ext3 /dev/mapper/backup -L CryptBackupDisk
mke2fs 1.41.12 (17-May-2010)
Filesystem label=CryptBackupDisk
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
19537920 inodes, 78141903 blocks
3907095 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
2385 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632,
        2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

The partition is now ready to be used:

[thisPC:~] mount /dev/mapper/backup /mnt
...
[thisPC:~] umount /mnt
[thisPC:~] cryptsetup luksClose /dev/mapper/backup

Since the action "luksClose" will remove the device "backup", you must call the cryptsetup command "luksOpen" (and type the passphrase) again if you want to mount the partition once more. You must repeat the procedure every time you reconnect the removable disk or reboot the machine. One advantage of the LUKS extensions (as compared with the traditional "cryptsetup -c aes -y create backup /dev/sdc1") is that most environments can recognize that /dev/hdc1 is a LUKS encrypted partition and prompt the user for the passphrase when automatically mounting the removable devices.

Super-user privileges

So far I assumed root privileges, and for completeness I'll write below the configurations that should allow me to mount the partition as a regular user:

[thisPC:~] md /media/CryptBackupDisk
[thisPC:~] sudo chmod g+s /media/CryptBackupDisk
[thisPC:~] chown leo:leo /media/CryptBackupDisk
[thisPC:~] cat /etc/crypttab
sda2_crypt /dev/sda2 none luks
backup     /dev/sdc1 none luks
[thisPC:~] cat /etc/fstab
# /etc/fstab: static file system information
#
proc                    /proc   proc defaults      0    0
/dev/mapper/thisPC-root /       ext3 errors=remount-ro 0    1
# ... (other entries)
/dev/mapper/backup /media/CryptBackupDisk/ ext3 user,noauto 0 0

By creating a directory with the same name as the partition label ("CryptBackupDisk"), I should control the mounting process, but it is simply not working - it creates another directory called /media/CryptBackupDisk_ (notice the underscore), and with root permissions only. For now I removed the directory (since it will be dynamically created) and created a directory inside the encrypted partition (after being properly mounted) to which I have access as a regular user. There is another option using cryptomount but I haven't tried it with LUKS. It seems an easy alternative to the whole process described above, since you can create the encrypted partition from scratch using it. For setting up an already created encrypted partition it was not so easy (but again, I haven't tried with a LUKS partition).

Files and DVDs

The procedure above works not only with partitions, but with files as well. The trick is to transform the file (like an ISO image) into a filesystem, using the loopback device (more information here). So a 1Gb partition can be created through the file /home/leo/fake.img:

[thisPC:/home/leo] dd if=/dev/zero of=/home/leo/fake.img \
bs=1M count=1024

A safer but slower alternative to /dev/zero is /dev/random. This file can thus be mounted as a partition, like /dev/sdc1 of the previous example:

[thisPC:/home/leo] losetup /dev/loop0 /home/leo/fake.img
...
[thisPC:/home/leo] losetup -d /dev/loop0

That is, you can encrypt the device with "cryptsetup luksFormat /dev/loop0", etc. Don't forget that "losetup" should be the first and last recipes ("-d" for deleting the device), before "luksOpen" and after "luksClose", respectively. As a last digression, here is how to create an encrypted data DVD (with my precious "articles" directory as an example):

[thisPC:/home/leo] sudo apt-get install aespipe mkisofs \
      loop-aes-utils; sudo modprobe aes; sudo modprobe cryptoloop
[thisPC:/home/leo] mkisofs -U -R -J -r /home/leo/articles | \
      aespipe -e aes256 > backup.iso #encrypted backup.iso image
[thisPC:/home/leo] sudo mount -t iso9660 backup.iso /mnt/iso \
      -o loop=/dev/loop0,encryption=aes256 #confirm that image is OK

After burning the backup.iso image to a DVD, you can mount and access it with

[thisPC:/home/leo] sudo mount -t iso9660 /dev/cdrom /mnt/iso \
-o loop=/dev/loop0,encryption=aes256

Recover an encrypted root partition

Now, my original problem. Assuming that I have a hard disk where even the root directory was part of an LVM which was encrypted inside partition /dev/sdc2, we decrypt it as before (obviously, you should know the passphrase!):

[thisPC:/home/leo] cryptsetup luksOpen /dev/sdc2 BrokenPCPartition
[thisPC:/home/leo] dmsetup ls
thisPC-swap_1     (254, 2)
thisPC-usr        (254, 3)
sda2_crypt        (254, 0)
thisPC-home       (254, 4)
BrokenPCPartition (254, 5)
thisPC-root       (254, 1)

But now BrokenPCPartition is not a regular device, it is the part of an LVM. We can scan all possible volume groups (LVM lingo...) with the command:

[thisPC:/home/leo] vgscan
Reading all physical volumes. This may take a while...
Found volume group "brokenPC" using metadata type lvm2
Found volume group "thisPC" using metadata type lvm2

The group "thisPC" is from my working machine, and the newly-recognized "brokenPC" is the one found through /dev/mapper/BrokenPCPartition (of course the real name was not "brokenPC" :). Now we make this volume group available, and scan all possible logical volumes. IOW, we will activate the logical volumes from this group and list them:

[thisPC:/home/leo] vgchange -a y brokenPC
   3 logical volume(s) in volume group "brokenPC" now active
[thisPC:/home/leo] lvscan
ACTIVE            '/dev/brokenPC/swap_1' [2.50 GiB] inherit
ACTIVE            '/dev/brokenPC/root' [59.60 GiB] inherit
ACTIVE            '/dev/brokenPC/home' [235.74 GiB] inherit
ACTIVE            '/dev/thisPC/root' [6.52 GiB] inherit
ACTIVE            '/dev/thisPC/swap_1' [7.92 GiB] inherit
ACTIVE            '/dev/thisPC/usr' [111.76 GiB] inherit
ACTIVE            '/dev/thisPC/home' [469.73 GiB] inherit

Now we have, besides the volumes from thisPC, the "recovered" volumes from brokenPC that can be regularly mounted and played with:

[thisPC:/home/leo] mount /dev/brokenPC/home /mnt/
[thisPC:/home/leo] ls /mnt/
leo/ lost+found/
[thisPC:/home/leo] umount /mnt/
[thisPC:/home/leo] vgchange -a n brokenPC
0 logical volume(s) in volume group "brokenPC" now active
[thisPC:/home/leo] cryptsetup luksClose /dev/mapper/BrokenPCPartition

Notice that the volumes names (like /dev/brokenPC/home) are just links to the device mapper (like /dev/mapper/brokenPC-home).

Thursday, June 10, 2010

This week's readings

Lift the veil of secrecy over peer review - New Scientist:

But the authors of the protest letter suggest a compromise, which should make abuses easier to spot while preserving the main strength of peer review: keep anonymity, so scientists aren't discouraged from being critical, but publish the reviews and editorial correspondence. Indeed, The EMBO Journal already does this.

For justice to be seen to be done more widely, other journals should follow this lead. Like democracy, peer review may be flawed but it's the least-worst system we've got. It should be nurtured, through exposure to a little more sunlight.

To be a science ignoramus is simply not on - Times Online:

To ordinary people our top universities often seem somewhat elite and mysterious where arcane subjects are explored. Universities should foster a culture of open access and strengthen educational activities in the community and in schools.

In the past, the mark of a civilised person was an appreciation of Shakespeare, Thucydides, Rembrandt and Beethoven. Today the pursuit of science is so demanding that scientists are more likely to neglect their cultural inheritance. Perhaps, by broadening our own interests, we scientists may also help non-scientists see science more as part of our culture.

Are We Training Too Many Scientists? - The Scientist:

Career disappointment for postdocs is not just about finding a job, it's finding a job that is rewarding: one that pays reasonably well and offers a career path. The recent prospects for PhDs - rising numbers of postdocs, few tenure track positions, and poor funding - do not live up to that expectation. (...) The unintended consequence of the shortage of faculty positions was longer postdoctoral fellowships as young scientists received low wages, endured little job security or respect, and delayed starting families while waiting for a job.

Shrinking budgets + skyrocketing subscription fees = UC boycott of NPG - Adventures in Ethics and Science:

Scientific publication, after all, isn't just about keeping score. It's also about communicating findings, ideas, techniques, and conclusions. And NPG surely has no monopoly on the technologies by which that communication can -- and will -- take place.

I also made a comment on the NatureNews report about the last Brazilian scientific meeting - which had very nice talks as well (link in pt_BR).

PS: HT to @BioMedCentral for some of the links.

Tuesday, June 8, 2010

More secure Secure Shell

Hakone-Zaiku, a japanese wooden "safe" (this one is closed)

When we connect to a remote machine using Secure Shell (SSH), we routinely must type the password that we use on this remote machine. But SSH allows for public key authentication, where the remote machine has a public key, and it grants access to whoever has the corresponding private key (broadly speaking; if you want a proper explanation please refer to e.g. this book's section or the "Authentication" section of the ssh(1) man page). A possible analogy is a lock-and-key pair, where the lock may be accessible to everybody as long as you keep the key with yourself. It is also similar to PGP cryptography.

But with SSH it is also possible to encrypt the private key with a passphrase, such that the file with the private key is not enough for authentication. In the lock-and-key analogy, this would be equivalent to these modern cars where the owner's fingerprint is necessary to unlock the doors, besides the key. So far I have always used empty passphrases (that is, the private key was unencrypted), but now that we're migrating to a more secure server I'm better trying a distinct passphrase for each remote machine I connect regularly.

To create the public/private key pair is relatively simple, this is how I did it:

23:30[myPC:~] ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/leo/.ssh/id_rsa): /home/leo/.ssh/id_rsa-theirPC
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/leo/.ssh/id_rsa-theirPC.
Your public key has been saved in /home/leo/.ssh/id_rsa-theirPC.pub.
The key fingerprint is:
a2:c3:1f:1c:fc:ae:df:2f:af:77:aa:1d:81:00:6a:ed leo@myPC
The key's randomart image is:
+--[ RSA 2048]----+
|=+oo             |
|+=*+o            |
|+=o++.           |
| .=.E.           |
| o o. o S        |
| o o .         |
+-----------------+

Here I have chosen the standard RSA algorithm, but gave a custom name to my private (id_rsa-theirPC) and public (id_rsa-theirPC.pub) files. This is because I want to use this pair for connecting only to theirPC machine (any name would do, BTW). You cannot see, but I chose a fancy passphrase for that too. The next step is to send the public file id_rsa-theirPC.pub to the remote machine and there, to include it into the list of public keys:

23:35[theirPC:~] cat id_rsa-theirPC.pub >> /home/leo/.ssh/authorized_keys

To test if this configuration is working, we can do the following:

23:37[myPC:~] ssh -i ~/.ssh/id_rsa-theirPC theirPC.bigcorp.com

(notice how we must explicitly tell SSH about the location of the private key, otherwise it would fall back to your password at the remote host). If everything went fine, we can automatize the process by creating a configuration file for the SSH client with our customizations:

23:39[myPC:~] cat ~/.ssh/config
Host theirPC
HostName theirPC.bigcorp.com
IdentityFile ~/.ssh/id_rsa-theirPC

This way we can easily include more and more remote machines, and in the example above I could connect with the command "ssh theirPC". But the problem is that with more machines we connect to, the more times we need to type long passphrases into. To solve this you can use ssh-agent to keep the authentication keys (the unencrypted private keys) in memory for all commands spawned from it. That is, ssh-agent acts over a command (which can be a terminal, or startx), and all commands called from it will inherit the authorization keys. But sometimes this is not enough, you might want authentication keys shared between sessions (even after logout). For me the solution was keychain (not the Mac OSX application!), which is a wrapper around ssh-agent (and its low-level friend ssh-add).

Both keychain and ssh-add can read from the terminal prompt, but normally they will use a program that provides the passphrase dialog window. This program is generally called ssh-askpass, but each window manager offers its own implementation as well. In my case I'm using ksshaskpass, that can be integrated with the KDE Wallet system (which is installed under the name kwalletmanager). The kwallet is an application that manages passwords and passphrases, storing them in encrypted "wallets". That is, you only need to type one master password per wallet.

I will skip the details of kwallet configuration - it suffices to say that you should create at least one wallet and it will prompt you when creating or retrieving passwords - and comment on the keychain program. The installation instructions always tell us to call keychain from .bash_profile (like here), but this didn't work for me since I must make sure it is called only after KDE is up. When following the instructions, I had kwallet asking for my wallet's password (or ssh-askpass asking for the passphrase for id_rsa-theirPC, when I deactivated kwallet) before startkde being called, and thus the keyboard was unresponsive. Maybe because I have an exotic keyboard configuration. Anyway, the solution to that was to create an executable to be called after startkde, and the place to put these custom "daemons" is in ~/.kde/Autostart:

23:39[myPC:~] cat ~/.kde/Autostart/keychain.sh
#!/bin/bash

# initialize SSH key management ("keychain" wrapper for ssh-agent)
# depends on a ssh_askpass (which for KDE is ksshaskpass)
keychain ~/.ssh/id_rsa-theirPC ~/.ssh/id_rsa-anotherPC
source ~/.keychain/${HOSTNAME}-sh

Keychain can also handle GnuPG signatures and several SSH private keys, like in the id_rsa-anotherPC example. Now, when I start a KDE session I am asked about my wallet password, which then offers my SSH passphrases whenever requested. To be germane, it is actually more nuanced: the program requesting the passphrase will trigger the "upper-level" program to or offer the passphrase or ask another program for it (like in the hierarchy ssh -> ssh-agent -> keychain -> ssh-askpass -> kwallet).

Some general comments:

your private keys should not be accessible or readable by anybody but you ("chmod 600 /home/leo/ssh/id_rsa-theirPC");
the passphrase can have spaces, punctuation marks etc. So you can mix passwords, but avoid literature quotes! The larger the better;
avoid leaving private keys on your remote hosts, that you are unlikely to use to access other machines;
do not share private keys among machines, these should be present only in the machines you physically access (in my case these are my desktop and my laptop, which have different configurations);
you can change the passphrase later with "ssh-keygen -p id_rsa" (I haven't tried this, but I believe in the man page ;)
within kwallet, you can see the passwords and passphrases of open wallets in plain text. I usually panic, but these wallets are themselves encrypted...
I haven't tried, but it should be also possible to a) work with ssh-agent and ssh-add without keychain (launching them before the KDM or GDM display managers, or even in Autostart); b) do not use kwallet/ksshaskpass, by using the original x11-ssh-askpass or on the terminal; c) some other killer-app that I don't know about yet.

A Hakone-Zaiku opened. I never cracked the encryption key of these :(

Saturday, May 29, 2010

Geno-luddites on the rise

There is a very nice, though short discussion on the cut-and-paste of a bacterial genome onto its cousin at Edge's Reality Club. This comment by Nassim N. Taleb called my attention:

If I understand this well, to the creationists, this should be an insult to God; but, further, to the evolutionist, this is certainly an insult to evolution. And to the risk manager/probabilist, like myself & my peers, this is an insult to human Prudence, the beginning of the mother-of-all exposure to Black Swans.

And if I understood correctly his position, we shouldn't do anything that exposes us so much to Black Swans. I agree that some aspects of this research (and all the news coverage that followed) downplays the role of evolution. But as reminded by PZ Myers in the same discussion, there is a much larger number and variety of naturally-occurring threats to us. (HT @aerocleber).

PS: The title comes from this post, brought to my attention by @phylogenomics.

Monday, May 17, 2010

Anything goes on the internet

The "Special report: Living in denial" issue of the New Scientist (12 May 2010) has some interesting articles. Two excerpts below.

When a sceptic isn't a sceptic (by Michael Shermer):

A climate sceptic, for example, examines specific claims one by one, carefully considers the evidence for each, and is willing to follow the facts wherever they lead. A climate denier has a position staked out in advance, and sorts through the data employing "confirmation bias" - the tendency to look for and find confirmatory evidence for pre-existing beliefs and ignore or dismiss the rest.
(...)
It has, for example, become fashionable in some circles for anyone who dares to challenge the climate science "consensus" to be tarred as a denier and heaved into a vat of feathers. Do you believe in global warming? Answer with anything but an unequivocal yes and you risk being written off as a climate denier, in the same bag as Holocaust and evolution naysayers. (...) When I say "I believe in evolution" or "I believe in the big bang", this is something different from when I say, "I believe in a flat tax" or "I believe in liberal democracy".
(...)
What sometimes happens is that people confuse these two types of questions - scientific and ideological. Sometimes the confusion is deliberate. Denial is one outcome. Thus, one practical way to distinguish between a sceptic and a denier is the extent to which they are willing to update their positions in response to new information.

Questioning science isn't blasphemy (Michael Fitzpatrick):

As philosopher Edward Skidelsky of the University of Exeter, UK, has argued, crying denialism is a form of ad hominem argument: "the aim is not so much to refute your opponent as to discredit his motives". The expanding deployment of the concept, he argues, threatens to reverse one of the great achievements of the Enlightenment - "the liberation of historical and scientific inquiry from dogma". Don't get me wrong: the popular appeal of pseudoscience is undoubtedly a problem. But name-calling is neither a legitimate nor an effective response.
(...)
Such attempts to combat pseudoscience by branding it a secular form of blasphemy are illiberal and intolerant. They are also ineffective, tending not only to reinforce cynicism about science but also to promote a distrust for scientific and medical authority that provides a rallying point for pseudoscience.

I also liked Jim Giles's article about how easy it is to spread a lie, and how hard to fix it nowadays.