梁祝小提琴协奏曲

这两天反复在听梁祝的小提琴协奏曲，喜欢的紧。其实初衷是这个曲子近半个小时，很长时间不用换，呵呵，二泉映月也很喜欢，只是才7分钟。以前在udayton的小剧院听过一次梁祝的现场演奏，当时感觉非常好，但那种震撼随时间的推移而淡化了。梁祝的故事从小听到大，虽然是乐盲，还是能猜出各个段落代表的含义，其中发现十八相送之后那一大段疾风骤雨般的章节和楼台会及抗婚的段落很有越剧的风格，特别是其中的有快板那一段，完全就是一小段越剧。也难怪，梁祝故事本身就发生在上虞和杭州，而且作曲家之一何占豪原来是在越剧团里拉二胡的，虽然根据一些文章记载（http://baike.baidu.com/view/14011.htm#4），当初他自己不觉得这段经历有帮助。

我妈最爱看越剧，我外婆最爱看戏，于是我也跟着看过一些。梁祝的故事发生在杭州的万松书院，在杭州的时候没去过。以前以为北宋大儒张载是万松书院的，现在发现不是，那是陕西的横渠书院。横渠四句“为天地立心，为生民立命，为往圣继绝学，为万世开太平”一直谨记在心。

Shittest day. Lots of shit happened.

昨天在atlanta的craigslist上看到一个htc G1的deal, 就跟卖家约定今天晚上7点后在一个地址见面. 家里没米了, 刚好可以顺带去一下Atlanta的亚洲超市, 中午跟室友一说, 刚好下午他也有空, 就准备下午4,5点钟出发去Atlanta. 从Clemson到Atlanta要两个小时, 我们计划坐4:18pm的公车从学校回家, 然后开车先去买手机, 然后去Farmer's market买米买菜, 接着找家餐馆吃一顿,然后开回家. 这是所谓的perfect plan. Then, loads of shit happened, tick tick.

4:10pm, 实验室里耽搁了一会, 4:13pm奔向公车站, 细雨中狂奔400米, 发现这班公车早到, 加快步伐, 差几步眼看公交车关门, 缓缓驶离, 室友在车上看着我错过. 干等半个小时, 坐下一班.

5:10pm, 开车出发, 发现天基本黑透了, 还下雨,小心开吧. 跟卖家发短信, 可能7点半到, 卖家回复: cool. 2小时车程暂且不表, 开到Atlanta市中心, 发现给的地址是豪华的Marriot酒店, 找了旁边的另一家酒店的停车库, 拿了一张parking ticket, 就是记录进入时间的那种卡片, 进酒店大厅等. 此时近7:50pm, 给卖家打很多个电话, 无人接, 发短信没反应, 干等半小时, 喝了一杯咖啡, 放弃, 去超市买菜吧.

走到停车库门口, 发现parking ticket不见了, shit, 回想, 很可能是买咖啡时掏钱包掉出来了, 今天忘了把卡片收进钱包里, 问了一下停车库门口的mam, ticket掉了要交20刀, 算算找回来的话只要交4-5刀就够了, 折回starbucks, 店员告知, 他们捡到了ticket, 等了几分钟没人领, 就扔了, 扔到哪个垃圾桶忘了, 找了一圈, 没找到, 就对我摊摊手. So, 回到停车库, 交了20刀.

开到Farmer's Market是9点过几分, 中途接到一个电话, 结果不是卖家, 而是一个朋友. 买米买菜, 甚欢, 就是没啥海鲜, 几个柜台的鱼都卖光了, 买完接近10pm, 准备找餐馆吃饭, 室友推荐Sichuan House, 在回家的路上, GPS上没找到, 打电话给同学找到地址, 开过去, 关门了, 旁边有个steak house好像还开着. 先去加油, 结果加油管是漏的, 搞得满手汽油, 加了8加仑, 漏了就近1/3加仑. 去旁边的steak house, 好几张桌子上还有客人, 暗喜. 但接待台没人, 过两分钟一个侍应生过来说, it's after 10pm, and we are closing, sorry, if you came in 15 minutes earlier, blabla. 至少进去洗了个手, 满手油味换成了洗涤剂的味道.

出了steak house环顾四周, 基本都是关的, 心灰意冷准备直接开回家做点好的. 于是在漆黑的高速上捱饿狂奔两个小时, 中途受不了喝了杯咖啡吃了半根咸的要死的牛肉棒, 回到家是00:30am. 蒸了一袋速冻蟹黄糖包, 室友做了一个盐津肉, 总算吃了点好的.

卖家始终没有再联系, 被彻底的放了鸽子, 加上一连串的事情, 真是 shittest day. 幸好买了很多菜, 没有彻底白跑. 明天还要赶两个report, 生活还要继续.

记录一下, 以后不顺的时候可以对比一下哪次更背.

Update: 第二天早上9点钟, 突然想起来9点有appointment要去医院拿药, 只是人在家里, 而且周二错过一次这个appointment了. shit still happens.

Where am I?

我醒过来，发现躺在一个宿舍里，靠窗的一个双层铁架子床的上铺，格局跟高三的寝室一样。宿舍里就我一个人，其他床铺有的铺着床单。我下了床，看到靠门那有张单人沙发，上面堆着一个包，压着一个开着的13寸银色macbook pro。把玩了一下，没什么东西，又原样放好。窗外有一些阳光，不知何处的广播嘈杂的放着一些声音，我在想，这是在那？ 偏生脑子空荡荡的。

进来一位大妈，可能是打扫卫生的，自称姓耿。我就问，耿大妈，啊不，耿姐，这是在那，中科院？清华？浙大？ 大妈笑了笑，你这小伙，连在哪都不知道。我恍然大悟，在浙大，我在读博呢，已经念了个硕士，很有兴趣再接着读，对，在浙大。打开宿舍门，走进常见的黑黢黢的宿舍过道，过道尽头的盥洗室灯火通明，好几个光着上身的男生端着脸盆挂着毛巾进进出出，有十三，有郭波，有达达，我大声跟他们打着招呼，都回来了呀。从过道的另一头走来同一装束的一个高中同学，我用乡音打了个招呼，你也回来了呀。

这时，我又醒了一次，发现自己睡在一张靠门的单人沙发上，在实验室里，窗外的阳光亮的晃眼，屋里两张写字台，几个显示器，我缓过神来，哦，我在Clemson，而且硕士也不是在浙大念的。对了，明天要考试，要继续看课件。刚才的温馨一瞬，原来只是个连环梦。

我想我是有点想家了。

xorg-server update in gentoo

Whenever xorg-server get a major version update, such as 1.5->1.6 recently, make sure follow the tips after update and re-emerge x11-drivers.

Portage will tell you:

You must rebuild all drivers if upgrading from xorg-server 1.5 or earlier, because the ABI changed. If you cannot start X because of module version mismatch errors, this is your problem.

You can generate a list of all installed packages in the x11-drivers category using this command:
emerge portage-utils; qlist -I -C x11-drivers/

Then after executing `qlist -I -C x11-drivers/', a list of x11 drivers package will show. RE-EMERGE THEM before startx, otherwise these drivers will not work. In my case, the keyboard and mouse stops responding and it takes me one hour to fix it.

Another weird thing is that when compiling xorg-server, I have to turn off the `hal' USE flag. Otherwise the mouse and keyboard will also stop responding. This is just my experience.

sth else: keep the USE flag of perl and libperl the same, otherwise when compiling some program, it will fail and say a long list of repeated complaints like unknown variable PERL_Gth..., whose exact term I can not recall. This happens to me while I was installing inkscape. emake failed

从平原到山地

500 miles, 800公里, 从Dayton OH到Clemson SC, 从小城市到小镇,从北边到南边,从EO到CE,做了好大的转变,继续前行. 本以为孤家寡人没什么行李,结果拾掇拾掇整理出一堆,车后座堆到一人高,垃圾还是攒的很多. 小潘曾给我看的一个老头的talkshow, "My shit is stuff, their stuff is shit", 还是有些应景. 路上9个小时,前半段一直在下雨,路上就是在山里面绕来绕去,路很窄,上上下下的. 南方天气又潮又热,我这个浙江人在ohio呆了这么久都已经忘了家乡的夏天了,颇有些不习惯.

房子是已经联系好的, 2 bedroom town house, 挺满意的, 室友人很好. 环境非常乡下, 好像就是在丛林中开辟出来的一条小道和一片房子. 公交车免费, 去学校转了转, 大太阳底下按照地图找房子,更何况没有东南西北的意识,很晕,以后熟悉了就好了吧. 中午吃的是asian express, 白饭加两个炒得发甜的肉,已经是惊喜了,这比光汉堡皮萨好多了.

实验室的人都还没到,老板还在外地开会,系里的小秘没找到,这周可以过几天自在日子,下周就要开始忙了.

wpa wireless setup in gentoo

Reference: http://www.thinkwiki.org/wiki/How_to_install_wpa_supplicant


1. # echo "net-wireless/wpa_supplicant madwifi" >> /etc/portage/package.use
2. # emerge -av wpa_supplicant
3. # wpa_passphrase "ssid" "wpa_passphrase" % get the psk
4. % create /etc/wpa_supplicant.conf
# cat /etc/wpa_supplicant.conf

ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=0
eapol_version=1
# ap_scan=1 was the one for me you may try 0 or 2 indstead of 1
ap_scan=1
fast_reauth=1

network={
ssid="ssid"
proto=WPA
key_mgmt=WPA-PSK
pairwise=TKIP
group=TKIP
#psk="passphrase"
psk=xxxxx # a bunch of numbers and letters from wpa_passphrase
}


To start the network:

1. # ifconfig ath0 up
2. # wpa_supplicant -B -c /etc/wpa_supplicant.conf -iath0 -Dmadwifi
%% -B is the daemon mode, -d is the live mode
3. dhcpcd ath0

%% if dhcpcd has a time out, use `iwlist ath0 scan' to check the channel number and then change (ap_scan) in the config file. restart wpa_supplicant (if it's in daemon mode, kill the process and then restart)

浙江大学校歌MTV

浙江大学校歌

　　大不自多，海纳江河。惟学无际，际于天地。
　　形上谓道兮，形下谓器。礼主别异兮，乐主和同。
　　知其不二兮，尔听斯聪。国有成均，在浙之滨。
　　昔言求是，实启尔求真。习坎示教，始见经纶。
　　无曰已是，无曰遂真。靡革匪因，靡故匪新。
　　何以新之？开物前民。嗟尔髦士，尚其有闻。
　　念哉典学，思睿观通。有文有质，有农有工。
　　兼总条贯，知至知终。成章乃达，若金之在熔。
　　尚亨于野，无吝于宗。树我邦国，天下来同。

TestDisk saves my ass !!!

Today I accidentally did a "rm -rf *" under a folder, SHOOT!!! This is a FAT32 partition. Then I quickly unmount it and google for solution.

Then TestDisk comes to me and saves my ass! http://www.cgsecurity.org/wiki/TestDisk I emerged testdisk from gentoo portage and uses 'photorec' to undelete files. The recovered files are named randomly with a correct suffix. That's very enough for me. Great tools!

Thank this blog entry:
http://blog.lxpages.com/2007/06/21/linux-file-recovery/

BTW: for NTFS partition, ntfsundelete is useful, it's included in ntfsprogs package.

Running concurrent ML on smlnj

Check this wiki page:
http://en.wikipedia.org/wiki/Concurrent_ML

Note: 'heap2exec' is currently supported on ppc-unix (Mac OS X), and x86-unix (no amd64 support). It requires heap2asm, which is not installed by default. It can be added by editing config/targets

快过年了

离多聚少人寂寂，冬去春来柳纷纷.



500 Miles

If you miss the train I'm on
You will know that I am gone
You can hear the whistle blow a hundred miles
A hundred miles, a hundred miles, a hundred miles,
a hundred miles
You can hear the whistle blow a hundred miles

Lord, I'm one, Lord, I'm two, Lord, I'm three
Lord, I'm four, Lord, I'm five hundred miles from my home
Five hundred miles, five hundred miles, five hundred miles
five hundred miles
Lord, I'm five hundred miles from my home

Not a shirt on my back, not a penny to my name
Lord, I can't go back home this a way
This away, this away, this away, this away
Lord I can't go back home this away

If you miss the train I'm on
You will know that I am gone
You can hear the whistle blow a hundred miles
A hundred miles, a hundred miles, a hundred miles,
a hundred miles
You can hear the whistle blow a hundred miles

zz Recovering Deleted Files in Linux

Brian Buckeye and Kevin Liston

Most systems administrators have experienced a situation where a vital file has accidentally been deleted without a recent backup. In this article, we’ll explore how to recover files in Linux. To begin, however, there are several caveats:

1. The methods described are emergency measures. They do not replace a working backup process to protect your data. You should also consider version control methods to protect your data from accidents.

2. File recovery is usually a time-consuming process, and often is not completely successful. Once a file is deleted, the space it occupied on the hard drive is marked as “available” and can be overwritten. DO NOT install any file recovery software on the drive that houses the file you want to recover.

3. These data recovery techniques involve elements of luck and timing, in addition to technique. If you’ve suffered an accidental deletion in the first place, luck isn’t necessarily on your side.

4. Even if you do recover the file, there is no guarantee that it will have the same information that was contained in the original. Inspect anything you retrieve and verify the information before you use it in production.

5. There are several factors acting against a successful recovery, including: time, file size, congestion of the disk partition, and the system activity:

• The more time that passes between the deletion of the file and the initiation of the recovery process, the less likely the process will succeed.

• The larger the size of the deleted file, the more likely damage has occurred.

• The more active the system, the more likely the blocks freed by the deletion will be overwritten by new data.

• If there is little free space on the disk partition, the smaller pool of available blocks increases the chance that the deleted data blocks will be re-used.

With those caveats in mind, we’ll examine some options.

Linux and ext2

The default file system used by Linux is the Second Extended File system, referred to as ext2. (Ext3 with its use of journaling has also recently become common, but we will not cover it in this article.) The role of the file system itself is to abstract the physical structure of the storage media. On a physical level, a drive is a series of 512-byte sectors, addressable from 0 to n-1. The file system is responsible for organizing these sectors into files and directories eventually used by applications via the operating system.

Blocks

The Linux file system, ext2, collects sectors into blocks. Ext2 supports block sizes of 1024, 2048, and 4096 bytes. Blocks are organized into block groups. Blocks are either data blocks or superblocks. Data blocks are general-purpose blocks used to store files and directories. Superblocks reside on the border of block groups and contain settings and status of the file system (e.g., formatting and cleanliness state). Block groups consist of a superblock, block allocation bitmap, inode allocation bitmap, inode table, and data blocks. Block groups are usually organized into 8*block-size blocks (e.g., 8192 blocks in a 1024-byte block-sized system). The block allocation bitmap keeps track of which blocks in the block group are in use (allocated vs. free). Our 1024-byte block size example has 1024 bytes responsible for tracking 8192 blocks. Thus, each block is mapped to one bit in the bitmap. (A “1” denotes allocated and a “0” denotes the block to be free.) The make-up of a block group includes a superblock, block allocation bitmap, inode bitmap, inode table, and data blocks.

The inode allocation bitmap work similarly, but typically uses less space than allocated, unless you have defined the system to have one inode per data block (which would be the case in a system optimized to handle a large amount of small files such as a news server). Inodes are special data-structures, each 128 bytes in length, which represent a file. By default, mke2fs (used to format an ext2 partition) reserves an inode for every 4096 bytes of file system space. The first ten inodes in a file system are special purpose:

1 — Bad blocks inode

2 — Root inode

3 — acl index inode (not supported)

4 — acl data inode (not supported)

5 — Boot loader inode

6 — Undelete directory inode (not implemented)

7-10 — Reserved

The bad blocks inode lists all of the data blocks on the file system that have detected unrecoverable errors. The root inode points to the directory file of /. The acl-related and undelete directory inodes are currently not implemented.

Pointers

Inodes contain information about a file, such as modification, access, creation (and deletion) time, ownership, permissions, size, and pointers to the actual data blocks of the file. There are 15 pointers to data blocks; the first 12 are references to direct blocks (actual file-data). The 13th pointer references the indirect block, which is a data block containing a list of 4-byte pointers to direct blocks (i.e., another 256 direct blocks in a 1024-byte block-sized system, 1024 direct blocks in a 4096-byte block-sized inode). The 14th pointer references the doubly indirect block, which is a block containing pointers to 256 (in the case of a 1024-byte block-sized file system) indirect blocks. In other words, the 14th pointer serves as the root of a tree that references 65536 data blocks in a 1024-byte block-sized file-system. The 15th pointer points to the triply indirect block, or a block full of references to doubly indirect blocks. In other words, this forms an asymmetrical tree-structure, where the inode references 15 children, the first 12 are terminal, the 13th has 1 level, the 14th has 2 levels, and the 15th has 3 levels. This causes the 1.6-GB file-size limit on 1024-byte block-sized systems.

Everything is a File

In Linux, directories are simply special files. The second inode in the file system points to /. This directory links to other subdirectories (which are other directory files). Directories are simply lists of four-tuples, consisting of an inode number, entry length, name length, and filename. The entry length denotes the length of the directory entry itself. This structure allows the use of long filenames without wasting disk space, but there is some waste from directories due to block size. This is why you see a size such as 1024 for . and .. in the output of ls -la.

Also implemented with Linux is the /proc pseudo filesystem. Staying consistent with the UNIX everything-is-a-file metaphor, the /proc directory allows access to kernel data structures. The process structures are handy for data recovery. As root, change directory to /proc/, where is the process ID you’re interested in. You will see a number of directories, links, and files (note that they take up no space). Two of these directories are useful for recovering files: /proc//exe, and /proc//fd.

The exe link is an actual pointer to the file that is being executed. The fs link is a directory of file descriptors currently opened or in-use by the process. Every process will have at least three, which are listed first and denote STDIN, STDOUT, and STDERR, respectively. Other possible entries are network sockets (e.g., 20 -> \socket:[450], or port 450) and files (e.g., 4 ->/home/kliston/.list.swp).

In Linux, each inode keeps track of a file’s link count, which is the number of times that a directory lists the inode. When a file is deleted, its entry is removed from the directory file and the inode’s link count is decremented. If this link is reduced to 0, then the inode is marked as “free” in the inode bitmap, and all of the blocks referenced by that inode are marked as “free” in the block bitmap. The deletion time field is set in the inode. The OS also keeps track of the processes linked to an inode. This can be used to your advantage if you are notified of the accidental deletion in time.

Getting Your Files Back

This all may be interesting, but you still need to know how to get your files back. The first step is determining how important the information is, and how vital it is to get it back intact. In Linux, there are a few things you can try before mounting the affected partition in read-only mode.

If you need to recover an executable that happens to be currently running (such as in a forensics case where an intruder has a backdoor running, but has deleted it to cover his tracks), you can recover simply with:

cp /proc/415/exe /tmp/backdoor

If you have a process running that references a recently deleted file, you can try similar tricks with the /proc//fd directory. In the example above, we had:

/proc/415/fd/4 -> /home/kliston/.list.swp

This happened to be the swap file from a vi session. Performing strings 4 returned the contents of /home/kliston/list with some garbage as the header. Using the /proc//fd technique will require some understanding of the applications to be fully successful. To list the files currently open on a system, use lsof, or for a quick and dirty method to generate a list of candidates for this technique:

ls -l [0-9]*/fd|grep 

If you’re not lucky enough to have a case that can be solved by using the /proc recovery techniques, you need to cease write activity to the affected partition. Our examples will be recovering data from /home or /dev/hdc6.

Remount the partition in read-only mode:

mount -o ro,remount -n /home

This will allow you to access the system and stop processes from overwriting your to-be-recovered data blocks. The -n flag instructs mount to not write to /etc/mtab, enabling you to recover data from partitions that contain /etc, such as /.

There are a few factors that can be used to gauge your chances for success. Before kernel 2.2.x, the indirect inode pointers (pointers 13 and above) were also zeroed out when a file was deleted. If you are working with a kernel older than 2.2.0 (use uname -r to find out), you’re limited to the file size that you can recover using a direct inode reference technique. This recoverable limit is 12*block size. You can pull the system’s block size from the superblock by doing the following (where /dev/hdc6 is an example file system):

echo stats|debugfs /dev/hdc6

These examples were performed on a system running kernel version 2.2.19-6. The file system had a block size of 4096 and 10 block groups. Files were recovered from the /dev/hdc6 partition using /home as a mount point. The server saw low-to-moderate activity as a general-purpose server in a home/lab environment.

Using the debugfs utility, you can generate a list of deleted inodes, or inodes that have a non-zero time in their “Deleted Time” field. Generate a list of deleted inodes:

echo lsdel | /sbin/debugfs /dev/hdc6 > /tmp/lsdel.out

This generates an output similar to:

debugfs:   7344 deleted inodes found.

Inode   Owner Mode      Size   Blocks   Time deleted
62984   511   100600   12288   3/   3   Thu Dec 27  10:38:44 2001
62980   511   100644     693   1/   1   Thu Dec 27  10:39:09 2001
110212  511   100644 2123710   520/ 520 Thu Dec 27  10:54:35 2001

Needless to say, a lot of entries were omitted, and we’ve only shown the last three that belong to our user id since that’s what we’re interested in. To examine these files a bit more, use the stat command in debugfs to pull additional information about the file referenced by the inode:

debugfs /dev/hdc6
> stat <110212>

This will return the link count (probably 0), the creation, access, modify, and deletion times, and a list of all of the blocks that make up the file. This information will determine whether this inode is your candidate. To actually recover the data, use debugfs to dump the data to which the inode is pointing to a new file:

debugfs /dev/hdc6
dump <110212> /tmp/recovered

To recover all three of these files, edit /tmp/lsdel.out down to the desired files as /tmp/lsdel.edited and do something like this:

     awk '{print $1}' /tmp/lsdel.edited > /tmp/inodes
    for i in $(cat /tmp/inodes); do echo <$i> \
       -p /tmp/recovered.$u\i" | debugfs
/dev/hdc6; done

This creates a series of files in /tmp, but there is still the task of discovering their names and where to place them.

An alternative method (which is more risky but can work when you don’t have another partition to restore to, and this is rarely the case) involves directly editing the inode itself. Zero-out the deletion date and create a link to the inode (both raising the link count to one, and providing an access point in a directory):

debugfs -w /dev/hdc6
> mi <110212>

This action will walk us through the settings of the inode. It will show the current setting and offer to change it. Press “Return” to accept the current (or default) setting. When you arrive at the “Deleted Time” field, enter “0” and then continue accepting the rest of the settings. Then, change directory to where you want to link the file. Note that the top directory in debugfs will be the mount point, /home in our example:

> cd kliston/
> link <110212> recovered_file

It is important to unmount the altered partition and run fsck upon it. It will discover that there are blocks that are marked as free in the block allocation table, yet linked to an active inode. Let fsck make the required fixes. Now your file will remain safe, otherwise the data blocks will still be marked as available and eventually other files will reuse them and corrupt data.

It is simply a matter of chance should these techniques work. In test recoveries, we were able to help successfully recover log files on December 27th that had been deleted on October 11th. This was from a low-to-mid-use home/lab server, so these results are probably atypical.

Known-Text Recovery

What if the file wasn’t rmed? What if your unfortunate user typed:

cat /dev/null > important_file

In this case, the inode isn’t deleted but all of the data block pointers are zeroed and the data blocks are freed up in the block allocation bitmap. The odds of recovery have just decreased by an order of magnitude, but there are some other options.

The “known-text recovery method” is more of an art than a science and is less likely to succeed, but it has the advantage of working on file systems other than ext2 (such as Solaris’s ufs). This technique involves searching for a known pattern through an image of the affected file system. The pattern should be unique to the file that needs restoration. Crafting the search pattern is the artistic part of the process. A poorly written pattern can return too many hits, or no hits at all.

The example here involves recovering a DNS database file from the catastrophic cat /dev/null > important_domain.com.db. Because we’re looking for a bind data file, we could search on a pattern containing “IN SOA”, or for a known host of the missing domain.

The first active step involved in this technique is the creation of the recovery copy of the partition. By this time, the partition should have already been unmounted, or mounted read-only (see above techniques). Copy the partition to another file system (which must be large enough to hold the affected partition) with a command such as:

dd if=/dev/hdc6 of=/opt/hdc6.image

Apply an fgrep filter to locate the pattern (a unique hostname, in this case) in the recovery image:

fgrep "elmenop" /opt/hdc6.image

Here, we’re looking for the domain record that defined elmenop.important_domain.com. In the test case, this returned most of the domain record surrounded by nulls. It probably recovered unused space from a temporary file that referenced the file, rather than the file itself. If you need to search or use regular expressions, you can use egrep in lieu of fgrep, which will output all instances of your search pattern. Then, based on either knowledge, or trial and error, use fgrep’s -A and -B flags to pull a slice out of your recovery copy into (hopefully) an editable file that can be cleaned up for use.

The -A flag denotes how many lines after the match to print, and -B instructs how many lines before the match that grep will print. In the example, elmenop is a hostname that appears in the domain file. Using some guesswork (based on inspecting other domain data files that were not deleted), there is a window size of seven lines before, and ten lines after. There is added buffer room to our estimates to increase the odds of grabbing all of the usable data in one pass. In this special case, we lacked physical access to the server, and we didn’t have enough space to create a recovery copy, so the action was performed on a live pattern (not recommended unless you’re intentionally pushing your luck as we were):

dd if=/dev/hdc6 | fgrep -B 7 -A 10 --text "elmenop" > \
 /tmp/pattern_match.1

This approach created an editable output, capable of rebuilding the original file. This was successful after cat /dev/null > important_domain.com.db was used to “destroy” the file. The recovery attempt was made less than 24 hours later only to find that the data blocks had been overwritten. Once again, we find that time is not your friend when it comes to data recovery.

Recovery Tools

Are there programs out there to make this any easier? Absolutely. But, as sys admins, we know that you need at least three ways to fix a problem — none of them will work, but they’ll give you an idea for a fourth way that probably will. Taking time to work through the abstraction of the operating system and understand what is happening at a lower level may help you see the problem differently. Tools tend to hide what is going on and may blind you to another answer. Realistically, working through the problem yourself is not always the most expeditious path. These tools may make administration a little easier for you:

The Coroner’s Toolkit

(http://www.fish.com/tct) — A collection of tools originally created for computer forensics work. It includes the data recovery tools unrm and lazarus, both of which can be used to recover accidentally deleted data.

The Recover Tool

(http://recover.sourceforge.net) — Automates the direct inode recovery technique described above. It’s good to use if you have a large number of files to recover.

Conclusion

In the end, retrieving a file on Linux comes down to luck, timing, luck, technique, and luck. Most file recovery tools are fairly inexpensive and easily available and should be a standard part of any systems administrator’s toolbox. So, the next time a user accidentally deletes that vital file, you can say, “Relax, it’s probably already too late. But maybe, just maybe, there’s something I can do.”

References

Ferlito, John and Widdowson, Liam. “Tales from the Abyss: UNIX File Recovery,” Sys Admin magazine, November 2001:

Mandia, Kevin and Prossise, Chris. Incident Response: Investigating Computer Crime. Osborne/McGraw Hill, New York 2001.

Wall, Kurt, Watson, Mark, and Whitis, Mark. Linux Programming Unleashed. Sams, 1999.

Ext2 file Undeletion: http://www.billjonas.com/papers/undeletion.html

Crane, Aaron. Linux Ext2fs Undeletion mini-HOWTO: http://www.praeclarus.demon.co.uk/tech/e2-undel/howto.txt

Card, Remy, Ts’o,Theodore, and Tweedie, Stephen. Design and Implementation of the Second Extended Filesystem: http://e2fsprogs.sourceforge.net/ext2intro.htm

Oxman, Gadi. The extended-2 filesystem overview: http://www.nondot.org/sabre/os/files/FileSystems/Ext2fs-overview-0.1.pdf

Brian Buckeye is the Director of IT for a medium sized Ohio business. He can be reached at: brian@blindpanic.com.

Kevin Liston is a consulting security engineer. He can be reached at: kliston@infornographic.com.