Speeding up dpkg

I always had problems with the speed and memory consumption of dpkg. When I’m doing apt-get upgrade, my laptop becomes unusable for several minutes. And I do it every day, since I’m using debian testing. I made a little research, and found out that a huge part of the time is spent in the parsedb function with parsing the text files containing package information. That’s when you see "(Reading database ..."

Therefore I decided to replace the textfile backend with sqlite3. The first step is almost finished: the package information has been imported into an sqlite database. I just wrote a 100 line C++ prog, and it works.

The preliminary results are self explanatory:

Using dpkg as it is now (file backend, 17782 packages):

hikaru:/home/robi/prog/dpkg/trunk/utils# time dpkg -l "gnome-terminal"

ii gnome-terminal 2.14.2-1 The GNOME 2 terminal emulator application
real 0m6.817s
user 0m1.360s
sys 0m0.092s

Using an sql query with sqlite (same database, 17782 packages):

hikaru:/home/robi/prog/dpkg/trunk/utils# time echo "SELECT * FROM packages WHERE Package='gnome-terminal';" | sqlite3 available.db
gnome-terminal|optional|gnome|404|Guilherme de S. Pastore |i386||2.14.2-1|b18bd66347ef25707e7b92957142de0b
real 0m0.851s
user 0m0.004s
sys 0m0.004s

The next step is to fix some bugs (eg. having ‘ in the descriptions is a problem at the moment), and then replace the code in parsedb. But first I have to get some sleep.