I always had problems with the speed and memory consumption of dpkg. When I’m doing apt-get upgrade, my laptop becomes unusable for several minutes. And I do it every day, since I’m using debian testing. I made a little research, and found out that a huge part of the time is spent in the parsedb function with parsing the text files containing package information. That’s when you see "(Reading database ..."
Therefore I decided to replace the textfile backend with sqlite3. The first step is almost finished: the package information has been imported into an sqlite database. I just wrote a 100 line C++ prog, and it works.
The preliminary results are self explanatory:
Using dpkg as it is now (file backend, 17782 packages):
hikaru:/home/robi/prog/dpkg/trunk/utils# time dpkg -l "gnome-terminal"
[...]
ii gnome-terminal 2.14.2-1 The GNOME 2 terminal emulator application
[...]
real 0m6.817s
user 0m1.360s
sys 0m0.092s
Using an sql query with sqlite (same database, 17782 packages):
hikaru:/home/robi/prog/dpkg/trunk/utils# time echo "SELECT * FROM packages WHERE Package='gnome-terminal';" | sqlite3 available.db
gnome-terminal|optional|gnome|404|Guilherme de S. Pastore
[...]
real 0m0.851s
user 0m0.004s
sys 0m0.004s
The next step is to fix some bugs (eg. having ‘ in the descriptions is a problem at the moment), and then replace the code in parsedb. But first I have to get some sleep.