The arrow of time

The theory behind the trick is to use two compressors, one fast and one slow; The fast one should be fast enough to make use of available disk bandwidth and the slow one should be fast enough to read through whatever the first one feeds it. This two-stage model should be extensible to more stages / CPUs, but it gets tricky with regards to tuning the speed. Since the decompression stage is so much faster I doubt it will see much multi-CPU usage (of course it still needs two passes).

I found that it sort-of works, but the bottleneck soon becomes available disk bandwidth :)

last pid: 36628;  load averages:  1.48,  0.64,  0.37  up 4+03:31:30  14:34:54
184 processes: 3 running, 181 sleeping
CPU: 31.5% user,  0.0% nice,  2.2% system,  0.3% interrupt, 65.9% idle
Mem: 1130M Active, 1488M Inact, 935M Wired, 52M Cache, 392M Buf, 75M Free
Swap: 4094M Total, 2972K Used, 4091M Free

  PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME   WCPU COMMAND
36626 ivoras        1 113    0  8032K  1384K CPU1    1   0:38 81.98% gzip -2
36627 ivoras        1 111    0  8032K  1396K CPU2    2   0:33 69.19% gzip -9