Fossil Forum
Post: `fossil ls --age` takes ~30 s on Fossil's own repository since an SQLite update (foci scanned in the inner loop)
On a clone of Fossil's own repository, fossil ls --age -v -r trunk takes
0.04 s with 2.26 and 31.6 s with 2.28 and current trunk:
$ time fossil ls --age -v -R fossil.fossil -r trunk > /dev/null
real 0m31.6s
Bisected to 3041904dae
("Update the built-in SQLite to the latest trunk check-in for testing");
59879abe16 before it is fast. The code of compute_fileage() (browse.c) did
not change, but its query is planned differently: before, foci was scanned
once as the outer loop; now it is the innermost one, so files_of_checkin
parses the check-in's manifest again for every row (--sqltrace shows the
manifest read 63,000+ times and 437 million VM steps for that statement).
EXPLAIN QUERY PLAN before:
|--SCAN foci VIRTUAL TABLE INDEX 1:
|--SEARCH filename USING COVERING INDEX sqlite_autoindex_filename_1 (name=?)
|--SEARCH blob USING COVERING INDEX sqlite_autoindex_blob_1 (uuid=?)
|--SEARCH mlink USING INDEX mlink_i3 (fid=?)
...
and after:
|--SEARCH event USING INTEGER PRIMARY KEY (rowid=?)
...
|--SEARCH mlink USING INDEX mlink_i1 (mid=?)
|--SEARCH blob USING INTEGER PRIMARY KEY (rowid=?)
|--SCAN foci VIRTUAL TABLE INDEX 1:
...
Writing the FROM clause of that query as
foci CROSS JOIN filename CROSS JOIN blob CROSS JOIN mlink CROSS JOIN event
restores the old plan: the same 2928 rows in 0.04 s instead of 31 s.
Perhaps foci's xBestIndex should also report a higher cost for a scan.
Perhaps foci's xBestIndex should also report a higher cost for a scan.
Your repro takes 1m7s on my system :/. Changing that cost from 1 (trunk at the time of your report) to 10 has no appreciable effect but 100 restores order to the world (0.096s on my system) so is now checked in.
Thank you for the report!