Fossil Forum

serhiy.storchaka 5 days, 18 hours ago

Post: `fossil ls --age` takes ~30 s on Fossil's own repository since an SQLite update (foci scanned in the inner loop)

On a clone of Fossil's own repository, fossil ls --age -v -r trunk takes 0.04 s with 2.26 and 31.6 s with 2.28 and current trunk:

$ time fossil ls --age -v -R fossil.fossil -r trunk > /dev/null
real    0m31.6s

Bisected to 3041904dae ("Update the built-in SQLite to the latest trunk check-in for testing"); 59879abe16 before it is fast. The code of compute_fileage() (browse.c) did not change, but its query is planned differently: before, foci was scanned once as the outer loop; now it is the innermost one, so files_of_checkin parses the check-in's manifest again for every row (--sqltrace shows the manifest read 63,000+ times and 437 million VM steps for that statement).

EXPLAIN QUERY PLAN before:

|--SCAN foci VIRTUAL TABLE INDEX 1:
|--SEARCH filename USING COVERING INDEX sqlite_autoindex_filename_1 (name=?)
|--SEARCH blob USING COVERING INDEX sqlite_autoindex_blob_1 (uuid=?)
|--SEARCH mlink USING INDEX mlink_i3 (fid=?)
...

and after:

|--SEARCH event USING INTEGER PRIMARY KEY (rowid=?)
...
|--SEARCH mlink USING INDEX mlink_i1 (mid=?)
|--SEARCH blob USING INTEGER PRIMARY KEY (rowid=?)
|--SCAN foci VIRTUAL TABLE INDEX 1:
...

Writing the FROM clause of that query as foci CROSS JOIN filename CROSS JOIN blob CROSS JOIN mlink CROSS JOIN event restores the old plan: the same 2928 rows in 0.04 s instead of 31 s. Perhaps foci's xBestIndex should also report a higher cost for a scan.

stephan 1 day, 23 hours ago

Perhaps foci's xBestIndex should also report a higher cost for a scan.

Your repro takes 1m7s on my system :/. Changing that cost from 1 (trunk at the time of your report) to 10 has no appreciable effect but 100 restores order to the world (0.096s on my system) so is now checked in.

Thank you for the report!

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button