Fossil SCM

update branch before making changes

venkat 2011-08-16 18:35 UTC venks-emacs merge
Commit 84e6f172af56c6fd863e541f096af1362d880357
Parent 690ba8cbdb20c57…
83 files changed +38 -16 -3 +3 +9 +1 +2 +35 +1 +31 +63 +30 +314 +307 +1 +9 +144 +2 +130 +21 +3 +74 -21 +51 -32 +1 -1 +13 -3 +92 -15 +1 +17 -3 +5 -1 +1 +180 -52 +35 +8 -7 +22 -16 +161 -39 +35 -7 +9 +4 -4 +2 -1 +61 -1 +30 -6 +14 -8 +10 -4 +16 -7 +1 -1 +38 -31 +4 -5 +10 -2 +1 -6 +5 +1 -1 +1 -1 +1 -1 +11 -2 +5173 -2758 +48 -15 +1 +2 -1 +2 -1 +280 -29 +76 -15 +1 -1 +57 +8 -20 +644 -9 +7 -1 +3 -3 +4 -4 +5 -5 +93 -8 -10 +63 -26 +1 +73 -16 +60 +4 +21 -15 +11 -6 +11 -1 +15 -7 +3 +6 +21
~ BUILD.txt - Makefile ~ Makefile.classic + Makefile.classic ~ Makefile.in ~ VERSION ~ auto.def ~ autosetup/LICENSE ~ autosetup/README.autosetup ~ autosetup/autosetup ~ autosetup/cc-lib.tcl ~ autosetup/cc-shared.tcl ~ autosetup/cc.tcl ~ autosetup/config.guess ~ autosetup/config.sub ~ autosetup/find-tclsh ~ autosetup/jimsh0.c ~ autosetup/local.tcl ~ autosetup/system.tcl ~ autosetup/test-tclsh ~ configure ~ src/add.c ~ src/branch.c ~ src/browse.c ~ src/cgi.c ~ src/checkin.c ~ src/checkout.c ~ src/clone.c ~ src/config.h ~ src/configure.c ~ src/db.c ~ src/descendants.c ~ src/diff.c ~ src/diffcmd.c ~ src/export.c ~ src/file.c ~ src/finfo.c ~ src/glob.c ~ src/http.c ~ src/http_ssl.c ~ src/import.c ~ src/info.c ~ src/main.c ~ src/main.mk ~ src/makeheaders.c ~ src/makemake.tcl ~ src/manifest.c ~ src/merge.c ~ src/mkversion.c + src/mkversion.c ~ src/path.c ~ src/printf.c ~ src/rebuild.c ~ src/setup.c ~ src/sqlite3.c ~ src/sqlite3.h ~ src/stash.c ~ src/stat.c ~ src/style.c ~ src/tar.c ~ src/timeline.c ~ src/tkt.c ~ src/update.c ~ src/wiki.c ~ src/winhttp.c ~ test/release-checklist.wiki ~ win/Makefile.PellesCGMake ~ win/Makefile.dmc ~ win/Makefile.mingw ~ win/Makefile.msc - win/version.c ~ www/build.wiki ~ www/changes.wiki ~ www/checkin.wiki ~ www/foss-cklist.wiki ~ www/index.wiki ~ www/makefile.wiki ~ www/mkindex.tcl ~ www/permutedindex.wiki ~ www/quickstart.wiki ~ www/server.wiki ~ www/settings.wiki ~ www/ssl.wiki
M BUILD.txt
+38 -16
--- BUILD.txt
+++ BUILD.txt
@@ -1,41 +1,60 @@
1
-All of the source code for fossil is contained in the src/ subdirectory.
2
-But there is a lot of generated code, so you will probably want to
3
-use the Makefile. To do a complete build on unix, just type:
1
+To do a complete build, just type:
2
+
3
+ ./configure; make
4
+
5
+The ./configure script builds Makefile from Makefile.in based on
6
+your system and any options you select (run "./configure --help"
7
+for a listing of the available options.)
8
+
9
+If you wish to use the original Makefile with no configuration, you can
10
+instead use:
411
5
- make
12
+ make -f Makefile.classic
613
714
 On a windows box, use one of the Makefiles in the win/ subdirectory,
8
-according to your compiler and environment. For example:
15
+according to your compiler and environment. If you have GCC and MSYS
16
+installed on your system, then consider:
17
+
18
+ make -f win/Makefile.mingw
19
+
20
+If you have VC++ installed on your system, then consider:
921
10
- make -f win/Makefile.w32
22
+ cd win; nmake /f Makefile.msc
1123
1224
 If you have trouble, or you want to do something fancy, just look at
13
-top level makefile. There are 6 configuration options that are all well
14
-commented. Instead of editing the Makefile, consider copying the Makefile
15
-to an alternative name such as "GNUMakefile", "BSDMakefile", or "makefile"
16
-and editing the copy.
25
+Makefile.classic. There are 6 configuration options that are all well
26
+commented. Instead of editing the Makefile.classic, consider copying
27
+Makefile.classic to an alternative name such as "GNUMakefile",
28
+"BSDMakefile", or "makefile" and editing the copy.
1729
1830
1931
 BUILDING OUTSIDE THE SOURCE TREE
2032
2133
 An out of source build is pretty easy:
2234
23
- 1. Make a new directory to do the builds in.
24
- 2. Copy "Makefile" from the source into the build directory and
25
- modify the SRCDIR macro along the lines of:
35
+ 1. Make and change to a new directory to do the builds in.
36
+ 2. Run the "configure" script from this directory.
37
+ 3. Type: "make"
38
+
39
+For example:
2640
27
- SRCDIR=../src
28
-
29
- 3. type: "make"
41
+ mkdir build
42
+ cd build
43
+ ../configure
44
+ make
3045
3146
 This will now keep all generates files seperate from the maintained
3247
 source code.
3348
3449
 --------------------------------------------------------------------------
3550
3651
 Here are some notes on what is happening behind the scenes:
52
+
53
+* The configure script (if used) examines the options given
54
+ and runs various tests with the C compiler to create Makefile
55
+ from the Makefile.in template as well as autoconfig.h
3756
3857
 * The Makefile just sets up a few macros and then invokes the
3958
 real makefile in src/main.mk. The src/main.mk makefile is
4059
 automatically generated by a TCL script found at src/makemake.tcl.
4160
 Do not edit src/main.mk directly. Update src/makemake.tcl and
@@ -51,5 +70,8 @@
5170
 A header comment in src/translate.c explains in detail what it does.
5271
5372
 * The src/mkindex.c program generates some C code that implements
5473
 static lookup tables. See the header comment in the source code
5574
 for details on what it does.
75
+
76
+Additional information on the build process is available from
77
+http://www.fossil-scm.org/fossil/doc/trunk/www/makefile.wiki
5678
5779
DELETED Makefile
5880
ADDED Makefile.classic
5981
ADDED Makefile.in
6082
ADDED VERSION
6183
ADDED auto.def
6284
ADDED autosetup/LICENSE
6385
ADDED autosetup/README.autosetup
6486
ADDED autosetup/autosetup
6587
ADDED autosetup/cc-lib.tcl
6688
ADDED autosetup/cc-shared.tcl
6789
ADDED autosetup/cc.tcl
6890
ADDED autosetup/config.guess
6991
ADDED autosetup/config.sub
7092
ADDED autosetup/find-tclsh
7193
ADDED autosetup/jimsh0.c
7294
ADDED autosetup/local.tcl
7395
ADDED autosetup/system.tcl
7496
ADDED autosetup/test-tclsh
7597
ADDED configure
--- BUILD.txt
+++ BUILD.txt
@@ -1,41 +1,60 @@
1 All of the source code for fossil is contained in the src/ subdirectory.
2 But there is a lot of generated code, so you will probably want to
3 use the Makefile. To do a complete build on unix, just type:
 
 
 
 
 
 
 
4
5 make
6
7 On a windows box, use one of the Makefiles in the win/ subdirectory,
8 according to your compiler and environment. For example:
 
 
 
 
 
9
10 make -f win/Makefile.w32
11
12 If you have trouble, or you want to do something fancy, just look at
13 top level makefile. There are 6 configuration options that are all well
14 commented. Instead of editing the Makefile, consider copying the Makefile
15 to an alternative name such as "GNUMakefile", "BSDMakefile", or "makefile"
16 and editing the copy.
17
18
19 BUILDING OUTSIDE THE SOURCE TREE
20
21 An out of source build is pretty easy:
22
23 1. Make a new directory to do the builds in.
24 2. Copy "Makefile" from the source into the build directory and
25 modify the SRCDIR macro along the lines of:
 
 
26
27 SRCDIR=../src
28
29 3. type: "make"
 
30
31 This will now keep all generates files seperate from the maintained
32 source code.
33
34 --------------------------------------------------------------------------
35
36 Here are some notes on what is happening behind the scenes:
 
 
 
 
37
38 * The Makefile just sets up a few macros and then invokes the
39 real makefile in src/main.mk. The src/main.mk makefile is
40 automatically generated by a TCL script found at src/makemake.tcl.
41 Do not edit src/main.mk directly. Update src/makemake.tcl and
@@ -51,5 +70,8 @@
51 A header comment in src/translate.c explains in detail what it does.
52
53 * The src/mkindex.c program generates some C code that implements
54 static lookup tables. See the header comment in the source code
55 for details on what it does.
 
 
 
56
57 ELETED Makefile
58 DDED Makefile.classic
59 DDED Makefile.in
60 DDED VERSION
61 DDED auto.def
62 DDED autosetup/LICENSE
63 DDED autosetup/README.autosetup
64 DDED autosetup/autosetup
65 DDED autosetup/cc-lib.tcl
66 DDED autosetup/cc-shared.tcl
67 DDED autosetup/cc.tcl
68 DDED autosetup/config.guess
69 DDED autosetup/config.sub
70 DDED autosetup/find-tclsh
71 DDED autosetup/jimsh0.c
72 DDED autosetup/local.tcl
73 DDED autosetup/system.tcl
74 DDED autosetup/test-tclsh
75 DDED configure
--- BUILD.txt
+++ BUILD.txt
@@ -1,41 +1,60 @@
1 To do a complete build, just type:
2
3 ./configure; make
4
5 The ./configure script builds Makefile from Makefile.in based on
6 your system and any options you select (run "./configure --help"
7 for a listing of the available options.)
8
9 If you wish to use the original Makefile with no configuration, you can
10 instead use:
11
12 make -f Makefile.classic
13
14 On a windows box, use one of the Makefiles in the win/ subdirectory,
15 according to your compiler and environment. If you have GCC and MSYS
16 installed on your system, then consider:
17
18 make -f win/Makefile.mingw
19
20 If you have VC++ installed on your system, then consider:
21
22 cd win; nmake /f Makefile.msc
23
24 If you have trouble, or you want to do something fancy, just look at
25 Makefile.classic. There are 6 configuration options that are all well
26 commented. Instead of editing the Makefile.classic, consider copying
27 Makefile.classic to an alternative name such as "GNUMakefile",
28 "BSDMakefile", or "makefile" and editing the copy.
29
30
31 BUILDING OUTSIDE THE SOURCE TREE
32
33 An out of source build is pretty easy:
34
35 1. Make and change to a new directory to do the builds in.
36 2. Run the "configure" script from this directory.
37 3. Type: "make"
38
39 For example:
40
41 mkdir build
42 cd build
43 ../configure
44 make
45
46 This will now keep all generates files seperate from the maintained
47 source code.
48
49 --------------------------------------------------------------------------
50
51 Here are some notes on what is happening behind the scenes:
52
53 * The configure script (if used) examines the options given
54 and runs various tests with the C compiler to create Makefile
55 from the Makefile.in template as well as autoconfig.h
56
57 * The Makefile just sets up a few macros and then invokes the
58 real makefile in src/main.mk. The src/main.mk makefile is
59 automatically generated by a TCL script found at src/makemake.tcl.
60 Do not edit src/main.mk directly. Update src/makemake.tcl and
@@ -51,5 +70,8 @@
70 A header comment in src/translate.c explains in detail what it does.
71
72 * The src/mkindex.c program generates some C code that implements
73 static lookup tables. See the header comment in the source code
74 for details on what it does.
75
76 Additional information on the build process is available from
77 http://www.fossil-scm.org/fossil/doc/trunk/www/makefile.wiki
78
79 ELETED Makefile
80 DDED Makefile.classic
81 DDED Makefile.in
82 DDED VERSION
83 DDED auto.def
84 DDED autosetup/LICENSE
85 DDED autosetup/README.autosetup
86 DDED autosetup/autosetup
87 DDED autosetup/cc-lib.tcl
88 DDED autosetup/cc-shared.tcl
89 DDED autosetup/cc.tcl
90 DDED autosetup/config.guess
91 DDED autosetup/config.sub
92 DDED autosetup/find-tclsh
93 DDED autosetup/jimsh0.c
94 DDED autosetup/local.tcl
95 DDED autosetup/system.tcl
96 DDED autosetup/test-tclsh
97 DDED configure
D Makefile
-3
--- a/Makefile
+++ b/Makefile
@@ -1,3 +0,0 @@
1
-#!/usr/bin/make
2
-#
3
-# This is tg the build. This is used
--- a/Makefile
+++ b/Makefile
@@ -1,3 +0,0 @@
1 #!/usr/bin/make
2 #
3 # This is tg the build. This is used
--- a/Makefile
+++ b/Makefile
@@ -1,3 +0,0 @@
 
 
 
M Makefile.classic

No diff available

A Makefile.classic
+3
--- a/Makefile.classic
+++ b/Makefile.classic
@@ -0,0 +1,3 @@
1
+#!/usr/bin/make
2
+#
3
+# This is tg the build. This is used
--- a/Makefile.classic
+++ b/Makefile.classic
@@ -0,0 +1,3 @@
 
 
 
--- a/Makefile.classic
+++ b/Makefile.classic
@@ -0,0 +1,3 @@
1 #!/usr/bin/make
2 #
3 # This is tg the build. This is used
M Makefile.in
+9
--- a/Makefile.in
+++ b/Makefile.in
@@ -0,0 +1,9 @@
1
+ -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzz -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzzSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -D_HAVE_SQLITE_ssitclsh -DFOSSIL_ENABLE_LEGACY_MV_RM=1@CFLAGS@@CFLAGS@Rdi make source
2
+#
3
+#
4
+OBJDIR = ./@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzz -D_HAVE_SQLITE_ssil-@FOSSILendif
5
+ifeq ($(findstriSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzz -D_HAVE_SQLITE$(MAKE) reconfig
6
+endif
7
+@AUTOREMAKE@ && $(MAKE)
8
+endif
9
+FOSSIL_ENABLE_M +
--- a/Makefile.in
+++ b/Makefile.in
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
--- a/Makefile.in
+++ b/Makefile.in
@@ -0,0 +1,9 @@
1 -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzz -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -D_HAVE_SQLITE_ssil-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzzSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX-@FOSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -D_HAVE_SQLITE_ssitclsh -DFOSSIL_ENABLE_LEGACY_MV_RM=1@CFLAGS@@CFLAGS@Rdi make source
2 #
3 #
4 OBJDIR = ./@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzz -D_HAVE_SQLITE_ssil-@FOSSILendif
5 ifeq ($(findstriSSIL_CI_PFX@P@1BE,N:fossil:@FOSSIL_CI_PFX -fuzz -D_HAVE_SQLITE$(MAKE) reconfig
6 endif
7 @AUTOREMAKE@ && $(MAKE)
8 endif
9 FOSSIL_ENABLE_M +
M VERSION
+1
--- a/VERSION
+++ b/VERSION
@@ -0,0 +1 @@
1
+1.19
--- a/VERSION
+++ b/VERSION
@@ -0,0 +1 @@
 
--- a/VERSION
+++ b/VERSION
@@ -0,0 +1 @@
1 1.19
M auto.def
+2
--- a/auto.def
+++ b/auto.def
@@ -0,0 +1,2 @@
1
+D markdown=0 => {Build with markdown enginemarkdown]} {
2
+MARKDOWMARKDOWipv6=1 =>
--- a/auto.def
+++ b/auto.def
@@ -0,0 +1,2 @@
 
 
--- a/auto.def
+++ b/auto.def
@@ -0,0 +1,2 @@
1 D markdown=0 => {Build with markdown enginemarkdown]} {
2 MARKDOWMARKDOWipv6=1 =>
M autosetup/LICENSE
+35
--- a/autosetup/LICENSE
+++ b/autosetup/LICENSE
@@ -0,0 +1,35 @@
1
+Unless explicitly stated, all files which form part of autosetup
2
+are released under the following license:
3
+
4
+---------------------------------------------------------------------
5
+autosetup - A build environment "autoconfigurator"
6
+
7
+Copyright (c) 2010-2011, WorkWare Systems <http://workware.net.au/>
8
+
9
+Redistribution and use in source and binary forms, with or without
10
+modification, are permitted provided that the following conditions
11
+are met:
12
+
13
+1. Redistributions of source code must retain the above copyright
14
+ notice, this list of conditions and the following disclaimer.
15
+2. Redistributions in binary form must reproduce the above
16
+ copyright notice, this list of conditions and the following
17
+ disclaimer in the documentation and/or other materials
18
+ provided with the distribution.
19
+
20
+THIS SOFTWARE IS PROVIDED BY THE WORKWARE SYSTEMS ``AS IS'' AND ANY
21
+EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
22
+THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
23
+PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WORKWARE
24
+SYSTEMS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
25
+INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
26
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
27
+OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
28
+HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
29
+STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
30
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
31
+ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
32
+
33
+The views and conclusions contained in the software and documentation
34
+are those of the authors and should not be interpreted as representing
35
+official policies, either expressed or implied, of WorkWare Systems.
--- a/autosetup/LICENSE
+++ b/autosetup/LICENSE
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/LICENSE
+++ b/autosetup/LICENSE
@@ -0,0 +1,35 @@
1 Unless explicitly stated, all files which form part of autosetup
2 are released under the following license:
3
4 ---------------------------------------------------------------------
5 autosetup - A build environment "autoconfigurator"
6
7 Copyright (c) 2010-2011, WorkWare Systems <http://workware.net.au/>
8
9 Redistribution and use in source and binary forms, with or without
10 modification, are permitted provided that the following conditions
11 are met:
12
13 1. Redistributions of source code must retain the above copyright
14 notice, this list of conditions and the following disclaimer.
15 2. Redistributions in binary form must reproduce the above
16 copyright notice, this list of conditions and the following
17 disclaimer in the documentation and/or other materials
18 provided with the distribution.
19
20 THIS SOFTWARE IS PROVIDED BY THE WORKWARE SYSTEMS ``AS IS'' AND ANY
21 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
22 THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
23 PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WORKWARE
24 SYSTEMS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
25 INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
26 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
27 OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
28 HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
29 STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
30 ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
31 ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
32
33 The views and conclusions contained in the software and documentation
34 are those of the authors and should not be interpreted as representing
35 official policies, either expressed or implied, of WorkWare Systems.
M autosetup/README.autosetup
+1
--- a/autosetup/README.autosetup
+++ b/autosetup/README.autosetup
@@ -0,0 +1 @@
1
+This is autosetup v0.6.2. S
--- a/autosetup/README.autosetup
+++ b/autosetup/README.autosetup
@@ -0,0 +1 @@
 
--- a/autosetup/README.autosetup
+++ b/autosetup/README.autosetup
@@ -0,0 +1 @@
1 This is autosetup v0.6.2. S
M autosetup/autosetup
+31
--- a/autosetup/autosetup
+++ b/autosetup/autosetup
@@ -0,0 +1,31 @@
1
+#!/bin/sh
2
+4 => "create an initial 'confbool init]$source#!/bin/sh
3
+parray define
4
+ } else {
5
+ return $alias
6
+ }[error-location $msg]use cc cc-sharedupleve`"$dir/find-tclsh" || echo false`eduplevel #0#!/bin/sh
7
+${m}.tcl
8
+uplevel #0 [list source $source]put2put3 "Looking for $source"![string m 2>/dev/null} {
9
+ set create_configure 1Use --initset cre#!/bin/sh
10
+4 => "creat # Jim uses system() for exec under mingw, so
11
+ # we need to fetch the output ourselves
12
+ set tmpfile auto[format %04x [rand 10000]].tmp
13
+ set rc [catch [list exec {*}$readfile $tmpfile]
14
+ file delete $tmpfile
15
+ return -code $rc $result breakn/sh
16
+4 => "create an in"create an initial 'c[lrange $argv $i end]`"$dir/find-tclsh" || echo false`autoconf-hsh
17
+4 => "create an initial 'confbool init]$source#!/bin/sh
18
+parray define
19
+ } else {
20
+ return $alias
21
+ }[error-location $msg]use cc cc-shareduplevel #0#!/bin/sh
22
+${m}.tcl
23
+uplevel #0 [list source $source]put2put3 "Looking for $source"![string m 2>/dev/null} {
24
+ set create_configure 1Use --initset cre#!/bin/sh
25
+4 => "creat # Jim uses system() for exec under mingw, so
26
+ # we need to fetch the output ourselves
27
+ set tmpfile auto[format %04x [rand 10000]].tmp
28
+ set rc [catch [list exec {*}$readfile $tmpfile]
29
+ file delete $tmpfile
30
+ return -code $rc $result!/bin/sh
31
+4 => "create an
--- a/autosetup/autosetup
+++ b/autosetup/autosetup
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/autosetup
+++ b/autosetup/autosetup
@@ -0,0 +1,31 @@
1 #!/bin/sh
2 4 => "create an initial 'confbool init]$source#!/bin/sh
3 parray define
4 } else {
5 return $alias
6 }[error-location $msg]use cc cc-sharedupleve`"$dir/find-tclsh" || echo false`eduplevel #0#!/bin/sh
7 ${m}.tcl
8 uplevel #0 [list source $source]put2put3 "Looking for $source"![string m 2>/dev/null} {
9 set create_configure 1Use --initset cre#!/bin/sh
10 4 => "creat # Jim uses system() for exec under mingw, so
11 # we need to fetch the output ourselves
12 set tmpfile auto[format %04x [rand 10000]].tmp
13 set rc [catch [list exec {*}$readfile $tmpfile]
14 file delete $tmpfile
15 return -code $rc $result breakn/sh
16 4 => "create an in"create an initial 'c[lrange $argv $i end]`"$dir/find-tclsh" || echo false`autoconf-hsh
17 4 => "create an initial 'confbool init]$source#!/bin/sh
18 parray define
19 } else {
20 return $alias
21 }[error-location $msg]use cc cc-shareduplevel #0#!/bin/sh
22 ${m}.tcl
23 uplevel #0 [list source $source]put2put3 "Looking for $source"![string m 2>/dev/null} {
24 set create_configure 1Use --initset cre#!/bin/sh
25 4 => "creat # Jim uses system() for exec under mingw, so
26 # we need to fetch the output ourselves
27 set tmpfile auto[format %04x [rand 10000]].tmp
28 set rc [catch [list exec {*}$readfile $tmpfile]
29 file delete $tmpfile
30 return -code $rc $result!/bin/sh
31 4 => "create an
M autosetup/cc-lib.tcl
+63
--- a/autosetup/cc-lib.tcl
+++ b/autosetup/cc-lib.tcl
@@ -0,0 +1,63 @@
1
+# Copyright (c) 2011 WorkWare Systems http://www.workware.net.au/
2
+# All rights reserved
3
+
4
+# @synopsis:
5
+#
6
+# Provides a library of common tests on top ofmodule-options {}of the 'cc' module.
7
+
8
+use cc
9
+
10
+# @cc-check-lfsAC_SYS_LARGEFILE macro
11
+#
12
+# defines 'HAVE_LFS' if L S is available,
13
+# and defines '_FILE_OFFSET_BITS=64' if necessary
14
+#
15
+# Returns 1 if 'LFS' is available or 0 otherwise
16
+#
17
+proc cc-check-lfs {} {
18
+ cc-check-includes sys/types.h
19
+ msg-checking "Checking if -D_FILE_OFFSET_BITS=64 is needed..."
20
+ set lfs 1
21
+ if {[msg-quiet cc-with {-includes sys/types.h} {cc-check-sizeof off_t}] == 8} {
22
+ msg-result no
23
+ } elseif {[msg-quiet cc-with {-includes sys/types.h -cflags -D_FILE_OFFSET_BITS=64} {cc-check-sizeof off_t}] == 8} {
24
+ define _FILE_OFFSET_BITS 64
25
+ msg-result yes
26
+ } else {
27
+ set lfs 0
28
+ msg-result none
29
+ }
30
+ define-feature lfs $lfs
31
+ return $lfs
32
+}
33
+
34
+# @cc-check-endian
35
+#
36
+# The equivalent of th eAC_C_BIGENDIAN macroth e 'AC_C_BIGENDIAN' macro.
37
+#
38
+# defines 'HAVE_BIG_ENDIAN' if endian is known to be big,
39
+# or 'HAVE_LITTLE_ENDIAN' if endian is known to be little.
40
+#
41
+# Returns 1 if determined, or 0 if not.
42
+#
43
+proc cc-check-endian {} {
44
+ cc-check-includes sys/types.h sys/param.h
45
+ set rc 0
46
+ msg-checking "Checking endian..."
47
+ cc-with {-includes {sys/types.h sys/param.h}} {
48
+ if {[cctest -code {
49
+ #if! ! defined(LITTLE_ENDIAN) || !defined(BYTE_ORDER)
50
+ #error unknown
51
+ #elif BYTE_ORDER != LITTLE_ENDIAN
52
+ #error big
53
+ #endif
54
+ }]} {
55
+ define-feature little-endian
56
+ msg-result "little"
57
+ set rc 1
58
+ } else {
59
+ msg-result "unknown"
60
+ }
61
+ }
62
+ return $rc
63
+}
--- a/autosetup/cc-lib.tcl
+++ b/autosetup/cc-lib.tcl
@@ -0,0 +1,63 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/cc-lib.tcl
+++ b/autosetup/cc-lib.tcl
@@ -0,0 +1,63 @@
1 # Copyright (c) 2011 WorkWare Systems http://www.workware.net.au/
2 # All rights reserved
3
4 # @synopsis:
5 #
6 # Provides a library of common tests on top ofmodule-options {}of the 'cc' module.
7
8 use cc
9
10 # @cc-check-lfsAC_SYS_LARGEFILE macro
11 #
12 # defines 'HAVE_LFS' if L S is available,
13 # and defines '_FILE_OFFSET_BITS=64' if necessary
14 #
15 # Returns 1 if 'LFS' is available or 0 otherwise
16 #
17 proc cc-check-lfs {} {
18 cc-check-includes sys/types.h
19 msg-checking "Checking if -D_FILE_OFFSET_BITS=64 is needed..."
20 set lfs 1
21 if {[msg-quiet cc-with {-includes sys/types.h} {cc-check-sizeof off_t}] == 8} {
22 msg-result no
23 } elseif {[msg-quiet cc-with {-includes sys/types.h -cflags -D_FILE_OFFSET_BITS=64} {cc-check-sizeof off_t}] == 8} {
24 define _FILE_OFFSET_BITS 64
25 msg-result yes
26 } else {
27 set lfs 0
28 msg-result none
29 }
30 define-feature lfs $lfs
31 return $lfs
32 }
33
34 # @cc-check-endian
35 #
36 # The equivalent of th eAC_C_BIGENDIAN macroth e 'AC_C_BIGENDIAN' macro.
37 #
38 # defines 'HAVE_BIG_ENDIAN' if endian is known to be big,
39 # or 'HAVE_LITTLE_ENDIAN' if endian is known to be little.
40 #
41 # Returns 1 if determined, or 0 if not.
42 #
43 proc cc-check-endian {} {
44 cc-check-includes sys/types.h sys/param.h
45 set rc 0
46 msg-checking "Checking endian..."
47 cc-with {-includes {sys/types.h sys/param.h}} {
48 if {[cctest -code {
49 #if! ! defined(LITTLE_ENDIAN) || !defined(BYTE_ORDER)
50 #error unknown
51 #elif BYTE_ORDER != LITTLE_ENDIAN
52 #error big
53 #endif
54 }]} {
55 define-feature little-endian
56 msg-result "little"
57 set rc 1
58 } else {
59 msg-result "unknown"
60 }
61 }
62 return $rc
63 }
M autosetup/cc-shared.tcl
+30
--- a/autosetup/cc-shared.tcl
+++ b/autosetup/cc-shared.tcl
@@ -0,0 +1,30 @@
1
+# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
2
+# All rights reserved
3
+
4
+# @synopsis:
5
+#
6
+# The 'cc-shared' module provides support for shared libraries and shared objects.
7
+# It defines the following variables:
8
+#
9
+## SH_CFLAGS Flags to use compiling sources destined for a shared library
10
+## SH_LDFLAGS Flags to use linking t the soname when cread libs - %s = version
11
+## SHOBJ_CFLAGS Flags to use compiling sources destined for a shared object
12
+## ll symbols table, %s = path
13
+## SH_LINKFLAGS Flags to use linking an executable which will load shared objects
14
+## LD_LIBRARY_PATH Environment variable which specifie
15
+module-options {}
16
+
17
+foreach i {SH_LINKFLAGS SH_CFLAGS SH_LDFLAGS SHOBJ_CFLAGS SHOBJ_LLD_LIBRARY_PATH define SHOBJ_LDFLAGS_R -bobject
18
+## SHOBJ_LDFLAGS Flags to are System# Copyright ( Flags to usDYLD_LIBRARY_PATH
19
+ }
20
+ *-*-ming* {sharedOBJ_LDFLAGSshared}
21
+ *-*-solaris* {
22
+# Copyright (c) 20mwed
23
+## SHOBJ_LD-Kpic Flags to us-Kpic"-G"
24
+define LD_LIBthe toolch-strip-unneeded
25
+
26
+# Note: Thiain
27
+# http://sourceforge sparc* {
28
+AGS SHOBJ_LDFLAGS_rdynamicference for identify-fPICshared Flags to us-fPIC}
29
+ * {
30
+ # Generic Unix settingsstems http://www.workwa# Copyrigference for identifying PICmicference for idCtify-fPfPICicference for identify-f
--- a/autosetup/cc-shared.tcl
+++ b/autosetup/cc-shared.tcl
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/cc-shared.tcl
+++ b/autosetup/cc-shared.tcl
@@ -0,0 +1,30 @@
1 # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
2 # All rights reserved
3
4 # @synopsis:
5 #
6 # The 'cc-shared' module provides support for shared libraries and shared objects.
7 # It defines the following variables:
8 #
9 ## SH_CFLAGS Flags to use compiling sources destined for a shared library
10 ## SH_LDFLAGS Flags to use linking t the soname when cread libs - %s = version
11 ## SHOBJ_CFLAGS Flags to use compiling sources destined for a shared object
12 ## ll symbols table, %s = path
13 ## SH_LINKFLAGS Flags to use linking an executable which will load shared objects
14 ## LD_LIBRARY_PATH Environment variable which specifie
15 module-options {}
16
17 foreach i {SH_LINKFLAGS SH_CFLAGS SH_LDFLAGS SHOBJ_CFLAGS SHOBJ_LLD_LIBRARY_PATH define SHOBJ_LDFLAGS_R -bobject
18 ## SHOBJ_LDFLAGS Flags to are System# Copyright ( Flags to usDYLD_LIBRARY_PATH
19 }
20 *-*-ming* {sharedOBJ_LDFLAGSshared}
21 *-*-solaris* {
22 # Copyright (c) 20mwed
23 ## SHOBJ_LD-Kpic Flags to us-Kpic"-G"
24 define LD_LIBthe toolch-strip-unneeded
25
26 # Note: Thiain
27 # http://sourceforge sparc* {
28 AGS SHOBJ_LDFLAGS_rdynamicference for identify-fPICshared Flags to us-fPIC}
29 * {
30 # Generic Unix settingsstems http://www.workwa# Copyrigference for identifying PICmicference for idCtify-fPfPICicference for identify-f
M autosetup/cc.tcl
+314
--- a/autosetup/cc.tcl
+++ b/autosetup/cc.tcl
@@ -0,0 +1,314 @@
1
+# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
2
+# All rights reserved
3
+
4
+# @synopsis:
5
+#
6
+# The 'cc' module supports checking various 'features' of the C or C++
7
+# compiler/linker environment. Common commands are ,
8
+# cc-check-types, cc-check-functions, cc-with, make-autoconf-h and make-templatement variables are used if set:
9
+#
10
+## CC - C compiler
11
+## CXX - C++ compiler
12
+## CPP - C preprocessor
13
+##CACHE - Set to "none" to disable automatic use of ccache
14
+## Ciler flags
15
+## CXXFLAGS - Additional C++ compiler flags
16
+## LDFLAGS - Additional compiler flags during linking
17
+## LINKFLAGS - ?How is this diffes to use (for all tests)
18
+## CROSS - Tool prefix for cross compilation
19
+#
20
+# The following variables are defined from the corresponding
21
+# environment variables if set.
22
+#
23
+## CC_FOR_BUILD
24
+## LD
25
+
26
+use systPPFLAGS
27
+## LINKFLAGSmodule-options {}
28
+
29
+# Checks for the existence of the giveNote that the return code is not meaningful
30
+proc cc-check-something {name code} {
31
+ uplevel 1 $codee given type/strunction {function} {
32
+ cctest -link 1 -declare "extern void $function\(void);" -code "$function\();"
33
+}
34
+
35
+# Checks for the existence of the given type by compiling
36
+proc cctest_type {type} {
37
+ cctest -code "$type _x;"
38
+}
39
+
40
+# Checks for the existence of the given type/structure member.
41
+# e.g. "struct stat.st_mtime"
42
+proc cctest_member {struct_member} {
43
+ # split at the first dot
44
+ regexp {^([^.]+)[.](.*)$} $lassign me;\n#endif"
45
+}
46
+
47
+# @cc-ch
48
+# e.g. "struct stypeses a variable with the 4 8 1 2 16 32} {
49
+ if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} {
50
+ set size $i
51
+ break
52
+ }
53
+ }
54
+ msg-result $size
55
+ set define [feature-define-name $type SIZEOF_]
56
+ define $define $size
57
+ }
58
+ # Return the last result
59
+ get-define $define
60
+}
61
+
62
+# Checks for each feature in $list by using the given script.
63
+#
64
+# When the script is evaluated, $each is set to the feature
65
+# being checked, and $extra is set to any additional cctest args.
66
+#
67
+# Returns 1 if all features were found, or 0 otherwise.
68
+proc cc-check-some-feature {list script} {
69
+ set ret 1
70
+ foreach each $list {
71
+ if {![check-feature $each $script]} {
72
+ set ret 0
73
+ }
74
+ }
75
+ return $ret
76
+}
77
+
78
+# @cc-check-includes includes ...
79
+#
80
+# Checks that the given include files can be used.
81
+proc cc-check-includes {args} {
82
+ c
83
+procproc cctest_define {cc-check-some-xp {^([^.]+)[.](.*)$} $struct_member -> struct member
84
+ cctest -code "static $struct _s; return sizeof(_s.$member);"
85
+}
86
+
87
+# Checks for the existence of the given define by compiling
88
+#
89
+proc cctest_define {name} {
90
+ cctest -code "#ifndef $name\n#error not defined\n#endif"
91
+}
92
+
93
+# Checks forcks that the given include files can be used.
94
+proc cc-check-includes {args} {
95
+ cc-check-some-feature $args {
96
+ set with {}
97
+ if {[dict exists $::autosetup(cc-include-deps) $each]} {
98
+ set deps! cc-check-include$exeecks tha]} {
99
+ cc-with [list -includes $w }
100
+ define $TOOL $exeed if set:
101
+#
102
+## CC
103
+# @synopsis:
104
+#
105
+# The 'cc' module supports checking various 'features' of the C or C++
106
+# compiler/linker environment. Common commands are ,
107
+# cc-check-types, cc-check-functions, cc-with, make-autoconf-h and make-templatement variables are used if set:
108
+#
109
+## CC - C compiler
110
+## CXX - C++ compiler
111
+## CPP - C preprocessor
112
+##CACHE - Set to "none" to disable automatic use of ccache
113
+## Ciler flags
114
+## CXXFLAGS - Additional C++ compiler flags
115
+## LDFLAGS - Additional compiler flags during linking
116
+## LINKFLAGS - ?How is this diffes to use (for all tests)
117
+## CROSS - Tool prefix for cross compilation
118
+#
119
+# The following variables are defined from the corresponding
120
+# environment variables if set.
121
+#
122
+## CC_FOR_BUILD
123
+## LD
124
+
125
+use systPPFLAGS
126
+## LINKFLAGSmodule-options {}
127
+
128
+# Checks for the existence of the giveNote that the return code is not meaningful
129
+proc cc-check-something {name code} {
130
+ uplevel 1 $codee given type/strunction {function} {
131
+ cctest -link 1 -declare "extern void $function\(void);" -code "$function\();"
132
+}
133
+
134
+# Checks for the existence of the given type by compiling
135
+proc cctest_type {type} {
136
+ cctest -code "$type _x;"
137
+}
138
+
139
+# Checks for the existence of the given type/structure member.
140
+# e.g. "struct stat.st_mtime"
141
+proc cctest_member {struct_member} {
142
+ # split at the first dot
143
+ regexp {^([^.]+)[.](.*)$} $lassign [split $struct_member .]rn sizeof(_s.$member);"
144
+}
145
+
146
+# Checks for the existence of the given define by compiling
147
+#
148
+proc cctest_define {name} {
149
+ cctest -code "#ifndef $name\n#error not defined\n#endif"
150
+}
151
+
152
+# Checks for the existence of the given name either as
153
+# a macro (#define) or an rvalue (such as an enum)
154
+#
155
+proc cctest_decl {name} {
156
+ cctest -code "#ifndef $name\n(void)$name;\n#endif"
157
+}
158
+
159
+# @cc-check-sizeof type ...
160
+#
161
+# Checks the size of the given types (between 1 and 32, inclusive).
162
+# Defines a variable with the size determined, or 'unknown'"unknown"ng', defines 'SIZEOF_LSIZEOF_LONG_LONGe.
163
+#
164
+proc cc-check-sizeof {args} {
165
+ foreach type $args {
166
+ msg-checking "Checking for sizeof $type..."
167
+ set size unknown
168
+ # Try the most common sizes first
169
+ foreach i {4 8 1 2 16 32} {
170
+ if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} {
171
+ set size $i
172
+ break
173
+ }
174
+ }
175
+ msg-result $size
176
+ set define [feature-define-name $type SIZEOF_]
177
+ define $define $size
178
+ }
179
+ # Return the last result
180
+ get-define $define
181
+}
182
+
183
+# Checks for each feature in $list by using the given script.
184
+#
185
+# When the script is evaluated, $each is set to the feature
186
+# being checked, and $extra is set to any additional cctest args.
187
+#
188
+# Returns 1 if all features were found, or 0 otherwise.
189
+proc cc-check-some-feature {list script} {
190
+ set ret 1
191
+ foreach each $list {
192
+ if {![check-feature $each $script]} {
193
+ set ret 0
194
+ }
195
+ }
196
+ return $ret
197
+}
198
+
199
+# @cc-check-includes includes ...
200
+#
201
+# Checks that the given include files can be used.
202
+proc cc-check-includes {args} {
203
+ c
204
+procproc cctest_define {cc-check-some-xp {^([^.]+)[.](.*)$} $struct_member -> struct member
205
+ cctest -code "static $struct _s; return sizeof(_s.$member);"
206
+}
207
+
208
+# Checks for the existence of the given define by compiling
209
+#
210
+proc cctest_define {name} {
211
+ cctest -code "#ifndef $name\n#error not defined\n#endif"
212
+}
213
+
214
+# Checks for the existence of the given name either as
215
+# a macro (#define) or an rvalue (such as an enum)
216
+#
217
+proc cctest_decl {name} {
218
+ cctest -code "#ifndef $name\n(void)$name;\n#endif"
219
+}
220
+ given types (bet#includeddetermined, or 'unknown' otherwise.
221
+# e.g. for type 'long long', defines 'SIZEOF_LONG_LONG'.
222
+# Returns the size of the last type.
223
+#
224
+proc cc-check-sizeof {args} {
225
+ foreach type $adfirst dot
226
+ regexp {^([^.]+)[.](.*)$} $struct_member -> struct member
227
+ cctest -code "static $struct _s; return sizeof(_s.$member);"
228
+}
229
+
230
+# Checks for the existence of the given define by compiling
231
+#
232
+proc cctest_define FLAGS - ?How is this di# @synopsis:
233
+#
234
+# The 'cc' module supports checking various 'features' of the C or C++
235
+# compiler/linker environment. Common commands are ,
236
+# cc-check-types, cc-check-functions, cc-with, make-autoconf-h and make-templatement variables are used if set:
237
+#
238
+## CC - C compiler
239
+## CXX - C++ compiler
240
+## CPP - C preprocessor
241
+##CACHE - Set to "none" to disfor a declsable automatic use of ccache
242
+## Ciler flags
243
+## C# Copyright (c) 2010 Work foreach type $args {
244
+ msg-checking "Checking for sizeof $type..."
245
+ set size unknown
246
+ # Try the most common sizes first
247
+ foreach i {4 8 1 2 16 32} {
248
+ if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} {
249
+ set size $i
250
+ break
251
+ }
252
+ }
253
+ msg-result $size
254
+ set define [feature-define-name $type SIZEOF_]
255
+ define $define $size
256
+ }
257
+ # Return the last result
258
+ get-define $define
259
+}
260
+
261
+# Checks for each feature in $list by using the given script.
262
+#
263
+# When the script is evaluated, $each is set to the feature
264
+# being checked, and $extra is set to any additional cctest args.
265
+#
266
+# Returns 1 if all features were found, or 0 otherwise.
267
+proc cc-check-some-feature {list script} {
268
+ set ret 1ch $script]} {
269
+ set ret 0
270
+ }
271
+ }
272
+ return $ret
273
+}
274
+
275
+# @cc-check-includes includes ...
276
+#
277
+# Checks that the given include files can be used.
278
+proc cc-check-includes {args} {
279
+ cc-check-some-feature $args {
280
+ set with {}
281
+ if {[dict exists $::autosetup(cc-include-deps) $each]} {
282
+ set deps [dict keys [dict get $::autosetup(cc-include-deps) $each]]
283
+ msg-quiet cc-check-includes {*}$deps
284
+ foreach i $deps {
285
+ if {[have-feature $i]} {
286
+ lappend with $i
287
+ure $each $scriptargs {
288
+ set new($name) $value
289
+ } $i]} {
290
+ lappend$newme\n(void)$name;\n#th]} {
291
+ cc-with [list -includes $with] {
292
+ cctest -includes $each
293
+ }
294
+ } else {
295
+ cctest -includes $each
296
+ }
297
+ }
298
+}
299
+
300
+# @cc-include-needs include required ...
301
+#
302
+# Ensures that when checking for '$include', a check is first
303
+# made for eachtk is first
304
+# made for each '$required' file, and if found, it is included with '#include'.
305
+proc cc-include-needs {file args} {
306
+ foreach depfile $args {
307
+ dict set ::autosetup(cc-include-deps) $file $depfile 1
308
+ }
309
+}
310
+
311
+# C *-*-darwin* {
312
+ # Don't-gstabs
313
+ }
314
+ }$value$valuecc-check-tools ld.o
--- a/autosetup/cc.tcl
+++ b/autosetup/cc.tcl
@@ -0,0 +1,314 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/cc.tcl
+++ b/autosetup/cc.tcl
@@ -0,0 +1,314 @@
1 # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
2 # All rights reserved
3
4 # @synopsis:
5 #
6 # The 'cc' module supports checking various 'features' of the C or C++
7 # compiler/linker environment. Common commands are ,
8 # cc-check-types, cc-check-functions, cc-with, make-autoconf-h and make-templatement variables are used if set:
9 #
10 ## CC - C compiler
11 ## CXX - C++ compiler
12 ## CPP - C preprocessor
13 ##CACHE - Set to "none" to disable automatic use of ccache
14 ## Ciler flags
15 ## CXXFLAGS - Additional C++ compiler flags
16 ## LDFLAGS - Additional compiler flags during linking
17 ## LINKFLAGS - ?How is this diffes to use (for all tests)
18 ## CROSS - Tool prefix for cross compilation
19 #
20 # The following variables are defined from the corresponding
21 # environment variables if set.
22 #
23 ## CC_FOR_BUILD
24 ## LD
25
26 use systPPFLAGS
27 ## LINKFLAGSmodule-options {}
28
29 # Checks for the existence of the giveNote that the return code is not meaningful
30 proc cc-check-something {name code} {
31 uplevel 1 $codee given type/strunction {function} {
32 cctest -link 1 -declare "extern void $function\(void);" -code "$function\();"
33 }
34
35 # Checks for the existence of the given type by compiling
36 proc cctest_type {type} {
37 cctest -code "$type _x;"
38 }
39
40 # Checks for the existence of the given type/structure member.
41 # e.g. "struct stat.st_mtime"
42 proc cctest_member {struct_member} {
43 # split at the first dot
44 regexp {^([^.]+)[.](.*)$} $lassign me;\n#endif"
45 }
46
47 # @cc-ch
48 # e.g. "struct stypeses a variable with the 4 8 1 2 16 32} {
49 if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} {
50 set size $i
51 break
52 }
53 }
54 msg-result $size
55 set define [feature-define-name $type SIZEOF_]
56 define $define $size
57 }
58 # Return the last result
59 get-define $define
60 }
61
62 # Checks for each feature in $list by using the given script.
63 #
64 # When the script is evaluated, $each is set to the feature
65 # being checked, and $extra is set to any additional cctest args.
66 #
67 # Returns 1 if all features were found, or 0 otherwise.
68 proc cc-check-some-feature {list script} {
69 set ret 1
70 foreach each $list {
71 if {![check-feature $each $script]} {
72 set ret 0
73 }
74 }
75 return $ret
76 }
77
78 # @cc-check-includes includes ...
79 #
80 # Checks that the given include files can be used.
81 proc cc-check-includes {args} {
82 c
83 procproc cctest_define {cc-check-some-xp {^([^.]+)[.](.*)$} $struct_member -> struct member
84 cctest -code "static $struct _s; return sizeof(_s.$member);"
85 }
86
87 # Checks for the existence of the given define by compiling
88 #
89 proc cctest_define {name} {
90 cctest -code "#ifndef $name\n#error not defined\n#endif"
91 }
92
93 # Checks forcks that the given include files can be used.
94 proc cc-check-includes {args} {
95 cc-check-some-feature $args {
96 set with {}
97 if {[dict exists $::autosetup(cc-include-deps) $each]} {
98 set deps! cc-check-include$exeecks tha]} {
99 cc-with [list -includes $w }
100 define $TOOL $exeed if set:
101 #
102 ## CC
103 # @synopsis:
104 #
105 # The 'cc' module supports checking various 'features' of the C or C++
106 # compiler/linker environment. Common commands are ,
107 # cc-check-types, cc-check-functions, cc-with, make-autoconf-h and make-templatement variables are used if set:
108 #
109 ## CC - C compiler
110 ## CXX - C++ compiler
111 ## CPP - C preprocessor
112 ##CACHE - Set to "none" to disable automatic use of ccache
113 ## Ciler flags
114 ## CXXFLAGS - Additional C++ compiler flags
115 ## LDFLAGS - Additional compiler flags during linking
116 ## LINKFLAGS - ?How is this diffes to use (for all tests)
117 ## CROSS - Tool prefix for cross compilation
118 #
119 # The following variables are defined from the corresponding
120 # environment variables if set.
121 #
122 ## CC_FOR_BUILD
123 ## LD
124
125 use systPPFLAGS
126 ## LINKFLAGSmodule-options {}
127
128 # Checks for the existence of the giveNote that the return code is not meaningful
129 proc cc-check-something {name code} {
130 uplevel 1 $codee given type/strunction {function} {
131 cctest -link 1 -declare "extern void $function\(void);" -code "$function\();"
132 }
133
134 # Checks for the existence of the given type by compiling
135 proc cctest_type {type} {
136 cctest -code "$type _x;"
137 }
138
139 # Checks for the existence of the given type/structure member.
140 # e.g. "struct stat.st_mtime"
141 proc cctest_member {struct_member} {
142 # split at the first dot
143 regexp {^([^.]+)[.](.*)$} $lassign [split $struct_member .]rn sizeof(_s.$member);"
144 }
145
146 # Checks for the existence of the given define by compiling
147 #
148 proc cctest_define {name} {
149 cctest -code "#ifndef $name\n#error not defined\n#endif"
150 }
151
152 # Checks for the existence of the given name either as
153 # a macro (#define) or an rvalue (such as an enum)
154 #
155 proc cctest_decl {name} {
156 cctest -code "#ifndef $name\n(void)$name;\n#endif"
157 }
158
159 # @cc-check-sizeof type ...
160 #
161 # Checks the size of the given types (between 1 and 32, inclusive).
162 # Defines a variable with the size determined, or 'unknown'"unknown"ng', defines 'SIZEOF_LSIZEOF_LONG_LONGe.
163 #
164 proc cc-check-sizeof {args} {
165 foreach type $args {
166 msg-checking "Checking for sizeof $type..."
167 set size unknown
168 # Try the most common sizes first
169 foreach i {4 8 1 2 16 32} {
170 if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} {
171 set size $i
172 break
173 }
174 }
175 msg-result $size
176 set define [feature-define-name $type SIZEOF_]
177 define $define $size
178 }
179 # Return the last result
180 get-define $define
181 }
182
183 # Checks for each feature in $list by using the given script.
184 #
185 # When the script is evaluated, $each is set to the feature
186 # being checked, and $extra is set to any additional cctest args.
187 #
188 # Returns 1 if all features were found, or 0 otherwise.
189 proc cc-check-some-feature {list script} {
190 set ret 1
191 foreach each $list {
192 if {![check-feature $each $script]} {
193 set ret 0
194 }
195 }
196 return $ret
197 }
198
199 # @cc-check-includes includes ...
200 #
201 # Checks that the given include files can be used.
202 proc cc-check-includes {args} {
203 c
204 procproc cctest_define {cc-check-some-xp {^([^.]+)[.](.*)$} $struct_member -> struct member
205 cctest -code "static $struct _s; return sizeof(_s.$member);"
206 }
207
208 # Checks for the existence of the given define by compiling
209 #
210 proc cctest_define {name} {
211 cctest -code "#ifndef $name\n#error not defined\n#endif"
212 }
213
214 # Checks for the existence of the given name either as
215 # a macro (#define) or an rvalue (such as an enum)
216 #
217 proc cctest_decl {name} {
218 cctest -code "#ifndef $name\n(void)$name;\n#endif"
219 }
220 given types (bet#includeddetermined, or 'unknown' otherwise.
221 # e.g. for type 'long long', defines 'SIZEOF_LONG_LONG'.
222 # Returns the size of the last type.
223 #
224 proc cc-check-sizeof {args} {
225 foreach type $adfirst dot
226 regexp {^([^.]+)[.](.*)$} $struct_member -> struct member
227 cctest -code "static $struct _s; return sizeof(_s.$member);"
228 }
229
230 # Checks for the existence of the given define by compiling
231 #
232 proc cctest_define FLAGS - ?How is this di# @synopsis:
233 #
234 # The 'cc' module supports checking various 'features' of the C or C++
235 # compiler/linker environment. Common commands are ,
236 # cc-check-types, cc-check-functions, cc-with, make-autoconf-h and make-templatement variables are used if set:
237 #
238 ## CC - C compiler
239 ## CXX - C++ compiler
240 ## CPP - C preprocessor
241 ##CACHE - Set to "none" to disfor a declsable automatic use of ccache
242 ## Ciler flags
243 ## C# Copyright (c) 2010 Work foreach type $args {
244 msg-checking "Checking for sizeof $type..."
245 set size unknown
246 # Try the most common sizes first
247 foreach i {4 8 1 2 16 32} {
248 if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} {
249 set size $i
250 break
251 }
252 }
253 msg-result $size
254 set define [feature-define-name $type SIZEOF_]
255 define $define $size
256 }
257 # Return the last result
258 get-define $define
259 }
260
261 # Checks for each feature in $list by using the given script.
262 #
263 # When the script is evaluated, $each is set to the feature
264 # being checked, and $extra is set to any additional cctest args.
265 #
266 # Returns 1 if all features were found, or 0 otherwise.
267 proc cc-check-some-feature {list script} {
268 set ret 1ch $script]} {
269 set ret 0
270 }
271 }
272 return $ret
273 }
274
275 # @cc-check-includes includes ...
276 #
277 # Checks that the given include files can be used.
278 proc cc-check-includes {args} {
279 cc-check-some-feature $args {
280 set with {}
281 if {[dict exists $::autosetup(cc-include-deps) $each]} {
282 set deps [dict keys [dict get $::autosetup(cc-include-deps) $each]]
283 msg-quiet cc-check-includes {*}$deps
284 foreach i $deps {
285 if {[have-feature $i]} {
286 lappend with $i
287 ure $each $scriptargs {
288 set new($name) $value
289 } $i]} {
290 lappend$newme\n(void)$name;\n#th]} {
291 cc-with [list -includes $with] {
292 cctest -includes $each
293 }
294 } else {
295 cctest -includes $each
296 }
297 }
298 }
299
300 # @cc-include-needs include required ...
301 #
302 # Ensures that when checking for '$include', a check is first
303 # made for eachtk is first
304 # made for each '$required' file, and if found, it is included with '#include'.
305 proc cc-include-needs {file args} {
306 foreach depfile $args {
307 dict set ::autosetup(cc-include-deps) $file $depfile 1
308 }
309 }
310
311 # C *-*-darwin* {
312 # Don't-gstabs
313 }
314 }$value$valuecc-check-tools ld.o
M autosetup/config.guess
+307
--- a/autosetup/config.guess
+++ b/autosetup/config.guess
@@ -0,0 +1,307 @@
1
+03-23 the terms of the GNU General Public License as published by
2
+# t(C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
3
+# 2000, 2001, 2002, 2003, 2004, 2005, 2006,oftware Foundation, 0-09-24r
4
+# (at your option) any later version.
5
+#
6
+# This program is distributed in the hope that it will be useful, but
7
+# WITHOUT ANY WARRANTY; without even the implied w2 or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
8
+# General Public License for more details.
9
+#
10
+# You should have received a copy of the GNU General Public License
11
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
12
+#
13
+# As a special exceptionto the GNU General Public License, if you
14
+# distribute this file as part of a program that contains a
15
+# configuration script generated by Autoconwrite to the Free Software
16
+# Foundation, Inc., 51 Franklin Street - Fifth Floor, Bmay include it under
17
+# the same distribution terms that you use for the rest of that
18
+# program. This Exception is an additional permission under section 7
19
+# of the GNU General Public License, version 3 ("GPLv3").
20
+#
21
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
22
+#
23
+# You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
24
+
25
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
26
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
27
+# Attempt to guess a canonical system name.
28
+# Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
29
+# under the terms of the GNU General Public License as published by
30
+# t;e Free Software Foundation, either version 3 of the L;cense, or
31
+# (at your option) any later version.
32
+#
33
+# This program is distributed in the hope that it will be useful, but
34
+# WITHOUT ANY WARRANTY; without even the implied warranty of
35
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
36
+# General Public License for more details.
37
+#
38
+# You should have received a copy of the GNU General Public License
39
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
40
+#
41
+# As a special exceptionto the GNU General Public License, if you
42
+# distribute this file as part of a program that contains a
43
+# configuration script generated by Autoconf, you may include it under
44
+# the same distribution terms that you use for the rest of that
45
+# program. This Exception is an additional permission under section 7
46
+# of the GNU General Public License, version 3 ("GPLv3").
47
+#
48
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
49
+#
50
+# You can get the latest version of this script from:
51
+# https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
52
+
53
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
54
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILD -E $dummy.c'^LIBC' | sed 's, ,,g'`{UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}/sbin/$sysctl/usr/sbin/$sysctl
55
+# Attempt to guess a canoniAttempt to g! /bin/sh
56
+# Attemp#*) machine=${! /bin/sh
57
+# Attemp#! /bin/sh
58
+# Attempt to guess a canonical system name.
59
+# Copyright 1992-2018canonical system name.
60
+# Copyrig! /bin/sh
61
+# Attemp#! /b{UNAME_VERSION}${UNAME_REL{machine}-${os}${release}${! /bin/sh
62
+# Attemp#! /bin/sh
63
+# Attempt to guess a ${! /bin/sh
64
+# Attemp#! /bin/sh
65
+# Attempt to guess a cekkoBSD:*:*)
66
+ echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}SolidBSD:*:*)
67
+ echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}${UNAME_RELEASE}${UNAME_MACHINE}-unk"alpha""alpha""alpha""alphaev5"A"alphaev56""alphapca56""alphapca57""alphaev6""alphaev67""alphaev68""alphaev68""alphaev68""alphaev69""alphaev7""alphaev79"${UNAME_MACHINE}-dec-osf`echo RELEASE}'' ''` /bin/sh
68
+# Attempt to guess a canonical system name.
69
+# Copyright 1992-2018 Free Software FouAttempt to guess a canonical system name.
70
+# Copyright 1992-2018 Free Software Foundation, Inc.
71
+
72
+timestamp='2018-03-08it and/or modify it
73
+# under the terms of the GNU General Public License as published by
74
+# t;e Free Software Foundation, either version 3 of the L;cense, or
75
+# (at your o program.
76
+
77
+
78
+# neral Public License
79
+# along with he GNU General Public03-23 the terms of the and include a ChangeLog
80
+# entrhe GNU General Public License as published by
81
+# t;e Free Software Foundation, either version 3 of the L;cense, or
82
+# (at your option) any later version.
83
+#
84
+# This program is distributed in the hope that it will be useful, but
85
+# WITHOUT ANY WARRANTY; without even the implied warranty of
86
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
87
+# General Public License for more details.
88
+#
89
+# You should have received a copy of the GNU General Public License
90
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
91
+#
92
+# As a special exceptionto the GNU General Public License, if you
93
+# distribute this file as part of a program that contains a
94
+# configuration script generated by Autoconf, you may include it under
95
+# the same distribution terms that you use for the rest of that
96
+# program. This Exception is an additional permission under section 7
97
+# of the GNU General Public License, version 3 ("GPLv3").
98
+#
99
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
100
+#
101
+# You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
102
+
103
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
104
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
105
+# Attempt to guess a canonical system name.
106
+# Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
107
+# under the terms of the GNU General Public License as published by
108
+# t;e Free Software Foundation, either version 3 of the L;cense, or
109
+# (at your option) any later version.
110
+#
111
+# This program is distributed in the hope that it will be useful, but
112
+# WITHOUT ANY WARRANTY; without even the implied warranty of
113
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
114
+# General Public License for more details.
115
+#
116
+# You should have received a copy of the GNU General Public License
117
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
118
+#
119
+# As a special exceptionto the GNU General Public License, if you
120
+# distribute this file as part of a program that contains a
121
+# configuration script generated by Autoconf, you may include it under
122
+# the same distribution terms that you use for the rest of that
123
+# program. This Exception is an additional permission under section 7
124
+# of the GNU General Public License, version 3 ("GPLv3").
125
+#
126
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
127
+#
128
+# You can get the latest version of this script from:
129
+# https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
130
+
131
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
132
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILD -E $dummy.c'^LIBC' | sed 's, ,,g'`{UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}/sbin/$sysctl/usr/sbin/$sysctl
133
+# Attempt to guess a canoniAttempt to g! /bin/sh
134
+# Attemp#*) machine=${! /bin/sh
135
+# Attemp#! /bin/sh
136
+# Attempt to guess a canonical system name.
137
+# Copyright 1992-2018canonical system name.
138
+# Copyrig! /bin/sh
139
+# Attemp#! /b{UNAME_VERSION}${UNAME_REL{machine}-${os}${release}${! /bin/sh
140
+# Attemp#! /bin/sh
141
+# Attempt to guess a ${! /bin/sh
142
+# Attemp#! /bin/sh
143
+# Attempt to guess a cekkoBSD:*:*)
144
+ echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}SolidBSD:*:*)
145
+ echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}${UNAME_RELEASE}${UNAME_MACHINE}-unk"alpha""alpha""alpha""alphaev5"A"alphaev56""alphapca56""alphapca57""alphaev6""alphaev67""alphaev68""alphaev68""alphaev68""alphaev69""alphaev7""alphaev79"${UNAME_MACHINE}-dec-osf`echo RELEASE}'' ''` /bin/sh
146
+# Attempt to guess a canonical system name.
147
+# Copyright 1992-2018 Free Software FouAttempt to guess a canonical system name.
148
+# Copyright 1992-2018 Free Software Foundation, Inc.
149
+
150
+timestamp='2018-03-08it and/or modify it
151
+# under the terms of the GNU General Public License as published by
152
+# t;e Free Software Foundation, either version 3 of the L;cense, or
153
+# (at your o.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
154
+# Attempt to guess a canonical system name.
155
+# Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
156
+# under the terms of the GNU General Public License as published by
157
+# t;e Free Software Foundation, either version 3 of the L;cense, or
158
+# (at your option) any later version.
159
+#
160
+# This program is distributed in the hope that it will be useful, but
161
+# WITHOUT ANY WARRANTY; without even the implied warranty of
162
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
163
+# General Public License for more details.
164
+#
165
+# You should have received a copy of the GNU General Public License
166
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
167
+#
168
+# As a special exceptionto the GNU General Public License, if you
169
+# distribute this file as part of a program that contains a
170
+# configuration script generated by Autoconf, you may include it under
171
+# the same distribution terms that you use for the rest of that
172
+# program. This Exception is an additional permission under section 7
173
+# of the GNU General Public License, version 3 ("GPLv3").
174
+#
175
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
176
+#
177
+# You can get the latest version of this script from:
178
+# https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
179
+
180
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
181
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILD -E $dummy.c'^LIBC' | sed 's, ,,g'`{UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}/sbin/$sysctl/usr/sbin/$sysctl
182
+# Attempt to guess a canoniAttempt to g! /bin/sh
183
+# Attemp#*) machine=${! /bin/sh
184
+# Attemp#! /bin/sh
185
+# Attempt to guess a canonical system name.
186
+# Copyright 1992-2018canonical system name.
187
+# Copyrig! /bin/sh
188
+# Attemp#! /b{UNAME_VERSION}${UNAME_REL{machine}-${os}${release}${! /bin/sh
189
+# Attemp#! /bin/sh
190
+# Attempt to guess a ${! /bin/sh
191
+# Attemp#! /bin/sh
192
+# Attempt to guess a cekkoBSD:*:*)
193
+ echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}SolidBSD:*:*)
194
+ echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}${UNAME_RELEASE}${UNAME_MACHINE}-unk"alpha""alpha""alpha""alphaev5"A"alphaev56""alphapca56""alphapca57""alphaev6""alphaev67""alphaev68""alphaev68""alphaev68""alphaev69""alphaev7""alphaev79"${UNAME_MACHINE}-dec-osf`echo RELEASE}'' ''` /bin/sh
195
+# Attempt to guess a canonical system name.
196
+# Copyright 1992-2018 Free Software FouAttempt to guess a canonical system name.
197
+# Copyright 1992-2018 Free Software Foundation, Inc.
198
+
199
+timestamp='2018-03-08it and/or modify it
200
+# under the terms of the GNU General Public License as published by
201
+# t;e Free Software Foundation, either version 3 of the L;cense, or
202
+# (at your oished by
203
+# t;e Free Software Foundation, either version 3 of the L;cense, or
204
+# (at your option) any later version.
205
+#
206
+# This program is distributed in the hope that it will be useful, but
207
+# WITHOUT ANY WARRANTY; without even the implied warranty of
208
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
209
+# General Public License for more details.
210
+#
211
+# You should have received a copy of the GNU General Public License
212
+# along with this progra(C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000,
213
+2001, 2002, 2003, 2004, 2005, 2006, 2implied warranty ofot, see <https://www.gnu.org/licenses/>.
214
+#
215
+# As a special exceptionto the GNU General Public License, if you
216
+# distribute this file as part of a program that contains a
217
+# configuration script generated by Autoconf, you may include it under
218
+# the same distribution terms that you use for the rest of that
219
+# program. This Exception is an additional permission under section 7
220
+# of the GNU General Public License, version 3 ("GPLv3").
221
+#
222
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
223
+#
224
+# You can get the latest version of this script from:
225
+# https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
226
+
227
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
228
+ cat <<-EOF > $dummy.c`$CC_FORHUP INT TERMneral Public License, version 3 ("GPLv3").
229
+#
230
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
231
+#
232
+# You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
233
+
234
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
235
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
236
+# Attempt to guess a canonical system name.
237
+# Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
238
+# under the terms of the GNU General Public License as published by
239
+# t;e Free Software Foundation, either version 3 of the L;cense, or
240
+# (at your optionHUP INT PIPE TERM03-23 the terms of the GNU General Public License as published by
241
+# t;e Free Software Foundation, either version 3 of the L;cense, or
242
+# (at your option) any later version.
243
+#
244
+# This program is distributed in the hope tha03-23 the terms of the GNU General Public License as published by
245
+# t;e Free Software Foundation, either version 3 of the L;cense, or
246
+# (at your option) any later version.
247
+#
248
+# This program is distributed in the hope that it will be useful, but
249
+# WITHOUT ANY WARRANTY; without even the implied warranty of
250
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
251
+# General Public License for more details.
252
+#
253
+# You should have received a copy of the GNU General Public License
254
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
255
+#
256
+# As a special exceptionto the GNU General Public License, if you
257
+# distribute this file as part of a program that contains a
258
+# configuration script generated by Autoconf, you may include it under
259
+# the same distribution terms that you use for the rest of that
260
+# program. This Exception is an additional permission under section 7
261
+# of the GNU General Public License, version 3 ("GPLv3").
262
+#
263
+# Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
264
+#
265
+# You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
266
+
267
+$dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
268
+ cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
269
+# Attempt to guess a canonical system name.
270
+# Open exit :riscos:*:*|arm t it will be useful, but
271
+03-23 th 23 the terms of the GNU General Public License as published by
272
+# t;e Free Software Foundation, either version 3 of the L;cense, or
273
+# (at your option) any later version.
274
+#
275
+# This program is distributed in the hope that it will be useful, but
276
+# WITHOUT ANY WARRANTY; without even the implied warranty of
277
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
278
+# General Public License for more details.
279
+#
280
+# You should have received a copy of the GNU General Public License
281
+# along with this program; if not, see <https://www.gnu.org/licenses/>.
282
+#
283
+# As a special exceptionto the GNU General Public License, if you
284
+# distribute this file as part of a program that contains a
285
+# configuration script generated by Autoconf, you may include it under
286
+# the same distribution terms that you use for the rest of that
287
+# program. This Exception is an additional permission under section 7
288
+# of the GNU General Public License, version 3 ("GPLv3").
289
+#
290
+# Originally written ms of the GNU Genera532)case "${sc_kernel_bits}" in
291
+ 32) HP_ARCH="hppa2.0n" ;;
292
+esac ;;
293
+
294
+ #include <stdlib.h>
295
+ #include <unistd.h>
296
+
297
+ int main ()
298
+ {
299
+ #if
300
+ #endif
301
+switch (cpu)
302
+ "); break;
303
+ 2_0:
304
+ #if
305
+ switch (bits)
306
+ {
307
+
--- a/autosetup/config.guess
+++ b/autosetup/config.guess
@@ -0,0 +1,307 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/config.guess
+++ b/autosetup/config.guess
@@ -0,0 +1,307 @@
1 03-23 the terms of the GNU General Public License as published by
2 # t(C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999,
3 # 2000, 2001, 2002, 2003, 2004, 2005, 2006,oftware Foundation, 0-09-24r
4 # (at your option) any later version.
5 #
6 # This program is distributed in the hope that it will be useful, but
7 # WITHOUT ANY WARRANTY; without even the implied w2 or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
8 # General Public License for more details.
9 #
10 # You should have received a copy of the GNU General Public License
11 # along with this program; if not, see <https://www.gnu.org/licenses/>.
12 #
13 # As a special exceptionto the GNU General Public License, if you
14 # distribute this file as part of a program that contains a
15 # configuration script generated by Autoconwrite to the Free Software
16 # Foundation, Inc., 51 Franklin Street - Fifth Floor, Bmay include it under
17 # the same distribution terms that you use for the rest of that
18 # program. This Exception is an additional permission under section 7
19 # of the GNU General Public License, version 3 ("GPLv3").
20 #
21 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
22 #
23 # You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
24
25 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
26 cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
27 # Attempt to guess a canonical system name.
28 # Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
29 # under the terms of the GNU General Public License as published by
30 # t;e Free Software Foundation, either version 3 of the L;cense, or
31 # (at your option) any later version.
32 #
33 # This program is distributed in the hope that it will be useful, but
34 # WITHOUT ANY WARRANTY; without even the implied warranty of
35 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
36 # General Public License for more details.
37 #
38 # You should have received a copy of the GNU General Public License
39 # along with this program; if not, see <https://www.gnu.org/licenses/>.
40 #
41 # As a special exceptionto the GNU General Public License, if you
42 # distribute this file as part of a program that contains a
43 # configuration script generated by Autoconf, you may include it under
44 # the same distribution terms that you use for the rest of that
45 # program. This Exception is an additional permission under section 7
46 # of the GNU General Public License, version 3 ("GPLv3").
47 #
48 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
49 #
50 # You can get the latest version of this script from:
51 # https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
52
53 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
54 cat <<-EOF > $dummy.c`$CC_FOR_BUILD -E $dummy.c'^LIBC' | sed 's, ,,g'`{UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}/sbin/$sysctl/usr/sbin/$sysctl
55 # Attempt to guess a canoniAttempt to g! /bin/sh
56 # Attemp#*) machine=${! /bin/sh
57 # Attemp#! /bin/sh
58 # Attempt to guess a canonical system name.
59 # Copyright 1992-2018canonical system name.
60 # Copyrig! /bin/sh
61 # Attemp#! /b{UNAME_VERSION}${UNAME_REL{machine}-${os}${release}${! /bin/sh
62 # Attemp#! /bin/sh
63 # Attempt to guess a ${! /bin/sh
64 # Attemp#! /bin/sh
65 # Attempt to guess a cekkoBSD:*:*)
66 echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}SolidBSD:*:*)
67 echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}${UNAME_RELEASE}${UNAME_MACHINE}-unk"alpha""alpha""alpha""alphaev5"A"alphaev56""alphapca56""alphapca57""alphaev6""alphaev67""alphaev68""alphaev68""alphaev68""alphaev69""alphaev7""alphaev79"${UNAME_MACHINE}-dec-osf`echo RELEASE}'' ''` /bin/sh
68 # Attempt to guess a canonical system name.
69 # Copyright 1992-2018 Free Software FouAttempt to guess a canonical system name.
70 # Copyright 1992-2018 Free Software Foundation, Inc.
71
72 timestamp='2018-03-08it and/or modify it
73 # under the terms of the GNU General Public License as published by
74 # t;e Free Software Foundation, either version 3 of the L;cense, or
75 # (at your o program.
76
77
78 # neral Public License
79 # along with he GNU General Public03-23 the terms of the and include a ChangeLog
80 # entrhe GNU General Public License as published by
81 # t;e Free Software Foundation, either version 3 of the L;cense, or
82 # (at your option) any later version.
83 #
84 # This program is distributed in the hope that it will be useful, but
85 # WITHOUT ANY WARRANTY; without even the implied warranty of
86 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
87 # General Public License for more details.
88 #
89 # You should have received a copy of the GNU General Public License
90 # along with this program; if not, see <https://www.gnu.org/licenses/>.
91 #
92 # As a special exceptionto the GNU General Public License, if you
93 # distribute this file as part of a program that contains a
94 # configuration script generated by Autoconf, you may include it under
95 # the same distribution terms that you use for the rest of that
96 # program. This Exception is an additional permission under section 7
97 # of the GNU General Public License, version 3 ("GPLv3").
98 #
99 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
100 #
101 # You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
102
103 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
104 cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
105 # Attempt to guess a canonical system name.
106 # Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
107 # under the terms of the GNU General Public License as published by
108 # t;e Free Software Foundation, either version 3 of the L;cense, or
109 # (at your option) any later version.
110 #
111 # This program is distributed in the hope that it will be useful, but
112 # WITHOUT ANY WARRANTY; without even the implied warranty of
113 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
114 # General Public License for more details.
115 #
116 # You should have received a copy of the GNU General Public License
117 # along with this program; if not, see <https://www.gnu.org/licenses/>.
118 #
119 # As a special exceptionto the GNU General Public License, if you
120 # distribute this file as part of a program that contains a
121 # configuration script generated by Autoconf, you may include it under
122 # the same distribution terms that you use for the rest of that
123 # program. This Exception is an additional permission under section 7
124 # of the GNU General Public License, version 3 ("GPLv3").
125 #
126 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
127 #
128 # You can get the latest version of this script from:
129 # https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
130
131 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
132 cat <<-EOF > $dummy.c`$CC_FOR_BUILD -E $dummy.c'^LIBC' | sed 's, ,,g'`{UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}/sbin/$sysctl/usr/sbin/$sysctl
133 # Attempt to guess a canoniAttempt to g! /bin/sh
134 # Attemp#*) machine=${! /bin/sh
135 # Attemp#! /bin/sh
136 # Attempt to guess a canonical system name.
137 # Copyright 1992-2018canonical system name.
138 # Copyrig! /bin/sh
139 # Attemp#! /b{UNAME_VERSION}${UNAME_REL{machine}-${os}${release}${! /bin/sh
140 # Attemp#! /bin/sh
141 # Attempt to guess a ${! /bin/sh
142 # Attemp#! /bin/sh
143 # Attempt to guess a cekkoBSD:*:*)
144 echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}SolidBSD:*:*)
145 echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}${UNAME_RELEASE}${UNAME_MACHINE}-unk"alpha""alpha""alpha""alphaev5"A"alphaev56""alphapca56""alphapca57""alphaev6""alphaev67""alphaev68""alphaev68""alphaev68""alphaev69""alphaev7""alphaev79"${UNAME_MACHINE}-dec-osf`echo RELEASE}'' ''` /bin/sh
146 # Attempt to guess a canonical system name.
147 # Copyright 1992-2018 Free Software FouAttempt to guess a canonical system name.
148 # Copyright 1992-2018 Free Software Foundation, Inc.
149
150 timestamp='2018-03-08it and/or modify it
151 # under the terms of the GNU General Public License as published by
152 # t;e Free Software Foundation, either version 3 of the L;cense, or
153 # (at your o.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
154 # Attempt to guess a canonical system name.
155 # Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
156 # under the terms of the GNU General Public License as published by
157 # t;e Free Software Foundation, either version 3 of the L;cense, or
158 # (at your option) any later version.
159 #
160 # This program is distributed in the hope that it will be useful, but
161 # WITHOUT ANY WARRANTY; without even the implied warranty of
162 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
163 # General Public License for more details.
164 #
165 # You should have received a copy of the GNU General Public License
166 # along with this program; if not, see <https://www.gnu.org/licenses/>.
167 #
168 # As a special exceptionto the GNU General Public License, if you
169 # distribute this file as part of a program that contains a
170 # configuration script generated by Autoconf, you may include it under
171 # the same distribution terms that you use for the rest of that
172 # program. This Exception is an additional permission under section 7
173 # of the GNU General Public License, version 3 ("GPLv3").
174 #
175 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
176 #
177 # You can get the latest version of this script from:
178 # https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
179
180 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
181 cat <<-EOF > $dummy.c`$CC_FOR_BUILD -E $dummy.c'^LIBC' | sed 's, ,,g'`{UNAME_MACHINE}:${UNAME_SYSTEM}:${UNAME_RELEASE}:${UNAME_VERSION}/sbin/$sysctl/usr/sbin/$sysctl
182 # Attempt to guess a canoniAttempt to g! /bin/sh
183 # Attemp#*) machine=${! /bin/sh
184 # Attemp#! /bin/sh
185 # Attempt to guess a canonical system name.
186 # Copyright 1992-2018canonical system name.
187 # Copyrig! /bin/sh
188 # Attemp#! /b{UNAME_VERSION}${UNAME_REL{machine}-${os}${release}${! /bin/sh
189 # Attemp#! /bin/sh
190 # Attempt to guess a ${! /bin/sh
191 # Attemp#! /bin/sh
192 # Attempt to guess a cekkoBSD:*:*)
193 echo ${UNAME_MACHINE}-unknown-ekkobsd${UNAME_RELEASE}SolidBSD:*:*)
194 echo ${UNAME_MACHINE}-unknown-solidbsd${UNAME_RELEASE}${UNAME_RELEASE}${UNAME_MACHINE}-unk"alpha""alpha""alpha""alphaev5"A"alphaev56""alphapca56""alphapca57""alphaev6""alphaev67""alphaev68""alphaev68""alphaev68""alphaev69""alphaev7""alphaev79"${UNAME_MACHINE}-dec-osf`echo RELEASE}'' ''` /bin/sh
195 # Attempt to guess a canonical system name.
196 # Copyright 1992-2018 Free Software FouAttempt to guess a canonical system name.
197 # Copyright 1992-2018 Free Software Foundation, Inc.
198
199 timestamp='2018-03-08it and/or modify it
200 # under the terms of the GNU General Public License as published by
201 # t;e Free Software Foundation, either version 3 of the L;cense, or
202 # (at your oished by
203 # t;e Free Software Foundation, either version 3 of the L;cense, or
204 # (at your option) any later version.
205 #
206 # This program is distributed in the hope that it will be useful, but
207 # WITHOUT ANY WARRANTY; without even the implied warranty of
208 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
209 # General Public License for more details.
210 #
211 # You should have received a copy of the GNU General Public License
212 # along with this progra(C) 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000,
213 2001, 2002, 2003, 2004, 2005, 2006, 2implied warranty ofot, see <https://www.gnu.org/licenses/>.
214 #
215 # As a special exceptionto the GNU General Public License, if you
216 # distribute this file as part of a program that contains a
217 # configuration script generated by Autoconf, you may include it under
218 # the same distribution terms that you use for the rest of that
219 # program. This Exception is an additional permission under section 7
220 # of the GNU General Public License, version 3 ("GPLv3").
221 #
222 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
223 #
224 # You can get the latest version of this script from:
225 # https://git.savannah.gnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
226
227 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
228 cat <<-EOF > $dummy.c`$CC_FORHUP INT TERMneral Public License, version 3 ("GPLv3").
229 #
230 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
231 #
232 # You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
233
234 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
235 cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
236 # Attempt to guess a canonical system name.
237 # Copyright 1992-2014 Free Software Found4-11-04it and/or modify it
238 # under the terms of the GNU General Public License as published by
239 # t;e Free Software Foundation, either version 3 of the L;cense, or
240 # (at your optionHUP INT PIPE TERM03-23 the terms of the GNU General Public License as published by
241 # t;e Free Software Foundation, either version 3 of the L;cense, or
242 # (at your option) any later version.
243 #
244 # This program is distributed in the hope tha03-23 the terms of the GNU General Public License as published by
245 # t;e Free Software Foundation, either version 3 of the L;cense, or
246 # (at your option) any later version.
247 #
248 # This program is distributed in the hope that it will be useful, but
249 # WITHOUT ANY WARRANTY; without even the implied warranty of
250 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
251 # General Public License for more details.
252 #
253 # You should have received a copy of the GNU General Public License
254 # along with this program; if not, see <https://www.gnu.org/licenses/>.
255 #
256 # As a special exceptionto the GNU General Public License, if you
257 # distribute this file as part of a program that contains a
258 # configuration script generated by Autoconf, you may include it under
259 # the same distribution terms that you use for the rest of that
260 # program. This Exception is an additional permission under section 7
261 # of the GNU General Public License, version 3 ("GPLv3").
262 #
263 # Originally written by Per Bothner; maintained since 2000 by Ben Elliston.
264 #
265 # You can ggnu.org/guess;hb=HEADeration mode4 Free Software Foundation, Inc.
266
267 $dummy.c$dummy.o $dummy.c{UNAME_SYSTEM}
268 cat <<-EOF > $dummy.c`$CC_FOR_BUILwith a ChangeLog entry to to guess a canonical s#! /bin/sh
269 # Attempt to guess a canonical system name.
270 # Open exit :riscos:*:*|arm t it will be useful, but
271 03-23 th 23 the terms of the GNU General Public License as published by
272 # t;e Free Software Foundation, either version 3 of the L;cense, or
273 # (at your option) any later version.
274 #
275 # This program is distributed in the hope that it will be useful, but
276 # WITHOUT ANY WARRANTY; without even the implied warranty of
277 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
278 # General Public License for more details.
279 #
280 # You should have received a copy of the GNU General Public License
281 # along with this program; if not, see <https://www.gnu.org/licenses/>.
282 #
283 # As a special exceptionto the GNU General Public License, if you
284 # distribute this file as part of a program that contains a
285 # configuration script generated by Autoconf, you may include it under
286 # the same distribution terms that you use for the rest of that
287 # program. This Exception is an additional permission under section 7
288 # of the GNU General Public License, version 3 ("GPLv3").
289 #
290 # Originally written ms of the GNU Genera532)case "${sc_kernel_bits}" in
291 32) HP_ARCH="hppa2.0n" ;;
292 esac ;;
293
294 #include <stdlib.h>
295 #include <unistd.h>
296
297 int main ()
298 {
299 #if
300 #endif
301 switch (cpu)
302 "); break;
303 2_0:
304 #if
305 switch (bits)
306 {
307
M autosetup/config.sub
+1
--- a/autosetup/config.sub
+++ b/autosetup/config.sub
@@ -0,0 +1 @@
1
+#! /bin/ncr3#! /bin/
--- a/autosetup/config.sub
+++ b/autosetup/config.sub
@@ -0,0 +1 @@
 
--- a/autosetup/config.sub
+++ b/autosetup/config.sub
@@ -0,0 +1 @@
1 #! /bin/ncr3#! /bin/
M autosetup/find-tclsh
+9
--- a/autosetup/find-tclsh
+++ b/autosetup/find-tclsh
@@ -0,0 +1,9 @@
1
+#!/bin/sh
2
+# Looks for a suitable tclsh or jimsh in the PATH
3
+# If not foundstrap jimsh d=`dirname "$0"`
4
+{ "$d/jimshd/${1-ac"; } 2>&imsh0"
5
+for cc in tclsh8.7; do
6
+ jimsh0-tclsh}"; } 2>/devexit 0
7
+done
8
+echo 1>&2 "No installed jimsh or tclsh, building local bootstrap jimsh0"
9
+for cc in ${CC_FOR_BUILD:-PATH="$PATH:$d2>/dev/null
--- a/autosetup/find-tclsh
+++ b/autosetup/find-tclsh
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
--- a/autosetup/find-tclsh
+++ b/autosetup/find-tclsh
@@ -0,0 +1,9 @@
1 #!/bin/sh
2 # Looks for a suitable tclsh or jimsh in the PATH
3 # If not foundstrap jimsh d=`dirname "$0"`
4 { "$d/jimshd/${1-ac"; } 2>&imsh0"
5 for cc in tclsh8.7; do
6 jimsh0-tclsh}"; } 2>/devexit 0
7 done
8 echo 1>&2 "No installed jimsh or tclsh, building local bootstrap jimsh0"
9 for cc in ${CC_FOR_BUILD:-PATH="$PATH:$d2>/dev/null
M autosetup/jimsh0.c
+144
--- a/autosetup/jimsh0.c
+++ b/autosetup/jimsh0.c
@@ -0,0 +1,144 @@
1
+3cl. See http://jim.berlios.de/ */
2
+#define _GNU_SOURCEREFERENCES single source file,/* This is single source file, bootstrap version of Jim Tcl. See http://jim.tcl.tk/ */
3
+#define JIM_TCL_COMPAT_H
4
+#define TCL_LIBRARY "."
5
+#define jim_ext_stdlib
6
+#define jimregexpxec
7
+#define jim_efilexec
8
+#define jim_eexecxec
9
+#define jim_eclockxec
10
+#define jim_ee TCL_PLATFORM_OS "windows"
11
+#define TCL_PLATFORM_PLATFORM "windows"
12
+#define TCL_PLATFORM_PATH_SEPARATOR ";"
13
+#define HAVE_MKDIR_ONE_ARG
14
+#define HAVE_SYSTEM
15
+#elif defined(__MINGW32__)
16
+#defi HAVE_DIRENT_H
17
+#define HAVE_UNISTD_H
18
+#define HAVE_UMASK
19
+#include elseSEPARATOR ":"
20
+#ifdef _ATFORM_PLATFORM "unix"
21
+#define TCL_PLATFORM_PATH_SEPARMKSTEMP
22
+#define HAVE_LINK
23
+#define HAVE_SYS_TIME_H
24
+#define HAVE_DITFORM "unix"
25
+#define TCL_PLATFndif
26
+#define JIM_VERS3cl. See http://jim.berlios.de/ */
27
+#define _GNU_SOURCEREFERENCES single source file,/* /**
28
+ * UTF-8 utility functions
29
+ *
30
+ * (c) 2010 Steve Bennett <[email protected]>
31
+ *
32
+ * See LICENCE for licence details.
33
+ */
34
+
35
+/**
36
+ * Converts the given unicode codepoint (0 - 0xffff) to utf-8
37
+ * and stores the result at 'p'.
38
+ *
39
+ * Returns the number of utf-8 characters (1-3).
40
+ */
41
+intJim Tcl. See http://jim.tcl.tk/ */
42
+#define JIM_TCL_COMPAT_H
43
+#define/* No utf-8 support. 1 byte = 1 char */
44
+#define utf8_strlen(S, B) TCL_LIBRARY "."
45
+#define j
46
+#defineS, CP) (*(CP) = #define jim_eexecxec
47
+#define jim_eclockxec
48
+#define jim_ee TCL_PLATFORM_OS "windows"
49
+#define TCL_PLATFORM_PLATFORM "windows"
50
+#define TCL_PLAT/**
51
+ * Returns the length of the utf-8 sequence starting with 'c'.
52
+ *
53
+ * Returns 1-4, or -1 if this is not a valid start byte.
54
+ *
55
+ * Note that charlen=4 is not supported by the rest of the API.
56
+ */
57
+int utf8_charlen(int c);
58
+
59
+/**
60
+ * Returns the number of characters in the utf-8
61
+ * string of the given byte length.
62
+ *
63
+ * Any bytes which are not part of an valid utf-8
64
+ * sequence are treated as individual characters.
65
+ *
66
+ * The string *must* be null terminated.
67
+ *
68
+ * Does not support unicode code points > \uffff
69
+ */
70
+int utf8_strlen(bytelen);
71
+
72
+/**
73
+ * Returns the byte index of the given character in the utf-8 string.
74
+ *
75
+ * The string *must* be null terminated.
76
+ *
77
+ * This will return the byte length of a utf-8 string
78
+ * if given the char charindex);
79
+
80
+/**
81
+ * Returns the unicode codepoint corresponding to the
82
+ * utf-8 sequence 'str'.
83
+ *
84
+ * Stores the result in *uc and returns the number of bytes
85
+ * consumed.
86
+ *
87
+ * If 'str' is null terminated, then an invalid utf-8 sequence
88
+ * at the end of the string will be returned as individual bytes.
89
+ *
90
+ * If it is not null terminated, the length *must* be checked first.
91
+ *
92
+ * Does not support unicode c*uc);
93
+
94
+/**
95
+ * Returns the number of bytes before 'str' that the previous
96
+ * utf-8 character sequence starts (which may be the middle of a sequence).
97
+ *
98
+ * Looks back at most 'len' bytes backwards, which must be > 0.
99
+ * If no start char is found, returns -len
100
+ */
101
+int utf8_prev_len(
102
+/**
103
+ * Returns the upper-case variant of the given unicode codepoint.
104
+ *
105
+ * Does not support unicode code points > \uffff
106
+ */
107
+int utf8_upper(int uc);
108
+
109
+/**
110
+ * Returns the lower-case variant of the given unicode codepoint.
111
+ *
112
+ * NOTE: Use utf8_upper() in preference for case-insensitive matching.
113
+ *
114
+ * Does not support unicode code points > \uffff
115
+ */
116
+int utf8_lower(int uc);
117
+
118
+#endif
119
+
120
+#endif
121
+/* Jim - A small embeddable Tcl interpreter
122
+ *
123
+ * Copyright 2005 Salvatore Sanfilippo <[email protected]>
124
+ * Copyright 2005 Clemens Hintze <[email protected]>
125
+ * Copyright 2005 patthoyts - Pat Thoyts <[email protected]>
126
+ * Copyright 2008 oharboe - �yvind Harboe - [email protected]
127
+ * Copyright 2008 Andrew Lunn <[email protected]>
128
+ * Copyright 2008 Duane Ellis <[email protected]>
129
+ * Copyright 2008 Uwe Klein <[email protected]>
130
+ *
131
+ * Redistribution and use in source and binary forms, with or without
132
+ * modification, are permitted provided that the following conditions
133
+ * are met:
134
+ *
135
+ * 1. Redistributions of source code must retain the above copyright
136
+ * notice, this list of conditions and the following disclaimer.
137
+ * 2. Redistributions in binary form must reproduce the above
138
+ * copyright notice, this list of conditions and the following
139
+ * disclaimer in the documentation and/or other materials
140
+ * provided with the distribution.
141
+ *
142
+ * THIS SOFTWARE IS PROVIDED BY THE JIM TCL PROJECT ``AS IS'' AND ANY
143
+ * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
144
+ * THE IMPLIED WARRANTIES OF MERCHANTABILI
--- a/autosetup/jimsh0.c
+++ b/autosetup/jimsh0.c
@@ -0,0 +1,144 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/jimsh0.c
+++ b/autosetup/jimsh0.c
@@ -0,0 +1,144 @@
1 3cl. See http://jim.berlios.de/ */
2 #define _GNU_SOURCEREFERENCES single source file,/* This is single source file, bootstrap version of Jim Tcl. See http://jim.tcl.tk/ */
3 #define JIM_TCL_COMPAT_H
4 #define TCL_LIBRARY "."
5 #define jim_ext_stdlib
6 #define jimregexpxec
7 #define jim_efilexec
8 #define jim_eexecxec
9 #define jim_eclockxec
10 #define jim_ee TCL_PLATFORM_OS "windows"
11 #define TCL_PLATFORM_PLATFORM "windows"
12 #define TCL_PLATFORM_PATH_SEPARATOR ";"
13 #define HAVE_MKDIR_ONE_ARG
14 #define HAVE_SYSTEM
15 #elif defined(__MINGW32__)
16 #defi HAVE_DIRENT_H
17 #define HAVE_UNISTD_H
18 #define HAVE_UMASK
19 #include elseSEPARATOR ":"
20 #ifdef _ATFORM_PLATFORM "unix"
21 #define TCL_PLATFORM_PATH_SEPARMKSTEMP
22 #define HAVE_LINK
23 #define HAVE_SYS_TIME_H
24 #define HAVE_DITFORM "unix"
25 #define TCL_PLATFndif
26 #define JIM_VERS3cl. See http://jim.berlios.de/ */
27 #define _GNU_SOURCEREFERENCES single source file,/* /**
28 * UTF-8 utility functions
29 *
30 * (c) 2010 Steve Bennett <[email protected]>
31 *
32 * See LICENCE for licence details.
33 */
34
35 /**
36 * Converts the given unicode codepoint (0 - 0xffff) to utf-8
37 * and stores the result at 'p'.
38 *
39 * Returns the number of utf-8 characters (1-3).
40 */
41 intJim Tcl. See http://jim.tcl.tk/ */
42 #define JIM_TCL_COMPAT_H
43 #define/* No utf-8 support. 1 byte = 1 char */
44 #define utf8_strlen(S, B) TCL_LIBRARY "."
45 #define j
46 #defineS, CP) (*(CP) = #define jim_eexecxec
47 #define jim_eclockxec
48 #define jim_ee TCL_PLATFORM_OS "windows"
49 #define TCL_PLATFORM_PLATFORM "windows"
50 #define TCL_PLAT/**
51 * Returns the length of the utf-8 sequence starting with 'c'.
52 *
53 * Returns 1-4, or -1 if this is not a valid start byte.
54 *
55 * Note that charlen=4 is not supported by the rest of the API.
56 */
57 int utf8_charlen(int c);
58
59 /**
60 * Returns the number of characters in the utf-8
61 * string of the given byte length.
62 *
63 * Any bytes which are not part of an valid utf-8
64 * sequence are treated as individual characters.
65 *
66 * The string *must* be null terminated.
67 *
68 * Does not support unicode code points > \uffff
69 */
70 int utf8_strlen(bytelen);
71
72 /**
73 * Returns the byte index of the given character in the utf-8 string.
74 *
75 * The string *must* be null terminated.
76 *
77 * This will return the byte length of a utf-8 string
78 * if given the char charindex);
79
80 /**
81 * Returns the unicode codepoint corresponding to the
82 * utf-8 sequence 'str'.
83 *
84 * Stores the result in *uc and returns the number of bytes
85 * consumed.
86 *
87 * If 'str' is null terminated, then an invalid utf-8 sequence
88 * at the end of the string will be returned as individual bytes.
89 *
90 * If it is not null terminated, the length *must* be checked first.
91 *
92 * Does not support unicode c*uc);
93
94 /**
95 * Returns the number of bytes before 'str' that the previous
96 * utf-8 character sequence starts (which may be the middle of a sequence).
97 *
98 * Looks back at most 'len' bytes backwards, which must be > 0.
99 * If no start char is found, returns -len
100 */
101 int utf8_prev_len(
102 /**
103 * Returns the upper-case variant of the given unicode codepoint.
104 *
105 * Does not support unicode code points > \uffff
106 */
107 int utf8_upper(int uc);
108
109 /**
110 * Returns the lower-case variant of the given unicode codepoint.
111 *
112 * NOTE: Use utf8_upper() in preference for case-insensitive matching.
113 *
114 * Does not support unicode code points > \uffff
115 */
116 int utf8_lower(int uc);
117
118 #endif
119
120 #endif
121 /* Jim - A small embeddable Tcl interpreter
122 *
123 * Copyright 2005 Salvatore Sanfilippo <[email protected]>
124 * Copyright 2005 Clemens Hintze <[email protected]>
125 * Copyright 2005 patthoyts - Pat Thoyts <[email protected]>
126 * Copyright 2008 oharboe - �yvind Harboe - [email protected]
127 * Copyright 2008 Andrew Lunn <[email protected]>
128 * Copyright 2008 Duane Ellis <[email protected]>
129 * Copyright 2008 Uwe Klein <[email protected]>
130 *
131 * Redistribution and use in source and binary forms, with or without
132 * modification, are permitted provided that the following conditions
133 * are met:
134 *
135 * 1. Redistributions of source code must retain the above copyright
136 * notice, this list of conditions and the following disclaimer.
137 * 2. Redistributions in binary form must reproduce the above
138 * copyright notice, this list of conditions and the following
139 * disclaimer in the documentation and/or other materials
140 * provided with the distribution.
141 *
142 * THIS SOFTWARE IS PROVIDED BY THE JIM TCL PROJECT ``AS IS'' AND ANY
143 * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
144 * THE IMPLIED WARRANTIES OF MERCHANTABILI
M autosetup/local.tcl
+2
--- a/autosetup/local.tcl
+++ b/autosetup/local.tcl
@@ -0,0 +1,2 @@
1
+# For this project, d
2
+set useropts(nopager)tosetup(optdefault) nopager 1
--- a/autosetup/local.tcl
+++ b/autosetup/local.tcl
@@ -0,0 +1,2 @@
 
 
--- a/autosetup/local.tcl
+++ b/autosetup/local.tcl
@@ -0,0 +1,2 @@
1 # For this project, d
2 set useropts(nopager)tosetup(optdefault) nopager 1
M autosetup/system.tcl
+130
--- a/autosetup/system.tcl
+++ b/autosetup/system.tcl
@@ -0,0 +1,130 @@
1
+# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
2
+# All rights reserved
3
+
4
+# @synopsis:
5
+#
6
+# This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
7
+#
8
+# It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
9
+#
10
+# It defines the fol--prefix unless overridden by th
11
+# Note that the hidden unless 'options-defaults { prefix compatibility
12
+ module-optionsation and options
13
+# such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
14
+#
15
+# It also support the "feature" naming convention, where searching
16
+# for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
17
+#
18
+## datadir
19
+## sysconfdir
20
+## sharedstatedir
21
+## localstatedir
22
+## infodir
23
+## mandir
24
+## includedir
25
+#
26
+# If '--pre
27
+# defines featuresysconf {
28
+ msg-checking "Che 2010 WorkWare Systems http://www.workware.net.au/
29
+# All rights reserved
30
+
31
+# @synopsis:
32
+#
33
+# This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
34
+#
35
+# It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
36
+#
37
+# It defines the fol--prefix unless overridden by the module-optionsation and options
38
+# such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
39
+#
40
+# It also support the "feature" naming convention, where searching
41
+# for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
42
+#
43
+## datadir
44
+## sysconfdir
45
+## sharedstatedir
46
+## localstatedir
47
+## infodir
48
+## mandir
49
+## includedir
50
+#
51
+# If '--prefix' is not /usr/localaults to '/usr/local' unless 'options-defaults { prefix ... }'prefix ... }' #
52
+ program-transform-name:
53
+}
54
+
55
+# @check-feature name { script }
56
+#
57
+# defines feature '$nReturns 1 if exists, or 0 if not
58
+#ode} {
59
+ msg-checking "Checking for $name..."
60
+ set r [uplevel 1 $code
61
+ define-feature $name $r
62
+ if {$r} {
63
+ msg-result "ok"
64
+ } else {
65
+ msg-result "not found"
66
+ }
67
+ return $r
68
+}
69
+
70
+# @have-feature name ?default=0?
71
+#
72
+# Returns the value of feature '$name' if defined,the feature if de
73
+# is translated thehave-feature {name {defathe 0}. Use @top_sr# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
74
+# All rights reserved
75
+
76
+# @synopsis:
77
+#
78
+# This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
79
+#
80
+# It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
81
+#
82
+# It defines the fol--prefix unless overridden by the module-optionsation and options
83
+# such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
84
+#
85
+# It also support the "feature" naming convention, where searching
86
+# for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
87
+#
88
+## datadir
89
+## sysconfdir
90
+## sharedstatedir
91
+## localstatedir
92
+## infodir
93
+## mandir
94
+## includedir
95
+#
96
+# If '--prefix' is not /usr/localaults to '/usr/local' unless 'options-defaults { prefix ... }'prefix ... }' #
97
+ program-transform-name:
98
+}
99
+
100
+# @check-feature set prefixdefine[get-env exec-prefix \${prefix}]]\${exec_prefix}/bi# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
101
+# All rights reserved
102
+
103
+# @synopsis:
104
+#
105
+# This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
106
+#
107
+# It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
108
+#
109
+# It defines the fol--prefix unless overridden by the module-optionsation and options
110
+# such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
111
+#
112
+# It also support the "feature" naming convention, where searching
113
+# for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
114
+#
115
+## datadir
116
+## sysconfdir
117
+## sharedstatedir
118
+## localstatedir
119
+## infodir
120
+## mandir
121
+## includreadfile $infileAnd less common ones too
122
+, 'buil \${prefix}
123
+define define define libexecdir [get-env ]
124
+define datadir \${prefix}/share
125
+]
126
+define define localstatedir [get-env localvar]
127
+define define infodir [get-env s:
128
+#
129
+# This module supports c# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
130
+# All rights rese
--- a/autosetup/system.tcl
+++ b/autosetup/system.tcl
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/system.tcl
+++ b/autosetup/system.tcl
@@ -0,0 +1,130 @@
1 # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
2 # All rights reserved
3
4 # @synopsis:
5 #
6 # This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
7 #
8 # It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
9 #
10 # It defines the fol--prefix unless overridden by th
11 # Note that the hidden unless 'options-defaults { prefix compatibility
12 module-optionsation and options
13 # such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
14 #
15 # It also support the "feature" naming convention, where searching
16 # for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
17 #
18 ## datadir
19 ## sysconfdir
20 ## sharedstatedir
21 ## localstatedir
22 ## infodir
23 ## mandir
24 ## includedir
25 #
26 # If '--pre
27 # defines featuresysconf {
28 msg-checking "Che 2010 WorkWare Systems http://www.workware.net.au/
29 # All rights reserved
30
31 # @synopsis:
32 #
33 # This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
34 #
35 # It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
36 #
37 # It defines the fol--prefix unless overridden by the module-optionsation and options
38 # such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
39 #
40 # It also support the "feature" naming convention, where searching
41 # for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
42 #
43 ## datadir
44 ## sysconfdir
45 ## sharedstatedir
46 ## localstatedir
47 ## infodir
48 ## mandir
49 ## includedir
50 #
51 # If '--prefix' is not /usr/localaults to '/usr/local' unless 'options-defaults { prefix ... }'prefix ... }' #
52 program-transform-name:
53 }
54
55 # @check-feature name { script }
56 #
57 # defines feature '$nReturns 1 if exists, or 0 if not
58 #ode} {
59 msg-checking "Checking for $name..."
60 set r [uplevel 1 $code
61 define-feature $name $r
62 if {$r} {
63 msg-result "ok"
64 } else {
65 msg-result "not found"
66 }
67 return $r
68 }
69
70 # @have-feature name ?default=0?
71 #
72 # Returns the value of feature '$name' if defined,the feature if de
73 # is translated thehave-feature {name {defathe 0}. Use @top_sr# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
74 # All rights reserved
75
76 # @synopsis:
77 #
78 # This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
79 #
80 # It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
81 #
82 # It defines the fol--prefix unless overridden by the module-optionsation and options
83 # such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
84 #
85 # It also support the "feature" naming convention, where searching
86 # for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
87 #
88 ## datadir
89 ## sysconfdir
90 ## sharedstatedir
91 ## localstatedir
92 ## infodir
93 ## mandir
94 ## includedir
95 #
96 # If '--prefix' is not /usr/localaults to '/usr/local' unless 'options-defaults { prefix ... }'prefix ... }' #
97 program-transform-name:
98 }
99
100 # @check-feature set prefixdefine[get-env exec-prefix \${prefix}]]\${exec_prefix}/bi# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
101 # All rights reserved
102
103 # @synopsis:
104 #
105 # This module supports common system interroga--host, --build, --prefix, and setting srcdir, builddir, and EXEXT.
106 #
107 # It naming convention, where searchisys/type.h defines HAVE_SYS_TYPES_H
108 #
109 # It defines the fol--prefix unless overridden by the module-optionsation and options
110 # such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'.
111 #
112 # It also support the "feature" naming convention, where searching
113 # for a feature such as 'sys/type => ines the followins to '$defaultprefix overridden by the user:
114 #
115 ## datadir
116 ## sysconfdir
117 ## sharedstatedir
118 ## localstatedir
119 ## infodir
120 ## mandir
121 ## includreadfile $infileAnd less common ones too
122 , 'buil \${prefix}
123 define define define libexecdir [get-env ]
124 define datadir \${prefix}/share
125 ]
126 define define localstatedir [get-env localvar]
127 define define infodir [get-env s:
128 #
129 # This module supports c# Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/
130 # All rights rese
M autosetup/test-tclsh
+21
--- a/autosetup/test-tclsh
+++ b/autosetup/test-tclsh
@@ -0,0 +1,21 @@
1
+# A small Tcl script to verify that the chosen
2
+# interpreter works. Sometimes we might e.g. pick up
3
+# an interpreter for a different arch.
4
+# Outputs the full path to the interpreter
5
+
6
+if {[catch {info version} version] == 0} {
7
+ # This is Jim Tcl
8
+ if {$version >= 0.70} {
9
+ # Ensure that rege
10
+ # Older versions of jimsh mayinfo patchlevel]]} {
11
+ puts [info nameofexecutable]
12
+ exit 0
13
+ }
14
+}
15
+exit 1
16
+puts [file join [p]
17
+ exit 0
18
+ }
19
+} elseif {[catch {info tclversion} version] == 0} {
20
+ if {$version >= 8.5 && ![string match 8.5a* [info patchlevel]]} {
21
+ puts [info nameofexecut
--- a/autosetup/test-tclsh
+++ b/autosetup/test-tclsh
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/autosetup/test-tclsh
+++ b/autosetup/test-tclsh
@@ -0,0 +1,21 @@
1 # A small Tcl script to verify that the chosen
2 # interpreter works. Sometimes we might e.g. pick up
3 # an interpreter for a different arch.
4 # Outputs the full path to the interpreter
5
6 if {[catch {info version} version] == 0} {
7 # This is Jim Tcl
8 if {$version >= 0.70} {
9 # Ensure that rege
10 # Older versions of jimsh mayinfo patchlevel]]} {
11 puts [info nameofexecutable]
12 exit 0
13 }
14 }
15 exit 1
16 puts [file join [p]
17 exit 0
18 }
19 } elseif {[catch {info tclversion} version] == 0} {
20 if {$version >= 8.5 && ![string match 8.5a* [info patchlevel]]} {
21 puts [info nameofexecut
M configure
+3
--- a/configure
+++ b/configure
@@ -0,0 +1,3 @@
1
+#!/bin/sh
2
+dir=autosetup"
3
+#st -f `"ir/auhen exec" || echo false`
--- a/configure
+++ b/configure
@@ -0,0 +1,3 @@
 
 
 
--- a/configure
+++ b/configure
@@ -0,0 +1,3 @@
1 #!/bin/sh
2 dir=autosetup"
3 #st -f `"ir/auhen exec" || echo false`
M src/add.c
+74 -21
--- src/add.c
+++ src/add.c
@@ -92,66 +92,76 @@
9292
 **
9393
 ** Omit any file whose name is pOmit.
9494
 */
9595
 static int add_one_file(
9696
 const char *zPath, /* Tree-name of file to add. */
97
- int vid /* Add to this VFILE */
97
+ int vid, /* Add to this VFILE */
98
+ int caseSensitive /* True if filenames are case sensitive */
9899
 ){
100
+ const char *zCollate = caseSensitive ? "binary" : "nocase";
99101
 if( !file_is_simple_pathname(zPath) ){
100102
 fossil_fatal("filename contains illegal characters: %s", zPath);
101103
 }
102
-#if defined(_WIN32)
103104
 if( db_exists("SELECT 1 FROM vfile"
104
- " WHERE pathname=%Q COLLATE nocase", zPath) ){
105
+ " WHERE pathname=%Q COLLATE %s", zPath, zCollate) ){
105106
 db_multi_exec("UPDATE vfile SET deleted=0"
106
- " WHERE pathname=%Q COLLATE nocase", zPath);
107
- }
108
-#else
109
- if( db_exists("SELECT 1 FROM vfile WHERE pathname=%Q", zPath) ){
110
- db_multi_exec("UPDATE vfile SET deleted=0 WHERE pathname=%Q", zPath);
111
- }
112
-#endif
113
- else{
107
+ " WHERE pathname=%Q COLLATE %s", zPath, zCollate);
108
+ }else{
114109
 char *zFullname = mprintf("%s%s", g.zLocalRoot, zPath);
115110
 db_multi_exec(
116111
 "INSERT INTO vfile(vid,deleted,rid,mrid,pathname,isexe)"
117112
 "VALUES(%d,0,0,0,%Q,%d)",
118113
 vid, zPath, file_isexe(zFullname));
119114
 fossil_free(zFullname);
120115
 }
121
- fossil_print("ADDED %s\n", zPath);
122
- return 1;
116
+ if( db_changes() ){
117
+ fossil_print("ADDED %s\n", zPath);
118
+ return 1;
119
+ }else{
120
+ fossil_print("SKIP %s\n", zPath);
121
+ return 0;
122
+ }
123123
 }
124124
125125
 /*
126126
 ** Add all files in the sfile temp table.
127127
 **
128128
 ** Automatically exclude the repository file.
129129
 */
130
-static int add_files_in_sfile(int vid){
130
+static int add_files_in_sfile(int vid, int caseSensitive){
131131
 const char *zRepo; /* Name of the repository database file */
132132
 int nAdd = 0; /* Number of files added */
133133
 int i; /* Loop counter */
134134
 const char *zReserved; /* Name of a reserved file */
135135
 Blob repoName; /* Treename of the repository */
136136
 Stmt loop; /* SQL to loop over all files to add */
137
+ int (*xCmp)(const char*,const char*);
137138
138139
 if( !file_tree_name(g.zRepositoryName, &repoName, 0) ){
139140
 blob_zero(&repoName);
140141
 zRepo = "";
141142
 }else{
142143
 zRepo = blob_str(&repoName);
144
+ }
145
+ if( caseSensitive ){
146
+ xCmp = fossil_strcmp;
147
+ }else{
148
+ xCmp = fossil_stricmp;
149
+ db_multi_exec(
150
+ "CREATE INDEX IF NOT EXISTS vfile_nocase"
151
+ " ON vfile(pathname COLLATE nocase)"
152
+ );
143153
 }
144154
 db_prepare(&loop, "SELECT x FROM sfile ORDER BY x");
145155
 while( db_step(&loop)==SQLITE_ROW ){
146156
 const char *zToAdd = db_column_text(&loop, 0);
147157
 if( fossil_strcmp(zToAdd, zRepo)==0 ) continue;
148158
 for(i=0; (zReserved = fossil_reserved_name(i))!=0; i++){
149
- if( fossil_strcmp(zToAdd, zReserved)==0 ) break;
159
+ if( xCmp(zToAdd, zReserved)==0 ) break;
150160
 }
151161
 if( zReserved ) continue;
152
- nAdd += add_one_file(zToAdd, vid);
162
+ nAdd += add_one_file(zToAdd, vid, caseSensitive);
153163
 }
154164
 db_finalize(&loop);
155165
 blob_reset(&repoName);
156166
 return nAdd;
157167
 }
@@ -180,14 +190,17 @@
180190
 int i; /* Loop counter */
181191
 int vid; /* Currently checked out version */
182192
 int nRoot; /* Full path characters in g.zLocalRoot */
183193
 const char *zIgnoreFlag; /* The --ignore option or ignore-glob setting */
184194
 Glob *pIgnore; /* Ignore everything matching this glob pattern */
195
+ int caseSensitive; /* True if filenames are case sensitive */
185196
186197
 zIgnoreFlag = find_option("ignore",0,1);
187198
 includeDotFiles = find_option("dotfiles",0,0)!=0;
199
+ capture_case_sensitive_option();
188200
 db_must_be_within_tree();
201
+ caseSensitive = filenames_are_case_sensitive();
189202
 if( zIgnoreFlag==0 ){
190203
 zIgnoreFlag = db_get("ignore-glob", 0);
191204
 }
192205
 vid = db_lget_int("checkout",0);
193206
 if( vid==0 ){
@@ -229,11 +242,11 @@
229242
 }
230243
 blob_reset(&fullName);
231244
 }
232245
 glob_free(pIgnore);
233246
234
- add_files_in_sfile(vid);
247
+ add_files_in_sfile(vid, caseSensitive);
235248
 db_end_transaction(0);
236249
 }
237250
238251
 /*
239252
 ** COMMAND: rm
@@ -289,10 +302,48 @@
289302
 "UPDATE vfile SET deleted=1 WHERE pathname IN sfile;"
290303
 "DELETE FROM vfile WHERE rid=0 AND deleted;"
291304
 );
292305
 db_end_transaction(0);
293306
 }
307
+
308
+/*
309
+** Capture the command-line --case-sensitive option.
310
+*/
311
+static const char *zCaseSensitive = 0;
312
+void capture_case_sensitive_option(void){
313
+ if( zCaseSensitive==0 ){
314
+ zCaseSensitive = find_option("case-sensitive",0,1);
315
+ }
316
+}
317
+
318
+/*
319
+** This routine determines if files should be case-sensitive or not.
320
+** In other words, this routine determines if two filenames that
321
+** differ only in case should be considered the same name or not.
322
+**
323
+** The case-sensitive setting determines the default value. If
324
+** the case-sensitive setting is undefined, then case sensitivity
325
+** defaults on for Mac and Windows and off for all other unix.
326
+**
327
+** The --case-sensitive BOOLEAN command-line option overrides any
328
+** setting.
329
+*/
330
+int filenames_are_case_sensitive(void){
331
+ int caseSensitive;
332
+
333
+ if( zCaseSensitive ){
334
+ caseSensitive = is_truth(zCaseSensitive);
335
+ }else{
336
+#if !defined(_WIN32) && !defined(__DARWIN__) && !defined(__APPLE__)
337
+ caseSensitive = 1;
338
+#else
339
+ caseSensitive = 0;
340
+#endif
341
+ caseSensitive = db_get_boolean("case-sensitive",caseSensitive);
342
+ }
343
+ return caseSensitive;
344
+}
294345
295346
 /*
296347
 ** COMMAND: addremove
297348
 **
298349
 ** Usage: %fossil addremove ?--dotfiles? ?--ignore GLOBPATTERN? ?--test?
@@ -319,27 +370,29 @@
319370
 **
320371
 ** The --test option shows what would happen without actually doing anything.
321372
 **
322373
 ** This command can be used to track third party software.
323374
 **
324
-**
325375
 ** SUMMARY: fossil addremove
326
-** Options: ?--dotfiles? ?--ignore GLOBPATTERN? ?--test?
376
+** Options: ?--dotfiles? ?--ignore GLOB? ?--test? ?--case-sensitive BOOL?
327377
 */
328
-void import_cmd(void){
378
+void addremove_cmd(void){
329379
 Blob path;
330380
 const char *zIgnoreFlag = find_option("ignore",0,1);
331381
 int allFlag = find_option("dotfiles",0,0)!=0;
332382
 int isTest = find_option("test",0,0)!=0;
383
+ int caseSensitive;
333384
 int n;
334385
 Stmt q;
335386
 int vid;
336387
 int nAdd = 0;
337388
 int nDelete = 0;
338389
 Glob *pIgnore;
339390
391
+ capture_case_sensitive_option();
340392
 db_must_be_within_tree();
393
+ caseSensitive = filenames_are_case_sensitive();
341394
 if( zIgnoreFlag==0 ){
342395
 zIgnoreFlag = db_get("ignore-glob", 0);
343396
 }
344397
 vid = db_lget_int("checkout",0);
345398
 if( vid==0 ){
@@ -358,11 +411,11 @@
358411
 blob_init(&path, g.zLocalRoot, n-1);
359412
 /* now we read the complete file structure into a temp table */
360413
 pIgnore = glob_create(zIgnoreFlag);
361414
 vfile_scan(&path, blob_size(&path), allFlag, pIgnore);
362415
 glob_free(pIgnore);
363
- nAdd = add_files_in_sfile(vid);
416
+ nAdd = add_files_in_sfile(vid, caseSensitive);
364417
365418
 /* step 2: search for missing files */
366419
 db_prepare(&q,
367420
 "SELECT pathname, %Q || pathname, deleted FROM vfile"
368421
 " WHERE NOT deleted"
369422
--- src/add.c
+++ src/add.c
@@ -92,66 +92,76 @@
92 **
93 ** Omit any file whose name is pOmit.
94 */
95 static int add_one_file(
96 const char *zPath, /* Tree-name of file to add. */
97 int vid /* Add to this VFILE */
 
98 ){
 
99 if( !file_is_simple_pathname(zPath) ){
100 fossil_fatal("filename contains illegal characters: %s", zPath);
101 }
102 #if defined(_WIN32)
103 if( db_exists("SELECT 1 FROM vfile"
104 " WHERE pathname=%Q COLLATE nocase", zPath) ){
105 db_multi_exec("UPDATE vfile SET deleted=0"
106 " WHERE pathname=%Q COLLATE nocase", zPath);
107 }
108 #else
109 if( db_exists("SELECT 1 FROM vfile WHERE pathname=%Q", zPath) ){
110 db_multi_exec("UPDATE vfile SET deleted=0 WHERE pathname=%Q", zPath);
111 }
112 #endif
113 else{
114 char *zFullname = mprintf("%s%s", g.zLocalRoot, zPath);
115 db_multi_exec(
116 "INSERT INTO vfile(vid,deleted,rid,mrid,pathname,isexe)"
117 "VALUES(%d,0,0,0,%Q,%d)",
118 vid, zPath, file_isexe(zFullname));
119 fossil_free(zFullname);
120 }
121 fossil_print("ADDED %s\n", zPath);
122 return 1;
 
 
 
 
 
123 }
124
125 /*
126 ** Add all files in the sfile temp table.
127 **
128 ** Automatically exclude the repository file.
129 */
130 static int add_files_in_sfile(int vid){
131 const char *zRepo; /* Name of the repository database file */
132 int nAdd = 0; /* Number of files added */
133 int i; /* Loop counter */
134 const char *zReserved; /* Name of a reserved file */
135 Blob repoName; /* Treename of the repository */
136 Stmt loop; /* SQL to loop over all files to add */
 
137
138 if( !file_tree_name(g.zRepositoryName, &repoName, 0) ){
139 blob_zero(&repoName);
140 zRepo = "";
141 }else{
142 zRepo = blob_str(&repoName);
 
 
 
 
 
 
 
 
 
143 }
144 db_prepare(&loop, "SELECT x FROM sfile ORDER BY x");
145 while( db_step(&loop)==SQLITE_ROW ){
146 const char *zToAdd = db_column_text(&loop, 0);
147 if( fossil_strcmp(zToAdd, zRepo)==0 ) continue;
148 for(i=0; (zReserved = fossil_reserved_name(i))!=0; i++){
149 if( fossil_strcmp(zToAdd, zReserved)==0 ) break;
150 }
151 if( zReserved ) continue;
152 nAdd += add_one_file(zToAdd, vid);
153 }
154 db_finalize(&loop);
155 blob_reset(&repoName);
156 return nAdd;
157 }
@@ -180,14 +190,17 @@
180 int i; /* Loop counter */
181 int vid; /* Currently checked out version */
182 int nRoot; /* Full path characters in g.zLocalRoot */
183 const char *zIgnoreFlag; /* The --ignore option or ignore-glob setting */
184 Glob *pIgnore; /* Ignore everything matching this glob pattern */
 
185
186 zIgnoreFlag = find_option("ignore",0,1);
187 includeDotFiles = find_option("dotfiles",0,0)!=0;
 
188 db_must_be_within_tree();
 
189 if( zIgnoreFlag==0 ){
190 zIgnoreFlag = db_get("ignore-glob", 0);
191 }
192 vid = db_lget_int("checkout",0);
193 if( vid==0 ){
@@ -229,11 +242,11 @@
229 }
230 blob_reset(&fullName);
231 }
232 glob_free(pIgnore);
233
234 add_files_in_sfile(vid);
235 db_end_transaction(0);
236 }
237
238 /*
239 ** COMMAND: rm
@@ -289,10 +302,48 @@
289 "UPDATE vfile SET deleted=1 WHERE pathname IN sfile;"
290 "DELETE FROM vfile WHERE rid=0 AND deleted;"
291 );
292 db_end_transaction(0);
293 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
294
295 /*
296 ** COMMAND: addremove
297 **
298 ** Usage: %fossil addremove ?--dotfiles? ?--ignore GLOBPATTERN? ?--test?
@@ -319,27 +370,29 @@
319 **
320 ** The --test option shows what would happen without actually doing anything.
321 **
322 ** This command can be used to track third party software.
323 **
324 **
325 ** SUMMARY: fossil addremove
326 ** Options: ?--dotfiles? ?--ignore GLOBPATTERN? ?--test?
327 */
328 void import_cmd(void){
329 Blob path;
330 const char *zIgnoreFlag = find_option("ignore",0,1);
331 int allFlag = find_option("dotfiles",0,0)!=0;
332 int isTest = find_option("test",0,0)!=0;
 
333 int n;
334 Stmt q;
335 int vid;
336 int nAdd = 0;
337 int nDelete = 0;
338 Glob *pIgnore;
339
 
340 db_must_be_within_tree();
 
341 if( zIgnoreFlag==0 ){
342 zIgnoreFlag = db_get("ignore-glob", 0);
343 }
344 vid = db_lget_int("checkout",0);
345 if( vid==0 ){
@@ -358,11 +411,11 @@
358 blob_init(&path, g.zLocalRoot, n-1);
359 /* now we read the complete file structure into a temp table */
360 pIgnore = glob_create(zIgnoreFlag);
361 vfile_scan(&path, blob_size(&path), allFlag, pIgnore);
362 glob_free(pIgnore);
363 nAdd = add_files_in_sfile(vid);
364
365 /* step 2: search for missing files */
366 db_prepare(&q,
367 "SELECT pathname, %Q || pathname, deleted FROM vfile"
368 " WHERE NOT deleted"
369
--- src/add.c
+++ src/add.c
@@ -92,66 +92,76 @@
92 **
93 ** Omit any file whose name is pOmit.
94 */
95 static int add_one_file(
96 const char *zPath, /* Tree-name of file to add. */
97 int vid, /* Add to this VFILE */
98 int caseSensitive /* True if filenames are case sensitive */
99 ){
100 const char *zCollate = caseSensitive ? "binary" : "nocase";
101 if( !file_is_simple_pathname(zPath) ){
102 fossil_fatal("filename contains illegal characters: %s", zPath);
103 }
 
104 if( db_exists("SELECT 1 FROM vfile"
105 " WHERE pathname=%Q COLLATE %s", zPath, zCollate) ){
106 db_multi_exec("UPDATE vfile SET deleted=0"
107 " WHERE pathname=%Q COLLATE %s", zPath, zCollate);
108 }else{
 
 
 
 
 
 
109 char *zFullname = mprintf("%s%s", g.zLocalRoot, zPath);
110 db_multi_exec(
111 "INSERT INTO vfile(vid,deleted,rid,mrid,pathname,isexe)"
112 "VALUES(%d,0,0,0,%Q,%d)",
113 vid, zPath, file_isexe(zFullname));
114 fossil_free(zFullname);
115 }
116 if( db_changes() ){
117 fossil_print("ADDED %s\n", zPath);
118 return 1;
119 }else{
120 fossil_print("SKIP %s\n", zPath);
121 return 0;
122 }
123 }
124
125 /*
126 ** Add all files in the sfile temp table.
127 **
128 ** Automatically exclude the repository file.
129 */
130 static int add_files_in_sfile(int vid, int caseSensitive){
131 const char *zRepo; /* Name of the repository database file */
132 int nAdd = 0; /* Number of files added */
133 int i; /* Loop counter */
134 const char *zReserved; /* Name of a reserved file */
135 Blob repoName; /* Treename of the repository */
136 Stmt loop; /* SQL to loop over all files to add */
137 int (*xCmp)(const char*,const char*);
138
139 if( !file_tree_name(g.zRepositoryName, &repoName, 0) ){
140 blob_zero(&repoName);
141 zRepo = "";
142 }else{
143 zRepo = blob_str(&repoName);
144 }
145 if( caseSensitive ){
146 xCmp = fossil_strcmp;
147 }else{
148 xCmp = fossil_stricmp;
149 db_multi_exec(
150 "CREATE INDEX IF NOT EXISTS vfile_nocase"
151 " ON vfile(pathname COLLATE nocase)"
152 );
153 }
154 db_prepare(&loop, "SELECT x FROM sfile ORDER BY x");
155 while( db_step(&loop)==SQLITE_ROW ){
156 const char *zToAdd = db_column_text(&loop, 0);
157 if( fossil_strcmp(zToAdd, zRepo)==0 ) continue;
158 for(i=0; (zReserved = fossil_reserved_name(i))!=0; i++){
159 if( xCmp(zToAdd, zReserved)==0 ) break;
160 }
161 if( zReserved ) continue;
162 nAdd += add_one_file(zToAdd, vid, caseSensitive);
163 }
164 db_finalize(&loop);
165 blob_reset(&repoName);
166 return nAdd;
167 }
@@ -180,14 +190,17 @@
190 int i; /* Loop counter */
191 int vid; /* Currently checked out version */
192 int nRoot; /* Full path characters in g.zLocalRoot */
193 const char *zIgnoreFlag; /* The --ignore option or ignore-glob setting */
194 Glob *pIgnore; /* Ignore everything matching this glob pattern */
195 int caseSensitive; /* True if filenames are case sensitive */
196
197 zIgnoreFlag = find_option("ignore",0,1);
198 includeDotFiles = find_option("dotfiles",0,0)!=0;
199 capture_case_sensitive_option();
200 db_must_be_within_tree();
201 caseSensitive = filenames_are_case_sensitive();
202 if( zIgnoreFlag==0 ){
203 zIgnoreFlag = db_get("ignore-glob", 0);
204 }
205 vid = db_lget_int("checkout",0);
206 if( vid==0 ){
@@ -229,11 +242,11 @@
242 }
243 blob_reset(&fullName);
244 }
245 glob_free(pIgnore);
246
247 add_files_in_sfile(vid, caseSensitive);
248 db_end_transaction(0);
249 }
250
251 /*
252 ** COMMAND: rm
@@ -289,10 +302,48 @@
302 "UPDATE vfile SET deleted=1 WHERE pathname IN sfile;"
303 "DELETE FROM vfile WHERE rid=0 AND deleted;"
304 );
305 db_end_transaction(0);
306 }
307
308 /*
309 ** Capture the command-line --case-sensitive option.
310 */
311 static const char *zCaseSensitive = 0;
312 void capture_case_sensitive_option(void){
313 if( zCaseSensitive==0 ){
314 zCaseSensitive = find_option("case-sensitive",0,1);
315 }
316 }
317
318 /*
319 ** This routine determines if files should be case-sensitive or not.
320 ** In other words, this routine determines if two filenames that
321 ** differ only in case should be considered the same name or not.
322 **
323 ** The case-sensitive setting determines the default value. If
324 ** the case-sensitive setting is undefined, then case sensitivity
325 ** defaults on for Mac and Windows and off for all other unix.
326 **
327 ** The --case-sensitive BOOLEAN command-line option overrides any
328 ** setting.
329 */
330 int filenames_are_case_sensitive(void){
331 int caseSensitive;
332
333 if( zCaseSensitive ){
334 caseSensitive = is_truth(zCaseSensitive);
335 }else{
336 #if !defined(_WIN32) && !defined(__DARWIN__) && !defined(__APPLE__)
337 caseSensitive = 1;
338 #else
339 caseSensitive = 0;
340 #endif
341 caseSensitive = db_get_boolean("case-sensitive",caseSensitive);
342 }
343 return caseSensitive;
344 }
345
346 /*
347 ** COMMAND: addremove
348 **
349 ** Usage: %fossil addremove ?--dotfiles? ?--ignore GLOBPATTERN? ?--test?
@@ -319,27 +370,29 @@
370 **
371 ** The --test option shows what would happen without actually doing anything.
372 **
373 ** This command can be used to track third party software.
374 **
 
375 ** SUMMARY: fossil addremove
376 ** Options: ?--dotfiles? ?--ignore GLOB? ?--test? ?--case-sensitive BOOL?
377 */
378 void addremove_cmd(void){
379 Blob path;
380 const char *zIgnoreFlag = find_option("ignore",0,1);
381 int allFlag = find_option("dotfiles",0,0)!=0;
382 int isTest = find_option("test",0,0)!=0;
383 int caseSensitive;
384 int n;
385 Stmt q;
386 int vid;
387 int nAdd = 0;
388 int nDelete = 0;
389 Glob *pIgnore;
390
391 capture_case_sensitive_option();
392 db_must_be_within_tree();
393 caseSensitive = filenames_are_case_sensitive();
394 if( zIgnoreFlag==0 ){
395 zIgnoreFlag = db_get("ignore-glob", 0);
396 }
397 vid = db_lget_int("checkout",0);
398 if( vid==0 ){
@@ -358,11 +411,11 @@
411 blob_init(&path, g.zLocalRoot, n-1);
412 /* now we read the complete file structure into a temp table */
413 pIgnore = glob_create(zIgnoreFlag);
414 vfile_scan(&path, blob_size(&path), allFlag, pIgnore);
415 glob_free(pIgnore);
416 nAdd = add_files_in_sfile(vid, caseSensitive);
417
418 /* step 2: search for missing files */
419 db_prepare(&q,
420 "SELECT pathname, %Q || pathname, deleted FROM vfile"
421 " WHERE NOT deleted"
422
M src/branch.c
+51 -32
--- src/branch.c
+++ src/branch.c
@@ -176,10 +176,47 @@
176176
 db_end_transaction(0);
177177
178178
 /* Do an autosync push, if requested */
179179
 autosync(AUTOSYNC_PUSH);
180180
 }
181
+
182
+/*
183
+** Prepare a query that will list all branches.
184
+*/
185
+static void prepareBranchQuery(Stmt *pQuery, int showAll, int showClosed){
186
+ if( showClosed ){
187
+ db_prepare(pQuery,
188
+ "SELECT value FROM tagxref"
189
+ " WHERE tagid=%d AND value NOT NULL "
190
+ "EXCEPT "
191
+ "SELECT value FROM tagxref"
192
+ " WHERE tagid=%d"
193
+ " AND rid IN leaf"
194
+ " AND NOT %z"
195
+ " ORDER BY value COLLATE nocase /*sort*/",
196
+ TAG_BRANCH, TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
197
+ );
198
+ }else if( showAll ){
199
+ db_prepare(pQuery,
200
+ "SELECT DISTINCT value FROM tagxref"
201
+ " WHERE tagid=%d AND value NOT NULL"
202
+ " AND rid IN leaf"
203
+ " ORDER BY value COLLATE nocase /*sort*/",
204
+ TAG_BRANCH
205
+ );
206
+ }else{
207
+ db_prepare(pQuery,
208
+ "SELECT DISTINCT value FROM tagxref"
209
+ " WHERE tagid=%d AND value NOT NULL"
210
+ " AND rid IN leaf"
211
+ " AND NOT %z"
212
+ " ORDER BY value COLLATE nocase /*sort*/",
213
+ TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
214
+ );
215
+ }
216
+}
217
+
181218
182219
 /*
183220
 ** COMMAND: branch
184221
 **
185222
 ** Usage: %fossil branch SUBCOMMAND ... ?-R|--repository FILE?
@@ -194,11 +231,12 @@
194231
 ** --private option makes the branch private.
195232
 **
196233
 ** %fossil branch list
197234
 ** %fossil branch ls
198235
 **
199
-** List all branches
236
+** List all branches. Use --all or --closed to list all branches
237
+** or closed branches. The default is to show only open branches.
200238
 **
201239
 */
202240
 void branch_cmd(void){
203241
 int n;
204242
 const char *zCmd = "list";
@@ -212,24 +250,19 @@
212250
 branch_new();
213251
 }else if( (strncmp(zCmd,"list",n)==0)||(strncmp(zCmd, "ls", n)==0) ){
214252
 Stmt q;
215253
 int vid;
216254
 char *zCurrent = 0;
255
+ int showAll = find_option("all",0,0)!=0;
256
+ int showClosed = find_option("closed",0,0)!=0;
217257
218258
 if( g.localOpen ){
219259
 vid = db_lget_int("checkout", 0);
220260
 zCurrent = db_text(0, "SELECT value FROM tagxref"
221261
 " WHERE rid=%d AND tagid=%d", vid, TAG_BRANCH);
222262
 }
223
- db_prepare(&q,
224
- "SELECT DISTINCT value FROM tagxref"
225
- " WHERE tagid=%d AND value NOT NULL"
226
- " AND rid IN leaf"
227
- " AND NOT %z"
228
- " ORDER BY value /*sort*/",
229
- TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
230
- );
263
+ prepareBranchQuery(&q, showAll, showClosed);
231264
 while( db_step(&q)==SQLITE_ROW ){
232265
 const char *zBr = db_column_text(&q, 0);
233266
 int isCur = zCurrent!=0 && fossil_strcmp(zCurrent,zBr)==0;
234267
 fossil_print("%s%s\n", (isCur ? "* " : " "), zBr);
235268
 }
@@ -247,19 +280,26 @@
247280
 */
248281
 void brlist_page(void){
249282
 Stmt q;
250283
 int cnt;
251284
 int showClosed = P("closed")!=0;
285
+ int showAll = P("all")!=0;
252286
253287
 login_check_credentials();
254288
 if( !g.okRead ){ login_needed(); return; }
255289
256
- style_header(showClosed ? "Closed Branches" : "Open Branches");
290
+ style_header(showClosed ? "Closed Branches" :
291
+ showAll ? "All Branches" : "Open Branches");
257292
 style_submenu_element("Timeline", "Timeline", "brtimeline");
258293
 if( showClosed ){
294
+ style_submenu_element("All", "All", "brlist?all");
295
+ style_submenu_element("Open","Open","brlist");
296
+ }else if( showAll ){
297
+ style_submenu_element("Closed", "Closed", "brlist?closed");
259298
 style_submenu_element("Open","Open","brlist");
260299
 }else{
300
+ style_submenu_element("All", "All", "brlist?all");
261301
 style_submenu_element("Closed","Closed","brlist?closed");
262302
 }
263303
 login_anonymous_available();
264304
 style_sidebox_begin("Nomenclature:", "33%");
265305
 @ <ol>
@@ -275,33 +315,12 @@
275315
 @ Closed branches are fixed and do not change (unless they are first
276316
 @ reopened)</li>
277317
 @ </ol>
278318
 style_sidebox_end();
279319
320
+ prepareBranchQuery(&q, showAll, showClosed);
280321
 cnt = 0;
281
- if( showClosed ){
282
- db_prepare(&q,
283
- "SELECT value FROM tagxref"
284
- " WHERE tagid=%d AND value NOT NULL "
285
- "EXCEPT "
286
- "SELECT value FROM tagxref"
287
- " WHERE tagid=%d"
288
- " AND rid IN leaf"
289
- " AND NOT %z"
290
- " ORDER BY value /*sort*/",
291
- TAG_BRANCH, TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
292
- );
293
- }else{
294
- db_prepare(&q,
295
- "SELECT DISTINCT value FROM tagxref"
296
- " WHERE tagid=%d AND value NOT NULL"
297
- " AND rid IN leaf"
298
- " AND NOT %z"
299
- " ORDER BY value /*sort*/",
300
- TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
301
- );
302
- }
303322
 while( db_step(&q)==SQLITE_ROW ){
304323
 const char *zBr = db_column_text(&q, 0);
305324
 if( cnt==0 ){
306325
 if( showClosed ){
307326
 @ <h2>Closed Branches:</h2>
308327
--- src/branch.c
+++ src/branch.c
@@ -176,10 +176,47 @@
176 db_end_transaction(0);
177
178 /* Do an autosync push, if requested */
179 autosync(AUTOSYNC_PUSH);
180 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
181
182 /*
183 ** COMMAND: branch
184 **
185 ** Usage: %fossil branch SUBCOMMAND ... ?-R|--repository FILE?
@@ -194,11 +231,12 @@
194 ** --private option makes the branch private.
195 **
196 ** %fossil branch list
197 ** %fossil branch ls
198 **
199 ** List all branches
 
200 **
201 */
202 void branch_cmd(void){
203 int n;
204 const char *zCmd = "list";
@@ -212,24 +250,19 @@
212 branch_new();
213 }else if( (strncmp(zCmd,"list",n)==0)||(strncmp(zCmd, "ls", n)==0) ){
214 Stmt q;
215 int vid;
216 char *zCurrent = 0;
 
 
217
218 if( g.localOpen ){
219 vid = db_lget_int("checkout", 0);
220 zCurrent = db_text(0, "SELECT value FROM tagxref"
221 " WHERE rid=%d AND tagid=%d", vid, TAG_BRANCH);
222 }
223 db_prepare(&q,
224 "SELECT DISTINCT value FROM tagxref"
225 " WHERE tagid=%d AND value NOT NULL"
226 " AND rid IN leaf"
227 " AND NOT %z"
228 " ORDER BY value /*sort*/",
229 TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
230 );
231 while( db_step(&q)==SQLITE_ROW ){
232 const char *zBr = db_column_text(&q, 0);
233 int isCur = zCurrent!=0 && fossil_strcmp(zCurrent,zBr)==0;
234 fossil_print("%s%s\n", (isCur ? "* " : " "), zBr);
235 }
@@ -247,19 +280,26 @@
247 */
248 void brlist_page(void){
249 Stmt q;
250 int cnt;
251 int showClosed = P("closed")!=0;
 
252
253 login_check_credentials();
254 if( !g.okRead ){ login_needed(); return; }
255
256 style_header(showClosed ? "Closed Branches" : "Open Branches");
 
257 style_submenu_element("Timeline", "Timeline", "brtimeline");
258 if( showClosed ){
 
 
 
 
259 style_submenu_element("Open","Open","brlist");
260 }else{
 
261 style_submenu_element("Closed","Closed","brlist?closed");
262 }
263 login_anonymous_available();
264 style_sidebox_begin("Nomenclature:", "33%");
265 @ <ol>
@@ -275,33 +315,12 @@
275 @ Closed branches are fixed and do not change (unless they are first
276 @ reopened)</li>
277 @ </ol>
278 style_sidebox_end();
279
 
280 cnt = 0;
281 if( showClosed ){
282 db_prepare(&q,
283 "SELECT value FROM tagxref"
284 " WHERE tagid=%d AND value NOT NULL "
285 "EXCEPT "
286 "SELECT value FROM tagxref"
287 " WHERE tagid=%d"
288 " AND rid IN leaf"
289 " AND NOT %z"
290 " ORDER BY value /*sort*/",
291 TAG_BRANCH, TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
292 );
293 }else{
294 db_prepare(&q,
295 "SELECT DISTINCT value FROM tagxref"
296 " WHERE tagid=%d AND value NOT NULL"
297 " AND rid IN leaf"
298 " AND NOT %z"
299 " ORDER BY value /*sort*/",
300 TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
301 );
302 }
303 while( db_step(&q)==SQLITE_ROW ){
304 const char *zBr = db_column_text(&q, 0);
305 if( cnt==0 ){
306 if( showClosed ){
307 @ <h2>Closed Branches:</h2>
308
--- src/branch.c
+++ src/branch.c
@@ -176,10 +176,47 @@
176 db_end_transaction(0);
177
178 /* Do an autosync push, if requested */
179 autosync(AUTOSYNC_PUSH);
180 }
181
182 /*
183 ** Prepare a query that will list all branches.
184 */
185 static void prepareBranchQuery(Stmt *pQuery, int showAll, int showClosed){
186 if( showClosed ){
187 db_prepare(pQuery,
188 "SELECT value FROM tagxref"
189 " WHERE tagid=%d AND value NOT NULL "
190 "EXCEPT "
191 "SELECT value FROM tagxref"
192 " WHERE tagid=%d"
193 " AND rid IN leaf"
194 " AND NOT %z"
195 " ORDER BY value COLLATE nocase /*sort*/",
196 TAG_BRANCH, TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
197 );
198 }else if( showAll ){
199 db_prepare(pQuery,
200 "SELECT DISTINCT value FROM tagxref"
201 " WHERE tagid=%d AND value NOT NULL"
202 " AND rid IN leaf"
203 " ORDER BY value COLLATE nocase /*sort*/",
204 TAG_BRANCH
205 );
206 }else{
207 db_prepare(pQuery,
208 "SELECT DISTINCT value FROM tagxref"
209 " WHERE tagid=%d AND value NOT NULL"
210 " AND rid IN leaf"
211 " AND NOT %z"
212 " ORDER BY value COLLATE nocase /*sort*/",
213 TAG_BRANCH, leaf_is_closed_sql("tagxref.rid")
214 );
215 }
216 }
217
218
219 /*
220 ** COMMAND: branch
221 **
222 ** Usage: %fossil branch SUBCOMMAND ... ?-R|--repository FILE?
@@ -194,11 +231,12 @@
231 ** --private option makes the branch private.
232 **
233 ** %fossil branch list
234 ** %fossil branch ls
235 **
236 ** List all branches. Use --all or --closed to list all branches
237 ** or closed branches. The default is to show only open branches.
238 **
239 */
240 void branch_cmd(void){
241 int n;
242 const char *zCmd = "list";
@@ -212,24 +250,19 @@
250 branch_new();
251 }else if( (strncmp(zCmd,"list",n)==0)||(strncmp(zCmd, "ls", n)==0) ){
252 Stmt q;
253 int vid;
254 char *zCurrent = 0;
255 int showAll = find_option("all",0,0)!=0;
256 int showClosed = find_option("closed",0,0)!=0;
257
258 if( g.localOpen ){
259 vid = db_lget_int("checkout", 0);
260 zCurrent = db_text(0, "SELECT value FROM tagxref"
261 " WHERE rid=%d AND tagid=%d", vid, TAG_BRANCH);
262 }
263 prepareBranchQuery(&q, showAll, showClosed);
 
 
 
 
 
 
 
264 while( db_step(&q)==SQLITE_ROW ){
265 const char *zBr = db_column_text(&q, 0);
266 int isCur = zCurrent!=0 && fossil_strcmp(zCurrent,zBr)==0;
267 fossil_print("%s%s\n", (isCur ? "* " : " "), zBr);
268 }
@@ -247,19 +280,26 @@
280 */
281 void brlist_page(void){
282 Stmt q;
283 int cnt;
284 int showClosed = P("closed")!=0;
285 int showAll = P("all")!=0;
286
287 login_check_credentials();
288 if( !g.okRead ){ login_needed(); return; }
289
290 style_header(showClosed ? "Closed Branches" :
291 showAll ? "All Branches" : "Open Branches");
292 style_submenu_element("Timeline", "Timeline", "brtimeline");
293 if( showClosed ){
294 style_submenu_element("All", "All", "brlist?all");
295 style_submenu_element("Open","Open","brlist");
296 }else if( showAll ){
297 style_submenu_element("Closed", "Closed", "brlist?closed");
298 style_submenu_element("Open","Open","brlist");
299 }else{
300 style_submenu_element("All", "All", "brlist?all");
301 style_submenu_element("Closed","Closed","brlist?closed");
302 }
303 login_anonymous_available();
304 style_sidebox_begin("Nomenclature:", "33%");
305 @ <ol>
@@ -275,33 +315,12 @@
315 @ Closed branches are fixed and do not change (unless they are first
316 @ reopened)</li>
317 @ </ol>
318 style_sidebox_end();
319
320 prepareBranchQuery(&q, showAll, showClosed);
321 cnt = 0;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
322 while( db_step(&q)==SQLITE_ROW ){
323 const char *zBr = db_column_text(&q, 0);
324 if( cnt==0 ){
325 if( showClosed ){
326 @ <h2>Closed Branches:</h2>
327
M src/browse.c
+1 -1
--- src/browse.c
+++ src/browse.c
@@ -267,11 +267,11 @@
267267
 if( zFN[0]=='/' ){
268268
 zFN++;
269269
 @ <li><a href="%s(zSubdirLink)%T(zFN)">%h(zFN)/</a></li>
270270
 }else if( zCI ){
271271
 const char *zUuid = db_column_text(&q, 1);
272
- @ <li><a href="%s(g.zTop)/artifact?name=%s(zUuid)">%h(zFN)</a></li>
272
+ @ <li><a href="%s(g.zTop)/artifact/%s(zUuid)">%h(zFN)</a></li>
273273
 }else{
274274
 @ <li><a href="%s(g.zTop)/finfo?name=%T(zPrefix)%T(zFN)">%h(zFN)
275275
 @ </a></li>
276276
 }
277277
 }
278278
--- src/browse.c
+++ src/browse.c
@@ -267,11 +267,11 @@
267 if( zFN[0]=='/' ){
268 zFN++;
269 @ <li><a href="%s(zSubdirLink)%T(zFN)">%h(zFN)/</a></li>
270 }else if( zCI ){
271 const char *zUuid = db_column_text(&q, 1);
272 @ <li><a href="%s(g.zTop)/artifact?name=%s(zUuid)">%h(zFN)</a></li>
273 }else{
274 @ <li><a href="%s(g.zTop)/finfo?name=%T(zPrefix)%T(zFN)">%h(zFN)
275 @ </a></li>
276 }
277 }
278
--- src/browse.c
+++ src/browse.c
@@ -267,11 +267,11 @@
267 if( zFN[0]=='/' ){
268 zFN++;
269 @ <li><a href="%s(zSubdirLink)%T(zFN)">%h(zFN)/</a></li>
270 }else if( zCI ){
271 const char *zUuid = db_column_text(&q, 1);
272 @ <li><a href="%s(g.zTop)/artifact/%s(zUuid)">%h(zFN)</a></li>
273 }else{
274 @ <li><a href="%s(g.zTop)/finfo?name=%T(zPrefix)%T(zFN)">%h(zFN)
275 @ </a></li>
276 }
277 }
278
M src/cgi.c
+13 -3
--- src/cgi.c
+++ src/cgi.c
@@ -295,11 +295,11 @@
295295
 if( blob_size(&extraHeader)>0 ){
296296
 fprintf(g.httpOut, "%s", blob_buffer(&extraHeader));
297297
 }
298298
299299
 /* Add headers to turn on useful security options in browsers. */
300
- fprintf(g.httpOut, "X-Frame-Options: DENY\r\n");
300
+ fprintf(g.httpOut, "X-Frame-Options: SAMEORIGIN\r\n");
301301
 /* This stops fossil pages appearing in frames or iframes, preventing
302302
 ** click-jacking attacks on supporting browsers.
303303
 **
304304
 ** Other good headers would be
305305
 ** Strict-Transport-Security: max-age=62208000
@@ -353,10 +353,11 @@
353353
 if( size>0 ){
354354
 fwrite(blob_buffer(&cgiContent[i]), 1, size, g.httpOut);
355355
 }
356356
 }
357357
 }
358
+ fflush(g.httpOut);
358359
 CGIDEBUG(("DONE\n"));
359360
 }
360361
361362
 /*
362363
 ** Do a redirect request to the URL given in the argument.
@@ -887,17 +888,26 @@
887888
 }
888889
889890
 /*
890891
 ** Print all query parameters on standard output. Format the
891892
 ** parameters as HTML. This is used for testing and debugging.
893
+** Release builds omit the values of the cookies to avoid defeating
894
+** the purpose of setting HttpOnly cookies.
892895
 */
893896
 void cgi_print_all(void){
894897
 int i;
898
+ int showAll = 0;
899
+#ifdef FOSSIL_DEBUG
900
+ /* Show the values of cookies in debug mode. */
901
+ showAll = 1;
902
+#endif
895903
 cgi_parameter("",""); /* Force the parameters into sorted order */
896904
 for(i=0; i<nUsedQP; i++){
897
- cgi_printf("%s = %s <br />\n",
898
- htmlize(aParamQP[i].zName, -1), htmlize(aParamQP[i].zValue, -1));
905
+ if( showAll || (fossil_stricmp("HTTP_COOKIE",aParamQP[i].zName)!=0 && fossil_strnicmp("fossil-",aParamQP[i].zName,7)!=0) ){
906
+ cgi_printf("%s = %s <br />\n",
907
+ htmlize(aParamQP[i].zName, -1), htmlize(aParamQP[i].zValue, -1));
908
+ }
899909
 }
900910
 }
901911
902912
 /*
903913
 ** This routine works like "printf" except that it has the
904914
--- src/cgi.c
+++ src/cgi.c
@@ -295,11 +295,11 @@
295 if( blob_size(&extraHeader)>0 ){
296 fprintf(g.httpOut, "%s", blob_buffer(&extraHeader));
297 }
298
299 /* Add headers to turn on useful security options in browsers. */
300 fprintf(g.httpOut, "X-Frame-Options: DENY\r\n");
301 /* This stops fossil pages appearing in frames or iframes, preventing
302 ** click-jacking attacks on supporting browsers.
303 **
304 ** Other good headers would be
305 ** Strict-Transport-Security: max-age=62208000
@@ -353,10 +353,11 @@
353 if( size>0 ){
354 fwrite(blob_buffer(&cgiContent[i]), 1, size, g.httpOut);
355 }
356 }
357 }
 
358 CGIDEBUG(("DONE\n"));
359 }
360
361 /*
362 ** Do a redirect request to the URL given in the argument.
@@ -887,17 +888,26 @@
887 }
888
889 /*
890 ** Print all query parameters on standard output. Format the
891 ** parameters as HTML. This is used for testing and debugging.
 
 
892 */
893 void cgi_print_all(void){
894 int i;
 
 
 
 
 
895 cgi_parameter("",""); /* Force the parameters into sorted order */
896 for(i=0; i<nUsedQP; i++){
897 cgi_printf("%s = %s <br />\n",
898 htmlize(aParamQP[i].zName, -1), htmlize(aParamQP[i].zValue, -1));
 
 
899 }
900 }
901
902 /*
903 ** This routine works like "printf" except that it has the
904
--- src/cgi.c
+++ src/cgi.c
@@ -295,11 +295,11 @@
295 if( blob_size(&extraHeader)>0 ){
296 fprintf(g.httpOut, "%s", blob_buffer(&extraHeader));
297 }
298
299 /* Add headers to turn on useful security options in browsers. */
300 fprintf(g.httpOut, "X-Frame-Options: SAMEORIGIN\r\n");
301 /* This stops fossil pages appearing in frames or iframes, preventing
302 ** click-jacking attacks on supporting browsers.
303 **
304 ** Other good headers would be
305 ** Strict-Transport-Security: max-age=62208000
@@ -353,10 +353,11 @@
353 if( size>0 ){
354 fwrite(blob_buffer(&cgiContent[i]), 1, size, g.httpOut);
355 }
356 }
357 }
358 fflush(g.httpOut);
359 CGIDEBUG(("DONE\n"));
360 }
361
362 /*
363 ** Do a redirect request to the URL given in the argument.
@@ -887,17 +888,26 @@
888 }
889
890 /*
891 ** Print all query parameters on standard output. Format the
892 ** parameters as HTML. This is used for testing and debugging.
893 ** Release builds omit the values of the cookies to avoid defeating
894 ** the purpose of setting HttpOnly cookies.
895 */
896 void cgi_print_all(void){
897 int i;
898 int showAll = 0;
899 #ifdef FOSSIL_DEBUG
900 /* Show the values of cookies in debug mode. */
901 showAll = 1;
902 #endif
903 cgi_parameter("",""); /* Force the parameters into sorted order */
904 for(i=0; i<nUsedQP; i++){
905 if( showAll || (fossil_stricmp("HTTP_COOKIE",aParamQP[i].zName)!=0 && fossil_strnicmp("fossil-",aParamQP[i].zName,7)!=0) ){
906 cgi_printf("%s = %s <br />\n",
907 htmlize(aParamQP[i].zName, -1), htmlize(aParamQP[i].zValue, -1));
908 }
909 }
910 }
911
912 /*
913 ** This routine works like "printf" except that it has the
914
M src/checkin.c
+92 -15
--- src/checkin.c
+++ src/checkin.c
@@ -32,60 +32,72 @@
3232
 ** are not true files results in a fatal error.
3333
 */
3434
 static void status_report(
3535
 Blob *report, /* Append the status report here */
3636
 const char *zPrefix, /* Prefix on each line of the report */
37
- int missingIsFatal /* MISSING and NOT_A_FILE are fatal errors */
37
+ int missingIsFatal, /* MISSING and NOT_A_FILE are fatal errors */
38
+ int cwdRelative /* Report relative to the current working dir */
3839
 ){
3940
 Stmt q;
4041
 int nPrefix = strlen(zPrefix);
4142
 int nErr = 0;
43
+ Blob rewrittenPathname;
4244
 db_prepare(&q,
4345
 "SELECT pathname, deleted, chnged, rid, coalesce(origname!=pathname,0)"
4446
 " FROM vfile "
4547
 " WHERE file_is_selected(id)"
4648
 " AND (chnged OR deleted OR rid=0 OR pathname!=origname) ORDER BY 1"
4749
 );
50
+ blob_zero(&rewrittenPathname);
4851
 while( db_step(&q)==SQLITE_ROW ){
4952
 const char *zPathname = db_column_text(&q,0);
53
+ const char *zDisplayName = zPathname;
5054
 int isDeleted = db_column_int(&q, 1);
5155
 int isChnged = db_column_int(&q,2);
5256
 int isNew = db_column_int(&q,3)==0;
5357
 int isRenamed = db_column_int(&q,4);
5458
 char *zFullName = mprintf("%s%s", g.zLocalRoot, zPathname);
59
+ if( cwdRelative ){
60
+ file_relative_name(zFullName, &rewrittenPathname);
61
+ zDisplayName = blob_str(&rewrittenPathname);
62
+ if( zDisplayName[0]=='.' && zDisplayName[1]=='/' ){
63
+ zDisplayName += 2; /* no unnecessary ./ prefix */
64
+ }
65
+ }
5566
 blob_append(report, zPrefix, nPrefix);
5667
 if( isDeleted ){
57
- blob_appendf(report, "DELETED %s\n", zPathname);
68
+ blob_appendf(report, "DELETED %s\n", zDisplayName);
5869
 }else if( !file_isfile(zFullName) ){
5970
 if( file_access(zFullName, 0)==0 ){
60
- blob_appendf(report, "NOT_A_FILE %s\n", zPathname);
71
+ blob_appendf(report, "NOT_A_FILE %s\n", zDisplayName);
6172
 if( missingIsFatal ){
62
- fossil_warning("not a file: %s", zPathname);
73
+ fossil_warning("not a file: %s", zDisplayName);
6374
 nErr++;
6475
 }
6576
 }else{
66
- blob_appendf(report, "MISSING %s\n", zPathname);
77
+ blob_appendf(report, "MISSING %s\n", zDisplayName);
6778
 if( missingIsFatal ){
68
- fossil_warning("missing file: %s", zPathname);
79
+ fossil_warning("missing file: %s", zDisplayName);
6980
 nErr++;
7081
 }
7182
 }
7283
 }else if( isNew ){
73
- blob_appendf(report, "ADDED %s\n", zPathname);
84
+ blob_appendf(report, "ADDED %s\n", zDisplayName);
7485
 }else if( isDeleted ){
75
- blob_appendf(report, "DELETED %s\n", zPathname);
86
+ blob_appendf(report, "DELETED %s\n", zDisplayName);
7687
 }else if( isChnged==2 ){
77
- blob_appendf(report, "UPDATED_BY_MERGE %s\n", zPathname);
88
+ blob_appendf(report, "UPDATED_BY_MERGE %s\n", zDisplayName);
7889
 }else if( isChnged==3 ){
79
- blob_appendf(report, "ADDED_BY_MERGE %s\n", zPathname);
90
+ blob_appendf(report, "ADDED_BY_MERGE %s\n", zDisplayName);
8091
 }else if( isChnged==1 ){
81
- blob_appendf(report, "EDITED %s\n", zPathname);
92
+ blob_appendf(report, "EDITED %s\n", zDisplayName);
8293
 }else if( isRenamed ){
83
- blob_appendf(report, "RENAMED %s\n", zPathname);
94
+ blob_appendf(report, "RENAMED %s\n", zDisplayName);
8495
 }
8596
 free(zFullName);
8697
 }
98
+ blob_reset(&rewrittenPathname);
8799
 db_finalize(&q);
88100
 db_prepare(&q, "SELECT uuid FROM vmerge JOIN blob ON merge=rid"
89101
 " WHERE id=0");
90102
 while( db_step(&q)==SQLITE_ROW ){
91103
 blob_append(report, zPrefix, nPrefix);
@@ -94,33 +106,59 @@
94106
 db_finalize(&q);
95107
 if( nErr ){
96108
 fossil_fatal("aborting due to prior errors");
97109
 }
98110
 }
111
+
112
+/*
113
+** Use the "relative-paths" setting and the --abs-paths and
114
+** --rel-paths command line options to determine whether the
115
+** status report should be shown relative to the current
116
+** working directory.
117
+*/
118
+static int determine_cwd_relative_option()
119
+{
120
+ int relativePaths = db_get_boolean("relative-paths", 1);
121
+ int absPathOption = find_option("abs-paths", 0, 0)!=0;
122
+ int relPathOption = find_option("rel-paths", 0, 0)!=0;
123
+ if( absPathOption ){ relativePaths = 0; }
124
+ if( relPathOption ){ relativePaths = 1; }
125
+ return relativePaths;
126
+}
99127
100128
 /*
101129
 ** COMMAND: changes
102130
 **
103131
 ** Usage: %fossil changes
104132
 **
105133
 ** Report on the edit status of all files in the current checkout.
106134
 ** See also the "status" and "extra" commands.
107135
 **
136
+** Pathnames are displayed according to the "relative-paths" setting,
137
+** unless overridden by the --abs-paths or --rel-paths options.
138
+**
108139
 ** Options:
109140
 **
110141
 ** --sha1sum Verify file status using SHA1 hashing rather
111142
 ** than relying on file mtimes.
143
+**
144
+** --abs-paths Display absolute pathnames.
145
+**
146
+** --rel-paths Display pathnames relative to the current working
147
+** directory.
112148
 */
113149
 void changes_cmd(void){
114150
 Blob report;
115151
 int vid;
116152
 int useSha1sum = find_option("sha1sum", 0, 0)!=0;
153
+ int cwdRelative = 0;
117154
 db_must_be_within_tree();
155
+ cwdRelative = determine_cwd_relative_option();
118156
 blob_zero(&report);
119157
 vid = db_lget_int("checkout", 0);
120158
 vfile_check_signature(vid, 0, useSha1sum);
121
- status_report(&report, "", 0);
159
+ status_report(&report, "", 0, cwdRelative);
122160
 blob_write_to_file(&report, "-");
123161
 }
124162
125163
 /*
126164
 ** COMMAND: status
@@ -127,14 +165,22 @@
127165
 **
128166
 ** Usage: %fossil status
129167
 **
130168
 ** Report on the status of the current checkout.
131169
 **
170
+** Pathnames are displayed according to the "relative-paths" setting,
171
+** unless overridden by the --abs-paths or --rel-paths options.
172
+**
132173
 ** Options:
133174
 **
134175
 ** --sha1sum Verify file status using SHA1 hashing rather
135176
 ** than relying on file mtimes.
177
+**
178
+** --abs-paths Display absolute pathnames.
179
+**
180
+** --rel-paths Display pathnames relative to the current working
181
+** directory.
136182
 */
137183
 void status_cmd(void){
138184
 int vid;
139185
 db_must_be_within_tree();
140186
 /* 012345678901234 */
@@ -212,22 +258,41 @@
212258
 ** ignored but can be included by adding the --dotfiles option.
213259
 **
214260
 ** The GLOBPATTERN is a comma-separated list of GLOB expressions for
215261
 ** files that are ignored. The GLOBPATTERN specified by the "ignore-glob"
216262
 ** is used if the --ignore option is omitted.
263
+**
264
+** Pathnames are displayed according to the "relative-paths" setting,
265
+** unless overridden by the --abs-paths or --rel-paths options.
266
+**
267
+** Options:
268
+**
269
+** --dotfiles Include files with names beginning with "."
270
+**
271
+** --ignore GLOBPATTERN
272
+** Override the "ignore-glob" setting.
273
+**
274
+** --abs-paths Display absolute pathnames.
275
+**
276
+** --rel-paths Display pathnames relative to the current working
277
+** directory.
217278
 */
218279
 void extra_cmd(void){
219280
 Blob path;
220281
 Blob repo;
221282
 Stmt q;
222283
 int n;
223284
 const char *zIgnoreFlag = find_option("ignore",0,1);
224285
 int allFlag = find_option("dotfiles",0,0)!=0;
286
+ int cwdRelative = 0;
225287
 int outputManifest;
226288
 Glob *pIgnore;
289
+ Blob rewrittenPathname;
290
+ const char *zPathname, *zDisplayName;
227291
228292
 db_must_be_within_tree();
293
+ cwdRelative = determine_cwd_relative_option();
229294
 outputManifest = db_get_boolean("manifest",0);
230295
 db_multi_exec("CREATE TEMP TABLE sfile(x TEXT PRIMARY KEY)");
231296
 n = strlen(g.zLocalRoot);
232297
 blob_init(&path, g.zLocalRoot, n-1);
233298
 if( zIgnoreFlag==0 ){
@@ -243,13 +308,25 @@
243308
 fossil_all_reserved_names()
244309
 );
245310
 if( file_tree_name(g.zRepositoryName, &repo, 0) ){
246311
 db_multi_exec("DELETE FROM sfile WHERE x=%B", &repo);
247312
 }
313
+ blob_zero(&rewrittenPathname);
248314
 while( db_step(&q)==SQLITE_ROW ){
249
- fossil_print("%s\n", db_column_text(&q, 0));
315
+ zDisplayName = zPathname = db_column_text(&q, 0);
316
+ if( cwdRelative ) {
317
+ char *zFullName = mprintf("%s%s", g.zLocalRoot, zPathname);
318
+ file_relative_name(zFullName, &rewrittenPathname);
319
+ free(zFullName);
320
+ zDisplayName = blob_str(&rewrittenPathname);
321
+ if( zDisplayName[0]=='.' && zDisplayName[1]=='/' ){
322
+ zDisplayName += 2; /* no unnecessary ./ prefix */
323
+ }
324
+ }
325
+ fossil_print("%s\n", zDisplayName);
250326
 }
327
+ blob_reset(&rewrittenPathname);
251328
 db_finalize(&q);
252329
 }
253330
254331
 /*
255332
 ** COMMAND: clean
@@ -367,11 +444,11 @@
367444
 "# PRIVATE BRANCH: This check-in will be private and will not sync to\n"
368445
 "# repositories.\n"
369446
 "#\n", -1
370447
 );
371448
 }
372
- status_report(&text, "# ", 1);
449
+ status_report(&text, "# ", 1, 0);
373450
 zEditor = db_get("editor", 0);
374451
 if( zEditor==0 ){
375452
 zEditor = getenv("VISUAL");
376453
 }
377454
 if( zEditor==0 ){
378455
--- src/checkin.c
+++ src/checkin.c
@@ -32,60 +32,72 @@
32 ** are not true files results in a fatal error.
33 */
34 static void status_report(
35 Blob *report, /* Append the status report here */
36 const char *zPrefix, /* Prefix on each line of the report */
37 int missingIsFatal /* MISSING and NOT_A_FILE are fatal errors */
 
38 ){
39 Stmt q;
40 int nPrefix = strlen(zPrefix);
41 int nErr = 0;
 
42 db_prepare(&q,
43 "SELECT pathname, deleted, chnged, rid, coalesce(origname!=pathname,0)"
44 " FROM vfile "
45 " WHERE file_is_selected(id)"
46 " AND (chnged OR deleted OR rid=0 OR pathname!=origname) ORDER BY 1"
47 );
 
48 while( db_step(&q)==SQLITE_ROW ){
49 const char *zPathname = db_column_text(&q,0);
 
50 int isDeleted = db_column_int(&q, 1);
51 int isChnged = db_column_int(&q,2);
52 int isNew = db_column_int(&q,3)==0;
53 int isRenamed = db_column_int(&q,4);
54 char *zFullName = mprintf("%s%s", g.zLocalRoot, zPathname);
 
 
 
 
 
 
 
55 blob_append(report, zPrefix, nPrefix);
56 if( isDeleted ){
57 blob_appendf(report, "DELETED %s\n", zPathname);
58 }else if( !file_isfile(zFullName) ){
59 if( file_access(zFullName, 0)==0 ){
60 blob_appendf(report, "NOT_A_FILE %s\n", zPathname);
61 if( missingIsFatal ){
62 fossil_warning("not a file: %s", zPathname);
63 nErr++;
64 }
65 }else{
66 blob_appendf(report, "MISSING %s\n", zPathname);
67 if( missingIsFatal ){
68 fossil_warning("missing file: %s", zPathname);
69 nErr++;
70 }
71 }
72 }else if( isNew ){
73 blob_appendf(report, "ADDED %s\n", zPathname);
74 }else if( isDeleted ){
75 blob_appendf(report, "DELETED %s\n", zPathname);
76 }else if( isChnged==2 ){
77 blob_appendf(report, "UPDATED_BY_MERGE %s\n", zPathname);
78 }else if( isChnged==3 ){
79 blob_appendf(report, "ADDED_BY_MERGE %s\n", zPathname);
80 }else if( isChnged==1 ){
81 blob_appendf(report, "EDITED %s\n", zPathname);
82 }else if( isRenamed ){
83 blob_appendf(report, "RENAMED %s\n", zPathname);
84 }
85 free(zFullName);
86 }
 
87 db_finalize(&q);
88 db_prepare(&q, "SELECT uuid FROM vmerge JOIN blob ON merge=rid"
89 " WHERE id=0");
90 while( db_step(&q)==SQLITE_ROW ){
91 blob_append(report, zPrefix, nPrefix);
@@ -94,33 +106,59 @@
94 db_finalize(&q);
95 if( nErr ){
96 fossil_fatal("aborting due to prior errors");
97 }
98 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
100 /*
101 ** COMMAND: changes
102 **
103 ** Usage: %fossil changes
104 **
105 ** Report on the edit status of all files in the current checkout.
106 ** See also the "status" and "extra" commands.
107 **
 
 
 
108 ** Options:
109 **
110 ** --sha1sum Verify file status using SHA1 hashing rather
111 ** than relying on file mtimes.
 
 
 
 
 
112 */
113 void changes_cmd(void){
114 Blob report;
115 int vid;
116 int useSha1sum = find_option("sha1sum", 0, 0)!=0;
 
117 db_must_be_within_tree();
 
118 blob_zero(&report);
119 vid = db_lget_int("checkout", 0);
120 vfile_check_signature(vid, 0, useSha1sum);
121 status_report(&report, "", 0);
122 blob_write_to_file(&report, "-");
123 }
124
125 /*
126 ** COMMAND: status
@@ -127,14 +165,22 @@
127 **
128 ** Usage: %fossil status
129 **
130 ** Report on the status of the current checkout.
131 **
 
 
 
132 ** Options:
133 **
134 ** --sha1sum Verify file status using SHA1 hashing rather
135 ** than relying on file mtimes.
 
 
 
 
 
136 */
137 void status_cmd(void){
138 int vid;
139 db_must_be_within_tree();
140 /* 012345678901234 */
@@ -212,22 +258,41 @@
212 ** ignored but can be included by adding the --dotfiles option.
213 **
214 ** The GLOBPATTERN is a comma-separated list of GLOB expressions for
215 ** files that are ignored. The GLOBPATTERN specified by the "ignore-glob"
216 ** is used if the --ignore option is omitted.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
217 */
218 void extra_cmd(void){
219 Blob path;
220 Blob repo;
221 Stmt q;
222 int n;
223 const char *zIgnoreFlag = find_option("ignore",0,1);
224 int allFlag = find_option("dotfiles",0,0)!=0;
 
225 int outputManifest;
226 Glob *pIgnore;
 
 
227
228 db_must_be_within_tree();
 
229 outputManifest = db_get_boolean("manifest",0);
230 db_multi_exec("CREATE TEMP TABLE sfile(x TEXT PRIMARY KEY)");
231 n = strlen(g.zLocalRoot);
232 blob_init(&path, g.zLocalRoot, n-1);
233 if( zIgnoreFlag==0 ){
@@ -243,13 +308,25 @@
243 fossil_all_reserved_names()
244 );
245 if( file_tree_name(g.zRepositoryName, &repo, 0) ){
246 db_multi_exec("DELETE FROM sfile WHERE x=%B", &repo);
247 }
 
248 while( db_step(&q)==SQLITE_ROW ){
249 fossil_print("%s\n", db_column_text(&q, 0));
 
 
 
 
 
 
 
 
 
 
250 }
 
251 db_finalize(&q);
252 }
253
254 /*
255 ** COMMAND: clean
@@ -367,11 +444,11 @@
367 "# PRIVATE BRANCH: This check-in will be private and will not sync to\n"
368 "# repositories.\n"
369 "#\n", -1
370 );
371 }
372 status_report(&text, "# ", 1);
373 zEditor = db_get("editor", 0);
374 if( zEditor==0 ){
375 zEditor = getenv("VISUAL");
376 }
377 if( zEditor==0 ){
378
--- src/checkin.c
+++ src/checkin.c
@@ -32,60 +32,72 @@
32 ** are not true files results in a fatal error.
33 */
34 static void status_report(
35 Blob *report, /* Append the status report here */
36 const char *zPrefix, /* Prefix on each line of the report */
37 int missingIsFatal, /* MISSING and NOT_A_FILE are fatal errors */
38 int cwdRelative /* Report relative to the current working dir */
39 ){
40 Stmt q;
41 int nPrefix = strlen(zPrefix);
42 int nErr = 0;
43 Blob rewrittenPathname;
44 db_prepare(&q,
45 "SELECT pathname, deleted, chnged, rid, coalesce(origname!=pathname,0)"
46 " FROM vfile "
47 " WHERE file_is_selected(id)"
48 " AND (chnged OR deleted OR rid=0 OR pathname!=origname) ORDER BY 1"
49 );
50 blob_zero(&rewrittenPathname);
51 while( db_step(&q)==SQLITE_ROW ){
52 const char *zPathname = db_column_text(&q,0);
53 const char *zDisplayName = zPathname;
54 int isDeleted = db_column_int(&q, 1);
55 int isChnged = db_column_int(&q,2);
56 int isNew = db_column_int(&q,3)==0;
57 int isRenamed = db_column_int(&q,4);
58 char *zFullName = mprintf("%s%s", g.zLocalRoot, zPathname);
59 if( cwdRelative ){
60 file_relative_name(zFullName, &rewrittenPathname);
61 zDisplayName = blob_str(&rewrittenPathname);
62 if( zDisplayName[0]=='.' && zDisplayName[1]=='/' ){
63 zDisplayName += 2; /* no unnecessary ./ prefix */
64 }
65 }
66 blob_append(report, zPrefix, nPrefix);
67 if( isDeleted ){
68 blob_appendf(report, "DELETED %s\n", zDisplayName);
69 }else if( !file_isfile(zFullName) ){
70 if( file_access(zFullName, 0)==0 ){
71 blob_appendf(report, "NOT_A_FILE %s\n", zDisplayName);
72 if( missingIsFatal ){
73 fossil_warning("not a file: %s", zDisplayName);
74 nErr++;
75 }
76 }else{
77 blob_appendf(report, "MISSING %s\n", zDisplayName);
78 if( missingIsFatal ){
79 fossil_warning("missing file: %s", zDisplayName);
80 nErr++;
81 }
82 }
83 }else if( isNew ){
84 blob_appendf(report, "ADDED %s\n", zDisplayName);
85 }else if( isDeleted ){
86 blob_appendf(report, "DELETED %s\n", zDisplayName);
87 }else if( isChnged==2 ){
88 blob_appendf(report, "UPDATED_BY_MERGE %s\n", zDisplayName);
89 }else if( isChnged==3 ){
90 blob_appendf(report, "ADDED_BY_MERGE %s\n", zDisplayName);
91 }else if( isChnged==1 ){
92 blob_appendf(report, "EDITED %s\n", zDisplayName);
93 }else if( isRenamed ){
94 blob_appendf(report, "RENAMED %s\n", zDisplayName);
95 }
96 free(zFullName);
97 }
98 blob_reset(&rewrittenPathname);
99 db_finalize(&q);
100 db_prepare(&q, "SELECT uuid FROM vmerge JOIN blob ON merge=rid"
101 " WHERE id=0");
102 while( db_step(&q)==SQLITE_ROW ){
103 blob_append(report, zPrefix, nPrefix);
@@ -94,33 +106,59 @@
106 db_finalize(&q);
107 if( nErr ){
108 fossil_fatal("aborting due to prior errors");
109 }
110 }
111
112 /*
113 ** Use the "relative-paths" setting and the --abs-paths and
114 ** --rel-paths command line options to determine whether the
115 ** status report should be shown relative to the current
116 ** working directory.
117 */
118 static int determine_cwd_relative_option()
119 {
120 int relativePaths = db_get_boolean("relative-paths", 1);
121 int absPathOption = find_option("abs-paths", 0, 0)!=0;
122 int relPathOption = find_option("rel-paths", 0, 0)!=0;
123 if( absPathOption ){ relativePaths = 0; }
124 if( relPathOption ){ relativePaths = 1; }
125 return relativePaths;
126 }
127
128 /*
129 ** COMMAND: changes
130 **
131 ** Usage: %fossil changes
132 **
133 ** Report on the edit status of all files in the current checkout.
134 ** See also the "status" and "extra" commands.
135 **
136 ** Pathnames are displayed according to the "relative-paths" setting,
137 ** unless overridden by the --abs-paths or --rel-paths options.
138 **
139 ** Options:
140 **
141 ** --sha1sum Verify file status using SHA1 hashing rather
142 ** than relying on file mtimes.
143 **
144 ** --abs-paths Display absolute pathnames.
145 **
146 ** --rel-paths Display pathnames relative to the current working
147 ** directory.
148 */
149 void changes_cmd(void){
150 Blob report;
151 int vid;
152 int useSha1sum = find_option("sha1sum", 0, 0)!=0;
153 int cwdRelative = 0;
154 db_must_be_within_tree();
155 cwdRelative = determine_cwd_relative_option();
156 blob_zero(&report);
157 vid = db_lget_int("checkout", 0);
158 vfile_check_signature(vid, 0, useSha1sum);
159 status_report(&report, "", 0, cwdRelative);
160 blob_write_to_file(&report, "-");
161 }
162
163 /*
164 ** COMMAND: status
@@ -127,14 +165,22 @@
165 **
166 ** Usage: %fossil status
167 **
168 ** Report on the status of the current checkout.
169 **
170 ** Pathnames are displayed according to the "relative-paths" setting,
171 ** unless overridden by the --abs-paths or --rel-paths options.
172 **
173 ** Options:
174 **
175 ** --sha1sum Verify file status using SHA1 hashing rather
176 ** than relying on file mtimes.
177 **
178 ** --abs-paths Display absolute pathnames.
179 **
180 ** --rel-paths Display pathnames relative to the current working
181 ** directory.
182 */
183 void status_cmd(void){
184 int vid;
185 db_must_be_within_tree();
186 /* 012345678901234 */
@@ -212,22 +258,41 @@
258 ** ignored but can be included by adding the --dotfiles option.
259 **
260 ** The GLOBPATTERN is a comma-separated list of GLOB expressions for
261 ** files that are ignored. The GLOBPATTERN specified by the "ignore-glob"
262 ** is used if the --ignore option is omitted.
263 **
264 ** Pathnames are displayed according to the "relative-paths" setting,
265 ** unless overridden by the --abs-paths or --rel-paths options.
266 **
267 ** Options:
268 **
269 ** --dotfiles Include files with names beginning with "."
270 **
271 ** --ignore GLOBPATTERN
272 ** Override the "ignore-glob" setting.
273 **
274 ** --abs-paths Display absolute pathnames.
275 **
276 ** --rel-paths Display pathnames relative to the current working
277 ** directory.
278 */
279 void extra_cmd(void){
280 Blob path;
281 Blob repo;
282 Stmt q;
283 int n;
284 const char *zIgnoreFlag = find_option("ignore",0,1);
285 int allFlag = find_option("dotfiles",0,0)!=0;
286 int cwdRelative = 0;
287 int outputManifest;
288 Glob *pIgnore;
289 Blob rewrittenPathname;
290 const char *zPathname, *zDisplayName;
291
292 db_must_be_within_tree();
293 cwdRelative = determine_cwd_relative_option();
294 outputManifest = db_get_boolean("manifest",0);
295 db_multi_exec("CREATE TEMP TABLE sfile(x TEXT PRIMARY KEY)");
296 n = strlen(g.zLocalRoot);
297 blob_init(&path, g.zLocalRoot, n-1);
298 if( zIgnoreFlag==0 ){
@@ -243,13 +308,25 @@
308 fossil_all_reserved_names()
309 );
310 if( file_tree_name(g.zRepositoryName, &repo, 0) ){
311 db_multi_exec("DELETE FROM sfile WHERE x=%B", &repo);
312 }
313 blob_zero(&rewrittenPathname);
314 while( db_step(&q)==SQLITE_ROW ){
315 zDisplayName = zPathname = db_column_text(&q, 0);
316 if( cwdRelative ) {
317 char *zFullName = mprintf("%s%s", g.zLocalRoot, zPathname);
318 file_relative_name(zFullName, &rewrittenPathname);
319 free(zFullName);
320 zDisplayName = blob_str(&rewrittenPathname);
321 if( zDisplayName[0]=='.' && zDisplayName[1]=='/' ){
322 zDisplayName += 2; /* no unnecessary ./ prefix */
323 }
324 }
325 fossil_print("%s\n", zDisplayName);
326 }
327 blob_reset(&rewrittenPathname);
328 db_finalize(&q);
329 }
330
331 /*
332 ** COMMAND: clean
@@ -367,11 +444,11 @@
444 "# PRIVATE BRANCH: This check-in will be private and will not sync to\n"
445 "# repositories.\n"
446 "#\n", -1
447 );
448 }
449 status_report(&text, "# ", 1, 0);
450 zEditor = db_get("editor", 0);
451 if( zEditor==0 ){
452 zEditor = getenv("VISUAL");
453 }
454 if( zEditor==0 ){
455
M src/checkout.c
+1
--- src/checkout.c
+++ src/checkout.c
@@ -232,10 +232,11 @@
232232
 if( !keepFlag ){
233233
 vfile_to_disk(vid, 0, 1, promptFlag);
234234
 }
235235
 checkout_set_all_exe(vid);
236236
 manifest_to_disk(vid);
237
+ ensure_empty_dirs_created();
237238
 db_lset_int("checkout", vid);
238239
 undo_reset();
239240
 db_multi_exec("DELETE FROM vmerge");
240241
 if( !keepFlag && db_get_boolean("repo-cksum",1) ){
241242
 vfile_aggregate_checksum_manifest(vid, &cksum1, &cksum1b);
242243
--- src/checkout.c
+++ src/checkout.c
@@ -232,10 +232,11 @@
232 if( !keepFlag ){
233 vfile_to_disk(vid, 0, 1, promptFlag);
234 }
235 checkout_set_all_exe(vid);
236 manifest_to_disk(vid);
 
237 db_lset_int("checkout", vid);
238 undo_reset();
239 db_multi_exec("DELETE FROM vmerge");
240 if( !keepFlag && db_get_boolean("repo-cksum",1) ){
241 vfile_aggregate_checksum_manifest(vid, &cksum1, &cksum1b);
242
--- src/checkout.c
+++ src/checkout.c
@@ -232,10 +232,11 @@
232 if( !keepFlag ){
233 vfile_to_disk(vid, 0, 1, promptFlag);
234 }
235 checkout_set_all_exe(vid);
236 manifest_to_disk(vid);
237 ensure_empty_dirs_created();
238 db_lset_int("checkout", vid);
239 undo_reset();
240 db_multi_exec("DELETE FROM vmerge");
241 if( !keepFlag && db_get_boolean("repo-cksum",1) ){
242 vfile_aggregate_checksum_manifest(vid, &cksum1, &cksum1b);
243
M src/clone.c
+17 -3
--- src/clone.c
+++ src/clone.c
@@ -37,15 +37,17 @@
3737
 **
3838
 ** Options:
3939
 **
4040
 ** --admin-user|-A USERNAME Make USERNAME the administrator
4141
 ** --private Also clone private branches
42
+** --ssl-identity=filename Use the SSL identity if requested by the server
4243
 **
4344
 */
4445
 void clone_cmd(void){
4546
 char *zPassword;
4647
 const char *zDefaultUser; /* Optional name of the default user */
48
+ const char *zPw; /* The user clone password */
4749
 int nErr = 0;
4850
 int bPrivate; /* Also clone private branches */
4951
5052
 bPrivate = find_option("private",0,0)!=0;
5153
 url_proxy_options();
@@ -76,13 +78,15 @@
7678
 "DELETE FROM blob WHERE rid IN private;"
7779
 "DELETE FROM delta wHERE rid IN private;"
7880
 "DELETE FROM private;"
7981
 );
8082
 shun_artifacts();
81
- g.zLogin = db_text(0, "SELECT login FROM user WHERE cap LIKE '%%s%%'");
82
- if( g.zLogin==0 ){
83
- db_create_default_users(1,zDefaultUser);
83
+ db_create_default_users(1, zDefaultUser);
84
+ if( zDefaultUser ){
85
+ g.zLogin = zDefaultUser;
86
+ }else{
87
+ g.zLogin = db_text(0, "SELECT login FROM user WHERE cap LIKE '%%s%%'");
8488
 }
8589
 fossil_print("Repository cloned into %s\n", g.argv[3]);
8690
 }else{
8791
 db_create_repository(g.argv[3]);
8892
 db_open_repository(g.argv[3]);
@@ -91,10 +95,18 @@
9195
 db_initial_setup(0, zDefaultUser, 0);
9296
 user_select();
9397
 db_set("content-schema", CONTENT_SCHEMA, 0);
9498
 db_set("aux-schema", AUX_SCHEMA, 0);
9599
 db_set("last-sync-url", g.argv[2], 0);
100
+ if( g.zSSLIdentity!=0 ){
101
+ /* If the --ssl-identity option was specified, store it as a setting */
102
+ Blob fn;
103
+ blob_zero(&fn);
104
+ file_canonical_name(g.zSSLIdentity, &fn);
105
+ db_set("ssl-identity", blob_str(&fn), 0);
106
+ blob_reset(&fn);
107
+ }
96108
 db_multi_exec(
97109
 "REPLACE INTO config(name,value,mtime)"
98110
 " VALUES('server-code', lower(hex(randomblob(20))), now());"
99111
 );
100112
 url_enable_proxy(0);
@@ -116,7 +128,9 @@
116128
 rebuild_db(0, 1, 0);
117129
 fossil_print("project-id: %s\n", db_get("project-code", 0));
118130
 fossil_print("server-id: %s\n", db_get("server-code", 0));
119131
 zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
120132
 fossil_print("admin-user: %s (password is \"%s\")\n", g.zLogin, zPassword);
133
+ zPw = g.urlPasswd;
134
+ if( !g.dontKeepUrl && zPw) db_set("last-sync-pw", obscure(zPw), 0);
121135
 db_end_transaction(0);
122136
 }
123137
--- src/clone.c
+++ src/clone.c
@@ -37,15 +37,17 @@
37 **
38 ** Options:
39 **
40 ** --admin-user|-A USERNAME Make USERNAME the administrator
41 ** --private Also clone private branches
 
42 **
43 */
44 void clone_cmd(void){
45 char *zPassword;
46 const char *zDefaultUser; /* Optional name of the default user */
 
47 int nErr = 0;
48 int bPrivate; /* Also clone private branches */
49
50 bPrivate = find_option("private",0,0)!=0;
51 url_proxy_options();
@@ -76,13 +78,15 @@
76 "DELETE FROM blob WHERE rid IN private;"
77 "DELETE FROM delta wHERE rid IN private;"
78 "DELETE FROM private;"
79 );
80 shun_artifacts();
81 g.zLogin = db_text(0, "SELECT login FROM user WHERE cap LIKE '%%s%%'");
82 if( g.zLogin==0 ){
83 db_create_default_users(1,zDefaultUser);
 
 
84 }
85 fossil_print("Repository cloned into %s\n", g.argv[3]);
86 }else{
87 db_create_repository(g.argv[3]);
88 db_open_repository(g.argv[3]);
@@ -91,10 +95,18 @@
91 db_initial_setup(0, zDefaultUser, 0);
92 user_select();
93 db_set("content-schema", CONTENT_SCHEMA, 0);
94 db_set("aux-schema", AUX_SCHEMA, 0);
95 db_set("last-sync-url", g.argv[2], 0);
 
 
 
 
 
 
 
 
96 db_multi_exec(
97 "REPLACE INTO config(name,value,mtime)"
98 " VALUES('server-code', lower(hex(randomblob(20))), now());"
99 );
100 url_enable_proxy(0);
@@ -116,7 +128,9 @@
116 rebuild_db(0, 1, 0);
117 fossil_print("project-id: %s\n", db_get("project-code", 0));
118 fossil_print("server-id: %s\n", db_get("server-code", 0));
119 zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
120 fossil_print("admin-user: %s (password is \"%s\")\n", g.zLogin, zPassword);
 
 
121 db_end_transaction(0);
122 }
123
--- src/clone.c
+++ src/clone.c
@@ -37,15 +37,17 @@
37 **
38 ** Options:
39 **
40 ** --admin-user|-A USERNAME Make USERNAME the administrator
41 ** --private Also clone private branches
42 ** --ssl-identity=filename Use the SSL identity if requested by the server
43 **
44 */
45 void clone_cmd(void){
46 char *zPassword;
47 const char *zDefaultUser; /* Optional name of the default user */
48 const char *zPw; /* The user clone password */
49 int nErr = 0;
50 int bPrivate; /* Also clone private branches */
51
52 bPrivate = find_option("private",0,0)!=0;
53 url_proxy_options();
@@ -76,13 +78,15 @@
78 "DELETE FROM blob WHERE rid IN private;"
79 "DELETE FROM delta wHERE rid IN private;"
80 "DELETE FROM private;"
81 );
82 shun_artifacts();
83 db_create_default_users(1, zDefaultUser);
84 if( zDefaultUser ){
85 g.zLogin = zDefaultUser;
86 }else{
87 g.zLogin = db_text(0, "SELECT login FROM user WHERE cap LIKE '%%s%%'");
88 }
89 fossil_print("Repository cloned into %s\n", g.argv[3]);
90 }else{
91 db_create_repository(g.argv[3]);
92 db_open_repository(g.argv[3]);
@@ -91,10 +95,18 @@
95 db_initial_setup(0, zDefaultUser, 0);
96 user_select();
97 db_set("content-schema", CONTENT_SCHEMA, 0);
98 db_set("aux-schema", AUX_SCHEMA, 0);
99 db_set("last-sync-url", g.argv[2], 0);
100 if( g.zSSLIdentity!=0 ){
101 /* If the --ssl-identity option was specified, store it as a setting */
102 Blob fn;
103 blob_zero(&fn);
104 file_canonical_name(g.zSSLIdentity, &fn);
105 db_set("ssl-identity", blob_str(&fn), 0);
106 blob_reset(&fn);
107 }
108 db_multi_exec(
109 "REPLACE INTO config(name,value,mtime)"
110 " VALUES('server-code', lower(hex(randomblob(20))), now());"
111 );
112 url_enable_proxy(0);
@@ -116,7 +128,9 @@
128 rebuild_db(0, 1, 0);
129 fossil_print("project-id: %s\n", db_get("project-code", 0));
130 fossil_print("server-id: %s\n", db_get("server-code", 0));
131 zPassword = db_text(0, "SELECT pw FROM user WHERE login=%Q", g.zLogin);
132 fossil_print("admin-user: %s (password is \"%s\")\n", g.zLogin, zPassword);
133 zPw = g.urlPasswd;
134 if( !g.dontKeepUrl && zPw) db_set("last-sync-pw", obscure(zPw), 0);
135 db_end_transaction(0);
136 }
137
M src/config.h
+5 -1
--- src/config.h
+++ src/config.h
@@ -24,10 +24,14 @@
2424
 #define _LARGE_FILE 1
2525
 #ifndef _FILE_OFFSET_BITS
2626
 # define _FILE_OFFSET_BITS 64
2727
 #endif
2828
 #define _LARGEFILE_SOURCE 1
29
+
30
+#ifdef HAVE_AUTOCONFIG_H
31
+#include "autoconfig.h"
32
+#endif
2933
3034
 #ifndef _RC_COMPILE_
3135
3236
 /*
3337
 ** System header files used by all modules
@@ -85,11 +89,11 @@
8589
 #include "sqlite3.h"
8690
8791
 /*
8892
 ** On Solaris, getpass() will only return up to 8 characters. getpassphrase() returns up to 257.
8993
 */
90
-#if defined(__sun__) || defined(sun)
94
+#if HAVE_GETPASSPHRASE
9195
 #define getpass getpassphrase
9296
 #endif
9397
9498
 /*
9599
 ** Typedef for a 64-bit integer
96100
--- src/config.h
+++ src/config.h
@@ -24,10 +24,14 @@
24 #define _LARGE_FILE 1
25 #ifndef _FILE_OFFSET_BITS
26 # define _FILE_OFFSET_BITS 64
27 #endif
28 #define _LARGEFILE_SOURCE 1
 
 
 
 
29
30 #ifndef _RC_COMPILE_
31
32 /*
33 ** System header files used by all modules
@@ -85,11 +89,11 @@
85 #include "sqlite3.h"
86
87 /*
88 ** On Solaris, getpass() will only return up to 8 characters. getpassphrase() returns up to 257.
89 */
90 #if defined(__sun__) || defined(sun)
91 #define getpass getpassphrase
92 #endif
93
94 /*
95 ** Typedef for a 64-bit integer
96
--- src/config.h
+++ src/config.h
@@ -24,10 +24,14 @@
24 #define _LARGE_FILE 1
25 #ifndef _FILE_OFFSET_BITS
26 # define _FILE_OFFSET_BITS 64
27 #endif
28 #define _LARGEFILE_SOURCE 1
29
30 #ifdef HAVE_AUTOCONFIG_H
31 #include "autoconfig.h"
32 #endif
33
34 #ifndef _RC_COMPILE_
35
36 /*
37 ** System header files used by all modules
@@ -85,11 +89,11 @@
89 #include "sqlite3.h"
90
91 /*
92 ** On Solaris, getpass() will only return up to 8 characters. getpassphrase() returns up to 257.
93 */
94 #if HAVE_GETPASSPHRASE
95 #define getpass getpassphrase
96 #endif
97
98 /*
99 ** Typedef for a 64-bit integer
100
M src/configure.c
+1
--- src/configure.c
+++ src/configure.c
@@ -81,10 +81,11 @@
8181
 { "project-name", CONFIGSET_PROJ },
8282
 { "project-description", CONFIGSET_PROJ },
8383
 { "manifest", CONFIGSET_PROJ },
8484
 { "ignore-glob", CONFIGSET_PROJ },
8585
 { "crnl-glob", CONFIGSET_PROJ },
86
+ { "empty-dirs", CONFIGSET_PROJ },
8687
 { "index-page", CONFIGSET_SKIN },
8788
 { "timeline-block-markup", CONFIGSET_SKIN },
8889
 { "timeline-max-comment", CONFIGSET_SKIN },
8990
 { "ticket-table", CONFIGSET_TKT },
9091
 { "ticket-common", CONFIGSET_TKT },
9192
--- src/configure.c
+++ src/configure.c
@@ -81,10 +81,11 @@
81 { "project-name", CONFIGSET_PROJ },
82 { "project-description", CONFIGSET_PROJ },
83 { "manifest", CONFIGSET_PROJ },
84 { "ignore-glob", CONFIGSET_PROJ },
85 { "crnl-glob", CONFIGSET_PROJ },
 
86 { "index-page", CONFIGSET_SKIN },
87 { "timeline-block-markup", CONFIGSET_SKIN },
88 { "timeline-max-comment", CONFIGSET_SKIN },
89 { "ticket-table", CONFIGSET_TKT },
90 { "ticket-common", CONFIGSET_TKT },
91
--- src/configure.c
+++ src/configure.c
@@ -81,10 +81,11 @@
81 { "project-name", CONFIGSET_PROJ },
82 { "project-description", CONFIGSET_PROJ },
83 { "manifest", CONFIGSET_PROJ },
84 { "ignore-glob", CONFIGSET_PROJ },
85 { "crnl-glob", CONFIGSET_PROJ },
86 { "empty-dirs", CONFIGSET_PROJ },
87 { "index-page", CONFIGSET_SKIN },
88 { "timeline-block-markup", CONFIGSET_SKIN },
89 { "timeline-max-comment", CONFIGSET_SKIN },
90 { "ticket-table", CONFIGSET_TKT },
91 { "ticket-common", CONFIGSET_TKT },
92
M src/db.c
+180 -52
--- src/db.c
+++ src/db.c
@@ -153,13 +153,17 @@
153153
 ** Force a rollback and shutdown the database
154154
 */
155155
 void db_force_rollback(void){
156156
 int i;
157157
 static int busy = 0;
158
+ sqlite3_stmt *pStmt = 0;
158159
 if( busy || g.db==0 ) return;
159160
 busy = 1;
160161
 undo_rollback();
162
+ while( (pStmt = sqlite3_next_stmt(g.db,pStmt))!=0 ){
163
+ sqlite3_reset(pStmt);
164
+ }
161165
 while( pAllStmt ){
162166
 db_finalize(pAllStmt);
163167
 }
164168
 if( nBegin ){
165169
 sqlite3_exec(g.db, "ROLLBACK", 0, 0, 0);
@@ -809,10 +813,11 @@
809813
 static const char *aDbName[] = { "/_FOSSIL_", "/.fos" };
810814
811815
 if( g.localOpen) return 1;
812816
 file_getcwd(zPwd, sizeof(zPwd)-20);
813817
 n = strlen(zPwd);
818
+ if( n==1 && zPwd[0]=='/' ) zPwd[0] = '.';
814819
 while( n>0 ){
815820
 if( file_access(zPwd, W_OK) ) break;
816821
 for(i=0; i<sizeof(aDbName)/sizeof(aDbName[0]); i++){
817822
 sqlite3_snprintf(sizeof(zPwd)-n, &zPwd[n], "%s", aDbName[i]);
818823
 if( isValidLocalDb(zPwd) ){
@@ -1058,14 +1063,13 @@
10581063
10591064
 /*
10601065
 ** Create the default user accounts in the USER table.
10611066
 */
10621067
 void db_create_default_users(int setupUserOnly, const char *zDefaultUser){
1063
- const char *zUser;
1064
- zUser = db_get("default-user", 0);
1068
+ const char *zUser = zDefaultUser;
10651069
 if( zUser==0 ){
1066
- zUser = zDefaultUser;
1070
+ zUser = db_get("default-user", 0);
10671071
 }
10681072
 if( zUser==0 ){
10691073
 #if defined(_WIN32)
10701074
 zUser = getenv("USERNAME");
10711075
 #else
@@ -1074,12 +1078,15 @@
10741078
 }
10751079
 if( zUser==0 ){
10761080
 zUser = "root";
10771081
 }
10781082
 db_multi_exec(
1079
- "INSERT INTO user(login, pw, cap, info)"
1080
- "VALUES(%Q,lower(hex(randomblob(3))),'s','')", zUser
1083
+ "INSERT OR IGNORE INTO user(login, info) VALUES(%Q,'')", zUser
1084
+ );
1085
+ db_multi_exec(
1086
+ "UPDATE user SET cap='s', pw=lower(hex(randomblob(3)))"
1087
+ " WHERE login=%Q", zUser
10811088
 );
10821089
 if( !setupUserOnly ){
10831090
 db_multi_exec(
10841091
 "INSERT INTO user(login,pw,cap,info)"
10851092
 " VALUES('anonymous',hex(randomblob(8)),'hmncz','Anon');"
@@ -1363,19 +1370,19 @@
13631370
 */
13641371
 int is_truth(const char *zVal){
13651372
 static const char *azOn[] = { "on", "yes", "true", "1" };
13661373
 int i;
13671374
 for(i=0; i<sizeof(azOn)/sizeof(azOn[0]); i++){
1368
- if( fossil_strcmp(zVal,azOn[i])==0 ) return 1;
1375
+ if( fossil_stricmp(zVal,azOn[i])==0 ) return 1;
13691376
 }
13701377
 return 0;
13711378
 }
13721379
 int is_false(const char *zVal){
13731380
 static const char *azOff[] = { "off", "no", "false", "0" };
13741381
 int i;
13751382
 for(i=0; i<sizeof(azOff)/sizeof(azOff[0]); i++){
1376
- if( fossil_strcmp(zVal,azOff[i])==0 ) return 1;
1383
+ if( fossil_stricmp(zVal,azOff[i])==0 ) return 1;
13771384
 }
13781385
 return 0;
13791386
 }
13801387
13811388
 /*
@@ -1392,24 +1399,93 @@
13921399
 sqlite3 *dbTemp = g.db;
13931400
 g.db = g.dbConfig;
13941401
 g.dbConfig = dbTemp;
13951402
 }
13961403
 }
1404
+
1405
+/*
1406
+** Logic for reading potentially versioned settings from
1407
+** .fossil-settings/<name> , and emits warnings if necessary.
1408
+** Returns the non-versioned value without modification if there is no
1409
+** versioned value.
1410
+*/
1411
+static char *db_get_do_versionable(const char *zName, char *zNonVersionedSetting){
1412
+ /* Attempt to load the versioned setting from a checked out file */
1413
+ char *zVersionedSetting = 0;
1414
+ int noWarn = 0;
1415
+
1416
+ if( db_open_local() ){
1417
+ Blob versionedPathname;
1418
+ char *zVersionedPathname;
1419
+ blob_zero(&versionedPathname);
1420
+ blob_appendf(&versionedPathname, "%s/.fossil-settings/%s",
1421
+ g.zLocalRoot, zName);
1422
+ zVersionedPathname = blob_str(&versionedPathname);
1423
+ if( file_size(zVersionedPathname)>=0 ){
1424
+ /* File exists, and contains the value for this setting. Load from
1425
+ ** the file. */
1426
+ Blob setting;
1427
+ blob_zero(&setting);
1428
+ if( blob_read_from_file(&setting, zVersionedPathname) >= 0 ){
1429
+ blob_trim(&setting); /* Avoid non-obvious problems with line endings
1430
+ ** on boolean properties */
1431
+ zVersionedSetting = strdup(blob_str(&setting));
1432
+ }
1433
+ blob_reset(&setting);
1434
+ /* See if there's a no-warn flag */
1435
+ blob_append(&versionedPathname, ".no-warn", -1);
1436
+ if( file_size(blob_str(&versionedPathname))>=0 ){
1437
+ noWarn = 1;
1438
+ }
1439
+ }
1440
+ blob_reset(&versionedPathname);
1441
+ }
1442
+ /* Display a warning? */
1443
+ if( zVersionedSetting!=0 && zNonVersionedSetting!=0
1444
+ && zNonVersionedSetting[0]!='\0' && !noWarn
1445
+ ){
1446
+ /* There's a versioned setting, and a non-versioned setting. Tell
1447
+ ** the user about the conflict */
1448
+ fossil_warning(
1449
+ "setting %s has both versioned and non-versioned values: using "
1450
+ "versioned value from file .fossil-settings/%s (to silence this "
1451
+ "warning, either create an empty file named "
1452
+ ".fossil-settings/%s.no-warn or delete the non-versioned setting "
1453
+ " with \"fossil unset %s\")", zName, zName, zName, zName
1454
+ );
1455
+ }
1456
+ /* Prefer the versioned setting */
1457
+ return ( zVersionedSetting!=0 ) ? zVersionedSetting : zNonVersionedSetting;
1458
+}
1459
+
13971460
13981461
 /*
13991462
 ** Get and set values from the CONFIG, GLOBAL_CONFIG and VVAR table in the
14001463
 ** repository and local databases.
14011464
 */
14021465
 char *db_get(const char *zName, char *zDefault){
14031466
 char *z = 0;
1467
+ int i;
1468
+ const struct stControlSettings *ctrlSetting = 0;
1469
+ /* Is this a setting? */
1470
+ for(i=0; ctrlSettings[i].name; i++){
1471
+ if( strcmp(ctrlSettings[i].name, zName)==0 ){
1472
+ ctrlSetting = &(ctrlSettings[i]);
1473
+ break;
1474
+ }
1475
+ }
14041476
 if( g.repositoryOpen ){
14051477
 z = db_text(0, "SELECT value FROM config WHERE name=%Q", zName);
14061478
 }
14071479
 if( z==0 && g.configOpen ){
14081480
 db_swap_connections();
14091481
 z = db_text(0, "SELECT value FROM global_config WHERE name=%Q", zName);
14101482
 db_swap_connections();
1483
+ }
1484
+ if( ctrlSetting!=0 && ctrlSetting->versionable ){
1485
+ /* This is a versionable setting, try and get the info from a checked out file */
1486
+ z = db_get_do_versionable(zName, z);
14111487
 }
14121488
 if( z==0 ){
14131489
 z = zDefault;
14141490
 }
14151491
 return z;
@@ -1596,30 +1672,39 @@
15961672
 }
15971673
15981674
 /*
15991675
 ** Print the value of a setting named zName
16001676
 */
1601
-static void print_setting(const char *zName){
1677
+static void print_setting(const struct stControlSettings *ctrlSetting, int localOpen){
16021678
 Stmt q;
16031679
 if( g.repositoryOpen ){
16041680
 db_prepare(&q,
16051681
 "SELECT '(local)', value FROM config WHERE name=%Q"
16061682
 " UNION ALL "
16071683
 "SELECT '(global)', value FROM global_config WHERE name=%Q",
1608
- zName, zName
1684
+ ctrlSetting->name, ctrlSetting->name
16091685
 );
16101686
 }else{
16111687
 db_prepare(&q,
16121688
 "SELECT '(global)', value FROM global_config WHERE name=%Q",
1613
- zName
1689
+ ctrlSetting->name
16141690
 );
16151691
 }
16161692
 if( db_step(&q)==SQLITE_ROW ){
1617
- fossil_print("%-20s %-8s %s\n", zName, db_column_text(&q, 0),
1693
+ fossil_print("%-20s %-8s %s\n", ctrlSetting->name, db_column_text(&q, 0),
16181694
 db_column_text(&q, 1));
16191695
 }else{
1620
- fossil_print("%-20s\n", zName);
1696
+ fossil_print("%-20s\n", ctrlSetting->name);
1697
+ }
1698
+ if( ctrlSetting->versionable && localOpen ){
1699
+ /* Check to see if this is overridden by a versionable settings file */
1700
+ Blob versionedPathname;
1701
+ blob_zero(&versionedPathname);
1702
+ blob_appendf(&versionedPathname, "%s/.fossil-settings/%s", g.zLocalRoot, ctrlSetting->name);
1703
+ if( file_size(blob_str(&versionedPathname))>=0 ){
1704
+ fossil_print(" (overridden by contents of file .fossil-settings/%s)\n", ctrlSetting->name);
1705
+ }
16211706
 }
16221707
 db_finalize(&q);
16231708
 }
16241709
16251710
@@ -1635,42 +1720,49 @@
16351720
 #if INTERFACE
16361721
 struct stControlSettings {
16371722
 char const *name; /* Name of the setting */
16381723
 char const *var; /* Internal variable name used by db_set() */
16391724
 int width; /* Width of display. 0 for boolean values */
1725
+ int versionable; /* Is this setting versionable? */
16401726
 char const *def; /* Default value */
16411727
 };
16421728
 #endif /* INTERFACE */
16431729
 struct stControlSettings const ctrlSettings[] = {
1644
- { "access-log", 0, 0, "off" },
1645
- { "auto-captcha", "autocaptcha", 0, "on" },
1646
- { "auto-shun", 0, 0, "on" },
1647
- { "autosync", 0, 0, "on" },
1648
- { "binary-glob", 0, 32, "" },
1649
- { "clearsign", 0, 0, "off" },
1650
- { "crnl-glob", 0, 16, "" },
1651
- { "default-perms", 0, 16, "u" },
1652
- { "diff-command", 0, 16, "" },
1653
- { "dont-push", 0, 0, "off" },
1654
- { "editor", 0, 16, "" },
1655
- { "gdiff-command", 0, 16, "gdiff" },
1656
- { "gmerge-command",0, 40, "" },
1657
- { "https-login", 0, 0, "off" },
1658
- { "ignore-glob", 0, 40, "" },
1659
- { "http-port", 0, 16, "8080" },
1660
- { "localauth", 0, 0, "off" },
1661
- { "main-branch", 0, 40, "trunk" },
1662
- { "manifest", 0, 0, "off" },
1663
- { "max-upload", 0, 25, "250000" },
1664
- { "mtime-changes", 0, 0, "on" },
1665
- { "pgp-command", 0, 32, "gpg --clearsign -o " },
1666
- { "proxy", 0, 32, "off" },
1667
- { "repo-cksum", 0, 0, "on" },
1668
- { "self-register", 0, 0, "off" },
1669
- { "ssh-command", 0, 32, "" },
1670
- { "web-browser", 0, 32, "" },
1671
- { 0,0,0,0 }
1730
+ { "access-log", 0, 0, 0, "off" },
1731
+ { "auto-captcha", "autocaptcha", 0, 0, "on" },
1732
+ { "auto-shun", 0, 0, 0, "on" },
1733
+ { "autosync", 0, 0, 0, "on" },
1734
+ { "binary-glob", 0, 32, 1, "" },
1735
+ { "clearsign", 0, 0, 0, "off" },
1736
+ { "case-sensitive",0, 0, 0, "on" },
1737
+ { "crnl-glob", 0, 16, 1, "" },
1738
+ { "default-perms", 0, 16, 0, "u" },
1739
+ { "diff-command", 0, 16, 0, "" },
1740
+ { "dont-push", 0, 0, 0, "off" },
1741
+ { "editor", 0, 16, 0, "" },
1742
+ { "gdiff-command", 0, 16, 0, "gdiff" },
1743
+ { "gmerge-command",0, 40, 0, "" },
1744
+ { "https-login", 0, 0, 0, "off" },
1745
+ { "ignore-glob", 0, 40, 1, "" },
1746
+ { "empty-dirs", 0, 40, 1, "" },
1747
+ { "http-port", 0, 16, 0, "8080" },
1748
+ { "localauth", 0, 0, 0, "off" },
1749
+ { "main-branch", 0, 40, 0, "trunk" },
1750
+ { "manifest", 0, 0, 1, "off" },
1751
+ { "max-upload", 0, 25, 0, "250000" },
1752
+ { "mtime-changes", 0, 0, 0, "on" },
1753
+ { "pgp-command", 0, 32, 0, "gpg --clearsign -o " },
1754
+ { "proxy", 0, 32, 0, "off" },
1755
+ { "relative-paths",0, 0, 0, "on" },
1756
+ { "repo-cksum", 0, 0, 0, "on" },
1757
+ { "self-register", 0, 0, 0, "off" },
1758
+ { "ssl-ca-location",0, 40, 0, "" },
1759
+ { "ssl-identity", 0, 40, 0, "" },
1760
+ { "ssh-command", 0, 32, 0, "" },
1761
+ { "web-browser", 0, 32, 0, "" },
1762
+ { "white-foreground", 0, 0, 0, "off" },
1763
+ { 0,0,0,0,0 }
16721764
 };
16731765
16741766
 /*
16751767
 ** COMMAND: settings
16761768
 ** COMMAND: unset
@@ -1679,10 +1771,14 @@
16791771
 ** %fossil unset PROPERTY ?-global?
16801772
 **
16811773
 ** The "settings" command with no arguments lists all properties and their
16821774
 ** values. With just a property name it shows the value of that property.
16831775
 ** With a value argument it changes the property for the current repository.
1776
+**
1777
+** Settings marked as versionable are overridden by the contents of the
1778
+** file named .fossil-settings/PROPERTY in the checked out files, if that
1779
+** file exists.
16841780
 **
16851781
 ** The "unset" command clears a property setting.
16861782
 **
16871783
 **
16881784
 ** auto-captcha If enabled, the Login page provides a button to
@@ -1696,20 +1792,25 @@
16961792
 ** or update and automatically push after commit or
16971793
 ** tag or branch creation. If the value is "pullonly"
16981794
 ** then only pull operations occur automatically.
16991795
 ** Default: on
17001796
 **
1701
-** binary-glob The VALUE is a comma-separated list of GLOB patterns
1702
-** that should be treated as binary files for merging
1703
-** purposes. Example: *.xml
1797
+** binary-glob The VALUE is a comma or newline-separated list of
1798
+** (versionable) GLOB patterns that should be treated as binary files
1799
+** for merging purposes. Example: *.xml
1800
+**
1801
+** case-sensitive If TRUE, the files whose names differ only in case
1802
+** care considered distinct. If FALSE files whose names
1803
+** differ only in case are the same file. Defaults to
1804
+** TRUE for unix and FALSE for windows and mac.
17041805
 **
17051806
 ** clearsign When enabled, fossil will attempt to sign all commits
17061807
 ** with gpg. When disabled (the default), commits will
17071808
 ** be unsigned. Default: off
17081809
 **
1709
-** crnl-glob A comma-separated list of GLOB patterns for text files
1710
-** in which it is ok to have CR+NL line endings.
1810
+** crnl-glob A comma or newline-separated list of GLOB patterns for
1811
+** (versionable) text files in which it is ok to have CR+NL line endings.
17111812
 ** Set to "*" to disable CR+NL checking.
17121813
 **
17131814
 ** default-perms Permissions given automatically to new users. For more
17141815
 ** information on permissions see Users page in Server
17151816
 ** Administration of the HTTP UI. Default: u.
@@ -1717,10 +1818,15 @@
17171818
 ** diff-command External command to run when performing a diff.
17181819
 ** If undefined, the internal text diff will be used.
17191820
 **
17201821
 ** dont-push Prevent this repository from pushing from client to
17211822
 ** server. Useful when setting up a private branch.
1823
+**
1824
+** empty-dirs A comma or newline-separated list of pathnames. On
1825
+** (versionable) update and checkout commands, if no file or directory
1826
+** exists with that name, an empty directory will be
1827
+** created.
17221828
 **
17231829
 ** editor Text editor command used for check-in comments.
17241830
 **
17251831
 ** gdiff-command External command to run when performing a graphical
17261832
 ** diff. If undefined, text diff will be used.
@@ -1735,23 +1841,23 @@
17351841
 ** and "ui" commands. Default: 8080
17361842
 **
17371843
 ** https-login Send login creditials using HTTPS instead of HTTP
17381844
 ** even if the login page request came via HTTP.
17391845
 **
1740
-** ignore-glob The VALUE is a comma-separated list of GLOB patterns
1741
-** specifying files that the "extra" command will ignore.
1742
-** Example: *.o,*.obj,*.exe
1846
+** ignore-glob The VALUE is a comma or newline-separated list of GLOB
1847
+** (versionable) patterns specifying files that the "extra" command will
1848
+** ignore. Example: *.o,*.obj,*.exe
17431849
 **
17441850
 ** localauth If enabled, require that HTTP connections from
17451851
 ** 127.0.0.1 be authenticated by password. If
17461852
 ** false, all HTTP requests from localhost have
17471853
 ** unrestricted access to the repository.
17481854
 **
17491855
 ** main-branch The primary branch for the project. Default: trunk
17501856
 **
17511857
 ** manifest If enabled, automatically create files "manifest" and
1752
-** "manifest.uuid" in every checkout. The SQLite and
1858
+** (versionable) "manifest.uuid" in every checkout. The SQLite and
17531859
 ** Fossil repositories both require this. Default: off.
17541860
 **
17551861
 ** max-upload A limit on the size of uplink HTTP requests. The
17561862
 ** default is 250000 bytes.
17571863
 **
@@ -1763,10 +1869,13 @@
17631869
 **
17641870
 ** proxy URL of the HTTP proxy. If undefined or "off" then
17651871
 ** the "http_proxy" environment variable is consulted.
17661872
 ** If the http_proxy environment variable is undefined
17671873
 ** then a direct HTTP connection is used.
1874
+**
1875
+** relative-paths When showing changes and extras, report paths relative
1876
+** to the current working directory. Default: "on"
17681877
 **
17691878
 ** repo-cksum Compute checksums over all files in each checkout
17701879
 ** as a double-check of correctness. Defaults to "on".
17711880
 ** Disable on large repositories for a performance
17721881
 ** improvement.
@@ -1773,10 +1882,28 @@
17731882
 **
17741883
 ** self-register Allow users to register themselves through the HTTP UI.
17751884
 ** This is useful if you want to see other names than
17761885
 ** "Anonymous" in e.g. ticketing system. On the other hand
17771886
 ** users can not be deleted. Default: off.
1887
+**
1888
+** ssl-ca-location The full pathname to a file containing PEM encoded
1889
+** CA root certificates, or a directory of certificates
1890
+** with filenames formed from the certificate hashes as
1891
+** required by OpenSSL.
1892
+** If set, this will override the OS default list of
1893
+** OpenSSL CAs. If unset, the default list will be used.
1894
+** Some platforms may add additional certificates.
1895
+** Check your platform behaviour is as required if the
1896
+** exact contents of the CA root is critical for your
1897
+** application.
1898
+**
1899
+** ssl-identity The full pathname to a file containing a certificate
1900
+** and private key in PEM format. Create by concatenating
1901
+** the certificate and private key files.
1902
+** This identity will be presented to SSL servers to
1903
+** authenticate this client, in addition to the normal
1904
+** password authentication.
17781905
 **
17791906
 ** ssh-command Command used to talk to a remote machine with
17801907
 ** the "ssh://" protocol.
17811908
 **
17821909
 ** web-browser A shell command used to launch your preferred
@@ -1797,12 +1924,13 @@
17971924
 }
17981925
 if( unsetFlag && g.argc!=3 ){
17991926
 usage("PROPERTY ?-global?");
18001927
 }
18011928
 if( g.argc==2 ){
1929
+ int openLocal = db_open_local();
18021930
 for(i=0; ctrlSettings[i].name; i++){
1803
- print_setting(ctrlSettings[i].name);
1931
+ print_setting(&ctrlSettings[i], openLocal);
18041932
 }
18051933
 }else if( g.argc==3 || g.argc==4 ){
18061934
 const char *zName = g.argv[2];
18071935
 int isManifest;
18081936
 int n = strlen(zName);
@@ -1820,13 +1948,13 @@
18201948
 db_unset(ctrlSettings[i].name, globalFlag);
18211949
 }else if( g.argc==4 ){
18221950
 db_set(ctrlSettings[i].name, g.argv[3], globalFlag);
18231951
 }else{
18241952
 isManifest = 0;
1825
- print_setting(ctrlSettings[i].name);
1953
+ print_setting(&ctrlSettings[i], db_open_local());
18261954
 }
1827
- if( isManifest ){
1955
+ if( isManifest && g.localOpen ){
18281956
 manifest_to_disk(db_lget_int("checkout", 0));
18291957
 }
18301958
 }else{
18311959
 usage("?PROPERTY? ?VALUE?");
18321960
 }
18331961
--- src/db.c
+++ src/db.c
@@ -153,13 +153,17 @@
153 ** Force a rollback and shutdown the database
154 */
155 void db_force_rollback(void){
156 int i;
157 static int busy = 0;
 
158 if( busy || g.db==0 ) return;
159 busy = 1;
160 undo_rollback();
 
 
 
161 while( pAllStmt ){
162 db_finalize(pAllStmt);
163 }
164 if( nBegin ){
165 sqlite3_exec(g.db, "ROLLBACK", 0, 0, 0);
@@ -809,10 +813,11 @@
809 static const char *aDbName[] = { "/_FOSSIL_", "/.fos" };
810
811 if( g.localOpen) return 1;
812 file_getcwd(zPwd, sizeof(zPwd)-20);
813 n = strlen(zPwd);
 
814 while( n>0 ){
815 if( file_access(zPwd, W_OK) ) break;
816 for(i=0; i<sizeof(aDbName)/sizeof(aDbName[0]); i++){
817 sqlite3_snprintf(sizeof(zPwd)-n, &zPwd[n], "%s", aDbName[i]);
818 if( isValidLocalDb(zPwd) ){
@@ -1058,14 +1063,13 @@
1058
1059 /*
1060 ** Create the default user accounts in the USER table.
1061 */
1062 void db_create_default_users(int setupUserOnly, const char *zDefaultUser){
1063 const char *zUser;
1064 zUser = db_get("default-user", 0);
1065 if( zUser==0 ){
1066 zUser = zDefaultUser;
1067 }
1068 if( zUser==0 ){
1069 #if defined(_WIN32)
1070 zUser = getenv("USERNAME");
1071 #else
@@ -1074,12 +1078,15 @@
1074 }
1075 if( zUser==0 ){
1076 zUser = "root";
1077 }
1078 db_multi_exec(
1079 "INSERT INTO user(login, pw, cap, info)"
1080 "VALUES(%Q,lower(hex(randomblob(3))),'s','')", zUser
 
 
 
1081 );
1082 if( !setupUserOnly ){
1083 db_multi_exec(
1084 "INSERT INTO user(login,pw,cap,info)"
1085 " VALUES('anonymous',hex(randomblob(8)),'hmncz','Anon');"
@@ -1363,19 +1370,19 @@
1363 */
1364 int is_truth(const char *zVal){
1365 static const char *azOn[] = { "on", "yes", "true", "1" };
1366 int i;
1367 for(i=0; i<sizeof(azOn)/sizeof(azOn[0]); i++){
1368 if( fossil_strcmp(zVal,azOn[i])==0 ) return 1;
1369 }
1370 return 0;
1371 }
1372 int is_false(const char *zVal){
1373 static const char *azOff[] = { "off", "no", "false", "0" };
1374 int i;
1375 for(i=0; i<sizeof(azOff)/sizeof(azOff[0]); i++){
1376 if( fossil_strcmp(zVal,azOff[i])==0 ) return 1;
1377 }
1378 return 0;
1379 }
1380
1381 /*
@@ -1392,24 +1399,93 @@
1392 sqlite3 *dbTemp = g.db;
1393 g.db = g.dbConfig;
1394 g.dbConfig = dbTemp;
1395 }
1396 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1397
1398 /*
1399 ** Get and set values from the CONFIG, GLOBAL_CONFIG and VVAR table in the
1400 ** repository and local databases.
1401 */
1402 char *db_get(const char *zName, char *zDefault){
1403 char *z = 0;
 
 
 
 
 
 
 
 
 
1404 if( g.repositoryOpen ){
1405 z = db_text(0, "SELECT value FROM config WHERE name=%Q", zName);
1406 }
1407 if( z==0 && g.configOpen ){
1408 db_swap_connections();
1409 z = db_text(0, "SELECT value FROM global_config WHERE name=%Q", zName);
1410 db_swap_connections();
 
 
 
 
1411 }
1412 if( z==0 ){
1413 z = zDefault;
1414 }
1415 return z;
@@ -1596,30 +1672,39 @@
1596 }
1597
1598 /*
1599 ** Print the value of a setting named zName
1600 */
1601 static void print_setting(const char *zName){
1602 Stmt q;
1603 if( g.repositoryOpen ){
1604 db_prepare(&q,
1605 "SELECT '(local)', value FROM config WHERE name=%Q"
1606 " UNION ALL "
1607 "SELECT '(global)', value FROM global_config WHERE name=%Q",
1608 zName, zName
1609 );
1610 }else{
1611 db_prepare(&q,
1612 "SELECT '(global)', value FROM global_config WHERE name=%Q",
1613 zName
1614 );
1615 }
1616 if( db_step(&q)==SQLITE_ROW ){
1617 fossil_print("%-20s %-8s %s\n", zName, db_column_text(&q, 0),
1618 db_column_text(&q, 1));
1619 }else{
1620 fossil_print("%-20s\n", zName);
 
 
 
 
 
 
 
 
 
1621 }
1622 db_finalize(&q);
1623 }
1624
1625
@@ -1635,42 +1720,49 @@
1635 #if INTERFACE
1636 struct stControlSettings {
1637 char const *name; /* Name of the setting */
1638 char const *var; /* Internal variable name used by db_set() */
1639 int width; /* Width of display. 0 for boolean values */
 
1640 char const *def; /* Default value */
1641 };
1642 #endif /* INTERFACE */
1643 struct stControlSettings const ctrlSettings[] = {
1644 { "access-log", 0, 0, "off" },
1645 { "auto-captcha", "autocaptcha", 0, "on" },
1646 { "auto-shun", 0, 0, "on" },
1647 { "autosync", 0, 0, "on" },
1648 { "binary-glob", 0, 32, "" },
1649 { "clearsign", 0, 0, "off" },
1650 { "crnl-glob", 0, 16, "" },
1651 { "default-perms", 0, 16, "u" },
1652 { "diff-command", 0, 16, "" },
1653 { "dont-push", 0, 0, "off" },
1654 { "editor", 0, 16, "" },
1655 { "gdiff-command", 0, 16, "gdiff" },
1656 { "gmerge-command",0, 40, "" },
1657 { "https-login", 0, 0, "off" },
1658 { "ignore-glob", 0, 40, "" },
1659 { "http-port", 0, 16, "8080" },
1660 { "localauth", 0, 0, "off" },
1661 { "main-branch", 0, 40, "trunk" },
1662 { "manifest", 0, 0, "off" },
1663 { "max-upload", 0, 25, "250000" },
1664 { "mtime-changes", 0, 0, "on" },
1665 { "pgp-command", 0, 32, "gpg --clearsign -o " },
1666 { "proxy", 0, 32, "off" },
1667 { "repo-cksum", 0, 0, "on" },
1668 { "self-register", 0, 0, "off" },
1669 { "ssh-command", 0, 32, "" },
1670 { "web-browser", 0, 32, "" },
1671 { 0,0,0,0 }
 
 
 
 
 
 
1672 };
1673
1674 /*
1675 ** COMMAND: settings
1676 ** COMMAND: unset
@@ -1679,10 +1771,14 @@
1679 ** %fossil unset PROPERTY ?-global?
1680 **
1681 ** The "settings" command with no arguments lists all properties and their
1682 ** values. With just a property name it shows the value of that property.
1683 ** With a value argument it changes the property for the current repository.
 
 
 
 
1684 **
1685 ** The "unset" command clears a property setting.
1686 **
1687 **
1688 ** auto-captcha If enabled, the Login page provides a button to
@@ -1696,20 +1792,25 @@
1696 ** or update and automatically push after commit or
1697 ** tag or branch creation. If the value is "pullonly"
1698 ** then only pull operations occur automatically.
1699 ** Default: on
1700 **
1701 ** binary-glob The VALUE is a comma-separated list of GLOB patterns
1702 ** that should be treated as binary files for merging
1703 ** purposes. Example: *.xml
 
 
 
 
 
1704 **
1705 ** clearsign When enabled, fossil will attempt to sign all commits
1706 ** with gpg. When disabled (the default), commits will
1707 ** be unsigned. Default: off
1708 **
1709 ** crnl-glob A comma-separated list of GLOB patterns for text files
1710 ** in which it is ok to have CR+NL line endings.
1711 ** Set to "*" to disable CR+NL checking.
1712 **
1713 ** default-perms Permissions given automatically to new users. For more
1714 ** information on permissions see Users page in Server
1715 ** Administration of the HTTP UI. Default: u.
@@ -1717,10 +1818,15 @@
1717 ** diff-command External command to run when performing a diff.
1718 ** If undefined, the internal text diff will be used.
1719 **
1720 ** dont-push Prevent this repository from pushing from client to
1721 ** server. Useful when setting up a private branch.
 
 
 
 
 
1722 **
1723 ** editor Text editor command used for check-in comments.
1724 **
1725 ** gdiff-command External command to run when performing a graphical
1726 ** diff. If undefined, text diff will be used.
@@ -1735,23 +1841,23 @@
1735 ** and "ui" commands. Default: 8080
1736 **
1737 ** https-login Send login creditials using HTTPS instead of HTTP
1738 ** even if the login page request came via HTTP.
1739 **
1740 ** ignore-glob The VALUE is a comma-separated list of GLOB patterns
1741 ** specifying files that the "extra" command will ignore.
1742 ** Example: *.o,*.obj,*.exe
1743 **
1744 ** localauth If enabled, require that HTTP connections from
1745 ** 127.0.0.1 be authenticated by password. If
1746 ** false, all HTTP requests from localhost have
1747 ** unrestricted access to the repository.
1748 **
1749 ** main-branch The primary branch for the project. Default: trunk
1750 **
1751 ** manifest If enabled, automatically create files "manifest" and
1752 ** "manifest.uuid" in every checkout. The SQLite and
1753 ** Fossil repositories both require this. Default: off.
1754 **
1755 ** max-upload A limit on the size of uplink HTTP requests. The
1756 ** default is 250000 bytes.
1757 **
@@ -1763,10 +1869,13 @@
1763 **
1764 ** proxy URL of the HTTP proxy. If undefined or "off" then
1765 ** the "http_proxy" environment variable is consulted.
1766 ** If the http_proxy environment variable is undefined
1767 ** then a direct HTTP connection is used.
 
 
 
1768 **
1769 ** repo-cksum Compute checksums over all files in each checkout
1770 ** as a double-check of correctness. Defaults to "on".
1771 ** Disable on large repositories for a performance
1772 ** improvement.
@@ -1773,10 +1882,28 @@
1773 **
1774 ** self-register Allow users to register themselves through the HTTP UI.
1775 ** This is useful if you want to see other names than
1776 ** "Anonymous" in e.g. ticketing system. On the other hand
1777 ** users can not be deleted. Default: off.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1778 **
1779 ** ssh-command Command used to talk to a remote machine with
1780 ** the "ssh://" protocol.
1781 **
1782 ** web-browser A shell command used to launch your preferred
@@ -1797,12 +1924,13 @@
1797 }
1798 if( unsetFlag && g.argc!=3 ){
1799 usage("PROPERTY ?-global?");
1800 }
1801 if( g.argc==2 ){
 
1802 for(i=0; ctrlSettings[i].name; i++){
1803 print_setting(ctrlSettings[i].name);
1804 }
1805 }else if( g.argc==3 || g.argc==4 ){
1806 const char *zName = g.argv[2];
1807 int isManifest;
1808 int n = strlen(zName);
@@ -1820,13 +1948,13 @@
1820 db_unset(ctrlSettings[i].name, globalFlag);
1821 }else if( g.argc==4 ){
1822 db_set(ctrlSettings[i].name, g.argv[3], globalFlag);
1823 }else{
1824 isManifest = 0;
1825 print_setting(ctrlSettings[i].name);
1826 }
1827 if( isManifest ){
1828 manifest_to_disk(db_lget_int("checkout", 0));
1829 }
1830 }else{
1831 usage("?PROPERTY? ?VALUE?");
1832 }
1833
--- src/db.c
+++ src/db.c
@@ -153,13 +153,17 @@
153 ** Force a rollback and shutdown the database
154 */
155 void db_force_rollback(void){
156 int i;
157 static int busy = 0;
158 sqlite3_stmt *pStmt = 0;
159 if( busy || g.db==0 ) return;
160 busy = 1;
161 undo_rollback();
162 while( (pStmt = sqlite3_next_stmt(g.db,pStmt))!=0 ){
163 sqlite3_reset(pStmt);
164 }
165 while( pAllStmt ){
166 db_finalize(pAllStmt);
167 }
168 if( nBegin ){
169 sqlite3_exec(g.db, "ROLLBACK", 0, 0, 0);
@@ -809,10 +813,11 @@
813 static const char *aDbName[] = { "/_FOSSIL_", "/.fos" };
814
815 if( g.localOpen) return 1;
816 file_getcwd(zPwd, sizeof(zPwd)-20);
817 n = strlen(zPwd);
818 if( n==1 && zPwd[0]=='/' ) zPwd[0] = '.';
819 while( n>0 ){
820 if( file_access(zPwd, W_OK) ) break;
821 for(i=0; i<sizeof(aDbName)/sizeof(aDbName[0]); i++){
822 sqlite3_snprintf(sizeof(zPwd)-n, &zPwd[n], "%s", aDbName[i]);
823 if( isValidLocalDb(zPwd) ){
@@ -1058,14 +1063,13 @@
1063
1064 /*
1065 ** Create the default user accounts in the USER table.
1066 */
1067 void db_create_default_users(int setupUserOnly, const char *zDefaultUser){
1068 const char *zUser = zDefaultUser;
 
1069 if( zUser==0 ){
1070 zUser = db_get("default-user", 0);
1071 }
1072 if( zUser==0 ){
1073 #if defined(_WIN32)
1074 zUser = getenv("USERNAME");
1075 #else
@@ -1074,12 +1078,15 @@
1078 }
1079 if( zUser==0 ){
1080 zUser = "root";
1081 }
1082 db_multi_exec(
1083 "INSERT OR IGNORE INTO user(login, info) VALUES(%Q,'')", zUser
1084 );
1085 db_multi_exec(
1086 "UPDATE user SET cap='s', pw=lower(hex(randomblob(3)))"
1087 " WHERE login=%Q", zUser
1088 );
1089 if( !setupUserOnly ){
1090 db_multi_exec(
1091 "INSERT INTO user(login,pw,cap,info)"
1092 " VALUES('anonymous',hex(randomblob(8)),'hmncz','Anon');"
@@ -1363,19 +1370,19 @@
1370 */
1371 int is_truth(const char *zVal){
1372 static const char *azOn[] = { "on", "yes", "true", "1" };
1373 int i;
1374 for(i=0; i<sizeof(azOn)/sizeof(azOn[0]); i++){
1375 if( fossil_stricmp(zVal,azOn[i])==0 ) return 1;
1376 }
1377 return 0;
1378 }
1379 int is_false(const char *zVal){
1380 static const char *azOff[] = { "off", "no", "false", "0" };
1381 int i;
1382 for(i=0; i<sizeof(azOff)/sizeof(azOff[0]); i++){
1383 if( fossil_stricmp(zVal,azOff[i])==0 ) return 1;
1384 }
1385 return 0;
1386 }
1387
1388 /*
@@ -1392,24 +1399,93 @@
1399 sqlite3 *dbTemp = g.db;
1400 g.db = g.dbConfig;
1401 g.dbConfig = dbTemp;
1402 }
1403 }
1404
1405 /*
1406 ** Logic for reading potentially versioned settings from
1407 ** .fossil-settings/<name> , and emits warnings if necessary.
1408 ** Returns the non-versioned value without modification if there is no
1409 ** versioned value.
1410 */
1411 static char *db_get_do_versionable(const char *zName, char *zNonVersionedSetting){
1412 /* Attempt to load the versioned setting from a checked out file */
1413 char *zVersionedSetting = 0;
1414 int noWarn = 0;
1415
1416 if( db_open_local() ){
1417 Blob versionedPathname;
1418 char *zVersionedPathname;
1419 blob_zero(&versionedPathname);
1420 blob_appendf(&versionedPathname, "%s/.fossil-settings/%s",
1421 g.zLocalRoot, zName);
1422 zVersionedPathname = blob_str(&versionedPathname);
1423 if( file_size(zVersionedPathname)>=0 ){
1424 /* File exists, and contains the value for this setting. Load from
1425 ** the file. */
1426 Blob setting;
1427 blob_zero(&setting);
1428 if( blob_read_from_file(&setting, zVersionedPathname) >= 0 ){
1429 blob_trim(&setting); /* Avoid non-obvious problems with line endings
1430 ** on boolean properties */
1431 zVersionedSetting = strdup(blob_str(&setting));
1432 }
1433 blob_reset(&setting);
1434 /* See if there's a no-warn flag */
1435 blob_append(&versionedPathname, ".no-warn", -1);
1436 if( file_size(blob_str(&versionedPathname))>=0 ){
1437 noWarn = 1;
1438 }
1439 }
1440 blob_reset(&versionedPathname);
1441 }
1442 /* Display a warning? */
1443 if( zVersionedSetting!=0 && zNonVersionedSetting!=0
1444 && zNonVersionedSetting[0]!='\0' && !noWarn
1445 ){
1446 /* There's a versioned setting, and a non-versioned setting. Tell
1447 ** the user about the conflict */
1448 fossil_warning(
1449 "setting %s has both versioned and non-versioned values: using "
1450 "versioned value from file .fossil-settings/%s (to silence this "
1451 "warning, either create an empty file named "
1452 ".fossil-settings/%s.no-warn or delete the non-versioned setting "
1453 " with \"fossil unset %s\")", zName, zName, zName, zName
1454 );
1455 }
1456 /* Prefer the versioned setting */
1457 return ( zVersionedSetting!=0 ) ? zVersionedSetting : zNonVersionedSetting;
1458 }
1459
1460
1461 /*
1462 ** Get and set values from the CONFIG, GLOBAL_CONFIG and VVAR table in the
1463 ** repository and local databases.
1464 */
1465 char *db_get(const char *zName, char *zDefault){
1466 char *z = 0;
1467 int i;
1468 const struct stControlSettings *ctrlSetting = 0;
1469 /* Is this a setting? */
1470 for(i=0; ctrlSettings[i].name; i++){
1471 if( strcmp(ctrlSettings[i].name, zName)==0 ){
1472 ctrlSetting = &(ctrlSettings[i]);
1473 break;
1474 }
1475 }
1476 if( g.repositoryOpen ){
1477 z = db_text(0, "SELECT value FROM config WHERE name=%Q", zName);
1478 }
1479 if( z==0 && g.configOpen ){
1480 db_swap_connections();
1481 z = db_text(0, "SELECT value FROM global_config WHERE name=%Q", zName);
1482 db_swap_connections();
1483 }
1484 if( ctrlSetting!=0 && ctrlSetting->versionable ){
1485 /* This is a versionable setting, try and get the info from a checked out file */
1486 z = db_get_do_versionable(zName, z);
1487 }
1488 if( z==0 ){
1489 z = zDefault;
1490 }
1491 return z;
@@ -1596,30 +1672,39 @@
1672 }
1673
1674 /*
1675 ** Print the value of a setting named zName
1676 */
1677 static void print_setting(const struct stControlSettings *ctrlSetting, int localOpen){
1678 Stmt q;
1679 if( g.repositoryOpen ){
1680 db_prepare(&q,
1681 "SELECT '(local)', value FROM config WHERE name=%Q"
1682 " UNION ALL "
1683 "SELECT '(global)', value FROM global_config WHERE name=%Q",
1684 ctrlSetting->name, ctrlSetting->name
1685 );
1686 }else{
1687 db_prepare(&q,
1688 "SELECT '(global)', value FROM global_config WHERE name=%Q",
1689 ctrlSetting->name
1690 );
1691 }
1692 if( db_step(&q)==SQLITE_ROW ){
1693 fossil_print("%-20s %-8s %s\n", ctrlSetting->name, db_column_text(&q, 0),
1694 db_column_text(&q, 1));
1695 }else{
1696 fossil_print("%-20s\n", ctrlSetting->name);
1697 }
1698 if( ctrlSetting->versionable && localOpen ){
1699 /* Check to see if this is overridden by a versionable settings file */
1700 Blob versionedPathname;
1701 blob_zero(&versionedPathname);
1702 blob_appendf(&versionedPathname, "%s/.fossil-settings/%s", g.zLocalRoot, ctrlSetting->name);
1703 if( file_size(blob_str(&versionedPathname))>=0 ){
1704 fossil_print(" (overridden by contents of file .fossil-settings/%s)\n", ctrlSetting->name);
1705 }
1706 }
1707 db_finalize(&q);
1708 }
1709
1710
@@ -1635,42 +1720,49 @@
1720 #if INTERFACE
1721 struct stControlSettings {
1722 char const *name; /* Name of the setting */
1723 char const *var; /* Internal variable name used by db_set() */
1724 int width; /* Width of display. 0 for boolean values */
1725 int versionable; /* Is this setting versionable? */
1726 char const *def; /* Default value */
1727 };
1728 #endif /* INTERFACE */
1729 struct stControlSettings const ctrlSettings[] = {
1730 { "access-log", 0, 0, 0, "off" },
1731 { "auto-captcha", "autocaptcha", 0, 0, "on" },
1732 { "auto-shun", 0, 0, 0, "on" },
1733 { "autosync", 0, 0, 0, "on" },
1734 { "binary-glob", 0, 32, 1, "" },
1735 { "clearsign", 0, 0, 0, "off" },
1736 { "case-sensitive",0, 0, 0, "on" },
1737 { "crnl-glob", 0, 16, 1, "" },
1738 { "default-perms", 0, 16, 0, "u" },
1739 { "diff-command", 0, 16, 0, "" },
1740 { "dont-push", 0, 0, 0, "off" },
1741 { "editor", 0, 16, 0, "" },
1742 { "gdiff-command", 0, 16, 0, "gdiff" },
1743 { "gmerge-command",0, 40, 0, "" },
1744 { "https-login", 0, 0, 0, "off" },
1745 { "ignore-glob", 0, 40, 1, "" },
1746 { "empty-dirs", 0, 40, 1, "" },
1747 { "http-port", 0, 16, 0, "8080" },
1748 { "localauth", 0, 0, 0, "off" },
1749 { "main-branch", 0, 40, 0, "trunk" },
1750 { "manifest", 0, 0, 1, "off" },
1751 { "max-upload", 0, 25, 0, "250000" },
1752 { "mtime-changes", 0, 0, 0, "on" },
1753 { "pgp-command", 0, 32, 0, "gpg --clearsign -o " },
1754 { "proxy", 0, 32, 0, "off" },
1755 { "relative-paths",0, 0, 0, "on" },
1756 { "repo-cksum", 0, 0, 0, "on" },
1757 { "self-register", 0, 0, 0, "off" },
1758 { "ssl-ca-location",0, 40, 0, "" },
1759 { "ssl-identity", 0, 40, 0, "" },
1760 { "ssh-command", 0, 32, 0, "" },
1761 { "web-browser", 0, 32, 0, "" },
1762 { "white-foreground", 0, 0, 0, "off" },
1763 { 0,0,0,0,0 }
1764 };
1765
1766 /*
1767 ** COMMAND: settings
1768 ** COMMAND: unset
@@ -1679,10 +1771,14 @@
1771 ** %fossil unset PROPERTY ?-global?
1772 **
1773 ** The "settings" command with no arguments lists all properties and their
1774 ** values. With just a property name it shows the value of that property.
1775 ** With a value argument it changes the property for the current repository.
1776 **
1777 ** Settings marked as versionable are overridden by the contents of the
1778 ** file named .fossil-settings/PROPERTY in the checked out files, if that
1779 ** file exists.
1780 **
1781 ** The "unset" command clears a property setting.
1782 **
1783 **
1784 ** auto-captcha If enabled, the Login page provides a button to
@@ -1696,20 +1792,25 @@
1792 ** or update and automatically push after commit or
1793 ** tag or branch creation. If the value is "pullonly"
1794 ** then only pull operations occur automatically.
1795 ** Default: on
1796 **
1797 ** binary-glob The VALUE is a comma or newline-separated list of
1798 ** (versionable) GLOB patterns that should be treated as binary files
1799 ** for merging purposes. Example: *.xml
1800 **
1801 ** case-sensitive If TRUE, the files whose names differ only in case
1802 ** care considered distinct. If FALSE files whose names
1803 ** differ only in case are the same file. Defaults to
1804 ** TRUE for unix and FALSE for windows and mac.
1805 **
1806 ** clearsign When enabled, fossil will attempt to sign all commits
1807 ** with gpg. When disabled (the default), commits will
1808 ** be unsigned. Default: off
1809 **
1810 ** crnl-glob A comma or newline-separated list of GLOB patterns for
1811 ** (versionable) text files in which it is ok to have CR+NL line endings.
1812 ** Set to "*" to disable CR+NL checking.
1813 **
1814 ** default-perms Permissions given automatically to new users. For more
1815 ** information on permissions see Users page in Server
1816 ** Administration of the HTTP UI. Default: u.
@@ -1717,10 +1818,15 @@
1818 ** diff-command External command to run when performing a diff.
1819 ** If undefined, the internal text diff will be used.
1820 **
1821 ** dont-push Prevent this repository from pushing from client to
1822 ** server. Useful when setting up a private branch.
1823 **
1824 ** empty-dirs A comma or newline-separated list of pathnames. On
1825 ** (versionable) update and checkout commands, if no file or directory
1826 ** exists with that name, an empty directory will be
1827 ** created.
1828 **
1829 ** editor Text editor command used for check-in comments.
1830 **
1831 ** gdiff-command External command to run when performing a graphical
1832 ** diff. If undefined, text diff will be used.
@@ -1735,23 +1841,23 @@
1841 ** and "ui" commands. Default: 8080
1842 **
1843 ** https-login Send login creditials using HTTPS instead of HTTP
1844 ** even if the login page request came via HTTP.
1845 **
1846 ** ignore-glob The VALUE is a comma or newline-separated list of GLOB
1847 ** (versionable) patterns specifying files that the "extra" command will
1848 ** ignore. Example: *.o,*.obj,*.exe
1849 **
1850 ** localauth If enabled, require that HTTP connections from
1851 ** 127.0.0.1 be authenticated by password. If
1852 ** false, all HTTP requests from localhost have
1853 ** unrestricted access to the repository.
1854 **
1855 ** main-branch The primary branch for the project. Default: trunk
1856 **
1857 ** manifest If enabled, automatically create files "manifest" and
1858 ** (versionable) "manifest.uuid" in every checkout. The SQLite and
1859 ** Fossil repositories both require this. Default: off.
1860 **
1861 ** max-upload A limit on the size of uplink HTTP requests. The
1862 ** default is 250000 bytes.
1863 **
@@ -1763,10 +1869,13 @@
1869 **
1870 ** proxy URL of the HTTP proxy. If undefined or "off" then
1871 ** the "http_proxy" environment variable is consulted.
1872 ** If the http_proxy environment variable is undefined
1873 ** then a direct HTTP connection is used.
1874 **
1875 ** relative-paths When showing changes and extras, report paths relative
1876 ** to the current working directory. Default: "on"
1877 **
1878 ** repo-cksum Compute checksums over all files in each checkout
1879 ** as a double-check of correctness. Defaults to "on".
1880 ** Disable on large repositories for a performance
1881 ** improvement.
@@ -1773,10 +1882,28 @@
1882 **
1883 ** self-register Allow users to register themselves through the HTTP UI.
1884 ** This is useful if you want to see other names than
1885 ** "Anonymous" in e.g. ticketing system. On the other hand
1886 ** users can not be deleted. Default: off.
1887 **
1888 ** ssl-ca-location The full pathname to a file containing PEM encoded
1889 ** CA root certificates, or a directory of certificates
1890 ** with filenames formed from the certificate hashes as
1891 ** required by OpenSSL.
1892 ** If set, this will override the OS default list of
1893 ** OpenSSL CAs. If unset, the default list will be used.
1894 ** Some platforms may add additional certificates.
1895 ** Check your platform behaviour is as required if the
1896 ** exact contents of the CA root is critical for your
1897 ** application.
1898 **
1899 ** ssl-identity The full pathname to a file containing a certificate
1900 ** and private key in PEM format. Create by concatenating
1901 ** the certificate and private key files.
1902 ** This identity will be presented to SSL servers to
1903 ** authenticate this client, in addition to the normal
1904 ** password authentication.
1905 **
1906 ** ssh-command Command used to talk to a remote machine with
1907 ** the "ssh://" protocol.
1908 **
1909 ** web-browser A shell command used to launch your preferred
@@ -1797,12 +1924,13 @@
1924 }
1925 if( unsetFlag && g.argc!=3 ){
1926 usage("PROPERTY ?-global?");
1927 }
1928 if( g.argc==2 ){
1929 int openLocal = db_open_local();
1930 for(i=0; ctrlSettings[i].name; i++){
1931 print_setting(&ctrlSettings[i], openLocal);
1932 }
1933 }else if( g.argc==3 || g.argc==4 ){
1934 const char *zName = g.argv[2];
1935 int isManifest;
1936 int n = strlen(zName);
@@ -1820,13 +1948,13 @@
1948 db_unset(ctrlSettings[i].name, globalFlag);
1949 }else if( g.argc==4 ){
1950 db_set(ctrlSettings[i].name, g.argv[3], globalFlag);
1951 }else{
1952 isManifest = 0;
1953 print_setting(&ctrlSettings[i], db_open_local());
1954 }
1955 if( isManifest && g.localOpen ){
1956 manifest_to_disk(db_lget_int("checkout", 0));
1957 }
1958 }else{
1959 usage("?PROPERTY? ?VALUE?");
1960 }
1961
M src/descendants.c
+35
--- src/descendants.c
+++ src/descendants.c
@@ -186,10 +186,45 @@
186186
 }
187187
 db_reset(&q);
188188
 }
189189
 bag_clear(&seen);
190190
 pqueue_clear(&queue);
191
+ db_finalize(&ins);
192
+ db_finalize(&q);
193
+}
194
+
195
+/*
196
+** Compute up to N direct ancestors (merge ancestors do not count)
197
+** for the check-in rid and put them in a table named "ancestor".
198
+** Label each generation with consecutive integers going backwards
199
+** in time such that rid has the smallest generation number and the oldest
200
+** direct ancestor as the largest generation number.
201
+*/
202
+void compute_direct_ancestors(int rid, int N){
203
+ Stmt ins;
204
+ Stmt q;
205
+ int gen = 0;
206
+ db_multi_exec(
207
+ "CREATE TEMP TABLE ancestor(rid INTEGER, generation INTEGER PRIMARY KEY);"
208
+ "INSERT INTO ancestor VALUES(%d, 0);", rid
209
+ );
210
+ db_prepare(&ins, "INSERT INTO ancestor VALUES(:rid, :gen)");
211
+ db_prepare(&q,
212
+ "SELECT pid FROM plink"
213
+ " WHERE cid=:rid AND isprim"
214
+ );
215
+ while( (N--)>0 ){
216
+ db_bind_int(&q, ":rid", rid);
217
+ if( db_step(&q)!=SQLITE_ROW ) break;
218
+ rid = db_column_int(&q, 0);
219
+ db_reset(&q);
220
+ gen++;
221
+ db_bind_int(&ins, ":rid", rid);
222
+ db_bind_int(&ins, ":gen", gen);
223
+ db_step(&ins);
224
+ db_reset(&ins);
225
+ }
191226
 db_finalize(&ins);
192227
 db_finalize(&q);
193228
 }
194229
195230
 /*
196231
--- src/descendants.c
+++ src/descendants.c
@@ -186,10 +186,45 @@
186 }
187 db_reset(&q);
188 }
189 bag_clear(&seen);
190 pqueue_clear(&queue);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
191 db_finalize(&ins);
192 db_finalize(&q);
193 }
194
195 /*
196
--- src/descendants.c
+++ src/descendants.c
@@ -186,10 +186,45 @@
186 }
187 db_reset(&q);
188 }
189 bag_clear(&seen);
190 pqueue_clear(&queue);
191 db_finalize(&ins);
192 db_finalize(&q);
193 }
194
195 /*
196 ** Compute up to N direct ancestors (merge ancestors do not count)
197 ** for the check-in rid and put them in a table named "ancestor".
198 ** Label each generation with consecutive integers going backwards
199 ** in time such that rid has the smallest generation number and the oldest
200 ** direct ancestor as the largest generation number.
201 */
202 void compute_direct_ancestors(int rid, int N){
203 Stmt ins;
204 Stmt q;
205 int gen = 0;
206 db_multi_exec(
207 "CREATE TEMP TABLE ancestor(rid INTEGER, generation INTEGER PRIMARY KEY);"
208 "INSERT INTO ancestor VALUES(%d, 0);", rid
209 );
210 db_prepare(&ins, "INSERT INTO ancestor VALUES(:rid, :gen)");
211 db_prepare(&q,
212 "SELECT pid FROM plink"
213 " WHERE cid=:rid AND isprim"
214 );
215 while( (N--)>0 ){
216 db_bind_int(&q, ":rid", rid);
217 if( db_step(&q)!=SQLITE_ROW ) break;
218 rid = db_column_int(&q, 0);
219 db_reset(&q);
220 gen++;
221 db_bind_int(&ins, ":rid", rid);
222 db_bind_int(&ins, ":gen", gen);
223 db_step(&ins);
224 db_reset(&ins);
225 }
226 db_finalize(&ins);
227 db_finalize(&q);
228 }
229
230 /*
231
M src/diff.c
+8 -7
--- src/diff.c
+++ src/diff.c
@@ -758,16 +758,16 @@
758758
 ** it was checked in (mlink.mid).
759759
 */
760760
 static void annotate_file(
761761
 Annotator *p, /* The annotator */
762762
 int fnid, /* The name of the file to be annotated */
763
- int mid, /* The specific version of the file for this step */
763
+ int mid, /* Use the version of the file in this check-in */
764764
 int webLabel, /* Use web-style annotations if true */
765765
 int iLimit, /* Limit the number of levels if greater than zero */
766766
 int annFlags /* Flags to alter the annotation */
767767
 ){
768
- Blob toAnnotate; /* Text of the final version of the file */
768
+ Blob toAnnotate; /* Text of the final (mid) version of the file */
769769
 Blob step; /* Text of previous revision */
770770
 int rid; /* Artifact ID of the file being annotated */
771771
 char *zLabel; /* Label to apply to a line */
772772
 Stmt q; /* Query returning all ancestor versions */
773773
@@ -778,23 +778,24 @@
778778
 }
779779
 if( !content_get(rid, &toAnnotate) ){
780780
 fossil_panic("unable to retrieve content of artifact #%d", rid);
781781
 }
782782
 db_multi_exec("CREATE TEMP TABLE ok(rid INTEGER PRIMARY KEY)");
783
- compute_ancestors(mid, 1000000000);
783
+ if( iLimit<=0 ) iLimit = 1000000000;
784
+ compute_direct_ancestors(mid, iLimit);
784785
 annotation_start(p, &toAnnotate);
785786
786787
 db_prepare(&q,
787788
 "SELECT mlink.fid,"
788789
 " (SELECT uuid FROM blob WHERE rid=mlink.%s),"
789790
 " date(event.mtime), "
790791
 " coalesce(event.euser,event.user) "
791
- " FROM mlink, event"
792
+ " FROM ancestor, mlink, event"
792793
 " WHERE mlink.fnid=%d"
793
- " AND mlink.mid IN ok"
794
- " AND event.objid=mlink.mid"
795
- " ORDER BY event.mtime DESC"
794
+ " AND mlink.mid=ancestor.rid"
795
+ " AND event.objid=ancestor.rid"
796
+ " ORDER BY ancestor.generation ASC"
796797
 " LIMIT %d",
797798
 (annFlags & ANN_FILE_VERS)!=0 ? "fid" : "mid",
798799
 fnid,
799800
 iLimit>0 ? iLimit : 10000000
800801
 );
801802
--- src/diff.c
+++ src/diff.c
@@ -758,16 +758,16 @@
758 ** it was checked in (mlink.mid).
759 */
760 static void annotate_file(
761 Annotator *p, /* The annotator */
762 int fnid, /* The name of the file to be annotated */
763 int mid, /* The specific version of the file for this step */
764 int webLabel, /* Use web-style annotations if true */
765 int iLimit, /* Limit the number of levels if greater than zero */
766 int annFlags /* Flags to alter the annotation */
767 ){
768 Blob toAnnotate; /* Text of the final version of the file */
769 Blob step; /* Text of previous revision */
770 int rid; /* Artifact ID of the file being annotated */
771 char *zLabel; /* Label to apply to a line */
772 Stmt q; /* Query returning all ancestor versions */
773
@@ -778,23 +778,24 @@
778 }
779 if( !content_get(rid, &toAnnotate) ){
780 fossil_panic("unable to retrieve content of artifact #%d", rid);
781 }
782 db_multi_exec("CREATE TEMP TABLE ok(rid INTEGER PRIMARY KEY)");
783 compute_ancestors(mid, 1000000000);
 
784 annotation_start(p, &toAnnotate);
785
786 db_prepare(&q,
787 "SELECT mlink.fid,"
788 " (SELECT uuid FROM blob WHERE rid=mlink.%s),"
789 " date(event.mtime), "
790 " coalesce(event.euser,event.user) "
791 " FROM mlink, event"
792 " WHERE mlink.fnid=%d"
793 " AND mlink.mid IN ok"
794 " AND event.objid=mlink.mid"
795 " ORDER BY event.mtime DESC"
796 " LIMIT %d",
797 (annFlags & ANN_FILE_VERS)!=0 ? "fid" : "mid",
798 fnid,
799 iLimit>0 ? iLimit : 10000000
800 );
801
--- src/diff.c
+++ src/diff.c
@@ -758,16 +758,16 @@
758 ** it was checked in (mlink.mid).
759 */
760 static void annotate_file(
761 Annotator *p, /* The annotator */
762 int fnid, /* The name of the file to be annotated */
763 int mid, /* Use the version of the file in this check-in */
764 int webLabel, /* Use web-style annotations if true */
765 int iLimit, /* Limit the number of levels if greater than zero */
766 int annFlags /* Flags to alter the annotation */
767 ){
768 Blob toAnnotate; /* Text of the final (mid) version of the file */
769 Blob step; /* Text of previous revision */
770 int rid; /* Artifact ID of the file being annotated */
771 char *zLabel; /* Label to apply to a line */
772 Stmt q; /* Query returning all ancestor versions */
773
@@ -778,23 +778,24 @@
778 }
779 if( !content_get(rid, &toAnnotate) ){
780 fossil_panic("unable to retrieve content of artifact #%d", rid);
781 }
782 db_multi_exec("CREATE TEMP TABLE ok(rid INTEGER PRIMARY KEY)");
783 if( iLimit<=0 ) iLimit = 1000000000;
784 compute_direct_ancestors(mid, iLimit);
785 annotation_start(p, &toAnnotate);
786
787 db_prepare(&q,
788 "SELECT mlink.fid,"
789 " (SELECT uuid FROM blob WHERE rid=mlink.%s),"
790 " date(event.mtime), "
791 " coalesce(event.euser,event.user) "
792 " FROM ancestor, mlink, event"
793 " WHERE mlink.fnid=%d"
794 " AND mlink.mid=ancestor.rid"
795 " AND event.objid=ancestor.rid"
796 " ORDER BY ancestor.generation ASC"
797 " LIMIT %d",
798 (annFlags & ANN_FILE_VERS)!=0 ? "fid" : "mid",
799 fnid,
800 iLimit>0 ? iLimit : 10000000
801 );
802
M src/diffcmd.c
+22 -16
--- src/diffcmd.c
+++ src/diffcmd.c
@@ -175,23 +175,24 @@
175175
 blob_reset(&cmd);
176176
 }
177177
 }
178178
179179
 /*
180
-** Do a diff against a single file named in g.argv[2] from version zFrom
180
+** Do a diff against a single file named in zFileTreeName from version zFrom
181181
 ** against the same file on disk.
182182
 */
183183
 static void diff_one_against_disk(
184184
 const char *zFrom, /* Name of file */
185185
 const char *zDiffCmd, /* Use this "diff" command */
186
- int ignoreEolWs /* Ignore whitespace changes at end of lines */
186
+ int ignoreEolWs, /* Ignore whitespace changes at end of lines */
187
+ const char *zFileTreeName
187188
 ){
188189
 Blob fname;
189190
 Blob content;
190
- file_tree_name(g.argv[2], &fname, 1);
191
+ file_tree_name(zFileTreeName, &fname, 1);
191192
 historical_version_of_file(zFrom, blob_str(&fname), &content, 0, 0);
192
- diff_file(&content, g.argv[2], g.argv[2], zDiffCmd, ignoreEolWs);
193
+ diff_file(&content, zFileTreeName, zFileTreeName, zDiffCmd, ignoreEolWs);
193194
 blob_reset(&content);
194195
 blob_reset(&fname);
195196
 }
196197
197198
 /*
@@ -295,22 +296,22 @@
295296
 }
296297
297298
 /*
298299
 ** Output the differences between two versions of a single file.
299300
 ** zFrom and zTo are the check-ins containing the two file versions.
300
-** The filename is contained in g.argv[2].
301301
 */
302302
 static void diff_one_two_versions(
303303
 const char *zFrom,
304304
 const char *zTo,
305305
 const char *zDiffCmd,
306
- int ignoreEolWs
306
+ int ignoreEolWs,
307
+ const char *zFileTreeName
307308
 ){
308309
 char *zName;
309310
 Blob fname;
310311
 Blob v1, v2;
311
- file_tree_name(g.argv[2], &fname, 1);
312
+ file_tree_name(zFileTreeName, &fname, 1);
312313
 zName = blob_str(&fname);
313314
 historical_version_of_file(zFrom, zName, &v1, 0, 0);
314315
 historical_version_of_file(zTo, zName, &v2, 0, 0);
315316
 diff_file_mem(&v1, &v2, zName, zDiffCmd, ignoreEolWs);
316317
 blob_reset(&v1);
@@ -408,16 +409,16 @@
408409
409410
 /*
410411
 ** COMMAND: diff
411412
 ** COMMAND: gdiff
412413
 **
413
-** Usage: %fossil diff|gdiff ?options? ?FILE?
414
+** Usage: %fossil diff|gdiff ?options? ?FILE1? ?FILE2 ...?
414415
 **
415
-** Show the difference between the current version of FILE (as it
416
-** exists on disk) and that same file as it was checked out. Or
417
-** if the FILE argument is omitted, show the unsaved changed currently
418
-** in the working check-out.
416
+** Show the difference between the current version of each of the FILEs
417
+** specified (as they exist on disk) and that same file as it was checked
418
+** out. Or if the FILE arguments are omitted, show the unsaved changed
419
+** currently in the working check-out.
419420
 **
420421
 ** If the "--from VERSION" or "-r VERSION" option is used it specifies
421422
 ** the source check-in for the diff operation. If not specified, the
422423
 ** source check-in is the base check-in for the current check-out.
423424
 **
@@ -440,10 +441,11 @@
440441
 int hasNFlag; /* True if -N or --new-file flag is used */
441442
 const char *zFrom; /* Source version number */
442443
 const char *zTo; /* Target version number */
443444
 const char *zDiffCmd = 0; /* External diff command. NULL for internal diff */
444445
 int diffFlags = 0; /* Flags to control the DIFF */
446
+ int f;
445447
446448
 isGDiff = g.argv[1][0]=='g';
447449
 isInternDiff = find_option("internal","i",0)!=0;
448450
 zFrom = find_option("from", "r", 1);
449451
 zTo = find_option("to", 0, 1);
@@ -455,12 +457,14 @@
455457
 db_must_be_within_tree();
456458
 verify_all_options();
457459
 if( !isInternDiff ){
458460
 zDiffCmd = db_get(isGDiff ? "gdiff-command" : "diff-command", 0);
459461
 }
460
- if( g.argc==3 ){
461
- diff_one_against_disk(zFrom, zDiffCmd, 0);
462
+ if( g.argc>=3 ){
463
+ for(f=2; f<g.argc; ++f){
464
+ diff_one_against_disk(zFrom, zDiffCmd, 0, g.argv[f]);
465
+ }
462466
 }else{
463467
 diff_all_against_disk(zFrom, zDiffCmd, diffFlags);
464468
 }
465469
 }else if( zFrom==0 ){
466470
 fossil_fatal("must use --from if --to is present");
@@ -468,12 +472,14 @@
468472
 db_find_and_open_repository(0, 0);
469473
 verify_all_options();
470474
 if( !isInternDiff ){
471475
 zDiffCmd = db_get(isGDiff ? "gdiff-command" : "diff-command", 0);
472476
 }
473
- if( g.argc==3 ){
474
- diff_one_two_versions(zFrom, zTo, zDiffCmd, 0);
477
+ if( g.argc>=3 ){
478
+ for(f=2; f<g.argc; ++f){
479
+ diff_one_two_versions(zFrom, zTo, zDiffCmd, 0, g.argv[f]);
480
+ }
475481
 }else{
476482
 diff_all_two_versions(zFrom, zTo, zDiffCmd, diffFlags);
477483
 }
478484
 }
479485
 }
480486
--- src/diffcmd.c
+++ src/diffcmd.c
@@ -175,23 +175,24 @@
175 blob_reset(&cmd);
176 }
177 }
178
179 /*
180 ** Do a diff against a single file named in g.argv[2] from version zFrom
181 ** against the same file on disk.
182 */
183 static void diff_one_against_disk(
184 const char *zFrom, /* Name of file */
185 const char *zDiffCmd, /* Use this "diff" command */
186 int ignoreEolWs /* Ignore whitespace changes at end of lines */
 
187 ){
188 Blob fname;
189 Blob content;
190 file_tree_name(g.argv[2], &fname, 1);
191 historical_version_of_file(zFrom, blob_str(&fname), &content, 0, 0);
192 diff_file(&content, g.argv[2], g.argv[2], zDiffCmd, ignoreEolWs);
193 blob_reset(&content);
194 blob_reset(&fname);
195 }
196
197 /*
@@ -295,22 +296,22 @@
295 }
296
297 /*
298 ** Output the differences between two versions of a single file.
299 ** zFrom and zTo are the check-ins containing the two file versions.
300 ** The filename is contained in g.argv[2].
301 */
302 static void diff_one_two_versions(
303 const char *zFrom,
304 const char *zTo,
305 const char *zDiffCmd,
306 int ignoreEolWs
 
307 ){
308 char *zName;
309 Blob fname;
310 Blob v1, v2;
311 file_tree_name(g.argv[2], &fname, 1);
312 zName = blob_str(&fname);
313 historical_version_of_file(zFrom, zName, &v1, 0, 0);
314 historical_version_of_file(zTo, zName, &v2, 0, 0);
315 diff_file_mem(&v1, &v2, zName, zDiffCmd, ignoreEolWs);
316 blob_reset(&v1);
@@ -408,16 +409,16 @@
408
409 /*
410 ** COMMAND: diff
411 ** COMMAND: gdiff
412 **
413 ** Usage: %fossil diff|gdiff ?options? ?FILE?
414 **
415 ** Show the difference between the current version of FILE (as it
416 ** exists on disk) and that same file as it was checked out. Or
417 ** if the FILE argument is omitted, show the unsaved changed currently
418 ** in the working check-out.
419 **
420 ** If the "--from VERSION" or "-r VERSION" option is used it specifies
421 ** the source check-in for the diff operation. If not specified, the
422 ** source check-in is the base check-in for the current check-out.
423 **
@@ -440,10 +441,11 @@
440 int hasNFlag; /* True if -N or --new-file flag is used */
441 const char *zFrom; /* Source version number */
442 const char *zTo; /* Target version number */
443 const char *zDiffCmd = 0; /* External diff command. NULL for internal diff */
444 int diffFlags = 0; /* Flags to control the DIFF */
 
445
446 isGDiff = g.argv[1][0]=='g';
447 isInternDiff = find_option("internal","i",0)!=0;
448 zFrom = find_option("from", "r", 1);
449 zTo = find_option("to", 0, 1);
@@ -455,12 +457,14 @@
455 db_must_be_within_tree();
456 verify_all_options();
457 if( !isInternDiff ){
458 zDiffCmd = db_get(isGDiff ? "gdiff-command" : "diff-command", 0);
459 }
460 if( g.argc==3 ){
461 diff_one_against_disk(zFrom, zDiffCmd, 0);
 
 
462 }else{
463 diff_all_against_disk(zFrom, zDiffCmd, diffFlags);
464 }
465 }else if( zFrom==0 ){
466 fossil_fatal("must use --from if --to is present");
@@ -468,12 +472,14 @@
468 db_find_and_open_repository(0, 0);
469 verify_all_options();
470 if( !isInternDiff ){
471 zDiffCmd = db_get(isGDiff ? "gdiff-command" : "diff-command", 0);
472 }
473 if( g.argc==3 ){
474 diff_one_two_versions(zFrom, zTo, zDiffCmd, 0);
 
 
475 }else{
476 diff_all_two_versions(zFrom, zTo, zDiffCmd, diffFlags);
477 }
478 }
479 }
480
--- src/diffcmd.c
+++ src/diffcmd.c
@@ -175,23 +175,24 @@
175 blob_reset(&cmd);
176 }
177 }
178
179 /*
180 ** Do a diff against a single file named in zFileTreeName from version zFrom
181 ** against the same file on disk.
182 */
183 static void diff_one_against_disk(
184 const char *zFrom, /* Name of file */
185 const char *zDiffCmd, /* Use this "diff" command */
186 int ignoreEolWs, /* Ignore whitespace changes at end of lines */
187 const char *zFileTreeName
188 ){
189 Blob fname;
190 Blob content;
191 file_tree_name(zFileTreeName, &fname, 1);
192 historical_version_of_file(zFrom, blob_str(&fname), &content, 0, 0);
193 diff_file(&content, zFileTreeName, zFileTreeName, zDiffCmd, ignoreEolWs);
194 blob_reset(&content);
195 blob_reset(&fname);
196 }
197
198 /*
@@ -295,22 +296,22 @@
296 }
297
298 /*
299 ** Output the differences between two versions of a single file.
300 ** zFrom and zTo are the check-ins containing the two file versions.
 
301 */
302 static void diff_one_two_versions(
303 const char *zFrom,
304 const char *zTo,
305 const char *zDiffCmd,
306 int ignoreEolWs,
307 const char *zFileTreeName
308 ){
309 char *zName;
310 Blob fname;
311 Blob v1, v2;
312 file_tree_name(zFileTreeName, &fname, 1);
313 zName = blob_str(&fname);
314 historical_version_of_file(zFrom, zName, &v1, 0, 0);
315 historical_version_of_file(zTo, zName, &v2, 0, 0);
316 diff_file_mem(&v1, &v2, zName, zDiffCmd, ignoreEolWs);
317 blob_reset(&v1);
@@ -408,16 +409,16 @@
409
410 /*
411 ** COMMAND: diff
412 ** COMMAND: gdiff
413 **
414 ** Usage: %fossil diff|gdiff ?options? ?FILE1? ?FILE2 ...?
415 **
416 ** Show the difference between the current version of each of the FILEs
417 ** specified (as they exist on disk) and that same file as it was checked
418 ** out. Or if the FILE arguments are omitted, show the unsaved changed
419 ** currently in the working check-out.
420 **
421 ** If the "--from VERSION" or "-r VERSION" option is used it specifies
422 ** the source check-in for the diff operation. If not specified, the
423 ** source check-in is the base check-in for the current check-out.
424 **
@@ -440,10 +441,11 @@
441 int hasNFlag; /* True if -N or --new-file flag is used */
442 const char *zFrom; /* Source version number */
443 const char *zTo; /* Target version number */
444 const char *zDiffCmd = 0; /* External diff command. NULL for internal diff */
445 int diffFlags = 0; /* Flags to control the DIFF */
446 int f;
447
448 isGDiff = g.argv[1][0]=='g';
449 isInternDiff = find_option("internal","i",0)!=0;
450 zFrom = find_option("from", "r", 1);
451 zTo = find_option("to", 0, 1);
@@ -455,12 +457,14 @@
457 db_must_be_within_tree();
458 verify_all_options();
459 if( !isInternDiff ){
460 zDiffCmd = db_get(isGDiff ? "gdiff-command" : "diff-command", 0);
461 }
462 if( g.argc>=3 ){
463 for(f=2; f<g.argc; ++f){
464 diff_one_against_disk(zFrom, zDiffCmd, 0, g.argv[f]);
465 }
466 }else{
467 diff_all_against_disk(zFrom, zDiffCmd, diffFlags);
468 }
469 }else if( zFrom==0 ){
470 fossil_fatal("must use --from if --to is present");
@@ -468,12 +472,14 @@
472 db_find_and_open_repository(0, 0);
473 verify_all_options();
474 if( !isInternDiff ){
475 zDiffCmd = db_get(isGDiff ? "gdiff-command" : "diff-command", 0);
476 }
477 if( g.argc>=3 ){
478 for(f=2; f<g.argc; ++f){
479 diff_one_two_versions(zFrom, zTo, zDiffCmd, 0, g.argv[f]);
480 }
481 }else{
482 diff_all_two_versions(zFrom, zTo, zDiffCmd, diffFlags);
483 }
484 }
485 }
486
M src/export.c
+161 -39
--- src/export.c
+++ src/export.c
@@ -39,11 +39,11 @@
3939
 db_bind_text(&q, ":user", zUser);
4040
 if( db_step(&q)!=SQLITE_ROW ){
4141
 db_reset(&q);
4242
 for(i=0; zUser[i] && zUser[i]!='>' && zUser[i]!='<'; i++){}
4343
 if( zUser[i]==0 ){
44
- printf(" <%s>", zUser);
44
+ printf(" %s <%s>", zUser, zUser);
4545
 return;
4646
 }
4747
 zName = mprintf("%s", zUser);
4848
 for(i=j=0; zName[i]; i++){
4949
 if( zName[i]!='<' && zName[i]!='>' ){
@@ -56,11 +56,11 @@
5656
 return;
5757
 }
5858
 zContact = db_column_text(&q, 0);
5959
 for(i=0; zContact[i] && zContact[i]!='>' && zContact[i]!='<'; i++){}
6060
 if( zContact[i]==0 ){
61
- printf(" %s <%s>", zContact, zUser);
61
+ printf(" %s <%s>", zContact[0] ? zContact : zUser, zUser);
6262
 db_reset(&q);
6363
 return;
6464
 }
6565
 if( zContact[i]=='<' ){
6666
 zEmail = mprintf("%s", &zContact[i]);
@@ -78,15 +78,17 @@
7878
 free(zName);
7979
 free(zEmail);
8080
 db_reset(&q);
8181
 }
8282
83
+#define BLOBMARK(rid) ((rid) * 2)
84
+#define COMMITMARK(rid) ((rid) * 2 + 1)
8385
8486
 /*
8587
 ** COMMAND: export
8688
 **
87
-** Usage: %fossil export --git ?REPOSITORY?
89
+** Usage: %fossil export --git ?options? ?REPOSITORY?
8890
 **
8991
 ** Write an export of all check-ins to standard output. The export is
9092
 ** written in the git-fast-export file format assuming the --git option is
9193
 ** provided. The git-fast-export format is currently the only VCS
9294
 ** interchange format supported, though other formats may be added in
@@ -95,98 +97,190 @@
9597
 ** Run this command within a checkout. Or use the -R or --repository
9698
 ** option to specify a Fossil repository to be exported.
9799
 **
98100
 ** Only check-ins are exported using --git. Git does not support tickets
99101
 ** or wiki or events or attachments, so none of those are exported.
102
+**
103
+** If the "--import-marks FILE" option is used, it contains a list of
104
+** rids to skip.
105
+**
106
+** If the "--export-marks FILE" option is used, the rid of all commits and
107
+** blobs written on exit for use with "--import-marks" on the next run.
100108
 */
101109
 void export_cmd(void){
102
- Stmt q;
110
+ Stmt q, q2, q3;
103111
 int i;
104
- int firstCkin; /* Integer offset to check-in marks */
105112
 Bag blobs, vers;
113
+ const char *markfile_in;
114
+ const char *markfile_out;
115
+
106116
 bag_init(&blobs);
107117
 bag_init(&vers);
108118
109119
 find_option("git", 0, 0); /* Ignore the --git option for now */
120
+ markfile_in = find_option("import-marks", 0, 1);
121
+ markfile_out = find_option("export-marks", 0, 1);
122
+
110123
 db_find_and_open_repository(0, 2);
111124
 verify_all_options();
112125
 if( g.argc!=2 && g.argc!=3 ){ usage("--git ?REPOSITORY?"); }
126
+
127
+ db_multi_exec("CREATE TEMPORARY TABLE oldblob(rid INTEGER PRIMARY KEY)");
128
+ db_multi_exec("CREATE TEMPORARY TABLE oldcommit(rid INTEGER PRIMARY KEY)");
129
+ if( markfile_in!=0 ){
130
+ Stmt qb,qc;
131
+ char line[100];
132
+ FILE *f;
133
+
134
+ f = fopen(markfile_in, "r");
135
+ if( f==0 ){
136
+ fossil_panic("cannot open %s for reading", markfile_in);
137
+ }
138
+ db_prepare(&qb, "INSERT OR IGNORE INTO oldblob VALUES (:rid)");
139
+ db_prepare(&qc, "INSERT OR IGNORE INTO oldcommit VALUES (:rid)");
140
+ while( fgets(line, sizeof(line), f)!=0 ){
141
+ if( *line == 'b' ){
142
+ db_bind_text(&qb, ":rid", line + 1);
143
+ db_step(&qb);
144
+ db_reset(&qb);
145
+ bag_insert(&blobs, atoi(line + 1));
146
+ }else if( *line == 'c' ){
147
+ db_bind_text(&qc, ":rid", line + 1);
148
+ db_step(&qc);
149
+ db_reset(&qc);
150
+ bag_insert(&vers, atoi(line + 1));
151
+ }else{
152
+ fossil_panic("bad input from %s: %s", markfile_in, line);
153
+ }
154
+ }
155
+ db_finalize(&qb);
156
+ db_finalize(&qc);
157
+ fclose(f);
158
+ }
113159
114160
 /* Step 1: Generate "blob" records for every artifact that is part
115161
 ** of a check-in
116162
 */
117163
 fossil_binary_mode(stdout);
118
- db_prepare(&q, "SELECT DISTINCT fid FROM mlink WHERE fid>0");
164
+ db_multi_exec("CREATE TEMPORARY TABLE newblob(rid INTEGER KEY, srcid INTEGER)");
165
+ db_multi_exec("CREATE INDEX newblob_src ON newblob(srcid)");
166
+ db_multi_exec(
167
+ "INSERT INTO newblob"
168
+ " SELECT DISTINCT fid,"
169
+ " CASE WHEN EXISTS(SELECT 1 FROM delta WHERE rid=fid AND NOT EXISTS(SELECT 1 FROM oldblob WHERE srcid=fid))"
170
+ " THEN (SELECT srcid FROM delta WHERE rid=fid)"
171
+ " ELSE 0"
172
+ " END"
173
+ " FROM mlink"
174
+ " WHERE fid>0 AND NOT EXISTS(SELECT 1 FROM oldblob WHERE rid=fid)");
175
+ db_prepare(&q,
176
+ "SELECT DISTINCT fid FROM mlink"
177
+ " WHERE fid>0 AND NOT EXISTS(SELECT 1 FROM oldblob WHERE rid=fid)");
178
+ db_prepare(&q2, "INSERT INTO oldblob VALUES (:rid)");
179
+ db_prepare(&q3, "SELECT rid FROM newblob WHERE srcid= (:srcid)");
119180
 while( db_step(&q)==SQLITE_ROW ){
120181
 int rid = db_column_int(&q, 0);
121182
 Blob content;
122
- content_get(rid, &content);
123
- printf("blob\nmark :%d\ndata %d\n", rid, blob_size(&content));
124
- bag_insert(&blobs, rid);
125
- fwrite(blob_buffer(&content), 1, blob_size(&content), stdout);
126
- printf("\n");
127
- blob_reset(&content);
183
+
184
+ while( !bag_find(&blobs, rid) ){
185
+ content_get(rid, &content);
186
+ db_bind_int(&q2, ":rid", rid);
187
+ db_step(&q2);
188
+ db_reset(&q2);
189
+ printf("blob\nmark :%d\ndata %d\n", BLOBMARK(rid), blob_size(&content));
190
+ bag_insert(&blobs, rid);
191
+ fwrite(blob_buffer(&content), 1, blob_size(&content), stdout);
192
+ printf("\n");
193
+ blob_reset(&content);
194
+
195
+ db_bind_int(&q3, ":srcid", rid);
196
+ if( db_step(&q3) != SQLITE_ROW ){
197
+ db_reset(&q3);
198
+ break;
199
+ }
200
+ rid = db_column_int(&q3, 0);
201
+ db_reset(&q3);
202
+ }
128203
 }
129204
 db_finalize(&q);
205
+ db_finalize(&q2);
206
+ db_finalize(&q3);
130207
131208
 /* Output the commit records.
132209
 */
133
- firstCkin = db_int(0, "SELECT max(rid) FROM blob")+1;
134210
 db_prepare(&q,
135211
 "SELECT strftime('%%s',mtime), objid, coalesce(comment,ecomment),"
136212
 " coalesce(user,euser),"
137213
 " (SELECT value FROM tagxref WHERE rid=objid AND tagid=%d)"
138214
 " FROM event"
139
- " WHERE type='ci'"
215
+ " WHERE type='ci' AND NOT EXISTS (SELECT 1 FROM oldcommit WHERE objid=rid)"
140216
 " ORDER BY mtime ASC",
141217
 TAG_BRANCH
142218
 );
219
+ db_prepare(&q2, "INSERT INTO oldcommit VALUES (:rid)");
143220
 while( db_step(&q)==SQLITE_ROW ){
221
+ Stmt q4;
144222
 const char *zSecondsSince1970 = db_column_text(&q, 0);
145223
 int ckinId = db_column_int(&q, 1);
146224
 const char *zComment = db_column_text(&q, 2);
147225
 const char *zUser = db_column_text(&q, 3);
148226
 const char *zBranch = db_column_text(&q, 4);
149227
 char *zBr;
150
- Manifest *p;
151
- ManifestFile *pFile;
152
- const char *zFromType;
153228
154229
 bag_insert(&vers, ckinId);
230
+ db_bind_int(&q2, ":rid", ckinId);
231
+ db_step(&q2);
232
+ db_reset(&q2);
155233
 if( zBranch==0 ) zBranch = "trunk";
156234
 zBr = mprintf("%s", zBranch);
157235
 for(i=0; zBr[i]; i++){
158236
 if( !fossil_isalnum(zBr[i]) ) zBr[i] = '_';
159237
 }
160
- printf("commit refs/heads/%s\nmark :%d\n", zBr, ckinId+firstCkin);
238
+ printf("commit refs/heads/%s\nmark :%d\n", zBr, COMMITMARK(ckinId));
161239
 free(zBr);
162240
 printf("committer");
163241
 print_person(zUser);
164242
 printf(" %s +0000\n", zSecondsSince1970);
165243
 if( zComment==0 ) zComment = "null comment";
166244
 printf("data %d\n%s\n", (int)strlen(zComment), zComment);
167
- p = manifest_get(ckinId, CFTYPE_ANY);
168
- zFromType = "from";
169
- for(i=0; i<p->nParent; i++){
170
- int pid = fast_uuid_to_rid(p->azParent[i]);
171
- if( pid==0 || !bag_find(&vers, pid) ) continue;
172
- printf("%s :%d\n", zFromType, fast_uuid_to_rid(p->azParent[i])+firstCkin);
173
- zFromType = "merge";
174
- }
175
- printf("deleteall\n");
176
- manifest_file_rewind(p);
177
- while( (pFile=manifest_file_next(p, 0))!=0 ){
178
- int fid = fast_uuid_to_rid(pFile->zUuid);
179
- const char *zPerm = "100644";
180
- if( fid==0 ) continue;
181
- if( pFile->zPerm && strstr(pFile->zPerm,"x") ) zPerm = "100755";
182
- if( !bag_find(&blobs, fid) ) continue;
183
- printf("M %s :%d %s\n", zPerm, fid, pFile->zName);
184
- }
185
- manifest_cache_insert(p);
245
+ db_prepare(&q3, "SELECT pid FROM plink WHERE cid=%d AND isprim", ckinId);
246
+ if( db_step(&q3) == SQLITE_ROW ){
247
+ printf("from :%d\n", COMMITMARK(db_column_int(&q3, 0)));
248
+ db_prepare(&q4,
249
+ "SELECT pid FROM plink"
250
+ " WHERE cid=%d AND NOT isprim"
251
+ " AND NOT EXISTS(SELECT 1 FROM phantom WHERE rid=pid)"
252
+ " ORDER BY pid",
253
+ ckinId);
254
+ while( db_step(&q4)==SQLITE_ROW ){
255
+ printf("merge :%d\n", COMMITMARK(db_column_int(&q4,0)));
256
+ }
257
+ db_finalize(&q4);
258
+ }else{
259
+ printf("deleteall\n");
260
+ }
261
+
262
+ db_prepare(&q4,
263
+ "SELECT filename.name, mlink.fid, mlink.mperm FROM mlink"
264
+ " JOIN filename ON filename.fnid=mlink.fnid"
265
+ " WHERE mlink.mid=%d",
266
+ ckinId
267
+ );
268
+ while( db_step(&q4)==SQLITE_ROW ){
269
+ const char *zName = db_column_text(&q4,0);
270
+ int zNew = db_column_int(&q4,1);
271
+ int mPerm = db_column_int(&q4,2);
272
+ if( zNew==0)
273
+ printf("D %s\n", zName);
274
+ else if( bag_find(&blobs, zNew) )
275
+ printf("M %s :%d %s\n", mPerm ? "100755" : "100644", BLOBMARK(zNew), zName);
276
+ }
277
+ db_finalize(&q4);
278
+ db_finalize(&q3);
186279
 printf("\n");
187280
 }
281
+ db_finalize(&q2);
188282
 db_finalize(&q);
189283
 bag_clear(&blobs);
190284
 manifest_cache_clear();
191285
192286
@@ -196,17 +290,45 @@
196290
 " FROM tagxref JOIN tag USING(tagid)"
197291
 " WHERE tagtype=1 AND tagname GLOB 'sym-*'"
198292
 );
199293
 while( db_step(&q)==SQLITE_ROW ){
200294
 const char *zTagname = db_column_text(&q, 0);
295
+ char *zEncoded = 0;
201296
 int rid = db_column_int(&q, 1);
202297
 const char *zSecSince1970 = db_column_text(&q, 2);
298
+ int i;
203299
 if( rid==0 || !bag_find(&vers, rid) ) continue;
204300
 zTagname += 4;
205
- printf("tag %s\n", zTagname);
206
- printf("from :%d\n", rid+firstCkin);
301
+ zEncoded = mprintf("%s", zTagname);
302
+ for(i=0; zEncoded[i]; i++){
303
+ if( !fossil_isalnum(zEncoded[i]) ) zEncoded[i] = '_';
304
+ }
305
+ printf("tag %s\n", zEncoded);
306
+ printf("from :%d\n", COMMITMARK(rid));
207307
 printf("tagger <tagger> %s +0000\n", zSecSince1970);
208308
 printf("data 0\n");
309
+ fossil_free(zEncoded);
209310
 }
210311
 db_finalize(&q);
211312
 bag_clear(&vers);
313
+
314
+ if( markfile_out!=0 ){
315
+ FILE *f;
316
+ f = fopen(markfile_out, "w");
317
+ if( f == 0 ){
318
+ fossil_panic("cannot open %s for writing", markfile_out);
319
+ }
320
+ db_prepare(&q, "SELECT rid FROM oldblob");
321
+ while( db_step(&q)==SQLITE_ROW ){
322
+ fprintf(f, "b%d\n", db_column_int(&q, 0));
323
+ }
324
+ db_finalize(&q);
325
+ db_prepare(&q, "SELECT rid FROM oldcommit");
326
+ while( db_step(&q)==SQLITE_ROW ){
327
+ fprintf(f, "c%d\n", db_column_int(&q, 0));
328
+ }
329
+ db_finalize(&q);
330
+ if( ferror(f)!=0 || fclose(f)!=0 ) {
331
+ fossil_panic("error while writing %s", markfile_out);
332
+ }
333
+ }
212334
 }
213335
--- src/export.c
+++ src/export.c
@@ -39,11 +39,11 @@
39 db_bind_text(&q, ":user", zUser);
40 if( db_step(&q)!=SQLITE_ROW ){
41 db_reset(&q);
42 for(i=0; zUser[i] && zUser[i]!='>' && zUser[i]!='<'; i++){}
43 if( zUser[i]==0 ){
44 printf(" <%s>", zUser);
45 return;
46 }
47 zName = mprintf("%s", zUser);
48 for(i=j=0; zName[i]; i++){
49 if( zName[i]!='<' && zName[i]!='>' ){
@@ -56,11 +56,11 @@
56 return;
57 }
58 zContact = db_column_text(&q, 0);
59 for(i=0; zContact[i] && zContact[i]!='>' && zContact[i]!='<'; i++){}
60 if( zContact[i]==0 ){
61 printf(" %s <%s>", zContact, zUser);
62 db_reset(&q);
63 return;
64 }
65 if( zContact[i]=='<' ){
66 zEmail = mprintf("%s", &zContact[i]);
@@ -78,15 +78,17 @@
78 free(zName);
79 free(zEmail);
80 db_reset(&q);
81 }
82
 
 
83
84 /*
85 ** COMMAND: export
86 **
87 ** Usage: %fossil export --git ?REPOSITORY?
88 **
89 ** Write an export of all check-ins to standard output. The export is
90 ** written in the git-fast-export file format assuming the --git option is
91 ** provided. The git-fast-export format is currently the only VCS
92 ** interchange format supported, though other formats may be added in
@@ -95,98 +97,190 @@
95 ** Run this command within a checkout. Or use the -R or --repository
96 ** option to specify a Fossil repository to be exported.
97 **
98 ** Only check-ins are exported using --git. Git does not support tickets
99 ** or wiki or events or attachments, so none of those are exported.
 
 
 
 
 
 
100 */
101 void export_cmd(void){
102 Stmt q;
103 int i;
104 int firstCkin; /* Integer offset to check-in marks */
105 Bag blobs, vers;
 
 
 
106 bag_init(&blobs);
107 bag_init(&vers);
108
109 find_option("git", 0, 0); /* Ignore the --git option for now */
 
 
 
110 db_find_and_open_repository(0, 2);
111 verify_all_options();
112 if( g.argc!=2 && g.argc!=3 ){ usage("--git ?REPOSITORY?"); }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
113
114 /* Step 1: Generate "blob" records for every artifact that is part
115 ** of a check-in
116 */
117 fossil_binary_mode(stdout);
118 db_prepare(&q, "SELECT DISTINCT fid FROM mlink WHERE fid>0");
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119 while( db_step(&q)==SQLITE_ROW ){
120 int rid = db_column_int(&q, 0);
121 Blob content;
122 content_get(rid, &content);
123 printf("blob\nmark :%d\ndata %d\n", rid, blob_size(&content));
124 bag_insert(&blobs, rid);
125 fwrite(blob_buffer(&content), 1, blob_size(&content), stdout);
126 printf("\n");
127 blob_reset(&content);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
128 }
129 db_finalize(&q);
 
 
130
131 /* Output the commit records.
132 */
133 firstCkin = db_int(0, "SELECT max(rid) FROM blob")+1;
134 db_prepare(&q,
135 "SELECT strftime('%%s',mtime), objid, coalesce(comment,ecomment),"
136 " coalesce(user,euser),"
137 " (SELECT value FROM tagxref WHERE rid=objid AND tagid=%d)"
138 " FROM event"
139 " WHERE type='ci'"
140 " ORDER BY mtime ASC",
141 TAG_BRANCH
142 );
 
143 while( db_step(&q)==SQLITE_ROW ){
 
144 const char *zSecondsSince1970 = db_column_text(&q, 0);
145 int ckinId = db_column_int(&q, 1);
146 const char *zComment = db_column_text(&q, 2);
147 const char *zUser = db_column_text(&q, 3);
148 const char *zBranch = db_column_text(&q, 4);
149 char *zBr;
150 Manifest *p;
151 ManifestFile *pFile;
152 const char *zFromType;
153
154 bag_insert(&vers, ckinId);
 
 
 
155 if( zBranch==0 ) zBranch = "trunk";
156 zBr = mprintf("%s", zBranch);
157 for(i=0; zBr[i]; i++){
158 if( !fossil_isalnum(zBr[i]) ) zBr[i] = '_';
159 }
160 printf("commit refs/heads/%s\nmark :%d\n", zBr, ckinId+firstCkin);
161 free(zBr);
162 printf("committer");
163 print_person(zUser);
164 printf(" %s +0000\n", zSecondsSince1970);
165 if( zComment==0 ) zComment = "null comment";
166 printf("data %d\n%s\n", (int)strlen(zComment), zComment);
167 p = manifest_get(ckinId, CFTYPE_ANY);
168 zFromType = "from";
169 for(i=0; i<p->nParent; i++){
170 int pid = fast_uuid_to_rid(p->azParent[i]);
171 if( pid==0 || !bag_find(&vers, pid) ) continue;
172 printf("%s :%d\n", zFromType, fast_uuid_to_rid(p->azParent[i])+firstCkin);
173 zFromType = "merge";
174 }
175 printf("deleteall\n");
176 manifest_file_rewind(p);
177 while( (pFile=manifest_file_next(p, 0))!=0 ){
178 int fid = fast_uuid_to_rid(pFile->zUuid);
179 const char *zPerm = "100644";
180 if( fid==0 ) continue;
181 if( pFile->zPerm && strstr(pFile->zPerm,"x") ) zPerm = "100755";
182 if( !bag_find(&blobs, fid) ) continue;
183 printf("M %s :%d %s\n", zPerm, fid, pFile->zName);
184 }
185 manifest_cache_insert(p);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
186 printf("\n");
187 }
 
188 db_finalize(&q);
189 bag_clear(&blobs);
190 manifest_cache_clear();
191
192
@@ -196,17 +290,45 @@
196 " FROM tagxref JOIN tag USING(tagid)"
197 " WHERE tagtype=1 AND tagname GLOB 'sym-*'"
198 );
199 while( db_step(&q)==SQLITE_ROW ){
200 const char *zTagname = db_column_text(&q, 0);
 
201 int rid = db_column_int(&q, 1);
202 const char *zSecSince1970 = db_column_text(&q, 2);
 
203 if( rid==0 || !bag_find(&vers, rid) ) continue;
204 zTagname += 4;
205 printf("tag %s\n", zTagname);
206 printf("from :%d\n", rid+firstCkin);
 
 
 
 
207 printf("tagger <tagger> %s +0000\n", zSecSince1970);
208 printf("data 0\n");
 
209 }
210 db_finalize(&q);
211 bag_clear(&vers);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
212 }
213
--- src/export.c
+++ src/export.c
@@ -39,11 +39,11 @@
39 db_bind_text(&q, ":user", zUser);
40 if( db_step(&q)!=SQLITE_ROW ){
41 db_reset(&q);
42 for(i=0; zUser[i] && zUser[i]!='>' && zUser[i]!='<'; i++){}
43 if( zUser[i]==0 ){
44 printf(" %s <%s>", zUser, zUser);
45 return;
46 }
47 zName = mprintf("%s", zUser);
48 for(i=j=0; zName[i]; i++){
49 if( zName[i]!='<' && zName[i]!='>' ){
@@ -56,11 +56,11 @@
56 return;
57 }
58 zContact = db_column_text(&q, 0);
59 for(i=0; zContact[i] && zContact[i]!='>' && zContact[i]!='<'; i++){}
60 if( zContact[i]==0 ){
61 printf(" %s <%s>", zContact[0] ? zContact : zUser, zUser);
62 db_reset(&q);
63 return;
64 }
65 if( zContact[i]=='<' ){
66 zEmail = mprintf("%s", &zContact[i]);
@@ -78,15 +78,17 @@
78 free(zName);
79 free(zEmail);
80 db_reset(&q);
81 }
82
83 #define BLOBMARK(rid) ((rid) * 2)
84 #define COMMITMARK(rid) ((rid) * 2 + 1)
85
86 /*
87 ** COMMAND: export
88 **
89 ** Usage: %fossil export --git ?options? ?REPOSITORY?
90 **
91 ** Write an export of all check-ins to standard output. The export is
92 ** written in the git-fast-export file format assuming the --git option is
93 ** provided. The git-fast-export format is currently the only VCS
94 ** interchange format supported, though other formats may be added in
@@ -95,98 +97,190 @@
97 ** Run this command within a checkout. Or use the -R or --repository
98 ** option to specify a Fossil repository to be exported.
99 **
100 ** Only check-ins are exported using --git. Git does not support tickets
101 ** or wiki or events or attachments, so none of those are exported.
102 **
103 ** If the "--import-marks FILE" option is used, it contains a list of
104 ** rids to skip.
105 **
106 ** If the "--export-marks FILE" option is used, the rid of all commits and
107 ** blobs written on exit for use with "--import-marks" on the next run.
108 */
109 void export_cmd(void){
110 Stmt q, q2, q3;
111 int i;
 
112 Bag blobs, vers;
113 const char *markfile_in;
114 const char *markfile_out;
115
116 bag_init(&blobs);
117 bag_init(&vers);
118
119 find_option("git", 0, 0); /* Ignore the --git option for now */
120 markfile_in = find_option("import-marks", 0, 1);
121 markfile_out = find_option("export-marks", 0, 1);
122
123 db_find_and_open_repository(0, 2);
124 verify_all_options();
125 if( g.argc!=2 && g.argc!=3 ){ usage("--git ?REPOSITORY?"); }
126
127 db_multi_exec("CREATE TEMPORARY TABLE oldblob(rid INTEGER PRIMARY KEY)");
128 db_multi_exec("CREATE TEMPORARY TABLE oldcommit(rid INTEGER PRIMARY KEY)");
129 if( markfile_in!=0 ){
130 Stmt qb,qc;
131 char line[100];
132 FILE *f;
133
134 f = fopen(markfile_in, "r");
135 if( f==0 ){
136 fossil_panic("cannot open %s for reading", markfile_in);
137 }
138 db_prepare(&qb, "INSERT OR IGNORE INTO oldblob VALUES (:rid)");
139 db_prepare(&qc, "INSERT OR IGNORE INTO oldcommit VALUES (:rid)");
140 while( fgets(line, sizeof(line), f)!=0 ){
141 if( *line == 'b' ){
142 db_bind_text(&qb, ":rid", line + 1);
143 db_step(&qb);
144 db_reset(&qb);
145 bag_insert(&blobs, atoi(line + 1));
146 }else if( *line == 'c' ){
147 db_bind_text(&qc, ":rid", line + 1);
148 db_step(&qc);
149 db_reset(&qc);
150 bag_insert(&vers, atoi(line + 1));
151 }else{
152 fossil_panic("bad input from %s: %s", markfile_in, line);
153 }
154 }
155 db_finalize(&qb);
156 db_finalize(&qc);
157 fclose(f);
158 }
159
160 /* Step 1: Generate "blob" records for every artifact that is part
161 ** of a check-in
162 */
163 fossil_binary_mode(stdout);
164 db_multi_exec("CREATE TEMPORARY TABLE newblob(rid INTEGER KEY, srcid INTEGER)");
165 db_multi_exec("CREATE INDEX newblob_src ON newblob(srcid)");
166 db_multi_exec(
167 "INSERT INTO newblob"
168 " SELECT DISTINCT fid,"
169 " CASE WHEN EXISTS(SELECT 1 FROM delta WHERE rid=fid AND NOT EXISTS(SELECT 1 FROM oldblob WHERE srcid=fid))"
170 " THEN (SELECT srcid FROM delta WHERE rid=fid)"
171 " ELSE 0"
172 " END"
173 " FROM mlink"
174 " WHERE fid>0 AND NOT EXISTS(SELECT 1 FROM oldblob WHERE rid=fid)");
175 db_prepare(&q,
176 "SELECT DISTINCT fid FROM mlink"
177 " WHERE fid>0 AND NOT EXISTS(SELECT 1 FROM oldblob WHERE rid=fid)");
178 db_prepare(&q2, "INSERT INTO oldblob VALUES (:rid)");
179 db_prepare(&q3, "SELECT rid FROM newblob WHERE srcid= (:srcid)");
180 while( db_step(&q)==SQLITE_ROW ){
181 int rid = db_column_int(&q, 0);
182 Blob content;
183
184 while( !bag_find(&blobs, rid) ){
185 content_get(rid, &content);
186 db_bind_int(&q2, ":rid", rid);
187 db_step(&q2);
188 db_reset(&q2);
189 printf("blob\nmark :%d\ndata %d\n", BLOBMARK(rid), blob_size(&content));
190 bag_insert(&blobs, rid);
191 fwrite(blob_buffer(&content), 1, blob_size(&content), stdout);
192 printf("\n");
193 blob_reset(&content);
194
195 db_bind_int(&q3, ":srcid", rid);
196 if( db_step(&q3) != SQLITE_ROW ){
197 db_reset(&q3);
198 break;
199 }
200 rid = db_column_int(&q3, 0);
201 db_reset(&q3);
202 }
203 }
204 db_finalize(&q);
205 db_finalize(&q2);
206 db_finalize(&q3);
207
208 /* Output the commit records.
209 */
 
210 db_prepare(&q,
211 "SELECT strftime('%%s',mtime), objid, coalesce(comment,ecomment),"
212 " coalesce(user,euser),"
213 " (SELECT value FROM tagxref WHERE rid=objid AND tagid=%d)"
214 " FROM event"
215 " WHERE type='ci' AND NOT EXISTS (SELECT 1 FROM oldcommit WHERE objid=rid)"
216 " ORDER BY mtime ASC",
217 TAG_BRANCH
218 );
219 db_prepare(&q2, "INSERT INTO oldcommit VALUES (:rid)");
220 while( db_step(&q)==SQLITE_ROW ){
221 Stmt q4;
222 const char *zSecondsSince1970 = db_column_text(&q, 0);
223 int ckinId = db_column_int(&q, 1);
224 const char *zComment = db_column_text(&q, 2);
225 const char *zUser = db_column_text(&q, 3);
226 const char *zBranch = db_column_text(&q, 4);
227 char *zBr;
 
 
 
228
229 bag_insert(&vers, ckinId);
230 db_bind_int(&q2, ":rid", ckinId);
231 db_step(&q2);
232 db_reset(&q2);
233 if( zBranch==0 ) zBranch = "trunk";
234 zBr = mprintf("%s", zBranch);
235 for(i=0; zBr[i]; i++){
236 if( !fossil_isalnum(zBr[i]) ) zBr[i] = '_';
237 }
238 printf("commit refs/heads/%s\nmark :%d\n", zBr, COMMITMARK(ckinId));
239 free(zBr);
240 printf("committer");
241 print_person(zUser);
242 printf(" %s +0000\n", zSecondsSince1970);
243 if( zComment==0 ) zComment = "null comment";
244 printf("data %d\n%s\n", (int)strlen(zComment), zComment);
245 db_prepare(&q3, "SELECT pid FROM plink WHERE cid=%d AND isprim", ckinId);
246 if( db_step(&q3) == SQLITE_ROW ){
247 printf("from :%d\n", COMMITMARK(db_column_int(&q3, 0)));
248 db_prepare(&q4,
249 "SELECT pid FROM plink"
250 " WHERE cid=%d AND NOT isprim"
251 " AND NOT EXISTS(SELECT 1 FROM phantom WHERE rid=pid)"
252 " ORDER BY pid",
253 ckinId);
254 while( db_step(&q4)==SQLITE_ROW ){
255 printf("merge :%d\n", COMMITMARK(db_column_int(&q4,0)));
256 }
257 db_finalize(&q4);
258 }else{
259 printf("deleteall\n");
260 }
261
262 db_prepare(&q4,
263 "SELECT filename.name, mlink.fid, mlink.mperm FROM mlink"
264 " JOIN filename ON filename.fnid=mlink.fnid"
265 " WHERE mlink.mid=%d",
266 ckinId
267 );
268 while( db_step(&q4)==SQLITE_ROW ){
269 const char *zName = db_column_text(&q4,0);
270 int zNew = db_column_int(&q4,1);
271 int mPerm = db_column_int(&q4,2);
272 if( zNew==0)
273 printf("D %s\n", zName);
274 else if( bag_find(&blobs, zNew) )
275 printf("M %s :%d %s\n", mPerm ? "100755" : "100644", BLOBMARK(zNew), zName);
276 }
277 db_finalize(&q4);
278 db_finalize(&q3);
279 printf("\n");
280 }
281 db_finalize(&q2);
282 db_finalize(&q);
283 bag_clear(&blobs);
284 manifest_cache_clear();
285
286
@@ -196,17 +290,45 @@
290 " FROM tagxref JOIN tag USING(tagid)"
291 " WHERE tagtype=1 AND tagname GLOB 'sym-*'"
292 );
293 while( db_step(&q)==SQLITE_ROW ){
294 const char *zTagname = db_column_text(&q, 0);
295 char *zEncoded = 0;
296 int rid = db_column_int(&q, 1);
297 const char *zSecSince1970 = db_column_text(&q, 2);
298 int i;
299 if( rid==0 || !bag_find(&vers, rid) ) continue;
300 zTagname += 4;
301 zEncoded = mprintf("%s", zTagname);
302 for(i=0; zEncoded[i]; i++){
303 if( !fossil_isalnum(zEncoded[i]) ) zEncoded[i] = '_';
304 }
305 printf("tag %s\n", zEncoded);
306 printf("from :%d\n", COMMITMARK(rid));
307 printf("tagger <tagger> %s +0000\n", zSecSince1970);
308 printf("data 0\n");
309 fossil_free(zEncoded);
310 }
311 db_finalize(&q);
312 bag_clear(&vers);
313
314 if( markfile_out!=0 ){
315 FILE *f;
316 f = fopen(markfile_out, "w");
317 if( f == 0 ){
318 fossil_panic("cannot open %s for writing", markfile_out);
319 }
320 db_prepare(&q, "SELECT rid FROM oldblob");
321 while( db_step(&q)==SQLITE_ROW ){
322 fprintf(f, "b%d\n", db_column_int(&q, 0));
323 }
324 db_finalize(&q);
325 db_prepare(&q, "SELECT rid FROM oldcommit");
326 while( db_step(&q)==SQLITE_ROW ){
327 fprintf(f, "c%d\n", db_column_int(&q, 0));
328 }
329 db_finalize(&q);
330 if( ferror(f)!=0 || fclose(f)!=0 ) {
331 fossil_panic("error while writing %s", markfile_out);
332 }
333 }
334 }
335
M src/file.c
+35 -7
--- src/file.c
+++ src/file.c
@@ -19,10 +19,12 @@
1919
 */
2020
 #include "config.h"
2121
 #include <sys/types.h>
2222
 #include <sys/stat.h>
2323
 #include <unistd.h>
24
+#include <string.h>
25
+#include <errno.h>
2426
 #include "file.h"
2527
2628
 /*
2729
 ** The file status information from the most recent stat() call.
2830
 **
@@ -397,11 +399,11 @@
397399
 char *zPwdUtf8;
398400
 int nPwd;
399401
 int i;
400402
 char zPwd[2000];
401403
 if( getcwd(zPwd, sizeof(zPwd)-1)==0 ){
402
- fossil_fatal("pwd too big: max %d\n", (int)sizeof(zPwd)-1);
404
+ fossil_fatal("cannot find the current working directory.");
403405
 }
404406
 zPwdUtf8 = fossil_mbcs_to_utf8(zPwd);
405407
 nPwd = strlen(zPwdUtf8);
406408
 if( nPwd > nBuf-1 ){
407409
 fossil_fatal("pwd too big: max %d\n", nBuf-1);
@@ -409,11 +411,16 @@
409411
 for(i=0; zPwdUtf8[i]; i++) if( zPwdUtf8[i]=='\\' ) zPwdUtf8[i] = '/';
410412
 memcpy(zBuf, zPwdUtf8, nPwd+1);
411413
 fossil_mbcs_free(zPwdUtf8);
412414
 #else
413415
 if( getcwd(zBuf, nBuf-1)==0 ){
414
- fossil_fatal("pwd too big: max %d\n", nBuf-1);
416
+ if( errno==ERANGE ){
417
+ fossil_fatal("pwd too big: max %d\n", nBuf-1);
418
+ }else{
419
+ fossil_fatal("cannot find current working directory; %s",
420
+ strerror(errno));
421
+ }
415422
 }
416423
 #endif
417424
 }
418425
419426
 /*
@@ -492,26 +499,45 @@
492499
 }
493500
 }
494501
 }
495502
 return 1;
496503
 }
504
+
505
+/*
506
+** Return a pointer to the first character in a pathname past the
507
+** drive letter. This routine is a no-op on unix.
508
+*/
509
+char *file_without_drive_letter(char *zIn){
510
+#ifdef _WIN32
511
+ if( fossil_isalpha(zIn[0]) && zIn[1]==':' ) zIn += 2;
512
+#endif
513
+ return zIn;
514
+}
497515
498516
 /*
499517
 ** Compute a pathname for a file or directory that is relative
500518
 ** to the current directory.
501519
 */
502520
 void file_relative_name(const char *zOrigName, Blob *pOut){
503521
 char *zPath;
504522
 blob_set(pOut, zOrigName);
505523
 blob_resize(pOut, file_simplify_name(blob_buffer(pOut), blob_size(pOut)));
506
- zPath = blob_buffer(pOut);
524
+ zPath = file_without_drive_letter(blob_buffer(pOut));
507525
 if( zPath[0]=='/' ){
508526
 int i, j;
509527
 Blob tmp;
510
- char zPwd[2000];
511
- file_getcwd(zPwd, sizeof(zPwd)-20);
512
- for(i=1; zPath[i] && zPwd[i]==zPath[i]; i++){}
528
+ char *zPwd;
529
+ char zBuf[2000];
530
+ zPwd = zBuf;
531
+ file_getcwd(zBuf, sizeof(zBuf)-20);
532
+ zPwd = file_without_drive_letter(zBuf);
533
+ i = 1;
534
+#ifdef _WIN32
535
+ while( zPath[i] && fossil_tolower(zPwd[i])==fossil_tolower(zPath[i]) ) i++;
536
+#else
537
+ while( zPath[i] && zPwd[i]==zPath[i] ) i++;
538
+#endif
513539
 if( zPath[i]==0 ){
514540
 blob_reset(pOut);
515541
 if( zPwd[i]==0 ){
516542
 blob_append(pOut, ".", 1);
517543
 }else{
@@ -672,10 +698,11 @@
672698
 "abcdefghijklmnopqrstuvwxyz"
673699
 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
674700
 "0123456789";
675701
 unsigned int i, j;
676702
 const char *zDir = ".";
703
+ int cnt = 0;
677704
678705
 for(i=0; i<sizeof(azDirs)/sizeof(azDirs[0]); i++){
679706
 if( !file_isdir(azDirs[i]) ) continue;
680707
 zDir = azDirs[i];
681708
 break;
@@ -687,18 +714,19 @@
687714
 if( (strlen(zDir) + 17) >= (size_t)nBuf ){
688715
 fossil_fatal("insufficient space for temporary filename");
689716
 }
690717
691718
 do{
719
+ if( cnt++>20 ) fossil_panic("cannot generate a temporary filename");
692720
 sqlite3_snprintf(nBuf-17, zBuf, "%s/", zDir);
693721
 j = (int)strlen(zBuf);
694722
 sqlite3_randomness(15, &zBuf[j]);
695723
 for(i=0; i<15; i++, j++){
696724
 zBuf[j] = (char)zChars[ ((unsigned char)zBuf[j])%(sizeof(zChars)-1) ];
697725
 }
698726
 zBuf[j] = 0;
699
- }while( file_size(zBuf)<0 );
727
+ }while( file_size(zBuf)>=0 );
700728
 }
701729
702730
703731
 /*
704732
 ** Return true if a file named zName exists and has identical content
705733
--- src/file.c
+++ src/file.c
@@ -19,10 +19,12 @@
19 */
20 #include "config.h"
21 #include <sys/types.h>
22 #include <sys/stat.h>
23 #include <unistd.h>
 
 
24 #include "file.h"
25
26 /*
27 ** The file status information from the most recent stat() call.
28 **
@@ -397,11 +399,11 @@
397 char *zPwdUtf8;
398 int nPwd;
399 int i;
400 char zPwd[2000];
401 if( getcwd(zPwd, sizeof(zPwd)-1)==0 ){
402 fossil_fatal("pwd too big: max %d\n", (int)sizeof(zPwd)-1);
403 }
404 zPwdUtf8 = fossil_mbcs_to_utf8(zPwd);
405 nPwd = strlen(zPwdUtf8);
406 if( nPwd > nBuf-1 ){
407 fossil_fatal("pwd too big: max %d\n", nBuf-1);
@@ -409,11 +411,16 @@
409 for(i=0; zPwdUtf8[i]; i++) if( zPwdUtf8[i]=='\\' ) zPwdUtf8[i] = '/';
410 memcpy(zBuf, zPwdUtf8, nPwd+1);
411 fossil_mbcs_free(zPwdUtf8);
412 #else
413 if( getcwd(zBuf, nBuf-1)==0 ){
414 fossil_fatal("pwd too big: max %d\n", nBuf-1);
 
 
 
 
 
415 }
416 #endif
417 }
418
419 /*
@@ -492,26 +499,45 @@
492 }
493 }
494 }
495 return 1;
496 }
 
 
 
 
 
 
 
 
 
 
 
497
498 /*
499 ** Compute a pathname for a file or directory that is relative
500 ** to the current directory.
501 */
502 void file_relative_name(const char *zOrigName, Blob *pOut){
503 char *zPath;
504 blob_set(pOut, zOrigName);
505 blob_resize(pOut, file_simplify_name(blob_buffer(pOut), blob_size(pOut)));
506 zPath = blob_buffer(pOut);
507 if( zPath[0]=='/' ){
508 int i, j;
509 Blob tmp;
510 char zPwd[2000];
511 file_getcwd(zPwd, sizeof(zPwd)-20);
512 for(i=1; zPath[i] && zPwd[i]==zPath[i]; i++){}
 
 
 
 
 
 
 
 
513 if( zPath[i]==0 ){
514 blob_reset(pOut);
515 if( zPwd[i]==0 ){
516 blob_append(pOut, ".", 1);
517 }else{
@@ -672,10 +698,11 @@
672 "abcdefghijklmnopqrstuvwxyz"
673 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
674 "0123456789";
675 unsigned int i, j;
676 const char *zDir = ".";
 
677
678 for(i=0; i<sizeof(azDirs)/sizeof(azDirs[0]); i++){
679 if( !file_isdir(azDirs[i]) ) continue;
680 zDir = azDirs[i];
681 break;
@@ -687,18 +714,19 @@
687 if( (strlen(zDir) + 17) >= (size_t)nBuf ){
688 fossil_fatal("insufficient space for temporary filename");
689 }
690
691 do{
 
692 sqlite3_snprintf(nBuf-17, zBuf, "%s/", zDir);
693 j = (int)strlen(zBuf);
694 sqlite3_randomness(15, &zBuf[j]);
695 for(i=0; i<15; i++, j++){
696 zBuf[j] = (char)zChars[ ((unsigned char)zBuf[j])%(sizeof(zChars)-1) ];
697 }
698 zBuf[j] = 0;
699 }while( file_size(zBuf)<0 );
700 }
701
702
703 /*
704 ** Return true if a file named zName exists and has identical content
705
--- src/file.c
+++ src/file.c
@@ -19,10 +19,12 @@
19 */
20 #include "config.h"
21 #include <sys/types.h>
22 #include <sys/stat.h>
23 #include <unistd.h>
24 #include <string.h>
25 #include <errno.h>
26 #include "file.h"
27
28 /*
29 ** The file status information from the most recent stat() call.
30 **
@@ -397,11 +399,11 @@
399 char *zPwdUtf8;
400 int nPwd;
401 int i;
402 char zPwd[2000];
403 if( getcwd(zPwd, sizeof(zPwd)-1)==0 ){
404 fossil_fatal("cannot find the current working directory.");
405 }
406 zPwdUtf8 = fossil_mbcs_to_utf8(zPwd);
407 nPwd = strlen(zPwdUtf8);
408 if( nPwd > nBuf-1 ){
409 fossil_fatal("pwd too big: max %d\n", nBuf-1);
@@ -409,11 +411,16 @@
411 for(i=0; zPwdUtf8[i]; i++) if( zPwdUtf8[i]=='\\' ) zPwdUtf8[i] = '/';
412 memcpy(zBuf, zPwdUtf8, nPwd+1);
413 fossil_mbcs_free(zPwdUtf8);
414 #else
415 if( getcwd(zBuf, nBuf-1)==0 ){
416 if( errno==ERANGE ){
417 fossil_fatal("pwd too big: max %d\n", nBuf-1);
418 }else{
419 fossil_fatal("cannot find current working directory; %s",
420 strerror(errno));
421 }
422 }
423 #endif
424 }
425
426 /*
@@ -492,26 +499,45 @@
499 }
500 }
501 }
502 return 1;
503 }
504
505 /*
506 ** Return a pointer to the first character in a pathname past the
507 ** drive letter. This routine is a no-op on unix.
508 */
509 char *file_without_drive_letter(char *zIn){
510 #ifdef _WIN32
511 if( fossil_isalpha(zIn[0]) && zIn[1]==':' ) zIn += 2;
512 #endif
513 return zIn;
514 }
515
516 /*
517 ** Compute a pathname for a file or directory that is relative
518 ** to the current directory.
519 */
520 void file_relative_name(const char *zOrigName, Blob *pOut){
521 char *zPath;
522 blob_set(pOut, zOrigName);
523 blob_resize(pOut, file_simplify_name(blob_buffer(pOut), blob_size(pOut)));
524 zPath = file_without_drive_letter(blob_buffer(pOut));
525 if( zPath[0]=='/' ){
526 int i, j;
527 Blob tmp;
528 char *zPwd;
529 char zBuf[2000];
530 zPwd = zBuf;
531 file_getcwd(zBuf, sizeof(zBuf)-20);
532 zPwd = file_without_drive_letter(zBuf);
533 i = 1;
534 #ifdef _WIN32
535 while( zPath[i] && fossil_tolower(zPwd[i])==fossil_tolower(zPath[i]) ) i++;
536 #else
537 while( zPath[i] && zPwd[i]==zPath[i] ) i++;
538 #endif
539 if( zPath[i]==0 ){
540 blob_reset(pOut);
541 if( zPwd[i]==0 ){
542 blob_append(pOut, ".", 1);
543 }else{
@@ -672,10 +698,11 @@
698 "abcdefghijklmnopqrstuvwxyz"
699 "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
700 "0123456789";
701 unsigned int i, j;
702 const char *zDir = ".";
703 int cnt = 0;
704
705 for(i=0; i<sizeof(azDirs)/sizeof(azDirs[0]); i++){
706 if( !file_isdir(azDirs[i]) ) continue;
707 zDir = azDirs[i];
708 break;
@@ -687,18 +714,19 @@
714 if( (strlen(zDir) + 17) >= (size_t)nBuf ){
715 fossil_fatal("insufficient space for temporary filename");
716 }
717
718 do{
719 if( cnt++>20 ) fossil_panic("cannot generate a temporary filename");
720 sqlite3_snprintf(nBuf-17, zBuf, "%s/", zDir);
721 j = (int)strlen(zBuf);
722 sqlite3_randomness(15, &zBuf[j]);
723 for(i=0; i<15; i++, j++){
724 zBuf[j] = (char)zChars[ ((unsigned char)zBuf[j])%(sizeof(zChars)-1) ];
725 }
726 zBuf[j] = 0;
727 }while( file_size(zBuf)>=0 );
728 }
729
730
731 /*
732 ** Return true if a file named zName exists and has identical content
733
M src/finfo.c
+9
--- src/finfo.c
+++ src/finfo.c
@@ -193,10 +193,12 @@
193193
 ** Additional query parameters:
194194
 **
195195
 ** a=DATE Only show changes after DATE
196196
 ** b=DATE Only show changes before DATE
197197
 ** n=NUM Show the first NUM changes only
198
+** brbg Background color by branch name
199
+** ubg Background color by user name
198200
 */
199201
 void finfo_page(void){
200202
 Stmt q;
201203
 const char *zFilename;
202204
 char zPrevDate[20];
@@ -204,10 +206,12 @@
204206
 const char *zB;
205207
 int n;
206208
 Blob title;
207209
 Blob sql;
208210
 GraphContext *pGraph;
211
+ int brBg = P("brbg")!=0;
212
+ int uBg = P("ubg")!=0;
209213
210214
 login_check_credentials();
211215
 if( !g.okRead ){ login_needed(); return; }
212216
 style_header("File History");
213217
 login_anonymous_available();
@@ -268,10 +272,15 @@
268272
 int gidx;
269273
 char zTime[10];
270274
 char zShort[20];
271275
 char zShortCkin[20];
272276
 if( zBr==0 ) zBr = "trunk";
277
+ if( uBg ){
278
+ zBgClr = hash_color(zUser);
279
+ }else if( brBg || zBgClr==0 || zBgClr[0]==0 ){
280
+ zBgClr = strcmp(zBr,"trunk")==0 ? "white" : hash_color(zBr);
281
+ }
273282
 gidx = graph_add_row(pGraph, frid, fpid>0 ? 1 : 0, &fpid, zBr, zBgClr, 0);
274283
 if( memcmp(zDate, zPrevDate, 10) ){
275284
 sqlite3_snprintf(sizeof(zPrevDate), zPrevDate, "%.10s", zDate);
276285
 @ <tr><td>
277286
 @ <div class="divider">%s(zPrevDate)</div>
278287
--- src/finfo.c
+++ src/finfo.c
@@ -193,10 +193,12 @@
193 ** Additional query parameters:
194 **
195 ** a=DATE Only show changes after DATE
196 ** b=DATE Only show changes before DATE
197 ** n=NUM Show the first NUM changes only
 
 
198 */
199 void finfo_page(void){
200 Stmt q;
201 const char *zFilename;
202 char zPrevDate[20];
@@ -204,10 +206,12 @@
204 const char *zB;
205 int n;
206 Blob title;
207 Blob sql;
208 GraphContext *pGraph;
 
 
209
210 login_check_credentials();
211 if( !g.okRead ){ login_needed(); return; }
212 style_header("File History");
213 login_anonymous_available();
@@ -268,10 +272,15 @@
268 int gidx;
269 char zTime[10];
270 char zShort[20];
271 char zShortCkin[20];
272 if( zBr==0 ) zBr = "trunk";
 
 
 
 
 
273 gidx = graph_add_row(pGraph, frid, fpid>0 ? 1 : 0, &fpid, zBr, zBgClr, 0);
274 if( memcmp(zDate, zPrevDate, 10) ){
275 sqlite3_snprintf(sizeof(zPrevDate), zPrevDate, "%.10s", zDate);
276 @ <tr><td>
277 @ <div class="divider">%s(zPrevDate)</div>
278
--- src/finfo.c
+++ src/finfo.c
@@ -193,10 +193,12 @@
193 ** Additional query parameters:
194 **
195 ** a=DATE Only show changes after DATE
196 ** b=DATE Only show changes before DATE
197 ** n=NUM Show the first NUM changes only
198 ** brbg Background color by branch name
199 ** ubg Background color by user name
200 */
201 void finfo_page(void){
202 Stmt q;
203 const char *zFilename;
204 char zPrevDate[20];
@@ -204,10 +206,12 @@
206 const char *zB;
207 int n;
208 Blob title;
209 Blob sql;
210 GraphContext *pGraph;
211 int brBg = P("brbg")!=0;
212 int uBg = P("ubg")!=0;
213
214 login_check_credentials();
215 if( !g.okRead ){ login_needed(); return; }
216 style_header("File History");
217 login_anonymous_available();
@@ -268,10 +272,15 @@
272 int gidx;
273 char zTime[10];
274 char zShort[20];
275 char zShortCkin[20];
276 if( zBr==0 ) zBr = "trunk";
277 if( uBg ){
278 zBgClr = hash_color(zUser);
279 }else if( brBg || zBgClr==0 || zBgClr[0]==0 ){
280 zBgClr = strcmp(zBr,"trunk")==0 ? "white" : hash_color(zBr);
281 }
282 gidx = graph_add_row(pGraph, frid, fpid>0 ? 1 : 0, &fpid, zBr, zBgClr, 0);
283 if( memcmp(zDate, zPrevDate, 10) ){
284 sqlite3_snprintf(sizeof(zPrevDate), zPrevDate, "%.10s", zDate);
285 @ <tr><td>
286 @ <div class="divider">%s(zPrevDate)</div>
287
M src/glob.c
+4 -4
--- src/glob.c
+++ src/glob.c
@@ -110,24 +110,24 @@
110110
 p = fossil_malloc( sizeof(*p) + nList+1 );
111111
 memset(p, 0, sizeof(*p));
112112
 z = (char*)&p[1];
113113
 memcpy(z, zPatternList, nList+1);
114114
 while( z[0] ){
115
- while( z[0]==',' || z[0]==' ' ) z++; /* Skip leading spaces */
115
+ while( z[0]==',' || z[0]==' ' || z[0]=='\n' || z[0]=='\r' ) z++; /* Skip leading spaces and newlines */
116116
 if( z[0]=='\'' || z[0]=='"' ){
117117
 delimiter = z[0];
118118
 z++;
119119
 }else{
120120
 delimiter = ',';
121121
 }
122122
 if( z[0]==0 ) break;
123123
 p->azPattern = fossil_realloc(p->azPattern, (p->nPattern+1)*sizeof(char*) );
124124
 p->azPattern[p->nPattern++] = z;
125
- for(i=0; z[i] && z[i]!=delimiter; i++){}
125
+ for(i=0; z[i] && z[i]!=delimiter && z[i]!='\n' && z[i]!='\r'; i++){}
126126
 if( delimiter==',' ){
127
- /* Remove trailing spaces on a comma-delimited pattern */
128
- for(j=i; j>1 && z[j-1]==' '; j--){}
127
+ /* Remove trailing spaces / newlines on a comma-delimited pattern */
128
+ for(j=i; j>1 && (z[j-1]==' ' || z[j-1]=='\n' || z[j-1]=='\r'); j--){}
129129
 if( j<i ) z[j] = 0;
130130
 }
131131
 if( z[i]==0 ) break;
132132
 z[i] = 0;
133133
 z += i+1;
134134
--- src/glob.c
+++ src/glob.c
@@ -110,24 +110,24 @@
110 p = fossil_malloc( sizeof(*p) + nList+1 );
111 memset(p, 0, sizeof(*p));
112 z = (char*)&p[1];
113 memcpy(z, zPatternList, nList+1);
114 while( z[0] ){
115 while( z[0]==',' || z[0]==' ' ) z++; /* Skip leading spaces */
116 if( z[0]=='\'' || z[0]=='"' ){
117 delimiter = z[0];
118 z++;
119 }else{
120 delimiter = ',';
121 }
122 if( z[0]==0 ) break;
123 p->azPattern = fossil_realloc(p->azPattern, (p->nPattern+1)*sizeof(char*) );
124 p->azPattern[p->nPattern++] = z;
125 for(i=0; z[i] && z[i]!=delimiter; i++){}
126 if( delimiter==',' ){
127 /* Remove trailing spaces on a comma-delimited pattern */
128 for(j=i; j>1 && z[j-1]==' '; j--){}
129 if( j<i ) z[j] = 0;
130 }
131 if( z[i]==0 ) break;
132 z[i] = 0;
133 z += i+1;
134
--- src/glob.c
+++ src/glob.c
@@ -110,24 +110,24 @@
110 p = fossil_malloc( sizeof(*p) + nList+1 );
111 memset(p, 0, sizeof(*p));
112 z = (char*)&p[1];
113 memcpy(z, zPatternList, nList+1);
114 while( z[0] ){
115 while( z[0]==',' || z[0]==' ' || z[0]=='\n' || z[0]=='\r' ) z++; /* Skip leading spaces and newlines */
116 if( z[0]=='\'' || z[0]=='"' ){
117 delimiter = z[0];
118 z++;
119 }else{
120 delimiter = ',';
121 }
122 if( z[0]==0 ) break;
123 p->azPattern = fossil_realloc(p->azPattern, (p->nPattern+1)*sizeof(char*) );
124 p->azPattern[p->nPattern++] = z;
125 for(i=0; z[i] && z[i]!=delimiter && z[i]!='\n' && z[i]!='\r'; i++){}
126 if( delimiter==',' ){
127 /* Remove trailing spaces / newlines on a comma-delimited pattern */
128 for(j=i; j>1 && (z[j-1]==' ' || z[j-1]=='\n' || z[j-1]=='\r'); j--){}
129 if( j<i ) z[j] = 0;
130 }
131 if( z[i]==0 ) break;
132 z[i] = 0;
133 z += i+1;
134
M src/http.c
+2 -1
--- src/http.c
+++ src/http.c
@@ -110,11 +110,12 @@
110110
 blob_appendf(pHdr, "Authorization: Basic %s\r\n", zEncoded);
111111
 fossil_free(zEncoded);
112112
 fossil_free(zCredentials);
113113
 }
114114
 blob_appendf(pHdr, "Host: %s\r\n", g.urlHostname);
115
- blob_appendf(pHdr, "User-Agent: Fossil/" MANIFEST_VERSION "\r\n");
115
+ blob_appendf(pHdr, "User-Agent: Fossil/" RELEASE_VERSION
116
+ "-" MANIFEST_VERSION "\r\n");
116117
 if( g.fHttpTrace ){
117118
 blob_appendf(pHdr, "Content-Type: application/x-fossil-debug\r\n");
118119
 }else{
119120
 blob_appendf(pHdr, "Content-Type: application/x-fossil\r\n");
120121
 }
121122
--- src/http.c
+++ src/http.c
@@ -110,11 +110,12 @@
110 blob_appendf(pHdr, "Authorization: Basic %s\r\n", zEncoded);
111 fossil_free(zEncoded);
112 fossil_free(zCredentials);
113 }
114 blob_appendf(pHdr, "Host: %s\r\n", g.urlHostname);
115 blob_appendf(pHdr, "User-Agent: Fossil/" MANIFEST_VERSION "\r\n");
 
116 if( g.fHttpTrace ){
117 blob_appendf(pHdr, "Content-Type: application/x-fossil-debug\r\n");
118 }else{
119 blob_appendf(pHdr, "Content-Type: application/x-fossil\r\n");
120 }
121
--- src/http.c
+++ src/http.c
@@ -110,11 +110,12 @@
110 blob_appendf(pHdr, "Authorization: Basic %s\r\n", zEncoded);
111 fossil_free(zEncoded);
112 fossil_free(zCredentials);
113 }
114 blob_appendf(pHdr, "Host: %s\r\n", g.urlHostname);
115 blob_appendf(pHdr, "User-Agent: Fossil/" RELEASE_VERSION
116 "-" MANIFEST_VERSION "\r\n");
117 if( g.fHttpTrace ){
118 blob_appendf(pHdr, "Content-Type: application/x-fossil-debug\r\n");
119 }else{
120 blob_appendf(pHdr, "Content-Type: application/x-fossil\r\n");
121 }
122
M src/http_ssl.c
+61 -1
--- src/http_ssl.c
+++ src/http_ssl.c
@@ -78,23 +78,76 @@
7878
 ** Return the current SSL error message
7979
 */
8080
 const char *ssl_errmsg(void){
8181
 return sslErrMsg;
8282
 }
83
+
84
+/*
85
+** When a server requests a client certificate that hasn't been provided,
86
+** display a warning message explaining what to do next.
87
+*/
88
+static int ssl_client_cert_callback(SSL *ssl, X509 **x509, EVP_PKEY **pkey){
89
+ fossil_warning("The remote server requested a client certificate for "
90
+ "authentication. Specify the pathname to a file containing the PEM "
91
+ "encoded certificate and private key with the --ssl-identity option "
92
+ "or the ssl-identity setting.");
93
+ return 0; /* no cert available */
94
+}
8395
8496
 /*
8597
 ** Call this routine once before any other use of the SSL interface.
8698
 ** This routine does initial configuration of the SSL module.
8799
 */
88100
 void ssl_global_init(void){
101
+ const char *zCaSetting = 0, *zCaFile = 0, *zCaDirectory = 0;
102
+
89103
 if( sslIsInit==0 ){
90104
 SSL_library_init();
91105
 SSL_load_error_strings();
92106
 ERR_load_BIO_strings();
93107
 OpenSSL_add_all_algorithms();
94108
 sslCtx = SSL_CTX_new(SSLv23_client_method());
95
- X509_STORE_set_default_paths(SSL_CTX_get_cert_store(sslCtx));
109
+
110
+ /* Set up acceptable CA root certificates */
111
+ zCaSetting = db_get("ssl-ca-location", 0);
112
+ if( zCaSetting==0 || zCaSetting[0]=='\0' ){
113
+ /* CA location not specified, use platform's default certificate store */
114
+ X509_STORE_set_default_paths(SSL_CTX_get_cert_store(sslCtx));
115
+ }else{
116
+ /* User has specified a CA location, make sure it exists and use it */
117
+ switch( file_isdir(zCaSetting) ){
118
+ case 0: { /* doesn't exist */
119
+ fossil_fatal("ssl-ca-location is set to '%s', "
120
+ "but is not a file or directory", zCaSetting);
121
+ break;
122
+ }
123
+ case 1: { /* directory */
124
+ zCaDirectory = zCaSetting;
125
+ break;
126
+ }
127
+ case 2: { /* file */
128
+ zCaFile = zCaSetting;
129
+ break;
130
+ }
131
+ }
132
+ if( SSL_CTX_load_verify_locations(sslCtx, zCaFile, zCaDirectory)==0 ){
133
+ fossil_fatal("Failed to use CA root certificates from "
134
+ "ssl-ca-location '%s'", zCaSetting);
135
+ }
136
+ }
137
+
138
+ /* Load client SSL identity, preferring the filename specified on the command line */
139
+ const char *identityFile = ( g.zSSLIdentity!= 0) ? g.zSSLIdentity : db_get("ssl-identity", 0);
140
+ if( identityFile!=0 && identityFile[0]!='\0' ){
141
+ if( SSL_CTX_use_certificate_file(sslCtx, identityFile, SSL_FILETYPE_PEM)!= 1
142
+ || SSL_CTX_use_PrivateKey_file(sslCtx, identityFile, SSL_FILETYPE_PEM)!=1 ){
143
+ fossil_fatal("Could not load SSL identity from %s", identityFile);
144
+ }
145
+ }
146
+ /* Register a callback to tell the user what to do when the server asks for a cert */
147
+ SSL_CTX_set_client_cert_cb(sslCtx, ssl_client_cert_callback);
148
+
96149
 sslIsInit = 1;
97150
 }
98151
 }
99152
100153
 /*
@@ -204,10 +257,17 @@
204257
 if( hasSavedCertificate ){
205258
 warning = "WARNING: Certificate doesn't match the "
206259
 "saved certificate for this host!";
207260
 }
208261
 prompt = mprintf("\nUnknown SSL certificate:\n\n%s\n\n%s\n"
262
+ "Either:\n"
263
+ " * verify the certificate is correct using the "
264
+ "SHA1 fingerprint above\n"
265
+ " * use the global ssl-ca-location setting to specify your CA root\n"
266
+ " certificates list\n\n"
267
+ "If you are not expecting this message, answer no and "
268
+ "contact your server\nadministrator.\n\n"
209269
 "Accept certificate [a=always/y/N]? ", desc, warning);
210270
 BIO_free(mem);
211271
212272
 prompt_user(prompt, &ans);
213273
 free(prompt);
214274
--- src/http_ssl.c
+++ src/http_ssl.c
@@ -78,23 +78,76 @@
78 ** Return the current SSL error message
79 */
80 const char *ssl_errmsg(void){
81 return sslErrMsg;
82 }
 
 
 
 
 
 
 
 
 
 
 
 
83
84 /*
85 ** Call this routine once before any other use of the SSL interface.
86 ** This routine does initial configuration of the SSL module.
87 */
88 void ssl_global_init(void){
 
 
89 if( sslIsInit==0 ){
90 SSL_library_init();
91 SSL_load_error_strings();
92 ERR_load_BIO_strings();
93 OpenSSL_add_all_algorithms();
94 sslCtx = SSL_CTX_new(SSLv23_client_method());
95 X509_STORE_set_default_paths(SSL_CTX_get_cert_store(sslCtx));
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
96 sslIsInit = 1;
97 }
98 }
99
100 /*
@@ -204,10 +257,17 @@
204 if( hasSavedCertificate ){
205 warning = "WARNING: Certificate doesn't match the "
206 "saved certificate for this host!";
207 }
208 prompt = mprintf("\nUnknown SSL certificate:\n\n%s\n\n%s\n"
 
 
 
 
 
 
 
209 "Accept certificate [a=always/y/N]? ", desc, warning);
210 BIO_free(mem);
211
212 prompt_user(prompt, &ans);
213 free(prompt);
214
--- src/http_ssl.c
+++ src/http_ssl.c
@@ -78,23 +78,76 @@
78 ** Return the current SSL error message
79 */
80 const char *ssl_errmsg(void){
81 return sslErrMsg;
82 }
83
84 /*
85 ** When a server requests a client certificate that hasn't been provided,
86 ** display a warning message explaining what to do next.
87 */
88 static int ssl_client_cert_callback(SSL *ssl, X509 **x509, EVP_PKEY **pkey){
89 fossil_warning("The remote server requested a client certificate for "
90 "authentication. Specify the pathname to a file containing the PEM "
91 "encoded certificate and private key with the --ssl-identity option "
92 "or the ssl-identity setting.");
93 return 0; /* no cert available */
94 }
95
96 /*
97 ** Call this routine once before any other use of the SSL interface.
98 ** This routine does initial configuration of the SSL module.
99 */
100 void ssl_global_init(void){
101 const char *zCaSetting = 0, *zCaFile = 0, *zCaDirectory = 0;
102
103 if( sslIsInit==0 ){
104 SSL_library_init();
105 SSL_load_error_strings();
106 ERR_load_BIO_strings();
107 OpenSSL_add_all_algorithms();
108 sslCtx = SSL_CTX_new(SSLv23_client_method());
109
110 /* Set up acceptable CA root certificates */
111 zCaSetting = db_get("ssl-ca-location", 0);
112 if( zCaSetting==0 || zCaSetting[0]=='\0' ){
113 /* CA location not specified, use platform's default certificate store */
114 X509_STORE_set_default_paths(SSL_CTX_get_cert_store(sslCtx));
115 }else{
116 /* User has specified a CA location, make sure it exists and use it */
117 switch( file_isdir(zCaSetting) ){
118 case 0: { /* doesn't exist */
119 fossil_fatal("ssl-ca-location is set to '%s', "
120 "but is not a file or directory", zCaSetting);
121 break;
122 }
123 case 1: { /* directory */
124 zCaDirectory = zCaSetting;
125 break;
126 }
127 case 2: { /* file */
128 zCaFile = zCaSetting;
129 break;
130 }
131 }
132 if( SSL_CTX_load_verify_locations(sslCtx, zCaFile, zCaDirectory)==0 ){
133 fossil_fatal("Failed to use CA root certificates from "
134 "ssl-ca-location '%s'", zCaSetting);
135 }
136 }
137
138 /* Load client SSL identity, preferring the filename specified on the command line */
139 const char *identityFile = ( g.zSSLIdentity!= 0) ? g.zSSLIdentity : db_get("ssl-identity", 0);
140 if( identityFile!=0 && identityFile[0]!='\0' ){
141 if( SSL_CTX_use_certificate_file(sslCtx, identityFile, SSL_FILETYPE_PEM)!= 1
142 || SSL_CTX_use_PrivateKey_file(sslCtx, identityFile, SSL_FILETYPE_PEM)!=1 ){
143 fossil_fatal("Could not load SSL identity from %s", identityFile);
144 }
145 }
146 /* Register a callback to tell the user what to do when the server asks for a cert */
147 SSL_CTX_set_client_cert_cb(sslCtx, ssl_client_cert_callback);
148
149 sslIsInit = 1;
150 }
151 }
152
153 /*
@@ -204,10 +257,17 @@
257 if( hasSavedCertificate ){
258 warning = "WARNING: Certificate doesn't match the "
259 "saved certificate for this host!";
260 }
261 prompt = mprintf("\nUnknown SSL certificate:\n\n%s\n\n%s\n"
262 "Either:\n"
263 " * verify the certificate is correct using the "
264 "SHA1 fingerprint above\n"
265 " * use the global ssl-ca-location setting to specify your CA root\n"
266 " certificates list\n\n"
267 "If you are not expecting this message, answer no and "
268 "contact your server\nadministrator.\n\n"
269 "Accept certificate [a=always/y/N]? ", desc, warning);
270 BIO_free(mem);
271
272 prompt_user(prompt, &ans);
273 free(prompt);
274
M src/import.c
+30 -6
--- src/import.c
+++ src/import.c
@@ -207,10 +207,19 @@
207207
 static int mfile_cmp(const void *pLeft, const void *pRight){
208208
 const ImportFile *pA = (const ImportFile*)pLeft;
209209
 const ImportFile *pB = (const ImportFile*)pRight;
210210
 return fossil_strcmp(pA->zName, pB->zName);
211211
 }
212
+
213
+/*
214
+** Compare two strings for sorting.
215
+*/
216
+static int string_cmp(const void *pLeft, const void *pRight){
217
+ const char *zLeft = *(char const **)pLeft;
218
+ const char *zRight = *(char const **)pRight;
219
+ return fossil_strcmp(zLeft, zRight);
220
+}
212221
213222
 /* Forward reference */
214223
 static void import_prior_files(void);
215224
216225
 /*
@@ -218,11 +227,14 @@
218227
 ** manifest artifact to the BLOB table.
219228
 */
220229
 static void finish_commit(void){
221230
 int i;
222231
 char *zFromBranch;
232
+ char *aTCard[4]; /* Array of T cards for manifest */
233
+ int nTCard = 0; /* Entries used in aTCard[] */
223234
 Blob record, cksum;
235
+
224236
 import_prior_files();
225237
 qsort(gg.aFile, gg.nFile, sizeof(gg.aFile[0]), mfile_cmp);
226238
 blob_zero(&record);
227239
 blob_appendf(&record, "C %F\n", gg.zComment);
228240
 blob_appendf(&record, "D %s\n", gg.zDate);
@@ -245,21 +257,33 @@
245257
 zFromBranch = db_text(0, "SELECT brnm FROM xbranch WHERE tname=%Q",
246258
 gg.zFromMark);
247259
 }else{
248260
 zFromBranch = 0;
249261
 }
262
+
263
+ /* Add the required "T" cards to the manifest. Make sure they are added
264
+ ** in sorted order and without any duplicates. Otherwise, fossil will not
265
+ ** recognize the document as a valid manifest. */
250266
 if( !gg.tagCommit && fossil_strcmp(zFromBranch, gg.zBranch)!=0 ){
251
- blob_appendf(&record, "T *branch * %F\n", gg.zBranch);
252
- blob_appendf(&record, "T *sym-%F *\n", gg.zBranch);
267
+ aTCard[nTCard++] = mprintf("T *branch * %F\n", gg.zBranch);
268
+ aTCard[nTCard++] = mprintf("T *sym-%F *\n", gg.zBranch);
253269
 if( zFromBranch ){
254
- blob_appendf(&record, "T -sym-%F *\n", zFromBranch);
270
+ aTCard[nTCard++] = mprintf("T -sym-%F *\n", zFromBranch);
271
+ }
272
+ }
273
+ if( gg.zFrom==0 ){
274
+ aTCard[nTCard++] = mprintf("T *sym-trunk *\n");
275
+ }
276
+ qsort(aTCard, nTCard, sizeof(char *), string_cmp);
277
+ for(i=0; i<nTCard; i++){
278
+ if( i==0 || fossil_strcmp(aTCard[i-1], aTCard[i]) ){
279
+ blob_appendf(&record, "%s", aTCard[i]);
255280
 }
256281
 }
282
+ for(i=0; i<nTCard; i++) free(aTCard[i]);
283
+
257284
 free(zFromBranch);
258
- if( gg.zFrom==0 ){
259
- blob_appendf(&record, "T *sym-trunk *\n");
260
- }
261285
 db_multi_exec("INSERT INTO xbranch(tname, brnm) VALUES(%Q,%Q)",
262286
 gg.zMark, gg.zBranch);
263287
 blob_appendf(&record, "U %F\n", gg.zUser);
264288
 md5sum_blob(&record, &cksum);
265289
 blob_appendf(&record, "Z %b\n", &cksum);
266290
--- src/import.c
+++ src/import.c
@@ -207,10 +207,19 @@
207 static int mfile_cmp(const void *pLeft, const void *pRight){
208 const ImportFile *pA = (const ImportFile*)pLeft;
209 const ImportFile *pB = (const ImportFile*)pRight;
210 return fossil_strcmp(pA->zName, pB->zName);
211 }
 
 
 
 
 
 
 
 
 
212
213 /* Forward reference */
214 static void import_prior_files(void);
215
216 /*
@@ -218,11 +227,14 @@
218 ** manifest artifact to the BLOB table.
219 */
220 static void finish_commit(void){
221 int i;
222 char *zFromBranch;
 
 
223 Blob record, cksum;
 
224 import_prior_files();
225 qsort(gg.aFile, gg.nFile, sizeof(gg.aFile[0]), mfile_cmp);
226 blob_zero(&record);
227 blob_appendf(&record, "C %F\n", gg.zComment);
228 blob_appendf(&record, "D %s\n", gg.zDate);
@@ -245,21 +257,33 @@
245 zFromBranch = db_text(0, "SELECT brnm FROM xbranch WHERE tname=%Q",
246 gg.zFromMark);
247 }else{
248 zFromBranch = 0;
249 }
 
 
 
 
250 if( !gg.tagCommit && fossil_strcmp(zFromBranch, gg.zBranch)!=0 ){
251 blob_appendf(&record, "T *branch * %F\n", gg.zBranch);
252 blob_appendf(&record, "T *sym-%F *\n", gg.zBranch);
253 if( zFromBranch ){
254 blob_appendf(&record, "T -sym-%F *\n", zFromBranch);
 
 
 
 
 
 
 
 
 
255 }
256 }
 
 
257 free(zFromBranch);
258 if( gg.zFrom==0 ){
259 blob_appendf(&record, "T *sym-trunk *\n");
260 }
261 db_multi_exec("INSERT INTO xbranch(tname, brnm) VALUES(%Q,%Q)",
262 gg.zMark, gg.zBranch);
263 blob_appendf(&record, "U %F\n", gg.zUser);
264 md5sum_blob(&record, &cksum);
265 blob_appendf(&record, "Z %b\n", &cksum);
266
--- src/import.c
+++ src/import.c
@@ -207,10 +207,19 @@
207 static int mfile_cmp(const void *pLeft, const void *pRight){
208 const ImportFile *pA = (const ImportFile*)pLeft;
209 const ImportFile *pB = (const ImportFile*)pRight;
210 return fossil_strcmp(pA->zName, pB->zName);
211 }
212
213 /*
214 ** Compare two strings for sorting.
215 */
216 static int string_cmp(const void *pLeft, const void *pRight){
217 const char *zLeft = *(char const **)pLeft;
218 const char *zRight = *(char const **)pRight;
219 return fossil_strcmp(zLeft, zRight);
220 }
221
222 /* Forward reference */
223 static void import_prior_files(void);
224
225 /*
@@ -218,11 +227,14 @@
227 ** manifest artifact to the BLOB table.
228 */
229 static void finish_commit(void){
230 int i;
231 char *zFromBranch;
232 char *aTCard[4]; /* Array of T cards for manifest */
233 int nTCard = 0; /* Entries used in aTCard[] */
234 Blob record, cksum;
235
236 import_prior_files();
237 qsort(gg.aFile, gg.nFile, sizeof(gg.aFile[0]), mfile_cmp);
238 blob_zero(&record);
239 blob_appendf(&record, "C %F\n", gg.zComment);
240 blob_appendf(&record, "D %s\n", gg.zDate);
@@ -245,21 +257,33 @@
257 zFromBranch = db_text(0, "SELECT brnm FROM xbranch WHERE tname=%Q",
258 gg.zFromMark);
259 }else{
260 zFromBranch = 0;
261 }
262
263 /* Add the required "T" cards to the manifest. Make sure they are added
264 ** in sorted order and without any duplicates. Otherwise, fossil will not
265 ** recognize the document as a valid manifest. */
266 if( !gg.tagCommit && fossil_strcmp(zFromBranch, gg.zBranch)!=0 ){
267 aTCard[nTCard++] = mprintf("T *branch * %F\n", gg.zBranch);
268 aTCard[nTCard++] = mprintf("T *sym-%F *\n", gg.zBranch);
269 if( zFromBranch ){
270 aTCard[nTCard++] = mprintf("T -sym-%F *\n", zFromBranch);
271 }
272 }
273 if( gg.zFrom==0 ){
274 aTCard[nTCard++] = mprintf("T *sym-trunk *\n");
275 }
276 qsort(aTCard, nTCard, sizeof(char *), string_cmp);
277 for(i=0; i<nTCard; i++){
278 if( i==0 || fossil_strcmp(aTCard[i-1], aTCard[i]) ){
279 blob_appendf(&record, "%s", aTCard[i]);
280 }
281 }
282 for(i=0; i<nTCard; i++) free(aTCard[i]);
283
284 free(zFromBranch);
 
 
 
285 db_multi_exec("INSERT INTO xbranch(tname, brnm) VALUES(%Q,%Q)",
286 gg.zMark, gg.zBranch);
287 blob_appendf(&record, "U %F\n", gg.zUser);
288 md5sum_blob(&record, &cksum);
289 blob_appendf(&record, "Z %b\n", &cksum);
290
M src/info.c
+14 -8
--- src/info.c
+++ src/info.c
@@ -190,11 +190,11 @@
190190
 " (SELECT uuid FROM blob WHERE rid=tagxref.srcid AND rid!=%d),"
191191
 " value, datetime(tagxref.mtime,'localtime'), tagtype,"
192192
 " (SELECT uuid FROM blob WHERE rid=tagxref.origid AND rid!=%d)"
193193
 " FROM tagxref JOIN tag ON tagxref.tagid=tag.tagid"
194194
 " WHERE tagxref.rid=%d AND tagname NOT GLOB '%s'"
195
- " ORDER BY tagname", rid, rid, rid, zNotGlob
195
+ " ORDER BY tagname /*sort*/", rid, rid, rid, zNotGlob
196196
 );
197197
 while( db_step(&q)==SQLITE_ROW ){
198198
 const char *zTagname = db_column_text(&q, 1);
199199
 const char *zSrcUuid = db_column_text(&q, 2);
200200
 const char *zValue = db_column_text(&q, 3);
@@ -525,11 +525,11 @@
525525
 " (SELECT uuid FROM blob WHERE rid=mlink.pid),"
526526
 " (SELECT uuid FROM blob WHERE rid=mlink.fid),"
527527
 " (SELECT name FROM filename WHERE filename.fnid=mlink.pfnid)"
528528
 " FROM mlink JOIN filename ON filename.fnid=mlink.fnid"
529529
 " WHERE mlink.mid=%d"
530
- " ORDER BY name",
530
+ " ORDER BY name /*sort*/",
531531
 rid
532532
 );
533533
 while( db_step(&q)==SQLITE_ROW ){
534534
 const char *zName = db_column_text(&q,0);
535535
 int mperm = db_column_int(&q, 1);
@@ -1213,12 +1213,18 @@
12131213
 }
12141214
12151215
12161216
 /*
12171217
 ** WEBPAGE: artifact
1218
-** URL: /artifact?name=ARTIFACTID
1218
+** URL: /artifact/ARTIFACTID
12191219
 ** URL: /artifact?ci=CHECKIN&filename=PATH
1220
+**
1221
+** Additional query parameters:
1222
+**
1223
+** ln - show line numbers
1224
+** ln=N - highlight line number N
1225
+** ln=M-N - highlight lines M through N inclusive
12201226
 **
12211227
 ** Show the complete content of a file identified by ARTIFACTID
12221228
 ** as preformatted text.
12231229
 */
12241230
 void artifact_page(void){
@@ -1260,24 +1266,24 @@
12601266
 zMime = mimetype_from_name(blob_str(&downloadName));
12611267
 if( zMime ){
12621268
 if( fossil_strcmp(zMime, "text/html")==0 ){
12631269
 if( P("txt") ){
12641270
 style_submenu_element("Html", "Html",
1265
- "%s/artifact?name=%s", g.zTop, zUuid);
1271
+ "%s/artifact/%s", g.zTop, zUuid);
12661272
 }else{
12671273
 renderAsHtml = 1;
12681274
 style_submenu_element("Text", "Text",
1269
- "%s/artifact?name=%s&amp;txt=1", g.zTop, zUuid);
1275
+ "%s/artifact/%s?txt=1", g.zTop, zUuid);
12701276
 }
12711277
 }else if( fossil_strcmp(zMime, "application/x-fossil-wiki")==0 ){
12721278
 if( P("txt") ){
12731279
 style_submenu_element("Wiki", "Wiki",
1274
- "%s/artifact?name=%s", g.zTop, zUuid);
1280
+ "%s/artifact/%s", g.zTop, zUuid);
12751281
 }else{
12761282
 renderAsWiki = 1;
12771283
 style_submenu_element("Text", "Text",
1278
- "%s/artifact?name=%s&amp;txt=1", g.zTop, zUuid);
1284
+ "%s/artifact/%s?txt=1", g.zTop, zUuid);
12791285
 }
12801286
 }
12811287
 }
12821288
 @ </p></blockquote>
12831289
 @ <hr />
@@ -1790,11 +1796,11 @@
17901796
 @ <input type="text" style="width:15;" name="tagname" value="%h(zNewTag)" />
17911797
 db_prepare(&q,
17921798
 "SELECT tag.tagid, tagname FROM tagxref, tag"
17931799
 " WHERE tagxref.rid=%d AND tagtype>0 AND tagxref.tagid=tag.tagid"
17941800
 " ORDER BY CASE WHEN tagname GLOB 'sym-*' THEN substr(tagname,5)"
1795
- " ELSE tagname END",
1801
+ " ELSE tagname END /*sort*/",
17961802
 rid
17971803
 );
17981804
 while( db_step(&q)==SQLITE_ROW ){
17991805
 int tagid = db_column_int(&q, 0);
18001806
 const char *zTagName = db_column_text(&q, 1);
18011807
--- src/info.c
+++ src/info.c
@@ -190,11 +190,11 @@
190 " (SELECT uuid FROM blob WHERE rid=tagxref.srcid AND rid!=%d),"
191 " value, datetime(tagxref.mtime,'localtime'), tagtype,"
192 " (SELECT uuid FROM blob WHERE rid=tagxref.origid AND rid!=%d)"
193 " FROM tagxref JOIN tag ON tagxref.tagid=tag.tagid"
194 " WHERE tagxref.rid=%d AND tagname NOT GLOB '%s'"
195 " ORDER BY tagname", rid, rid, rid, zNotGlob
196 );
197 while( db_step(&q)==SQLITE_ROW ){
198 const char *zTagname = db_column_text(&q, 1);
199 const char *zSrcUuid = db_column_text(&q, 2);
200 const char *zValue = db_column_text(&q, 3);
@@ -525,11 +525,11 @@
525 " (SELECT uuid FROM blob WHERE rid=mlink.pid),"
526 " (SELECT uuid FROM blob WHERE rid=mlink.fid),"
527 " (SELECT name FROM filename WHERE filename.fnid=mlink.pfnid)"
528 " FROM mlink JOIN filename ON filename.fnid=mlink.fnid"
529 " WHERE mlink.mid=%d"
530 " ORDER BY name",
531 rid
532 );
533 while( db_step(&q)==SQLITE_ROW ){
534 const char *zName = db_column_text(&q,0);
535 int mperm = db_column_int(&q, 1);
@@ -1213,12 +1213,18 @@
1213 }
1214
1215
1216 /*
1217 ** WEBPAGE: artifact
1218 ** URL: /artifact?name=ARTIFACTID
1219 ** URL: /artifact?ci=CHECKIN&filename=PATH
 
 
 
 
 
 
1220 **
1221 ** Show the complete content of a file identified by ARTIFACTID
1222 ** as preformatted text.
1223 */
1224 void artifact_page(void){
@@ -1260,24 +1266,24 @@
1260 zMime = mimetype_from_name(blob_str(&downloadName));
1261 if( zMime ){
1262 if( fossil_strcmp(zMime, "text/html")==0 ){
1263 if( P("txt") ){
1264 style_submenu_element("Html", "Html",
1265 "%s/artifact?name=%s", g.zTop, zUuid);
1266 }else{
1267 renderAsHtml = 1;
1268 style_submenu_element("Text", "Text",
1269 "%s/artifact?name=%s&amp;txt=1", g.zTop, zUuid);
1270 }
1271 }else if( fossil_strcmp(zMime, "application/x-fossil-wiki")==0 ){
1272 if( P("txt") ){
1273 style_submenu_element("Wiki", "Wiki",
1274 "%s/artifact?name=%s", g.zTop, zUuid);
1275 }else{
1276 renderAsWiki = 1;
1277 style_submenu_element("Text", "Text",
1278 "%s/artifact?name=%s&amp;txt=1", g.zTop, zUuid);
1279 }
1280 }
1281 }
1282 @ </p></blockquote>
1283 @ <hr />
@@ -1790,11 +1796,11 @@
1790 @ <input type="text" style="width:15;" name="tagname" value="%h(zNewTag)" />
1791 db_prepare(&q,
1792 "SELECT tag.tagid, tagname FROM tagxref, tag"
1793 " WHERE tagxref.rid=%d AND tagtype>0 AND tagxref.tagid=tag.tagid"
1794 " ORDER BY CASE WHEN tagname GLOB 'sym-*' THEN substr(tagname,5)"
1795 " ELSE tagname END",
1796 rid
1797 );
1798 while( db_step(&q)==SQLITE_ROW ){
1799 int tagid = db_column_int(&q, 0);
1800 const char *zTagName = db_column_text(&q, 1);
1801
--- src/info.c
+++ src/info.c
@@ -190,11 +190,11 @@
190 " (SELECT uuid FROM blob WHERE rid=tagxref.srcid AND rid!=%d),"
191 " value, datetime(tagxref.mtime,'localtime'), tagtype,"
192 " (SELECT uuid FROM blob WHERE rid=tagxref.origid AND rid!=%d)"
193 " FROM tagxref JOIN tag ON tagxref.tagid=tag.tagid"
194 " WHERE tagxref.rid=%d AND tagname NOT GLOB '%s'"
195 " ORDER BY tagname /*sort*/", rid, rid, rid, zNotGlob
196 );
197 while( db_step(&q)==SQLITE_ROW ){
198 const char *zTagname = db_column_text(&q, 1);
199 const char *zSrcUuid = db_column_text(&q, 2);
200 const char *zValue = db_column_text(&q, 3);
@@ -525,11 +525,11 @@
525 " (SELECT uuid FROM blob WHERE rid=mlink.pid),"
526 " (SELECT uuid FROM blob WHERE rid=mlink.fid),"
527 " (SELECT name FROM filename WHERE filename.fnid=mlink.pfnid)"
528 " FROM mlink JOIN filename ON filename.fnid=mlink.fnid"
529 " WHERE mlink.mid=%d"
530 " ORDER BY name /*sort*/",
531 rid
532 );
533 while( db_step(&q)==SQLITE_ROW ){
534 const char *zName = db_column_text(&q,0);
535 int mperm = db_column_int(&q, 1);
@@ -1213,12 +1213,18 @@
1213 }
1214
1215
1216 /*
1217 ** WEBPAGE: artifact
1218 ** URL: /artifact/ARTIFACTID
1219 ** URL: /artifact?ci=CHECKIN&filename=PATH
1220 **
1221 ** Additional query parameters:
1222 **
1223 ** ln - show line numbers
1224 ** ln=N - highlight line number N
1225 ** ln=M-N - highlight lines M through N inclusive
1226 **
1227 ** Show the complete content of a file identified by ARTIFACTID
1228 ** as preformatted text.
1229 */
1230 void artifact_page(void){
@@ -1260,24 +1266,24 @@
1266 zMime = mimetype_from_name(blob_str(&downloadName));
1267 if( zMime ){
1268 if( fossil_strcmp(zMime, "text/html")==0 ){
1269 if( P("txt") ){
1270 style_submenu_element("Html", "Html",
1271 "%s/artifact/%s", g.zTop, zUuid);
1272 }else{
1273 renderAsHtml = 1;
1274 style_submenu_element("Text", "Text",
1275 "%s/artifact/%s?txt=1", g.zTop, zUuid);
1276 }
1277 }else if( fossil_strcmp(zMime, "application/x-fossil-wiki")==0 ){
1278 if( P("txt") ){
1279 style_submenu_element("Wiki", "Wiki",
1280 "%s/artifact/%s", g.zTop, zUuid);
1281 }else{
1282 renderAsWiki = 1;
1283 style_submenu_element("Text", "Text",
1284 "%s/artifact/%s?txt=1", g.zTop, zUuid);
1285 }
1286 }
1287 }
1288 @ </p></blockquote>
1289 @ <hr />
@@ -1790,11 +1796,11 @@
1796 @ <input type="text" style="width:15;" name="tagname" value="%h(zNewTag)" />
1797 db_prepare(&q,
1798 "SELECT tag.tagid, tagname FROM tagxref, tag"
1799 " WHERE tagxref.rid=%d AND tagtype>0 AND tagxref.tagid=tag.tagid"
1800 " ORDER BY CASE WHEN tagname GLOB 'sym-*' THEN substr(tagname,5)"
1801 " ELSE tagname END /*sort*/",
1802 rid
1803 );
1804 while( db_step(&q)==SQLITE_ROW ){
1805 int tagid = db_column_int(&q, 0);
1806 const char *zTagName = db_column_text(&q, 1);
1807
M src/main.c
+10 -4
--- src/main.c
+++ src/main.c
@@ -2,11 +2,11 @@
22
 ** Copyright (c) 2006 D. Richard Hipp
33
 **
44
 ** This program is free software; you can redistribute it and/or
55
 ** modify it under the terms of the Simplified BSD License (also
66
 ** known as the "2-Clause License" or "FreeBSD License".)
7
-
7
+**
88
 ** This program is distributed in the hope that it will be useful,
99
 ** but without any warranty; without even the implied warranty of
1010
 ** merchantability or fitness for a particular purpose.
1111
 **
1212
 ** Author contact information:
@@ -105,10 +105,11 @@
105105
 char *urlProxyAuth; /* Proxy-Authorizer: string */
106106
 char *urlFossil; /* The path of the ?fossil=path suffix on ssh: */
107107
 int dontKeepUrl; /* Do not persist the URL */
108108
109109
 const char *zLogin; /* Login name. "" if not logged in. */
110
+ const char *zSSLIdentity; /* Value of --ssl-identity option, filename of SSL client identity */
110111
 int useLocalauth; /* No login required if from 127.0.0.1 */
111112
 int noPswd; /* Logged in without password (on 127.0.0.1) */
112113
 int userUid; /* Integer user id */
113114
114115
 /* Information used to populate the RCVFROM table */
@@ -251,10 +252,11 @@
251252
 g.fSystemTrace = find_option("systemtrace", 0, 0)!=0;
252253
 if( g.fSqlTrace ) g.fSqlStats = 1;
253254
 g.fSqlPrint = find_option("sqlprint", 0, 0)!=0;
254255
 g.fHttpTrace = find_option("httptrace", 0, 0)!=0;
255256
 g.zLogin = find_option("user", "U", 1);
257
+ g.zSSLIdentity = find_option("ssl-identity", 0, 1);
256258
 if( find_option("help",0,0)!=0 ){
257259
 /* --help anywhere on the command line is translated into
258260
 ** "fossil help argv[1] argv[2]..." */
259261
 int i;
260262
 char **zNewArgv = fossil_malloc( sizeof(char*)*(g.argc+2) );
@@ -644,11 +646,11 @@
644646
 ** Usage: %fossil version
645647
 **
646648
 ** Print the source code version number for the fossil executable.
647649
 */
648650
 void version_cmd(void){
649
- fossil_print("This is fossil version "
651
+ fossil_print("This is fossil version " RELEASE_VERSION " "
650652
 MANIFEST_VERSION " " MANIFEST_DATE " UTC\n");
651653
 }
652654
653655
654656
 /*
@@ -1102,11 +1104,12 @@
11021104
 if( blob_eq(&key, "HOME:") && blob_token(&line, &value) ){
11031105
 cgi_setenv("HOME", blob_str(&value));
11041106
 blob_reset(&value);
11051107
 continue;
11061108
 }
1107
- if( blob_eq(&key, "repository:") && blob_token(&line, &value) ){
1109
+ if( blob_eq(&key, "repository:") && blob_tail(&line, &value) ){
1110
+ blob_trim(&value);
11081111
 db_open_repository(blob_str(&value));
11091112
 blob_reset(&value);
11101113
 continue;
11111114
 }
11121115
 if( blob_eq(&key, "directory:") && blob_token(&line, &value) ){
@@ -1437,11 +1440,14 @@
14371440
 if( isUiCmd ){
14381441
 zBrowser = db_get("web-browser", "start");
14391442
 zBrowserCmd = mprintf("%s http://127.0.0.1:%%d/", zBrowser);
14401443
 }
14411444
 db_close(1);
1442
- win32_http_server(iPort, mxPort, zBrowserCmd, zStopperFile, zNotFound, flags);
1445
+ if( win32_http_service(iPort, zNotFound, flags) ){
1446
+ win32_http_server(iPort, mxPort, zBrowserCmd,
1447
+ zStopperFile, zNotFound, flags);
1448
+ }
14431449
 #endif
14441450
 }
14451451
14461452
 /*
14471453
 ** COMMAND: test-echo
14481454
--- src/main.c
+++ src/main.c
@@ -2,11 +2,11 @@
2 ** Copyright (c) 2006 D. Richard Hipp
3 **
4 ** This program is free software; you can redistribute it and/or
5 ** modify it under the terms of the Simplified BSD License (also
6 ** known as the "2-Clause License" or "FreeBSD License".)
7
8 ** This program is distributed in the hope that it will be useful,
9 ** but without any warranty; without even the implied warranty of
10 ** merchantability or fitness for a particular purpose.
11 **
12 ** Author contact information:
@@ -105,10 +105,11 @@
105 char *urlProxyAuth; /* Proxy-Authorizer: string */
106 char *urlFossil; /* The path of the ?fossil=path suffix on ssh: */
107 int dontKeepUrl; /* Do not persist the URL */
108
109 const char *zLogin; /* Login name. "" if not logged in. */
 
110 int useLocalauth; /* No login required if from 127.0.0.1 */
111 int noPswd; /* Logged in without password (on 127.0.0.1) */
112 int userUid; /* Integer user id */
113
114 /* Information used to populate the RCVFROM table */
@@ -251,10 +252,11 @@
251 g.fSystemTrace = find_option("systemtrace", 0, 0)!=0;
252 if( g.fSqlTrace ) g.fSqlStats = 1;
253 g.fSqlPrint = find_option("sqlprint", 0, 0)!=0;
254 g.fHttpTrace = find_option("httptrace", 0, 0)!=0;
255 g.zLogin = find_option("user", "U", 1);
 
256 if( find_option("help",0,0)!=0 ){
257 /* --help anywhere on the command line is translated into
258 ** "fossil help argv[1] argv[2]..." */
259 int i;
260 char **zNewArgv = fossil_malloc( sizeof(char*)*(g.argc+2) );
@@ -644,11 +646,11 @@
644 ** Usage: %fossil version
645 **
646 ** Print the source code version number for the fossil executable.
647 */
648 void version_cmd(void){
649 fossil_print("This is fossil version "
650 MANIFEST_VERSION " " MANIFEST_DATE " UTC\n");
651 }
652
653
654 /*
@@ -1102,11 +1104,12 @@
1102 if( blob_eq(&key, "HOME:") && blob_token(&line, &value) ){
1103 cgi_setenv("HOME", blob_str(&value));
1104 blob_reset(&value);
1105 continue;
1106 }
1107 if( blob_eq(&key, "repository:") && blob_token(&line, &value) ){
 
1108 db_open_repository(blob_str(&value));
1109 blob_reset(&value);
1110 continue;
1111 }
1112 if( blob_eq(&key, "directory:") && blob_token(&line, &value) ){
@@ -1437,11 +1440,14 @@
1437 if( isUiCmd ){
1438 zBrowser = db_get("web-browser", "start");
1439 zBrowserCmd = mprintf("%s http://127.0.0.1:%%d/", zBrowser);
1440 }
1441 db_close(1);
1442 win32_http_server(iPort, mxPort, zBrowserCmd, zStopperFile, zNotFound, flags);
 
 
 
1443 #endif
1444 }
1445
1446 /*
1447 ** COMMAND: test-echo
1448
--- src/main.c
+++ src/main.c
@@ -2,11 +2,11 @@
2 ** Copyright (c) 2006 D. Richard Hipp
3 **
4 ** This program is free software; you can redistribute it and/or
5 ** modify it under the terms of the Simplified BSD License (also
6 ** known as the "2-Clause License" or "FreeBSD License".)
7 **
8 ** This program is distributed in the hope that it will be useful,
9 ** but without any warranty; without even the implied warranty of
10 ** merchantability or fitness for a particular purpose.
11 **
12 ** Author contact information:
@@ -105,10 +105,11 @@
105 char *urlProxyAuth; /* Proxy-Authorizer: string */
106 char *urlFossil; /* The path of the ?fossil=path suffix on ssh: */
107 int dontKeepUrl; /* Do not persist the URL */
108
109 const char *zLogin; /* Login name. "" if not logged in. */
110 const char *zSSLIdentity; /* Value of --ssl-identity option, filename of SSL client identity */
111 int useLocalauth; /* No login required if from 127.0.0.1 */
112 int noPswd; /* Logged in without password (on 127.0.0.1) */
113 int userUid; /* Integer user id */
114
115 /* Information used to populate the RCVFROM table */
@@ -251,10 +252,11 @@
252 g.fSystemTrace = find_option("systemtrace", 0, 0)!=0;
253 if( g.fSqlTrace ) g.fSqlStats = 1;
254 g.fSqlPrint = find_option("sqlprint", 0, 0)!=0;
255 g.fHttpTrace = find_option("httptrace", 0, 0)!=0;
256 g.zLogin = find_option("user", "U", 1);
257 g.zSSLIdentity = find_option("ssl-identity", 0, 1);
258 if( find_option("help",0,0)!=0 ){
259 /* --help anywhere on the command line is translated into
260 ** "fossil help argv[1] argv[2]..." */
261 int i;
262 char **zNewArgv = fossil_malloc( sizeof(char*)*(g.argc+2) );
@@ -644,11 +646,11 @@
646 ** Usage: %fossil version
647 **
648 ** Print the source code version number for the fossil executable.
649 */
650 void version_cmd(void){
651 fossil_print("This is fossil version " RELEASE_VERSION " "
652 MANIFEST_VERSION " " MANIFEST_DATE " UTC\n");
653 }
654
655
656 /*
@@ -1102,11 +1104,12 @@
1104 if( blob_eq(&key, "HOME:") && blob_token(&line, &value) ){
1105 cgi_setenv("HOME", blob_str(&value));
1106 blob_reset(&value);
1107 continue;
1108 }
1109 if( blob_eq(&key, "repository:") && blob_tail(&line, &value) ){
1110 blob_trim(&value);
1111 db_open_repository(blob_str(&value));
1112 blob_reset(&value);
1113 continue;
1114 }
1115 if( blob_eq(&key, "directory:") && blob_token(&line, &value) ){
@@ -1437,11 +1440,14 @@
1440 if( isUiCmd ){
1441 zBrowser = db_get("web-browser", "start");
1442 zBrowserCmd = mprintf("%s http://127.0.0.1:%%d/", zBrowser);
1443 }
1444 db_close(1);
1445 if( win32_http_service(iPort, zNotFound, flags) ){
1446 win32_http_server(iPort, mxPort, zBrowserCmd,
1447 zStopperFile, zNotFound, flags);
1448 }
1449 #endif
1450 }
1451
1452 /*
1453 ** COMMAND: test-echo
1454
M src/main.mk
+16 -7
--- src/main.mk
+++ src/main.mk
@@ -281,22 +281,31 @@
281281
 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
282282
283283
 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
284284
 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
285285
286
+$(OBJDIR)/mkversion: $(SRCDIR)/mkversion.c
287
+ $(BCC) -o $(OBJDIR)/mkversion $(SRCDIR)/mkversion.c
288
+
286289
 # WARNING. DANGER. Running the testsuite modifies the repository the
287290
 # build is done from, i.e. the checkout belongs to. Do not sync/push
288291
 # the repository after running the tests.
289292
 test: $(APPNAME)
290293
 $(TCLSH) test/tester.tcl $(APPNAME)
291294
292
-$(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest
293
- awk '{ printf "#define MANIFEST_UUID \"%s\"\n", $$1}' $(SRCDIR)/../manifest.uuid >$(OBJDIR)/VERSION.h
294
- awk '{ printf "#define MANIFEST_VERSION \"[%.10s]\"\n", $$1}' $(SRCDIR)/../manifest.uuid >>$(OBJDIR)/VERSION.h
295
- awk '$$1=="D"{printf "#define MANIFEST_DATE \"%s %s\"\n", substr($$2,1,10),substr($$2,12,8)}' $(SRCDIR)/../manifest >>$(OBJDIR)/VERSION.h
295
+$(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION $(OBJDIR)/mkversion
296
+ $(OBJDIR)/mkversion $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
296297
297
-EXTRAOBJ = $(OBJDIR)/sqlite3.o $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
298
+# The USE_SYSTEM_SQLITE variable may be undefined, set to 0, or set
299
+# to 1. If it is set to 1, then there is no need to build or link
300
+# the sqlite3.o object. Instead, the system sqlite will be linked
301
+# using -lsqlite3.
302
+SQLITE3_OBJ.1 =
303
+SQLITE3_OBJ.0 = $(OBJDIR)/sqlite3.o
304
+SQLITE3_OBJ. = $(SQLITE3_OBJ.0)
305
+
306
+EXTRAOBJ = $(SQLITE3_OBJ.$(USE_SYSTEM_SQLITE)) $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
298307
299308
 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ)
300309
 $(TCC) -o $(APPNAME) $(OBJ) $(EXTRAOBJ) $(LIB)
301310
302311
 # This rule prevents make from using its default rules to try build
@@ -889,16 +898,16 @@
889898
 $(OBJDIR)/zip.o: $(OBJDIR)/zip_.c $(OBJDIR)/zip.h $(SRCDIR)/config.h
890899
 $(XTCC) -o $(OBJDIR)/zip.o -c $(OBJDIR)/zip_.c
891900
892901
 $(OBJDIR)/zip.h: $(OBJDIR)/headers
893902
 $(OBJDIR)/sqlite3.o: $(SRCDIR)/sqlite3.c
894
- $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT2 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
903
+ $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT3 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
895904
896
-$(OBJDIR)/shell.o: $(SRCDIR)/shell.c
905
+$(OBJDIR)/shell.o: $(SRCDIR)/shell.c $(SRCDIR)/sqlite3.h
897906
 $(XTCC) -Dmain=sqlite3_shell -DSQLITE_OMIT_LOAD_EXTENSION=1 -c $(SRCDIR)/shell.c -o $(OBJDIR)/shell.o
898907
899908
 $(OBJDIR)/th.o: $(SRCDIR)/th.c
900909
 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th.c -o $(OBJDIR)/th.o
901910
902911
 $(OBJDIR)/th_lang.o: $(SRCDIR)/th_lang.c
903912
 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th_lang.c -o $(OBJDIR)/th_lang.o
904913
905914
--- src/main.mk
+++ src/main.mk
@@ -281,22 +281,31 @@
281 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
282
283 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
284 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
285
 
 
 
286 # WARNING. DANGER. Running the testsuite modifies the repository the
287 # build is done from, i.e. the checkout belongs to. Do not sync/push
288 # the repository after running the tests.
289 test: $(APPNAME)
290 $(TCLSH) test/tester.tcl $(APPNAME)
291
292 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest
293 awk '{ printf "#define MANIFEST_UUID \"%s\"\n", $$1}' $(SRCDIR)/../manifest.uuid >$(OBJDIR)/VERSION.h
294 awk '{ printf "#define MANIFEST_VERSION \"[%.10s]\"\n", $$1}' $(SRCDIR)/../manifest.uuid >>$(OBJDIR)/VERSION.h
295 awk '$$1=="D"{printf "#define MANIFEST_DATE \"%s %s\"\n", substr($$2,1,10),substr($$2,12,8)}' $(SRCDIR)/../manifest >>$(OBJDIR)/VERSION.h
296
297 EXTRAOBJ = $(OBJDIR)/sqlite3.o $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
 
 
 
 
 
 
 
 
298
299 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ)
300 $(TCC) -o $(APPNAME) $(OBJ) $(EXTRAOBJ) $(LIB)
301
302 # This rule prevents make from using its default rules to try build
@@ -889,16 +898,16 @@
889 $(OBJDIR)/zip.o: $(OBJDIR)/zip_.c $(OBJDIR)/zip.h $(SRCDIR)/config.h
890 $(XTCC) -o $(OBJDIR)/zip.o -c $(OBJDIR)/zip_.c
891
892 $(OBJDIR)/zip.h: $(OBJDIR)/headers
893 $(OBJDIR)/sqlite3.o: $(SRCDIR)/sqlite3.c
894 $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT2 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
895
896 $(OBJDIR)/shell.o: $(SRCDIR)/shell.c
897 $(XTCC) -Dmain=sqlite3_shell -DSQLITE_OMIT_LOAD_EXTENSION=1 -c $(SRCDIR)/shell.c -o $(OBJDIR)/shell.o
898
899 $(OBJDIR)/th.o: $(SRCDIR)/th.c
900 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th.c -o $(OBJDIR)/th.o
901
902 $(OBJDIR)/th_lang.o: $(SRCDIR)/th_lang.c
903 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th_lang.c -o $(OBJDIR)/th_lang.o
904
905
--- src/main.mk
+++ src/main.mk
@@ -281,22 +281,31 @@
281 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
282
283 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
284 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
285
286 $(OBJDIR)/mkversion: $(SRCDIR)/mkversion.c
287 $(BCC) -o $(OBJDIR)/mkversion $(SRCDIR)/mkversion.c
288
289 # WARNING. DANGER. Running the testsuite modifies the repository the
290 # build is done from, i.e. the checkout belongs to. Do not sync/push
291 # the repository after running the tests.
292 test: $(APPNAME)
293 $(TCLSH) test/tester.tcl $(APPNAME)
294
295 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION $(OBJDIR)/mkversion
296 $(OBJDIR)/mkversion $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
 
 
297
298 # The USE_SYSTEM_SQLITE variable may be undefined, set to 0, or set
299 # to 1. If it is set to 1, then there is no need to build or link
300 # the sqlite3.o object. Instead, the system sqlite will be linked
301 # using -lsqlite3.
302 SQLITE3_OBJ.1 =
303 SQLITE3_OBJ.0 = $(OBJDIR)/sqlite3.o
304 SQLITE3_OBJ. = $(SQLITE3_OBJ.0)
305
306 EXTRAOBJ = $(SQLITE3_OBJ.$(USE_SYSTEM_SQLITE)) $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
307
308 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ)
309 $(TCC) -o $(APPNAME) $(OBJ) $(EXTRAOBJ) $(LIB)
310
311 # This rule prevents make from using its default rules to try build
@@ -889,16 +898,16 @@
898 $(OBJDIR)/zip.o: $(OBJDIR)/zip_.c $(OBJDIR)/zip.h $(SRCDIR)/config.h
899 $(XTCC) -o $(OBJDIR)/zip.o -c $(OBJDIR)/zip_.c
900
901 $(OBJDIR)/zip.h: $(OBJDIR)/headers
902 $(OBJDIR)/sqlite3.o: $(SRCDIR)/sqlite3.c
903 $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT3 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
904
905 $(OBJDIR)/shell.o: $(SRCDIR)/shell.c $(SRCDIR)/sqlite3.h
906 $(XTCC) -Dmain=sqlite3_shell -DSQLITE_OMIT_LOAD_EXTENSION=1 -c $(SRCDIR)/shell.c -o $(OBJDIR)/shell.o
907
908 $(OBJDIR)/th.o: $(SRCDIR)/th.c
909 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th.c -o $(OBJDIR)/th.o
910
911 $(OBJDIR)/th_lang.o: $(SRCDIR)/th_lang.c
912 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th_lang.c -o $(OBJDIR)/th_lang.o
913
914
M src/makeheaders.c
+1 -1
--- src/makeheaders.c
+++ src/makeheaders.c
@@ -3066,11 +3066,11 @@
30663066
 */
30673067
 if( zSrc[nSrc]==':' ){
30683068
 int nHdr;
30693069
 char *zHdr;
30703070
 zHdr = &zSrc[nSrc+1];
3071
- for(nHdr=0; zHdr[nHdr] && zHdr[nHdr]!=':'; nHdr++){}
3071
+ for(nHdr=0; zHdr[nHdr]; nHdr++){}
30723072
 pFile->zHdr = StrDup(zHdr,nHdr);
30733073
 }
30743074
30753075
 /* Look for any 'c' or 'C' in the suffix of the file name and change
30763076
 ** that character to 'h' or 'H' respectively. If no 'c' or 'C' is found,
30773077
--- src/makeheaders.c
+++ src/makeheaders.c
@@ -3066,11 +3066,11 @@
3066 */
3067 if( zSrc[nSrc]==':' ){
3068 int nHdr;
3069 char *zHdr;
3070 zHdr = &zSrc[nSrc+1];
3071 for(nHdr=0; zHdr[nHdr] && zHdr[nHdr]!=':'; nHdr++){}
3072 pFile->zHdr = StrDup(zHdr,nHdr);
3073 }
3074
3075 /* Look for any 'c' or 'C' in the suffix of the file name and change
3076 ** that character to 'h' or 'H' respectively. If no 'c' or 'C' is found,
3077
--- src/makeheaders.c
+++ src/makeheaders.c
@@ -3066,11 +3066,11 @@
3066 */
3067 if( zSrc[nSrc]==':' ){
3068 int nHdr;
3069 char *zHdr;
3070 zHdr = &zSrc[nSrc+1];
3071 for(nHdr=0; zHdr[nHdr]; nHdr++){}
3072 pFile->zHdr = StrDup(zHdr,nHdr);
3073 }
3074
3075 /* Look for any 'c' or 'C' in the suffix of the file name and change
3076 ** that character to 'h' or 'H' respectively. If no 'c' or 'C' is found,
3077
M src/makemake.tcl
+38 -31
--- src/makemake.tcl
+++ src/makemake.tcl
@@ -176,27 +176,34 @@
176176
 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
177177
178178
 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
179179
 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
180180
181
+$(OBJDIR)/mkversion: $(SRCDIR)/mkversion.c
182
+ $(BCC) -o $(OBJDIR)/mkversion $(SRCDIR)/mkversion.c
183
+
181184
 # WARNING. DANGER. Running the testsuite modifies the repository the
182185
 # build is done from, i.e. the checkout belongs to. Do not sync/push
183186
 # the repository after running the tests.
184187
 test: $(APPNAME)
185188
 $(TCLSH) test/tester.tcl $(APPNAME)
186189
187
-$(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest
188
- awk '{ printf "#define MANIFEST_UUID \"%s\"\n", $$1}' \
189
- $(SRCDIR)/../manifest.uuid >$(OBJDIR)/VERSION.h
190
- awk '{ printf "#define MANIFEST_VERSION \"[%.10s]\"\n", $$1}' \
191
- $(SRCDIR)/../manifest.uuid >>$(OBJDIR)/VERSION.h
192
- awk '$$1=="D"{printf "#define MANIFEST_DATE \"%s %s\"\n",\
193
- substr($$2,1,10),substr($$2,12,8)}' \
194
- $(SRCDIR)/../manifest >>$(OBJDIR)/VERSION.h
190
+$(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION $(OBJDIR)/mkversion
191
+ $(OBJDIR)/mkversion $(SRCDIR)/../manifest.uuid \
192
+ $(SRCDIR)/../manifest \
193
+ $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
194
+
195
+# The USE_SYSTEM_SQLITE variable may be undefined, set to 0, or set
196
+# to 1. If it is set to 1, then there is no need to build or link
197
+# the sqlite3.o object. Instead, the system sqlite will be linked
198
+# using -lsqlite3.
199
+SQLITE3_OBJ.1 =
200
+SQLITE3_OBJ.0 = $(OBJDIR)/sqlite3.o
201
+SQLITE3_OBJ. = $(SQLITE3_OBJ.0)
195202
196203
 EXTRAOBJ = \
197
- $(OBJDIR)/sqlite3.o \
204
+ $(SQLITE3_OBJ.$(USE_SYSTEM_SQLITE)) \
198205
 $(OBJDIR)/shell.o \
199206
 $(OBJDIR)/th.o \
200207
 $(OBJDIR)/th_lang.o
201208
202209
 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ)
@@ -241,17 +248,17 @@
241248
242249
 writeln "\$(OBJDIR)/sqlite3.o:\t\$(SRCDIR)/sqlite3.c"
243250
 set opt {-DSQLITE_OMIT_LOAD_EXTENSION=1}
244251
 append opt " -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4"
245252
 #append opt " -DSQLITE_ENABLE_FTS3=1"
246
-append opt " -DSQLITE_ENABLE_STAT2"
253
+append opt " -DSQLITE_ENABLE_STAT3"
247254
 append opt " -Dlocaltime=fossil_localtime"
248255
 append opt " -DSQLITE_ENABLE_LOCKING_STYLE=0"
249256
 set SQLITE_OPTIONS $opt
250257
 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/sqlite3.c -o \$(OBJDIR)/sqlite3.o\n"
251258
252
-writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c"
259
+writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c \$(SRCDIR)/sqlite3.h"
253260
 set opt {-Dmain=sqlite3_shell}
254261
 append opt " -DSQLITE_OMIT_LOAD_EXTENSION=1"
255262
 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/shell.c -o \$(OBJDIR)/shell.o\n"
256263
257264
 writeln "\$(OBJDIR)/th.o:\t\$(SRCDIR)/th.c"
@@ -389,21 +396,21 @@
389396
 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
390397
391398
 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
392399
 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
393400
394
-$(VERSION): $(SRCDIR)/../win/version.c
395
- $(BCC) -o $(OBJDIR)/version $(SRCDIR)/../win/version.c
401
+$(VERSION): $(SRCDIR)/mkversion.c
402
+ $(BCC) -o $(OBJDIR)/version $(SRCDIR)/mkversion.c
396403
397404
 # WARNING. DANGER. Running the testsuite modifies the repository the
398405
 # build is done from, i.e. the checkout belongs to. Do not sync/push
399406
 # the repository after running the tests.
400407
 test: $(APPNAME)
401408
 $(TCLSH) test/tester.tcl $(APPNAME)
402409
403410
 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(VERSION)
404
- $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest >$(OBJDIR)/VERSION.h
411
+ $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
405412
406413
 EXTRAOBJ = \
407414
 $(OBJDIR)/sqlite3.o \
408415
 $(OBJDIR)/shell.o \
409416
 $(OBJDIR)/th.o \
@@ -459,11 +466,11 @@
459466
460467
 writeln "\$(OBJDIR)/sqlite3.o:\t\$(SRCDIR)/sqlite3.c"
461468
 set opt $SQLITE_OPTIONS
462469
 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/sqlite3.c -o \$(OBJDIR)/sqlite3.o\n"
463470
464
-writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c"
471
+writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c \$(SRCDIR)/sqlite3.h"
465472
 set opt {-Dmain=sqlite3_shell}
466473
 append opt " -DSQLITE_OMIT_LOAD_EXTENSION=1"
467474
 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/shell.c -o \$(OBJDIR)/shell.o\n"
468475
469476
 writeln "\$(OBJDIR)/th.o:\t\$(SRCDIR)/th.c"
@@ -556,11 +563,11 @@
556563
 $(BCC) -o$@ $**
557564
558565
 mkindex$E: $(SRCDIR)\mkindex.c
559566
 $(BCC) -o$@ $**
560567
561
-version$E: $B\win\version.c
568
+version$E: $B\src\mkversion.c
562569
 $(BCC) -o$@ $**
563570
564571
 $(OBJDIR)\shell$O : $(SRCDIR)\shell.c
565572
 $(TCC) -o$@ -c -Dmain=sqlite3_shell $(SQLITE_OPTIONS) $**
566573
@@ -571,11 +578,11 @@
571578
 $(TCC) -o$@ -c $**
572579
573580
 $(OBJDIR)\th_lang$O : $(SRCDIR)\th_lang.c
574581
 $(TCC) -o$@ -c $**
575582
576
-VERSION.h : version$E $B\manifest.uuid $B\manifest
583
+VERSION.h : version$E $B\manifest.uuid $B\manifest $B\VERSION
577584
 +$** > $@
578585
579586
 page_index.h: mkindex$E $(SRC)
580587
 +$** > $@
581588
@@ -582,11 +589,11 @@
582589
 clean:
583590
 -del $(OBJDIR)\*.obj
584591
 -del *.obj *_.c *.h *.map
585592
586593
 realclean:
587
- -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
594
+ -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
588595
589596
 }
590597
 foreach s [lsort $src] {
591598
 writeln "\$(OBJDIR)\\$s\$O : ${s}_.c ${s}.h"
592599
 writeln "\t\$(TCC) -o\$@ -c ${s}_.c\n"
@@ -646,11 +653,11 @@
646653
 INCL = -I. -I$(SRCDIR) -I$B\win\include -I$(MSCDIR)\extra\include -I$(ZINCDIR)
647654
648655
 CFLAGS = -nologo -MT -O2
649656
 BCC = $(CC) $(CFLAGS)
650657
 TCC = $(CC) -c $(CFLAGS) $(MSCDEF) $(SSL) $(INCL)
651
-LIBS = $(ZLIB) ws2_32.lib $(SSLLIB)
658
+LIBS = $(ZLIB) ws2_32.lib advapi32.lib $(SSLLIB)
652659
 LIBDIR = -LIBPATH:$(MSCDIR)\extra\lib -LIBPATH:$(ZLIBDIR)
653660
 }
654661
 regsub -all {[-]D} $SQLITE_OPTIONS {/D} MSC_SQLITE_OPTIONS
655662
 writeln "SQLITE_OPTIONS = $MSC_SQLITE_OPTIONS\n"
656663
 writeln -nonewline "SRC = "
@@ -669,18 +676,18 @@
669676
670677
 all: $(OX) $(APPNAME)
671678
672679
 $(APPNAME) : translate$E mkindex$E headers $(OBJ) $(OX)\linkopts
673680
 cd $(OX)
674
- link -LINK -OUT:$@ $(LIBDIR) @linkopts
681
+ link /NODEFAULTLIB:msvcrt -OUT:$@ $(LIBDIR) @linkopts
675682
676683
 $(OX)\linkopts: $B\win\Makefile.msc}
677
-writeln -nonewline "\techo "
678
-foreach s [lsort $src] {
679
- writeln -nonewline "$s "
684
+set redir {>}
685
+foreach s [lsort [concat $src {shell sqlite3 th th_lang}]] {
686
+ writeln "\techo \$(OX)\\$s.obj $redir \$@"
687
+ set redir {>>}
680688
 }
681
-writeln "sqlite3 th th_lang > \$@"
682689
 writeln "\techo \$(LIBS) >> \$@\n\n"
683690
684691
 writeln {
685692
686693
 $(OX):
@@ -693,15 +700,15 @@
693700
 $(BCC) $**
694701
695702
 mkindex$E: $(SRCDIR)\mkindex.c
696703
 $(BCC) $**
697704
698
-version$E: $B\win\version.c
705
+mkversion$E: $B\src\mkversion.c
699706
 $(BCC) $**
700707
701708
 $(OX)\shell$O : $(SRCDIR)\shell.c
702
- $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c shell_.c
709
+ $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c $(SRCDIR)\shell.c
703710
704711
 $(OX)\sqlite3$O : $(SRCDIR)\sqlite3.c
705712
 $(TCC) /Fo$@ -c $(SQLITE_OPTIONS) $**
706713
707714
 $(OX)\th$O : $(SRCDIR)\th.c
@@ -708,11 +715,11 @@
708715
 $(TCC) /Fo$@ -c $**
709716
710717
 $(OX)\th_lang$O : $(SRCDIR)\th_lang.c
711718
 $(TCC) /Fo$@ -c $**
712719
713
-VERSION.h : version$E $B\manifest.uuid $B\manifest
720
+VERSION.h : mkversion$E $B\manifest.uuid $B\manifest $B\VERSION
714721
 $** > $@
715722
716723
 page_index.h: mkindex$E $(SRC)
717724
 $** > $@
718725
@@ -720,11 +727,11 @@
720727
 -del $(OX)\*.obj
721728
 -del *.obj *_.c *.h *.map
722729
 -del headers linkopts
723730
724731
 realclean:
725
- -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
732
+ -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
726733
727734
 }
728735
 foreach s [lsort $src] {
729736
 writeln "\$(OX)\\$s\$O : ${s}_.c ${s}.h"
730737
 writeln "\t\$(TCC) /Fo\$@ -c ${s}_.c\n"
@@ -880,11 +887,11 @@
880887
 # compiling standard fossil utils
881888
 $(UTILS_OBJ): %.obj: $(SRCDIR)%.c
882889
 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
883890
884891
 # compile special windows utils
885
-version.obj: $(WINDIR)version.c
892
+version.obj: $(SRCDIR)mkversion.c
886893
 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
887894
888895
 # generate the translated c-source files
889896
 $(TRANSLATEDSRC): %_.c: $(SRCDIR)%.c translate.exe
890897
 translate.exe $< >$@
@@ -892,12 +899,12 @@
892899
 # generate the index source, containing all web references,..
893900
 page_index.h: $(TRANSLATEDSRC) mkindex.exe
894901
 mkindex.exe $(TRANSLATEDSRC) >$@
895902
896903
 # extracting version info from manifest
897
-VERSION.h: version.exe ..\manifest.uuid ..\manifest
898
- version.exe ..\manifest.uuid ..\manifest > $@
904
+VERSION.h: version.exe ..\manifest.uuid ..\manifest ..\VERSION
905
+ version.exe ..\manifest.uuid ..\manifest ..\VERSION > $@
899906
900907
 # generate the simplified headers
901908
 headers: makeheaders.exe page_index.h VERSION.h ../src/sqlite3.h ../src/th.h VERSION.h
902909
 makeheaders.exe $(foreach ts,$(TRANSLATEDSRC),$(ts):$(ts:_.c=.h)) ../src/sqlite3.h ../src/th.h VERSION.h
903910
 echo Done >$@
904911
--- src/makemake.tcl
+++ src/makemake.tcl
@@ -176,27 +176,34 @@
176 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
177
178 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
179 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
180
 
 
 
181 # WARNING. DANGER. Running the testsuite modifies the repository the
182 # build is done from, i.e. the checkout belongs to. Do not sync/push
183 # the repository after running the tests.
184 test: $(APPNAME)
185 $(TCLSH) test/tester.tcl $(APPNAME)
186
187 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest
188 awk '{ printf "#define MANIFEST_UUID \"%s\"\n", $$1}' \
189 $(SRCDIR)/../manifest.uuid >$(OBJDIR)/VERSION.h
190 awk '{ printf "#define MANIFEST_VERSION \"[%.10s]\"\n", $$1}' \
191 $(SRCDIR)/../manifest.uuid >>$(OBJDIR)/VERSION.h
192 awk '$$1=="D"{printf "#define MANIFEST_DATE \"%s %s\"\n",\
193 substr($$2,1,10),substr($$2,12,8)}' \
194 $(SRCDIR)/../manifest >>$(OBJDIR)/VERSION.h
 
 
 
 
195
196 EXTRAOBJ = \
197 $(OBJDIR)/sqlite3.o \
198 $(OBJDIR)/shell.o \
199 $(OBJDIR)/th.o \
200 $(OBJDIR)/th_lang.o
201
202 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ)
@@ -241,17 +248,17 @@
241
242 writeln "\$(OBJDIR)/sqlite3.o:\t\$(SRCDIR)/sqlite3.c"
243 set opt {-DSQLITE_OMIT_LOAD_EXTENSION=1}
244 append opt " -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4"
245 #append opt " -DSQLITE_ENABLE_FTS3=1"
246 append opt " -DSQLITE_ENABLE_STAT2"
247 append opt " -Dlocaltime=fossil_localtime"
248 append opt " -DSQLITE_ENABLE_LOCKING_STYLE=0"
249 set SQLITE_OPTIONS $opt
250 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/sqlite3.c -o \$(OBJDIR)/sqlite3.o\n"
251
252 writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c"
253 set opt {-Dmain=sqlite3_shell}
254 append opt " -DSQLITE_OMIT_LOAD_EXTENSION=1"
255 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/shell.c -o \$(OBJDIR)/shell.o\n"
256
257 writeln "\$(OBJDIR)/th.o:\t\$(SRCDIR)/th.c"
@@ -389,21 +396,21 @@
389 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
390
391 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
392 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
393
394 $(VERSION): $(SRCDIR)/../win/version.c
395 $(BCC) -o $(OBJDIR)/version $(SRCDIR)/../win/version.c
396
397 # WARNING. DANGER. Running the testsuite modifies the repository the
398 # build is done from, i.e. the checkout belongs to. Do not sync/push
399 # the repository after running the tests.
400 test: $(APPNAME)
401 $(TCLSH) test/tester.tcl $(APPNAME)
402
403 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(VERSION)
404 $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest >$(OBJDIR)/VERSION.h
405
406 EXTRAOBJ = \
407 $(OBJDIR)/sqlite3.o \
408 $(OBJDIR)/shell.o \
409 $(OBJDIR)/th.o \
@@ -459,11 +466,11 @@
459
460 writeln "\$(OBJDIR)/sqlite3.o:\t\$(SRCDIR)/sqlite3.c"
461 set opt $SQLITE_OPTIONS
462 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/sqlite3.c -o \$(OBJDIR)/sqlite3.o\n"
463
464 writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c"
465 set opt {-Dmain=sqlite3_shell}
466 append opt " -DSQLITE_OMIT_LOAD_EXTENSION=1"
467 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/shell.c -o \$(OBJDIR)/shell.o\n"
468
469 writeln "\$(OBJDIR)/th.o:\t\$(SRCDIR)/th.c"
@@ -556,11 +563,11 @@
556 $(BCC) -o$@ $**
557
558 mkindex$E: $(SRCDIR)\mkindex.c
559 $(BCC) -o$@ $**
560
561 version$E: $B\win\version.c
562 $(BCC) -o$@ $**
563
564 $(OBJDIR)\shell$O : $(SRCDIR)\shell.c
565 $(TCC) -o$@ -c -Dmain=sqlite3_shell $(SQLITE_OPTIONS) $**
566
@@ -571,11 +578,11 @@
571 $(TCC) -o$@ -c $**
572
573 $(OBJDIR)\th_lang$O : $(SRCDIR)\th_lang.c
574 $(TCC) -o$@ -c $**
575
576 VERSION.h : version$E $B\manifest.uuid $B\manifest
577 +$** > $@
578
579 page_index.h: mkindex$E $(SRC)
580 +$** > $@
581
@@ -582,11 +589,11 @@
582 clean:
583 -del $(OBJDIR)\*.obj
584 -del *.obj *_.c *.h *.map
585
586 realclean:
587 -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
588
589 }
590 foreach s [lsort $src] {
591 writeln "\$(OBJDIR)\\$s\$O : ${s}_.c ${s}.h"
592 writeln "\t\$(TCC) -o\$@ -c ${s}_.c\n"
@@ -646,11 +653,11 @@
646 INCL = -I. -I$(SRCDIR) -I$B\win\include -I$(MSCDIR)\extra\include -I$(ZINCDIR)
647
648 CFLAGS = -nologo -MT -O2
649 BCC = $(CC) $(CFLAGS)
650 TCC = $(CC) -c $(CFLAGS) $(MSCDEF) $(SSL) $(INCL)
651 LIBS = $(ZLIB) ws2_32.lib $(SSLLIB)
652 LIBDIR = -LIBPATH:$(MSCDIR)\extra\lib -LIBPATH:$(ZLIBDIR)
653 }
654 regsub -all {[-]D} $SQLITE_OPTIONS {/D} MSC_SQLITE_OPTIONS
655 writeln "SQLITE_OPTIONS = $MSC_SQLITE_OPTIONS\n"
656 writeln -nonewline "SRC = "
@@ -669,18 +676,18 @@
669
670 all: $(OX) $(APPNAME)
671
672 $(APPNAME) : translate$E mkindex$E headers $(OBJ) $(OX)\linkopts
673 cd $(OX)
674 link -LINK -OUT:$@ $(LIBDIR) @linkopts
675
676 $(OX)\linkopts: $B\win\Makefile.msc}
677 writeln -nonewline "\techo "
678 foreach s [lsort $src] {
679 writeln -nonewline "$s "
 
680 }
681 writeln "sqlite3 th th_lang > \$@"
682 writeln "\techo \$(LIBS) >> \$@\n\n"
683
684 writeln {
685
686 $(OX):
@@ -693,15 +700,15 @@
693 $(BCC) $**
694
695 mkindex$E: $(SRCDIR)\mkindex.c
696 $(BCC) $**
697
698 version$E: $B\win\version.c
699 $(BCC) $**
700
701 $(OX)\shell$O : $(SRCDIR)\shell.c
702 $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c shell_.c
703
704 $(OX)\sqlite3$O : $(SRCDIR)\sqlite3.c
705 $(TCC) /Fo$@ -c $(SQLITE_OPTIONS) $**
706
707 $(OX)\th$O : $(SRCDIR)\th.c
@@ -708,11 +715,11 @@
708 $(TCC) /Fo$@ -c $**
709
710 $(OX)\th_lang$O : $(SRCDIR)\th_lang.c
711 $(TCC) /Fo$@ -c $**
712
713 VERSION.h : version$E $B\manifest.uuid $B\manifest
714 $** > $@
715
716 page_index.h: mkindex$E $(SRC)
717 $** > $@
718
@@ -720,11 +727,11 @@
720 -del $(OX)\*.obj
721 -del *.obj *_.c *.h *.map
722 -del headers linkopts
723
724 realclean:
725 -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
726
727 }
728 foreach s [lsort $src] {
729 writeln "\$(OX)\\$s\$O : ${s}_.c ${s}.h"
730 writeln "\t\$(TCC) /Fo\$@ -c ${s}_.c\n"
@@ -880,11 +887,11 @@
880 # compiling standard fossil utils
881 $(UTILS_OBJ): %.obj: $(SRCDIR)%.c
882 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
883
884 # compile special windows utils
885 version.obj: $(WINDIR)version.c
886 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
887
888 # generate the translated c-source files
889 $(TRANSLATEDSRC): %_.c: $(SRCDIR)%.c translate.exe
890 translate.exe $< >$@
@@ -892,12 +899,12 @@
892 # generate the index source, containing all web references,..
893 page_index.h: $(TRANSLATEDSRC) mkindex.exe
894 mkindex.exe $(TRANSLATEDSRC) >$@
895
896 # extracting version info from manifest
897 VERSION.h: version.exe ..\manifest.uuid ..\manifest
898 version.exe ..\manifest.uuid ..\manifest > $@
899
900 # generate the simplified headers
901 headers: makeheaders.exe page_index.h VERSION.h ../src/sqlite3.h ../src/th.h VERSION.h
902 makeheaders.exe $(foreach ts,$(TRANSLATEDSRC),$(ts):$(ts:_.c=.h)) ../src/sqlite3.h ../src/th.h VERSION.h
903 echo Done >$@
904
--- src/makemake.tcl
+++ src/makemake.tcl
@@ -176,27 +176,34 @@
176 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
177
178 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
179 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
180
181 $(OBJDIR)/mkversion: $(SRCDIR)/mkversion.c
182 $(BCC) -o $(OBJDIR)/mkversion $(SRCDIR)/mkversion.c
183
184 # WARNING. DANGER. Running the testsuite modifies the repository the
185 # build is done from, i.e. the checkout belongs to. Do not sync/push
186 # the repository after running the tests.
187 test: $(APPNAME)
188 $(TCLSH) test/tester.tcl $(APPNAME)
189
190 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION $(OBJDIR)/mkversion
191 $(OBJDIR)/mkversion $(SRCDIR)/../manifest.uuid \
192 $(SRCDIR)/../manifest \
193 $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
194
195 # The USE_SYSTEM_SQLITE variable may be undefined, set to 0, or set
196 # to 1. If it is set to 1, then there is no need to build or link
197 # the sqlite3.o object. Instead, the system sqlite will be linked
198 # using -lsqlite3.
199 SQLITE3_OBJ.1 =
200 SQLITE3_OBJ.0 = $(OBJDIR)/sqlite3.o
201 SQLITE3_OBJ. = $(SQLITE3_OBJ.0)
202
203 EXTRAOBJ = \
204 $(SQLITE3_OBJ.$(USE_SYSTEM_SQLITE)) \
205 $(OBJDIR)/shell.o \
206 $(OBJDIR)/th.o \
207 $(OBJDIR)/th_lang.o
208
209 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ)
@@ -241,17 +248,17 @@
248
249 writeln "\$(OBJDIR)/sqlite3.o:\t\$(SRCDIR)/sqlite3.c"
250 set opt {-DSQLITE_OMIT_LOAD_EXTENSION=1}
251 append opt " -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4"
252 #append opt " -DSQLITE_ENABLE_FTS3=1"
253 append opt " -DSQLITE_ENABLE_STAT3"
254 append opt " -Dlocaltime=fossil_localtime"
255 append opt " -DSQLITE_ENABLE_LOCKING_STYLE=0"
256 set SQLITE_OPTIONS $opt
257 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/sqlite3.c -o \$(OBJDIR)/sqlite3.o\n"
258
259 writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c \$(SRCDIR)/sqlite3.h"
260 set opt {-Dmain=sqlite3_shell}
261 append opt " -DSQLITE_OMIT_LOAD_EXTENSION=1"
262 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/shell.c -o \$(OBJDIR)/shell.o\n"
263
264 writeln "\$(OBJDIR)/th.o:\t\$(SRCDIR)/th.c"
@@ -389,21 +396,21 @@
396 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
397
398 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
399 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
400
401 $(VERSION): $(SRCDIR)/mkversion.c
402 $(BCC) -o $(OBJDIR)/version $(SRCDIR)/mkversion.c
403
404 # WARNING. DANGER. Running the testsuite modifies the repository the
405 # build is done from, i.e. the checkout belongs to. Do not sync/push
406 # the repository after running the tests.
407 test: $(APPNAME)
408 $(TCLSH) test/tester.tcl $(APPNAME)
409
410 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(VERSION)
411 $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
412
413 EXTRAOBJ = \
414 $(OBJDIR)/sqlite3.o \
415 $(OBJDIR)/shell.o \
416 $(OBJDIR)/th.o \
@@ -459,11 +466,11 @@
466
467 writeln "\$(OBJDIR)/sqlite3.o:\t\$(SRCDIR)/sqlite3.c"
468 set opt $SQLITE_OPTIONS
469 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/sqlite3.c -o \$(OBJDIR)/sqlite3.o\n"
470
471 writeln "\$(OBJDIR)/shell.o:\t\$(SRCDIR)/shell.c \$(SRCDIR)/sqlite3.h"
472 set opt {-Dmain=sqlite3_shell}
473 append opt " -DSQLITE_OMIT_LOAD_EXTENSION=1"
474 writeln "\t\$(XTCC) $opt -c \$(SRCDIR)/shell.c -o \$(OBJDIR)/shell.o\n"
475
476 writeln "\$(OBJDIR)/th.o:\t\$(SRCDIR)/th.c"
@@ -556,11 +563,11 @@
563 $(BCC) -o$@ $**
564
565 mkindex$E: $(SRCDIR)\mkindex.c
566 $(BCC) -o$@ $**
567
568 version$E: $B\src\mkversion.c
569 $(BCC) -o$@ $**
570
571 $(OBJDIR)\shell$O : $(SRCDIR)\shell.c
572 $(TCC) -o$@ -c -Dmain=sqlite3_shell $(SQLITE_OPTIONS) $**
573
@@ -571,11 +578,11 @@
578 $(TCC) -o$@ -c $**
579
580 $(OBJDIR)\th_lang$O : $(SRCDIR)\th_lang.c
581 $(TCC) -o$@ -c $**
582
583 VERSION.h : version$E $B\manifest.uuid $B\manifest $B\VERSION
584 +$** > $@
585
586 page_index.h: mkindex$E $(SRC)
587 +$** > $@
588
@@ -582,11 +589,11 @@
589 clean:
590 -del $(OBJDIR)\*.obj
591 -del *.obj *_.c *.h *.map
592
593 realclean:
594 -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
595
596 }
597 foreach s [lsort $src] {
598 writeln "\$(OBJDIR)\\$s\$O : ${s}_.c ${s}.h"
599 writeln "\t\$(TCC) -o\$@ -c ${s}_.c\n"
@@ -646,11 +653,11 @@
653 INCL = -I. -I$(SRCDIR) -I$B\win\include -I$(MSCDIR)\extra\include -I$(ZINCDIR)
654
655 CFLAGS = -nologo -MT -O2
656 BCC = $(CC) $(CFLAGS)
657 TCC = $(CC) -c $(CFLAGS) $(MSCDEF) $(SSL) $(INCL)
658 LIBS = $(ZLIB) ws2_32.lib advapi32.lib $(SSLLIB)
659 LIBDIR = -LIBPATH:$(MSCDIR)\extra\lib -LIBPATH:$(ZLIBDIR)
660 }
661 regsub -all {[-]D} $SQLITE_OPTIONS {/D} MSC_SQLITE_OPTIONS
662 writeln "SQLITE_OPTIONS = $MSC_SQLITE_OPTIONS\n"
663 writeln -nonewline "SRC = "
@@ -669,18 +676,18 @@
676
677 all: $(OX) $(APPNAME)
678
679 $(APPNAME) : translate$E mkindex$E headers $(OBJ) $(OX)\linkopts
680 cd $(OX)
681 link /NODEFAULTLIB:msvcrt -OUT:$@ $(LIBDIR) @linkopts
682
683 $(OX)\linkopts: $B\win\Makefile.msc}
684 set redir {>}
685 foreach s [lsort [concat $src {shell sqlite3 th th_lang}]] {
686 writeln "\techo \$(OX)\\$s.obj $redir \$@"
687 set redir {>>}
688 }
 
689 writeln "\techo \$(LIBS) >> \$@\n\n"
690
691 writeln {
692
693 $(OX):
@@ -693,15 +700,15 @@
700 $(BCC) $**
701
702 mkindex$E: $(SRCDIR)\mkindex.c
703 $(BCC) $**
704
705 mkversion$E: $B\src\mkversion.c
706 $(BCC) $**
707
708 $(OX)\shell$O : $(SRCDIR)\shell.c
709 $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c $(SRCDIR)\shell.c
710
711 $(OX)\sqlite3$O : $(SRCDIR)\sqlite3.c
712 $(TCC) /Fo$@ -c $(SQLITE_OPTIONS) $**
713
714 $(OX)\th$O : $(SRCDIR)\th.c
@@ -708,11 +715,11 @@
715 $(TCC) /Fo$@ -c $**
716
717 $(OX)\th_lang$O : $(SRCDIR)\th_lang.c
718 $(TCC) /Fo$@ -c $**
719
720 VERSION.h : mkversion$E $B\manifest.uuid $B\manifest $B\VERSION
721 $** > $@
722
723 page_index.h: mkindex$E $(SRC)
724 $** > $@
725
@@ -720,11 +727,11 @@
727 -del $(OX)\*.obj
728 -del *.obj *_.c *.h *.map
729 -del headers linkopts
730
731 realclean:
732 -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
733
734 }
735 foreach s [lsort $src] {
736 writeln "\$(OX)\\$s\$O : ${s}_.c ${s}.h"
737 writeln "\t\$(TCC) /Fo\$@ -c ${s}_.c\n"
@@ -880,11 +887,11 @@
887 # compiling standard fossil utils
888 $(UTILS_OBJ): %.obj: $(SRCDIR)%.c
889 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
890
891 # compile special windows utils
892 version.obj: $(SRCDIR)mkversion.c
893 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
894
895 # generate the translated c-source files
896 $(TRANSLATEDSRC): %_.c: $(SRCDIR)%.c translate.exe
897 translate.exe $< >$@
@@ -892,12 +899,12 @@
899 # generate the index source, containing all web references,..
900 page_index.h: $(TRANSLATEDSRC) mkindex.exe
901 mkindex.exe $(TRANSLATEDSRC) >$@
902
903 # extracting version info from manifest
904 VERSION.h: version.exe ..\manifest.uuid ..\manifest ..\VERSION
905 version.exe ..\manifest.uuid ..\manifest ..\VERSION > $@
906
907 # generate the simplified headers
908 headers: makeheaders.exe page_index.h VERSION.h ../src/sqlite3.h ../src/th.h VERSION.h
909 makeheaders.exe $(foreach ts,$(TRANSLATEDSRC),$(ts):$(ts:_.c=.h)) ../src/sqlite3.h ../src/th.h VERSION.h
910 echo Done >$@
911
M src/manifest.c
+4 -5
--- src/manifest.c
+++ src/manifest.c
@@ -342,11 +342,11 @@
342342
 int sz = 0;
343343
344344
 /* Every control artifact ends with a '\n' character. Exit early
345345
 ** if that is not the case for this artifact.
346346
 */
347
- z = blob_buffer(pContent);
347
+ z = blob_materialize(pContent);
348348
 n = blob_size(pContent);
349349
 if( n<=0 || z[n-1]!='\n' ){
350350
 blob_reset(pContent);
351351
 return 0;
352352
 }
@@ -1323,14 +1323,13 @@
13231323
 if( pChildFile ){
13241324
 add_one_mlink(cid, 0, pChildFile->zUuid, pChildFile->zName, 0,
13251325
 isPublic, manifest_file_mperm(pChildFile));
13261326
 }
13271327
 }
1328
- }else if( pChild->zBaseline==0 && pParent->zBaseline!=0 ){
1329
- /* Parent is a delta but pChild is a baseline. Look for files that are
1330
- ** present in pParent but which are missing from pChild and mark them
1331
- ** has having been deleted. */
1328
+ }else if( pChild->zBaseline==0 ){
1329
+ /* pChild is a baseline. Look for files that are present in pParent
1330
+ ** but are missing from pChild and mark them as having been deleted. */
13321331
 manifest_file_rewind(pParent);
13331332
 while( (pParentFile = manifest_file_next(pParent,0))!=0 ){
13341333
 pChildFile = manifest_file_seek(pChild, pParentFile->zName);
13351334
 if( pChildFile==0 && pParentFile->zUuid!=0 ){
13361335
 add_one_mlink(cid, pParentFile->zUuid, 0, pParentFile->zName, 0,
13371336
--- src/manifest.c
+++ src/manifest.c
@@ -342,11 +342,11 @@
342 int sz = 0;
343
344 /* Every control artifact ends with a '\n' character. Exit early
345 ** if that is not the case for this artifact.
346 */
347 z = blob_buffer(pContent);
348 n = blob_size(pContent);
349 if( n<=0 || z[n-1]!='\n' ){
350 blob_reset(pContent);
351 return 0;
352 }
@@ -1323,14 +1323,13 @@
1323 if( pChildFile ){
1324 add_one_mlink(cid, 0, pChildFile->zUuid, pChildFile->zName, 0,
1325 isPublic, manifest_file_mperm(pChildFile));
1326 }
1327 }
1328 }else if( pChild->zBaseline==0 && pParent->zBaseline!=0 ){
1329 /* Parent is a delta but pChild is a baseline. Look for files that are
1330 ** present in pParent but which are missing from pChild and mark them
1331 ** has having been deleted. */
1332 manifest_file_rewind(pParent);
1333 while( (pParentFile = manifest_file_next(pParent,0))!=0 ){
1334 pChildFile = manifest_file_seek(pChild, pParentFile->zName);
1335 if( pChildFile==0 && pParentFile->zUuid!=0 ){
1336 add_one_mlink(cid, pParentFile->zUuid, 0, pParentFile->zName, 0,
1337
--- src/manifest.c
+++ src/manifest.c
@@ -342,11 +342,11 @@
342 int sz = 0;
343
344 /* Every control artifact ends with a '\n' character. Exit early
345 ** if that is not the case for this artifact.
346 */
347 z = blob_materialize(pContent);
348 n = blob_size(pContent);
349 if( n<=0 || z[n-1]!='\n' ){
350 blob_reset(pContent);
351 return 0;
352 }
@@ -1323,14 +1323,13 @@
1323 if( pChildFile ){
1324 add_one_mlink(cid, 0, pChildFile->zUuid, pChildFile->zName, 0,
1325 isPublic, manifest_file_mperm(pChildFile));
1326 }
1327 }
1328 }else if( pChild->zBaseline==0 ){
1329 /* pChild is a baseline. Look for files that are present in pParent
1330 ** but are missing from pChild and mark them as having been deleted. */
 
1331 manifest_file_rewind(pParent);
1332 while( (pParentFile = manifest_file_next(pParent,0))!=0 ){
1333 pChildFile = manifest_file_seek(pChild, pParentFile->zName);
1334 if( pChildFile==0 && pParentFile->zUuid!=0 ){
1335 add_one_mlink(cid, pParentFile->zUuid, 0, pParentFile->zName, 0,
1336
M src/merge.c
+10 -2
--- src/merge.c
+++ src/merge.c
@@ -53,10 +53,14 @@
5353
 ** and do not try to merge parallel changes. This
5454
 ** option overrides the "binary-glob" setting.
5555
 **
5656
 ** --nochange | -n Dryrun: do not actually make any changes; just
5757
 ** show what would have happened.
58
+**
59
+** --case-sensitive BOOL Overwrite the case-sensitive setting. If false,
60
+** files whose names differ only in case are taken
61
+** to be the same file.
5862
 */
5963
 void merge_cmd(void){
6064
 int vid; /* Current version "V" */
6165
 int mid; /* Version we are merging from "M" */
6266
 int pid; /* The pivot version - most recent common ancestor P */
@@ -69,10 +73,11 @@
6973
 int debugFlag; /* True if --debug is present */
7074
 int nChng; /* Number of file name changes */
7175
 int *aChng; /* An array of file name changes */
7276
 int i; /* Loop counter */
7377
 int nConflict = 0; /* Number of conflicts seen */
78
+ int caseSensitive; /* True for case-sensitive filenames */
7479
 Stmt q;
7580
7681
7782
 /* Notation:
7883
 **
@@ -87,14 +92,16 @@
8792
 backoutFlag = find_option("backout",0,0)!=0;
8893
 debugFlag = find_option("debug",0,0)!=0;
8994
 zBinGlob = find_option("binary",0,1);
9095
 nochangeFlag = find_option("nochange","n",0)!=0;
9196
 zPivot = find_option("baseline",0,1);
97
+ capture_case_sensitive_option();
9298
 if( g.argc!=3 ){
9399
 usage("VERSION");
94100
 }
95101
 db_must_be_within_tree();
102
+ caseSensitive = filenames_are_case_sensitive();
96103
 if( zBinGlob==0 ) zBinGlob = db_get("binary-glob",0);
97104
 vid = db_lget_int("checkout", 0);
98105
 if( vid==0 ){
99106
 fossil_fatal("nothing is checked out");
100107
 }
@@ -150,11 +157,11 @@
150157
 ** in the current checkout, the pivot, and the version being merged.
151158
 */
152159
 db_multi_exec(
153160
 "DROP TABLE IF EXISTS fv;"
154161
 "CREATE TEMP TABLE fv("
155
- " fn TEXT PRIMARY KEY," /* The filename */
162
+ " fn TEXT PRIMARY KEY COLLATE %s," /* The filename */
156163
 " idv INTEGER," /* VFILE entry for current version */
157164
 " idp INTEGER," /* VFILE entry for the pivot */
158165
 " idm INTEGER," /* VFILE entry for version merging in */
159166
 " chnged BOOLEAN," /* True if current version has been edited */
160167
 " ridv INTEGER," /* Record ID for current version */
@@ -161,11 +168,12 @@
161168
 " ridp INTEGER," /* Record ID for pivot */
162169
 " ridm INTEGER," /* Record ID for merge */
163170
 " isexe BOOLEAN," /* Execute permission enabled */
164171
 " fnp TEXT," /* The filename in the pivot */
165172
 " fnm TEXT" /* the filename in the merged version */
166
- ");"
173
+ ");",
174
+ caseSensitive ? "binary" : "nocase"
167175
 );
168176
169177
 /* Add files found in V
170178
 */
171179
 db_multi_exec(
172180
173181
ADDED src/mkversion.c
--- src/merge.c
+++ src/merge.c
@@ -53,10 +53,14 @@
53 ** and do not try to merge parallel changes. This
54 ** option overrides the "binary-glob" setting.
55 **
56 ** --nochange | -n Dryrun: do not actually make any changes; just
57 ** show what would have happened.
 
 
 
 
58 */
59 void merge_cmd(void){
60 int vid; /* Current version "V" */
61 int mid; /* Version we are merging from "M" */
62 int pid; /* The pivot version - most recent common ancestor P */
@@ -69,10 +73,11 @@
69 int debugFlag; /* True if --debug is present */
70 int nChng; /* Number of file name changes */
71 int *aChng; /* An array of file name changes */
72 int i; /* Loop counter */
73 int nConflict = 0; /* Number of conflicts seen */
 
74 Stmt q;
75
76
77 /* Notation:
78 **
@@ -87,14 +92,16 @@
87 backoutFlag = find_option("backout",0,0)!=0;
88 debugFlag = find_option("debug",0,0)!=0;
89 zBinGlob = find_option("binary",0,1);
90 nochangeFlag = find_option("nochange","n",0)!=0;
91 zPivot = find_option("baseline",0,1);
 
92 if( g.argc!=3 ){
93 usage("VERSION");
94 }
95 db_must_be_within_tree();
 
96 if( zBinGlob==0 ) zBinGlob = db_get("binary-glob",0);
97 vid = db_lget_int("checkout", 0);
98 if( vid==0 ){
99 fossil_fatal("nothing is checked out");
100 }
@@ -150,11 +157,11 @@
150 ** in the current checkout, the pivot, and the version being merged.
151 */
152 db_multi_exec(
153 "DROP TABLE IF EXISTS fv;"
154 "CREATE TEMP TABLE fv("
155 " fn TEXT PRIMARY KEY," /* The filename */
156 " idv INTEGER," /* VFILE entry for current version */
157 " idp INTEGER," /* VFILE entry for the pivot */
158 " idm INTEGER," /* VFILE entry for version merging in */
159 " chnged BOOLEAN," /* True if current version has been edited */
160 " ridv INTEGER," /* Record ID for current version */
@@ -161,11 +168,12 @@
161 " ridp INTEGER," /* Record ID for pivot */
162 " ridm INTEGER," /* Record ID for merge */
163 " isexe BOOLEAN," /* Execute permission enabled */
164 " fnp TEXT," /* The filename in the pivot */
165 " fnm TEXT" /* the filename in the merged version */
166 ");"
 
167 );
168
169 /* Add files found in V
170 */
171 db_multi_exec(
172
173 DDED src/mkversion.c
--- src/merge.c
+++ src/merge.c
@@ -53,10 +53,14 @@
53 ** and do not try to merge parallel changes. This
54 ** option overrides the "binary-glob" setting.
55 **
56 ** --nochange | -n Dryrun: do not actually make any changes; just
57 ** show what would have happened.
58 **
59 ** --case-sensitive BOOL Overwrite the case-sensitive setting. If false,
60 ** files whose names differ only in case are taken
61 ** to be the same file.
62 */
63 void merge_cmd(void){
64 int vid; /* Current version "V" */
65 int mid; /* Version we are merging from "M" */
66 int pid; /* The pivot version - most recent common ancestor P */
@@ -69,10 +73,11 @@
73 int debugFlag; /* True if --debug is present */
74 int nChng; /* Number of file name changes */
75 int *aChng; /* An array of file name changes */
76 int i; /* Loop counter */
77 int nConflict = 0; /* Number of conflicts seen */
78 int caseSensitive; /* True for case-sensitive filenames */
79 Stmt q;
80
81
82 /* Notation:
83 **
@@ -87,14 +92,16 @@
92 backoutFlag = find_option("backout",0,0)!=0;
93 debugFlag = find_option("debug",0,0)!=0;
94 zBinGlob = find_option("binary",0,1);
95 nochangeFlag = find_option("nochange","n",0)!=0;
96 zPivot = find_option("baseline",0,1);
97 capture_case_sensitive_option();
98 if( g.argc!=3 ){
99 usage("VERSION");
100 }
101 db_must_be_within_tree();
102 caseSensitive = filenames_are_case_sensitive();
103 if( zBinGlob==0 ) zBinGlob = db_get("binary-glob",0);
104 vid = db_lget_int("checkout", 0);
105 if( vid==0 ){
106 fossil_fatal("nothing is checked out");
107 }
@@ -150,11 +157,11 @@
157 ** in the current checkout, the pivot, and the version being merged.
158 */
159 db_multi_exec(
160 "DROP TABLE IF EXISTS fv;"
161 "CREATE TEMP TABLE fv("
162 " fn TEXT PRIMARY KEY COLLATE %s," /* The filename */
163 " idv INTEGER," /* VFILE entry for current version */
164 " idp INTEGER," /* VFILE entry for the pivot */
165 " idm INTEGER," /* VFILE entry for version merging in */
166 " chnged BOOLEAN," /* True if current version has been edited */
167 " ridv INTEGER," /* Record ID for current version */
@@ -161,11 +168,12 @@
168 " ridp INTEGER," /* Record ID for pivot */
169 " ridm INTEGER," /* Record ID for merge */
170 " isexe BOOLEAN," /* Execute permission enabled */
171 " fnp TEXT," /* The filename in the pivot */
172 " fnm TEXT" /* the filename in the merged version */
173 ");",
174 caseSensitive ? "binary" : "nocase"
175 );
176
177 /* Add files found in V
178 */
179 db_multi_exec(
180
181 DDED src/mkversion.c
M src/mkversion.c
+1 -6
--- a/src/mkversion.c
+++ b/src/mkversion.c
@@ -1,10 +1,5 @@
1
-fopen(argv[1],"gv[1],exists to do the job that AWK would do for the unix
2
-** makefile - to extract information from the "mainfest" and "manifest.uuid"
3
-** files for this p)
1
+fopen(argv[1],"gv[1],"fMSC_VER)
42
 d = _MSC_VER / 1 return 0;
53
 }
64
 b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
75
 }
8
-;
9
- char b[1024b[strlen(b)-1] =0; return 0"gv[1],"fMSC_VER)
10
- fopen(argv[1
--- a/src/mkversion.c
+++ b/src/mkversion.c
@@ -1,10 +1,5 @@
1 fopen(argv[1],"gv[1],exists to do the job that AWK would do for the unix
2 ** makefile - to extract information from the "mainfest" and "manifest.uuid"
3 ** files for this p)
4 d = _MSC_VER / 1 return 0;
5 }
6 b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
7 }
8 ;
9 char b[1024b[strlen(b)-1] =0; return 0"gv[1],"fMSC_VER)
10 fopen(argv[1
--- a/src/mkversion.c
+++ b/src/mkversion.c
@@ -1,10 +1,5 @@
1 fopen(argv[1],"gv[1],"fMSC_VER)
 
 
2 d = _MSC_VER / 1 return 0;
3 }
4 b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
5 }
 
 
 
A src/mkversion.c
+5
--- a/src/mkversion.c
+++ b/src/mkversion.c
@@ -0,0 +1,5 @@
1
+fopen(argv[1],"gv[1],"fMSC_VER)
2
+ d = _MSC_VER / 1 return 0;
3
+}
4
+b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
5
+}
--- a/src/mkversion.c
+++ b/src/mkversion.c
@@ -0,0 +1,5 @@
 
 
 
 
 
--- a/src/mkversion.c
+++ b/src/mkversion.c
@@ -0,0 +1,5 @@
1 fopen(argv[1],"gv[1],"fMSC_VER)
2 d = _MSC_VER / 1 return 0;
3 }
4 b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
5 }
M src/path.c
+1 -1
--- src/path.c
+++ src/path.c
@@ -366,11 +366,11 @@
366366
367367
 *pnChng = 0;
368368
 *aiChng = 0;
369369
 if( iFrom==iTo ) return;
370370
 path_reset();
371
- p = path_shortest(iFrom, iTo, 0);
371
+ p = path_shortest(iFrom, iTo, 1);
372372
 if( p==0 ) return;
373373
 path_reverse_path();
374374
 db_prepare(&q1,
375375
 "SELECT pfnid, fnid FROM mlink WHERE mid=:mid AND pfnid>0"
376376
 );
377377
--- src/path.c
+++ src/path.c
@@ -366,11 +366,11 @@
366
367 *pnChng = 0;
368 *aiChng = 0;
369 if( iFrom==iTo ) return;
370 path_reset();
371 p = path_shortest(iFrom, iTo, 0);
372 if( p==0 ) return;
373 path_reverse_path();
374 db_prepare(&q1,
375 "SELECT pfnid, fnid FROM mlink WHERE mid=:mid AND pfnid>0"
376 );
377
--- src/path.c
+++ src/path.c
@@ -366,11 +366,11 @@
366
367 *pnChng = 0;
368 *aiChng = 0;
369 if( iFrom==iTo ) return;
370 path_reset();
371 p = path_shortest(iFrom, iTo, 1);
372 if( p==0 ) return;
373 path_reverse_path();
374 db_prepare(&q1,
375 "SELECT pfnid, fnid FROM mlink WHERE mid=:mid AND pfnid>0"
376 );
377
M src/printf.c
+1 -1
--- src/printf.c
+++ src/printf.c
@@ -859,11 +859,11 @@
859859
 int a, b;
860860
 do{
861861
 a = *zA++;
862862
 b = *zB++;
863863
 }while( a==b && a!=0 );
864
- return a - b;
864
+ return ((unsigned char)a) - (unsigned char)b;
865865
 }
866866
 }
867867
868868
 /*
869869
 ** Case insensitive string comparison.
870870
--- src/printf.c
+++ src/printf.c
@@ -859,11 +859,11 @@
859 int a, b;
860 do{
861 a = *zA++;
862 b = *zB++;
863 }while( a==b && a!=0 );
864 return a - b;
865 }
866 }
867
868 /*
869 ** Case insensitive string comparison.
870
--- src/printf.c
+++ src/printf.c
@@ -859,11 +859,11 @@
859 int a, b;
860 do{
861 a = *zA++;
862 b = *zB++;
863 }while( a==b && a!=0 );
864 return ((unsigned char)a) - (unsigned char)b;
865 }
866 }
867
868 /*
869 ** Case insensitive string comparison.
870
M src/rebuild.c
+1 -1
--- src/rebuild.c
+++ src/rebuild.c
@@ -882,11 +882,11 @@
882882
 }
883883
884884
 /*
885885
 ** COMMAND: deconstruct
886886
 **
887
-** Usage %fossil deconstruct ?OPTIONS? DESTIONATION
887
+** Usage %fossil deconstruct ?OPTIONS? DESTINATION
888888
 **
889889
 ** Options:
890890
 ** -R|--repository REPOSITORY
891891
 ** -L|--prefixlength N
892892
 **
893893
--- src/rebuild.c
+++ src/rebuild.c
@@ -882,11 +882,11 @@
882 }
883
884 /*
885 ** COMMAND: deconstruct
886 **
887 ** Usage %fossil deconstruct ?OPTIONS? DESTIONATION
888 **
889 ** Options:
890 ** -R|--repository REPOSITORY
891 ** -L|--prefixlength N
892 **
893
--- src/rebuild.c
+++ src/rebuild.c
@@ -882,11 +882,11 @@
882 }
883
884 /*
885 ** COMMAND: deconstruct
886 **
887 ** Usage %fossil deconstruct ?OPTIONS? DESTINATION
888 **
889 ** Options:
890 ** -R|--repository REPOSITORY
891 ** -L|--prefixlength N
892 **
893
M src/setup.c
+11 -2
--- src/setup.c
+++ src/setup.c
@@ -1062,25 +1062,34 @@
10621062
 for(pSet=ctrlSettings; pSet->name!=0; pSet++){
10631063
 if( pSet->width==0 ){
10641064
 onoff_attribute(pSet->name, pSet->name,
10651065
 pSet->var!=0 ? pSet->var : pSet->name,
10661066
 is_truth(pSet->def));
1067
- @ <br />
1067
+ if( pSet->versionable ){
1068
+ @ (v)<br />
1069
+ } else {
1070
+ @ <br />
1071
+ }
10681072
 }
10691073
 }
10701074
 @ </td><td style="width: 30;"></td><td valign="top">
10711075
 for(pSet=ctrlSettings; pSet->name!=0; pSet++){
10721076
 if( pSet->width!=0 ){
10731077
 entry_attribute(pSet->name, /*pSet->width*/ 40, pSet->name,
10741078
 pSet->var!=0 ? pSet->var : pSet->name,
10751079
 (char*)pSet->def);
1076
- @ <br />
1080
+ if( pSet->versionable ){
1081
+ @ (v)<br />
1082
+ } else {
1083
+ @ <br />
1084
+ }
10771085
 }
10781086
 }
10791087
 @ </td></tr></table>
10801088
 @ <p><input type="submit" name="submit" value="Apply Changes" /></p>
10811089
 @ </div></form>
1090
+ @ <p>Settings marked with (v) are 'versionable' and will be overridden by the contents of files named <tt>.fossil-settings/PROPERTY</tt>.</p>
10821091
 @ <hr /><p>
10831092
 @ These settings work in the same way, as the <kbd>set</kbd> commandline:<br />
10841093
 @ </p><pre>%s(zHelp_setting_cmd)</pre>
10851094
 db_end_transaction(0);
10861095
 style_footer();
10871096
--- src/setup.c
+++ src/setup.c
@@ -1062,25 +1062,34 @@
1062 for(pSet=ctrlSettings; pSet->name!=0; pSet++){
1063 if( pSet->width==0 ){
1064 onoff_attribute(pSet->name, pSet->name,
1065 pSet->var!=0 ? pSet->var : pSet->name,
1066 is_truth(pSet->def));
1067 @ <br />
 
 
 
 
1068 }
1069 }
1070 @ </td><td style="width: 30;"></td><td valign="top">
1071 for(pSet=ctrlSettings; pSet->name!=0; pSet++){
1072 if( pSet->width!=0 ){
1073 entry_attribute(pSet->name, /*pSet->width*/ 40, pSet->name,
1074 pSet->var!=0 ? pSet->var : pSet->name,
1075 (char*)pSet->def);
1076 @ <br />
 
 
 
 
1077 }
1078 }
1079 @ </td></tr></table>
1080 @ <p><input type="submit" name="submit" value="Apply Changes" /></p>
1081 @ </div></form>
 
1082 @ <hr /><p>
1083 @ These settings work in the same way, as the <kbd>set</kbd> commandline:<br />
1084 @ </p><pre>%s(zHelp_setting_cmd)</pre>
1085 db_end_transaction(0);
1086 style_footer();
1087
--- src/setup.c
+++ src/setup.c
@@ -1062,25 +1062,34 @@
1062 for(pSet=ctrlSettings; pSet->name!=0; pSet++){
1063 if( pSet->width==0 ){
1064 onoff_attribute(pSet->name, pSet->name,
1065 pSet->var!=0 ? pSet->var : pSet->name,
1066 is_truth(pSet->def));
1067 if( pSet->versionable ){
1068 @ (v)<br />
1069 } else {
1070 @ <br />
1071 }
1072 }
1073 }
1074 @ </td><td style="width: 30;"></td><td valign="top">
1075 for(pSet=ctrlSettings; pSet->name!=0; pSet++){
1076 if( pSet->width!=0 ){
1077 entry_attribute(pSet->name, /*pSet->width*/ 40, pSet->name,
1078 pSet->var!=0 ? pSet->var : pSet->name,
1079 (char*)pSet->def);
1080 if( pSet->versionable ){
1081 @ (v)<br />
1082 } else {
1083 @ <br />
1084 }
1085 }
1086 }
1087 @ </td></tr></table>
1088 @ <p><input type="submit" name="submit" value="Apply Changes" /></p>
1089 @ </div></form>
1090 @ <p>Settings marked with (v) are 'versionable' and will be overridden by the contents of files named <tt>.fossil-settings/PROPERTY</tt>.</p>
1091 @ <hr /><p>
1092 @ These settings work in the same way, as the <kbd>set</kbd> commandline:<br />
1093 @ </p><pre>%s(zHelp_setting_cmd)</pre>
1094 db_end_transaction(0);
1095 style_footer();
1096
M src/sqlite3.c
+5173 -2758
--- src/sqlite3.c
+++ src/sqlite3.c
@@ -1,8 +1,8 @@
11
 /******************************************************************************
22
 ** This file is an amalgamation of many separate C source files from SQLite
3
-** version 3.7.7. By combining all the individual C code files into this
3
+** version 3.7.8. By combining all the individual C code files into this
44
 ** single large file, the entire code can be compiled as a single translation
55
 ** unit. This allows many compilers to do optimizations that would not be
66
 ** possible if the files were compiled separately. Performance improvements
77
 ** of 5% or more are commonly seen when SQLite is compiled as a single
88
 ** translation unit.
@@ -648,13 +648,13 @@
648648
 **
649649
 ** See also: [sqlite3_libversion()],
650650
 ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
651651
 ** [sqlite_version()] and [sqlite_source_id()].
652652
 */
653
-#define SQLITE_VERSION "3.7.7"
654
-#define SQLITE_VERSION_NUMBER 3007007
655
-#define SQLITE_SOURCE_ID "2011-06-03 14:19:10 0206bc6f87bb9393218a380fc5b18039d334a8d8"
653
+#define SQLITE_VERSION "3.7.8"
654
+#define SQLITE_VERSION_NUMBER 3007008
655
+#define SQLITE_SOURCE_ID "2011-08-16 02:07:04 9650d7962804d61f56cac944ff9bb2c7bc111957"
656656
657657
 /*
658658
 ** CAPI3REF: Run-Time Library Version Numbers
659659
 ** KEYWORDS: sqlite3_version, sqlite3_sourceid
660660
 **
@@ -851,11 +851,11 @@
851851
 ** semicolon-separate SQL statements passed into its 2nd argument,
852852
 ** in the context of the [database connection] passed in as its 1st
853853
 ** argument. ^If the callback function of the 3rd argument to
854854
 ** sqlite3_exec() is not NULL, then it is invoked for each result row
855855
 ** coming out of the evaluated SQL statements. ^The 4th argument to
856
-** to sqlite3_exec() is relayed through to the 1st argument of each
856
+** sqlite3_exec() is relayed through to the 1st argument of each
857857
 ** callback invocation. ^If the callback pointer to sqlite3_exec()
858858
 ** is NULL, then no callback is ever invoked and result rows are
859859
 ** ignored.
860860
 **
861861
 ** ^If an error occurs while evaluating the SQL statements passed into
@@ -1282,20 +1282,52 @@
12821282
 ** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most
12831283
 ** VFSes do not need this signal and should silently ignore this opcode.
12841284
 ** Applications should not call [sqlite3_file_control()] with this
12851285
 ** opcode as doing so may disrupt the operation of the specialized VFSes
12861286
 ** that do require it.
1287
+**
1288
+** ^The [SQLITE_FCNTL_WIN32_AV_RETRY] opcode is used to configure automatic
1289
+** retry counts and intervals for certain disk I/O operations for the
1290
+** windows [VFS] in order to work to provide robustness against
1291
+** anti-virus programs. By default, the windows VFS will retry file read,
1292
+** file write, and file delete opertions up to 10 times, with a delay
1293
+** of 25 milliseconds before the first retry and with the delay increasing
1294
+** by an additional 25 milliseconds with each subsequent retry. This
1295
+** opcode allows those to values (10 retries and 25 milliseconds of delay)
1296
+** to be adjusted. The values are changed for all database connections
1297
+** within the same process. The argument is a pointer to an array of two
1298
+** integers where the first integer i the new retry count and the second
1299
+** integer is the delay. If either integer is negative, then the setting
1300
+** is not changed but instead the prior value of that setting is written
1301
+** into the array entry, allowing the current retry settings to be
1302
+** interrogated. The zDbName parameter is ignored.
1303
+**
1304
+** ^The [SQLITE_FCNTL_PERSIST_WAL] opcode is used to set or query the
1305
+** persistent [WAL | Write AHead Log] setting. By default, the auxiliary
1306
+** write ahead log and shared memory files used for transaction control
1307
+** are automatically deleted when the latest connection to the database
1308
+** closes. Setting persistent WAL mode causes those files to persist after
1309
+** close. Persisting the files is useful when other processes that do not
1310
+** have write permission on the directory containing the database file want
1311
+** to read the database file, as the WAL and shared memory files must exist
1312
+** in order for the database to be readable. The fourth parameter to
1313
+** [sqlite3_file_control()] for this opcode should be a pointer to an integer.
1314
+** That integer is 0 to disable persistent WAL mode or 1 to enable persistent
1315
+** WAL mode. If the integer is -1, then it is overwritten with the current
1316
+** WAL persistence setting.
1317
+**
12871318
 */
12881319
 #define SQLITE_FCNTL_LOCKSTATE 1
12891320
 #define SQLITE_GET_LOCKPROXYFILE 2
12901321
 #define SQLITE_SET_LOCKPROXYFILE 3
12911322
 #define SQLITE_LAST_ERRNO 4
12921323
 #define SQLITE_FCNTL_SIZE_HINT 5
12931324
 #define SQLITE_FCNTL_CHUNK_SIZE 6
12941325
 #define SQLITE_FCNTL_FILE_POINTER 7
12951326
 #define SQLITE_FCNTL_SYNC_OMITTED 8
1296
-
1327
+#define SQLITE_FCNTL_WIN32_AV_RETRY 9
1328
+#define SQLITE_FCNTL_PERSIST_WAL 10
12971329
12981330
 /*
12991331
 ** CAPI3REF: Mutex Handle
13001332
 **
13011333
 ** The mutex module within SQLite defines [sqlite3_mutex] to be an
@@ -1443,11 +1475,11 @@
14431475
 ** The xSleep() method causes the calling thread to sleep for at
14441476
 ** least the number of microseconds given. ^The xCurrentTime()
14451477
 ** method returns a Julian Day Number for the current date and time as
14461478
 ** a floating point value.
14471479
 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian
1448
-** Day Number multipled by 86400000 (the number of milliseconds in
1480
+** Day Number multiplied by 86400000 (the number of milliseconds in
14491481
 ** a 24-hour day).
14501482
 ** ^SQLite will use the xCurrentTimeInt64() method to get the current
14511483
 ** date and time if that method is available (if iVersion is 2 or
14521484
 ** greater and the function pointer is not NULL) and will fall back
14531485
 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable.
@@ -1881,11 +1913,11 @@
18811913
 ** scratch memory beyond what is provided by this configuration option, then
18821914
 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd>
18831915
 **
18841916
 ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt>
18851917
 ** <dd> ^This option specifies a static memory buffer that SQLite can use for
1886
-** the database page cache with the default page cache implemenation.
1918
+** the database page cache with the default page cache implementation.
18871919
 ** This configuration should not be used if an application-define page
18881920
 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option.
18891921
 ** There are three arguments to this option: A pointer to 8-byte aligned
18901922
 ** memory, the size of each page buffer (sz), and the number of pages (N).
18911923
 ** The sz argument should be the size of the largest database page
@@ -2979,16 +3011,16 @@
29793011
 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
29803012
 **
29813013
 ** ^If [URI filename] interpretation is enabled, and the filename argument
29823014
 ** begins with "file:", then the filename is interpreted as a URI. ^URI
29833015
 ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is
2984
-** is set in the fourth argument to sqlite3_open_v2(), or if it has
3016
+** set in the fourth argument to sqlite3_open_v2(), or if it has
29853017
 ** been enabled globally using the [SQLITE_CONFIG_URI] option with the
29863018
 ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.
29873019
 ** As of SQLite version 3.7.7, URI filename interpretation is turned off
29883020
 ** by default, but future releases of SQLite might enable URI filename
2989
-** intepretation by default. See "[URI filenames]" for additional
3021
+** interpretation by default. See "[URI filenames]" for additional
29903022
 ** information.
29913023
 **
29923024
 ** URI filenames are parsed according to RFC 3986. ^If the URI contains an
29933025
 ** authority, then it must be either an empty string or the string
29943026
 ** "localhost". ^If the authority is not an empty string or "localhost", an
@@ -3365,11 +3397,11 @@
33653397
 ** a schema change, on the first [sqlite3_step()] call following any change
33663398
 ** to the [sqlite3_bind_text | bindings] of that [parameter].
33673399
 ** ^The specific value of WHERE-clause [parameter] might influence the
33683400
 ** choice of query plan if the parameter is the left-hand side of a [LIKE]
33693401
 ** or [GLOB] operator or if the parameter is compared to an indexed column
3370
-** and the [SQLITE_ENABLE_STAT2] compile-time option is enabled.
3402
+** and the [SQLITE_ENABLE_STAT3] compile-time option is enabled.
33713403
 ** the
33723404
 ** </li>
33733405
 ** </ol>
33743406
 */
33753407
 SQLITE_API int sqlite3_prepare(
@@ -3803,11 +3835,11 @@
38033835
 ** [extended result codes] might be returned as well.
38043836
 **
38053837
 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the
38063838
 ** database locks it needs to do its job. ^If the statement is a [COMMIT]
38073839
 ** or occurs outside of an explicit transaction, then you can retry the
3808
-** statement. If the statement is not a [COMMIT] and occurs within a
3840
+** statement. If the statement is not a [COMMIT] and occurs within an
38093841
 ** explicit transaction then you should rollback the transaction before
38103842
 ** continuing.
38113843
 **
38123844
 ** ^[SQLITE_DONE] means that the statement has finished executing
38133845
 ** successfully. sqlite3_step() should not be called again on this virtual
@@ -4082,11 +4114,11 @@
40824114
40834115
 /*
40844116
 ** CAPI3REF: Destroy A Prepared Statement Object
40854117
 **
40864118
 ** ^The sqlite3_finalize() function is called to delete a [prepared statement].
4087
-** ^If the most recent evaluation of the statement encountered no errors or
4119
+** ^If the most recent evaluation of the statement encountered no errors
40884120
 ** or if the statement is never been evaluated, then sqlite3_finalize() returns
40894121
 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then
40904122
 ** sqlite3_finalize(S) returns the appropriate [error code] or
40914123
 ** [extended error code].
40924124
 **
@@ -5996,11 +6028,11 @@
59966028
 ** versions of these routines, it should at least provide stubs that always
59976029
 ** return true so that one does not get spurious assertion failures.
59986030
 **
59996031
 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then
60006032
 ** the routine should return 1. This seems counter-intuitive since
6001
-** clearly the mutex cannot be held if it does not exist. But the
6033
+** clearly the mutex cannot be held if it does not exist. But
60026034
 ** the reason the mutex does not exist is because the build is not
60036035
 ** using mutexes. And we do not want the assert() containing the
60046036
 ** call to sqlite3_mutex_held() to fail, so a non-zero return is
60056037
 ** the appropriate thing to do. ^The sqlite3_mutex_notheld()
60066038
 ** interface should also return 1 when given a NULL pointer.
@@ -6119,11 +6151,12 @@
61196151
 #define SQLITE_TESTCTRL_RESERVE 14
61206152
 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15
61216153
 #define SQLITE_TESTCTRL_ISKEYWORD 16
61226154
 #define SQLITE_TESTCTRL_PGHDRSZ 17
61236155
 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18
6124
-#define SQLITE_TESTCTRL_LAST 18
6156
+#define SQLITE_TESTCTRL_LOCALTIME_FAULT 19
6157
+#define SQLITE_TESTCTRL_LAST 19
61256158
61266159
 /*
61276160
 ** CAPI3REF: SQLite Runtime Status
61286161
 **
61296162
 ** ^This interface is used to retrieve runtime status information
@@ -6505,11 +6538,11 @@
65056538
 ** [[the xFetch() page cache methods]]
65066539
 ** The xFetch() method locates a page in the cache and returns a pointer to
65076540
 ** the page, or a NULL pointer.
65086541
 ** A "page", in this context, means a buffer of szPage bytes aligned at an
65096542
 ** 8-byte boundary. The page to be fetched is determined by the key. ^The
6510
-** mimimum key value is 1. After it has been retrieved using xFetch, the page
6543
+** minimum key value is 1. After it has been retrieved using xFetch, the page
65116544
 ** is considered to be "pinned".
65126545
 **
65136546
 ** If the requested page is already in the page cache, then the page cache
65146547
 ** implementation must return a pointer to the page buffer with its content
65156548
 ** intact. If the requested page is not already in the cache, then the
@@ -7678,10 +7711,22 @@
76787711
 ** is 0x00000000ffffffff. But because of quirks of some compilers, we
76797712
 ** have to specify the value in the less intuitive manner shown:
76807713
 */
76817714
 #define SQLITE_MAX_U32 ((((u64)1)<<32)-1)
76827715
7716
+/*
7717
+** The datatype used to store estimates of the number of rows in a
7718
+** table or index. This is an unsigned integer type. For 99.9% of
7719
+** the world, a 32-bit integer is sufficient. But a 64-bit integer
7720
+** can be used at compile-time if desired.
7721
+*/
7722
+#ifdef SQLITE_64BIT_STATS
7723
+ typedef u64 tRowcnt; /* 64-bit only if requested at compile-time */
7724
+#else
7725
+ typedef u32 tRowcnt; /* 32-bit is the default */
7726
+#endif
7727
+
76837728
 /*
76847729
 ** Macros to determine whether the machine is big or little endian,
76857730
 ** evaluated at runtime.
76867731
 */
76877732
 #ifdef SQLITE_AMALGAMATION
@@ -8483,10 +8528,11 @@
84838528
 SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int);
84848529
 SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int);
84858530
 SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int);
84868531
 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int);
84878532
 SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp);
8533
+SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe*,int,char*);
84888534
 SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1);
84898535
 SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2);
84908536
 SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3);
84918537
 SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u8 P5);
84928538
 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr);
@@ -9282,11 +9328,11 @@
92829328
 ** sqlite3_close().
92839329
 *
92849330
 ** A thread must be holding a mutex on the corresponding Btree in order
92859331
 ** to access Schema content. This implies that the thread must also be
92869332
 ** holding a mutex on the sqlite3 connection pointer that owns the Btree.
9287
-** For a TEMP Schema, on the connection mutex is required.
9333
+** For a TEMP Schema, only the connection mutex is required.
92889334
 */
92899335
 struct Schema {
92909336
 int schema_cookie; /* Database schema version number for this file */
92919337
 int iGeneration; /* Generation counter. Incremented with each change */
92929338
 Hash tblHash; /* All tables indexed by name */
@@ -9555,10 +9601,12 @@
95559601
 #define SQLITE_IndexSort 0x04 /* Disable indexes for sorting */
95569602
 #define SQLITE_IndexSearch 0x08 /* Disable indexes for searching */
95579603
 #define SQLITE_IndexCover 0x10 /* Disable index covering table */
95589604
 #define SQLITE_GroupByOrder 0x20 /* Disable GROUPBY cover of ORDERBY */
95599605
 #define SQLITE_FactorOutConst 0x40 /* Disable factoring out constants */
9606
+#define SQLITE_IdxRealAsInt 0x80 /* Store REAL as INT in indices */
9607
+#define SQLITE_DistinctOpt 0x80 /* DISTINCT using indexes */
95609608
 #define SQLITE_OptMask 0xff /* Mask of all disablable opts */
95619609
95629610
 /*
95639611
 ** Possible values for the sqlite.magic field.
95649612
 ** The numbers are obtained at random and have no special meaning, other
@@ -9876,11 +9924,11 @@
98769924
 int iPKey; /* If not negative, use aCol[iPKey] as the primary key */
98779925
 int nCol; /* Number of columns in this table */
98789926
 Column *aCol; /* Information about each column */
98799927
 Index *pIndex; /* List of SQL indexes on this table. */
98809928
 int tnum; /* Root BTree node for this table (see note above) */
9881
- unsigned nRowEst; /* Estimated rows in table - from sqlite_stat1 table */
9929
+ tRowcnt nRowEst; /* Estimated rows in table - from sqlite_stat1 table */
98829930
 Select *pSelect; /* NULL for tables. Points to definition if a view. */
98839931
 u16 nRef; /* Number of pointers to this Table */
98849932
 u8 tabFlags; /* Mask of TF_* values */
98859933
 u8 keyConf; /* What to do in case of uniqueness conflict on iPKey */
98869934
 FKey *pFKey; /* Linked list of all foreign keys in this table */
@@ -10075,35 +10123,43 @@
1007510123
 */
1007610124
 struct Index {
1007710125
 char *zName; /* Name of this index */
1007810126
 int nColumn; /* Number of columns in the table used by this index */
1007910127
 int *aiColumn; /* Which columns are used by this index. 1st is 0 */
10080
- unsigned *aiRowEst; /* Result of ANALYZE: Est. rows selected by each column */
10128
+ tRowcnt *aiRowEst; /* Result of ANALYZE: Est. rows selected by each column */
1008110129
 Table *pTable; /* The SQL table being indexed */
1008210130
 int tnum; /* Page containing root of this index in database file */
1008310131
 u8 onError; /* OE_Abort, OE_Ignore, OE_Replace, or OE_None */
1008410132
 u8 autoIndex; /* True if is automatically created (ex: by UNIQUE) */
1008510133
 u8 bUnordered; /* Use this index for == or IN queries only */
10134
+ u8 nSample; /* Number of elements in aSample[] */
1008610135
 char *zColAff; /* String defining the affinity of each column */
1008710136
 Index *pNext; /* The next index associated with the same table */
1008810137
 Schema *pSchema; /* Schema containing this index */
1008910138
 u8 *aSortOrder; /* Array of size Index.nColumn. True==DESC, False==ASC */
1009010139
 char **azColl; /* Array of collation sequence names for index */
10091
- IndexSample *aSample; /* Array of SQLITE_INDEX_SAMPLES samples */
10140
+#ifdef SQLITE_ENABLE_STAT3
10141
+ tRowcnt avgEq; /* Average nEq value for key values not in aSample */
10142
+ IndexSample *aSample; /* Samples of the left-most key */
10143
+#endif
1009210144
 };
1009310145
1009410146
 /*
1009510147
 ** Each sample stored in the sqlite_stat2 table is represented in memory
1009610148
 ** using a structure of this type.
1009710149
 */
1009810150
 struct IndexSample {
1009910151
 union {
1010010152
 char *z; /* Value if eType is SQLITE_TEXT or SQLITE_BLOB */
10101
- double r; /* Value if eType is SQLITE_FLOAT or SQLITE_INTEGER */
10153
+ double r; /* Value if eType is SQLITE_FLOAT */
10154
+ i64 i; /* Value if eType is SQLITE_INTEGER */
1010210155
 } u;
1010310156
 u8 eType; /* SQLITE_NULL, SQLITE_INTEGER ... etc. */
10104
- u8 nByte; /* Size in byte of text or blob. */
10157
+ u16 nByte; /* Size in byte of text or blob. */
10158
+ tRowcnt nEq; /* Est. number of rows where the key equals this sample */
10159
+ tRowcnt nLt; /* Est. number of rows where key is less than this sample */
10160
+ tRowcnt nDLt; /* Est. number of distinct keys less than this sample */
1010510161
 };
1010610162
1010710163
 /*
1010810164
 ** Each token coming out of the lexer is an instance of
1010910165
 ** this structure. Tokens are also used as part of an expression.
@@ -10446,10 +10502,11 @@
1044610502
 Table *pTab; /* An SQL table corresponding to zName */
1044710503
 Select *pSelect; /* A SELECT statement used in place of a table name */
1044810504
 u8 isPopulated; /* Temporary table associated with SELECT is populated */
1044910505
 u8 jointype; /* Type of join between this able and the previous */
1045010506
 u8 notIndexed; /* True if there is a NOT INDEXED clause */
10507
+ u8 isCorrelated; /* True if sub-query is correlated */
1045110508
 #ifndef SQLITE_OMIT_EXPLAIN
1045210509
 u8 iSelectId; /* If pSelect!=0, the id of the sub-select in EQP */
1045310510
 #endif
1045410511
 int iCursor; /* The VDBE cursor number used to access this table */
1045510512
 Expr *pOn; /* The ON clause of a join */
@@ -10565,10 +10622,11 @@
1056510622
 struct WhereInfo {
1056610623
 Parse *pParse; /* Parsing and code generating context */
1056710624
 u16 wctrlFlags; /* Flags originally passed to sqlite3WhereBegin() */
1056810625
 u8 okOnePass; /* Ok to use one-pass algorithm for UPDATE or DELETE */
1056910626
 u8 untestedTerms; /* Not all WHERE terms resolved by outer loop */
10627
+ u8 eDistinct;
1057010628
 SrcList *pTabList; /* List of tables in the join */
1057110629
 int iTop; /* The very beginning of the WHERE loop */
1057210630
 int iContinue; /* Jump here to continue with next record */
1057310631
 int iBreak; /* Jump here to break out of the loop */
1057410632
 int nLevel; /* Number of nested loop */
@@ -10576,10 +10634,13 @@
1057610634
 double savedNQueryLoop; /* pParse->nQueryLoop outside the WHERE loop */
1057710635
 double nRowOut; /* Estimated number of output rows */
1057810636
 WhereLevel a[1]; /* Information about each nest loop in WHERE */
1057910637
 };
1058010638
10639
+#define WHERE_DISTINCT_UNIQUE 1
10640
+#define WHERE_DISTINCT_ORDERED 2
10641
+
1058110642
 /*
1058210643
 ** A NameContext defines a context in which to resolve table and column
1058310644
 ** names. The context consists of a list of tables (the pSrcList) field and
1058410645
 ** a list of named expression (pEList). The named expression list may
1058510646
 ** be NULL. The pSrc corresponds to the FROM clause of a SELECT or
@@ -11052,10 +11113,11 @@
1105211113
 int isPCacheInit; /* True after malloc is initialized */
1105311114
 sqlite3_mutex *pInitMutex; /* Mutex used by sqlite3_initialize() */
1105411115
 int nRefInitMutex; /* Number of users of pInitMutex */
1105511116
 void (*xLog)(void*,int,const char*); /* Function for logging */
1105611117
 void *pLogArg; /* First argument to xLog() */
11118
+ int bLocaltimeFault; /* True to fail localtime() calls */
1105711119
 };
1105811120
1105911121
 /*
1106011122
 ** Context pointer passed down through the tree-walk.
1106111123
 */
@@ -11299,10 +11361,11 @@
1129911361
 #else
1130011362
 # define sqlite3ViewGetColumnNames(A,B) 0
1130111363
 #endif
1130211364
1130311365
 SQLITE_PRIVATE void sqlite3DropTable(Parse*, SrcList*, int, int);
11366
+SQLITE_PRIVATE void sqlite3CodeDropTable(Parse*, Table*, int, int);
1130411367
 SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3*, Table*);
1130511368
 #ifndef SQLITE_OMIT_AUTOINCREMENT
1130611369
 SQLITE_PRIVATE void sqlite3AutoincrementBegin(Parse *pParse);
1130711370
 SQLITE_PRIVATE void sqlite3AutoincrementEnd(Parse *pParse);
1130811371
 #else
@@ -11336,11 +11399,11 @@
1133611399
 #if defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT) && !defined(SQLITE_OMIT_SUBQUERY)
1133711400
 SQLITE_PRIVATE Expr *sqlite3LimitWhere(Parse *, SrcList *, Expr *, ExprList *, Expr *, Expr *, char *);
1133811401
 #endif
1133911402
 SQLITE_PRIVATE void sqlite3DeleteFrom(Parse*, SrcList*, Expr*);
1134011403
 SQLITE_PRIVATE void sqlite3Update(Parse*, SrcList*, ExprList*, Expr*, int);
11341
-SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(Parse*, SrcList*, Expr*, ExprList**, u16);
11404
+SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(Parse*, SrcList*, Expr*, ExprList**,ExprList*,u16);
1134211405
 SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo*);
1134311406
 SQLITE_PRIVATE int sqlite3ExprCodeGetColumn(Parse*, Table*, int, int, int);
1134411407
 SQLITE_PRIVATE void sqlite3ExprCodeGetColumnOfTable(Vdbe*, Table*, int, int, int);
1134511408
 SQLITE_PRIVATE void sqlite3ExprCodeMove(Parse*, int, int, int);
1134611409
 SQLITE_PRIVATE void sqlite3ExprCodeCopy(Parse*, int, int, int);
@@ -11479,11 +11542,11 @@
1147911542
 SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8);
1148011543
 SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*);
1148111544
 SQLITE_PRIVATE int sqlite3Atoi(const char*);
1148211545
 SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar);
1148311546
 SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte);
11484
-SQLITE_PRIVATE int sqlite3Utf8Read(const u8*, const u8**);
11547
+SQLITE_PRIVATE u32 sqlite3Utf8Read(const u8*, const u8**);
1148511548
1148611549
 /*
1148711550
 ** Routines to read and write variable-length integers. These used to
1148811551
 ** be defined locally, but now we use the varint routines in the util.c
1148911552
 ** file. Code should use the MACRO forms below, as the Varint32 versions
@@ -11555,11 +11618,11 @@
1155511618
 SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8,
1155611619
 void(*)(void*));
1155711620
 SQLITE_PRIVATE void sqlite3ValueFree(sqlite3_value*);
1155811621
 SQLITE_PRIVATE sqlite3_value *sqlite3ValueNew(sqlite3 *);
1155911622
 SQLITE_PRIVATE char *sqlite3Utf16to8(sqlite3 *, const void*, int, u8);
11560
-#ifdef SQLITE_ENABLE_STAT2
11623
+#ifdef SQLITE_ENABLE_STAT3
1156111624
 SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *, u8, char *, int, int *);
1156211625
 #endif
1156311626
 SQLITE_PRIVATE int sqlite3ValueFromExpr(sqlite3 *, Expr *, u8, u8, sqlite3_value **);
1156411627
 SQLITE_PRIVATE void sqlite3ValueApplyAffinity(sqlite3_value *, u8, u8);
1156511628
 #ifndef SQLITE_AMALGAMATION
@@ -12012,10 +12075,11 @@
1201212075
 0, /* isPCacheInit */
1201312076
 0, /* pInitMutex */
1201412077
 0, /* nRefInitMutex */
1201512078
 0, /* xLog */
1201612079
 0, /* pLogArg */
12080
+ 0, /* bLocaltimeFault */
1201712081
 };
1201812082
1201912083
1202012084
 /*
1202112085
 ** Hash table for global functions - functions common to all
@@ -12180,10 +12244,13 @@
1218012244
 #ifdef SQLITE_ENABLE_RTREE
1218112245
 "ENABLE_RTREE",
1218212246
 #endif
1218312247
 #ifdef SQLITE_ENABLE_STAT2
1218412248
 "ENABLE_STAT2",
12249
+#endif
12250
+#ifdef SQLITE_ENABLE_STAT3
12251
+ "ENABLE_STAT3",
1218512252
 #endif
1218612253
 #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY
1218712254
 "ENABLE_UNLOCK_NOTIFY",
1218812255
 #endif
1218912256
 #ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT
@@ -13162,26 +13229,10 @@
1316213229
 */
1316313230
 #include <time.h>
1316413231
1316513232
 #ifndef SQLITE_OMIT_DATETIME_FUNCS
1316613233
13167
-/*
13168
-** On recent Windows platforms, the localtime_s() function is available
13169
-** as part of the "Secure CRT". It is essentially equivalent to
13170
-** localtime_r() available under most POSIX platforms, except that the
13171
-** order of the parameters is reversed.
13172
-**
13173
-** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx.
13174
-**
13175
-** If the user has not indicated to use localtime_r() or localtime_s()
13176
-** already, check for an MSVC build environment that provides
13177
-** localtime_s().
13178
-*/
13179
-#if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \
13180
- defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE)
13181
-#define HAVE_LOCALTIME_S 1
13182
-#endif
1318313234
1318413235
 /*
1318513236
 ** A structure for holding a single date and time.
1318613237
 */
1318713238
 typedef struct DateTime DateTime;
@@ -13522,20 +13573,90 @@
1352213573
 static void clearYMD_HMS_TZ(DateTime *p){
1352313574
 p->validYMD = 0;
1352413575
 p->validHMS = 0;
1352513576
 p->validTZ = 0;
1352613577
 }
13578
+
13579
+/*
13580
+** On recent Windows platforms, the localtime_s() function is available
13581
+** as part of the "Secure CRT". It is essentially equivalent to
13582
+** localtime_r() available under most POSIX platforms, except that the
13583
+** order of the parameters is reversed.
13584
+**
13585
+** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx.
13586
+**
13587
+** If the user has not indicated to use localtime_r() or localtime_s()
13588
+** already, check for an MSVC build environment that provides
13589
+** localtime_s().
13590
+*/
13591
+#if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \
13592
+ defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE)
13593
+#define HAVE_LOCALTIME_S 1
13594
+#endif
13595
+
13596
+#ifndef SQLITE_OMIT_LOCALTIME
13597
+/*
13598
+** The following routine implements the rough equivalent of localtime_r()
13599
+** using whatever operating-system specific localtime facility that
13600
+** is available. This routine returns 0 on success and
13601
+** non-zero on any kind of error.
13602
+**
13603
+** If the sqlite3GlobalConfig.bLocaltimeFault variable is true then this
13604
+** routine will always fail.
13605
+*/
13606
+static int osLocaltime(time_t *t, struct tm *pTm){
13607
+ int rc;
13608
+#if (!defined(HAVE_LOCALTIME_R) || !HAVE_LOCALTIME_R) \
13609
+ && (!defined(HAVE_LOCALTIME_S) || !HAVE_LOCALTIME_S)
13610
+ struct tm *pX;
13611
+#if SQLITE_THREADSAFE>0
13612
+ sqlite3_mutex *mutex = sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER);
13613
+#endif
13614
+ sqlite3_mutex_enter(mutex);
13615
+ pX = localtime(t);
13616
+#ifndef SQLITE_OMIT_BUILTIN_TEST
13617
+ if( sqlite3GlobalConfig.bLocaltimeFault ) pX = 0;
13618
+#endif
13619
+ if( pX ) *pTm = *pX;
13620
+ sqlite3_mutex_leave(mutex);
13621
+ rc = pX==0;
13622
+#else
13623
+#ifndef SQLITE_OMIT_BUILTIN_TEST
13624
+ if( sqlite3GlobalConfig.bLocaltimeFault ) return 1;
13625
+#endif
13626
+#if defined(HAVE_LOCALTIME_R) && HAVE_LOCALTIME_R
13627
+ rc = localtime_r(t, pTm)==0;
13628
+#else
13629
+ rc = localtime_s(pTm, t);
13630
+#endif /* HAVE_LOCALTIME_R */
13631
+#endif /* HAVE_LOCALTIME_R || HAVE_LOCALTIME_S */
13632
+ return rc;
13633
+}
13634
+#endif /* SQLITE_OMIT_LOCALTIME */
13635
+
1352713636
1352813637
 #ifndef SQLITE_OMIT_LOCALTIME
1352913638
 /*
13530
-** Compute the difference (in milliseconds)
13531
-** between localtime and UTC (a.k.a. GMT)
13532
-** for the time value p where p is in UTC.
13639
+** Compute the difference (in milliseconds) between localtime and UTC
13640
+** (a.k.a. GMT) for the time value p where p is in UTC. If no error occurs,
13641
+** return this value and set *pRc to SQLITE_OK.
13642
+**
13643
+** Or, if an error does occur, set *pRc to SQLITE_ERROR. The returned value
13644
+** is undefined in this case.
1353313645
 */
13534
-static sqlite3_int64 localtimeOffset(DateTime *p){
13646
+static sqlite3_int64 localtimeOffset(
13647
+ DateTime *p, /* Date at which to calculate offset */
13648
+ sqlite3_context *pCtx, /* Write error here if one occurs */
13649
+ int *pRc /* OUT: Error code. SQLITE_OK or ERROR */
13650
+){
1353513651
 DateTime x, y;
1353613652
 time_t t;
13653
+ struct tm sLocal;
13654
+
13655
+ /* Initialize the contents of sLocal to avoid a compiler warning. */
13656
+ memset(&sLocal, 0, sizeof(sLocal));
13657
+
1353713658
 x = *p;
1353813659
 computeYMD_HMS(&x);
1353913660
 if( x.Y<1971 || x.Y>=2038 ){
1354013661
 x.Y = 2000;
1354113662
 x.M = 1;
@@ -13549,51 +13670,27 @@
1354913670
 }
1355013671
 x.tz = 0;
1355113672
 x.validJD = 0;
1355213673
 computeJD(&x);
1355313674
 t = (time_t)(x.iJD/1000 - 21086676*(i64)10000);
13554
-#ifdef HAVE_LOCALTIME_R
13555
- {
13556
- struct tm sLocal;
13557
- localtime_r(&t, &sLocal);
13558
- y.Y = sLocal.tm_year + 1900;
13559
- y.M = sLocal.tm_mon + 1;
13560
- y.D = sLocal.tm_mday;
13561
- y.h = sLocal.tm_hour;
13562
- y.m = sLocal.tm_min;
13563
- y.s = sLocal.tm_sec;
13564
- }
13565
-#elif defined(HAVE_LOCALTIME_S) && HAVE_LOCALTIME_S
13566
- {
13567
- struct tm sLocal;
13568
- localtime_s(&sLocal, &t);
13569
- y.Y = sLocal.tm_year + 1900;
13570
- y.M = sLocal.tm_mon + 1;
13571
- y.D = sLocal.tm_mday;
13572
- y.h = sLocal.tm_hour;
13573
- y.m = sLocal.tm_min;
13574
- y.s = sLocal.tm_sec;
13575
- }
13576
-#else
13577
- {
13578
- struct tm *pTm;
13579
- sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER));
13580
- pTm = localtime(&t);
13581
- y.Y = pTm->tm_year + 1900;
13582
- y.M = pTm->tm_mon + 1;
13583
- y.D = pTm->tm_mday;
13584
- y.h = pTm->tm_hour;
13585
- y.m = pTm->tm_min;
13586
- y.s = pTm->tm_sec;
13587
- sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER));
13588
- }
13589
-#endif
13675
+ if( osLocaltime(&t, &sLocal) ){
13676
+ sqlite3_result_error(pCtx, "local time unavailable", -1);
13677
+ *pRc = SQLITE_ERROR;
13678
+ return 0;
13679
+ }
13680
+ y.Y = sLocal.tm_year + 1900;
13681
+ y.M = sLocal.tm_mon + 1;
13682
+ y.D = sLocal.tm_mday;
13683
+ y.h = sLocal.tm_hour;
13684
+ y.m = sLocal.tm_min;
13685
+ y.s = sLocal.tm_sec;
1359013686
 y.validYMD = 1;
1359113687
 y.validHMS = 1;
1359213688
 y.validJD = 0;
1359313689
 y.validTZ = 0;
1359413690
 computeJD(&y);
13691
+ *pRc = SQLITE_OK;
1359513692
 return y.iJD - x.iJD;
1359613693
 }
1359713694
 #endif /* SQLITE_OMIT_LOCALTIME */
1359813695
1359913696
 /*
@@ -13613,13 +13710,16 @@
1361313710
 ** weekday N
1361413711
 ** unixepoch
1361513712
 ** localtime
1361613713
 ** utc
1361713714
 **
13618
-** Return 0 on success and 1 if there is any kind of error.
13715
+** Return 0 on success and 1 if there is any kind of error. If the error
13716
+** is in a system call (i.e. localtime()), then an error message is written
13717
+** to context pCtx. If the error is an unrecognized modifier, no error is
13718
+** written to pCtx.
1361913719
 */
13620
-static int parseModifier(const char *zMod, DateTime *p){
13720
+static int parseModifier(sqlite3_context *pCtx, const char *zMod, DateTime *p){
1362113721
 int rc = 1;
1362213722
 int n;
1362313723
 double r;
1362413724
 char *z, zBuf[30];
1362513725
 z = zBuf;
@@ -13635,13 +13735,12 @@
1363513735
 ** Assuming the current time value is UTC (a.k.a. GMT), shift it to
1363613736
 ** show local time.
1363713737
 */
1363813738
 if( strcmp(z, "localtime")==0 ){
1363913739
 computeJD(p);
13640
- p->iJD += localtimeOffset(p);
13740
+ p->iJD += localtimeOffset(p, pCtx, &rc);
1364113741
 clearYMD_HMS_TZ(p);
13642
- rc = 0;
1364313742
 }
1364413743
 break;
1364513744
 }
1364613745
 #endif
1364713746
 case 'u': {
@@ -13658,15 +13757,16 @@
1365813757
 }
1365913758
 #ifndef SQLITE_OMIT_LOCALTIME
1366013759
 else if( strcmp(z, "utc")==0 ){
1366113760
 sqlite3_int64 c1;
1366213761
 computeJD(p);
13663
- c1 = localtimeOffset(p);
13664
- p->iJD -= c1;
13665
- clearYMD_HMS_TZ(p);
13666
- p->iJD += c1 - localtimeOffset(p);
13667
- rc = 0;
13762
+ c1 = localtimeOffset(p, pCtx, &rc);
13763
+ if( rc==SQLITE_OK ){
13764
+ p->iJD -= c1;
13765
+ clearYMD_HMS_TZ(p);
13766
+ p->iJD += c1 - localtimeOffset(p, pCtx, &rc);
13767
+ }
1366813768
 }
1366913769
 #endif
1367013770
 break;
1367113771
 }
1367213772
 case 'w': {
@@ -13843,13 +13943,12 @@
1384313943
 if( !z || parseDateOrTime(context, (char*)z, p) ){
1384413944
 return 1;
1384513945
 }
1384613946
 }
1384713947
 for(i=1; i<argc; i++){
13848
- if( (z = sqlite3_value_text(argv[i]))==0 || parseModifier((char*)z, p) ){
13849
- return 1;
13850
- }
13948
+ z = sqlite3_value_text(argv[i]);
13949
+ if( z==0 || parseModifier(context, (char*)z, p) ) return 1;
1385113950
 }
1385213951
 return 0;
1385313952
 }
1385413953
1385513954
@@ -14343,11 +14442,11 @@
1434314442
 DO_OS_MALLOC_TEST(0);
1434414443
 /* 0x87f3f is a mask of SQLITE_OPEN_ flags that are valid to be passed
1434514444
 ** down into the VFS layer. Some SQLITE_OPEN_ flags (for example,
1434614445
 ** SQLITE_OPEN_FULLMUTEX or SQLITE_OPEN_SHAREDCACHE) are blocked before
1434714446
 ** reaching the VFS. */
14348
- rc = pVfs->xOpen(pVfs, zPath, pFile, flags & 0x87f3f, pFlagsOut);
14447
+ rc = pVfs->xOpen(pVfs, zPath, pFile, flags & 0x87f7f, pFlagsOut);
1434914448
 assert( rc==SQLITE_OK || pFile->pMethods==0 );
1435014449
 return rc;
1435114450
 }
1435214451
 SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){
1435314452
 return pVfs->xDelete(pVfs, zPath, dirSync);
@@ -15804,11 +15903,11 @@
1580415903
 ** Free an outstanding memory allocation.
1580515904
 **
1580615905
 ** This function assumes that the necessary mutexes, if any, are
1580715906
 ** already held by the caller. Hence "Unsafe".
1580815907
 */
15809
-void memsys3FreeUnsafe(void *pOld){
15908
+static void memsys3FreeUnsafe(void *pOld){
1581015909
 Mem3Block *p = (Mem3Block*)pOld;
1581115910
 int i;
1581215911
 u32 size, x;
1581315912
 assert( sqlite3_mutex_held(mem3.mutex) );
1581415913
 assert( p>mem3.aPool && p<&mem3.aPool[mem3.nPool] );
@@ -15879,21 +15978,21 @@
1587915978
 }
1588015979
1588115980
 /*
1588215981
 ** Free memory.
1588315982
 */
15884
-void memsys3Free(void *pPrior){
15983
+static void memsys3Free(void *pPrior){
1588515984
 assert( pPrior );
1588615985
 memsys3Enter();
1588715986
 memsys3FreeUnsafe(pPrior);
1588815987
 memsys3Leave();
1588915988
 }
1589015989
1589115990
 /*
1589215991
 ** Change the size of an existing memory allocation
1589315992
 */
15894
-void *memsys3Realloc(void *pPrior, int nBytes){
15993
+static void *memsys3Realloc(void *pPrior, int nBytes){
1589515994
 int nOld;
1589615995
 void *p;
1589715996
 if( pPrior==0 ){
1589815997
 return sqlite3_malloc(nBytes);
1589915998
 }
@@ -20086,11 +20185,11 @@
2008620185
 } \
2008720186
 if( c<0x80 \
2008820187
 || (c&0xFFFFF800)==0xD800 \
2008920188
 || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \
2009020189
 }
20091
-SQLITE_PRIVATE int sqlite3Utf8Read(
20190
+SQLITE_PRIVATE u32 sqlite3Utf8Read(
2009220191
 const unsigned char *zIn, /* First byte of UTF-8 character */
2009320192
 const unsigned char **pzNext /* Write first byte past UTF-8 char here */
2009420193
 ){
2009520194
 unsigned int c;
2009620195
@@ -20387,11 +20486,11 @@
2038720486
 ** no longer required.
2038820487
 **
2038920488
 ** If a malloc failure occurs, NULL is returned and the db.mallocFailed
2039020489
 ** flag set.
2039120490
 */
20392
-#ifdef SQLITE_ENABLE_STAT2
20491
+#ifdef SQLITE_ENABLE_STAT3
2039320492
 SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *db, u8 enc, char *z, int n, int *pnOut){
2039420493
 Mem m;
2039520494
 memset(&m, 0, sizeof(m));
2039620495
 m.db = db;
2039720496
 sqlite3VdbeMemSetStr(&m, z, n, SQLITE_UTF8, SQLITE_STATIC);
@@ -21633,26 +21732,32 @@
2163321732
 return -x;
2163421733
 }
2163521734
2163621735
 #ifdef SQLITE_ENABLE_8_3_NAMES
2163721736
 /*
21638
-** If SQLITE_ENABLE_8_3_NAME is set at compile-time and if the database
21737
+** If SQLITE_ENABLE_8_3_NAMES is set at compile-time and if the database
2163921738
 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
2164021739
 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
2164121740
 ** three characters, then shorten the suffix on z[] to be the last three
2164221741
 ** characters of the original suffix.
21742
+**
21743
+** If SQLITE_ENABLE_8_3_NAMES is set to 2 at compile-time, then always
21744
+** do the suffix shortening regardless of URI parameter.
2164321745
 **
2164421746
 ** Examples:
2164521747
 **
2164621748
 ** test.db-journal => test.nal
2164721749
 ** test.db-wal => test.wal
2164821750
 ** test.db-shm => test.shm
2164921751
 */
2165021752
 SQLITE_PRIVATE void sqlite3FileSuffix3(const char *zBaseFilename, char *z){
21753
+#if SQLITE_ENABLE_8_3_NAMES<2
2165121754
 const char *zOk;
2165221755
 zOk = sqlite3_uri_parameter(zBaseFilename, "8_3_names");
21653
- if( zOk && sqlite3GetBoolean(zOk) ){
21756
+ if( zOk && sqlite3GetBoolean(zOk) )
21757
+#endif
21758
+ {
2165421759
 int i, sz;
2165521760
 sz = sqlite3Strlen30(z);
2165621761
 for(i=sz-1; i>0 && z[i]!='/' && z[i]!='.'; i--){}
2165721762
 if( z[i]=='.' && ALWAYS(sz>i+4) ) memcpy(&z[i+1], &z[sz-3], 4);
2165821763
 }
@@ -22182,14 +22287,17 @@
2218222287
 #ifdef MEMORY_DEBUG
2218322288
 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
2218422289
 #endif
2218522290
2218622291
 #ifdef SQLITE_DEBUG
22187
-SQLITE_PRIVATE int sqlite3OSTrace = 0;
22188
-#define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
22292
+# ifndef SQLITE_DEBUG_OS_TRACE
22293
+# define SQLITE_DEBUG_OS_TRACE 0
22294
+# endif
22295
+ int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
22296
+# define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
2218922297
 #else
22190
-#define OSTRACE(X)
22298
+# define OSTRACE(X)
2219122299
 #endif
2219222300
2219322301
 /*
2219422302
 ** Macros for performance tracing. Normally turned off. Only works
2219522303
 ** on i486 hardware.
@@ -24363,10 +24471,14 @@
2436324471
2436424472
 #if defined(__APPLE__) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS)
2436524473
 # include <sys/mount.h>
2436624474
 #endif
2436724475
24476
+#ifdef HAVE_UTIME
24477
+# include <utime.h>
24478
+#endif
24479
+
2436824480
 /*
2436924481
 ** Allowed values of unixFile.fsFlags
2437024482
 */
2437124483
 #define SQLITE_FSFLAGS_IS_MSDOS 0x1
2437224484
@@ -24428,11 +24540,10 @@
2442824540
 typedef struct unixFile unixFile;
2442924541
 struct unixFile {
2443024542
 sqlite3_io_methods const *pMethod; /* Always the first entry */
2443124543
 unixInodeInfo *pInode; /* Info about locks on this inode */
2443224544
 int h; /* The file descriptor */
24433
- int dirfd; /* File descriptor for the directory */
2443424545
 unsigned char eFileLock; /* The type of lock held on this fd */
2443524546
 unsigned char ctrlFlags; /* Behavioral bits. UNIXFILE_* flags */
2443624547
 int lastErrno; /* The unix errno from last I/O error */
2443724548
 void *lockingContext; /* Locking style specific state */
2443824549
 UnixUnusedFd *pUnused; /* Pre-allocated UnixUnusedFd */
@@ -24470,12 +24581,14 @@
2447024581
 };
2447124582
2447224583
 /*
2447324584
 ** Allowed values for the unixFile.ctrlFlags bitmask:
2447424585
 */
24475
-#define UNIXFILE_EXCL 0x01 /* Connections from one process only */
24476
-#define UNIXFILE_RDONLY 0x02 /* Connection is read only */
24586
+#define UNIXFILE_EXCL 0x01 /* Connections from one process only */
24587
+#define UNIXFILE_RDONLY 0x02 /* Connection is read only */
24588
+#define UNIXFILE_PERSIST_WAL 0x04 /* Persistent WAL mode */
24589
+#define UNIXFILE_DIRSYNC 0x08 /* Directory sync needed */
2447724590
2447824591
 /*
2447924592
 ** Include code that is common to all os_*.c files
2448024593
 */
2448124594
 /************** Include os_common.h in the middle of os_unix.c ***************/
@@ -24510,14 +24623,17 @@
2451024623
 #ifdef MEMORY_DEBUG
2451124624
 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
2451224625
 #endif
2451324626
2451424627
 #ifdef SQLITE_DEBUG
24515
-SQLITE_PRIVATE int sqlite3OSTrace = 0;
24516
-#define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
24628
+# ifndef SQLITE_DEBUG_OS_TRACE
24629
+# define SQLITE_DEBUG_OS_TRACE 0
24630
+# endif
24631
+ int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
24632
+# define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
2451724633
 #else
24518
-#define OSTRACE(X)
24634
+# define OSTRACE(X)
2451924635
 #endif
2452024636
2452124637
 /*
2452224638
 ** Macros for performance tracing. Normally turned off. Only works
2452324639
 ** on i486 hardware.
@@ -24722,10 +24838,13 @@
2472224838
 */
2472324839
 static int posixOpen(const char *zFile, int flags, int mode){
2472424840
 return open(zFile, flags, mode);
2472524841
 }
2472624842
24843
+/* Forward reference */
24844
+static int openDirectory(const char*, int*);
24845
+
2472724846
 /*
2472824847
 ** Many system calls are accessed through pointer-to-functions so that
2472924848
 ** they may be overridden at runtime to facilitate fault injection during
2473024849
 ** testing and sandboxing. The following array holds the names and pointers
2473124850
 ** to all overrideable system calls.
@@ -24818,10 +24937,16 @@
2481824937
 #else
2481924938
 { "fallocate", (sqlite3_syscall_ptr)0, 0 },
2482024939
 #endif
2482124940
 #define osFallocate ((int(*)(int,off_t,off_t))aSyscall[15].pCurrent)
2482224941
24942
+ { "unlink", (sqlite3_syscall_ptr)unlink, 0 },
24943
+#define osUnlink ((int(*)(const char*))aSyscall[16].pCurrent)
24944
+
24945
+ { "openDirectory", (sqlite3_syscall_ptr)openDirectory, 0 },
24946
+#define osOpenDirectory ((int(*)(const char*,int*))aSyscall[17].pCurrent)
24947
+
2482324948
 }; /* End of the overrideable system calls */
2482424949
2482524950
 /*
2482624951
 ** This is the xSetSystemCall() method of sqlite3_vfs for all of the
2482724952
 ** "unix" VFSes. Return SQLITE_OK opon successfully updating the
@@ -25102,11 +25227,13 @@
2510225227
 case EINVAL:
2510325228
 case ENOTCONN:
2510425229
 case ENODEV:
2510525230
 case ENXIO:
2510625231
 case ENOENT:
25232
+#ifdef ESTALE /* ESTALE is not defined on Interix systems */
2510725233
 case ESTALE:
25234
+#endif
2510825235
 case ENOSYS:
2510925236
 /* these should force the client to close the file and reconnect */
2511025237
2511125238
 default:
2511225239
 return sqliteIOErr;
@@ -26172,22 +26299,18 @@
2617226299
 ** even on VxWorks. A mutex will be acquired on VxWorks by the
2617326300
 ** vxworksReleaseFileId() routine.
2617426301
 */
2617526302
 static int closeUnixFile(sqlite3_file *id){
2617626303
 unixFile *pFile = (unixFile*)id;
26177
- if( pFile->dirfd>=0 ){
26178
- robust_close(pFile, pFile->dirfd, __LINE__);
26179
- pFile->dirfd=-1;
26180
- }
2618126304
 if( pFile->h>=0 ){
2618226305
 robust_close(pFile, pFile->h, __LINE__);
2618326306
 pFile->h = -1;
2618426307
 }
2618526308
 #if OS_VXWORKS
2618626309
 if( pFile->pId ){
2618726310
 if( pFile->isDelete ){
26188
- unlink(pFile->pId->zCanonicalName);
26311
+ osUnlink(pFile->pId->zCanonicalName);
2618926312
 }
2619026313
 vxworksReleaseFileId(pFile->pId);
2619126314
 pFile->pId = 0;
2619226315
 }
2619326316
 #endif
@@ -26368,12 +26491,14 @@
2636826491
 /* If we have any lock, then the lock file already exists. All we have
2636926492
 ** to do is adjust our internal record of the lock level.
2637026493
 */
2637126494
 if( pFile->eFileLock > NO_LOCK ){
2637226495
 pFile->eFileLock = eFileLock;
26373
-#if !OS_VXWORKS
2637426496
 /* Always update the timestamp on the old file */
26497
+#ifdef HAVE_UTIME
26498
+ utime(zLockFile, NULL);
26499
+#else
2637526500
 utimes(zLockFile, NULL);
2637626501
 #endif
2637726502
 return SQLITE_OK;
2637826503
 }
2637926504
@@ -26430,11 +26555,11 @@
2643026555
 return SQLITE_OK;
2643126556
 }
2643226557
2643326558
 /* To fully unlock the database, delete the lock file */
2643426559
 assert( eFileLock==NO_LOCK );
26435
- if( unlink(zLockFile) ){
26560
+ if( osUnlink(zLockFile) ){
2643626561
 int rc = 0;
2643726562
 int tErrno = errno;
2643826563
 if( ENOENT != tErrno ){
2643926564
 rc = SQLITE_IOERR_UNLOCK;
2644026565
 }
@@ -27665,10 +27790,54 @@
2766527790
 if( OS_VXWORKS && rc!= -1 ){
2766627791
 rc = 0;
2766727792
 }
2766827793
 return rc;
2766927794
 }
27795
+
27796
+/*
27797
+** Open a file descriptor to the directory containing file zFilename.
27798
+** If successful, *pFd is set to the opened file descriptor and
27799
+** SQLITE_OK is returned. If an error occurs, either SQLITE_NOMEM
27800
+** or SQLITE_CANTOPEN is returned and *pFd is set to an undefined
27801
+** value.
27802
+**
27803
+** The directory file descriptor is used for only one thing - to
27804
+** fsync() a directory to make sure file creation and deletion events
27805
+** are flushed to disk. Such fsyncs are not needed on newer
27806
+** journaling filesystems, but are required on older filesystems.
27807
+**
27808
+** This routine can be overridden using the xSetSysCall interface.
27809
+** The ability to override this routine was added in support of the
27810
+** chromium sandbox. Opening a directory is a security risk (we are
27811
+** told) so making it overrideable allows the chromium sandbox to
27812
+** replace this routine with a harmless no-op. To make this routine
27813
+** a no-op, replace it with a stub that returns SQLITE_OK but leaves
27814
+** *pFd set to a negative number.
27815
+**
27816
+** If SQLITE_OK is returned, the caller is responsible for closing
27817
+** the file descriptor *pFd using close().
27818
+*/
27819
+static int openDirectory(const char *zFilename, int *pFd){
27820
+ int ii;
27821
+ int fd = -1;
27822
+ char zDirname[MAX_PATHNAME+1];
27823
+
27824
+ sqlite3_snprintf(MAX_PATHNAME, zDirname, "%s", zFilename);
27825
+ for(ii=(int)strlen(zDirname); ii>1 && zDirname[ii]!='/'; ii--);
27826
+ if( ii>0 ){
27827
+ zDirname[ii] = '\0';
27828
+ fd = robust_open(zDirname, O_RDONLY|O_BINARY, 0);
27829
+ if( fd>=0 ){
27830
+#ifdef FD_CLOEXEC
27831
+ osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
27832
+#endif
27833
+ OSTRACE(("OPENDIR %-3d %s\n", fd, zDirname));
27834
+ }
27835
+ }
27836
+ *pFd = fd;
27837
+ return (fd>=0?SQLITE_OK:unixLogError(SQLITE_CANTOPEN_BKPT, "open", zDirname));
27838
+}
2767027839
2767127840
 /*
2767227841
 ** Make sure all writes to a particular file are committed to disk.
2767327842
 **
2767427843
 ** If dataOnly==0 then both the file itself and its metadata (file
@@ -27706,32 +27875,25 @@
2770627875
 SimulateIOError( rc=1 );
2770727876
 if( rc ){
2770827877
 pFile->lastErrno = errno;
2770927878
 return unixLogError(SQLITE_IOERR_FSYNC, "full_fsync", pFile->zPath);
2771027879
 }
27711
- if( pFile->dirfd>=0 ){
27712
- OSTRACE(("DIRSYNC %-3d (have_fullfsync=%d fullsync=%d)\n", pFile->dirfd,
27880
+
27881
+ /* Also fsync the directory containing the file if the DIRSYNC flag
27882
+ ** is set. This is a one-time occurrance. Many systems (examples: AIX)
27883
+ ** are unable to fsync a directory, so ignore errors on the fsync.
27884
+ */
27885
+ if( pFile->ctrlFlags & UNIXFILE_DIRSYNC ){
27886
+ int dirfd;
27887
+ OSTRACE(("DIRSYNC %s (have_fullfsync=%d fullsync=%d)\n", pFile->zPath,
2771327888
 HAVE_FULLFSYNC, isFullsync));
27714
-#ifndef SQLITE_DISABLE_DIRSYNC
27715
- /* The directory sync is only attempted if full_fsync is
27716
- ** turned off or unavailable. If a full_fsync occurred above,
27717
- ** then the directory sync is superfluous.
27718
- */
27719
- if( (!HAVE_FULLFSYNC || !isFullsync) && full_fsync(pFile->dirfd,0,0) ){
27720
- /*
27721
- ** We have received multiple reports of fsync() returning
27722
- ** errors when applied to directories on certain file systems.
27723
- ** A failed directory sync is not a big deal. So it seems
27724
- ** better to ignore the error. Ticket #1657
27725
- */
27726
- /* pFile->lastErrno = errno; */
27727
- /* return SQLITE_IOERR; */
27728
- }
27729
-#endif
27730
- /* Only need to sync once, so close the directory when we are done */
27731
- robust_close(pFile, pFile->dirfd, __LINE__);
27732
- pFile->dirfd = -1;
27889
+ rc = osOpenDirectory(pFile->zPath, &dirfd);
27890
+ if( rc==SQLITE_OK && dirfd>=0 ){
27891
+ full_fsync(dirfd, 0, 0);
27892
+ robust_close(pFile, dirfd, __LINE__);
27893
+ }
27894
+ pFile->ctrlFlags &= ~UNIXFILE_DIRSYNC;
2773327895
 }
2773427896
 return rc;
2773527897
 }
2773627898
2773727899
 /*
@@ -27816,17 +27978,23 @@
2781627978
 ** If the user has configured a chunk-size for this file, it could be
2781727979
 ** that the file needs to be extended at this point. Otherwise, the
2781827980
 ** SQLITE_FCNTL_SIZE_HINT operation is a no-op for Unix.
2781927981
 */
2782027982
 static int fcntlSizeHint(unixFile *pFile, i64 nByte){
27821
- if( pFile->szChunk ){
27983
+ { /* preserve indentation of removed "if" */
2782227984
 i64 nSize; /* Required file size */
27985
+ i64 szChunk; /* Chunk size */
2782327986
 struct stat buf; /* Used to hold return values of fstat() */
2782427987
2782527988
 if( osFstat(pFile->h, &buf) ) return SQLITE_IOERR_FSTAT;
2782627989
27827
- nSize = ((nByte+pFile->szChunk-1) / pFile->szChunk) * pFile->szChunk;
27990
+ szChunk = pFile->szChunk;
27991
+ if( szChunk==0 ){
27992
+ nSize = nByte;
27993
+ }else{
27994
+ nSize = ((nByte+szChunk-1) / szChunk) * szChunk;
27995
+ }
2782827996
 if( nSize>(i64)buf.st_size ){
2782927997
2783027998
 #if defined(HAVE_POSIX_FALLOCATE) && HAVE_POSIX_FALLOCATE
2783127999
 /* The code below is handling the return value of osFallocate()
2783228000
 ** correctly. posix_fallocate() is defined to "returns zero on success,
@@ -27865,25 +28033,37 @@
2786528033
2786628034
 /*
2786728035
 ** Information and control of an open file handle.
2786828036
 */
2786928037
 static int unixFileControl(sqlite3_file *id, int op, void *pArg){
28038
+ unixFile *pFile = (unixFile*)id;
2787028039
 switch( op ){
2787128040
 case SQLITE_FCNTL_LOCKSTATE: {
27872
- *(int*)pArg = ((unixFile*)id)->eFileLock;
28041
+ *(int*)pArg = pFile->eFileLock;
2787328042
 return SQLITE_OK;
2787428043
 }
2787528044
 case SQLITE_LAST_ERRNO: {
27876
- *(int*)pArg = ((unixFile*)id)->lastErrno;
28045
+ *(int*)pArg = pFile->lastErrno;
2787728046
 return SQLITE_OK;
2787828047
 }
2787928048
 case SQLITE_FCNTL_CHUNK_SIZE: {
27880
- ((unixFile*)id)->szChunk = *(int *)pArg;
28049
+ pFile->szChunk = *(int *)pArg;
2788128050
 return SQLITE_OK;
2788228051
 }
2788328052
 case SQLITE_FCNTL_SIZE_HINT: {
27884
- return fcntlSizeHint((unixFile *)id, *(i64 *)pArg);
28053
+ return fcntlSizeHint(pFile, *(i64 *)pArg);
28054
+ }
28055
+ case SQLITE_FCNTL_PERSIST_WAL: {
28056
+ int bPersist = *(int*)pArg;
28057
+ if( bPersist<0 ){
28058
+ *(int*)pArg = (pFile->ctrlFlags & UNIXFILE_PERSIST_WAL)!=0;
28059
+ }else if( bPersist==0 ){
28060
+ pFile->ctrlFlags &= ~UNIXFILE_PERSIST_WAL;
28061
+ }else{
28062
+ pFile->ctrlFlags |= UNIXFILE_PERSIST_WAL;
28063
+ }
28064
+ return SQLITE_OK;
2788528065
 }
2788628066
 #ifndef NDEBUG
2788728067
 /* The pager calls this method to signal that it has done
2788828068
 ** a rollback and that the database is therefore unchanged and
2788928069
 ** it hence it is OK for the transaction change counter to be
@@ -28095,11 +28275,11 @@
2809528275
 unixShmNode *p = pFd->pInode->pShmNode;
2809628276
 assert( unixMutexHeld() );
2809728277
 if( p && p->nRef==0 ){
2809828278
 int i;
2809928279
 assert( p->pInode==pFd->pInode );
28100
- if( p->mutex ) sqlite3_mutex_free(p->mutex);
28280
+ sqlite3_mutex_free(p->mutex);
2810128281
 for(i=0; i<p->nRegion; i++){
2810228282
 if( p->h>=0 ){
2810328283
 munmap(p->apRegion[i], p->szRegion);
2810428284
 }else{
2810528285
 sqlite3_free(p->apRegion[i]);
@@ -28564,11 +28744,11 @@
2856428744
 ** shared-memory file, too */
2856528745
 unixEnterMutex();
2856628746
 assert( pShmNode->nRef>0 );
2856728747
 pShmNode->nRef--;
2856828748
 if( pShmNode->nRef==0 ){
28569
- if( deleteFlag && pShmNode->h>=0 ) unlink(pShmNode->zFilename);
28749
+ if( deleteFlag && pShmNode->h>=0 ) osUnlink(pShmNode->zFilename);
2857028750
 unixShmPurge(pDbFd);
2857128751
 }
2857228752
 unixLeaveMutex();
2857328753
2857428754
 return SQLITE_OK;
@@ -28877,11 +29057,11 @@
2887729057
 ** Initialize the contents of the unixFile structure pointed to by pId.
2887829058
 */
2887929059
 static int fillInUnixFile(
2888029060
 sqlite3_vfs *pVfs, /* Pointer to vfs object */
2888129061
 int h, /* Open file descriptor of file being opened */
28882
- int dirfd, /* Directory file descriptor */
29062
+ int syncDir, /* True to sync directory on first sync */
2888329063
 sqlite3_file *pId, /* Write to the unixFile structure here */
2888429064
 const char *zFilename, /* Name of the file being opened */
2888529065
 int noLock, /* Omit locking if true */
2888629066
 int isDelete, /* Delete on close if true */
2888729067
 int isReadOnly /* True if the file is opened read-only */
@@ -28908,20 +29088,22 @@
2890829088
 assert( zFilename==0 || zFilename[0]=='/' );
2890929089
 #endif
2891029090
2891129091
 OSTRACE(("OPEN %-3d %s\n", h, zFilename));
2891229092
 pNew->h = h;
28913
- pNew->dirfd = dirfd;
2891429093
 pNew->zPath = zFilename;
2891529094
 if( memcmp(pVfs->zName,"unix-excl",10)==0 ){
2891629095
 pNew->ctrlFlags = UNIXFILE_EXCL;
2891729096
 }else{
2891829097
 pNew->ctrlFlags = 0;
2891929098
 }
2892029099
 if( isReadOnly ){
2892129100
 pNew->ctrlFlags |= UNIXFILE_RDONLY;
2892229101
 }
29102
+ if( syncDir ){
29103
+ pNew->ctrlFlags |= UNIXFILE_DIRSYNC;
29104
+ }
2892329105
2892429106
 #if OS_VXWORKS
2892529107
 pNew->pId = vxworksFindFileId(zFilename);
2892629108
 if( pNew->pId==0 ){
2892729109
 noLock = 1;
@@ -29044,56 +29226,24 @@
2904429226
 pNew->lastErrno = 0;
2904529227
 #if OS_VXWORKS
2904629228
 if( rc!=SQLITE_OK ){
2904729229
 if( h>=0 ) robust_close(pNew, h, __LINE__);
2904829230
 h = -1;
29049
- unlink(zFilename);
29231
+ osUnlink(zFilename);
2905029232
 isDelete = 0;
2905129233
 }
2905229234
 pNew->isDelete = isDelete;
2905329235
 #endif
2905429236
 if( rc!=SQLITE_OK ){
29055
- if( dirfd>=0 ) robust_close(pNew, dirfd, __LINE__);
2905629237
 if( h>=0 ) robust_close(pNew, h, __LINE__);
2905729238
 }else{
2905829239
 pNew->pMethod = pLockingStyle;
2905929240
 OpenCounter(+1);
2906029241
 }
2906129242
 return rc;
2906229243
 }
2906329244
29064
-/*
29065
-** Open a file descriptor to the directory containing file zFilename.
29066
-** If successful, *pFd is set to the opened file descriptor and
29067
-** SQLITE_OK is returned. If an error occurs, either SQLITE_NOMEM
29068
-** or SQLITE_CANTOPEN is returned and *pFd is set to an undefined
29069
-** value.
29070
-**
29071
-** If SQLITE_OK is returned, the caller is responsible for closing
29072
-** the file descriptor *pFd using close().
29073
-*/
29074
-static int openDirectory(const char *zFilename, int *pFd){
29075
- int ii;
29076
- int fd = -1;
29077
- char zDirname[MAX_PATHNAME+1];
29078
-
29079
- sqlite3_snprintf(MAX_PATHNAME, zDirname, "%s", zFilename);
29080
- for(ii=(int)strlen(zDirname); ii>1 && zDirname[ii]!='/'; ii--);
29081
- if( ii>0 ){
29082
- zDirname[ii] = '\0';
29083
- fd = robust_open(zDirname, O_RDONLY|O_BINARY, 0);
29084
- if( fd>=0 ){
29085
-#ifdef FD_CLOEXEC
29086
- osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
29087
-#endif
29088
- OSTRACE(("OPENDIR %-3d %s\n", fd, zDirname));
29089
- }
29090
- }
29091
- *pFd = fd;
29092
- return (fd>=0?SQLITE_OK:unixLogError(SQLITE_CANTOPEN_BKPT, "open", zDirname));
29093
-}
29094
-
2909529245
 /*
2909629246
 ** Return the name of a directory in which to put temporary files.
2909729247
 ** If no suitable temporary file directory can be found, return NULL.
2909829248
 */
2909929249
 static const char *unixTempFileDir(void){
@@ -29204,11 +29354,11 @@
2920429354
 ** ignored and -1 is returned. The caller will try to open a new file
2920529355
 ** descriptor on the same path, fail, and return an error to SQLite.
2920629356
 **
2920729357
 ** Even if a subsequent open() call does succeed, the consequences of
2920829358
 ** not searching for a resusable file descriptor are not dire. */
29209
- if( 0==stat(zPath, &sStat) ){
29359
+ if( 0==osStat(zPath, &sStat) ){
2921029360
 unixInodeInfo *pInode;
2921129361
2921229362
 unixEnterMutex();
2921329363
 pInode = inodeList;
2921429364
 while( pInode && (pInode->fileId.dev!=sStat.st_dev
@@ -29280,11 +29430,11 @@
2928029430
 while( nDb>0 && zPath[nDb]!='-' ) nDb--;
2928129431
 if( nDb==0 ) return SQLITE_OK;
2928229432
 memcpy(zDb, zPath, nDb);
2928329433
 zDb[nDb] = '\0';
2928429434
29285
- if( 0==stat(zDb, &sStat) ){
29435
+ if( 0==osStat(zDb, &sStat) ){
2928629436
 *pMode = sStat.st_mode & 0777;
2928729437
 }else{
2928829438
 rc = SQLITE_IOERR_FSTAT;
2928929439
 }
2929029440
 }else if( flags & SQLITE_OPEN_DELETEONCLOSE ){
@@ -29322,11 +29472,10 @@
2932229472
 int flags, /* Input flags to control the opening */
2932329473
 int *pOutFlags /* Output flags returned to SQLite core */
2932429474
 ){
2932529475
 unixFile *p = (unixFile *)pFile;
2932629476
 int fd = -1; /* File descriptor returned by open() */
29327
- int dirfd = -1; /* Directory file descriptor */
2932829477
 int openFlags = 0; /* Flags to pass to open() */
2932929478
 int eType = flags&0xFFFFFF00; /* Type of file to open */
2933029479
 int noLock; /* True to omit locking primitives */
2933129480
 int rc = SQLITE_OK; /* Function Return Code */
2933229481
@@ -29341,11 +29490,11 @@
2934129490
2934229491
 /* If creating a master or main-file journal, this function will open
2934329492
 ** a file-descriptor on the directory too. The first time unixSync()
2934429493
 ** is called the directory file descriptor will be fsync()ed and close()d.
2934529494
 */
29346
- int isOpenDirectory = (isCreate && (
29495
+ int syncDir = (isCreate && (
2934729496
 eType==SQLITE_OPEN_MASTER_JOURNAL
2934829497
 || eType==SQLITE_OPEN_MAIN_JOURNAL
2934929498
 || eType==SQLITE_OPEN_WAL
2935029499
 ));
2935129500
@@ -29395,11 +29544,11 @@
2939529544
 }
2939629545
 }
2939729546
 p->pUnused = pUnused;
2939829547
 }else if( !zName ){
2939929548
 /* If zName is NULL, the upper layer is requesting a temp file. */
29400
- assert(isDelete && !isOpenDirectory);
29549
+ assert(isDelete && !syncDir);
2940129550
 rc = unixGetTempname(MAX_PATHNAME+1, zTmpname);
2940229551
 if( rc!=SQLITE_OK ){
2940329552
 return rc;
2940429553
 }
2940529554
 zName = zTmpname;
@@ -29451,32 +29600,19 @@
2945129600
2945229601
 if( isDelete ){
2945329602
 #if OS_VXWORKS
2945429603
 zPath = zName;
2945529604
 #else
29456
- unlink(zName);
29605
+ osUnlink(zName);
2945729606
 #endif
2945829607
 }
2945929608
 #if SQLITE_ENABLE_LOCKING_STYLE
2946029609
 else{
2946129610
 p->openFlags = openFlags;
2946229611
 }
2946329612
 #endif
2946429613
29465
- if( isOpenDirectory ){
29466
- rc = openDirectory(zPath, &dirfd);
29467
- if( rc!=SQLITE_OK ){
29468
- /* It is safe to close fd at this point, because it is guaranteed not
29469
- ** to be open on a database file. If it were open on a database file,
29470
- ** it would not be safe to close as this would release any locks held
29471
- ** on the file by this process. */
29472
- assert( eType!=SQLITE_OPEN_MAIN_DB );
29473
- robust_close(p, fd, __LINE__);
29474
- goto open_finished;
29475
- }
29476
- }
29477
-
2947829614
 #ifdef FD_CLOEXEC
2947929615
 osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
2948029616
 #endif
2948129617
2948229618
 noLock = eType!=SQLITE_OPEN_MAIN_DB;
@@ -29484,11 +29620,10 @@
2948429620
2948529621
 #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
2948629622
 struct statfs fsInfo;
2948729623
 if( fstatfs(fd, &fsInfo) == -1 ){
2948829624
 ((unixFile*)pFile)->lastErrno = errno;
29489
- if( dirfd>=0 ) robust_close(p, dirfd, __LINE__);
2949029625
 robust_close(p, fd, __LINE__);
2949129626
 return SQLITE_IOERR_ACCESS;
2949229627
 }
2949329628
 if (0 == strncmp("msdos", fsInfo.f_fstypename, 5)) {
2949429629
 ((unixFile*)pFile)->fsFlags |= SQLITE_FSFLAGS_IS_MSDOS;
@@ -29516,21 +29651,18 @@
2951629651
 ** then the call to close() will cancel those locks. In practice,
2951729652
 ** we're assuming that statfs() doesn't fail very often. At least
2951829653
 ** not while other file descriptors opened by the same process on
2951929654
 ** the same file are working. */
2952029655
 p->lastErrno = errno;
29521
- if( dirfd>=0 ){
29522
- robust_close(p, dirfd, __LINE__);
29523
- }
2952429656
 robust_close(p, fd, __LINE__);
2952529657
 rc = SQLITE_IOERR_ACCESS;
2952629658
 goto open_finished;
2952729659
 }
2952829660
 useProxy = !(fsInfo.f_flags&MNT_LOCAL);
2952929661
 }
2953029662
 if( useProxy ){
29531
- rc = fillInUnixFile(pVfs, fd, dirfd, pFile, zPath, noLock,
29663
+ rc = fillInUnixFile(pVfs, fd, syncDir, pFile, zPath, noLock,
2953229664
 isDelete, isReadonly);
2953329665
 if( rc==SQLITE_OK ){
2953429666
 rc = proxyTransformUnixFile((unixFile*)pFile, ":auto:");
2953529667
 if( rc!=SQLITE_OK ){
2953629668
 /* Use unixClose to clean up the resources added in fillInUnixFile
@@ -29544,11 +29676,11 @@
2954429676
 goto open_finished;
2954529677
 }
2954629678
 }
2954729679
 #endif
2954829680
29549
- rc = fillInUnixFile(pVfs, fd, dirfd, pFile, zPath, noLock,
29681
+ rc = fillInUnixFile(pVfs, fd, syncDir, pFile, zPath, noLock,
2955029682
 isDelete, isReadonly);
2955129683
 open_finished:
2955229684
 if( rc!=SQLITE_OK ){
2955329685
 sqlite3_free(p->pUnused);
2955429686
 }
@@ -29566,17 +29698,17 @@
2956629698
 int dirSync /* If true, fsync() directory after deleting file */
2956729699
 ){
2956829700
 int rc = SQLITE_OK;
2956929701
 UNUSED_PARAMETER(NotUsed);
2957029702
 SimulateIOError(return SQLITE_IOERR_DELETE);
29571
- if( unlink(zPath)==(-1) && errno!=ENOENT ){
29703
+ if( osUnlink(zPath)==(-1) && errno!=ENOENT ){
2957229704
 return unixLogError(SQLITE_IOERR_DELETE, "unlink", zPath);
2957329705
 }
2957429706
 #ifndef SQLITE_DISABLE_DIRSYNC
2957529707
 if( dirSync ){
2957629708
 int fd;
29577
- rc = openDirectory(zPath, &fd);
29709
+ rc = osOpenDirectory(zPath, &fd);
2957829710
 if( rc==SQLITE_OK ){
2957929711
 #if OS_VXWORKS
2958029712
 if( fsync(fd)==-1 )
2958129713
 #else
2958229714
 if( fsync(fd) )
@@ -29625,11 +29757,11 @@
2962529757
 assert(!"Invalid flags argument");
2962629758
 }
2962729759
 *pResOut = (osAccess(zPath, amode)==0);
2962829760
 if( flags==SQLITE_ACCESS_EXISTS && *pResOut ){
2962929761
 struct stat buf;
29630
- if( 0==stat(zPath, &buf) && buf.st_size==0 ){
29762
+ if( 0==osStat(zPath, &buf) && buf.st_size==0 ){
2963129763
 *pResOut = 0;
2963229764
 }
2963329765
 }
2963429766
 return SQLITE_OK;
2963529767
 }
@@ -30144,11 +30276,10 @@
3014430276
 const char *path, /* path for the new unixFile */
3014530277
 unixFile **ppFile, /* unixFile created and returned by ref */
3014630278
 int islockfile /* if non zero missing dirs will be created */
3014730279
 ) {
3014830280
 int fd = -1;
30149
- int dirfd = -1;
3015030281
 unixFile *pNew;
3015130282
 int rc = SQLITE_OK;
3015230283
 int openFlags = O_RDWR | O_CREAT;
3015330284
 sqlite3_vfs dummyVfs;
3015430285
 int terrno = 0;
@@ -30209,11 +30340,11 @@
3020930340
 dummyVfs.zName = "dummy";
3021030341
 pUnused->fd = fd;
3021130342
 pUnused->flags = openFlags;
3021230343
 pNew->pUnused = pUnused;
3021330344
30214
- rc = fillInUnixFile(&dummyVfs, fd, dirfd, (sqlite3_file*)pNew, path, 0, 0, 0);
30345
+ rc = fillInUnixFile(&dummyVfs, fd, 0, (sqlite3_file*)pNew, path, 0, 0, 0);
3021530346
 if( rc==SQLITE_OK ){
3021630347
 *ppFile = pNew;
3021730348
 return SQLITE_OK;
3021830349
 }
3021930350
 end_create_proxy:
@@ -30323,11 +30454,11 @@
3032330454
 conchFile->openFlags = O_RDWR | O_CREAT;
3032430455
3032530456
 end_breaklock:
3032630457
 if( rc ){
3032730458
 if( fd>=0 ){
30328
- unlink(tPath);
30459
+ osUnlink(tPath);
3032930460
 robust_close(pFile, fd, __LINE__);
3033030461
 }
3033130462
 fprintf(stderr, "failed to break stale lock on %s, %s\n", cPath, errmsg);
3033230463
 }
3033330464
 return rc;
@@ -31146,11 +31277,11 @@
3114631277
 };
3114731278
 unsigned int i; /* Loop counter */
3114831279
3114931280
 /* Double-check that the aSyscall[] array has been constructed
3115031281
 ** correctly. See ticket [bb3a86e890c8e96ab] */
31151
- assert( ArraySize(aSyscall)==16 );
31282
+ assert( ArraySize(aSyscall)==18 );
3115231283
3115331284
 /* Register all VFSes defined in the aVfs[] array */
3115431285
 for(i=0; i<(sizeof(aVfs)/sizeof(sqlite3_vfs)); i++){
3115531286
 sqlite3_vfs_register(&aVfs[i], i==0);
3115631287
 }
@@ -31263,14 +31394,17 @@
3126331394
 #ifdef MEMORY_DEBUG
3126431395
 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
3126531396
 #endif
3126631397
3126731398
 #ifdef SQLITE_DEBUG
31268
-SQLITE_PRIVATE int sqlite3OSTrace = 0;
31269
-#define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
31399
+# ifndef SQLITE_DEBUG_OS_TRACE
31400
+# define SQLITE_DEBUG_OS_TRACE 0
31401
+# endif
31402
+ int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
31403
+# define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
3127031404
 #else
31271
-#define OSTRACE(X)
31405
+# define OSTRACE(X)
3127231406
 #endif
3127331407
3127431408
 /*
3127531409
 ** Macros for performance tracing. Normally turned off. Only works
3127631410
 ** on i486 hardware.
@@ -31478,12 +31612,13 @@
3147831612
 typedef struct winFile winFile;
3147931613
 struct winFile {
3148031614
 const sqlite3_io_methods *pMethod; /*** Must be first ***/
3148131615
 sqlite3_vfs *pVfs; /* The VFS used to open this file */
3148231616
 HANDLE h; /* Handle for accessing the file */
31483
- unsigned char locktype; /* Type of lock currently held on this file */
31617
+ u8 locktype; /* Type of lock currently held on this file */
3148431618
 short sharedLockByte; /* Randomly chosen byte used as a shared lock */
31619
+ u8 bPersistWal; /* True to persist WAL files */
3148531620
 DWORD lastErrno; /* The Windows errno from the last I/O error */
3148631621
 DWORD sectorSize; /* Sector size of the device file is on */
3148731622
 winShm *pShm; /* Instance of shared memory on this file */
3148831623
 const char *zPath; /* Full pathname of this file */
3148931624
 int szChunk; /* Chunk size configured by FCNTL_CHUNK_SIZE */
@@ -31777,10 +31912,58 @@
3177731912
 iLine, iErrno, zFunc, zPath, zMsg
3177831913
 );
3177931914
3178031915
 return errcode;
3178131916
 }
31917
+
31918
+/*
31919
+** The number of times that a ReadFile(), WriteFile(), and DeleteFile()
31920
+** will be retried following a locking error - probably caused by
31921
+** antivirus software. Also the initial delay before the first retry.
31922
+** The delay increases linearly with each retry.
31923
+*/
31924
+#ifndef SQLITE_WIN32_IOERR_RETRY
31925
+# define SQLITE_WIN32_IOERR_RETRY 10
31926
+#endif
31927
+#ifndef SQLITE_WIN32_IOERR_RETRY_DELAY
31928
+# define SQLITE_WIN32_IOERR_RETRY_DELAY 25
31929
+#endif
31930
+static int win32IoerrRetry = SQLITE_WIN32_IOERR_RETRY;
31931
+static int win32IoerrRetryDelay = SQLITE_WIN32_IOERR_RETRY_DELAY;
31932
+
31933
+/*
31934
+** If a ReadFile() or WriteFile() error occurs, invoke this routine
31935
+** to see if it should be retried. Return TRUE to retry. Return FALSE
31936
+** to give up with an error.
31937
+*/
31938
+static int retryIoerr(int *pnRetry){
31939
+ DWORD e;
31940
+ if( *pnRetry>=win32IoerrRetry ){
31941
+ return 0;
31942
+ }
31943
+ e = GetLastError();
31944
+ if( e==ERROR_ACCESS_DENIED ||
31945
+ e==ERROR_LOCK_VIOLATION ||
31946
+ e==ERROR_SHARING_VIOLATION ){
31947
+ Sleep(win32IoerrRetryDelay*(1+*pnRetry));
31948
+ ++*pnRetry;
31949
+ return 1;
31950
+ }
31951
+ return 0;
31952
+}
31953
+
31954
+/*
31955
+** Log a I/O error retry episode.
31956
+*/
31957
+static void logIoerr(int nRetry){
31958
+ if( nRetry ){
31959
+ sqlite3_log(SQLITE_IOERR,
31960
+ "delayed %dms for lock/sharing conflict",
31961
+ win32IoerrRetryDelay*nRetry*(nRetry+1)/2
31962
+ );
31963
+ }
31964
+}
3178231965
3178331966
 #if SQLITE_OS_WINCE
3178431967
 /*************************************************************************
3178531968
 ** This section contains code for WinCE only.
3178631969
 */
@@ -32195,22 +32378,25 @@
3219532378
 int amt, /* Number of bytes to read */
3219632379
 sqlite3_int64 offset /* Begin reading at this offset */
3219732380
 ){
3219832381
 winFile *pFile = (winFile*)id; /* file handle */
3219932382
 DWORD nRead; /* Number of bytes actually read from file */
32383
+ int nRetry = 0; /* Number of retrys */
3220032384
3220132385
 assert( id!=0 );
3220232386
 SimulateIOError(return SQLITE_IOERR_READ);
3220332387
 OSTRACE(("READ %d lock=%d\n", pFile->h, pFile->locktype));
3220432388
3220532389
 if( seekWinFile(pFile, offset) ){
3220632390
 return SQLITE_FULL;
3220732391
 }
32208
- if( !ReadFile(pFile->h, pBuf, amt, &nRead, 0) ){
32392
+ while( !ReadFile(pFile->h, pBuf, amt, &nRead, 0) ){
32393
+ if( retryIoerr(&nRetry) ) continue;
3220932394
 pFile->lastErrno = GetLastError();
3221032395
 return winLogError(SQLITE_IOERR_READ, "winRead", pFile->zPath);
3221132396
 }
32397
+ logIoerr(nRetry);
3221232398
 if( nRead<(DWORD)amt ){
3221332399
 /* Unread parts of the buffer must be zero-filled */
3221432400
 memset(&((char*)pBuf)[nRead], 0, amt-nRead);
3221532401
 return SQLITE_IOERR_SHORT_READ;
3221632402
 }
@@ -32228,10 +32414,11 @@
3222832414
 int amt, /* Number of bytes to write */
3222932415
 sqlite3_int64 offset /* Offset into the file to begin writing at */
3223032416
 ){
3223132417
 int rc; /* True if error has occured, else false */
3223232418
 winFile *pFile = (winFile*)id; /* File handle */
32419
+ int nRetry = 0; /* Number of retries */
3223332420
3223432421
 assert( amt>0 );
3223532422
 assert( pFile );
3223632423
 SimulateIOError(return SQLITE_IOERR_WRITE);
3223732424
 SimulateDiskfullError(return SQLITE_FULL);
@@ -32242,11 +32429,16 @@
3224232429
 if( rc==0 ){
3224332430
 u8 *aRem = (u8 *)pBuf; /* Data yet to be written */
3224432431
 int nRem = amt; /* Number of bytes yet to be written */
3224532432
 DWORD nWrite; /* Bytes written by each WriteFile() call */
3224632433
32247
- while( nRem>0 && WriteFile(pFile->h, aRem, nRem, &nWrite, 0) && nWrite>0 ){
32434
+ while( nRem>0 ){
32435
+ if( !WriteFile(pFile->h, aRem, nRem, &nWrite, 0) ){
32436
+ if( retryIoerr(&nRetry) ) continue;
32437
+ break;
32438
+ }
32439
+ if( nWrite<=0 ) break;
3224832440
 aRem += nWrite;
3224932441
 nRem -= nWrite;
3225032442
 }
3225132443
 if( nRem>0 ){
3225232444
 pFile->lastErrno = GetLastError();
@@ -32253,14 +32445,17 @@
3225332445
 rc = 1;
3225432446
 }
3225532447
 }
3225632448
3225732449
 if( rc ){
32258
- if( pFile->lastErrno==ERROR_HANDLE_DISK_FULL ){
32450
+ if( ( pFile->lastErrno==ERROR_HANDLE_DISK_FULL )
32451
+ || ( pFile->lastErrno==ERROR_DISK_FULL )){
3225932452
 return SQLITE_FULL;
3226032453
 }
3226132454
 return winLogError(SQLITE_IOERR_WRITE, "winWrite", pFile->zPath);
32455
+ }else{
32456
+ logIoerr(nRetry);
3226232457
 }
3226332458
 return SQLITE_OK;
3226432459
 }
3226532460
3226632461
 /*
@@ -32650,32 +32845,56 @@
3265032845
3265132846
 /*
3265232847
 ** Control and query of the open file handle.
3265332848
 */
3265432849
 static int winFileControl(sqlite3_file *id, int op, void *pArg){
32850
+ winFile *pFile = (winFile*)id;
3265532851
 switch( op ){
3265632852
 case SQLITE_FCNTL_LOCKSTATE: {
32657
- *(int*)pArg = ((winFile*)id)->locktype;
32853
+ *(int*)pArg = pFile->locktype;
3265832854
 return SQLITE_OK;
3265932855
 }
3266032856
 case SQLITE_LAST_ERRNO: {
32661
- *(int*)pArg = (int)((winFile*)id)->lastErrno;
32857
+ *(int*)pArg = (int)pFile->lastErrno;
3266232858
 return SQLITE_OK;
3266332859
 }
3266432860
 case SQLITE_FCNTL_CHUNK_SIZE: {
32665
- ((winFile*)id)->szChunk = *(int *)pArg;
32861
+ pFile->szChunk = *(int *)pArg;
3266632862
 return SQLITE_OK;
3266732863
 }
3266832864
 case SQLITE_FCNTL_SIZE_HINT: {
3266932865
 sqlite3_int64 sz = *(sqlite3_int64*)pArg;
3267032866
 SimulateIOErrorBenign(1);
3267132867
 winTruncate(id, sz);
3267232868
 SimulateIOErrorBenign(0);
3267332869
 return SQLITE_OK;
32870
+ }
32871
+ case SQLITE_FCNTL_PERSIST_WAL: {
32872
+ int bPersist = *(int*)pArg;
32873
+ if( bPersist<0 ){
32874
+ *(int*)pArg = pFile->bPersistWal;
32875
+ }else{
32876
+ pFile->bPersistWal = bPersist!=0;
32877
+ }
32878
+ return SQLITE_OK;
3267432879
 }
3267532880
 case SQLITE_FCNTL_SYNC_OMITTED: {
3267632881
 return SQLITE_OK;
32882
+ }
32883
+ case SQLITE_FCNTL_WIN32_AV_RETRY: {
32884
+ int *a = (int*)pArg;
32885
+ if( a[0]>0 ){
32886
+ win32IoerrRetry = a[0];
32887
+ }else{
32888
+ a[0] = win32IoerrRetry;
32889
+ }
32890
+ if( a[1]>0 ){
32891
+ win32IoerrRetryDelay = a[1];
32892
+ }else{
32893
+ a[1] = win32IoerrRetryDelay;
32894
+ }
32895
+ return SQLITE_OK;
3267732896
 }
3267832897
 }
3267932898
 return SQLITE_NOTFOUND;
3268032899
 }
3268132900
@@ -33481,10 +33700,11 @@
3348133700
 int isTemp = 0;
3348233701
 #endif
3348333702
 winFile *pFile = (winFile*)id;
3348433703
 void *zConverted; /* Filename in OS encoding */
3348533704
 const char *zUtf8Name = zName; /* Filename in UTF-8 encoding */
33705
+ int cnt = 0;
3348633706
3348733707
 /* If argument zPath is a NULL pointer, this function is required to open
3348833708
 ** a temporary file. Use this buffer to store the file name in.
3348933709
 */
3349033710
 char zTmpname[MAX_PATH+1]; /* Buffer used to create temp filename */
@@ -33600,34 +33820,34 @@
3360033820
 #if SQLITE_OS_WINCE
3360133821
 dwFlagsAndAttributes |= FILE_FLAG_RANDOM_ACCESS;
3360233822
 #endif
3360333823
3360433824
 if( isNT() ){
33605
- h = CreateFileW((WCHAR*)zConverted,
33606
- dwDesiredAccess,
33607
- dwShareMode,
33608
- NULL,
33609
- dwCreationDisposition,
33610
- dwFlagsAndAttributes,
33611
- NULL
33612
- );
33825
+ while( (h = CreateFileW((WCHAR*)zConverted,
33826
+ dwDesiredAccess,
33827
+ dwShareMode, NULL,
33828
+ dwCreationDisposition,
33829
+ dwFlagsAndAttributes,
33830
+ NULL))==INVALID_HANDLE_VALUE &&
33831
+ retryIoerr(&cnt) ){}
3361333832
 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed.
3361433833
 ** Since the ASCII version of these Windows API do not exist for WINCE,
3361533834
 ** it's important to not reference them for WINCE builds.
3361633835
 */
3361733836
 #if SQLITE_OS_WINCE==0
3361833837
 }else{
33619
- h = CreateFileA((char*)zConverted,
33620
- dwDesiredAccess,
33621
- dwShareMode,
33622
- NULL,
33623
- dwCreationDisposition,
33624
- dwFlagsAndAttributes,
33625
- NULL
33626
- );
33838
+ while( (h = CreateFileA((char*)zConverted,
33839
+ dwDesiredAccess,
33840
+ dwShareMode, NULL,
33841
+ dwCreationDisposition,
33842
+ dwFlagsAndAttributes,
33843
+ NULL))==INVALID_HANDLE_VALUE &&
33844
+ retryIoerr(&cnt) ){}
3362733845
 #endif
3362833846
 }
33847
+
33848
+ logIoerr(cnt);
3362933849
3363033850
 OSTRACE(("OPEN %d %s 0x%lx %s\n",
3363133851
 h, zName, dwDesiredAccess,
3363233852
 h==INVALID_HANDLE_VALUE ? "failed" : "ok"));
3363333853
@@ -33690,19 +33910,17 @@
3369033910
 ** file open, we will be unable to delete it. To work around this
3369133911
 ** problem, we delay 100 milliseconds and try to delete again. Up
3369233912
 ** to MX_DELETION_ATTEMPTs deletion attempts are run before giving
3369333913
 ** up and returning an error.
3369433914
 */
33695
-#define MX_DELETION_ATTEMPTS 5
3369633915
 static int winDelete(
3369733916
 sqlite3_vfs *pVfs, /* Not used on win32 */
3369833917
 const char *zFilename, /* Name of file to delete */
3369933918
 int syncDir /* Not used on win32 */
3370033919
 ){
3370133920
 int cnt = 0;
33702
- DWORD rc;
33703
- DWORD error = 0;
33921
+ int rc;
3370433922
 void *zConverted;
3370533923
 UNUSED_PARAMETER(pVfs);
3370633924
 UNUSED_PARAMETER(syncDir);
3370733925
3370833926
 SimulateIOError(return SQLITE_IOERR_DELETE);
@@ -33709,38 +33927,34 @@
3370933927
 zConverted = convertUtf8Filename(zFilename);
3371033928
 if( zConverted==0 ){
3371133929
 return SQLITE_NOMEM;
3371233930
 }
3371333931
 if( isNT() ){
33714
- do{
33715
- DeleteFileW(zConverted);
33716
- }while( ( ((rc = GetFileAttributesW(zConverted)) != INVALID_FILE_ATTRIBUTES)
33717
- || ((error = GetLastError()) == ERROR_ACCESS_DENIED))
33718
- && (++cnt < MX_DELETION_ATTEMPTS)
33719
- && (Sleep(100), 1) );
33932
+ rc = 1;
33933
+ while( GetFileAttributesW(zConverted)!=INVALID_FILE_ATTRIBUTES &&
33934
+ (rc = DeleteFileW(zConverted))==0 && retryIoerr(&cnt) ){}
33935
+ rc = rc ? SQLITE_OK : SQLITE_ERROR;
3372033936
 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed.
3372133937
 ** Since the ASCII version of these Windows API do not exist for WINCE,
3372233938
 ** it's important to not reference them for WINCE builds.
3372333939
 */
3372433940
 #if SQLITE_OS_WINCE==0
3372533941
 }else{
33726
- do{
33727
- DeleteFileA(zConverted);
33728
- }while( ( ((rc = GetFileAttributesA(zConverted)) != INVALID_FILE_ATTRIBUTES)
33729
- || ((error = GetLastError()) == ERROR_ACCESS_DENIED))
33730
- && (++cnt < MX_DELETION_ATTEMPTS)
33731
- && (Sleep(100), 1) );
33942
+ rc = 1;
33943
+ while( GetFileAttributesA(zConverted)!=INVALID_FILE_ATTRIBUTES &&
33944
+ (rc = DeleteFileA(zConverted))==0 && retryIoerr(&cnt) ){}
33945
+ rc = rc ? SQLITE_OK : SQLITE_ERROR;
3373233946
 #endif
3373333947
 }
33948
+ if( rc ){
33949
+ rc = winLogError(SQLITE_IOERR_DELETE, "winDelete", zFilename);
33950
+ }else{
33951
+ logIoerr(cnt);
33952
+ }
3373433953
 free(zConverted);
33735
- OSTRACE(("DELETE \"%s\" %s\n", zFilename,
33736
- ( (rc==INVALID_FILE_ATTRIBUTES) && (error==ERROR_FILE_NOT_FOUND)) ?
33737
- "ok" : "failed" ));
33738
-
33739
- return ( (rc == INVALID_FILE_ATTRIBUTES)
33740
- && (error == ERROR_FILE_NOT_FOUND)) ? SQLITE_OK :
33741
- winLogError(SQLITE_IOERR_DELETE, "winDelete", zFilename);
33954
+ OSTRACE(("DELETE \"%s\" %s\n", zFilename, (rc ? "failed" : "ok" )));
33955
+ return rc;
3374233956
 }
3374333957
3374433958
 /*
3374533959
 ** Check the existance and status of a file.
3374633960
 */
@@ -33759,15 +33973,17 @@
3375933973
 zConverted = convertUtf8Filename(zFilename);
3376033974
 if( zConverted==0 ){
3376133975
 return SQLITE_NOMEM;
3376233976
 }
3376333977
 if( isNT() ){
33978
+ int cnt = 0;
3376433979
 WIN32_FILE_ATTRIBUTE_DATA sAttrData;
3376533980
 memset(&sAttrData, 0, sizeof(sAttrData));
33766
- if( GetFileAttributesExW((WCHAR*)zConverted,
33981
+ while( !(rc = GetFileAttributesExW((WCHAR*)zConverted,
3376733982
 GetFileExInfoStandard,
33768
- &sAttrData) ){
33983
+ &sAttrData)) && retryIoerr(&cnt) ){}
33984
+ if( rc ){
3376933985
 /* For an SQLITE_ACCESS_EXISTS query, treat a zero-length file
3377033986
 ** as if it does not exist.
3377133987
 */
3377233988
 if( flags==SQLITE_ACCESS_EXISTS
3377333989
 && sAttrData.nFileSizeHigh==0
@@ -33775,10 +33991,11 @@
3377533991
 attr = INVALID_FILE_ATTRIBUTES;
3377633992
 }else{
3377733993
 attr = sAttrData.dwFileAttributes;
3377833994
 }
3377933995
 }else{
33996
+ logIoerr(cnt);
3378033997
 if( GetLastError()!=ERROR_FILE_NOT_FOUND ){
3378133998
 winLogError(SQLITE_IOERR_ACCESS, "winAccess", zFilename);
3378233999
 free(zConverted);
3378334000
 return SQLITE_IOERR_ACCESS;
3378434001
 }else{
@@ -33799,11 +34016,12 @@
3379934016
 case SQLITE_ACCESS_READ:
3380034017
 case SQLITE_ACCESS_EXISTS:
3380134018
 rc = attr!=INVALID_FILE_ATTRIBUTES;
3380234019
 break;
3380334020
 case SQLITE_ACCESS_READWRITE:
33804
- rc = (attr & FILE_ATTRIBUTE_READONLY)==0;
34021
+ rc = attr!=INVALID_FILE_ATTRIBUTES &&
34022
+ (attr & FILE_ATTRIBUTE_READONLY)==0;
3380534023
 break;
3380634024
 default:
3380734025
 assert(!"Invalid flags argument");
3380834026
 }
3380934027
 *pResOut = rc;
@@ -35805,11 +36023,11 @@
3580536023
 memset(pCache, 0, sz);
3580636024
 if( separateCache ){
3580736025
 pGroup = (PGroup*)&pCache[1];
3580836026
 pGroup->mxPinned = 10;
3580936027
 }else{
35810
- pGroup = &pcache1_g.grp;
36028
+ pGroup = &pcache1.grp;
3581136029
 }
3581236030
 pCache->pGroup = pGroup;
3581336031
 pCache->szPage = szPage;
3581436032
 pCache->bPurgeable = (bPurgeable ? 1 : 0);
3581536033
 if( bPurgeable ){
@@ -45383,17 +45601,19 @@
4538345601
 **
4538445602
 ** The EXCLUSIVE lock is not released before returning.
4538545603
 */
4538645604
 rc = sqlite3OsLock(pWal->pDbFd, SQLITE_LOCK_EXCLUSIVE);
4538745605
 if( rc==SQLITE_OK ){
45606
+ int bPersistWal = -1;
4538845607
 if( pWal->exclusiveMode==WAL_NORMAL_MODE ){
4538945608
 pWal->exclusiveMode = WAL_EXCLUSIVE_MODE;
4539045609
 }
4539145610
 rc = sqlite3WalCheckpoint(
4539245611
 pWal, SQLITE_CHECKPOINT_PASSIVE, 0, 0, sync_flags, nBuf, zBuf, 0, 0
4539345612
 );
45394
- if( rc==SQLITE_OK ){
45613
+ sqlite3OsFileControl(pWal->pDbFd, SQLITE_FCNTL_PERSIST_WAL, &bPersistWal);
45614
+ if( rc==SQLITE_OK && bPersistWal!=1 ){
4539545615
 isDelete = 1;
4539645616
 }
4539745617
 }
4539845618
4539945619
 walIndexClose(pWal, isDelete);
@@ -48324,10 +48544,12 @@
4832448544
 **
4832548545
 ** This routine works only for pages that do not contain overflow cells.
4832648546
 */
4832748547
 #define findCell(P,I) \
4832848548
 ((P)->aData + ((P)->maskPage & get2byte(&(P)->aData[(P)->cellOffset+2*(I)])))
48549
+#define findCellv2(D,M,O,I) (D+(M&get2byte(D+(O+2*(I)))))
48550
+
4832948551
4833048552
 /*
4833148553
 ** This a more complex version of findCell() that works for
4833248554
 ** pages that do contain overflow cells.
4833348555
 */
@@ -51918,11 +52140,11 @@
5191852140
 assert( pCur->apPage[pCur->iPage]->nCell==0 );
5191952141
 return SQLITE_OK;
5192052142
 }
5192152143
 assert( pCur->apPage[0]->intKey || pIdxKey );
5192252144
 for(;;){
51923
- int lwr, upr;
52145
+ int lwr, upr, idx;
5192452146
 Pgno chldPg;
5192552147
 MemPage *pPage = pCur->apPage[pCur->iPage];
5192652148
 int c;
5192752149
5192852150
 /* pPage->nCell must be greater than zero. If this is the root-page
@@ -51934,18 +52156,18 @@
5193452156
 assert( pPage->nCell>0 );
5193552157
 assert( pPage->intKey==(pIdxKey==0) );
5193652158
 lwr = 0;
5193752159
 upr = pPage->nCell-1;
5193852160
 if( biasRight ){
51939
- pCur->aiIdx[pCur->iPage] = (u16)upr;
52161
+ pCur->aiIdx[pCur->iPage] = (u16)(idx = upr);
5194052162
 }else{
51941
- pCur->aiIdx[pCur->iPage] = (u16)((upr+lwr)/2);
52163
+ pCur->aiIdx[pCur->iPage] = (u16)(idx = (upr+lwr)/2);
5194252164
 }
5194352165
 for(;;){
51944
- int idx = pCur->aiIdx[pCur->iPage]; /* Index of current cell in pPage */
5194552166
 u8 *pCell; /* Pointer to current cell in pPage */
5194652167
52168
+ assert( idx==pCur->aiIdx[pCur->iPage] );
5194752169
 pCur->info.nSize = 0;
5194852170
 pCell = findCell(pPage, idx) + pPage->childPtrSize;
5194952171
 if( pPage->intKey ){
5195052172
 i64 nCellKey;
5195152173
 if( pPage->hasData ){
@@ -52024,11 +52246,11 @@
5202452246
 upr = idx-1;
5202552247
 }
5202652248
 if( lwr>upr ){
5202752249
 break;
5202852250
 }
52029
- pCur->aiIdx[pCur->iPage] = (u16)((lwr+upr)/2);
52251
+ pCur->aiIdx[pCur->iPage] = (u16)(idx = (lwr+upr)/2);
5203052252
 }
5203152253
 assert( lwr==upr+1 );
5203252254
 assert( pPage->isInit );
5203352255
 if( pPage->leaf ){
5203452256
 chldPg = 0;
@@ -52886,13 +53108,13 @@
5288653108
 if( rc ){
5288753109
 *pRC = rc;
5288853110
 return;
5288953111
 }
5289053112
 endPtr = &data[pPage->cellOffset + 2*pPage->nCell - 2];
53113
+ assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */
5289153114
 while( ptr<endPtr ){
52892
- ptr[0] = ptr[2];
52893
- ptr[1] = ptr[3];
53115
+ *(u16*)ptr = *(u16*)&ptr[2];
5289453116
 ptr += 2;
5289553117
 }
5289653118
 pPage->nCell--;
5289753119
 put2byte(&data[hdr+3], pPage->nCell);
5289853120
 pPage->nFree += 2;
@@ -52929,10 +53151,11 @@
5292953151
 int end; /* First byte past the last cell pointer in data[] */
5293053152
 int ins; /* Index in data[] where new cell pointer is inserted */
5293153153
 int cellOffset; /* Address of first cell pointer in data[] */
5293253154
 u8 *data; /* The content of the whole page */
5293353155
 u8 *ptr; /* Used for moving information around in data[] */
53156
+ u8 *endPtr; /* End of the loop */
5293453157
5293553158
 int nSkip = (iChild ? 4 : 0);
5293653159
5293753160
 if( *pRC ) return;
5293853161
@@ -52979,13 +53202,16 @@
5297953202
 pPage->nFree -= (u16)(2 + sz);
5298053203
 memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip);
5298153204
 if( iChild ){
5298253205
 put4byte(&data[idx], iChild);
5298353206
 }
52984
- for(j=end, ptr=&data[j]; j>ins; j-=2, ptr-=2){
52985
- ptr[0] = ptr[-2];
52986
- ptr[1] = ptr[-1];
53207
+ ptr = &data[end];
53208
+ endPtr = &data[ins];
53209
+ assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */
53210
+ while( ptr>endPtr ){
53211
+ *(u16*)ptr = *(u16*)&ptr[-2];
53212
+ ptr -= 2;
5298753213
 }
5298853214
 put2byte(&data[ins], idx);
5298953215
 put2byte(&data[pPage->hdrOffset+3], pPage->nCell);
5299053216
 #ifndef SQLITE_OMIT_AUTOVACUUM
5299153217
 if( pPage->pBt->autoVacuum ){
@@ -53026,14 +53252,15 @@
5302653252
 assert( get2byteNotZero(&data[hdr+5])==nUsable );
5302753253
5302853254
 pCellptr = &data[pPage->cellOffset + nCell*2];
5302953255
 cellbody = nUsable;
5303053256
 for(i=nCell-1; i>=0; i--){
53257
+ u16 sz = aSize[i];
5303153258
 pCellptr -= 2;
53032
- cellbody -= aSize[i];
53259
+ cellbody -= sz;
5303353260
 put2byte(pCellptr, cellbody);
53034
- memcpy(&data[cellbody], apCell[i], aSize[i]);
53261
+ memcpy(&data[cellbody], apCell[i], sz);
5303553262
 }
5303653263
 put2byte(&data[hdr+3], nCell);
5303753264
 put2byte(&data[hdr+5], cellbody);
5303853265
 pPage->nFree -= (nCell*2 + nUsable - cellbody);
5303953266
 pPage->nCell = (u16)nCell;
@@ -53483,16 +53710,28 @@
5348353710
 memcpy(pOld, apOld[i], sizeof(MemPage));
5348453711
 pOld->aData = (void*)&pOld[1];
5348553712
 memcpy(pOld->aData, apOld[i]->aData, pBt->pageSize);
5348653713
5348753714
 limit = pOld->nCell+pOld->nOverflow;
53488
- for(j=0; j<limit; j++){
53489
- assert( nCell<nMaxCells );
53490
- apCell[nCell] = findOverflowCell(pOld, j);
53491
- szCell[nCell] = cellSizePtr(pOld, apCell[nCell]);
53492
- nCell++;
53493
- }
53715
+ if( pOld->nOverflow>0 ){
53716
+ for(j=0; j<limit; j++){
53717
+ assert( nCell<nMaxCells );
53718
+ apCell[nCell] = findOverflowCell(pOld, j);
53719
+ szCell[nCell] = cellSizePtr(pOld, apCell[nCell]);
53720
+ nCell++;
53721
+ }
53722
+ }else{
53723
+ u8 *aData = pOld->aData;
53724
+ u16 maskPage = pOld->maskPage;
53725
+ u16 cellOffset = pOld->cellOffset;
53726
+ for(j=0; j<limit; j++){
53727
+ assert( nCell<nMaxCells );
53728
+ apCell[nCell] = findCellv2(aData, maskPage, cellOffset, j);
53729
+ szCell[nCell] = cellSizePtr(pOld, apCell[nCell]);
53730
+ nCell++;
53731
+ }
53732
+ }
5349453733
 if( i<nOld-1 && !leafData){
5349553734
 u16 sz = (u16)szNew[i];
5349653735
 u8 *pTemp;
5349753736
 assert( nCell<nMaxCells );
5349853737
 szCell[nCell] = sz;
@@ -57353,15 +57592,15 @@
5735357592
 *ppVal = 0;
5735457593
 return SQLITE_OK;
5735557594
 }
5735657595
 op = pExpr->op;
5735757596
57358
- /* op can only be TK_REGISTER if we have compiled with SQLITE_ENABLE_STAT2.
57597
+ /* op can only be TK_REGISTER if we have compiled with SQLITE_ENABLE_STAT3.
5735957598
 ** The ifdef here is to enable us to achieve 100% branch test coverage even
57360
- ** when SQLITE_ENABLE_STAT2 is omitted.
57599
+ ** when SQLITE_ENABLE_STAT3 is omitted.
5736157600
 */
57362
-#ifdef SQLITE_ENABLE_STAT2
57601
+#ifdef SQLITE_ENABLE_STAT3
5736357602
 if( op==TK_REGISTER ) op = pExpr->op2;
5736457603
 #else
5736557604
 if( NEVER(op==TK_REGISTER) ) op = pExpr->op2;
5736657605
 #endif
5736757606
@@ -57637,17 +57876,10 @@
5763757876
 pOp->p1 = p1;
5763857877
 pOp->p2 = p2;
5763957878
 pOp->p3 = p3;
5764057879
 pOp->p4.p = 0;
5764157880
 pOp->p4type = P4_NOTUSED;
57642
- p->expired = 0;
57643
- if( op==OP_ParseSchema ){
57644
- /* Any program that uses the OP_ParseSchema opcode needs to lock
57645
- ** all btrees. */
57646
- int j;
57647
- for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j);
57648
- }
5764957881
 #ifdef SQLITE_DEBUG
5765057882
 pOp->zComment = 0;
5765157883
 if( sqlite3VdbeAddopTrace ) sqlite3VdbePrintOp(0, i, &p->aOp[i]);
5765257884
 #endif
5765357885
 #ifdef VDBE_PROFILE
@@ -57681,10 +57913,24 @@
5768157913
 ){
5768257914
 int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3);
5768357915
 sqlite3VdbeChangeP4(p, addr, zP4, p4type);
5768457916
 return addr;
5768557917
 }
57918
+
57919
+/*
57920
+** Add an OP_ParseSchema opcode. This routine is broken out from
57921
+** sqlite3VdbeAddOp4() since it needs to also local all btrees.
57922
+**
57923
+** The zWhere string must have been obtained from sqlite3_malloc().
57924
+** This routine will take ownership of the allocated memory.
57925
+*/
57926
+SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe *p, int iDb, char *zWhere){
57927
+ int j;
57928
+ int addr = sqlite3VdbeAddOp3(p, OP_ParseSchema, iDb, 0, 0);
57929
+ sqlite3VdbeChangeP4(p, addr, zWhere, P4_DYNAMIC);
57930
+ for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j);
57931
+}
5768657932
5768757933
 /*
5768857934
 ** Add an opcode that includes the p4 value as an integer.
5768957935
 */
5769057936
 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(
@@ -58046,12 +58292,12 @@
5804658292
 /*
5804758293
 ** Change the P2 operand of instruction addr so that it points to
5804858294
 ** the address of the next instruction to be coded.
5804958295
 */
5805058296
 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe *p, int addr){
58051
- assert( addr>=0 );
58052
- sqlite3VdbeChangeP2(p, addr, p->nOp);
58297
+ assert( addr>=0 || p->db->mallocFailed );
58298
+ if( addr>=0 ) sqlite3VdbeChangeP2(p, addr, p->nOp);
5805358299
 }
5805458300
5805558301
5805658302
 /*
5805758303
 ** If the input FuncDef structure is ephemeral, then free it. If
@@ -58979,10 +59225,11 @@
5897959225
 nMem = 10;
5898059226
 }
5898159227
 memset(zCsr, 0, zEnd-zCsr);
5898259228
 zCsr += (zCsr - (u8*)0)&7;
5898359229
 assert( EIGHT_BYTE_ALIGNMENT(zCsr) );
59230
+ p->expired = 0;
5898459231
5898559232
 /* Memory for registers, parameters, cursor, etc, is allocated in two
5898659233
 ** passes. On the first pass, we try to reuse unused space at the
5898759234
 ** end of the opcode array. If we are unable to satisfy all memory
5898859235
 ** requirements by reusing the opcode array tail, then the second
@@ -61178,10 +61425,18 @@
6117861425
 rc = db->errCode = p->rc;
6117961426
 }
6118061427
 return (rc&db->errMask);
6118161428
 }
6118261429
61430
+/*
61431
+** The maximum number of times that a statement will try to reparse
61432
+** itself before giving up and returning SQLITE_SCHEMA.
61433
+*/
61434
+#ifndef SQLITE_MAX_SCHEMA_RETRY
61435
+# define SQLITE_MAX_SCHEMA_RETRY 5
61436
+#endif
61437
+
6118361438
 /*
6118461439
 ** This is the top-level implementation of sqlite3_step(). Call
6118561440
 ** sqlite3Step() to do most of the work. If a schema error occurs,
6118661441
 ** call sqlite3Reprepare() and try again.
6118761442
 */
@@ -61196,14 +61451,14 @@
6119661451
 return SQLITE_MISUSE_BKPT;
6119761452
 }
6119861453
 db = v->db;
6119961454
 sqlite3_mutex_enter(db->mutex);
6120061455
 while( (rc = sqlite3Step(v))==SQLITE_SCHEMA
61201
- && cnt++ < 5
61456
+ && cnt++ < SQLITE_MAX_SCHEMA_RETRY
6120261457
 && (rc2 = rc = sqlite3Reprepare(v))==SQLITE_OK ){
6120361458
 sqlite3_reset(pStmt);
61204
- v->expired = 0;
61459
+ assert( v->expired==0 );
6120561460
 }
6120661461
 if( rc2!=SQLITE_OK && ALWAYS(v->isPrepareV2) && ALWAYS(db->pErr) ){
6120761462
 /* This case occurs after failing to recompile an sql statement.
6120861463
 ** The error message from the SQL compiler has already been loaded
6120961464
 ** into the database handle. This block copies the error message
@@ -63469,11 +63724,11 @@
6346963724
 break;
6347063725
 }
6347163726
6347263727
 /* Opcode: HaltIfNull P1 P2 P3 P4 *
6347363728
 **
63474
-** Check the value in register P3. If is is NULL then Halt using
63729
+** Check the value in register P3. If it is NULL then Halt using
6347563730
 ** parameter P1, P2, and P4 as if this were a Halt instruction. If the
6347663731
 ** value in register P3 is not NULL, then this routine is a no-op.
6347763732
 */
6347863733
 case OP_HaltIfNull: { /* in3 */
6347963734
 pIn3 = &aMem[pOp->p3];
@@ -63703,10 +63958,15 @@
6370363958
 assert( memIsValid(pIn1) );
6370463959
 memAboutToChange(p, pOut);
6370563960
 u.ac.zMalloc = pOut->zMalloc;
6370663961
 pOut->zMalloc = 0;
6370763962
 sqlite3VdbeMemMove(pOut, pIn1);
63963
+#ifdef SQLITE_DEBUG
63964
+ if( pOut->pScopyFrom>=&aMem[u.ac.p1] && pOut->pScopyFrom<&aMem[u.ac.p1+pOp->p3] ){
63965
+ pOut->pScopyFrom += u.ac.p1 - pOp->p2;
63966
+ }
63967
+#endif
6370863968
 pIn1->zMalloc = u.ac.zMalloc;
6370963969
 REGISTER_TRACE(u.ac.p2++, pOut);
6371063970
 pIn1++;
6371163971
 pOut++;
6371263972
 }
@@ -64083,10 +64343,20 @@
6408364343
 u.ag.ctx.pColl = pOp[-1].p4.pColl;
6408464344
 }
6408564345
 db->lastRowid = lastRowid;
6408664346
 (*u.ag.ctx.pFunc->xFunc)(&u.ag.ctx, u.ag.n, u.ag.apVal); /* IMP: R-24505-23230 */
6408764347
 lastRowid = db->lastRowid;
64348
+
64349
+ /* If any auxiliary data functions have been called by this user function,
64350
+ ** immediately call the destructor for any non-static values.
64351
+ */
64352
+ if( u.ag.ctx.pVdbeFunc ){
64353
+ sqlite3VdbeDeleteAuxData(u.ag.ctx.pVdbeFunc, pOp->p1);
64354
+ pOp->p4.pVdbeFunc = u.ag.ctx.pVdbeFunc;
64355
+ pOp->p4type = P4_VDBEFUNC;
64356
+ }
64357
+
6408864358
 if( db->mallocFailed ){
6408964359
 /* Even though a malloc() has failed, the implementation of the
6409064360
 ** user function may have called an sqlite3_result_XXX() function
6409164361
 ** to return a value. The following call releases any resources
6409264362
 ** associated with such a value.
@@ -64093,19 +64363,10 @@
6409364363
 */
6409464364
 sqlite3VdbeMemRelease(&u.ag.ctx.s);
6409564365
 goto no_mem;
6409664366
 }
6409764367
64098
- /* If any auxiliary data functions have been called by this user function,
64099
- ** immediately call the destructor for any non-static values.
64100
- */
64101
- if( u.ag.ctx.pVdbeFunc ){
64102
- sqlite3VdbeDeleteAuxData(u.ag.ctx.pVdbeFunc, pOp->p1);
64103
- pOp->p4.pVdbeFunc = u.ag.ctx.pVdbeFunc;
64104
- pOp->p4type = P4_VDBEFUNC;
64105
- }
64106
-
6410764368
 /* If the function returned an error, throw an exception */
6410864369
 if( u.ag.ctx.isError ){
6410964370
 sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.ag.ctx.s));
6411064371
 rc = u.ag.ctx.isError;
6411164372
 }
@@ -64405,11 +64666,11 @@
6440564666
 ** additional information.
6440664667
 **
6440764668
 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
6440864669
 ** true or false and is never NULL. If both operands are NULL then the result
6440964670
 ** of comparison is false. If either operand is NULL then the result is true.
64410
-** If neither operand is NULL the the result is the same as it would be if
64671
+** If neither operand is NULL the result is the same as it would be if
6441164672
 ** the SQLITE_NULLEQ flag were omitted from P5.
6441264673
 */
6441364674
 /* Opcode: Eq P1 P2 P3 P4 P5
6441464675
 **
6441564676
 ** This works just like the Lt opcode except that the jump is taken if
@@ -64417,11 +64678,11 @@
6441764678
 ** See the Lt opcode for additional information.
6441864679
 **
6441964680
 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
6442064681
 ** true or false and is never NULL. If both operands are NULL then the result
6442164682
 ** of comparison is true. If either operand is NULL then the result is false.
64422
-** If neither operand is NULL the the result is the same as it would be if
64683
+** If neither operand is NULL the result is the same as it would be if
6442364684
 ** the SQLITE_NULLEQ flag were omitted from P5.
6442464685
 */
6442564686
 /* Opcode: Le P1 P2 P3 P4 P5
6442664687
 **
6442764688
 ** This works just like the Lt opcode except that the jump is taken if
@@ -64702,17 +64963,17 @@
6470264963
 break;
6470364964
 }
6470464965
6470564966
 /* Opcode: If P1 P2 P3 * *
6470664967
 **
64707
-** Jump to P2 if the value in register P1 is true. The value is
64968
+** Jump to P2 if the value in register P1 is true. The value
6470864969
 ** is considered true if it is numeric and non-zero. If the value
6470964970
 ** in P1 is NULL then take the jump if P3 is true.
6471064971
 */
6471164972
 /* Opcode: IfNot P1 P2 P3 * *
6471264973
 **
64713
-** Jump to P2 if the value in register P1 is False. The value is
64974
+** Jump to P2 if the value in register P1 is False. The value
6471464975
 ** is considered true if it has a numeric value of zero. If the value
6471564976
 ** in P1 is NULL then take the jump if P3 is true.
6471664977
 */
6471764978
 case OP_If: /* jump, in1 */
6471864979
 case OP_IfNot: { /* jump, in1 */
@@ -65889,11 +66150,11 @@
6588966150
 ** automatically created table with root-page 1 (an BLOB_INTKEY table).
6589066151
 */
6589166152
 if( pOp->p4.pKeyInfo ){
6589266153
 int pgno;
6589366154
 assert( pOp->p4type==P4_KEYINFO );
65894
- rc = sqlite3BtreeCreateTable(u.ax.pCx->pBt, &pgno, BTREE_BLOBKEY);
66155
+ rc = sqlite3BtreeCreateTable(u.ax.pCx->pBt, &pgno, BTREE_BLOBKEY | pOp->p5);
6589566156
 if( rc==SQLITE_OK ){
6589666157
 assert( pgno==MASTER_ROOT+1 );
6589766158
 rc = sqlite3BtreeCursor(u.ax.pCx->pBt, pgno, 1,
6589866159
 (KeyInfo*)pOp->p4.z, u.ax.pCx->pCursor);
6589966160
 u.ax.pCx->pKeyInfo = pOp->p4.pKeyInfo;
@@ -66362,11 +66623,11 @@
6636266623
 break;
6636366624
 }
6636466625
6636566626
 /* Opcode: NotExists P1 P2 P3 * *
6636666627
 **
66367
-** Use the content of register P3 as a integer key. If a record
66628
+** Use the content of register P3 as an integer key. If a record
6636866629
 ** with that key does not exist in table of P1, then jump to P2.
6636966630
 ** If the record does exist, then fall through. The cursor is left
6637066631
 ** pointing to the record if it exists.
6637166632
 **
6637266633
 ** The difference between this operation and NotFound is that this
@@ -66440,11 +66701,11 @@
6644066701
 ** written to register P2.
6644166702
 **
6644266703
 ** If P3>0 then P3 is a register in the root frame of this VDBE that holds
6644366704
 ** the largest previously generated record number. No new record numbers are
6644466705
 ** allowed to be less than this value. When this value reaches its maximum,
66445
-** a SQLITE_FULL error is generated. The P3 register is updated with the '
66706
+** an SQLITE_FULL error is generated. The P3 register is updated with the '
6644666707
 ** generated record number. This P3 mechanism is used to help implement the
6644766708
 ** AUTOINCREMENT feature.
6644866709
 */
6644966710
 case OP_NewRowid: { /* out2-prerelease */
6645066711
 #if 0 /* local variables moved into u.be */
@@ -67082,11 +67343,11 @@
6708267343
 break;
6708367344
 }
6708467345
6708567346
 /* Opcode: IdxInsert P1 P2 P3 * P5
6708667347
 **
67087
-** Register P2 holds a SQL index key made using the
67348
+** Register P2 holds an SQL index key made using the
6708867349
 ** MakeRecord instructions. This opcode writes that key
6708967350
 ** into the index P1. Data for the entry is nil.
6709067351
 **
6709167352
 ** P3 is a flag that provides a hint to the b-tree layer that this
6709267353
 ** insert is likely to be an append.
@@ -70928,15 +71189,29 @@
7092871189
 /* Recursively resolve names in all subqueries
7092971190
 */
7093071191
 for(i=0; i<p->pSrc->nSrc; i++){
7093171192
 struct SrcList_item *pItem = &p->pSrc->a[i];
7093271193
 if( pItem->pSelect ){
71194
+ NameContext *pNC; /* Used to iterate name contexts */
71195
+ int nRef = 0; /* Refcount for pOuterNC and outer contexts */
7093371196
 const char *zSavedContext = pParse->zAuthContext;
71197
+
71198
+ /* Count the total number of references to pOuterNC and all of its
71199
+ ** parent contexts. After resolving references to expressions in
71200
+ ** pItem->pSelect, check if this value has changed. If so, then
71201
+ ** SELECT statement pItem->pSelect must be correlated. Set the
71202
+ ** pItem->isCorrelated flag if this is the case. */
71203
+ for(pNC=pOuterNC; pNC; pNC=pNC->pNext) nRef += pNC->nRef;
71204
+
7093471205
 if( pItem->zName ) pParse->zAuthContext = pItem->zName;
7093571206
 sqlite3ResolveSelectNames(pParse, pItem->pSelect, pOuterNC);
7093671207
 pParse->zAuthContext = zSavedContext;
7093771208
 if( pParse->nErr || db->mallocFailed ) return WRC_Abort;
71209
+
71210
+ for(pNC=pOuterNC; pNC; pNC=pNC->pNext) nRef -= pNC->nRef;
71211
+ assert( pItem->isCorrelated==0 && nRef<=0 );
71212
+ pItem->isCorrelated = (nRef!=0);
7093871213
 }
7093971214
 }
7094071215
7094171216
 /* If there are no aggregate functions in the result-set, and no GROUP BY
7094271217
 ** expression, do not allow aggregates in any of the other expressions.
@@ -72040,10 +72315,11 @@
7204072315
 pNewItem->zName = sqlite3DbStrDup(db, pOldItem->zName);
7204172316
 pNewItem->zAlias = sqlite3DbStrDup(db, pOldItem->zAlias);
7204272317
 pNewItem->jointype = pOldItem->jointype;
7204372318
 pNewItem->iCursor = pOldItem->iCursor;
7204472319
 pNewItem->isPopulated = pOldItem->isPopulated;
72320
+ pNewItem->isCorrelated = pOldItem->isCorrelated;
7204572321
 pNewItem->zIndex = sqlite3DbStrDup(db, pOldItem->zIndex);
7204672322
 pNewItem->notIndexed = pOldItem->notIndexed;
7204772323
 pNewItem->pIndex = pOldItem->pIndex;
7204872324
 pTab = pNewItem->pTab = pOldItem->pTab;
7204972325
 if( pTab ){
@@ -75255,18 +75531,18 @@
7525575531
 sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
7525675532
7525775533
 /* Reload the table, index and permanent trigger schemas. */
7525875534
 zWhere = sqlite3MPrintf(pParse->db, "tbl_name=%Q", zName);
7525975535
 if( !zWhere ) return;
75260
- sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC);
75536
+ sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere);
7526175537
7526275538
 #ifndef SQLITE_OMIT_TRIGGER
7526375539
 /* Now, if the table is not stored in the temp database, reload any temp
7526475540
 ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined.
7526575541
 */
7526675542
 if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){
75267
- sqlite3VdbeAddOp4(v, OP_ParseSchema, 1, 0, 0, zWhere, P4_DYNAMIC);
75543
+ sqlite3VdbeAddParseSchemaOp(v, 1, zWhere);
7526875544
 }
7526975545
 #endif
7527075546
 }
7527175547
7527275548
 /*
@@ -75736,10 +76012,100 @@
7573676012
 ** May you find forgiveness for yourself and forgive others.
7573776013
 ** May you share freely, never taking more than you give.
7573876014
 **
7573976015
 *************************************************************************
7574076016
 ** This file contains code associated with the ANALYZE command.
76017
+**
76018
+** The ANALYZE command gather statistics about the content of tables
76019
+** and indices. These statistics are made available to the query planner
76020
+** to help it make better decisions about how to perform queries.
76021
+**
76022
+** The following system tables are or have been supported:
76023
+**
76024
+** CREATE TABLE sqlite_stat1(tbl, idx, stat);
76025
+** CREATE TABLE sqlite_stat2(tbl, idx, sampleno, sample);
76026
+** CREATE TABLE sqlite_stat3(tbl, idx, nEq, nLt, nDLt, sample);
76027
+**
76028
+** Additional tables might be added in future releases of SQLite.
76029
+** The sqlite_stat2 table is not created or used unless the SQLite version
76030
+** is between 3.6.18 and 3.7.7, inclusive, and unless SQLite is compiled
76031
+** with SQLITE_ENABLE_STAT2. The sqlite_stat2 table is deprecated.
76032
+** The sqlite_stat2 table is superceded by sqlite_stat3, which is only
76033
+** created and used by SQLite versions after 2011-08-09 with
76034
+** SQLITE_ENABLE_STAT3 defined. The fucntionality of sqlite_stat3
76035
+** is a superset of sqlite_stat2.
76036
+**
76037
+** Format of sqlite_stat1:
76038
+**
76039
+** There is normally one row per index, with the index identified by the
76040
+** name in the idx column. The tbl column is the name of the table to
76041
+** which the index belongs. In each such row, the stat column will be
76042
+** a string consisting of a list of integers. The first integer in this
76043
+** list is the number of rows in the index and in the table. The second
76044
+** integer is the average number of rows in the index that have the same
76045
+** value in the first column of the index. The third integer is the average
76046
+** number of rows in the index that have the same value for the first two
76047
+** columns. The N-th integer (for N>1) is the average number of rows in
76048
+** the index which have the same value for the first N-1 columns. For
76049
+** a K-column index, there will be K+1 integers in the stat column. If
76050
+** the index is unique, then the last integer will be 1.
76051
+**
76052
+** The list of integers in the stat column can optionally be followed
76053
+** by the keyword "unordered". The "unordered" keyword, if it is present,
76054
+** must be separated from the last integer by a single space. If the
76055
+** "unordered" keyword is present, then the query planner assumes that
76056
+** the index is unordered and will not use the index for a range query.
76057
+**
76058
+** If the sqlite_stat1.idx column is NULL, then the sqlite_stat1.stat
76059
+** column contains a single integer which is the (estimated) number of
76060
+** rows in the table identified by sqlite_stat1.tbl.
76061
+**
76062
+** Format of sqlite_stat2:
76063
+**
76064
+** The sqlite_stat2 is only created and is only used if SQLite is compiled
76065
+** with SQLITE_ENABLE_STAT2 and if the SQLite version number is between
76066
+** 3.6.18 and 3.7.7. The "stat2" table contains additional information
76067
+** about the distribution of keys within an index. The index is identified by
76068
+** the "idx" column and the "tbl" column is the name of the table to which
76069
+** the index belongs. There are usually 10 rows in the sqlite_stat2
76070
+** table for each index.
76071
+**
76072
+** The sqlite_stat2 entries for an index that have sampleno between 0 and 9
76073
+** inclusive are samples of the left-most key value in the index taken at
76074
+** evenly spaced points along the index. Let the number of samples be S
76075
+** (10 in the standard build) and let C be the number of rows in the index.
76076
+** Then the sampled rows are given by:
76077
+**
76078
+** rownumber = (i*C*2 + C)/(S*2)
76079
+**
76080
+** For i between 0 and S-1. Conceptually, the index space is divided into
76081
+** S uniform buckets and the samples are the middle row from each bucket.
76082
+**
76083
+** The format for sqlite_stat2 is recorded here for legacy reference. This
76084
+** version of SQLite does not support sqlite_stat2. It neither reads nor
76085
+** writes the sqlite_stat2 table. This version of SQLite only supports
76086
+** sqlite_stat3.
76087
+**
76088
+** Format for sqlite_stat3:
76089
+**
76090
+** The sqlite_stat3 is an enhancement to sqlite_stat2. A new name is
76091
+** used to avoid compatibility problems.
76092
+**
76093
+** The format of the sqlite_stat3 table is similar to the format for
76094
+** the sqlite_stat2 table, with the following changes: (1)
76095
+** The sampleno column is removed. (2) Every sample has nEq, nLt, and nDLt
76096
+** columns which hold the approximate number of rows in the table that
76097
+** exactly match the sample, the approximate number of rows with values
76098
+** less than the sample, and the approximate number of distinct key values
76099
+** less than the sample, respectively. (3) The number of samples can vary
76100
+** from one table to the next; the sample count does not have to be
76101
+** exactly 10 as it is with sqlite_stat2.
76102
+**
76103
+** The ANALYZE command will typically generate sqlite_stat3 tables
76104
+** that contain between 10 and 40 samples which are distributed across
76105
+** the key space, though not uniformly, and which include samples with
76106
+** largest possible nEq values.
7574176107
 */
7574276108
 #ifndef SQLITE_OMIT_ANALYZE
7574376109
7574476110
 /*
7574576111
 ** This routine generates code that opens the sqlite_stat1 table for
@@ -75767,12 +76133,18 @@
7576776133
 static const struct {
7576876134
 const char *zName;
7576976135
 const char *zCols;
7577076136
 } aTable[] = {
7577176137
 { "sqlite_stat1", "tbl,idx,stat" },
75772
-#ifdef SQLITE_ENABLE_STAT2
75773
- { "sqlite_stat2", "tbl,idx,sampleno,sample" },
76138
+#ifdef SQLITE_ENABLE_STAT3
76139
+ { "sqlite_stat3", "tbl,idx,neq,nlt,ndlt,sample" },
76140
+#endif
76141
+ };
76142
+ static const char *azToDrop[] = {
76143
+ "sqlite_stat2",
76144
+#ifndef SQLITE_ENABLE_STAT3
76145
+ "sqlite_stat3",
7577476146
 #endif
7577576147
 };
7577676148
7577776149
 int aRoot[] = {0, 0};
7577876150
 u8 aCreateTbl[] = {0, 0};
@@ -75784,10 +76156,21 @@
7578476156
 if( v==0 ) return;
7578576157
 assert( sqlite3BtreeHoldsAllMutexes(db) );
7578676158
 assert( sqlite3VdbeDb(v)==db );
7578776159
 pDb = &db->aDb[iDb];
7578876160
76161
+ /* Drop all statistics tables that this version of SQLite does not
76162
+ ** understand.
76163
+ */
76164
+ for(i=0; i<ArraySize(azToDrop); i++){
76165
+ Table *pTab = sqlite3FindTable(db, azToDrop[i], pDb->zName);
76166
+ if( pTab ) sqlite3CodeDropTable(pParse, pTab, iDb, 0);
76167
+ }
76168
+
76169
+ /* Create new statistic tables if they do not exist, or clear them
76170
+ ** if they do already exist.
76171
+ */
7578976172
 for(i=0; i<ArraySize(aTable); i++){
7579076173
 const char *zTab = aTable[i].zName;
7579176174
 Table *pStat;
7579276175
 if( (pStat = sqlite3FindTable(db, zTab, pDb->zName))==0 ){
7579376176
 /* The sqlite_stat[12] table does not exist. Create it. Note that a
@@ -75814,17 +76197,238 @@
7581476197
 sqlite3VdbeAddOp2(v, OP_Clear, aRoot[i], iDb);
7581576198
 }
7581676199
 }
7581776200
 }
7581876201
75819
- /* Open the sqlite_stat[12] tables for writing. */
76202
+ /* Open the sqlite_stat[13] tables for writing. */
7582076203
 for(i=0; i<ArraySize(aTable); i++){
7582176204
 sqlite3VdbeAddOp3(v, OP_OpenWrite, iStatCur+i, aRoot[i], iDb);
7582276205
 sqlite3VdbeChangeP4(v, -1, (char *)3, P4_INT32);
7582376206
 sqlite3VdbeChangeP5(v, aCreateTbl[i]);
7582476207
 }
7582576208
 }
76209
+
76210
+/*
76211
+** Recommended number of samples for sqlite_stat3
76212
+*/
76213
+#ifndef SQLITE_STAT3_SAMPLES
76214
+# define SQLITE_STAT3_SAMPLES 24
76215
+#endif
76216
+
76217
+/*
76218
+** Three SQL functions - stat3_init(), stat3_push(), and stat3_pop() -
76219
+** share an instance of the following structure to hold their state
76220
+** information.
76221
+*/
76222
+typedef struct Stat3Accum Stat3Accum;
76223
+struct Stat3Accum {
76224
+ tRowcnt nRow; /* Number of rows in the entire table */
76225
+ tRowcnt nPSample; /* How often to do a periodic sample */
76226
+ int iMin; /* Index of entry with minimum nEq and hash */
76227
+ int mxSample; /* Maximum number of samples to accumulate */
76228
+ int nSample; /* Current number of samples */
76229
+ u32 iPrn; /* Pseudo-random number used for sampling */
76230
+ struct Stat3Sample {
76231
+ i64 iRowid; /* Rowid in main table of the key */
76232
+ tRowcnt nEq; /* sqlite_stat3.nEq */
76233
+ tRowcnt nLt; /* sqlite_stat3.nLt */
76234
+ tRowcnt nDLt; /* sqlite_stat3.nDLt */
76235
+ u8 isPSample; /* True if a periodic sample */
76236
+ u32 iHash; /* Tiebreaker hash */
76237
+ } *a; /* An array of samples */
76238
+};
76239
+
76240
+#ifdef SQLITE_ENABLE_STAT3
76241
+/*
76242
+** Implementation of the stat3_init(C,S) SQL function. The two parameters
76243
+** are the number of rows in the table or index (C) and the number of samples
76244
+** to accumulate (S).
76245
+**
76246
+** This routine allocates the Stat3Accum object.
76247
+**
76248
+** The return value is the Stat3Accum object (P).
76249
+*/
76250
+static void stat3Init(
76251
+ sqlite3_context *context,
76252
+ int argc,
76253
+ sqlite3_value **argv
76254
+){
76255
+ Stat3Accum *p;
76256
+ tRowcnt nRow;
76257
+ int mxSample;
76258
+ int n;
76259
+
76260
+ UNUSED_PARAMETER(argc);
76261
+ nRow = (tRowcnt)sqlite3_value_int64(argv[0]);
76262
+ mxSample = sqlite3_value_int(argv[1]);
76263
+ n = sizeof(*p) + sizeof(p->a[0])*mxSample;
76264
+ p = sqlite3_malloc( n );
76265
+ if( p==0 ){
76266
+ sqlite3_result_error_nomem(context);
76267
+ return;
76268
+ }
76269
+ memset(p, 0, n);
76270
+ p->a = (struct Stat3Sample*)&p[1];
76271
+ p->nRow = nRow;
76272
+ p->mxSample = mxSample;
76273
+ p->nPSample = p->nRow/(mxSample/3+1) + 1;
76274
+ sqlite3_randomness(sizeof(p->iPrn), &p->iPrn);
76275
+ sqlite3_result_blob(context, p, sizeof(p), sqlite3_free);
76276
+}
76277
+static const FuncDef stat3InitFuncdef = {
76278
+ 2, /* nArg */
76279
+ SQLITE_UTF8, /* iPrefEnc */
76280
+ 0, /* flags */
76281
+ 0, /* pUserData */
76282
+ 0, /* pNext */
76283
+ stat3Init, /* xFunc */
76284
+ 0, /* xStep */
76285
+ 0, /* xFinalize */
76286
+ "stat3_init", /* zName */
76287
+ 0, /* pHash */
76288
+ 0 /* pDestructor */
76289
+};
76290
+
76291
+
76292
+/*
76293
+** Implementation of the stat3_push(nEq,nLt,nDLt,rowid,P) SQL function. The
76294
+** arguments describe a single key instance. This routine makes the
76295
+** decision about whether or not to retain this key for the sqlite_stat3
76296
+** table.
76297
+**
76298
+** The return value is NULL.
76299
+*/
76300
+static void stat3Push(
76301
+ sqlite3_context *context,
76302
+ int argc,
76303
+ sqlite3_value **argv
76304
+){
76305
+ Stat3Accum *p = (Stat3Accum*)sqlite3_value_blob(argv[4]);
76306
+ tRowcnt nEq = sqlite3_value_int64(argv[0]);
76307
+ tRowcnt nLt = sqlite3_value_int64(argv[1]);
76308
+ tRowcnt nDLt = sqlite3_value_int64(argv[2]);
76309
+ i64 rowid = sqlite3_value_int64(argv[3]);
76310
+ u8 isPSample = 0;
76311
+ u8 doInsert = 0;
76312
+ int iMin = p->iMin;
76313
+ struct Stat3Sample *pSample;
76314
+ int i;
76315
+ u32 h;
76316
+
76317
+ UNUSED_PARAMETER(context);
76318
+ UNUSED_PARAMETER(argc);
76319
+ if( nEq==0 ) return;
76320
+ h = p->iPrn = p->iPrn*1103515245 + 12345;
76321
+ if( (nLt/p->nPSample)!=((nEq+nLt)/p->nPSample) ){
76322
+ doInsert = isPSample = 1;
76323
+ }else if( p->nSample<p->mxSample ){
76324
+ doInsert = 1;
76325
+ }else{
76326
+ if( nEq>p->a[iMin].nEq || (nEq==p->a[iMin].nEq && h>p->a[iMin].iHash) ){
76327
+ doInsert = 1;
76328
+ }
76329
+ }
76330
+ if( !doInsert ) return;
76331
+ if( p->nSample==p->mxSample ){
76332
+ if( iMin<p->nSample ){
76333
+ memcpy(&p->a[iMin], &p->a[iMin+1], sizeof(p->a[0])*(p->nSample-iMin));
76334
+ }
76335
+ pSample = &p->a[p->nSample-1];
76336
+ }else{
76337
+ pSample = &p->a[p->nSample++];
76338
+ }
76339
+ pSample->iRowid = rowid;
76340
+ pSample->nEq = nEq;
76341
+ pSample->nLt = nLt;
76342
+ pSample->nDLt = nDLt;
76343
+ pSample->iHash = h;
76344
+ pSample->isPSample = isPSample;
76345
+
76346
+ /* Find the new minimum */
76347
+ if( p->nSample==p->mxSample ){
76348
+ pSample = p->a;
76349
+ i = 0;
76350
+ while( pSample->isPSample ){
76351
+ i++;
76352
+ pSample++;
76353
+ assert( i<p->nSample );
76354
+ }
76355
+ nEq = pSample->nEq;
76356
+ h = pSample->iHash;
76357
+ iMin = i;
76358
+ for(i++, pSample++; i<p->nSample; i++, pSample++){
76359
+ if( pSample->isPSample ) continue;
76360
+ if( pSample->nEq<nEq
76361
+ || (pSample->nEq==nEq && pSample->iHash<h)
76362
+ ){
76363
+ iMin = i;
76364
+ nEq = pSample->nEq;
76365
+ h = pSample->iHash;
76366
+ }
76367
+ }
76368
+ p->iMin = iMin;
76369
+ }
76370
+}
76371
+static const FuncDef stat3PushFuncdef = {
76372
+ 5, /* nArg */
76373
+ SQLITE_UTF8, /* iPrefEnc */
76374
+ 0, /* flags */
76375
+ 0, /* pUserData */
76376
+ 0, /* pNext */
76377
+ stat3Push, /* xFunc */
76378
+ 0, /* xStep */
76379
+ 0, /* xFinalize */
76380
+ "stat3_push", /* zName */
76381
+ 0, /* pHash */
76382
+ 0 /* pDestructor */
76383
+};
76384
+
76385
+/*
76386
+** Implementation of the stat3_get(P,N,...) SQL function. This routine is
76387
+** used to query the results. Content is returned for the Nth sqlite_stat3
76388
+** row where N is between 0 and S-1 and S is the number of samples. The
76389
+** value returned depends on the number of arguments.
76390
+**
76391
+** argc==2 result: rowid
76392
+** argc==3 result: nEq
76393
+** argc==4 result: nLt
76394
+** argc==5 result: nDLt
76395
+*/
76396
+static void stat3Get(
76397
+ sqlite3_context *context,
76398
+ int argc,
76399
+ sqlite3_value **argv
76400
+){
76401
+ int n = sqlite3_value_int(argv[1]);
76402
+ Stat3Accum *p = (Stat3Accum*)sqlite3_value_blob(argv[0]);
76403
+
76404
+ assert( p!=0 );
76405
+ if( p->nSample<=n ) return;
76406
+ switch( argc ){
76407
+ case 2: sqlite3_result_int64(context, p->a[n].iRowid); break;
76408
+ case 3: sqlite3_result_int64(context, p->a[n].nEq); break;
76409
+ case 4: sqlite3_result_int64(context, p->a[n].nLt); break;
76410
+ case 5: sqlite3_result_int64(context, p->a[n].nDLt); break;
76411
+ }
76412
+}
76413
+static const FuncDef stat3GetFuncdef = {
76414
+ -1, /* nArg */
76415
+ SQLITE_UTF8, /* iPrefEnc */
76416
+ 0, /* flags */
76417
+ 0, /* pUserData */
76418
+ 0, /* pNext */
76419
+ stat3Get, /* xFunc */
76420
+ 0, /* xStep */
76421
+ 0, /* xFinalize */
76422
+ "stat3_get", /* zName */
76423
+ 0, /* pHash */
76424
+ 0 /* pDestructor */
76425
+};
76426
+#endif /* SQLITE_ENABLE_STAT3 */
76427
+
76428
+
76429
+
7582676430
7582776431
 /*
7582876432
 ** Generate code to do an analysis of all indices associated with
7582976433
 ** a single table.
7583076434
 */
@@ -75844,24 +76448,31 @@
7584476448
 int endOfLoop; /* The end of the loop */
7584576449
 int jZeroRows = -1; /* Jump from here if number of rows is zero */
7584676450
 int iDb; /* Index of database containing pTab */
7584776451
 int regTabname = iMem++; /* Register containing table name */
7584876452
 int regIdxname = iMem++; /* Register containing index name */
75849
- int regSampleno = iMem++; /* Register containing next sample number */
75850
- int regCol = iMem++; /* Content of a column analyzed table */
76453
+ int regStat1 = iMem++; /* The stat column of sqlite_stat1 */
76454
+#ifdef SQLITE_ENABLE_STAT3
76455
+ int regNumEq = regStat1; /* Number of instances. Same as regStat1 */
76456
+ int regNumLt = iMem++; /* Number of keys less than regSample */
76457
+ int regNumDLt = iMem++; /* Number of distinct keys less than regSample */
76458
+ int regSample = iMem++; /* The next sample value */
76459
+ int regRowid = regSample; /* Rowid of a sample */
76460
+ int regAccum = iMem++; /* Register to hold Stat3Accum object */
76461
+ int regLoop = iMem++; /* Loop counter */
76462
+ int regCount = iMem++; /* Number of rows in the table or index */
76463
+ int regTemp1 = iMem++; /* Intermediate register */
76464
+ int regTemp2 = iMem++; /* Intermediate register */
76465
+ int once = 1; /* One-time initialization */
76466
+ int shortJump = 0; /* Instruction address */
76467
+ int iTabCur = pParse->nTab++; /* Table cursor */
76468
+#endif
76469
+ int regCol = iMem++; /* Content of a column in analyzed table */
7585176470
 int regRec = iMem++; /* Register holding completed record */
7585276471
 int regTemp = iMem++; /* Temporary use register */
75853
- int regRowid = iMem++; /* Rowid for the inserted record */
75854
-
75855
-#ifdef SQLITE_ENABLE_STAT2
75856
- int addr = 0; /* Instruction address */
75857
- int regTemp2 = iMem++; /* Temporary use register */
75858
- int regSamplerecno = iMem++; /* Index of next sample to record */
75859
- int regRecno = iMem++; /* Current sample index */
75860
- int regLast = iMem++; /* Index of last sample to record */
75861
- int regFirst = iMem++; /* Index of first sample to record */
75862
-#endif
76472
+ int regNewRowid = iMem++; /* Rowid for the inserted record */
76473
+
7586376474
7586476475
 v = sqlite3GetVdbe(pParse);
7586576476
 if( v==0 || NEVER(pTab==0) ){
7586676477
 return;
7586776478
 }
@@ -75890,17 +76501,22 @@
7589076501
 iIdxCur = pParse->nTab++;
7589176502
 sqlite3VdbeAddOp4(v, OP_String8, 0, regTabname, 0, pTab->zName, 0);
7589276503
 for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
7589376504
 int nCol;
7589476505
 KeyInfo *pKey;
76506
+ int addrIfNot = 0; /* address of OP_IfNot */
76507
+ int *aChngAddr; /* Array of jump instruction addresses */
7589576508
7589676509
 if( pOnlyIdx && pOnlyIdx!=pIdx ) continue;
76510
+ VdbeNoopComment((v, "Begin analysis of %s", pIdx->zName));
7589776511
 nCol = pIdx->nColumn;
7589876512
 pKey = sqlite3IndexKeyinfo(pParse, pIdx);
7589976513
 if( iMem+1+(nCol*2)>pParse->nMem ){
7590076514
 pParse->nMem = iMem+1+(nCol*2);
7590176515
 }
76516
+ aChngAddr = sqlite3DbMallocRaw(db, sizeof(int)*pIdx->nColumn);
76517
+ if( aChngAddr==0 ) continue;
7590276518
7590376519
 /* Open a cursor to the index to be analyzed. */
7590476520
 assert( iDb==sqlite3SchemaToIndex(db, pIdx->pSchema) );
7590576521
 sqlite3VdbeAddOp4(v, OP_OpenRead, iIdxCur, pIdx->tnum, iDb,
7590676522
 (char *)pKey, P4_KEYINFO_HANDOFF);
@@ -75907,35 +76523,24 @@
7590776523
 VdbeComment((v, "%s", pIdx->zName));
7590876524
7590976525
 /* Populate the register containing the index name. */
7591076526
 sqlite3VdbeAddOp4(v, OP_String8, 0, regIdxname, 0, pIdx->zName, 0);
7591176527
75912
-#ifdef SQLITE_ENABLE_STAT2
75913
-
75914
- /* If this iteration of the loop is generating code to analyze the
75915
- ** first index in the pTab->pIndex list, then register regLast has
75916
- ** not been populated. In this case populate it now. */
75917
- if( pTab->pIndex==pIdx ){
75918
- sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES, regSamplerecno);
75919
- sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES*2-1, regTemp);
75920
- sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES*2, regTemp2);
75921
-
75922
- sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regLast);
75923
- sqlite3VdbeAddOp2(v, OP_Null, 0, regFirst);
75924
- addr = sqlite3VdbeAddOp3(v, OP_Lt, regSamplerecno, 0, regLast);
75925
- sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regLast, regFirst);
75926
- sqlite3VdbeAddOp3(v, OP_Multiply, regLast, regTemp, regLast);
75927
- sqlite3VdbeAddOp2(v, OP_AddImm, regLast, SQLITE_INDEX_SAMPLES*2-2);
75928
- sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regLast, regLast);
75929
- sqlite3VdbeJumpHere(v, addr);
75930
- }
75931
-
75932
- /* Zero the regSampleno and regRecno registers. */
75933
- sqlite3VdbeAddOp2(v, OP_Integer, 0, regSampleno);
75934
- sqlite3VdbeAddOp2(v, OP_Integer, 0, regRecno);
75935
- sqlite3VdbeAddOp2(v, OP_Copy, regFirst, regSamplerecno);
75936
-#endif
76528
+#ifdef SQLITE_ENABLE_STAT3
76529
+ if( once ){
76530
+ once = 0;
76531
+ sqlite3OpenTable(pParse, iTabCur, iDb, pTab, OP_OpenRead);
76532
+ }
76533
+ sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regCount);
76534
+ sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_STAT3_SAMPLES, regTemp1);
76535
+ sqlite3VdbeAddOp2(v, OP_Integer, 0, regNumEq);
76536
+ sqlite3VdbeAddOp2(v, OP_Integer, 0, regNumLt);
76537
+ sqlite3VdbeAddOp2(v, OP_Integer, -1, regNumDLt);
76538
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regCount, regAccum,
76539
+ (char*)&stat3InitFuncdef, P4_FUNCDEF);
76540
+ sqlite3VdbeChangeP5(v, 2);
76541
+#endif /* SQLITE_ENABLE_STAT3 */
7593776542
7593876543
 /* The block of memory cells initialized here is used as follows.
7593976544
 **
7594076545
 ** iMem:
7594176546
 ** The total number of rows in the table.
@@ -75961,79 +76566,87 @@
7596176566
 /* Start the analysis loop. This loop runs through all the entries in
7596276567
 ** the index b-tree. */
7596376568
 endOfLoop = sqlite3VdbeMakeLabel(v);
7596476569
 sqlite3VdbeAddOp2(v, OP_Rewind, iIdxCur, endOfLoop);
7596576570
 topOfLoop = sqlite3VdbeCurrentAddr(v);
75966
- sqlite3VdbeAddOp2(v, OP_AddImm, iMem, 1);
76571
+ sqlite3VdbeAddOp2(v, OP_AddImm, iMem, 1); /* Increment row counter */
7596776572
7596876573
 for(i=0; i<nCol; i++){
7596976574
 CollSeq *pColl;
7597076575
 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regCol);
7597176576
 if( i==0 ){
75972
-#ifdef SQLITE_ENABLE_STAT2
75973
- /* Check if the record that cursor iIdxCur points to contains a
75974
- ** value that should be stored in the sqlite_stat2 table. If so,
75975
- ** store it. */
75976
- int ne = sqlite3VdbeAddOp3(v, OP_Ne, regRecno, 0, regSamplerecno);
75977
- assert( regTabname+1==regIdxname
75978
- && regTabname+2==regSampleno
75979
- && regTabname+3==regCol
75980
- );
75981
- sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL);
75982
- sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 4, regRec, "aaab", 0);
75983
- sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur+1, regRowid);
75984
- sqlite3VdbeAddOp3(v, OP_Insert, iStatCur+1, regRec, regRowid);
75985
-
75986
- /* Calculate new values for regSamplerecno and regSampleno.
75987
- **
75988
- ** sampleno = sampleno + 1
75989
- ** samplerecno = samplerecno+(remaining records)/(remaining samples)
75990
- */
75991
- sqlite3VdbeAddOp2(v, OP_AddImm, regSampleno, 1);
75992
- sqlite3VdbeAddOp3(v, OP_Subtract, regRecno, regLast, regTemp);
75993
- sqlite3VdbeAddOp2(v, OP_AddImm, regTemp, -1);
75994
- sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES, regTemp2);
75995
- sqlite3VdbeAddOp3(v, OP_Subtract, regSampleno, regTemp2, regTemp2);
75996
- sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regTemp, regTemp);
75997
- sqlite3VdbeAddOp3(v, OP_Add, regSamplerecno, regTemp, regSamplerecno);
75998
-
75999
- sqlite3VdbeJumpHere(v, ne);
76000
- sqlite3VdbeAddOp2(v, OP_AddImm, regRecno, 1);
76001
-#endif
76002
-
7600376577
 /* Always record the very first row */
76004
- sqlite3VdbeAddOp1(v, OP_IfNot, iMem+1);
76578
+ addrIfNot = sqlite3VdbeAddOp1(v, OP_IfNot, iMem+1);
7600576579
 }
7600676580
 assert( pIdx->azColl!=0 );
7600776581
 assert( pIdx->azColl[i]!=0 );
7600876582
 pColl = sqlite3LocateCollSeq(pParse, pIdx->azColl[i]);
76009
- sqlite3VdbeAddOp4(v, OP_Ne, regCol, 0, iMem+nCol+i+1,
76010
- (char*)pColl, P4_COLLSEQ);
76583
+ aChngAddr[i] = sqlite3VdbeAddOp4(v, OP_Ne, regCol, 0, iMem+nCol+i+1,
76584
+ (char*)pColl, P4_COLLSEQ);
7601176585
 sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
76012
- }
76013
- if( db->mallocFailed ){
76014
- /* If a malloc failure has occurred, then the result of the expression
76015
- ** passed as the second argument to the call to sqlite3VdbeJumpHere()
76016
- ** below may be negative. Which causes an assert() to fail (or an
76017
- ** out-of-bounds write if SQLITE_DEBUG is not defined). */
76018
- return;
76586
+ VdbeComment((v, "jump if column %d changed", i));
76587
+#ifdef SQLITE_ENABLE_STAT3
76588
+ if( i==0 ){
76589
+ sqlite3VdbeAddOp2(v, OP_AddImm, regNumEq, 1);
76590
+ VdbeComment((v, "incr repeat count"));
76591
+ }
76592
+#endif
7601976593
 }
7602076594
 sqlite3VdbeAddOp2(v, OP_Goto, 0, endOfLoop);
7602176595
 for(i=0; i<nCol; i++){
76022
- int addr2 = sqlite3VdbeCurrentAddr(v) - (nCol*2);
76596
+ sqlite3VdbeJumpHere(v, aChngAddr[i]); /* Set jump dest for the OP_Ne */
7602376597
 if( i==0 ){
76024
- sqlite3VdbeJumpHere(v, addr2-1); /* Set jump dest for the OP_IfNot */
76598
+ sqlite3VdbeJumpHere(v, addrIfNot); /* Jump dest for OP_IfNot */
76599
+#ifdef SQLITE_ENABLE_STAT3
76600
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regNumEq, regTemp2,
76601
+ (char*)&stat3PushFuncdef, P4_FUNCDEF);
76602
+ sqlite3VdbeChangeP5(v, 5);
76603
+ sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, pIdx->nColumn, regRowid);
76604
+ sqlite3VdbeAddOp3(v, OP_Add, regNumEq, regNumLt, regNumLt);
76605
+ sqlite3VdbeAddOp2(v, OP_AddImm, regNumDLt, 1);
76606
+ sqlite3VdbeAddOp2(v, OP_Integer, 1, regNumEq);
76607
+#endif
7602576608
 }
76026
- sqlite3VdbeJumpHere(v, addr2); /* Set jump dest for the OP_Ne */
7602776609
 sqlite3VdbeAddOp2(v, OP_AddImm, iMem+i+1, 1);
7602876610
 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, iMem+nCol+i+1);
7602976611
 }
76612
+ sqlite3DbFree(db, aChngAddr);
7603076613
76031
- /* End of the analysis loop. */
76614
+ /* Always jump here after updating the iMem+1...iMem+1+nCol counters */
7603276615
 sqlite3VdbeResolveLabel(v, endOfLoop);
76616
+
7603376617
 sqlite3VdbeAddOp2(v, OP_Next, iIdxCur, topOfLoop);
7603476618
 sqlite3VdbeAddOp1(v, OP_Close, iIdxCur);
76619
+#ifdef SQLITE_ENABLE_STAT3
76620
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regNumEq, regTemp2,
76621
+ (char*)&stat3PushFuncdef, P4_FUNCDEF);
76622
+ sqlite3VdbeChangeP5(v, 5);
76623
+ sqlite3VdbeAddOp2(v, OP_Integer, -1, regLoop);
76624
+ shortJump =
76625
+ sqlite3VdbeAddOp2(v, OP_AddImm, regLoop, 1);
76626
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regTemp1,
76627
+ (char*)&stat3GetFuncdef, P4_FUNCDEF);
76628
+ sqlite3VdbeChangeP5(v, 2);
76629
+ sqlite3VdbeAddOp1(v, OP_IsNull, regTemp1);
76630
+ sqlite3VdbeAddOp3(v, OP_NotExists, iTabCur, shortJump, regTemp1);
76631
+ sqlite3VdbeAddOp3(v, OP_Column, iTabCur, pIdx->aiColumn[0], regSample);
76632
+ sqlite3ColumnDefault(v, pTab, pIdx->aiColumn[0], regSample);
76633
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumEq,
76634
+ (char*)&stat3GetFuncdef, P4_FUNCDEF);
76635
+ sqlite3VdbeChangeP5(v, 3);
76636
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumLt,
76637
+ (char*)&stat3GetFuncdef, P4_FUNCDEF);
76638
+ sqlite3VdbeChangeP5(v, 4);
76639
+ sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumDLt,
76640
+ (char*)&stat3GetFuncdef, P4_FUNCDEF);
76641
+ sqlite3VdbeChangeP5(v, 5);
76642
+ sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 6, regRec, "bbbbbb", 0);
76643
+ sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur+1, regNewRowid);
76644
+ sqlite3VdbeAddOp3(v, OP_Insert, iStatCur+1, regRec, regNewRowid);
76645
+ sqlite3VdbeAddOp2(v, OP_Goto, 0, shortJump);
76646
+ sqlite3VdbeJumpHere(v, shortJump+2);
76647
+#endif
7603576648
7603676649
 /* Store the results in sqlite_stat1.
7603776650
 **
7603876651
 ** The result is a single row of the sqlite_stat1 table. The first
7603976652
 ** two columns are the names of the table and index. The third column
@@ -76049,50 +76662,51 @@
7604976662
 **
7605076663
 ** If K==0 then no entry is made into the sqlite_stat1 table.
7605176664
 ** If K>0 then it is always the case the D>0 so division by zero
7605276665
 ** is never possible.
7605376666
 */
76054
- sqlite3VdbeAddOp2(v, OP_SCopy, iMem, regSampleno);
76667
+ sqlite3VdbeAddOp2(v, OP_SCopy, iMem, regStat1);
7605576668
 if( jZeroRows<0 ){
7605676669
 jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, iMem);
7605776670
 }
7605876671
 for(i=0; i<nCol; i++){
7605976672
 sqlite3VdbeAddOp4(v, OP_String8, 0, regTemp, 0, " ", 0);
76060
- sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regSampleno, regSampleno);
76673
+ sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regStat1, regStat1);
7606176674
 sqlite3VdbeAddOp3(v, OP_Add, iMem, iMem+i+1, regTemp);
7606276675
 sqlite3VdbeAddOp2(v, OP_AddImm, regTemp, -1);
7606376676
 sqlite3VdbeAddOp3(v, OP_Divide, iMem+i+1, regTemp, regTemp);
7606476677
 sqlite3VdbeAddOp1(v, OP_ToInt, regTemp);
76065
- sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regSampleno, regSampleno);
76678
+ sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regStat1, regStat1);
7606676679
 }
7606776680
 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0);
76068
- sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regRowid);
76069
- sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regRowid);
76681
+ sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regNewRowid);
76682
+ sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regNewRowid);
7607076683
 sqlite3VdbeChangeP5(v, OPFLAG_APPEND);
7607176684
 }
7607276685
7607376686
 /* If the table has no indices, create a single sqlite_stat1 entry
7607476687
 ** containing NULL as the index name and the row count as the content.
7607576688
 */
7607676689
 if( pTab->pIndex==0 ){
7607776690
 sqlite3VdbeAddOp3(v, OP_OpenRead, iIdxCur, pTab->tnum, iDb);
7607876691
 VdbeComment((v, "%s", pTab->zName));
76079
- sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regSampleno);
76692
+ sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regStat1);
7608076693
 sqlite3VdbeAddOp1(v, OP_Close, iIdxCur);
76081
- jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, regSampleno);
76694
+ jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, regStat1);
7608276695
 }else{
7608376696
 sqlite3VdbeJumpHere(v, jZeroRows);
7608476697
 jZeroRows = sqlite3VdbeAddOp0(v, OP_Goto);
7608576698
 }
7608676699
 sqlite3VdbeAddOp2(v, OP_Null, 0, regIdxname);
7608776700
 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0);
76088
- sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regRowid);
76089
- sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regRowid);
76701
+ sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regNewRowid);
76702
+ sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regNewRowid);
7609076703
 sqlite3VdbeChangeP5(v, OPFLAG_APPEND);
7609176704
 if( pParse->nMem<regRec ) pParse->nMem = regRec;
7609276705
 sqlite3VdbeJumpHere(v, jZeroRows);
7609376706
 }
76707
+
7609476708
7609576709
 /*
7609676710
 ** Generate code that will cause the most recent index analysis to
7609776711
 ** be loaded into internal hash tables where is can be used.
7609876712
 */
@@ -76113,11 +76727,11 @@
7611376727
 int iStatCur;
7611476728
 int iMem;
7611576729
7611676730
 sqlite3BeginWriteOperation(pParse, 0, iDb);
7611776731
 iStatCur = pParse->nTab;
76118
- pParse->nTab += 2;
76732
+ pParse->nTab += 3;
7611976733
 openStatTable(pParse, iDb, iStatCur, 0, 0);
7612076734
 iMem = pParse->nMem+1;
7612176735
 assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
7612276736
 for(k=sqliteHashFirst(&pSchema->tblHash); k; k=sqliteHashNext(k)){
7612376737
 Table *pTab = (Table*)sqliteHashData(k);
@@ -76138,11 +76752,11 @@
7613876752
 assert( pTab!=0 );
7613976753
 assert( sqlite3BtreeHoldsAllMutexes(pParse->db) );
7614076754
 iDb = sqlite3SchemaToIndex(pParse->db, pTab->pSchema);
7614176755
 sqlite3BeginWriteOperation(pParse, 0, iDb);
7614276756
 iStatCur = pParse->nTab;
76143
- pParse->nTab += 2;
76757
+ pParse->nTab += 3;
7614476758
 if( pOnlyIdx ){
7614576759
 openStatTable(pParse, iDb, iStatCur, pOnlyIdx->zName, "idx");
7614676760
 }else{
7614776761
 openStatTable(pParse, iDb, iStatCur, pTab->zName, "tbl");
7614876762
 }
@@ -76243,11 +76857,11 @@
7624376857
 static int analysisLoader(void *pData, int argc, char **argv, char **NotUsed){
7624476858
 analysisInfo *pInfo = (analysisInfo*)pData;
7624576859
 Index *pIndex;
7624676860
 Table *pTable;
7624776861
 int i, c, n;
76248
- unsigned int v;
76862
+ tRowcnt v;
7624976863
 const char *z;
7625076864
7625176865
 assert( argc==3 );
7625276866
 UNUSED_PARAMETER2(NotUsed, argc);
7625376867
@@ -76286,40 +76900,172 @@
7628676900
 /*
7628776901
 ** If the Index.aSample variable is not NULL, delete the aSample[] array
7628876902
 ** and its contents.
7628976903
 */
7629076904
 SQLITE_PRIVATE void sqlite3DeleteIndexSamples(sqlite3 *db, Index *pIdx){
76291
-#ifdef SQLITE_ENABLE_STAT2
76905
+#ifdef SQLITE_ENABLE_STAT3
7629276906
 if( pIdx->aSample ){
7629376907
 int j;
76294
- for(j=0; j<SQLITE_INDEX_SAMPLES; j++){
76908
+ for(j=0; j<pIdx->nSample; j++){
7629576909
 IndexSample *p = &pIdx->aSample[j];
7629676910
 if( p->eType==SQLITE_TEXT || p->eType==SQLITE_BLOB ){
76297
- sqlite3DbFree(db, p->u.z);
76911
+ sqlite3_free(p->u.z);
7629876912
 }
7629976913
 }
76300
- sqlite3DbFree(db, pIdx->aSample);
76914
+ sqlite3_free(pIdx->aSample);
7630176915
 }
76916
+ UNUSED_PARAMETER(db);
76917
+ pIdx->nSample = 0;
76918
+ pIdx->aSample = 0;
7630276919
 #else
7630376920
 UNUSED_PARAMETER(db);
7630476921
 UNUSED_PARAMETER(pIdx);
7630576922
 #endif
7630676923
 }
7630776924
76925
+#ifdef SQLITE_ENABLE_STAT3
7630876926
 /*
76309
-** Load the content of the sqlite_stat1 and sqlite_stat2 tables. The
76927
+** Load content from the sqlite_stat3 table into the Index.aSample[]
76928
+** arrays of all indices.
76929
+*/
76930
+static int loadStat3(sqlite3 *db, const char *zDb){
76931
+ int rc; /* Result codes from subroutines */
76932
+ sqlite3_stmt *pStmt = 0; /* An SQL statement being run */
76933
+ char *zSql; /* Text of the SQL statement */
76934
+ Index *pPrevIdx = 0; /* Previous index in the loop */
76935
+ int idx = 0; /* slot in pIdx->aSample[] for next sample */
76936
+ int eType; /* Datatype of a sample */
76937
+ IndexSample *pSample; /* A slot in pIdx->aSample[] */
76938
+
76939
+ if( !sqlite3FindTable(db, "sqlite_stat3", zDb) ){
76940
+ return SQLITE_OK;
76941
+ }
76942
+
76943
+ zSql = sqlite3MPrintf(db,
76944
+ "SELECT idx,count(*) FROM %Q.sqlite_stat3"
76945
+ " GROUP BY idx", zDb);
76946
+ if( !zSql ){
76947
+ return SQLITE_NOMEM;
76948
+ }
76949
+ rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
76950
+ sqlite3DbFree(db, zSql);
76951
+ if( rc ) return rc;
76952
+
76953
+ while( sqlite3_step(pStmt)==SQLITE_ROW ){
76954
+ char *zIndex; /* Index name */
76955
+ Index *pIdx; /* Pointer to the index object */
76956
+ int nSample; /* Number of samples */
76957
+
76958
+ zIndex = (char *)sqlite3_column_text(pStmt, 0);
76959
+ if( zIndex==0 ) continue;
76960
+ nSample = sqlite3_column_int(pStmt, 1);
76961
+ if( nSample>255 ) continue;
76962
+ pIdx = sqlite3FindIndex(db, zIndex, zDb);
76963
+ if( pIdx==0 ) continue;
76964
+ assert( pIdx->nSample==0 );
76965
+ pIdx->nSample = (u8)nSample;
76966
+ pIdx->aSample = sqlite3MallocZero( nSample*sizeof(IndexSample) );
76967
+ pIdx->avgEq = pIdx->aiRowEst[1];
76968
+ if( pIdx->aSample==0 ){
76969
+ db->mallocFailed = 1;
76970
+ sqlite3_finalize(pStmt);
76971
+ return SQLITE_NOMEM;
76972
+ }
76973
+ }
76974
+ sqlite3_finalize(pStmt);
76975
+
76976
+ zSql = sqlite3MPrintf(db,
76977
+ "SELECT idx,neq,nlt,ndlt,sample FROM %Q.sqlite_stat3", zDb);
76978
+ if( !zSql ){
76979
+ return SQLITE_NOMEM;
76980
+ }
76981
+ rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
76982
+ sqlite3DbFree(db, zSql);
76983
+ if( rc ) return rc;
76984
+
76985
+ while( sqlite3_step(pStmt)==SQLITE_ROW ){
76986
+ char *zIndex; /* Index name */
76987
+ Index *pIdx; /* Pointer to the index object */
76988
+ int i; /* Loop counter */
76989
+ tRowcnt sumEq; /* Sum of the nEq values */
76990
+
76991
+ zIndex = (char *)sqlite3_column_text(pStmt, 0);
76992
+ if( zIndex==0 ) continue;
76993
+ pIdx = sqlite3FindIndex(db, zIndex, zDb);
76994
+ if( pIdx==0 ) continue;
76995
+ if( pIdx==pPrevIdx ){
76996
+ idx++;
76997
+ }else{
76998
+ pPrevIdx = pIdx;
76999
+ idx = 0;
77000
+ }
77001
+ assert( idx<pIdx->nSample );
77002
+ pSample = &pIdx->aSample[idx];
77003
+ pSample->nEq = (tRowcnt)sqlite3_column_int64(pStmt, 1);
77004
+ pSample->nLt = (tRowcnt)sqlite3_column_int64(pStmt, 2);
77005
+ pSample->nDLt = (tRowcnt)sqlite3_column_int64(pStmt, 3);
77006
+ if( idx==pIdx->nSample-1 ){
77007
+ if( pSample->nDLt>0 ){
77008
+ for(i=0, sumEq=0; i<=idx-1; i++) sumEq += pIdx->aSample[i].nEq;
77009
+ pIdx->avgEq = (pSample->nLt - sumEq)/pSample->nDLt;
77010
+ }
77011
+ if( pIdx->avgEq<=0 ) pIdx->avgEq = 1;
77012
+ }
77013
+ eType = sqlite3_column_type(pStmt, 4);
77014
+ pSample->eType = (u8)eType;
77015
+ switch( eType ){
77016
+ case SQLITE_INTEGER: {
77017
+ pSample->u.i = sqlite3_column_int64(pStmt, 4);
77018
+ break;
77019
+ }
77020
+ case SQLITE_FLOAT: {
77021
+ pSample->u.r = sqlite3_column_double(pStmt, 4);
77022
+ break;
77023
+ }
77024
+ case SQLITE_NULL: {
77025
+ break;
77026
+ }
77027
+ default: assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB ); {
77028
+ const char *z = (const char *)(
77029
+ (eType==SQLITE_BLOB) ?
77030
+ sqlite3_column_blob(pStmt, 4):
77031
+ sqlite3_column_text(pStmt, 4)
77032
+ );
77033
+ int n = sqlite3_column_bytes(pStmt, 4);
77034
+ if( n>0xffff ) n = 0xffff;
77035
+ pSample->nByte = (u16)n;
77036
+ if( n < 1){
77037
+ pSample->u.z = 0;
77038
+ }else{
77039
+ pSample->u.z = sqlite3Malloc(n);
77040
+ if( pSample->u.z==0 ){
77041
+ db->mallocFailed = 1;
77042
+ sqlite3_finalize(pStmt);
77043
+ return SQLITE_NOMEM;
77044
+ }
77045
+ memcpy(pSample->u.z, z, n);
77046
+ }
77047
+ }
77048
+ }
77049
+ }
77050
+ return sqlite3_finalize(pStmt);
77051
+}
77052
+#endif /* SQLITE_ENABLE_STAT3 */
77053
+
77054
+/*
77055
+** Load the content of the sqlite_stat1 and sqlite_stat3 tables. The
7631077056
 ** contents of sqlite_stat1 are used to populate the Index.aiRowEst[]
76311
-** arrays. The contents of sqlite_stat2 are used to populate the
77057
+** arrays. The contents of sqlite_stat3 are used to populate the
7631277058
 ** Index.aSample[] arrays.
7631377059
 **
7631477060
 ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR
76315
-** is returned. In this case, even if SQLITE_ENABLE_STAT2 was defined
76316
-** during compilation and the sqlite_stat2 table is present, no data is
77061
+** is returned. In this case, even if SQLITE_ENABLE_STAT3 was defined
77062
+** during compilation and the sqlite_stat3 table is present, no data is
7631777063
 ** read from it.
7631877064
 **
76319
-** If SQLITE_ENABLE_STAT2 was defined during compilation and the
76320
-** sqlite_stat2 table is not present in the database, SQLITE_ERROR is
77065
+** If SQLITE_ENABLE_STAT3 was defined during compilation and the
77066
+** sqlite_stat3 table is not present in the database, SQLITE_ERROR is
7632177067
 ** returned. However, in this case, data is read from the sqlite_stat1
7632277068
 ** table (if it is present) before returning.
7632377069
 **
7632477070
 ** If an OOM error occurs, this function always sets db->mallocFailed.
7632577071
 ** This means if the caller does not care about other errors, the return
@@ -76337,12 +77083,14 @@
7633777083
 /* Clear any prior statistics */
7633877084
 assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
7633977085
 for(i=sqliteHashFirst(&db->aDb[iDb].pSchema->idxHash);i;i=sqliteHashNext(i)){
7634077086
 Index *pIdx = sqliteHashData(i);
7634177087
 sqlite3DefaultRowEst(pIdx);
77088
+#ifdef SQLITE_ENABLE_STAT3
7634277089
 sqlite3DeleteIndexSamples(db, pIdx);
7634377090
 pIdx->aSample = 0;
77091
+#endif
7634477092
 }
7634577093
7634677094
 /* Check to make sure the sqlite_stat1 table exists */
7634777095
 sInfo.db = db;
7634877096
 sInfo.zDatabase = db->aDb[iDb].zName;
@@ -76350,91 +77098,23 @@
7635077098
 return SQLITE_ERROR;
7635177099
 }
7635277100
7635377101
 /* Load new statistics out of the sqlite_stat1 table */
7635477102
 zSql = sqlite3MPrintf(db,
76355
- "SELECT tbl, idx, stat FROM %Q.sqlite_stat1", sInfo.zDatabase);
77103
+ "SELECT tbl,idx,stat FROM %Q.sqlite_stat1", sInfo.zDatabase);
7635677104
 if( zSql==0 ){
7635777105
 rc = SQLITE_NOMEM;
7635877106
 }else{
7635977107
 rc = sqlite3_exec(db, zSql, analysisLoader, &sInfo, 0);
7636077108
 sqlite3DbFree(db, zSql);
7636177109
 }
7636277110
7636377111
76364
- /* Load the statistics from the sqlite_stat2 table. */
76365
-#ifdef SQLITE_ENABLE_STAT2
76366
- if( rc==SQLITE_OK && !sqlite3FindTable(db, "sqlite_stat2", sInfo.zDatabase) ){
76367
- rc = SQLITE_ERROR;
76368
- }
76369
- if( rc==SQLITE_OK ){
76370
- sqlite3_stmt *pStmt = 0;
76371
-
76372
- zSql = sqlite3MPrintf(db,
76373
- "SELECT idx,sampleno,sample FROM %Q.sqlite_stat2", sInfo.zDatabase);
76374
- if( !zSql ){
76375
- rc = SQLITE_NOMEM;
76376
- }else{
76377
- rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
76378
- sqlite3DbFree(db, zSql);
76379
- }
76380
-
76381
- if( rc==SQLITE_OK ){
76382
- while( sqlite3_step(pStmt)==SQLITE_ROW ){
76383
- char *zIndex; /* Index name */
76384
- Index *pIdx; /* Pointer to the index object */
76385
-
76386
- zIndex = (char *)sqlite3_column_text(pStmt, 0);
76387
- pIdx = zIndex ? sqlite3FindIndex(db, zIndex, sInfo.zDatabase) : 0;
76388
- if( pIdx ){
76389
- int iSample = sqlite3_column_int(pStmt, 1);
76390
- if( iSample<SQLITE_INDEX_SAMPLES && iSample>=0 ){
76391
- int eType = sqlite3_column_type(pStmt, 2);
76392
-
76393
- if( pIdx->aSample==0 ){
76394
- static const int sz = sizeof(IndexSample)*SQLITE_INDEX_SAMPLES;
76395
- pIdx->aSample = (IndexSample *)sqlite3DbMallocRaw(0, sz);
76396
- if( pIdx->aSample==0 ){
76397
- db->mallocFailed = 1;
76398
- break;
76399
- }
76400
- memset(pIdx->aSample, 0, sz);
76401
- }
76402
-
76403
- assert( pIdx->aSample );
76404
- {
76405
- IndexSample *pSample = &pIdx->aSample[iSample];
76406
- pSample->eType = (u8)eType;
76407
- if( eType==SQLITE_INTEGER || eType==SQLITE_FLOAT ){
76408
- pSample->u.r = sqlite3_column_double(pStmt, 2);
76409
- }else if( eType==SQLITE_TEXT || eType==SQLITE_BLOB ){
76410
- const char *z = (const char *)(
76411
- (eType==SQLITE_BLOB) ?
76412
- sqlite3_column_blob(pStmt, 2):
76413
- sqlite3_column_text(pStmt, 2)
76414
- );
76415
- int n = sqlite3_column_bytes(pStmt, 2);
76416
- if( n>24 ){
76417
- n = 24;
76418
- }
76419
- pSample->nByte = (u8)n;
76420
- if( n < 1){
76421
- pSample->u.z = 0;
76422
- }else{
76423
- pSample->u.z = sqlite3DbStrNDup(0, z, n);
76424
- if( pSample->u.z==0 ){
76425
- db->mallocFailed = 1;
76426
- break;
76427
- }
76428
- }
76429
- }
76430
- }
76431
- }
76432
- }
76433
- }
76434
- rc = sqlite3_finalize(pStmt);
76435
- }
77112
+ /* Load the statistics from the sqlite_stat3 table. */
77113
+#ifdef SQLITE_ENABLE_STAT3
77114
+ if( rc==SQLITE_OK ){
77115
+ rc = loadStat3(db, sInfo.zDatabase);
7643677116
 }
7643777117
 #endif
7643877118
7643977119
 if( rc==SQLITE_NOMEM ){
7644077120
 db->mallocFailed = 1;
@@ -78875,12 +79555,12 @@
7887579555
 }
7887679556
 }
7887779557
 #endif
7887879558
7887979559
 /* Reparse everything to update our internal data structures */
78880
- sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0,
78881
- sqlite3MPrintf(db, "tbl_name='%q'",p->zName), P4_DYNAMIC);
79560
+ sqlite3VdbeAddParseSchemaOp(v, iDb,
79561
+ sqlite3MPrintf(db, "tbl_name='%q'", p->zName));
7888279562
 }
7888379563
7888479564
7888579565
 /* Add the table to the in-memory representation of the database.
7888679566
 */
@@ -79235,10 +79915,108 @@
7923579915
 iDestroyed = iLargest;
7923679916
 }
7923779917
 }
7923879918
 #endif
7923979919
 }
79920
+
79921
+/*
79922
+** Remove entries from the sqlite_stat1 and sqlite_stat2 tables
79923
+** after a DROP INDEX or DROP TABLE command.
79924
+*/
79925
+static void sqlite3ClearStatTables(
79926
+ Parse *pParse, /* The parsing context */
79927
+ int iDb, /* The database number */
79928
+ const char *zType, /* "idx" or "tbl" */
79929
+ const char *zName /* Name of index or table */
79930
+){
79931
+ static const char *azStatTab[] = {
79932
+ "sqlite_stat1",
79933
+ "sqlite_stat2",
79934
+ "sqlite_stat3",
79935
+ };
79936
+ int i;
79937
+ const char *zDbName = pParse->db->aDb[iDb].zName;
79938
+ for(i=0; i<ArraySize(azStatTab); i++){
79939
+ if( sqlite3FindTable(pParse->db, azStatTab[i], zDbName) ){
79940
+ sqlite3NestedParse(pParse,
79941
+ "DELETE FROM %Q.%s WHERE %s=%Q",
79942
+ zDbName, azStatTab[i], zType, zName
79943
+ );
79944
+ }
79945
+ }
79946
+}
79947
+
79948
+/*
79949
+** Generate code to drop a table.
79950
+*/
79951
+SQLITE_PRIVATE void sqlite3CodeDropTable(Parse *pParse, Table *pTab, int iDb, int isView){
79952
+ Vdbe *v;
79953
+ sqlite3 *db = pParse->db;
79954
+ Trigger *pTrigger;
79955
+ Db *pDb = &db->aDb[iDb];
79956
+
79957
+ v = sqlite3GetVdbe(pParse);
79958
+ assert( v!=0 );
79959
+ sqlite3BeginWriteOperation(pParse, 1, iDb);
79960
+
79961
+#ifndef SQLITE_OMIT_VIRTUALTABLE
79962
+ if( IsVirtual(pTab) ){
79963
+ sqlite3VdbeAddOp0(v, OP_VBegin);
79964
+ }
79965
+#endif
79966
+
79967
+ /* Drop all triggers associated with the table being dropped. Code
79968
+ ** is generated to remove entries from sqlite_master and/or
79969
+ ** sqlite_temp_master if required.
79970
+ */
79971
+ pTrigger = sqlite3TriggerList(pParse, pTab);
79972
+ while( pTrigger ){
79973
+ assert( pTrigger->pSchema==pTab->pSchema ||
79974
+ pTrigger->pSchema==db->aDb[1].pSchema );
79975
+ sqlite3DropTriggerPtr(pParse, pTrigger);
79976
+ pTrigger = pTrigger->pNext;
79977
+ }
79978
+
79979
+#ifndef SQLITE_OMIT_AUTOINCREMENT
79980
+ /* Remove any entries of the sqlite_sequence table associated with
79981
+ ** the table being dropped. This is done before the table is dropped
79982
+ ** at the btree level, in case the sqlite_sequence table needs to
79983
+ ** move as a result of the drop (can happen in auto-vacuum mode).
79984
+ */
79985
+ if( pTab->tabFlags & TF_Autoincrement ){
79986
+ sqlite3NestedParse(pParse,
79987
+ "DELETE FROM %Q.sqlite_sequence WHERE name=%Q",
79988
+ pDb->zName, pTab->zName
79989
+ );
79990
+ }
79991
+#endif
79992
+
79993
+ /* Drop all SQLITE_MASTER table and index entries that refer to the
79994
+ ** table. The program name loops through the master table and deletes
79995
+ ** every row that refers to a table of the same name as the one being
79996
+ ** dropped. Triggers are handled seperately because a trigger can be
79997
+ ** created in the temp database that refers to a table in another
79998
+ ** database.
79999
+ */
80000
+ sqlite3NestedParse(pParse,
80001
+ "DELETE FROM %Q.%s WHERE tbl_name=%Q and type!='trigger'",
80002
+ pDb->zName, SCHEMA_TABLE(iDb), pTab->zName);
80003
+ if( !isView && !IsVirtual(pTab) ){
80004
+ destroyTable(pParse, pTab);
80005
+ }
80006
+
80007
+ /* Remove the table entry from SQLite's internal schema and modify
80008
+ ** the schema cookie.
80009
+ */
80010
+ if( IsVirtual(pTab) ){
80011
+ sqlite3VdbeAddOp4(v, OP_VDestroy, iDb, 0, 0, pTab->zName, 0);
80012
+ }
80013
+ sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
80014
+ sqlite3ChangeCookie(pParse, iDb);
80015
+ sqliteViewResetAll(db, iDb);
80016
+
80017
+}
7924080018
7924180019
 /*
7924280020
 ** This routine is called to do the work of a DROP TABLE statement.
7924380021
 ** pName is the name of the table to be dropped.
7924480022
 */
@@ -79304,11 +80082,11 @@
7930480082
 if( sqlite3AuthCheck(pParse, SQLITE_DELETE, pTab->zName, 0, zDb) ){
7930580083
 goto exit_drop_table;
7930680084
 }
7930780085
 }
7930880086
 #endif
79309
- if( sqlite3StrNICmp(pTab->zName, "sqlite_", 7)==0 ){
80087
+ if( !pParse->nested && sqlite3StrNICmp(pTab->zName, "sqlite_", 7)==0 ){
7931080088
 sqlite3ErrorMsg(pParse, "table %s may not be dropped", pTab->zName);
7931180089
 goto exit_drop_table;
7931280090
 }
7931380091
7931480092
 #ifndef SQLITE_OMIT_VIEW
@@ -79328,79 +80106,15 @@
7932880106
 /* Generate code to remove the table from the master table
7932980107
 ** on disk.
7933080108
 */
7933180109
 v = sqlite3GetVdbe(pParse);
7933280110
 if( v ){
79333
- Trigger *pTrigger;
79334
- Db *pDb = &db->aDb[iDb];
7933580111
 sqlite3BeginWriteOperation(pParse, 1, iDb);
79336
-
79337
-#ifndef SQLITE_OMIT_VIRTUALTABLE
79338
- if( IsVirtual(pTab) ){
79339
- sqlite3VdbeAddOp0(v, OP_VBegin);
79340
- }
79341
-#endif
7934280112
 sqlite3FkDropTable(pParse, pName, pTab);
79343
-
79344
- /* Drop all triggers associated with the table being dropped. Code
79345
- ** is generated to remove entries from sqlite_master and/or
79346
- ** sqlite_temp_master if required.
79347
- */
79348
- pTrigger = sqlite3TriggerList(pParse, pTab);
79349
- while( pTrigger ){
79350
- assert( pTrigger->pSchema==pTab->pSchema ||
79351
- pTrigger->pSchema==db->aDb[1].pSchema );
79352
- sqlite3DropTriggerPtr(pParse, pTrigger);
79353
- pTrigger = pTrigger->pNext;
79354
- }
79355
-
79356
-#ifndef SQLITE_OMIT_AUTOINCREMENT
79357
- /* Remove any entries of the sqlite_sequence table associated with
79358
- ** the table being dropped. This is done before the table is dropped
79359
- ** at the btree level, in case the sqlite_sequence table needs to
79360
- ** move as a result of the drop (can happen in auto-vacuum mode).
79361
- */
79362
- if( pTab->tabFlags & TF_Autoincrement ){
79363
- sqlite3NestedParse(pParse,
79364
- "DELETE FROM %s.sqlite_sequence WHERE name=%Q",
79365
- pDb->zName, pTab->zName
79366
- );
79367
- }
79368
-#endif
79369
-
79370
- /* Drop all SQLITE_MASTER table and index entries that refer to the
79371
- ** table. The program name loops through the master table and deletes
79372
- ** every row that refers to a table of the same name as the one being
79373
- ** dropped. Triggers are handled seperately because a trigger can be
79374
- ** created in the temp database that refers to a table in another
79375
- ** database.
79376
- */
79377
- sqlite3NestedParse(pParse,
79378
- "DELETE FROM %Q.%s WHERE tbl_name=%Q and type!='trigger'",
79379
- pDb->zName, SCHEMA_TABLE(iDb), pTab->zName);
79380
-
79381
- /* Drop any statistics from the sqlite_stat1 table, if it exists */
79382
- if( sqlite3FindTable(db, "sqlite_stat1", db->aDb[iDb].zName) ){
79383
- sqlite3NestedParse(pParse,
79384
- "DELETE FROM %Q.sqlite_stat1 WHERE tbl=%Q", pDb->zName, pTab->zName
79385
- );
79386
- }
79387
-
79388
- if( !isView && !IsVirtual(pTab) ){
79389
- destroyTable(pParse, pTab);
79390
- }
79391
-
79392
- /* Remove the table entry from SQLite's internal schema and modify
79393
- ** the schema cookie.
79394
- */
79395
- if( IsVirtual(pTab) ){
79396
- sqlite3VdbeAddOp4(v, OP_VDestroy, iDb, 0, 0, pTab->zName, 0);
79397
- }
79398
- sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
79399
- sqlite3ChangeCookie(pParse, iDb);
79400
- }
79401
- sqliteViewResetAll(db, iDb);
80113
+ sqlite3CodeDropTable(pParse, pTab, iDb, isView);
80114
+ sqlite3ClearStatTables(pParse, iDb, "tbl", pTab->zName);
80115
+ }
7940280116
7940380117
 exit_drop_table:
7940480118
 sqlite3SrcListDelete(db, pName);
7940580119
 }
7940680120
@@ -79843,24 +80557,24 @@
7984380557
 */
7984480558
 nName = sqlite3Strlen30(zName);
7984580559
 nCol = pList->nExpr;
7984680560
 pIndex = sqlite3DbMallocZero(db,
7984780561
 sizeof(Index) + /* Index structure */
80562
+ sizeof(tRowcnt)*(nCol+1) + /* Index.aiRowEst */
7984880563
 sizeof(int)*nCol + /* Index.aiColumn */
79849
- sizeof(int)*(nCol+1) + /* Index.aiRowEst */
7985080564
 sizeof(char *)*nCol + /* Index.azColl */
7985180565
 sizeof(u8)*nCol + /* Index.aSortOrder */
7985280566
 nName + 1 + /* Index.zName */
7985380567
 nExtra /* Collation sequence names */
7985480568
 );
7985580569
 if( db->mallocFailed ){
7985680570
 goto exit_create_index;
7985780571
 }
79858
- pIndex->azColl = (char**)(&pIndex[1]);
80572
+ pIndex->aiRowEst = (tRowcnt*)(&pIndex[1]);
80573
+ pIndex->azColl = (char**)(&pIndex->aiRowEst[nCol+1]);
7985980574
 pIndex->aiColumn = (int *)(&pIndex->azColl[nCol]);
79860
- pIndex->aiRowEst = (unsigned *)(&pIndex->aiColumn[nCol]);
79861
- pIndex->aSortOrder = (u8 *)(&pIndex->aiRowEst[nCol+1]);
80575
+ pIndex->aSortOrder = (u8 *)(&pIndex->aiColumn[nCol]);
7986280576
 pIndex->zName = (char *)(&pIndex->aSortOrder[nCol]);
7986380577
 zExtra = (char *)(&pIndex->zName[nName+1]);
7986480578
 memcpy(pIndex->zName, zName, nName+1);
7986580579
 pIndex->pTable = pTab;
7986680580
 pIndex->nColumn = pList->nExpr;
@@ -80047,11 +80761,11 @@
8004780761
 if( pStart ){
8004880762
 assert( pEnd!=0 );
8004980763
 /* A named index with an explicit CREATE INDEX statement */
8005080764
 zStmt = sqlite3MPrintf(db, "CREATE%s INDEX %.*s",
8005180765
 onError==OE_None ? "" : " UNIQUE",
80052
- pEnd->z - pName->z + 1,
80766
+ (int)(pEnd->z - pName->z) + 1,
8005380767
 pName->z);
8005480768
 }else{
8005580769
 /* An automatic index created by a PRIMARY KEY or UNIQUE constraint */
8005680770
 /* zStmt = sqlite3MPrintf(""); */
8005780771
 zStmt = 0;
@@ -80073,13 +80787,12 @@
8007380787
 ** to invalidate all pre-compiled statements.
8007480788
 */
8007580789
 if( pTblName ){
8007680790
 sqlite3RefillIndex(pParse, pIndex, iMem);
8007780791
 sqlite3ChangeCookie(pParse, iDb);
80078
- sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0,
80079
- sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName),
80080
- P4_DYNAMIC);
80792
+ sqlite3VdbeAddParseSchemaOp(v, iDb,
80793
+ sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName));
8008180794
 sqlite3VdbeAddOp1(v, OP_Expire, 0);
8008280795
 }
8008380796
 }
8008480797
8008580798
 /* When adding an index to the list of indices for a table, make
@@ -80134,13 +80847,13 @@
8013480847
 ** Apart from that, we have little to go on besides intuition as to
8013580848
 ** how aiRowEst[] should be initialized. The numbers generated here
8013680849
 ** are based on typical values found in actual indices.
8013780850
 */
8013880851
 SQLITE_PRIVATE void sqlite3DefaultRowEst(Index *pIdx){
80139
- unsigned *a = pIdx->aiRowEst;
80852
+ tRowcnt *a = pIdx->aiRowEst;
8014080853
 int i;
80141
- unsigned n;
80854
+ tRowcnt n;
8014280855
 assert( a!=0 );
8014380856
 a[0] = pIdx->pTable->nRowEst;
8014480857
 if( a[0]<10 ) a[0] = 10;
8014580858
 n = 10;
8014680859
 for(i=1; i<=pIdx->nColumn; i++){
@@ -80206,19 +80919,13 @@
8020680919
 v = sqlite3GetVdbe(pParse);
8020780920
 if( v ){
8020880921
 sqlite3BeginWriteOperation(pParse, 1, iDb);
8020980922
 sqlite3NestedParse(pParse,
8021080923
 "DELETE FROM %Q.%s WHERE name=%Q AND type='index'",
80211
- db->aDb[iDb].zName, SCHEMA_TABLE(iDb),
80212
- pIndex->zName
80213
- );
80214
- if( sqlite3FindTable(db, "sqlite_stat1", db->aDb[iDb].zName) ){
80215
- sqlite3NestedParse(pParse,
80216
- "DELETE FROM %Q.sqlite_stat1 WHERE idx=%Q",
80217
- db->aDb[iDb].zName, pIndex->zName
80218
- );
80219
- }
80924
+ db->aDb[iDb].zName, SCHEMA_TABLE(iDb), pIndex->zName
80925
+ );
80926
+ sqlite3ClearStatTables(pParse, iDb, "idx", pIndex->zName);
8022080927
 sqlite3ChangeCookie(pParse, iDb);
8022180928
 destroyRootPage(pParse, pIndex->tnum, iDb);
8022280929
 sqlite3VdbeAddOp4(v, OP_DropIndex, iDb, 0, 0, pIndex->zName, 0);
8022380930
 }
8022480931
@@ -81843,11 +82550,13 @@
8184382550
 int regRowid; /* Actual register containing rowids */
8184482551
8184582552
 /* Collect rowids of every row to be deleted.
8184682553
 */
8184782554
 sqlite3VdbeAddOp2(v, OP_Null, 0, iRowSet);
81848
- pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere,0,WHERE_DUPLICATES_OK);
82555
+ pWInfo = sqlite3WhereBegin(
82556
+ pParse, pTabList, pWhere, 0, 0, WHERE_DUPLICATES_OK
82557
+ );
8184982558
 if( pWInfo==0 ) goto delete_from_cleanup;
8185082559
 regRowid = sqlite3ExprCodeGetColumn(pParse, pTab, -1, iCur, iRowid);
8185182560
 sqlite3VdbeAddOp2(v, OP_RowSetAdd, iRowSet, regRowid);
8185282561
 if( db->flags & SQLITE_CountRows ){
8185382562
 sqlite3VdbeAddOp2(v, OP_AddImm, memCnt, 1);
@@ -82108,12 +82817,18 @@
8210882817
 sqlite3VdbeAddOp3(v, OP_Column, iCur, idx, regBase+j);
8210982818
 sqlite3ColumnDefault(v, pTab, idx, -1);
8211082819
 }
8211182820
 }
8211282821
 if( doMakeRec ){
82822
+ const char *zAff;
82823
+ if( pTab->pSelect || (pParse->db->flags & SQLITE_IdxRealAsInt)!=0 ){
82824
+ zAff = 0;
82825
+ }else{
82826
+ zAff = sqlite3IndexAffinityStr(v, pIdx);
82827
+ }
8211382828
 sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol+1, regOut);
82114
- sqlite3VdbeChangeP4(v, -1, sqlite3IndexAffinityStr(v, pIdx), P4_TRANSIENT);
82829
+ sqlite3VdbeChangeP4(v, -1, zAff, P4_TRANSIENT);
8211582830
 }
8211682831
 sqlite3ReleaseTempRange(pParse, regBase, nCol+1);
8211782832
 return regBase;
8211882833
 }
8211982834
@@ -82621,14 +83336,14 @@
8262183336
 ** character is exactly one byte in size. Also, all characters are
8262283337
 ** able to participate in upper-case-to-lower-case mappings in EBCDIC
8262383338
 ** whereas only characters less than 0x80 do in ASCII.
8262483339
 */
8262583340
 #if defined(SQLITE_EBCDIC)
82626
-# define sqlite3Utf8Read(A,C) (*(A++))
82627
-# define GlogUpperToLower(A) A = sqlite3UpperToLower[A]
83341
+# define sqlite3Utf8Read(A,C) (*(A++))
83342
+# define GlogUpperToLower(A) A = sqlite3UpperToLower[A]
8262883343
 #else
82629
-# define GlogUpperToLower(A) if( A<0x80 ){ A = sqlite3UpperToLower[A]; }
83344
+# define GlogUpperToLower(A) if( !((A)&~0x7f) ){ A = sqlite3UpperToLower[A]; }
8263083345
 #endif
8263183346
8263283347
 static const struct compareInfo globInfo = { '*', '?', '[', 0 };
8263383348
 /* The correct SQL-92 behavior is for the LIKE operator to ignore
8263483349
 ** case. Thus 'a' LIKE 'A' would be true. */
@@ -82667,13 +83382,13 @@
8266783382
 */
8266883383
 static int patternCompare(
8266983384
 const u8 *zPattern, /* The glob pattern */
8267083385
 const u8 *zString, /* The string to compare against the glob */
8267183386
 const struct compareInfo *pInfo, /* Information about how to do the compare */
82672
- const int esc /* The escape character */
83387
+ u32 esc /* The escape character */
8267383388
 ){
82674
- int c, c2;
83389
+ u32 c, c2;
8267583390
 int invert;
8267683391
 int seen;
8267783392
 u8 matchOne = pInfo->matchOne;
8267883393
 u8 matchAll = pInfo->matchAll;
8267983394
 u8 matchSet = pInfo->matchSet;
@@ -82723,11 +83438,11 @@
8272383438
 }else if( !prevEscape && c==matchOne ){
8272483439
 if( sqlite3Utf8Read(zString, &zString)==0 ){
8272583440
 return 0;
8272683441
 }
8272783442
 }else if( c==matchSet ){
82728
- int prior_c = 0;
83443
+ u32 prior_c = 0;
8272983444
 assert( esc==0 ); /* This only occurs for GLOB, not LIKE */
8273083445
 seen = 0;
8273183446
 invert = 0;
8273283447
 c = sqlite3Utf8Read(zString, &zString);
8273383448
 if( c==0 ) return 0;
@@ -82799,11 +83514,11 @@
8279983514
 sqlite3_context *context,
8280083515
 int argc,
8280183516
 sqlite3_value **argv
8280283517
 ){
8280383518
 const unsigned char *zA, *zB;
82804
- int escape = 0;
83519
+ u32 escape = 0;
8280583520
 int nPat;
8280683521
 sqlite3 *db = sqlite3_context_db_handle(context);
8280783522
8280883523
 zB = sqlite3_value_text(argv[0]);
8280983524
 zA = sqlite3_value_text(argv[1]);
@@ -84110,17 +84825,29 @@
8411084825
 }
8411184826
8411284827
 /* If the parent table is the same as the child table, and we are about
8411384828
 ** to increment the constraint-counter (i.e. this is an INSERT operation),
8411484829
 ** then check if the row being inserted matches itself. If so, do not
84115
- ** increment the constraint-counter. */
84830
+ ** increment the constraint-counter.
84831
+ **
84832
+ ** If any of the parent-key values are NULL, then the row cannot match
84833
+ ** itself. So set JUMPIFNULL to make sure we do the OP_Found if any
84834
+ ** of the parent-key values are NULL (at this point it is known that
84835
+ ** none of the child key values are).
84836
+ */
8411684837
 if( pTab==pFKey->pFrom && nIncr==1 ){
8411784838
 int iJump = sqlite3VdbeCurrentAddr(v) + nCol + 1;
8411884839
 for(i=0; i<nCol; i++){
8411984840
 int iChild = aiCol[i]+1+regData;
8412084841
 int iParent = pIdx->aiColumn[i]+1+regData;
84842
+ assert( aiCol[i]!=pTab->iPKey );
84843
+ if( pIdx->aiColumn[i]==pTab->iPKey ){
84844
+ /* The parent key is a composite key that includes the IPK column */
84845
+ iParent = regData;
84846
+ }
8412184847
 sqlite3VdbeAddOp3(v, OP_Ne, iChild, iJump, iParent);
84848
+ sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL);
8412284849
 }
8412384850
 sqlite3VdbeAddOp2(v, OP_Goto, 0, iOk);
8412484851
 }
8412584852
8412684853
 sqlite3VdbeAddOp3(v, OP_MakeRecord, regTemp, nCol, regRec);
@@ -84272,11 +84999,11 @@
8427284999
8427385000
 /* Create VDBE to loop through the entries in pSrc that match the WHERE
8427485001
 ** clause. If the constraint is not deferred, throw an exception for
8427585002
 ** each row found. Otherwise, for deferred constraints, increment the
8427685003
 ** deferred constraint counter by nIncr for each row selected. */
84277
- pWInfo = sqlite3WhereBegin(pParse, pSrc, pWhere, 0, 0);
85004
+ pWInfo = sqlite3WhereBegin(pParse, pSrc, pWhere, 0, 0, 0);
8427885005
 if( nIncr>0 && pFKey->isDeferred==0 ){
8427985006
 sqlite3ParseToplevel(pParse)->mayAbort = 1;
8428085007
 }
8428185008
 sqlite3VdbeAddOp2(v, OP_FkCounter, pFKey->isDeferred, nIncr);
8428285009
 if( pWInfo ){
@@ -87139,10 +87866,13 @@
8713987866
 int (*strnicmp)(const char*,const char*,int);
8714087867
 int (*unlock_notify)(sqlite3*,void(*)(void**,int),void*);
8714187868
 int (*wal_autocheckpoint)(sqlite3*,int);
8714287869
 int (*wal_checkpoint)(sqlite3*,const char*);
8714387870
 void *(*wal_hook)(sqlite3*,int(*)(void*,sqlite3*,const char*,int),void*);
87871
+ int (*blob_reopen)(sqlite3_blob*,sqlite3_int64);
87872
+ int (*vtab_config)(sqlite3*,int op,...);
87873
+ int (*vtab_on_conflict)(sqlite3*);
8714487874
 };
8714587875
8714687876
 /*
8714787877
 ** The following macros redefine the API routines so that they are
8714887878
 ** redirected throught the global sqlite3_api structure.
@@ -87339,10 +88069,13 @@
8733988069
 #define sqlite3_strnicmp sqlite3_api->strnicmp
8734088070
 #define sqlite3_unlock_notify sqlite3_api->unlock_notify
8734188071
 #define sqlite3_wal_autocheckpoint sqlite3_api->wal_autocheckpoint
8734288072
 #define sqlite3_wal_checkpoint sqlite3_api->wal_checkpoint
8734388073
 #define sqlite3_wal_hook sqlite3_api->wal_hook
88074
+#define sqlite3_blob_reopen sqlite3_api->blob_reopen
88075
+#define sqlite3_vtab_config sqlite3_api->vtab_config
88076
+#define sqlite3_vtab_on_conflict sqlite3_api->vtab_on_conflict
8734488077
 #endif /* SQLITE_CORE */
8734588078
8734688079
 #define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api = 0;
8734788080
 #define SQLITE_EXTENSION_INIT2(v) sqlite3_api = v;
8734888081
@@ -87413,10 +88146,12 @@
8741388146
8741488147
 #ifdef SQLITE_OMIT_VIRTUALTABLE
8741588148
 # define sqlite3_create_module 0
8741688149
 # define sqlite3_create_module_v2 0
8741788150
 # define sqlite3_declare_vtab 0
88151
+# define sqlite3_vtab_config 0
88152
+# define sqlite3_vtab_on_conflict 0
8741888153
 #endif
8741988154
8742088155
 #ifdef SQLITE_OMIT_SHARED_CACHE
8742188156
 # define sqlite3_enable_shared_cache 0
8742288157
 #endif
@@ -87436,10 +88171,11 @@
8743688171
 #define sqlite3_blob_bytes 0
8743788172
 #define sqlite3_blob_close 0
8743888173
 #define sqlite3_blob_open 0
8743988174
 #define sqlite3_blob_read 0
8744088175
 #define sqlite3_blob_write 0
88176
+#define sqlite3_blob_reopen 0
8744188177
 #endif
8744288178
8744388179
 /*
8744488180
 ** The following structure contains pointers to all SQLite API routines.
8744588181
 ** A pointer to this structure is passed into extensions when they are
@@ -87701,10 +88437,13 @@
8770188437
 #else
8770288438
 0,
8770388439
 0,
8770488440
 0,
8770588441
 #endif
88442
+ sqlite3_blob_reopen,
88443
+ sqlite3_vtab_config,
88444
+ sqlite3_vtab_on_conflict,
8770688445
 };
8770788446
8770888447
 /*
8770988448
 ** Attempt to load an SQLite extension library contained in the file
8771088449
 ** zFile. The entry point is zProc. zProc may be 0 in which case a
@@ -94090,10 +94829,11 @@
9409094829
 Expr *pHaving; /* The HAVING clause. May be NULL */
9409194830
 int isDistinct; /* True if the DISTINCT keyword is present */
9409294831
 int distinct; /* Table to use for the distinct set */
9409394832
 int rc = 1; /* Value to return from this function */
9409494833
 int addrSortIndex; /* Address of an OP_OpenEphemeral instruction */
94834
+ int addrDistinctIndex; /* Address of an OP_OpenEphemeral instruction */
9409594835
 AggInfo sAggInfo; /* Information used by aggregate queries */
9409694836
 int iEnd; /* Address of the end of the query */
9409794837
 sqlite3 *db; /* The database connection */
9409894838
9409994839
 #ifndef SQLITE_OMIT_EXPLAIN
@@ -94216,20 +94956,10 @@
9421694956
 explainSetInteger(pParse->iSelectId, iRestoreSelectId);
9421794957
 return rc;
9421894958
 }
9421994959
 #endif
9422094960
94221
- /* If possible, rewrite the query to use GROUP BY instead of DISTINCT.
94222
- ** GROUP BY might use an index, DISTINCT never does.
94223
- */
94224
- assert( p->pGroupBy==0 || (p->selFlags & SF_Aggregate)!=0 );
94225
- if( (p->selFlags & (SF_Distinct|SF_Aggregate))==SF_Distinct ){
94226
- p->pGroupBy = sqlite3ExprListDup(db, p->pEList, 0);
94227
- pGroupBy = p->pGroupBy;
94228
- p->selFlags &= ~SF_Distinct;
94229
- }
94230
-
9423194961
 /* If there is both a GROUP BY and an ORDER BY clause and they are
9423294962
 ** identical, then disable the ORDER BY clause since the GROUP BY
9423394963
 ** will cause elements to come out in the correct order. This is
9423494964
 ** an optimization - the correct answer should result regardless.
9423594965
 ** Use the SQLITE_GroupByOrder flag with SQLITE_TESTCTRL_OPTIMIZER
@@ -94237,10 +94967,34 @@
9423794967
 */
9423894968
 if( sqlite3ExprListCompare(p->pGroupBy, pOrderBy)==0
9423994969
 && (db->flags & SQLITE_GroupByOrder)==0 ){
9424094970
 pOrderBy = 0;
9424194971
 }
94972
+
94973
+ /* If the query is DISTINCT with an ORDER BY but is not an aggregate, and
94974
+ ** if the select-list is the same as the ORDER BY list, then this query
94975
+ ** can be rewritten as a GROUP BY. In other words, this:
94976
+ **
94977
+ ** SELECT DISTINCT xyz FROM ... ORDER BY xyz
94978
+ **
94979
+ ** is transformed to:
94980
+ **
94981
+ ** SELECT xyz FROM ... GROUP BY xyz
94982
+ **
94983
+ ** The second form is preferred as a single index (or temp-table) may be
94984
+ ** used for both the ORDER BY and DISTINCT processing. As originally
94985
+ ** written the query must use a temp-table for at least one of the ORDER
94986
+ ** BY and DISTINCT, and an index or separate temp-table for the other.
94987
+ */
94988
+ if( (p->selFlags & (SF_Distinct|SF_Aggregate))==SF_Distinct
94989
+ && sqlite3ExprListCompare(pOrderBy, p->pEList)==0
94990
+ ){
94991
+ p->selFlags &= ~SF_Distinct;
94992
+ p->pGroupBy = sqlite3ExprListDup(db, p->pEList, 0);
94993
+ pGroupBy = p->pGroupBy;
94994
+ pOrderBy = 0;
94995
+ }
9424294996
9424394997
 /* If there is an ORDER BY clause, then this sorting
9424494998
 ** index might end up being unused if the data can be
9424594999
 ** extracted in pre-sorted order. If that is the case, then the
9424695000
 ** OP_OpenEphemeral instruction will be changed to an OP_Noop once
@@ -94273,26 +95027,25 @@
9427395027
9427495028
 /* Open a virtual index to use for the distinct set.
9427595029
 */
9427695030
 if( p->selFlags & SF_Distinct ){
9427795031
 KeyInfo *pKeyInfo;
94278
- assert( isAgg || pGroupBy );
9427995032
 distinct = pParse->nTab++;
9428095033
 pKeyInfo = keyInfoFromExprList(pParse, p->pEList);
94281
- sqlite3VdbeAddOp4(v, OP_OpenEphemeral, distinct, 0, 0,
94282
- (char*)pKeyInfo, P4_KEYINFO_HANDOFF);
95034
+ addrDistinctIndex = sqlite3VdbeAddOp4(v, OP_OpenEphemeral, distinct, 0, 0,
95035
+ (char*)pKeyInfo, P4_KEYINFO_HANDOFF);
9428395036
 sqlite3VdbeChangeP5(v, BTREE_UNORDERED);
9428495037
 }else{
94285
- distinct = -1;
95038
+ distinct = addrDistinctIndex = -1;
9428695039
 }
9428795040
9428895041
 /* Aggregate and non-aggregate queries are handled differently */
9428995042
 if( !isAgg && pGroupBy==0 ){
94290
- /* This case is for non-aggregate queries
94291
- ** Begin the database scan
94292
- */
94293
- pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pOrderBy, 0);
95043
+ ExprList *pDist = (isDistinct ? p->pEList : 0);
95044
+
95045
+ /* Begin the database scan. */
95046
+ pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pOrderBy, pDist, 0);
9429495047
 if( pWInfo==0 ) goto select_end;
9429595048
 if( pWInfo->nRowOut < p->nSelectRow ) p->nSelectRow = pWInfo->nRowOut;
9429695049
9429795050
 /* If sorting index that was created by a prior OP_OpenEphemeral
9429895051
 ** instruction ended up not being needed, then change the OP_OpenEphemeral
@@ -94301,14 +95054,56 @@
9430195054
 if( addrSortIndex>=0 && pOrderBy==0 ){
9430295055
 sqlite3VdbeChangeToNoop(v, addrSortIndex, 1);
9430395056
 p->addrOpenEphm[2] = -1;
9430495057
 }
9430595058
94306
- /* Use the standard inner loop
94307
- */
94308
- assert(!isDistinct);
94309
- selectInnerLoop(pParse, p, pEList, 0, 0, pOrderBy, -1, pDest,
95059
+ if( pWInfo->eDistinct ){
95060
+ VdbeOp *pOp; /* No longer required OpenEphemeral instr. */
95061
+
95062
+ assert( addrDistinctIndex>0 );
95063
+ pOp = sqlite3VdbeGetOp(v, addrDistinctIndex);
95064
+
95065
+ assert( isDistinct );
95066
+ assert( pWInfo->eDistinct==WHERE_DISTINCT_ORDERED
95067
+ || pWInfo->eDistinct==WHERE_DISTINCT_UNIQUE
95068
+ );
95069
+ distinct = -1;
95070
+ if( pWInfo->eDistinct==WHERE_DISTINCT_ORDERED ){
95071
+ int iJump;
95072
+ int iExpr;
95073
+ int iFlag = ++pParse->nMem;
95074
+ int iBase = pParse->nMem+1;
95075
+ int iBase2 = iBase + pEList->nExpr;
95076
+ pParse->nMem += (pEList->nExpr*2);
95077
+
95078
+ /* Change the OP_OpenEphemeral coded earlier to an OP_Integer. The
95079
+ ** OP_Integer initializes the "first row" flag. */
95080
+ pOp->opcode = OP_Integer;
95081
+ pOp->p1 = 1;
95082
+ pOp->p2 = iFlag;
95083
+
95084
+ sqlite3ExprCodeExprList(pParse, pEList, iBase, 1);
95085
+ iJump = sqlite3VdbeCurrentAddr(v) + 1 + pEList->nExpr + 1 + 1;
95086
+ sqlite3VdbeAddOp2(v, OP_If, iFlag, iJump-1);
95087
+ for(iExpr=0; iExpr<pEList->nExpr; iExpr++){
95088
+ CollSeq *pColl = sqlite3ExprCollSeq(pParse, pEList->a[iExpr].pExpr);
95089
+ sqlite3VdbeAddOp3(v, OP_Ne, iBase+iExpr, iJump, iBase2+iExpr);
95090
+ sqlite3VdbeChangeP4(v, -1, (const char *)pColl, P4_COLLSEQ);
95091
+ sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
95092
+ }
95093
+ sqlite3VdbeAddOp2(v, OP_Goto, 0, pWInfo->iContinue);
95094
+
95095
+ sqlite3VdbeAddOp2(v, OP_Integer, 0, iFlag);
95096
+ assert( sqlite3VdbeCurrentAddr(v)==iJump );
95097
+ sqlite3VdbeAddOp3(v, OP_Move, iBase, iBase2, pEList->nExpr);
95098
+ }else{
95099
+ pOp->opcode = OP_Noop;
95100
+ }
95101
+ }
95102
+
95103
+ /* Use the standard inner loop. */
95104
+ selectInnerLoop(pParse, p, pEList, 0, 0, pOrderBy, distinct, pDest,
9431095105
 pWInfo->iContinue, pWInfo->iBreak);
9431195106
9431295107
 /* End the database scan loop.
9431395108
 */
9431495109
 sqlite3WhereEnd(pWInfo);
@@ -94414,11 +95209,11 @@
9441495209
 ** This might involve two separate loops with an OP_Sort in between, or
9441595210
 ** it might be a single loop that uses an index to extract information
9441695211
 ** in the right order to begin with.
9441795212
 */
9441895213
 sqlite3VdbeAddOp2(v, OP_Gosub, regReset, addrReset);
94419
- pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pGroupBy, 0);
95214
+ pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pGroupBy, 0, 0);
9442095215
 if( pWInfo==0 ) goto select_end;
9442195216
 if( pGroupBy==0 ){
9442295217
 /* The optimizer is able to deliver rows in group by order so
9442395218
 ** we do not have to sort. The OP_OpenEphemeral table will be
9442495219
 ** cancelled later because we still need to use the pKeyInfo
@@ -94676,11 +95471,11 @@
9467695471
 /* This case runs if the aggregate has no GROUP BY clause. The
9467795472
 ** processing is much simpler since there is only a single row
9467895473
 ** of output.
9467995474
 */
9468095475
 resetAccumulator(pParse, &sAggInfo);
94681
- pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pMinMax, flag);
95476
+ pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pMinMax, 0, flag);
9468295477
 if( pWInfo==0 ){
9468395478
 sqlite3ExprListDelete(db, pDel);
9468495479
 goto select_end;
9468595480
 }
9468695481
 updateAccumulator(pParse, &sAggInfo);
@@ -95152,19 +95947,32 @@
9515295947
 iDb = sqlite3TwoPartName(pParse, pName1, pName2, &pName);
9515395948
 if( iDb<0 ){
9515495949
 goto trigger_cleanup;
9515595950
 }
9515695951
 }
95952
+ if( !pTableName || db->mallocFailed ){
95953
+ goto trigger_cleanup;
95954
+ }
95955
+
95956
+ /* A long-standing parser bug is that this syntax was allowed:
95957
+ **
95958
+ ** CREATE TRIGGER attached.demo AFTER INSERT ON attached.tab ....
95959
+ ** ^^^^^^^^
95960
+ **
95961
+ ** To maintain backwards compatibility, ignore the database
95962
+ ** name on pTableName if we are reparsing our of SQLITE_MASTER.
95963
+ */
95964
+ if( db->init.busy && iDb!=1 ){
95965
+ sqlite3DbFree(db, pTableName->a[0].zDatabase);
95966
+ pTableName->a[0].zDatabase = 0;
95967
+ }
9515795968
9515895969
 /* If the trigger name was unqualified, and the table is a temp table,
9515995970
 ** then set iDb to 1 to create the trigger in the temporary database.
9516095971
 ** If sqlite3SrcListLookup() returns 0, indicating the table does not
9516195972
 ** exist, the error is caught by the block below.
9516295973
 */
95163
- if( !pTableName || db->mallocFailed ){
95164
- goto trigger_cleanup;
95165
- }
9516695974
 pTab = sqlite3SrcListLookup(pParse, pTableName);
9516795975
 if( db->init.busy==0 && pName2->n==0 && pTab
9516895976
 && pTab->pSchema==db->aDb[1].pSchema ){
9516995977
 iDb = 1;
9517095978
 }
@@ -95336,13 +96144,12 @@
9533696144
 "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')",
9533796145
 db->aDb[iDb].zName, SCHEMA_TABLE(iDb), zName,
9533896146
 pTrig->table, z);
9533996147
 sqlite3DbFree(db, z);
9534096148
 sqlite3ChangeCookie(pParse, iDb);
95341
- sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, sqlite3MPrintf(
95342
- db, "type='trigger' AND name='%q'", zName), P4_DYNAMIC
95343
- );
96149
+ sqlite3VdbeAddParseSchemaOp(v, iDb,
96150
+ sqlite3MPrintf(db, "type='trigger' AND name='%q'", zName));
9534496151
 }
9534596152
9534696153
 if( db->init.busy ){
9534796154
 Trigger *pLink = pTrig;
9534896155
 Hash *pHash = &db->aDb[iDb].pSchema->trigHash;
@@ -96392,11 +97199,11 @@
9639297199
 aRegIdx = sqlite3DbMallocRaw(db, sizeof(Index*) * nIdx );
9639397200
 if( aRegIdx==0 ) goto update_cleanup;
9639497201
 }
9639597202
 for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){
9639697203
 int reg;
96397
- if( chngRowid ){
97204
+ if( hasFK || chngRowid ){
9639897205
 reg = ++pParse->nMem;
9639997206
 }else{
9640097207
 reg = 0;
9640197208
 for(i=0; i<pIdx->nColumn; i++){
9640297209
 if( aXRef[pIdx->aiColumn[i]]>=0 ){
@@ -96459,11 +97266,13 @@
9645997266
 }
9646097267
9646197268
 /* Begin the database scan
9646297269
 */
9646397270
 sqlite3VdbeAddOp2(v, OP_Null, 0, regOldRowid);
96464
- pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere,0, WHERE_ONEPASS_DESIRED);
97271
+ pWInfo = sqlite3WhereBegin(
97272
+ pParse, pTabList, pWhere, 0, 0, WHERE_ONEPASS_DESIRED
97273
+ );
9646597274
 if( pWInfo==0 ) goto update_cleanup;
9646697275
 okOnePass = pWInfo->okOnePass;
9646797276
9646897277
 /* Remember the rowid of every item to be updated.
9646997278
 */
@@ -97547,11 +98356,11 @@
9754798356
 v = sqlite3GetVdbe(pParse);
9754898357
 sqlite3ChangeCookie(pParse, iDb);
9754998358
9755098359
 sqlite3VdbeAddOp2(v, OP_Expire, 0, 0);
9755198360
 zWhere = sqlite3MPrintf(db, "name='%q' AND type='table'", pTab->zName);
97552
- sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC);
98361
+ sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere);
9755398362
 sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0,
9755498363
 pTab->zName, sqlite3Strlen30(pTab->zName) + 1);
9755598364
 }
9755698365
9755798366
 /* If we are rereading the sqlite_master table create the in-memory
@@ -98350,11 +99159,11 @@
9835099159
 #define TERM_CODED 0x04 /* This term is already coded */
9835199160
 #define TERM_COPIED 0x08 /* Has a child */
9835299161
 #define TERM_ORINFO 0x10 /* Need to free the WhereTerm.u.pOrInfo object */
9835399162
 #define TERM_ANDINFO 0x20 /* Need to free the WhereTerm.u.pAndInfo obj */
9835499163
 #define TERM_OR_OK 0x40 /* Used during OR-clause processing */
98355
-#ifdef SQLITE_ENABLE_STAT2
99164
+#ifdef SQLITE_ENABLE_STAT3
9835699165
 # define TERM_VNULL 0x80 /* Manufactured x>NULL or x<=NULL term */
9835799166
 #else
9835899167
 # define TERM_VNULL 0x00 /* Disabled if not using stat2 */
9835999168
 #endif
9836099169
@@ -98485,10 +99294,11 @@
9848599294
 #define WHERE_REVERSE 0x02000000 /* Scan in reverse order */
9848699295
 #define WHERE_UNIQUE 0x04000000 /* Selects no more than one row */
9848799296
 #define WHERE_VIRTUALTABLE 0x08000000 /* Use virtual-table processing */
9848899297
 #define WHERE_MULTI_OR 0x10000000 /* OR using multiple indices */
9848999298
 #define WHERE_TEMP_INDEX 0x20000000 /* Uses an ephemeral index */
99299
+#define WHERE_DISTINCT 0x40000000 /* Correct order for DISTINCT */
9849099300
9849199301
 /*
9849299302
 ** Initialize a preallocated WhereClause structure.
9849399303
 */
9849499304
 static void whereClauseInit(
@@ -99563,11 +100373,11 @@
99563100373
 pNewTerm->prereqAll = pTerm->prereqAll;
99564100374
 }
99565100375
 }
99566100376
 #endif /* SQLITE_OMIT_VIRTUALTABLE */
99567100377
99568
-#ifdef SQLITE_ENABLE_STAT2
100378
+#ifdef SQLITE_ENABLE_STAT3
99569100379
 /* When sqlite_stat2 histogram data is available an operator of the
99570100380
 ** form "x IS NOT NULL" can sometimes be evaluated more efficiently
99571100381
 ** as "x>NULL" if x is not an INTEGER PRIMARY KEY. So construct a
99572100382
 ** virtual term of that form.
99573100383
 **
@@ -99602,11 +100412,11 @@
99602100412
 pTerm->nChild = 1;
99603100413
 pTerm->wtFlags |= TERM_COPIED;
99604100414
 pNewTerm->prereqAll = pTerm->prereqAll;
99605100415
 }
99606100416
 }
99607
-#endif /* SQLITE_ENABLE_STAT2 */
100417
+#endif /* SQLITE_ENABLE_STAT */
99608100418
99609100419
 /* Prevent ON clause terms of a LEFT JOIN from being used to drive
99610100420
 ** an index for tables to the left of the join.
99611100421
 */
99612100422
 pTerm->prereqRight |= extraRight;
@@ -99629,10 +100439,166 @@
99629100439
 }
99630100440
 }
99631100441
 return 0;
99632100442
 }
99633100443
100444
+/*
100445
+** This function searches the expression list passed as the second argument
100446
+** for an expression of type TK_COLUMN that refers to the same column and
100447
+** uses the same collation sequence as the iCol'th column of index pIdx.
100448
+** Argument iBase is the cursor number used for the table that pIdx refers
100449
+** to.
100450
+**
100451
+** If such an expression is found, its index in pList->a[] is returned. If
100452
+** no expression is found, -1 is returned.
100453
+*/
100454
+static int findIndexCol(
100455
+ Parse *pParse, /* Parse context */
100456
+ ExprList *pList, /* Expression list to search */
100457
+ int iBase, /* Cursor for table associated with pIdx */
100458
+ Index *pIdx, /* Index to match column of */
100459
+ int iCol /* Column of index to match */
100460
+){
100461
+ int i;
100462
+ const char *zColl = pIdx->azColl[iCol];
100463
+
100464
+ for(i=0; i<pList->nExpr; i++){
100465
+ Expr *p = pList->a[i].pExpr;
100466
+ if( p->op==TK_COLUMN
100467
+ && p->iColumn==pIdx->aiColumn[iCol]
100468
+ && p->iTable==iBase
100469
+ ){
100470
+ CollSeq *pColl = sqlite3ExprCollSeq(pParse, p);
100471
+ if( ALWAYS(pColl) && 0==sqlite3StrICmp(pColl->zName, zColl) ){
100472
+ return i;
100473
+ }
100474
+ }
100475
+ }
100476
+
100477
+ return -1;
100478
+}
100479
+
100480
+/*
100481
+** This routine determines if pIdx can be used to assist in processing a
100482
+** DISTINCT qualifier. In other words, it tests whether or not using this
100483
+** index for the outer loop guarantees that rows with equal values for
100484
+** all expressions in the pDistinct list are delivered grouped together.
100485
+**
100486
+** For example, the query
100487
+**
100488
+** SELECT DISTINCT a, b, c FROM tbl WHERE a = ?
100489
+**
100490
+** can benefit from any index on columns "b" and "c".
100491
+*/
100492
+static int isDistinctIndex(
100493
+ Parse *pParse, /* Parsing context */
100494
+ WhereClause *pWC, /* The WHERE clause */
100495
+ Index *pIdx, /* The index being considered */
100496
+ int base, /* Cursor number for the table pIdx is on */
100497
+ ExprList *pDistinct, /* The DISTINCT expressions */
100498
+ int nEqCol /* Number of index columns with == */
100499
+){
100500
+ Bitmask mask = 0; /* Mask of unaccounted for pDistinct exprs */
100501
+ int i; /* Iterator variable */
100502
+
100503
+ if( pIdx->zName==0 || pDistinct==0 || pDistinct->nExpr>=BMS ) return 0;
100504
+ testcase( pDistinct->nExpr==BMS-1 );
100505
+
100506
+ /* Loop through all the expressions in the distinct list. If any of them
100507
+ ** are not simple column references, return early. Otherwise, test if the
100508
+ ** WHERE clause contains a "col=X" clause. If it does, the expression
100509
+ ** can be ignored. If it does not, and the column does not belong to the
100510
+ ** same table as index pIdx, return early. Finally, if there is no
100511
+ ** matching "col=X" expression and the column is on the same table as pIdx,
100512
+ ** set the corresponding bit in variable mask.
100513
+ */
100514
+ for(i=0; i<pDistinct->nExpr; i++){
100515
+ WhereTerm *pTerm;
100516
+ Expr *p = pDistinct->a[i].pExpr;
100517
+ if( p->op!=TK_COLUMN ) return 0;
100518
+ pTerm = findTerm(pWC, p->iTable, p->iColumn, ~(Bitmask)0, WO_EQ, 0);
100519
+ if( pTerm ){
100520
+ Expr *pX = pTerm->pExpr;
100521
+ CollSeq *p1 = sqlite3BinaryCompareCollSeq(pParse, pX->pLeft, pX->pRight);
100522
+ CollSeq *p2 = sqlite3ExprCollSeq(pParse, p);
100523
+ if( p1==p2 ) continue;
100524
+ }
100525
+ if( p->iTable!=base ) return 0;
100526
+ mask |= (((Bitmask)1) << i);
100527
+ }
100528
+
100529
+ for(i=nEqCol; mask && i<pIdx->nColumn; i++){
100530
+ int iExpr = findIndexCol(pParse, pDistinct, base, pIdx, i);
100531
+ if( iExpr<0 ) break;
100532
+ mask &= ~(((Bitmask)1) << iExpr);
100533
+ }
100534
+
100535
+ return (mask==0);
100536
+}
100537
+
100538
+
100539
+/*
100540
+** Return true if the DISTINCT expression-list passed as the third argument
100541
+** is redundant. A DISTINCT list is redundant if the database contains a
100542
+** UNIQUE index that guarantees that the result of the query will be distinct
100543
+** anyway.
100544
+*/
100545
+static int isDistinctRedundant(
100546
+ Parse *pParse,
100547
+ SrcList *pTabList,
100548
+ WhereClause *pWC,
100549
+ ExprList *pDistinct
100550
+){
100551
+ Table *pTab;
100552
+ Index *pIdx;
100553
+ int i;
100554
+ int iBase;
100555
+
100556
+ /* If there is more than one table or sub-select in the FROM clause of
100557
+ ** this query, then it will not be possible to show that the DISTINCT
100558
+ ** clause is redundant. */
100559
+ if( pTabList->nSrc!=1 ) return 0;
100560
+ iBase = pTabList->a[0].iCursor;
100561
+ pTab = pTabList->a[0].pTab;
100562
+
100563
+ /* If any of the expressions is an IPK column on table iBase, then return
100564
+ ** true. Note: The (p->iTable==iBase) part of this test may be false if the
100565
+ ** current SELECT is a correlated sub-query.
100566
+ */
100567
+ for(i=0; i<pDistinct->nExpr; i++){
100568
+ Expr *p = pDistinct->a[i].pExpr;
100569
+ if( p->op==TK_COLUMN && p->iTable==iBase && p->iColumn<0 ) return 1;
100570
+ }
100571
+
100572
+ /* Loop through all indices on the table, checking each to see if it makes
100573
+ ** the DISTINCT qualifier redundant. It does so if:
100574
+ **
100575
+ ** 1. The index is itself UNIQUE, and
100576
+ **
100577
+ ** 2. All of the columns in the index are either part of the pDistinct
100578
+ ** list, or else the WHERE clause contains a term of the form "col=X",
100579
+ ** where X is a constant value. The collation sequences of the
100580
+ ** comparison and select-list expressions must match those of the index.
100581
+ */
100582
+ for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
100583
+ if( pIdx->onError==OE_None ) continue;
100584
+ for(i=0; i<pIdx->nColumn; i++){
100585
+ int iCol = pIdx->aiColumn[i];
100586
+ if( 0==findTerm(pWC, iBase, iCol, ~(Bitmask)0, WO_EQ, pIdx)
100587
+ && 0>findIndexCol(pParse, pDistinct, iBase, pIdx, i)
100588
+ ){
100589
+ break;
100590
+ }
100591
+ }
100592
+ if( i==pIdx->nColumn ){
100593
+ /* This index implies that the DISTINCT qualifier is redundant. */
100594
+ return 1;
100595
+ }
100596
+ }
100597
+
100598
+ return 0;
100599
+}
99634100600
99635100601
 /*
99636100602
 ** This routine decides if pIdx can be used to satisfy the ORDER BY
99637100603
 ** clause. If it can, it returns 1. If pIdx cannot satisfy the
99638100604
 ** ORDER BY clause, this routine returns 0.
@@ -99665,11 +100631,14 @@
99665100631
 int sortOrder = 0; /* XOR of index and ORDER BY sort direction */
99666100632
 int nTerm; /* Number of ORDER BY terms */
99667100633
 struct ExprList_item *pTerm; /* A term of the ORDER BY clause */
99668100634
 sqlite3 *db = pParse->db;
99669100635
99670
- assert( pOrderBy!=0 );
100636
+ if( !pOrderBy ) return 0;
100637
+ if( wsFlags & WHERE_COLUMN_IN ) return 0;
100638
+ if( pIdx->bUnordered ) return 0;
100639
+
99671100640
 nTerm = pOrderBy->nExpr;
99672100641
 assert( nTerm>0 );
99673100642
99674100643
 /* Argument pIdx must either point to a 'real' named index structure,
99675100644
 ** or an index structure allocated on the stack by bestBtreeIndex() to
@@ -99978,10 +100947,14 @@
99978100947
 double costTempIdx; /* per-query cost of the transient index */
99979100948
 WhereTerm *pTerm; /* A single term of the WHERE clause */
99980100949
 WhereTerm *pWCEnd; /* End of pWC->a[] */
99981100950
 Table *pTable; /* Table tht might be indexed */
99982100951
100952
+ if( pParse->nQueryLoop<=(double)1 ){
100953
+ /* There is no point in building an automatic index for a single scan */
100954
+ return;
100955
+ }
99983100956
 if( (pParse->db->flags & SQLITE_AutoIndex)==0 ){
99984100957
 /* Automatic indices are disabled at run-time */
99985100958
 return;
99986100959
 }
99987100960
 if( (pCost->plan.wsFlags & WHERE_NOT_FULLSCAN)!=0 ){
@@ -99989,10 +100962,14 @@
99989100962
 return;
99990100963
 }
99991100964
 if( pSrc->notIndexed ){
99992100965
 /* The NOT INDEXED clause appears in the SQL. */
99993100966
 return;
100967
+ }
100968
+ if( pSrc->isCorrelated ){
100969
+ /* The source is a correlated sub-query. No point in indexing it. */
100970
+ return;
99994100971
 }
99995100972
99996100973
 assert( pParse->nQueryLoop >= (double)1 );
99997100974
 pTable = pSrc->pTab;
99998100975
 nTableRow = pTable->nRowEst;
@@ -100206,10 +101183,11 @@
100206101183
 if( pTerm->leftCursor != pSrc->iCursor ) continue;
100207101184
 assert( (pTerm->eOperator&(pTerm->eOperator-1))==0 );
100208101185
 testcase( pTerm->eOperator==WO_IN );
100209101186
 testcase( pTerm->eOperator==WO_ISNULL );
100210101187
 if( pTerm->eOperator & (WO_IN|WO_ISNULL) ) continue;
101188
+ if( pTerm->wtFlags & TERM_VNULL ) continue;
100211101189
 nTerm++;
100212101190
 }
100213101191
100214101192
 /* If the ORDER BY clause contains only columns in the current
100215101193
 ** virtual table then allocate space for the aOrderBy part of
@@ -100256,10 +101234,11 @@
100256101234
 if( pTerm->leftCursor != pSrc->iCursor ) continue;
100257101235
 assert( (pTerm->eOperator&(pTerm->eOperator-1))==0 );
100258101236
 testcase( pTerm->eOperator==WO_IN );
100259101237
 testcase( pTerm->eOperator==WO_ISNULL );
100260101238
 if( pTerm->eOperator & (WO_IN|WO_ISNULL) ) continue;
101239
+ if( pTerm->wtFlags & TERM_VNULL ) continue;
100261101240
 pIdxCons[j].iColumn = pTerm->u.leftColumn;
100262101241
 pIdxCons[j].iTermOffset = i;
100263101242
 pIdxCons[j].op = (u8)pTerm->eOperator;
100264101243
 /* The direct assignment in the previous line is possible only because
100265101244
 ** the WO_ and SQLITE_INDEX_CONSTRAINT_ codes are identical. The
@@ -100482,71 +101461,89 @@
100482101461
 */
100483101462
 bestOrClauseIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost);
100484101463
 }
100485101464
 #endif /* SQLITE_OMIT_VIRTUALTABLE */
100486101465
101466
+#ifdef SQLITE_ENABLE_STAT3
100487101467
 /*
100488
-** Argument pIdx is a pointer to an index structure that has an array of
100489
-** SQLITE_INDEX_SAMPLES evenly spaced samples of the first indexed column
100490
-** stored in Index.aSample. These samples divide the domain of values stored
100491
-** the index into (SQLITE_INDEX_SAMPLES+1) regions.
100492
-** Region 0 contains all values less than the first sample value. Region
100493
-** 1 contains values between the first and second samples. Region 2 contains
100494
-** values between samples 2 and 3. And so on. Region SQLITE_INDEX_SAMPLES
100495
-** contains values larger than the last sample.
100496
-**
100497
-** If the index contains many duplicates of a single value, then it is
100498
-** possible that two or more adjacent samples can hold the same value.
100499
-** When that is the case, the smallest possible region code is returned
100500
-** when roundUp is false and the largest possible region code is returned
100501
-** when roundUp is true.
100502
-**
100503
-** If successful, this function determines which of the regions value
100504
-** pVal lies in, sets *piRegion to the region index (a value between 0
100505
-** and SQLITE_INDEX_SAMPLES+1, inclusive) and returns SQLITE_OK.
100506
-** Or, if an OOM occurs while converting text values between encodings,
100507
-** SQLITE_NOMEM is returned and *piRegion is undefined.
100508
-*/
100509
-#ifdef SQLITE_ENABLE_STAT2
100510
-static int whereRangeRegion(
101468
+** Estimate the location of a particular key among all keys in an
101469
+** index. Store the results in aStat as follows:
101470
+**
101471
+** aStat[0] Est. number of rows less than pVal
101472
+** aStat[1] Est. number of rows equal to pVal
101473
+**
101474
+** Return SQLITE_OK on success.
101475
+*/
101476
+static int whereKeyStats(
100511101477
 Parse *pParse, /* Database connection */
100512101478
 Index *pIdx, /* Index to consider domain of */
100513101479
 sqlite3_value *pVal, /* Value to consider */
100514
- int roundUp, /* Return largest valid region if true */
100515
- int *piRegion /* OUT: Region of domain in which value lies */
101480
+ int roundUp, /* Round up if true. Round down if false */
101481
+ tRowcnt *aStat /* OUT: stats written here */
100516101482
 ){
101483
+ tRowcnt n;
101484
+ IndexSample *aSample;
101485
+ int i, eType;
101486
+ int isEq = 0;
101487
+ i64 v;
101488
+ double r, rS;
101489
+
100517101490
 assert( roundUp==0 || roundUp==1 );
100518
- if( ALWAYS(pVal) ){
100519
- IndexSample *aSample = pIdx->aSample;
100520
- int i = 0;
100521
- int eType = sqlite3_value_type(pVal);
100522
-
100523
- if( eType==SQLITE_INTEGER || eType==SQLITE_FLOAT ){
100524
- double r = sqlite3_value_double(pVal);
100525
- for(i=0; i<SQLITE_INDEX_SAMPLES; i++){
100526
- if( aSample[i].eType==SQLITE_NULL ) continue;
100527
- if( aSample[i].eType>=SQLITE_TEXT ) break;
100528
- if( roundUp ){
100529
- if( aSample[i].u.r>r ) break;
100530
- }else{
100531
- if( aSample[i].u.r>=r ) break;
100532
- }
100533
- }
100534
- }else if( eType==SQLITE_NULL ){
100535
- i = 0;
100536
- if( roundUp ){
100537
- while( i<SQLITE_INDEX_SAMPLES && aSample[i].eType==SQLITE_NULL ) i++;
100538
- }
100539
- }else{
101491
+ if( pVal==0 ) return SQLITE_ERROR;
101492
+ n = pIdx->aiRowEst[0];
101493
+ aSample = pIdx->aSample;
101494
+ i = 0;
101495
+ eType = sqlite3_value_type(pVal);
101496
+
101497
+ if( eType==SQLITE_INTEGER ){
101498
+ v = sqlite3_value_int64(pVal);
101499
+ r = (i64)v;
101500
+ for(i=0; i<pIdx->nSample; i++){
101501
+ if( aSample[i].eType==SQLITE_NULL ) continue;
101502
+ if( aSample[i].eType>=SQLITE_TEXT ) break;
101503
+ if( aSample[i].eType==SQLITE_INTEGER ){
101504
+ if( aSample[i].u.i>=v ){
101505
+ isEq = aSample[i].u.i==v;
101506
+ break;
101507
+ }
101508
+ }else{
101509
+ assert( aSample[i].eType==SQLITE_FLOAT );
101510
+ if( aSample[i].u.r>=r ){
101511
+ isEq = aSample[i].u.r==r;
101512
+ break;
101513
+ }
101514
+ }
101515
+ }
101516
+ }else if( eType==SQLITE_FLOAT ){
101517
+ r = sqlite3_value_double(pVal);
101518
+ for(i=0; i<pIdx->nSample; i++){
101519
+ if( aSample[i].eType==SQLITE_NULL ) continue;
101520
+ if( aSample[i].eType>=SQLITE_TEXT ) break;
101521
+ if( aSample[i].eType==SQLITE_FLOAT ){
101522
+ rS = aSample[i].u.r;
101523
+ }else{
101524
+ rS = aSample[i].u.i;
101525
+ }
101526
+ if( rS>=r ){
101527
+ isEq = rS==r;
101528
+ break;
101529
+ }
101530
+ }
101531
+ }else if( eType==SQLITE_NULL ){
101532
+ i = 0;
101533
+ if( pIdx->nSample>=1 && aSample[0].eType==SQLITE_NULL ) isEq = 1;
101534
+ }else{
101535
+ assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB );
101536
+ for(i=0; i<pIdx->nSample; i++){
101537
+ if( aSample[i].eType==SQLITE_TEXT || aSample[i].eType==SQLITE_BLOB ){
101538
+ break;
101539
+ }
101540
+ }
101541
+ if( i<pIdx->nSample ){
100540101542
 sqlite3 *db = pParse->db;
100541101543
 CollSeq *pColl;
100542101544
 const u8 *z;
100543
- int n;
100544
-
100545
- /* pVal comes from sqlite3ValueFromExpr() so the type cannot be NULL */
100546
- assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB );
100547
-
100548101545
 if( eType==SQLITE_BLOB ){
100549101546
 z = (const u8 *)sqlite3_value_blob(pVal);
100550101547
 pColl = db->pDfltColl;
100551101548
 assert( pColl->enc==SQLITE_UTF8 );
100552101549
 }else{
@@ -100561,16 +101558,16 @@
100561101558
 return SQLITE_NOMEM;
100562101559
 }
100563101560
 assert( z && pColl && pColl->xCmp );
100564101561
 }
100565101562
 n = sqlite3ValueBytes(pVal, pColl->enc);
100566
-
100567
- for(i=0; i<SQLITE_INDEX_SAMPLES; i++){
101563
+
101564
+ for(; i<pIdx->nSample; i++){
100568101565
 int c;
100569101566
 int eSampletype = aSample[i].eType;
100570
- if( eSampletype==SQLITE_NULL || eSampletype<eType ) continue;
100571
- if( (eSampletype!=eType) ) break;
101567
+ if( eSampletype<eType ) continue;
101568
+ if( eSampletype!=eType ) break;
100572101569
 #ifndef SQLITE_OMIT_UTF16
100573101570
 if( pColl->enc!=SQLITE_UTF8 ){
100574101571
 int nSample;
100575101572
 char *zSample = sqlite3Utf8to16(
100576101573
 db, pColl->enc, aSample[i].u.z, aSample[i].nByte, &nSample
@@ -100584,20 +101581,52 @@
100584101581
 }else
100585101582
 #endif
100586101583
 {
100587101584
 c = pColl->xCmp(pColl->pUser, aSample[i].nByte, aSample[i].u.z, n, z);
100588101585
 }
100589
- if( c-roundUp>=0 ) break;
101586
+ if( c>=0 ){
101587
+ if( c==0 ) isEq = 1;
101588
+ break;
101589
+ }
100590101590
 }
100591101591
 }
101592
+ }
100592101593
100593
- assert( i>=0 && i<=SQLITE_INDEX_SAMPLES );
100594
- *piRegion = i;
101594
+ /* At this point, aSample[i] is the first sample that is greater than
101595
+ ** or equal to pVal. Or if i==pIdx->nSample, then all samples are less
101596
+ ** than pVal. If aSample[i]==pVal, then isEq==1.
101597
+ */
101598
+ if( isEq ){
101599
+ assert( i<pIdx->nSample );
101600
+ aStat[0] = aSample[i].nLt;
101601
+ aStat[1] = aSample[i].nEq;
101602
+ }else{
101603
+ tRowcnt iLower, iUpper, iGap;
101604
+ if( i==0 ){
101605
+ iLower = 0;
101606
+ iUpper = aSample[0].nLt;
101607
+ }else{
101608
+ iUpper = i>=pIdx->nSample ? n : aSample[i].nLt;
101609
+ iLower = aSample[i-1].nEq + aSample[i-1].nLt;
101610
+ }
101611
+ aStat[1] = pIdx->avgEq;
101612
+ if( iLower>=iUpper ){
101613
+ iGap = 0;
101614
+ }else{
101615
+ iGap = iUpper - iLower;
101616
+ if( iGap>=aStat[1]/2 ) iGap -= aStat[1]/2;
101617
+ }
101618
+ if( roundUp ){
101619
+ iGap = (iGap*2)/3;
101620
+ }else{
101621
+ iGap = iGap/3;
101622
+ }
101623
+ aStat[0] = iLower + iGap;
100595101624
 }
100596101625
 return SQLITE_OK;
100597101626
 }
100598
-#endif /* #ifdef SQLITE_ENABLE_STAT2 */
101627
+#endif /* SQLITE_ENABLE_STAT3 */
100599101628
100600101629
 /*
100601101630
 ** If expression pExpr represents a literal value, set *pp to point to
100602101631
 ** an sqlite3_value structure containing the same value, with affinity
100603101632
 ** aff applied to it, before returning. It is the responsibility of the
@@ -100611,11 +101640,11 @@
100611101640
 **
100612101641
 ** If neither of the above apply, set *pp to NULL.
100613101642
 **
100614101643
 ** If an error occurs, return an error code. Otherwise, SQLITE_OK.
100615101644
 */
100616
-#ifdef SQLITE_ENABLE_STAT2
101645
+#ifdef SQLITE_ENABLE_STAT3
100617101646
 static int valueFromExpr(
100618101647
 Parse *pParse,
100619101648
 Expr *pExpr,
100620101649
 u8 aff,
100621101650
 sqlite3_value **pp
@@ -100659,106 +101688,92 @@
100659101688
 **
100660101689
 ** ... FROM t1 WHERE a > ? AND a < ? ...
100661101690
 **
100662101691
 ** then nEq should be passed 0.
100663101692
 **
100664
-** The returned value is an integer between 1 and 100, inclusive. A return
100665
-** value of 1 indicates that the proposed range scan is expected to visit
100666
-** approximately 1/100th (1%) of the rows selected by the nEq equality
100667
-** constraints (if any). A return value of 100 indicates that it is expected
100668
-** that the range scan will visit every row (100%) selected by the equality
100669
-** constraints.
101693
+** The returned value is an integer divisor to reduce the estimated
101694
+** search space. A return value of 1 means that range constraints are
101695
+** no help at all. A return value of 2 means range constraints are
101696
+** expected to reduce the search space by half. And so forth...
100670101697
 **
100671
-** In the absence of sqlite_stat2 ANALYZE data, each range inequality
100672
-** reduces the search space by 3/4ths. Hence a single constraint (x>?)
100673
-** results in a return of 25 and a range constraint (x>? AND x<?) results
100674
-** in a return of 6.
101698
+** In the absence of sqlite_stat3 ANALYZE data, each range inequality
101699
+** reduces the search space by a factor of 4. Hence a single constraint (x>?)
101700
+** results in a return of 4 and a range constraint (x>? AND x<?) results
101701
+** in a return of 16.
100675101702
 */
100676101703
 static int whereRangeScanEst(
100677101704
 Parse *pParse, /* Parsing & code generating context */
100678101705
 Index *p, /* The index containing the range-compared column; "x" */
100679101706
 int nEq, /* index into p->aCol[] of the range-compared column */
100680101707
 WhereTerm *pLower, /* Lower bound on the range. ex: "x>123" Might be NULL */
100681101708
 WhereTerm *pUpper, /* Upper bound on the range. ex: "x<455" Might be NULL */
100682
- int *piEst /* OUT: Return value */
101709
+ double *pRangeDiv /* OUT: Reduce search space by this divisor */
100683101710
 ){
100684101711
 int rc = SQLITE_OK;
100685101712
100686
-#ifdef SQLITE_ENABLE_STAT2
100687
-
100688
- if( nEq==0 && p->aSample ){
100689
- sqlite3_value *pLowerVal = 0;
100690
- sqlite3_value *pUpperVal = 0;
100691
- int iEst;
100692
- int iLower = 0;
100693
- int iUpper = SQLITE_INDEX_SAMPLES;
100694
- int roundUpUpper = 0;
100695
- int roundUpLower = 0;
101713
+#ifdef SQLITE_ENABLE_STAT3
101714
+
101715
+ if( nEq==0 && p->nSample ){
101716
+ sqlite3_value *pRangeVal;
101717
+ tRowcnt iLower = 0;
101718
+ tRowcnt iUpper = p->aiRowEst[0];
101719
+ tRowcnt a[2];
100696101720
 u8 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
100697101721
100698101722
 if( pLower ){
100699101723
 Expr *pExpr = pLower->pExpr->pRight;
100700
- rc = valueFromExpr(pParse, pExpr, aff, &pLowerVal);
101724
+ rc = valueFromExpr(pParse, pExpr, aff, &pRangeVal);
100701101725
 assert( pLower->eOperator==WO_GT || pLower->eOperator==WO_GE );
100702
- roundUpLower = (pLower->eOperator==WO_GT) ?1:0;
101726
+ if( rc==SQLITE_OK
101727
+ && whereKeyStats(pParse, p, pRangeVal, 0, a)==SQLITE_OK
101728
+ ){
101729
+ iLower = a[0];
101730
+ if( pLower->eOperator==WO_GT ) iLower += a[1];
101731
+ }
101732
+ sqlite3ValueFree(pRangeVal);
100703101733
 }
100704101734
 if( rc==SQLITE_OK && pUpper ){
100705101735
 Expr *pExpr = pUpper->pExpr->pRight;
100706
- rc = valueFromExpr(pParse, pExpr, aff, &pUpperVal);
101736
+ rc = valueFromExpr(pParse, pExpr, aff, &pRangeVal);
100707101737
 assert( pUpper->eOperator==WO_LT || pUpper->eOperator==WO_LE );
100708
- roundUpUpper = (pUpper->eOperator==WO_LE) ?1:0;
100709
- }
100710
-
100711
- if( rc!=SQLITE_OK || (pLowerVal==0 && pUpperVal==0) ){
100712
- sqlite3ValueFree(pLowerVal);
100713
- sqlite3ValueFree(pUpperVal);
100714
- goto range_est_fallback;
100715
- }else if( pLowerVal==0 ){
100716
- rc = whereRangeRegion(pParse, p, pUpperVal, roundUpUpper, &iUpper);
100717
- if( pLower ) iLower = iUpper/2;
100718
- }else if( pUpperVal==0 ){
100719
- rc = whereRangeRegion(pParse, p, pLowerVal, roundUpLower, &iLower);
100720
- if( pUpper ) iUpper = (iLower + SQLITE_INDEX_SAMPLES + 1)/2;
100721
- }else{
100722
- rc = whereRangeRegion(pParse, p, pUpperVal, roundUpUpper, &iUpper);
100723
- if( rc==SQLITE_OK ){
100724
- rc = whereRangeRegion(pParse, p, pLowerVal, roundUpLower, &iLower);
100725
- }
100726
- }
100727
- WHERETRACE(("range scan regions: %d..%d\n", iLower, iUpper));
100728
-
100729
- iEst = iUpper - iLower;
100730
- testcase( iEst==SQLITE_INDEX_SAMPLES );
100731
- assert( iEst<=SQLITE_INDEX_SAMPLES );
100732
- if( iEst<1 ){
100733
- *piEst = 50/SQLITE_INDEX_SAMPLES;
100734
- }else{
100735
- *piEst = (iEst*100)/SQLITE_INDEX_SAMPLES;
100736
- }
100737
- sqlite3ValueFree(pLowerVal);
100738
- sqlite3ValueFree(pUpperVal);
100739
- return rc;
100740
- }
100741
-range_est_fallback:
101738
+ if( rc==SQLITE_OK
101739
+ && whereKeyStats(pParse, p, pRangeVal, 1, a)==SQLITE_OK
101740
+ ){
101741
+ iUpper = a[0];
101742
+ if( pUpper->eOperator==WO_LE ) iUpper += a[1];
101743
+ }
101744
+ sqlite3ValueFree(pRangeVal);
101745
+ }
101746
+ if( rc==SQLITE_OK ){
101747
+ if( iUpper<=iLower ){
101748
+ *pRangeDiv = (double)p->aiRowEst[0];
101749
+ }else{
101750
+ *pRangeDiv = (double)p->aiRowEst[0]/(double)(iUpper - iLower);
101751
+ }
101752
+ WHERETRACE(("range scan regions: %u..%u div=%g\n",
101753
+ (u32)iLower, (u32)iUpper, *pRangeDiv));
101754
+ return SQLITE_OK;
101755
+ }
101756
+ }
100742101757
 #else
100743101758
 UNUSED_PARAMETER(pParse);
100744101759
 UNUSED_PARAMETER(p);
100745101760
 UNUSED_PARAMETER(nEq);
100746101761
 #endif
100747101762
 assert( pLower || pUpper );
100748
- *piEst = 100;
100749
- if( pLower && (pLower->wtFlags & TERM_VNULL)==0 ) *piEst /= 4;
100750
- if( pUpper ) *piEst /= 4;
101763
+ *pRangeDiv = (double)1;
101764
+ if( pLower && (pLower->wtFlags & TERM_VNULL)==0 ) *pRangeDiv *= (double)4;
101765
+ if( pUpper ) *pRangeDiv *= (double)4;
100751101766
 return rc;
100752101767
 }
100753101768
100754
-#ifdef SQLITE_ENABLE_STAT2
101769
+#ifdef SQLITE_ENABLE_STAT3
100755101770
 /*
100756101771
 ** Estimate the number of rows that will be returned based on
100757101772
 ** an equality constraint x=VALUE and where that VALUE occurs in
100758101773
 ** the histogram data. This only works when x is the left-most
100759
-** column of an index and sqlite_stat2 histogram data is available
101774
+** column of an index and sqlite_stat3 histogram data is available
100760101775
 ** for that index. When pExpr==NULL that means the constraint is
100761101776
 ** "x IS NULL" instead of "x=VALUE".
100762101777
 **
100763101778
 ** Write the estimated row count into *pnRow and return SQLITE_OK.
100764101779
 ** If unable to make an estimate, leave *pnRow unchanged and return
@@ -100774,14 +101789,13 @@
100774101789
 Index *p, /* The index whose left-most column is pTerm */
100775101790
 Expr *pExpr, /* Expression for VALUE in the x=VALUE constraint */
100776101791
 double *pnRow /* Write the revised row estimate here */
100777101792
 ){
100778101793
 sqlite3_value *pRhs = 0; /* VALUE on right-hand side of pTerm */
100779
- int iLower, iUpper; /* Range of histogram regions containing pRhs */
100780101794
 u8 aff; /* Column affinity */
100781101795
 int rc; /* Subfunction return code */
100782
- double nRowEst; /* New estimate of the number of rows */
101796
+ tRowcnt a[2]; /* Statistics */
100783101797
100784101798
 assert( p->aSample!=0 );
100785101799
 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
100786101800
 if( pExpr ){
100787101801
 rc = valueFromExpr(pParse, pExpr, aff, &pRhs);
@@ -100788,30 +101802,22 @@
100788101802
 if( rc ) goto whereEqualScanEst_cancel;
100789101803
 }else{
100790101804
 pRhs = sqlite3ValueNew(pParse->db);
100791101805
 }
100792101806
 if( pRhs==0 ) return SQLITE_NOTFOUND;
100793
- rc = whereRangeRegion(pParse, p, pRhs, 0, &iLower);
100794
- if( rc ) goto whereEqualScanEst_cancel;
100795
- rc = whereRangeRegion(pParse, p, pRhs, 1, &iUpper);
100796
- if( rc ) goto whereEqualScanEst_cancel;
100797
- WHERETRACE(("equality scan regions: %d..%d\n", iLower, iUpper));
100798
- if( iLower>=iUpper ){
100799
- nRowEst = p->aiRowEst[0]/(SQLITE_INDEX_SAMPLES*2);
100800
- if( nRowEst<*pnRow ) *pnRow = nRowEst;
100801
- }else{
100802
- nRowEst = (iUpper-iLower)*p->aiRowEst[0]/SQLITE_INDEX_SAMPLES;
100803
- *pnRow = nRowEst;
100804
- }
100805
-
101807
+ rc = whereKeyStats(pParse, p, pRhs, 0, a);
101808
+ if( rc==SQLITE_OK ){
101809
+ WHERETRACE(("equality scan regions: %d\n", (int)a[1]));
101810
+ *pnRow = a[1];
101811
+ }
100806101812
 whereEqualScanEst_cancel:
100807101813
 sqlite3ValueFree(pRhs);
100808101814
 return rc;
100809101815
 }
100810
-#endif /* defined(SQLITE_ENABLE_STAT2) */
101816
+#endif /* defined(SQLITE_ENABLE_STAT3) */
100811101817
100812
-#ifdef SQLITE_ENABLE_STAT2
101818
+#ifdef SQLITE_ENABLE_STAT3
100813101819
 /*
100814101820
 ** Estimate the number of rows that will be returned based on
100815101821
 ** an IN constraint where the right-hand side of the IN operator
100816101822
 ** is a list of values. Example:
100817101823
 **
@@ -100830,64 +101836,29 @@
100830101836
 Parse *pParse, /* Parsing & code generating context */
100831101837
 Index *p, /* The index whose left-most column is pTerm */
100832101838
 ExprList *pList, /* The value list on the RHS of "x IN (v1,v2,v3,...)" */
100833101839
 double *pnRow /* Write the revised row estimate here */
100834101840
 ){
100835
- sqlite3_value *pVal = 0; /* One value from list */
100836
- int iLower, iUpper; /* Range of histogram regions containing pRhs */
100837
- u8 aff; /* Column affinity */
100838
- int rc = SQLITE_OK; /* Subfunction return code */
100839
- double nRowEst; /* New estimate of the number of rows */
100840
- int nSpan = 0; /* Number of histogram regions spanned */
100841
- int nSingle = 0; /* Histogram regions hit by a single value */
100842
- int nNotFound = 0; /* Count of values that are not constants */
100843
- int i; /* Loop counter */
100844
- u8 aSpan[SQLITE_INDEX_SAMPLES+1]; /* Histogram regions that are spanned */
100845
- u8 aSingle[SQLITE_INDEX_SAMPLES+1]; /* Histogram regions hit once */
101841
+ int rc = SQLITE_OK; /* Subfunction return code */
101842
+ double nEst; /* Number of rows for a single term */
101843
+ double nRowEst = (double)0; /* New estimate of the number of rows */
101844
+ int i; /* Loop counter */
100846101845
100847101846
 assert( p->aSample!=0 );
100848
- aff = p->pTable->aCol[p->aiColumn[0]].affinity;
100849
- memset(aSpan, 0, sizeof(aSpan));
100850
- memset(aSingle, 0, sizeof(aSingle));
100851
- for(i=0; i<pList->nExpr; i++){
100852
- sqlite3ValueFree(pVal);
100853
- rc = valueFromExpr(pParse, pList->a[i].pExpr, aff, &pVal);
100854
- if( rc ) break;
100855
- if( pVal==0 || sqlite3_value_type(pVal)==SQLITE_NULL ){
100856
- nNotFound++;
100857
- continue;
100858
- }
100859
- rc = whereRangeRegion(pParse, p, pVal, 0, &iLower);
100860
- if( rc ) break;
100861
- rc = whereRangeRegion(pParse, p, pVal, 1, &iUpper);
100862
- if( rc ) break;
100863
- if( iLower>=iUpper ){
100864
- aSingle[iLower] = 1;
100865
- }else{
100866
- assert( iLower>=0 && iUpper<=SQLITE_INDEX_SAMPLES );
100867
- while( iLower<iUpper ) aSpan[iLower++] = 1;
100868
- }
101847
+ for(i=0; rc==SQLITE_OK && i<pList->nExpr; i++){
101848
+ nEst = p->aiRowEst[0];
101849
+ rc = whereEqualScanEst(pParse, p, pList->a[i].pExpr, &nEst);
101850
+ nRowEst += nEst;
100869101851
 }
100870101852
 if( rc==SQLITE_OK ){
100871
- for(i=nSpan=0; i<=SQLITE_INDEX_SAMPLES; i++){
100872
- if( aSpan[i] ){
100873
- nSpan++;
100874
- }else if( aSingle[i] ){
100875
- nSingle++;
100876
- }
100877
- }
100878
- nRowEst = (nSpan*2+nSingle)*p->aiRowEst[0]/(2*SQLITE_INDEX_SAMPLES)
100879
- + nNotFound*p->aiRowEst[1];
100880101853
 if( nRowEst > p->aiRowEst[0] ) nRowEst = p->aiRowEst[0];
100881101854
 *pnRow = nRowEst;
100882
- WHERETRACE(("IN row estimate: nSpan=%d, nSingle=%d, nNotFound=%d, est=%g\n",
100883
- nSpan, nSingle, nNotFound, nRowEst));
101855
+ WHERETRACE(("IN row estimate: est=%g\n", nRowEst));
100884101856
 }
100885
- sqlite3ValueFree(pVal);
100886101857
 return rc;
100887101858
 }
100888
-#endif /* defined(SQLITE_ENABLE_STAT2) */
101859
+#endif /* defined(SQLITE_ENABLE_STAT3) */
100889101860
100890101861
100891101862
 /*
100892101863
 ** Find the best query plan for accessing a particular table. Write the
100893101864
 ** best query plan and its cost into the WhereCost object supplied as the
@@ -100921,19 +101892,20 @@
100921101892
 WhereClause *pWC, /* The WHERE clause */
100922101893
 struct SrcList_item *pSrc, /* The FROM clause term to search */
100923101894
 Bitmask notReady, /* Mask of cursors not available for indexing */
100924101895
 Bitmask notValid, /* Cursors not available for any purpose */
100925101896
 ExprList *pOrderBy, /* The ORDER BY clause */
101897
+ ExprList *pDistinct, /* The select-list if query is DISTINCT */
100926101898
 WhereCost *pCost /* Lowest cost query plan */
100927101899
 ){
100928101900
 int iCur = pSrc->iCursor; /* The cursor of the table to be accessed */
100929101901
 Index *pProbe; /* An index we are evaluating */
100930101902
 Index *pIdx; /* Copy of pProbe, or zero for IPK index */
100931101903
 int eqTermMask; /* Current mask of valid equality operators */
100932101904
 int idxEqTermMask; /* Index mask of valid equality operators */
100933101905
 Index sPk; /* A fake index object for the primary key */
100934
- unsigned int aiRowEstPk[2]; /* The aiRowEst[] value for the sPk index */
101906
+ tRowcnt aiRowEstPk[2]; /* The aiRowEst[] value for the sPk index */
100935101907
 int aiColumnPk = -1; /* The aColumn[] value for the sPk index */
100936101908
 int wsFlagMask; /* Allowed flags in pCost->plan.wsFlag */
100937101909
100938101910
 /* Initialize the cost to a worst-case value */
100939101911
 memset(pCost, 0, sizeof(*pCost));
@@ -100984,11 +101956,11 @@
100984101956
 }
100985101957
100986101958
 /* Loop over all indices looking for the best one to use
100987101959
 */
100988101960
 for(; pProbe; pIdx=pProbe=pProbe->pNext){
100989
- const unsigned int * const aiRowEst = pProbe->aiRowEst;
101961
+ const tRowcnt * const aiRowEst = pProbe->aiRowEst;
100990101962
 double cost; /* Cost of using pProbe */
100991101963
 double nRow; /* Estimated number of rows in result set */
100992101964
 double log10N; /* base-10 logarithm of nRow (inexact) */
100993101965
 int rev; /* True to scan in reverse order */
100994101966
 int wsFlags = 0;
@@ -101027,18 +101999,16 @@
101027101999
 ** Set to true if there was at least one "x IN (SELECT ...)" term used
101028102000
 ** in determining the value of nInMul. Note that the RHS of the
101029102001
 ** IN operator must be a SELECT, not a value list, for this variable
101030102002
 ** to be true.
101031102003
 **
101032
- ** estBound:
101033
- ** An estimate on the amount of the table that must be searched. A
101034
- ** value of 100 means the entire table is searched. Range constraints
101035
- ** might reduce this to a value less than 100 to indicate that only
101036
- ** a fraction of the table needs searching. In the absence of
101037
- ** sqlite_stat2 ANALYZE data, a single inequality reduces the search
101038
- ** space to 1/4rd its original size. So an x>? constraint reduces
101039
- ** estBound to 25. Two constraints (x>? AND x<?) reduce estBound to 6.
102004
+ ** rangeDiv:
102005
+ ** An estimate of a divisor by which to reduce the search space due
102006
+ ** to inequality constraints. In the absence of sqlite_stat3 ANALYZE
102007
+ ** data, a single inequality reduces the search space to 1/4rd its
102008
+ ** original size (rangeDiv==4). Two inequalities reduce the search
102009
+ ** space to 1/16th of its original size (rangeDiv==16).
101040102010
 **
101041102011
 ** bSort:
101042102012
 ** Boolean. True if there is an ORDER BY clause that will require an
101043102013
 ** external sort (i.e. scanning the index being evaluated will not
101044102014
 ** correctly order records).
@@ -101059,16 +102029,17 @@
101059102029
 ** SELECT a, b, c FROM tbl WHERE a = 1;
101060102030
 */
101061102031
 int nEq; /* Number of == or IN terms matching index */
101062102032
 int bInEst = 0; /* True if "x IN (SELECT...)" seen */
101063102033
 int nInMul = 1; /* Number of distinct equalities to lookup */
101064
- int estBound = 100; /* Estimated reduction in search space */
102034
+ double rangeDiv = (double)1; /* Estimated reduction in search space */
101065102035
 int nBound = 0; /* Number of range constraints seen */
101066
- int bSort = 0; /* True if external sort required */
102036
+ int bSort = !!pOrderBy; /* True if external sort required */
102037
+ int bDist = !!pDistinct; /* True if index cannot help with DISTINCT */
101067102038
 int bLookup = 0; /* True if not a covering index */
101068102039
 WhereTerm *pTerm; /* A single term of the WHERE clause */
101069
-#ifdef SQLITE_ENABLE_STAT2
102040
+#ifdef SQLITE_ENABLE_STAT3
101070102041
 WhereTerm *pFirstTerm = 0; /* First term matching the index */
101071102042
 #endif
101072102043
101073102044
 /* Determine the values of nEq and nInMul */
101074102045
 for(nEq=0; nEq<pProbe->nColumn; nEq++){
@@ -101088,23 +102059,23 @@
101088102059
 nInMul *= pExpr->x.pList->nExpr;
101089102060
 }
101090102061
 }else if( pTerm->eOperator & WO_ISNULL ){
101091102062
 wsFlags |= WHERE_COLUMN_NULL;
101092102063
 }
101093
-#ifdef SQLITE_ENABLE_STAT2
102064
+#ifdef SQLITE_ENABLE_STAT3
101094102065
 if( nEq==0 && pProbe->aSample ) pFirstTerm = pTerm;
101095102066
 #endif
101096102067
 used |= pTerm->prereqRight;
101097102068
 }
101098102069
101099
- /* Determine the value of estBound. */
102070
+ /* Determine the value of rangeDiv */
101100102071
 if( nEq<pProbe->nColumn && pProbe->bUnordered==0 ){
101101102072
 int j = pProbe->aiColumn[nEq];
101102102073
 if( findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE|WO_GT|WO_GE, pIdx) ){
101103102074
 WhereTerm *pTop = findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE, pIdx);
101104102075
 WhereTerm *pBtm = findTerm(pWC, iCur, j, notReady, WO_GT|WO_GE, pIdx);
101105
- whereRangeScanEst(pParse, pProbe, nEq, pBtm, pTop, &estBound);
102076
+ whereRangeScanEst(pParse, pProbe, nEq, pBtm, pTop, &rangeDiv);
101106102077
 if( pTop ){
101107102078
 nBound = 1;
101108102079
 wsFlags |= WHERE_TOP_LIMIT;
101109102080
 used |= pTop->prereqRight;
101110102081
 }
@@ -101125,21 +102096,24 @@
101125102096
101126102097
 /* If there is an ORDER BY clause and the index being considered will
101127102098
 ** naturally scan rows in the required order, set the appropriate flags
101128102099
 ** in wsFlags. Otherwise, if there is an ORDER BY clause but the index
101129102100
 ** will scan rows in a different order, set the bSort variable. */
101130
- if( pOrderBy ){
101131
- if( (wsFlags & WHERE_COLUMN_IN)==0
101132
- && pProbe->bUnordered==0
101133
- && isSortingIndex(pParse, pWC->pMaskSet, pProbe, iCur, pOrderBy,
101134
- nEq, wsFlags, &rev)
101135
- ){
101136
- wsFlags |= WHERE_ROWID_RANGE|WHERE_COLUMN_RANGE|WHERE_ORDERBY;
101137
- wsFlags |= (rev ? WHERE_REVERSE : 0);
101138
- }else{
101139
- bSort = 1;
101140
- }
102101
+ if( isSortingIndex(
102102
+ pParse, pWC->pMaskSet, pProbe, iCur, pOrderBy, nEq, wsFlags, &rev)
102103
+ ){
102104
+ bSort = 0;
102105
+ wsFlags |= WHERE_ROWID_RANGE|WHERE_COLUMN_RANGE|WHERE_ORDERBY;
102106
+ wsFlags |= (rev ? WHERE_REVERSE : 0);
102107
+ }
102108
+
102109
+ /* If there is a DISTINCT qualifier and this index will scan rows in
102110
+ ** order of the DISTINCT expressions, clear bDist and set the appropriate
102111
+ ** flags in wsFlags. */
102112
+ if( isDistinctIndex(pParse, pWC, pProbe, iCur, pDistinct, nEq) ){
102113
+ bDist = 0;
102114
+ wsFlags |= WHERE_ROWID_RANGE|WHERE_COLUMN_RANGE|WHERE_DISTINCT;
101141102115
 }
101142102116
101143102117
 /* If currently calculating the cost of using an index (not the IPK
101144102118
 ** index), determine if all required column data may be obtained without
101145102119
 ** using the main table (i.e. if the index is a covering
@@ -101169,31 +102143,32 @@
101169102143
 if( bInEst && nRow*2>aiRowEst[0] ){
101170102144
 nRow = aiRowEst[0]/2;
101171102145
 nInMul = (int)(nRow / aiRowEst[nEq]);
101172102146
 }
101173102147
101174
-#ifdef SQLITE_ENABLE_STAT2
101175
- /* If the constraint is of the form x=VALUE and histogram
102148
+#ifdef SQLITE_ENABLE_STAT3
102149
+ /* If the constraint is of the form x=VALUE or x IN (E1,E2,...)
102150
+ ** and we do not think that values of x are unique and if histogram
101176102151
 ** data is available for column x, then it might be possible
101177102152
 ** to get a better estimate on the number of rows based on
101178102153
 ** VALUE and how common that value is according to the histogram.
101179102154
 */
101180
- if( nRow>(double)1 && nEq==1 && pFirstTerm!=0 ){
102155
+ if( nRow>(double)1 && nEq==1 && pFirstTerm!=0 && aiRowEst[1]>1 ){
101181102156
 if( pFirstTerm->eOperator & (WO_EQ|WO_ISNULL) ){
101182102157
 testcase( pFirstTerm->eOperator==WO_EQ );
101183102158
 testcase( pFirstTerm->eOperator==WO_ISNULL );
101184102159
 whereEqualScanEst(pParse, pProbe, pFirstTerm->pExpr->pRight, &nRow);
101185102160
 }else if( pFirstTerm->eOperator==WO_IN && bInEst==0 ){
101186102161
 whereInScanEst(pParse, pProbe, pFirstTerm->pExpr->x.pList, &nRow);
101187102162
 }
101188102163
 }
101189
-#endif /* SQLITE_ENABLE_STAT2 */
102164
+#endif /* SQLITE_ENABLE_STAT3 */
101190102165
101191102166
 /* Adjust the number of output rows and downward to reflect rows
101192102167
 ** that are excluded by range constraints.
101193102168
 */
101194
- nRow = (nRow * (double)estBound) / (double)100;
102169
+ nRow = nRow/rangeDiv;
101195102170
 if( nRow<1 ) nRow = 1;
101196102171
101197102172
 /* Experiments run on real SQLite databases show that the time needed
101198102173
 ** to do a binary search to locate a row in a table or index is roughly
101199102174
 ** log10(N) times the time to move from one row to the next row within
@@ -101252,10 +102227,13 @@
101252102227
 ** difference and select C of 3.0.
101253102228
 */
101254102229
 if( bSort ){
101255102230
 cost += nRow*estLog(nRow)*3;
101256102231
 }
102232
+ if( bDist ){
102233
+ cost += nRow*estLog(nRow)*3;
102234
+ }
101257102235
101258102236
 /**** Cost of using this index has now been computed ****/
101259102237
101260102238
 /* If there are additional constraints on this table that cannot
101261102239
 ** be used with the current index, but which might lower the number
@@ -101315,14 +102293,14 @@
101315102293
 if( nRow<2 ) nRow = 2;
101316102294
 }
101317102295
101318102296
101319102297
 WHERETRACE((
101320
- "%s(%s): nEq=%d nInMul=%d estBound=%d bSort=%d bLookup=%d wsFlags=0x%x\n"
102298
+ "%s(%s): nEq=%d nInMul=%d rangeDiv=%d bSort=%d bLookup=%d wsFlags=0x%x\n"
101321102299
 " notReady=0x%llx log10N=%.1f nRow=%.1f cost=%.1f used=0x%llx\n",
101322102300
 pSrc->pTab->zName, (pIdx ? pIdx->zName : "ipk"),
101323
- nEq, nInMul, estBound, bSort, bLookup, wsFlags,
102301
+ nEq, nInMul, (int)rangeDiv, bSort, bLookup, wsFlags,
101324102302
 notReady, log10N, nRow, cost, used
101325102303
 ));
101326102304
101327102305
 /* If this index is the best we have seen so far, then record this
101328102306
 ** index and its cost in the pCost structure.
@@ -101397,11 +102375,11 @@
101397102375
 }
101398102376
 sqlite3DbFree(pParse->db, p);
101399102377
 }else
101400102378
 #endif
101401102379
 {
101402
- bestBtreeIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost);
102380
+ bestBtreeIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, 0, pCost);
101403102381
 }
101404102382
 }
101405102383
101406102384
 /*
101407102385
 ** Disable a term in the WHERE clause. Except, do not disable the term
@@ -102359,11 +103337,11 @@
102359103337
 for(ii=0; ii<pOrWc->nTerm; ii++){
102360103338
 WhereTerm *pOrTerm = &pOrWc->a[ii];
102361103339
 if( pOrTerm->leftCursor==iCur || pOrTerm->eOperator==WO_AND ){
102362103340
 WhereInfo *pSubWInfo; /* Info for single OR-term scan */
102363103341
 /* Loop through table entries that match term pOrTerm. */
102364
- pSubWInfo = sqlite3WhereBegin(pParse, pOrTab, pOrTerm->pExpr, 0,
103342
+ pSubWInfo = sqlite3WhereBegin(pParse, pOrTab, pOrTerm->pExpr, 0, 0,
102365103343
 WHERE_OMIT_OPEN | WHERE_OMIT_CLOSE |
102366103344
 WHERE_FORCE_TABLE | WHERE_ONETABLE_ONLY);
102367103345
 if( pSubWInfo ){
102368103346
 explainOneScan(
102369103347
 pParse, pOrTab, &pSubWInfo->a[0], iLevel, pLevel->iFrom, 0
@@ -102600,10 +103578,11 @@
102600103578
 SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
102601103579
 Parse *pParse, /* The parser context */
102602103580
 SrcList *pTabList, /* A list of all tables to be scanned */
102603103581
 Expr *pWhere, /* The WHERE clause */
102604103582
 ExprList **ppOrderBy, /* An ORDER BY clause, or NULL */
103583
+ ExprList *pDistinct, /* The select-list for DISTINCT queries - or NULL */
102605103584
 u16 wctrlFlags /* One of the WHERE_* flags defined in sqliteInt.h */
102606103585
 ){
102607103586
 int i; /* Loop counter */
102608103587
 int nByteWInfo; /* Num. bytes allocated for WhereInfo struct */
102609103588
 int nTabList; /* Number of elements in pTabList */
@@ -102659,10 +103638,14 @@
102659103638
 pWInfo->iBreak = sqlite3VdbeMakeLabel(v);
102660103639
 pWInfo->pWC = pWC = (WhereClause *)&((u8 *)pWInfo)[nByteWInfo];
102661103640
 pWInfo->wctrlFlags = wctrlFlags;
102662103641
 pWInfo->savedNQueryLoop = pParse->nQueryLoop;
102663103642
 pMaskSet = (WhereMaskSet*)&pWC[1];
103643
+
103644
+ /* Disable the DISTINCT optimization if SQLITE_DistinctOpt is set via
103645
+ ** sqlite3_test_ctrl(SQLITE_TESTCTRL_OPTIMIZATIONS,...) */
103646
+ if( db->flags & SQLITE_DistinctOpt ) pDistinct = 0;
102664103647
102665103648
 /* Split the WHERE clause into separate subexpressions where each
102666103649
 ** subexpression is separated by an AND operator.
102667103650
 */
102668103651
 initMaskSet(pMaskSet);
@@ -102726,10 +103709,19 @@
102726103709
 */
102727103710
 exprAnalyzeAll(pTabList, pWC);
102728103711
 if( db->mallocFailed ){
102729103712
 goto whereBeginError;
102730103713
 }
103714
+
103715
+ /* Check if the DISTINCT qualifier, if there is one, is redundant.
103716
+ ** If it is, then set pDistinct to NULL and WhereInfo.eDistinct to
103717
+ ** WHERE_DISTINCT_UNIQUE to tell the caller to ignore the DISTINCT.
103718
+ */
103719
+ if( pDistinct && isDistinctRedundant(pParse, pTabList, pWC, pDistinct) ){
103720
+ pDistinct = 0;
103721
+ pWInfo->eDistinct = WHERE_DISTINCT_UNIQUE;
103722
+ }
102731103723
102732103724
 /* Chose the best index to use for each table in the FROM clause.
102733103725
 **
102734103726
 ** This loop fills in the following fields:
102735103727
 **
@@ -102810,10 +103802,11 @@
102810103802
 Bitmask mask; /* Mask of tables not yet ready */
102811103803
 for(j=iFrom, pTabItem=&pTabList->a[j]; j<nTabList; j++, pTabItem++){
102812103804
 int doNotReorder; /* True if this table should not be reordered */
102813103805
 WhereCost sCost; /* Cost information from best[Virtual]Index() */
102814103806
 ExprList *pOrderBy; /* ORDER BY clause for index to optimize */
103807
+ ExprList *pDist; /* DISTINCT clause for index to optimize */
102815103808
102816103809
 doNotReorder = (pTabItem->jointype & (JT_LEFT|JT_CROSS))!=0;
102817103810
 if( j!=iFrom && doNotReorder ) break;
102818103811
 m = getMask(pMaskSet, pTabItem->iCursor);
102819103812
 if( (m & notReady)==0 ){
@@ -102820,10 +103813,11 @@
102820103813
 if( j==iFrom ) iFrom++;
102821103814
 continue;
102822103815
 }
102823103816
 mask = (isOptimal ? m : notReady);
102824103817
 pOrderBy = ((i==0 && ppOrderBy )?*ppOrderBy:0);
103818
+ pDist = (i==0 ? pDistinct : 0);
102825103819
 if( pTabItem->pIndex==0 ) nUnconstrained++;
102826103820
102827103821
 WHERETRACE(("=== trying table %d with isOptimal=%d ===\n",
102828103822
 j, isOptimal));
102829103823
 assert( pTabItem->pTab );
@@ -102834,11 +103828,11 @@
102834103828
 &sCost, pp);
102835103829
 }else
102836103830
 #endif
102837103831
 {
102838103832
 bestBtreeIndex(pParse, pWC, pTabItem, mask, notReady, pOrderBy,
102839
- &sCost);
103833
+ pDist, &sCost);
102840103834
 }
102841103835
 assert( isOptimal || (sCost.used&notReady)==0 );
102842103836
102843103837
 /* If an INDEXED BY clause is present, then the plan must use that
102844103838
 ** index if it uses any index at all */
@@ -102894,10 +103888,14 @@
102894103888
 WHERETRACE(("*** Optimizer selects table %d for loop %d"
102895103889
 " with cost=%g and nRow=%g\n",
102896103890
 bestJ, pLevel-pWInfo->a, bestPlan.rCost, bestPlan.plan.nRow));
102897103891
 if( (bestPlan.plan.wsFlags & WHERE_ORDERBY)!=0 ){
102898103892
 *ppOrderBy = 0;
103893
+ }
103894
+ if( (bestPlan.plan.wsFlags & WHERE_DISTINCT)!=0 ){
103895
+ assert( pWInfo->eDistinct==0 );
103896
+ pWInfo->eDistinct = WHERE_DISTINCT_ORDERED;
102899103897
 }
102900103898
 andFlags &= bestPlan.plan.wsFlags;
102901103899
 pLevel->plan = bestPlan.plan;
102902103900
 testcase( bestPlan.plan.wsFlags & WHERE_INDEXED );
102903103901
 testcase( bestPlan.plan.wsFlags & WHERE_TEMP_INDEX );
@@ -107274,17 +108272,16 @@
107274108272
 #ifndef SQLITE_OMIT_BLOB_LITERAL
107275108273
 case 'x': case 'X': {
107276108274
 testcase( z[0]=='x' ); testcase( z[0]=='X' );
107277108275
 if( z[1]=='\'' ){
107278108276
 *tokenType = TK_BLOB;
107279
- for(i=2; (c=z[i])!=0 && c!='\''; i++){
107280
- if( !sqlite3Isxdigit(c) ){
107281
- *tokenType = TK_ILLEGAL;
107282
- }
108277
+ for(i=2; sqlite3Isxdigit(z[i]); i++){}
108278
+ if( z[i]!='\'' || i%2 ){
108279
+ *tokenType = TK_ILLEGAL;
108280
+ while( z[i] && z[i]!='\'' ){ i++; }
107283108281
 }
107284
- if( i%2 || !c ) *tokenType = TK_ILLEGAL;
107285
- if( c ) i++;
108282
+ if( z[i] ) i++;
107286108283
 return i;
107287108284
 }
107288108285
 /* Otherwise fall through to the next case */
107289108286
 }
107290108287
 #endif
@@ -110726,10 +111723,21 @@
110726111723
 if( sz ) *ppNew = sqlite3ScratchMalloc(sz);
110727111724
 sqlite3ScratchFree(pFree);
110728111725
 break;
110729111726
 }
110730111727
111728
+ /* sqlite3_test_control(SQLITE_TESTCTRL_LOCALTIME_FAULT, int onoff);
111729
+ **
111730
+ ** If parameter onoff is non-zero, configure the wrappers so that all
111731
+ ** subsequent calls to localtime() and variants fail. If onoff is zero,
111732
+ ** undo this setting.
111733
+ */
111734
+ case SQLITE_TESTCTRL_LOCALTIME_FAULT: {
111735
+ sqlite3GlobalConfig.bLocaltimeFault = va_arg(ap, int);
111736
+ break;
111737
+ }
111738
+
110731111739
 }
110732111740
 va_end(ap);
110733111741
 #endif /* SQLITE_OMIT_BUILTIN_TEST */
110734111742
 return rc;
110735111743
 }
@@ -111383,16 +112391,10 @@
111383112391
 ** TODO(shess) Provide a VACUUM type operation to clear out all
111384112392
 ** deletions and duplications. This would basically be a forced merge
111385112393
 ** into a single segment.
111386112394
 */
111387112395
111388
-#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
111389
-
111390
-#if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE)
111391
-# define SQLITE_CORE 1
111392
-#endif
111393
-
111394112396
 /************** Include fts3Int.h in the middle of fts3.c ********************/
111395112397
 /************** Begin file fts3Int.h *****************************************/
111396112398
 /*
111397112399
 ** 2009 Nov 12
111398112400
 **
@@ -111404,18 +112406,33 @@
111404112406
 ** May you share freely, never taking more than you give.
111405112407
 **
111406112408
 ******************************************************************************
111407112409
 **
111408112410
 */
111409
-
111410112411
 #ifndef _FTSINT_H
111411112412
 #define _FTSINT_H
111412112413
111413112414
 #if !defined(NDEBUG) && !defined(SQLITE_DEBUG)
111414112415
 # define NDEBUG 1
111415112416
 #endif
111416112417
112418
+/*
112419
+** FTS4 is really an extension for FTS3. It is enabled using the
112420
+** SQLITE_ENABLE_FTS3 macro. But to avoid confusion we also all
112421
+** the SQLITE_ENABLE_FTS4 macro to serve as an alisse for SQLITE_ENABLE_FTS3.
112422
+*/
112423
+#if defined(SQLITE_ENABLE_FTS4) && !defined(SQLITE_ENABLE_FTS3)
112424
+# define SQLITE_ENABLE_FTS3
112425
+#endif
112426
+
112427
+#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
112428
+
112429
+/* If not building as part of the core, include sqlite3ext.h. */
112430
+#ifndef SQLITE_CORE
112431
+SQLITE_API extern const sqlite3_api_routines *sqlite3_api;
112432
+#endif
112433
+
111417112434
 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/
111418112435
 /************** Begin file fts3_tokenizer.h **********************************/
111419112436
 /*
111420112437
 ** 2006 July 10
111421112438
 **
@@ -111710,16 +112727,39 @@
111710112727
 ** similar macro called ArraySize(). Use a different name to avoid
111711112728
 ** a collision when building an amalgamation with built-in FTS3.
111712112729
 */
111713112730
 #define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0])))
111714112731
112732
+
112733
+#ifndef MIN
112734
+# define MIN(x,y) ((x)<(y)?(x):(y))
112735
+#endif
112736
+
111715112737
 /*
111716112738
 ** Maximum length of a varint encoded integer. The varint format is different
111717112739
 ** from that used by SQLite, so the maximum length is 10, not 9.
111718112740
 */
111719112741
 #define FTS3_VARINT_MAX 10
111720112742
112743
+/*
112744
+** FTS4 virtual tables may maintain multiple indexes - one index of all terms
112745
+** in the document set and zero or more prefix indexes. All indexes are stored
112746
+** as one or more b+-trees in the %_segments and %_segdir tables.
112747
+**
112748
+** It is possible to determine which index a b+-tree belongs to based on the
112749
+** value stored in the "%_segdir.level" column. Given this value L, the index
112750
+** that the b+-tree belongs to is (L<<10). In other words, all b+-trees with
112751
+** level values between 0 and 1023 (inclusive) belong to index 0, all levels
112752
+** between 1024 and 2047 to index 1, and so on.
112753
+**
112754
+** It is considered impossible for an index to use more than 1024 levels. In
112755
+** theory though this may happen, but only after at least
112756
+** (FTS3_MERGE_COUNT^1024) separate flushes of the pending-terms tables.
112757
+*/
112758
+#define FTS3_SEGDIR_MAXLEVEL 1024
112759
+#define FTS3_SEGDIR_MAXLEVEL_STR "1024"
112760
+
111721112761
 /*
111722112762
 ** The testcase() macro is only used by the amalgamation. If undefined,
111723112763
 ** make it a no-op.
111724112764
 */
111725112765
 #ifndef testcase
@@ -111787,14 +112827,15 @@
111787112827
 typedef struct Fts3Cursor Fts3Cursor;
111788112828
 typedef struct Fts3Expr Fts3Expr;
111789112829
 typedef struct Fts3Phrase Fts3Phrase;
111790112830
 typedef struct Fts3PhraseToken Fts3PhraseToken;
111791112831
112832
+typedef struct Fts3Doclist Fts3Doclist;
111792112833
 typedef struct Fts3SegFilter Fts3SegFilter;
111793112834
 typedef struct Fts3DeferredToken Fts3DeferredToken;
111794112835
 typedef struct Fts3SegReader Fts3SegReader;
111795
-typedef struct Fts3SegReaderCursor Fts3SegReaderCursor;
112836
+typedef struct Fts3MultiSegReader Fts3MultiSegReader;
111796112837
111797112838
 /*
111798112839
 ** A connection to a fulltext index is an instance of the following
111799112840
 ** structure. The xCreate and xConnect methods create an instance
111800112841
 ** of this structure and xDestroy and xDisconnect free that instance.
@@ -111811,33 +112852,45 @@
111811112852
 sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */
111812112853
111813112854
 /* Precompiled statements used by the implementation. Each of these
111814112855
 ** statements is run and reset within a single virtual table API call.
111815112856
 */
111816
- sqlite3_stmt *aStmt[24];
112857
+ sqlite3_stmt *aStmt[27];
111817112858
111818112859
 char *zReadExprlist;
111819112860
 char *zWriteExprlist;
111820112861
111821112862
 int nNodeSize; /* Soft limit for node size */
111822112863
 u8 bHasStat; /* True if %_stat table exists */
111823112864
 u8 bHasDocsize; /* True if %_docsize table exists */
112865
+ u8 bDescIdx; /* True if doclists are in reverse order */
111824112866
 int nPgsz; /* Page size for host database */
111825112867
 char *zSegmentsTbl; /* Name of %_segments table */
111826112868
 sqlite3_blob *pSegments; /* Blob handle open on %_segments table */
111827112869
111828
- /* The following hash table is used to buffer pending index updates during
112870
+ /* TODO: Fix the first paragraph of this comment.
112871
+ **
112872
+ ** The following hash table is used to buffer pending index updates during
111829112873
 ** transactions. Variable nPendingData estimates the memory size of the
111830112874
 ** pending data, including hash table overhead, but not malloc overhead.
111831112875
 ** When nPendingData exceeds nMaxPendingData, the buffer is flushed
111832112876
 ** automatically. Variable iPrevDocid is the docid of the most recently
111833112877
 ** inserted record.
112878
+ **
112879
+ ** A single FTS4 table may have multiple full-text indexes. For each index
112880
+ ** there is an entry in the aIndex[] array. Index 0 is an index of all the
112881
+ ** terms that appear in the document set. Each subsequent index in aIndex[]
112882
+ ** is an index of prefixes of a specific length.
111834112883
 */
111835
- int nMaxPendingData;
111836
- int nPendingData;
111837
- sqlite_int64 iPrevDocid;
111838
- Fts3Hash pendingTerms;
112884
+ int nIndex; /* Size of aIndex[] */
112885
+ struct Fts3Index {
112886
+ int nPrefix; /* Prefix length (0 for main terms index) */
112887
+ Fts3Hash hPending; /* Pending terms table for this index */
112888
+ } *aIndex;
112889
+ int nMaxPendingData; /* Max pending data before flush to disk */
112890
+ int nPendingData; /* Current bytes of pending data */
112891
+ sqlite_int64 iPrevDocid; /* Docid of most recently inserted document */
111839112892
111840112893
 #if defined(SQLITE_DEBUG)
111841112894
 /* State variables used for validating that the transaction control
111842112895
 ** methods of the virtual table are called at appropriate times. These
111843112896
 ** values do not contribution to the FTS computation; they are used for
@@ -111864,13 +112917,14 @@
111864112917
 Fts3DeferredToken *pDeferred; /* Deferred search tokens, if any */
111865112918
 sqlite3_int64 iPrevId; /* Previous id read from aDoclist */
111866112919
 char *pNextId; /* Pointer into the body of aDoclist */
111867112920
 char *aDoclist; /* List of docids for full-text queries */
111868112921
 int nDoclist; /* Size of buffer at aDoclist */
111869
- int desc; /* True to sort in descending order */
112922
+ u8 bDesc; /* True to sort in descending order */
111870112923
 int eEvalmode; /* An FTS3_EVAL_XX constant */
111871112924
 int nRowAvg; /* Average size of database rows, in pages */
112925
+ sqlite3_int64 nDoc; /* Documents in table */
111872112926
111873112927
 int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */
111874112928
 u32 *aMatchinfo; /* Information about most recent match */
111875112929
 int nMatchinfo; /* Number of elements in aMatchinfo[] */
111876112930
 char *zMatchinfo; /* Matchinfo specification */
@@ -111897,66 +112951,90 @@
111897112951
 */
111898112952
 #define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */
111899112953
 #define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */
111900112954
 #define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */
111901112955
112956
+
112957
+struct Fts3Doclist {
112958
+ char *aAll; /* Array containing doclist (or NULL) */
112959
+ int nAll; /* Size of a[] in bytes */
112960
+ char *pNextDocid; /* Pointer to next docid */
112961
+
112962
+ sqlite3_int64 iDocid; /* Current docid (if pList!=0) */
112963
+ int bFreeList; /* True if pList should be sqlite3_free()d */
112964
+ char *pList; /* Pointer to position list following iDocid */
112965
+ int nList; /* Length of position list */
112966
+};
112967
+
111902112968
 /*
111903112969
 ** A "phrase" is a sequence of one or more tokens that must match in
111904112970
 ** sequence. A single token is the base case and the most common case.
111905112971
 ** For a sequence of tokens contained in double-quotes (i.e. "one two three")
111906112972
 ** nToken will be the number of tokens in the string.
111907
-**
111908
-** The nDocMatch and nMatch variables contain data that may be used by the
111909
-** matchinfo() function. They are populated when the full-text index is
111910
-** queried for hits on the phrase. If one or more tokens in the phrase
111911
-** are deferred, the nDocMatch and nMatch variables are populated based
111912
-** on the assumption that the
111913112973
 */
111914112974
 struct Fts3PhraseToken {
111915112975
 char *z; /* Text of the token */
111916112976
 int n; /* Number of bytes in buffer z */
111917112977
 int isPrefix; /* True if token ends with a "*" character */
111918
- int bFulltext; /* True if full-text index was used */
111919
- Fts3SegReaderCursor *pSegcsr; /* Segment-reader for this token */
112978
+
112979
+ /* Variables above this point are populated when the expression is
112980
+ ** parsed (by code in fts3_expr.c). Below this point the variables are
112981
+ ** used when evaluating the expression. */
111920112982
 Fts3DeferredToken *pDeferred; /* Deferred token object for this token */
112983
+ Fts3MultiSegReader *pSegcsr; /* Segment-reader for this token */
111921112984
 };
111922112985
111923112986
 struct Fts3Phrase {
111924
- /* Variables populated by fts3_expr.c when parsing a MATCH expression */
112987
+ /* Cache of doclist for this phrase. */
112988
+ Fts3Doclist doclist;
112989
+ int bIncr; /* True if doclist is loaded incrementally */
112990
+ int iDoclistToken;
112991
+
112992
+ /* Variables below this point are populated by fts3_expr.c when parsing
112993
+ ** a MATCH expression. Everything above is part of the evaluation phase.
112994
+ */
111925112995
 int nToken; /* Number of tokens in the phrase */
111926112996
 int iColumn; /* Index of column this phrase must match */
111927
- int isNot; /* Phrase prefixed by unary not (-) operator */
111928112997
 Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */
111929112998
 };
111930112999
111931113000
 /*
111932113001
 ** A tree of these objects forms the RHS of a MATCH operator.
111933113002
 **
111934
-** If Fts3Expr.eType is either FTSQUERY_NEAR or FTSQUERY_PHRASE and isLoaded
111935
-** is true, then aDoclist points to a malloced buffer, size nDoclist bytes,
111936
-** containing the results of the NEAR or phrase query in FTS3 doclist
111937
-** format. As usual, the initial "Length" field found in doclists stored
111938
-** on disk is omitted from this buffer.
113003
+** If Fts3Expr.eType is FTSQUERY_PHRASE and isLoaded is true, then aDoclist
113004
+** points to a malloced buffer, size nDoclist bytes, containing the results
113005
+** of this phrase query in FTS3 doclist format. As usual, the initial
113006
+** "Length" field found in doclists stored on disk is omitted from this
113007
+** buffer.
111939113008
 **
111940
-** Variable pCurrent always points to the start of a docid field within
111941
-** aDoclist. Since the doclist is usually scanned in docid order, this can
111942
-** be used to accelerate seeking to the required docid within the doclist.
113009
+** Variable aMI is used only for FTSQUERY_NEAR nodes to store the global
113010
+** matchinfo data. If it is not NULL, it points to an array of size nCol*3,
113011
+** where nCol is the number of columns in the queried FTS table. The array
113012
+** is populated as follows:
113013
+**
113014
+** aMI[iCol*3 + 0] = Undefined
113015
+** aMI[iCol*3 + 1] = Number of occurrences
113016
+** aMI[iCol*3 + 2] = Number of rows containing at least one instance
113017
+**
113018
+** The aMI array is allocated using sqlite3_malloc(). It should be freed
113019
+** when the expression node is.
111943113020
 */
111944113021
 struct Fts3Expr {
111945113022
 int eType; /* One of the FTSQUERY_XXX values defined below */
111946113023
 int nNear; /* Valid if eType==FTSQUERY_NEAR */
111947113024
 Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */
111948113025
 Fts3Expr *pLeft; /* Left operand */
111949113026
 Fts3Expr *pRight; /* Right operand */
111950113027
 Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */
111951113028
111952
- int isLoaded; /* True if aDoclist/nDoclist are initialized. */
111953
- char *aDoclist; /* Buffer containing doclist */
111954
- int nDoclist; /* Size of aDoclist in bytes */
113029
+ /* The following are used by the fts3_eval.c module. */
113030
+ sqlite3_int64 iDocid; /* Current docid */
113031
+ u8 bEof; /* True this expression is at EOF already */
113032
+ u8 bStart; /* True if iDocid is valid */
113033
+ u8 bDeferred; /* True if this expression is entirely deferred */
111955113034
111956
- sqlite3_int64 iCurrent;
111957
- char *pCurrent;
113035
+ u32 *aMI;
111958113036
 };
111959113037
111960113038
 /*
111961113039
 ** Candidate values for Fts3Query.eType. Note that the order of the first
111962113040
 ** four values is in order of precedence when parsing expressions. For
@@ -111980,35 +113058,36 @@
111980113058
 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *);
111981113059
 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *);
111982113060
 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *);
111983113061
 SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(int, sqlite3_int64,
111984113062
 sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**);
111985
-SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(Fts3Table*,const char*,int,int,Fts3SegReader**);
113063
+SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(
113064
+ Fts3Table*,int,const char*,int,int,Fts3SegReader**);
111986113065
 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *);
111987
-SQLITE_PRIVATE int sqlite3Fts3SegReaderCost(Fts3Cursor *, Fts3SegReader *, int *);
111988
-SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, sqlite3_stmt **);
113066
+SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, int, sqlite3_stmt **);
111989113067
 SQLITE_PRIVATE int sqlite3Fts3ReadLock(Fts3Table *);
111990
-SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*);
113068
+SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*, int*);
111991113069
111992113070
 SQLITE_PRIVATE int sqlite3Fts3SelectDoctotal(Fts3Table *, sqlite3_stmt **);
111993113071
 SQLITE_PRIVATE int sqlite3Fts3SelectDocsize(Fts3Table *, sqlite3_int64, sqlite3_stmt **);
111994113072
111995113073
 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *);
111996113074
 SQLITE_PRIVATE int sqlite3Fts3DeferToken(Fts3Cursor *, Fts3PhraseToken *, int);
111997113075
 SQLITE_PRIVATE int sqlite3Fts3CacheDeferredDoclists(Fts3Cursor *);
111998113076
 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *);
111999
-SQLITE_PRIVATE char *sqlite3Fts3DeferredDoclist(Fts3DeferredToken *, int *);
112000113077
 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *);
112001113078
112002
-#define FTS3_SEGCURSOR_PENDING -1
112003
-#define FTS3_SEGCURSOR_ALL -2
113079
+/* Special values interpreted by sqlite3SegReaderCursor() */
113080
+#define FTS3_SEGCURSOR_PENDING -1
113081
+#define FTS3_SEGCURSOR_ALL -2
112004113082
112005
-SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3SegReaderCursor*, Fts3SegFilter*);
112006
-SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3SegReaderCursor *);
112007
-SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3SegReaderCursor *);
113083
+SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3MultiSegReader*, Fts3SegFilter*);
113084
+SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3MultiSegReader *);
113085
+SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3MultiSegReader *);
113086
+
112008113087
 SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
112009
- Fts3Table *, int, const char *, int, int, int, Fts3SegReaderCursor *);
113088
+ Fts3Table *, int, int, const char *, int, int, int, Fts3MultiSegReader *);
112010113089
112011113090
 /* Flags allowed as part of the 4th argument to SegmentReaderIterate() */
112012113091
 #define FTS3_SEGMENT_REQUIRE_POS 0x00000001
112013113092
 #define FTS3_SEGMENT_IGNORE_EMPTY 0x00000002
112014113093
 #define FTS3_SEGMENT_COLUMN_FILTER 0x00000004
@@ -112021,21 +113100,25 @@
112021113100
 int nTerm;
112022113101
 int iCol;
112023113102
 int flags;
112024113103
 };
112025113104
112026
-struct Fts3SegReaderCursor {
113105
+struct Fts3MultiSegReader {
112027113106
 /* Used internally by sqlite3Fts3SegReaderXXX() calls */
112028113107
 Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */
112029113108
 int nSegment; /* Size of apSegment array */
112030113109
 int nAdvance; /* How many seg-readers to advance */
112031113110
 Fts3SegFilter *pFilter; /* Pointer to filter object */
112032113111
 char *aBuffer; /* Buffer to merge doclists in */
112033113112
 int nBuffer; /* Allocated size of aBuffer[] in bytes */
112034113113
112035
- /* Cost of running this iterator. Used by fts3.c only. */
112036
- int nCost;
113114
+ int iColFilter; /* If >=0, filter for this column */
113115
+ int bRestart;
113116
+
113117
+ /* Used by fts3.c only. */
113118
+ int nCost; /* Cost of running iterator */
113119
+ int bLookup; /* True if a lookup of a single entry. */
112037113120
112038113121
 /* Output values. Valid only after Fts3SegReaderStep() returns SQLITE_ROW. */
112039113122
 char *zTerm; /* Pointer to term buffer */
112040113123
 int nTerm; /* Size of zTerm in bytes */
112041113124
 char *aDoclist; /* Pointer to doclist buffer */
@@ -112046,15 +113129,13 @@
112046113129
 SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *, sqlite3_int64);
112047113130
 SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *, sqlite_int64 *);
112048113131
 SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *, int *);
112049113132
 SQLITE_PRIVATE int sqlite3Fts3VarintLen(sqlite3_uint64);
112050113133
 SQLITE_PRIVATE void sqlite3Fts3Dequote(char *);
113134
+SQLITE_PRIVATE void sqlite3Fts3DoclistPrev(int,char*,int,char**,sqlite3_int64*,int*,u8*);
112051113135
112052
-SQLITE_PRIVATE char *sqlite3Fts3FindPositions(Fts3Cursor *, Fts3Expr *, sqlite3_int64, int);
112053
-SQLITE_PRIVATE int sqlite3Fts3ExprLoadDoclist(Fts3Cursor *, Fts3Expr *);
112054
-SQLITE_PRIVATE int sqlite3Fts3ExprLoadFtDoclist(Fts3Cursor *, Fts3Expr *, char **, int *);
112055
-SQLITE_PRIVATE int sqlite3Fts3ExprNearTrim(Fts3Expr *, Fts3Expr *, int);
113136
+SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *);
112056113137
112057113138
 /* fts3_tokenizer.c */
112058113139
 SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *, int *);
112059113140
 SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *);
112060113141
 SQLITE_PRIVATE int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *,
@@ -112080,20 +113161,43 @@
112080113161
 #endif
112081113162
112082113163
 /* fts3_aux.c */
112083113164
 SQLITE_PRIVATE int sqlite3Fts3InitAux(sqlite3 *db);
112084113165
113166
+SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *);
113167
+
113168
+SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart(
113169
+ Fts3Table*, Fts3MultiSegReader*, int, const char*, int);
113170
+SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext(
113171
+ Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *);
113172
+SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol);
113173
+SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *);
113174
+SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr);
113175
+
113176
+SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(Fts3DeferredToken *, char **, int *);
113177
+
113178
+#endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */
112085113179
 #endif /* _FTSINT_H */
112086113180
112087113181
 /************** End of fts3Int.h *********************************************/
112088113182
 /************** Continuing where we left off in fts3.c ***********************/
113183
+#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
113184
+
113185
+#if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE)
113186
+# define SQLITE_CORE 1
113187
+#endif
112089113188
112090113189
112091113190
 #ifndef SQLITE_CORE
112092113191
 SQLITE_EXTENSION_INIT1
112093113192
 #endif
112094113193
113194
+static int fts3EvalNext(Fts3Cursor *pCsr);
113195
+static int fts3EvalStart(Fts3Cursor *pCsr);
113196
+static int fts3TermSegReaderCursor(
113197
+ Fts3Cursor *, const char *, int, int, Fts3MultiSegReader **);
113198
+
112095113199
 /*
112096113200
 ** Write a 64-bit variable-length integer to memory starting at p[0].
112097113201
 ** The length of data written will be between 1 and FTS3_VARINT_MAX bytes.
112098113202
 ** The number of bytes written is returned.
112099113203
 */
@@ -112200,16 +113304,16 @@
112200113304
112201113305
 /*
112202113306
 ** When this function is called, *pp points to the first byte following a
112203113307
 ** varint that is part of a doclist (or position-list, or any other list
112204113308
 ** of varints). This function moves *pp to point to the start of that varint,
112205
-** and decrements the value stored in *pVal by the varint value.
113309
+** and sets *pVal by the varint value.
112206113310
 **
112207113311
 ** Argument pStart points to the first byte of the doclist that the
112208113312
 ** varint is part of.
112209113313
 */
112210
-static void fts3GetReverseDeltaVarint(
113314
+static void fts3GetReverseVarint(
112211113315
 char **pp,
112212113316
 char *pStart,
112213113317
 sqlite3_int64 *pVal
112214113318
 ){
112215113319
 sqlite3_int64 iVal;
@@ -112221,25 +113325,11 @@
112221113325
 for(p = (*pp)-2; p>=pStart && *p&0x80; p--);
112222113326
 p++;
112223113327
 *pp = p;
112224113328
112225113329
 sqlite3Fts3GetVarint(p, &iVal);
112226
- *pVal -= iVal;
112227
-}
112228
-
112229
-/*
112230
-** As long as *pp has not reached its end (pEnd), then do the same
112231
-** as fts3GetDeltaVarint(): read a single varint and add it to *pVal.
112232
-** But if we have reached the end of the varint, just set *pp=0 and
112233
-** leave *pVal unchanged.
112234
-*/
112235
-static void fts3GetDeltaVarint2(char **pp, char *pEnd, sqlite3_int64 *pVal){
112236
- if( *pp>=pEnd ){
112237
- *pp = 0;
112238
- }else{
112239
- fts3GetDeltaVarint(pp, pVal);
112240
- }
113330
+ *pVal = iVal;
112241113331
 }
112242113332
112243113333
 /*
112244113334
 ** The xDisconnect() virtual table method.
112245113335
 */
@@ -112436,10 +113526,13 @@
112436113526
 rc = sqlite3_prepare(p->db, zSql, -1, &pStmt, 0);
112437113527
 if( rc==SQLITE_OK ){
112438113528
 sqlite3_step(pStmt);
112439113529
 p->nPgsz = sqlite3_column_int(pStmt, 0);
112440113530
 rc = sqlite3_finalize(pStmt);
113531
+ }else if( rc==SQLITE_AUTH ){
113532
+ p->nPgsz = 1024;
113533
+ rc = SQLITE_OK;
112441113534
 }
112442113535
 }
112443113536
 assert( p->nPgsz>0 || rc!=SQLITE_OK );
112444113537
 sqlite3_free(zSql);
112445113538
 *pRc = rc;
@@ -112608,10 +113701,91 @@
112608113701
 fts3Appendf(pRc, &zRet, ",%s(?)", zFunction);
112609113702
 }
112610113703
 sqlite3_free(zFree);
112611113704
 return zRet;
112612113705
 }
113706
+
113707
+/*
113708
+** This function interprets the string at (*pp) as a non-negative integer
113709
+** value. It reads the integer and sets *pnOut to the value read, then
113710
+** sets *pp to point to the byte immediately following the last byte of
113711
+** the integer value.
113712
+**
113713
+** Only decimal digits ('0'..'9') may be part of an integer value.
113714
+**
113715
+** If *pp does not being with a decimal digit SQLITE_ERROR is returned and
113716
+** the output value undefined. Otherwise SQLITE_OK is returned.
113717
+**
113718
+** This function is used when parsing the "prefix=" FTS4 parameter.
113719
+*/
113720
+static int fts3GobbleInt(const char **pp, int *pnOut){
113721
+ const char *p = *pp; /* Iterator pointer */
113722
+ int nInt = 0; /* Output value */
113723
+
113724
+ for(p=*pp; p[0]>='0' && p[0]<='9'; p++){
113725
+ nInt = nInt * 10 + (p[0] - '0');
113726
+ }
113727
+ if( p==*pp ) return SQLITE_ERROR;
113728
+ *pnOut = nInt;
113729
+ *pp = p;
113730
+ return SQLITE_OK;
113731
+}
113732
+
113733
+/*
113734
+** This function is called to allocate an array of Fts3Index structures
113735
+** representing the indexes maintained by the current FTS table. FTS tables
113736
+** always maintain the main "terms" index, but may also maintain one or
113737
+** more "prefix" indexes, depending on the value of the "prefix=" parameter
113738
+** (if any) specified as part of the CREATE VIRTUAL TABLE statement.
113739
+**
113740
+** Argument zParam is passed the value of the "prefix=" option if one was
113741
+** specified, or NULL otherwise.
113742
+**
113743
+** If no error occurs, SQLITE_OK is returned and *apIndex set to point to
113744
+** the allocated array. *pnIndex is set to the number of elements in the
113745
+** array. If an error does occur, an SQLite error code is returned.
113746
+**
113747
+** Regardless of whether or not an error is returned, it is the responsibility
113748
+** of the caller to call sqlite3_free() on the output array to free it.
113749
+*/
113750
+static int fts3PrefixParameter(
113751
+ const char *zParam, /* ABC in prefix=ABC parameter to parse */
113752
+ int *pnIndex, /* OUT: size of *apIndex[] array */
113753
+ struct Fts3Index **apIndex /* OUT: Array of indexes for this table */
113754
+){
113755
+ struct Fts3Index *aIndex; /* Allocated array */
113756
+ int nIndex = 1; /* Number of entries in array */
113757
+
113758
+ if( zParam && zParam[0] ){
113759
+ const char *p;
113760
+ nIndex++;
113761
+ for(p=zParam; *p; p++){
113762
+ if( *p==',' ) nIndex++;
113763
+ }
113764
+ }
113765
+
113766
+ aIndex = sqlite3_malloc(sizeof(struct Fts3Index) * nIndex);
113767
+ *apIndex = aIndex;
113768
+ *pnIndex = nIndex;
113769
+ if( !aIndex ){
113770
+ return SQLITE_NOMEM;
113771
+ }
113772
+
113773
+ memset(aIndex, 0, sizeof(struct Fts3Index) * nIndex);
113774
+ if( zParam ){
113775
+ const char *p = zParam;
113776
+ int i;
113777
+ for(i=1; i<nIndex; i++){
113778
+ int nPrefix;
113779
+ if( fts3GobbleInt(&p, &nPrefix) ) return SQLITE_ERROR;
113780
+ aIndex[i].nPrefix = nPrefix;
113781
+ p++;
113782
+ }
113783
+ }
113784
+
113785
+ return SQLITE_OK;
113786
+}
112613113787
112614113788
 /*
112615113789
 ** This function is the implementation of both the xConnect and xCreate
112616113790
 ** methods of the FTS3 virtual table.
112617113791
 **
@@ -112641,16 +113815,22 @@
112641113815
 int nCol = 0; /* Number of columns in the FTS table */
112642113816
 char *zCsr; /* Space for holding column names */
112643113817
 int nDb; /* Bytes required to hold database name */
112644113818
 int nName; /* Bytes required to hold table name */
112645113819
 int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */
112646
- int bNoDocsize = 0; /* True to omit %_docsize table */
112647113820
 const char **aCol; /* Array of column names */
112648113821
 sqlite3_tokenizer *pTokenizer = 0; /* Tokenizer for this table */
112649113822
112650
- char *zCompress = 0;
112651
- char *zUncompress = 0;
113823
+ int nIndex; /* Size of aIndex[] array */
113824
+ struct Fts3Index *aIndex = 0; /* Array of indexes for this table */
113825
+
113826
+ /* The results of parsing supported FTS4 key=value options: */
113827
+ int bNoDocsize = 0; /* True to omit %_docsize table */
113828
+ int bDescIdx = 0; /* True to store descending indexes */
113829
+ char *zPrefix = 0; /* Prefix parameter value (or NULL) */
113830
+ char *zCompress = 0; /* compress=? parameter (or NULL) */
113831
+ char *zUncompress = 0; /* uncompress=? parameter (or NULL) */
112652113832
112653113833
 assert( strlen(argv[0])==4 );
112654113834
 assert( (sqlite3_strnicmp(argv[0], "fts4", 4)==0 && isFts4)
112655113835
 || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4)
112656113836
 );
@@ -112687,32 +113867,76 @@
112687113867
 rc = sqlite3Fts3InitTokenizer(pHash, &z[9], &pTokenizer, pzErr);
112688113868
 }
112689113869
112690113870
 /* Check if it is an FTS4 special argument. */
112691113871
 else if( isFts4 && fts3IsSpecialColumn(z, &nKey, &zVal) ){
113872
+ struct Fts4Option {
113873
+ const char *zOpt;
113874
+ int nOpt;
113875
+ char **pzVar;
113876
+ } aFts4Opt[] = {
113877
+ { "matchinfo", 9, 0 }, /* 0 -> MATCHINFO */
113878
+ { "prefix", 6, 0 }, /* 1 -> PREFIX */
113879
+ { "compress", 8, 0 }, /* 2 -> COMPRESS */
113880
+ { "uncompress", 10, 0 }, /* 3 -> UNCOMPRESS */
113881
+ { "order", 5, 0 } /* 4 -> ORDER */
113882
+ };
113883
+
113884
+ int iOpt;
112692113885
 if( !zVal ){
112693113886
 rc = SQLITE_NOMEM;
112694
- goto fts3_init_out;
112695
- }
112696
- if( nKey==9 && 0==sqlite3_strnicmp(z, "matchinfo", 9) ){
112697
- if( strlen(zVal)==4 && 0==sqlite3_strnicmp(zVal, "fts3", 4) ){
112698
- bNoDocsize = 1;
112699
- }else{
112700
- *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal);
112701
- rc = SQLITE_ERROR;
112702
- }
112703
- }else if( nKey==8 && 0==sqlite3_strnicmp(z, "compress", 8) ){
112704
- zCompress = zVal;
112705
- zVal = 0;
112706
- }else if( nKey==10 && 0==sqlite3_strnicmp(z, "uncompress", 10) ){
112707
- zUncompress = zVal;
112708
- zVal = 0;
112709
- }else{
112710
- *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z);
112711
- rc = SQLITE_ERROR;
112712
- }
112713
- sqlite3_free(zVal);
113887
+ }else{
113888
+ for(iOpt=0; iOpt<SizeofArray(aFts4Opt); iOpt++){
113889
+ struct Fts4Option *pOp = &aFts4Opt[iOpt];
113890
+ if( nKey==pOp->nOpt && !sqlite3_strnicmp(z, pOp->zOpt, pOp->nOpt) ){
113891
+ break;
113892
+ }
113893
+ }
113894
+ if( iOpt==SizeofArray(aFts4Opt) ){
113895
+ *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z);
113896
+ rc = SQLITE_ERROR;
113897
+ }else{
113898
+ switch( iOpt ){
113899
+ case 0: /* MATCHINFO */
113900
+ if( strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "fts3", 4) ){
113901
+ *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal);
113902
+ rc = SQLITE_ERROR;
113903
+ }
113904
+ bNoDocsize = 1;
113905
+ break;
113906
+
113907
+ case 1: /* PREFIX */
113908
+ sqlite3_free(zPrefix);
113909
+ zPrefix = zVal;
113910
+ zVal = 0;
113911
+ break;
113912
+
113913
+ case 2: /* COMPRESS */
113914
+ sqlite3_free(zCompress);
113915
+ zCompress = zVal;
113916
+ zVal = 0;
113917
+ break;
113918
+
113919
+ case 3: /* UNCOMPRESS */
113920
+ sqlite3_free(zUncompress);
113921
+ zUncompress = zVal;
113922
+ zVal = 0;
113923
+ break;
113924
+
113925
+ case 4: /* ORDER */
113926
+ if( (strlen(zVal)!=3 || sqlite3_strnicmp(zVal, "asc", 3))
113927
+ && (strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "desc", 3))
113928
+ ){
113929
+ *pzErr = sqlite3_mprintf("unrecognized order: %s", zVal);
113930
+ rc = SQLITE_ERROR;
113931
+ }
113932
+ bDescIdx = (zVal[0]=='d' || zVal[0]=='D');
113933
+ break;
113934
+ }
113935
+ }
113936
+ sqlite3_free(zVal);
113937
+ }
112714113938
 }
112715113939
112716113940
 /* Otherwise, the argument is a column name. */
112717113941
 else {
112718113942
 nString += (int)(strlen(z) + 1);
@@ -112732,14 +113956,21 @@
112732113956
 rc = sqlite3Fts3InitTokenizer(pHash, "simple", &pTokenizer, pzErr);
112733113957
 if( rc!=SQLITE_OK ) goto fts3_init_out;
112734113958
 }
112735113959
 assert( pTokenizer );
112736113960
113961
+ rc = fts3PrefixParameter(zPrefix, &nIndex, &aIndex);
113962
+ if( rc==SQLITE_ERROR ){
113963
+ assert( zPrefix );
113964
+ *pzErr = sqlite3_mprintf("error parsing prefix parameter: %s", zPrefix);
113965
+ }
113966
+ if( rc!=SQLITE_OK ) goto fts3_init_out;
112737113967
112738113968
 /* Allocate and populate the Fts3Table structure. */
112739
- nByte = sizeof(Fts3Table) + /* Fts3Table */
113969
+ nByte = sizeof(Fts3Table) + /* Fts3Table */
112740113970
 nCol * sizeof(char *) + /* azColumn */
113971
+ nIndex * sizeof(struct Fts3Index) + /* aIndex */
112741113972
 nName + /* zName */
112742113973
 nDb + /* zDb */
112743113974
 nString; /* Space for azColumn strings */
112744113975
 p = (Fts3Table*)sqlite3_malloc(nByte);
112745113976
 if( p==0 ){
@@ -112750,20 +113981,26 @@
112750113981
 p->db = db;
112751113982
 p->nColumn = nCol;
112752113983
 p->nPendingData = 0;
112753113984
 p->azColumn = (char **)&p[1];
112754113985
 p->pTokenizer = pTokenizer;
112755
- p->nNodeSize = 1000;
112756113986
 p->nMaxPendingData = FTS3_MAX_PENDING_DATA;
112757113987
 p->bHasDocsize = (isFts4 && bNoDocsize==0);
112758113988
 p->bHasStat = isFts4;
113989
+ p->bDescIdx = bDescIdx;
112759113990
 TESTONLY( p->inTransaction = -1 );
112760113991
 TESTONLY( p->mxSavepoint = -1 );
112761
- fts3HashInit(&p->pendingTerms, FTS3_HASH_STRING, 1);
113992
+
113993
+ p->aIndex = (struct Fts3Index *)&p->azColumn[nCol];
113994
+ memcpy(p->aIndex, aIndex, sizeof(struct Fts3Index) * nIndex);
113995
+ p->nIndex = nIndex;
113996
+ for(i=0; i<nIndex; i++){
113997
+ fts3HashInit(&p->aIndex[i].hPending, FTS3_HASH_STRING, 1);
113998
+ }
112762113999
112763114000
 /* Fill in the zName and zDb fields of the vtab structure. */
112764
- zCsr = (char *)&p->azColumn[nCol];
114001
+ zCsr = (char *)&p->aIndex[nIndex];
112765114002
 p->zName = zCsr;
112766114003
 memcpy(zCsr, argv[2], nName);
112767114004
 zCsr += nName;
112768114005
 p->zDb = zCsr;
112769114006
 memcpy(zCsr, argv[1], nDb);
@@ -112770,11 +114007,11 @@
112770114007
 zCsr += nDb;
112771114008
112772114009
 /* Fill in the azColumn array */
112773114010
 for(iCol=0; iCol<nCol; iCol++){
112774114011
 char *z;
112775
- int n;
114012
+ int n = 0;
112776114013
 z = (char *)sqlite3Fts3NextToken(aCol[iCol], &n);
112777114014
 memcpy(zCsr, z, n);
112778114015
 zCsr[n] = '\0';
112779114016
 sqlite3Fts3Dequote(zCsr);
112780114017
 p->azColumn[iCol] = zCsr;
@@ -112797,19 +114034,20 @@
112797114034
 if( isCreate ){
112798114035
 rc = fts3CreateTables(p);
112799114036
 }
112800114037
112801114038
 /* Figure out the page-size for the database. This is required in order to
112802
- ** estimate the cost of loading large doclists from the database (see
112803
- ** function sqlite3Fts3SegReaderCost() for details).
112804
- */
114039
+ ** estimate the cost of loading large doclists from the database. */
112805114040
 fts3DatabasePageSize(&rc, p);
114041
+ p->nNodeSize = p->nPgsz-35;
112806114042
112807114043
 /* Declare the table schema to SQLite. */
112808114044
 fts3DeclareVtab(&rc, p);
112809114045
112810114046
 fts3_init_out:
114047
+ sqlite3_free(zPrefix);
114048
+ sqlite3_free(aIndex);
112811114049
 sqlite3_free(zCompress);
112812114050
 sqlite3_free(zUncompress);
112813114051
 sqlite3_free((void *)aCol);
112814114052
 if( rc!=SQLITE_OK ){
112815114053
 if( p ){
@@ -112816,10 +114054,11 @@
112816114054
 fts3DisconnectMethod((sqlite3_vtab *)p);
112817114055
 }else if( pTokenizer ){
112818114056
 pTokenizer->pModule->xDestroy(pTokenizer);
112819114057
 }
112820114058
 }else{
114059
+ assert( p->pSegments==0 );
112821114060
 *ppVTab = &p->base;
112822114061
 }
112823114062
 return rc;
112824114063
 }
112825114064
@@ -112913,14 +114152,15 @@
112913114152
 if( pOrder->desc ){
112914114153
 pInfo->idxStr = "DESC";
112915114154
 }else{
112916114155
 pInfo->idxStr = "ASC";
112917114156
 }
114157
+ pInfo->orderByConsumed = 1;
112918114158
 }
112919
- pInfo->orderByConsumed = 1;
112920114159
 }
112921114160
114161
+ assert( p->pSegments==0 );
112922114162
 return SQLITE_OK;
112923114163
 }
112924114164
112925114165
 /*
112926114166
 ** Implementation of xOpen method.
@@ -112952,10 +114192,11 @@
112952114192
 sqlite3_finalize(pCsr->pStmt);
112953114193
 sqlite3Fts3ExprFree(pCsr->pExpr);
112954114194
 sqlite3Fts3FreeDeferredTokens(pCsr);
112955114195
 sqlite3_free(pCsr->aDoclist);
112956114196
 sqlite3_free(pCsr->aMatchinfo);
114197
+ assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
112957114198
 sqlite3_free(pCsr);
112958114199
 return SQLITE_OK;
112959114200
 }
112960114201
112961114202
 /*
@@ -112963,12 +114204,12 @@
112963114204
 ** of the %_content table that contains the last match. Return
112964114205
 ** SQLITE_OK on success.
112965114206
 */
112966114207
 static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){
112967114208
 if( pCsr->isRequireSeek ){
112968
- pCsr->isRequireSeek = 0;
112969114209
 sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId);
114210
+ pCsr->isRequireSeek = 0;
112970114211
 if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){
112971114212
 return SQLITE_OK;
112972114213
 }else{
112973114214
 int rc = sqlite3_reset(pCsr->pStmt);
112974114215
 if( rc==SQLITE_OK ){
@@ -113145,21 +114386,21 @@
113145114386
 if( rc==SQLITE_OK && iHeight>1 ){
113146114387
 char *zBlob = 0; /* Blob read from %_segments table */
113147114388
 int nBlob; /* Size of zBlob in bytes */
113148114389
113149114390
 if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){
113150
- rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob);
114391
+ rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob, 0);
113151114392
 if( rc==SQLITE_OK ){
113152114393
 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, 0);
113153114394
 }
113154114395
 sqlite3_free(zBlob);
113155114396
 piLeaf = 0;
113156114397
 zBlob = 0;
113157114398
 }
113158114399
113159114400
 if( rc==SQLITE_OK ){
113160
- rc = sqlite3Fts3ReadBlock(p, piLeaf ? *piLeaf : *piLeaf2, &zBlob, &nBlob);
114401
+ rc = sqlite3Fts3ReadBlock(p, piLeaf?*piLeaf:*piLeaf2, &zBlob, &nBlob, 0);
113161114402
 }
113162114403
 if( rc==SQLITE_OK ){
113163114404
 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2);
113164114405
 }
113165114406
 sqlite3_free(zBlob);
@@ -113393,12 +114634,10 @@
113393114634
 *pp1 = p1 + 1;
113394114635
 *pp2 = p2 + 1;
113395114636
 }
113396114637
113397114638
 /*
113398
-** nToken==1 searches for adjacent positions.
113399
-**
113400114639
 ** This function is used to merge two position lists into one. When it is
113401114640
 ** called, *pp1 and *pp2 must both point to position lists. A position-list is
113402114641
 ** the part of a doclist that follows each document id. For example, if a row
113403114642
 ** contains:
113404114643
 **
@@ -113414,10 +114653,12 @@
113414114653
 ** If isSaveLeft is 0, an entry is added to the output position list for
113415114654
 ** each position in *pp2 for which there exists one or more positions in
113416114655
 ** *pp1 so that (pos(*pp2)>pos(*pp1) && pos(*pp2)-pos(*pp1)<=nToken). i.e.
113417114656
 ** when the *pp1 token appears before the *pp2 token, but not more than nToken
113418114657
 ** slots before it.
114658
+**
114659
+** e.g. nToken==1 searches for adjacent positions.
113419114660
 */
113420114661
 static int fts3PoslistPhraseMerge(
113421114662
 char **pp, /* IN/OUT: Preallocated output buffer */
113422114663
 int nToken, /* Maximum difference in token positions */
113423114664
 int isSaveLeft, /* Save the left position */
@@ -113531,11 +114772,23 @@
113531114772
 *pp = p;
113532114773
 return 1;
113533114774
 }
113534114775
113535114776
 /*
113536
-** Merge two position-lists as required by the NEAR operator.
114777
+** Merge two position-lists as required by the NEAR operator. The argument
114778
+** position lists correspond to the left and right phrases of an expression
114779
+** like:
114780
+**
114781
+** "phrase 1" NEAR "phrase number 2"
114782
+**
114783
+** Position list *pp1 corresponds to the left-hand side of the NEAR
114784
+** expression and *pp2 to the right. As usual, the indexes in the position
114785
+** lists are the offsets of the last token in each phrase (tokens "1" and "2"
114786
+** in the example above).
114787
+**
114788
+** The output position list - written to *pp - is a copy of *pp2 with those
114789
+** entries that are not sufficiently NEAR entries in *pp1 removed.
113537114790
 */
113538114791
 static int fts3PoslistNearMerge(
113539114792
 char **pp, /* Output buffer */
113540114793
 char *aTmp, /* Temporary buffer space */
113541114794
 int nRight, /* Maximum difference in token positions */
@@ -113544,230 +114797,248 @@
113544114797
 char **pp2 /* IN/OUT: Right input list */
113545114798
 ){
113546114799
 char *p1 = *pp1;
113547114800
 char *p2 = *pp2;
113548114801
113549
- if( !pp ){
113550
- if( fts3PoslistPhraseMerge(0, nRight, 0, 0, pp1, pp2) ) return 1;
113551
- *pp1 = p1;
113552
- *pp2 = p2;
113553
- return fts3PoslistPhraseMerge(0, nLeft, 0, 0, pp2, pp1);
113554
- }else{
113555
- char *pTmp1 = aTmp;
113556
- char *pTmp2;
113557
- char *aTmp2;
113558
- int res = 1;
113559
-
113560
- fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2);
113561
- aTmp2 = pTmp2 = pTmp1;
113562
- *pp1 = p1;
113563
- *pp2 = p2;
113564
- fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1);
113565
- if( pTmp1!=aTmp && pTmp2!=aTmp2 ){
113566
- fts3PoslistMerge(pp, &aTmp, &aTmp2);
113567
- }else if( pTmp1!=aTmp ){
113568
- fts3PoslistCopy(pp, &aTmp);
113569
- }else if( pTmp2!=aTmp2 ){
113570
- fts3PoslistCopy(pp, &aTmp2);
113571
- }else{
113572
- res = 0;
113573
- }
113574
-
113575
- return res;
113576
- }
113577
-}
113578
-
113579
-/*
113580
-** Values that may be used as the first parameter to fts3DoclistMerge().
113581
-*/
113582
-#define MERGE_NOT 2 /* D + D -> D */
113583
-#define MERGE_AND 3 /* D + D -> D */
113584
-#define MERGE_OR 4 /* D + D -> D */
113585
-#define MERGE_POS_OR 5 /* P + P -> P */
113586
-#define MERGE_PHRASE 6 /* P + P -> D */
113587
-#define MERGE_POS_PHRASE 7 /* P + P -> P */
113588
-#define MERGE_NEAR 8 /* P + P -> D */
113589
-#define MERGE_POS_NEAR 9 /* P + P -> P */
113590
-
113591
-/*
113592
-** Merge the two doclists passed in buffer a1 (size n1 bytes) and a2
113593
-** (size n2 bytes). The output is written to pre-allocated buffer aBuffer,
113594
-** which is guaranteed to be large enough to hold the results. The number
113595
-** of bytes written to aBuffer is stored in *pnBuffer before returning.
113596
-**
113597
-** If successful, SQLITE_OK is returned. Otherwise, if a malloc error
113598
-** occurs while allocating a temporary buffer as part of the merge operation,
113599
-** SQLITE_NOMEM is returned.
113600
-*/
113601
-static int fts3DoclistMerge(
113602
- int mergetype, /* One of the MERGE_XXX constants */
113603
- int nParam1, /* Used by MERGE_NEAR and MERGE_POS_NEAR */
113604
- int nParam2, /* Used by MERGE_NEAR and MERGE_POS_NEAR */
113605
- char *aBuffer, /* Pre-allocated output buffer */
113606
- int *pnBuffer, /* OUT: Bytes written to aBuffer */
113607
- char *a1, /* Buffer containing first doclist */
113608
- int n1, /* Size of buffer a1 */
113609
- char *a2, /* Buffer containing second doclist */
113610
- int n2, /* Size of buffer a2 */
113611
- int *pnDoc /* OUT: Number of docids in output */
114802
+ char *pTmp1 = aTmp;
114803
+ char *pTmp2;
114804
+ char *aTmp2;
114805
+ int res = 1;
114806
+
114807
+ fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2);
114808
+ aTmp2 = pTmp2 = pTmp1;
114809
+ *pp1 = p1;
114810
+ *pp2 = p2;
114811
+ fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1);
114812
+ if( pTmp1!=aTmp && pTmp2!=aTmp2 ){
114813
+ fts3PoslistMerge(pp, &aTmp, &aTmp2);
114814
+ }else if( pTmp1!=aTmp ){
114815
+ fts3PoslistCopy(pp, &aTmp);
114816
+ }else if( pTmp2!=aTmp2 ){
114817
+ fts3PoslistCopy(pp, &aTmp2);
114818
+ }else{
114819
+ res = 0;
114820
+ }
114821
+
114822
+ return res;
114823
+}
114824
+
114825
+/*
114826
+** An instance of this function is used to merge together the (potentially
114827
+** large number of) doclists for each term that matches a prefix query.
114828
+** See function fts3TermSelectMerge() for details.
114829
+*/
114830
+typedef struct TermSelect TermSelect;
114831
+struct TermSelect {
114832
+ char *aaOutput[16]; /* Malloc'd output buffers */
114833
+ int anOutput[16]; /* Size each output buffer in bytes */
114834
+};
114835
+
114836
+/*
114837
+** This function is used to read a single varint from a buffer. Parameter
114838
+** pEnd points 1 byte past the end of the buffer. When this function is
114839
+** called, if *pp points to pEnd or greater, then the end of the buffer
114840
+** has been reached. In this case *pp is set to 0 and the function returns.
114841
+**
114842
+** If *pp does not point to or past pEnd, then a single varint is read
114843
+** from *pp. *pp is then set to point 1 byte past the end of the read varint.
114844
+**
114845
+** If bDescIdx is false, the value read is added to *pVal before returning.
114846
+** If it is true, the value read is subtracted from *pVal before this
114847
+** function returns.
114848
+*/
114849
+static void fts3GetDeltaVarint3(
114850
+ char **pp, /* IN/OUT: Point to read varint from */
114851
+ char *pEnd, /* End of buffer */
114852
+ int bDescIdx, /* True if docids are descending */
114853
+ sqlite3_int64 *pVal /* IN/OUT: Integer value */
114854
+){
114855
+ if( *pp>=pEnd ){
114856
+ *pp = 0;
114857
+ }else{
114858
+ sqlite3_int64 iVal;
114859
+ *pp += sqlite3Fts3GetVarint(*pp, &iVal);
114860
+ if( bDescIdx ){
114861
+ *pVal -= iVal;
114862
+ }else{
114863
+ *pVal += iVal;
114864
+ }
114865
+ }
114866
+}
114867
+
114868
+/*
114869
+** This function is used to write a single varint to a buffer. The varint
114870
+** is written to *pp. Before returning, *pp is set to point 1 byte past the
114871
+** end of the value written.
114872
+**
114873
+** If *pbFirst is zero when this function is called, the value written to
114874
+** the buffer is that of parameter iVal.
114875
+**
114876
+** If *pbFirst is non-zero when this function is called, then the value
114877
+** written is either (iVal-*piPrev) (if bDescIdx is zero) or (*piPrev-iVal)
114878
+** (if bDescIdx is non-zero).
114879
+**
114880
+** Before returning, this function always sets *pbFirst to 1 and *piPrev
114881
+** to the value of parameter iVal.
114882
+*/
114883
+static void fts3PutDeltaVarint3(
114884
+ char **pp, /* IN/OUT: Output pointer */
114885
+ int bDescIdx, /* True for descending docids */
114886
+ sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */
114887
+ int *pbFirst, /* IN/OUT: True after first int written */
114888
+ sqlite3_int64 iVal /* Write this value to the list */
114889
+){
114890
+ sqlite3_int64 iWrite;
114891
+ if( bDescIdx==0 || *pbFirst==0 ){
114892
+ iWrite = iVal - *piPrev;
114893
+ }else{
114894
+ iWrite = *piPrev - iVal;
114895
+ }
114896
+ assert( *pbFirst || *piPrev==0 );
114897
+ assert( *pbFirst==0 || iWrite>0 );
114898
+ *pp += sqlite3Fts3PutVarint(*pp, iWrite);
114899
+ *piPrev = iVal;
114900
+ *pbFirst = 1;
114901
+}
114902
+
114903
+
114904
+/*
114905
+** This macro is used by various functions that merge doclists. The two
114906
+** arguments are 64-bit docid values. If the value of the stack variable
114907
+** bDescDoclist is 0 when this macro is invoked, then it returns (i1-i2).
114908
+** Otherwise, (i2-i1).
114909
+**
114910
+** Using this makes it easier to write code that can merge doclists that are
114911
+** sorted in either ascending or descending order.
114912
+*/
114913
+#define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i1-i2))
114914
+
114915
+/*
114916
+** This function does an "OR" merge of two doclists (output contains all
114917
+** positions contained in either argument doclist). If the docids in the
114918
+** input doclists are sorted in ascending order, parameter bDescDoclist
114919
+** should be false. If they are sorted in ascending order, it should be
114920
+** passed a non-zero value.
114921
+**
114922
+** If no error occurs, *paOut is set to point at an sqlite3_malloc'd buffer
114923
+** containing the output doclist and SQLITE_OK is returned. In this case
114924
+** *pnOut is set to the number of bytes in the output doclist.
114925
+**
114926
+** If an error occurs, an SQLite error code is returned. The output values
114927
+** are undefined in this case.
114928
+*/
114929
+static int fts3DoclistOrMerge(
114930
+ int bDescDoclist, /* True if arguments are desc */
114931
+ char *a1, int n1, /* First doclist */
114932
+ char *a2, int n2, /* Second doclist */
114933
+ char **paOut, int *pnOut /* OUT: Malloc'd doclist */
114934
+){
114935
+ sqlite3_int64 i1 = 0;
114936
+ sqlite3_int64 i2 = 0;
114937
+ sqlite3_int64 iPrev = 0;
114938
+ char *pEnd1 = &a1[n1];
114939
+ char *pEnd2 = &a2[n2];
114940
+ char *p1 = a1;
114941
+ char *p2 = a2;
114942
+ char *p;
114943
+ char *aOut;
114944
+ int bFirstOut = 0;
114945
+
114946
+ *paOut = 0;
114947
+ *pnOut = 0;
114948
+ aOut = sqlite3_malloc(n1+n2);
114949
+ if( !aOut ) return SQLITE_NOMEM;
114950
+
114951
+ p = aOut;
114952
+ fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1);
114953
+ fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2);
114954
+ while( p1 || p2 ){
114955
+ sqlite3_int64 iDiff = DOCID_CMP(i1, i2);
114956
+
114957
+ if( p2 && p1 && iDiff==0 ){
114958
+ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
114959
+ fts3PoslistMerge(&p, &p1, &p2);
114960
+ fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
114961
+ fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
114962
+ }else if( !p2 || (p1 && iDiff<0) ){
114963
+ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
114964
+ fts3PoslistCopy(&p, &p1);
114965
+ fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
114966
+ }else{
114967
+ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i2);
114968
+ fts3PoslistCopy(&p, &p2);
114969
+ fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
114970
+ }
114971
+ }
114972
+
114973
+ *paOut = aOut;
114974
+ *pnOut = (p-aOut);
114975
+ return SQLITE_OK;
114976
+}
114977
+
114978
+/*
114979
+** This function does a "phrase" merge of two doclists. In a phrase merge,
114980
+** the output contains a copy of each position from the right-hand input
114981
+** doclist for which there is a position in the left-hand input doclist
114982
+** exactly nDist tokens before it.
114983
+**
114984
+** If the docids in the input doclists are sorted in ascending order,
114985
+** parameter bDescDoclist should be false. If they are sorted in ascending
114986
+** order, it should be passed a non-zero value.
114987
+**
114988
+** The right-hand input doclist is overwritten by this function.
114989
+*/
114990
+static void fts3DoclistPhraseMerge(
114991
+ int bDescDoclist, /* True if arguments are desc */
114992
+ int nDist, /* Distance from left to right (1=adjacent) */
114993
+ char *aLeft, int nLeft, /* Left doclist */
114994
+ char *aRight, int *pnRight /* IN/OUT: Right/output doclist */
113612114995
 ){
113613114996
 sqlite3_int64 i1 = 0;
113614114997
 sqlite3_int64 i2 = 0;
113615114998
 sqlite3_int64 iPrev = 0;
113616
-
113617
- char *p = aBuffer;
113618
- char *p1 = a1;
113619
- char *p2 = a2;
113620
- char *pEnd1 = &a1[n1];
113621
- char *pEnd2 = &a2[n2];
113622
- int nDoc = 0;
113623
-
113624
- assert( mergetype==MERGE_OR || mergetype==MERGE_POS_OR
113625
- || mergetype==MERGE_AND || mergetype==MERGE_NOT
113626
- || mergetype==MERGE_PHRASE || mergetype==MERGE_POS_PHRASE
113627
- || mergetype==MERGE_NEAR || mergetype==MERGE_POS_NEAR
113628
- );
113629
-
113630
- if( !aBuffer ){
113631
- *pnBuffer = 0;
113632
- return SQLITE_NOMEM;
113633
- }
113634
-
113635
- /* Read the first docid from each doclist */
113636
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113637
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113638
-
113639
- switch( mergetype ){
113640
- case MERGE_OR:
113641
- case MERGE_POS_OR:
113642
- while( p1 || p2 ){
113643
- if( p2 && p1 && i1==i2 ){
113644
- fts3PutDeltaVarint(&p, &iPrev, i1);
113645
- if( mergetype==MERGE_POS_OR ) fts3PoslistMerge(&p, &p1, &p2);
113646
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113647
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113648
- }else if( !p2 || (p1 && i1<i2) ){
113649
- fts3PutDeltaVarint(&p, &iPrev, i1);
113650
- if( mergetype==MERGE_POS_OR ) fts3PoslistCopy(&p, &p1);
113651
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113652
- }else{
113653
- fts3PutDeltaVarint(&p, &iPrev, i2);
113654
- if( mergetype==MERGE_POS_OR ) fts3PoslistCopy(&p, &p2);
113655
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113656
- }
113657
- }
113658
- break;
113659
-
113660
- case MERGE_AND:
113661
- while( p1 && p2 ){
113662
- if( i1==i2 ){
113663
- fts3PutDeltaVarint(&p, &iPrev, i1);
113664
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113665
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113666
- nDoc++;
113667
- }else if( i1<i2 ){
113668
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113669
- }else{
113670
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113671
- }
113672
- }
113673
- break;
113674
-
113675
- case MERGE_NOT:
113676
- while( p1 ){
113677
- if( p2 && i1==i2 ){
113678
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113679
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113680
- }else if( !p2 || i1<i2 ){
113681
- fts3PutDeltaVarint(&p, &iPrev, i1);
113682
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113683
- }else{
113684
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113685
- }
113686
- }
113687
- break;
113688
-
113689
- case MERGE_POS_PHRASE:
113690
- case MERGE_PHRASE: {
113691
- char **ppPos = (mergetype==MERGE_PHRASE ? 0 : &p);
113692
- while( p1 && p2 ){
113693
- if( i1==i2 ){
113694
- char *pSave = p;
113695
- sqlite3_int64 iPrevSave = iPrev;
113696
- fts3PutDeltaVarint(&p, &iPrev, i1);
113697
- if( 0==fts3PoslistPhraseMerge(ppPos, nParam1, 0, 1, &p1, &p2) ){
113698
- p = pSave;
113699
- iPrev = iPrevSave;
113700
- }else{
113701
- nDoc++;
113702
- }
113703
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113704
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113705
- }else if( i1<i2 ){
113706
- fts3PoslistCopy(0, &p1);
113707
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113708
- }else{
113709
- fts3PoslistCopy(0, &p2);
113710
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113711
- }
113712
- }
113713
- break;
113714
- }
113715
-
113716
- default: assert( mergetype==MERGE_POS_NEAR || mergetype==MERGE_NEAR ); {
113717
- char *aTmp = 0;
113718
- char **ppPos = 0;
113719
-
113720
- if( mergetype==MERGE_POS_NEAR ){
113721
- ppPos = &p;
113722
- aTmp = sqlite3_malloc(2*(n1+n2+1));
113723
- if( !aTmp ){
113724
- return SQLITE_NOMEM;
113725
- }
113726
- }
113727
-
113728
- while( p1 && p2 ){
113729
- if( i1==i2 ){
113730
- char *pSave = p;
113731
- sqlite3_int64 iPrevSave = iPrev;
113732
- fts3PutDeltaVarint(&p, &iPrev, i1);
113733
-
113734
- if( !fts3PoslistNearMerge(ppPos, aTmp, nParam1, nParam2, &p1, &p2) ){
113735
- iPrev = iPrevSave;
113736
- p = pSave;
113737
- }
113738
-
113739
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113740
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113741
- }else if( i1<i2 ){
113742
- fts3PoslistCopy(0, &p1);
113743
- fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113744
- }else{
113745
- fts3PoslistCopy(0, &p2);
113746
- fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113747
- }
113748
- }
113749
- sqlite3_free(aTmp);
113750
- break;
113751
- }
113752
- }
113753
-
113754
- if( pnDoc ) *pnDoc = nDoc;
113755
- *pnBuffer = (int)(p-aBuffer);
113756
- return SQLITE_OK;
113757
-}
113758
-
113759
-/*
113760
-** A pointer to an instance of this structure is used as the context
113761
-** argument to sqlite3Fts3SegReaderIterate()
113762
-*/
113763
-typedef struct TermSelect TermSelect;
113764
-struct TermSelect {
113765
- int isReqPos;
113766
- char *aaOutput[16]; /* Malloc'd output buffer */
113767
- int anOutput[16]; /* Size of output in bytes */
113768
-};
114999
+ char *pEnd1 = &aLeft[nLeft];
115000
+ char *pEnd2 = &aRight[*pnRight];
115001
+ char *p1 = aLeft;
115002
+ char *p2 = aRight;
115003
+ char *p;
115004
+ int bFirstOut = 0;
115005
+ char *aOut = aRight;
115006
+
115007
+ assert( nDist>0 );
115008
+
115009
+ p = aOut;
115010
+ fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1);
115011
+ fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2);
115012
+
115013
+ while( p1 && p2 ){
115014
+ sqlite3_int64 iDiff = DOCID_CMP(i1, i2);
115015
+ if( iDiff==0 ){
115016
+ char *pSave = p;
115017
+ sqlite3_int64 iPrevSave = iPrev;
115018
+ int bFirstOutSave = bFirstOut;
115019
+
115020
+ fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
115021
+ if( 0==fts3PoslistPhraseMerge(&p, nDist, 0, 1, &p1, &p2) ){
115022
+ p = pSave;
115023
+ iPrev = iPrevSave;
115024
+ bFirstOut = bFirstOutSave;
115025
+ }
115026
+ fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
115027
+ fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
115028
+ }else if( iDiff<0 ){
115029
+ fts3PoslistCopy(0, &p1);
115030
+ fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
115031
+ }else{
115032
+ fts3PoslistCopy(0, &p2);
115033
+ fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
115034
+ }
115035
+ }
115036
+
115037
+ *pnRight = p - aOut;
115038
+}
115039
+
113769115040
113770115041
 /*
113771115042
 ** Merge all doclists in the TermSelect.aaOutput[] array into a single
113772115043
 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all
113773115044
 ** other doclists (except the aaOutput[0] one) and return SQLITE_OK.
@@ -113774,12 +115045,11 @@
113774115045
 **
113775115046
 ** If an OOM error occurs, return SQLITE_NOMEM. In this case it is
113776115047
 ** the responsibility of the caller to free any doclists left in the
113777115048
 ** TermSelect.aaOutput[] array.
113778115049
 */
113779
-static int fts3TermSelectMerge(TermSelect *pTS){
113780
- int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR);
115050
+static int fts3TermSelectFinishMerge(Fts3Table *p, TermSelect *pTS){
113781115051
 char *aOut = 0;
113782115052
 int nOut = 0;
113783115053
 int i;
113784115054
113785115055
 /* Loop through the doclists in the aaOutput[] array. Merge them all
@@ -113790,19 +115060,21 @@
113790115060
 if( !aOut ){
113791115061
 aOut = pTS->aaOutput[i];
113792115062
 nOut = pTS->anOutput[i];
113793115063
 pTS->aaOutput[i] = 0;
113794115064
 }else{
113795
- int nNew = nOut + pTS->anOutput[i];
113796
- char *aNew = sqlite3_malloc(nNew);
113797
- if( !aNew ){
115065
+ int nNew;
115066
+ char *aNew;
115067
+
115068
+ int rc = fts3DoclistOrMerge(p->bDescIdx,
115069
+ pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, &aNew, &nNew
115070
+ );
115071
+ if( rc!=SQLITE_OK ){
113798115072
 sqlite3_free(aOut);
113799
- return SQLITE_NOMEM;
115073
+ return rc;
113800115074
 }
113801
- fts3DoclistMerge(mergetype, 0, 0,
113802
- aNew, &nNew, pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, 0
113803
- );
115075
+
113804115076
 sqlite3_free(pTS->aaOutput[i]);
113805115077
 sqlite3_free(aOut);
113806115078
 pTS->aaOutput[i] = 0;
113807115079
 aOut = aNew;
113808115080
 nOut = nNew;
@@ -113814,288 +115086,329 @@
113814115086
 pTS->anOutput[0] = nOut;
113815115087
 return SQLITE_OK;
113816115088
 }
113817115089
113818115090
 /*
113819
-** This function is used as the sqlite3Fts3SegReaderIterate() callback when
113820
-** querying the full-text index for a doclist associated with a term or
113821
-** term-prefix.
115091
+** Merge the doclist aDoclist/nDoclist into the TermSelect object passed
115092
+** as the first argument. The merge is an "OR" merge (see function
115093
+** fts3DoclistOrMerge() for details).
115094
+**
115095
+** This function is called with the doclist for each term that matches
115096
+** a queried prefix. It merges all these doclists into one, the doclist
115097
+** for the specified prefix. Since there can be a very large number of
115098
+** doclists to merge, the merging is done pair-wise using the TermSelect
115099
+** object.
115100
+**
115101
+** This function returns SQLITE_OK if the merge is successful, or an
115102
+** SQLite error code (SQLITE_NOMEM) if an error occurs.
113822115103
 */
113823
-static int fts3TermSelectCb(
113824
- Fts3Table *p, /* Virtual table object */
113825
- void *pContext, /* Pointer to TermSelect structure */
113826
- char *zTerm,
113827
- int nTerm,
113828
- char *aDoclist,
113829
- int nDoclist
115104
+static int fts3TermSelectMerge(
115105
+ Fts3Table *p, /* FTS table handle */
115106
+ TermSelect *pTS, /* TermSelect object to merge into */
115107
+ char *aDoclist, /* Pointer to doclist */
115108
+ int nDoclist /* Size of aDoclist in bytes */
113830115109
 ){
113831
- TermSelect *pTS = (TermSelect *)pContext;
113832
-
113833
- UNUSED_PARAMETER(p);
113834
- UNUSED_PARAMETER(zTerm);
113835
- UNUSED_PARAMETER(nTerm);
113836
-
113837115110
 if( pTS->aaOutput[0]==0 ){
113838115111
 /* If this is the first term selected, copy the doclist to the output
113839
- ** buffer using memcpy(). TODO: Add a way to transfer control of the
113840
- ** aDoclist buffer from the caller so as to avoid the memcpy().
113841
- */
115112
+ ** buffer using memcpy(). */
113842115113
 pTS->aaOutput[0] = sqlite3_malloc(nDoclist);
113843115114
 pTS->anOutput[0] = nDoclist;
113844115115
 if( pTS->aaOutput[0] ){
113845115116
 memcpy(pTS->aaOutput[0], aDoclist, nDoclist);
113846115117
 }else{
113847115118
 return SQLITE_NOMEM;
113848115119
 }
113849115120
 }else{
113850
- int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR);
113851115121
 char *aMerge = aDoclist;
113852115122
 int nMerge = nDoclist;
113853115123
 int iOut;
113854115124
113855115125
 for(iOut=0; iOut<SizeofArray(pTS->aaOutput); iOut++){
113856
- char *aNew;
113857
- int nNew;
113858115126
 if( pTS->aaOutput[iOut]==0 ){
113859115127
 assert( iOut>0 );
113860115128
 pTS->aaOutput[iOut] = aMerge;
113861115129
 pTS->anOutput[iOut] = nMerge;
113862115130
 break;
113863
- }
113864
-
113865
- nNew = nMerge + pTS->anOutput[iOut];
113866
- aNew = sqlite3_malloc(nNew);
113867
- if( !aNew ){
113868
- if( aMerge!=aDoclist ){
113869
- sqlite3_free(aMerge);
113870
- }
113871
- return SQLITE_NOMEM;
113872
- }
113873
- fts3DoclistMerge(mergetype, 0, 0, aNew, &nNew,
113874
- pTS->aaOutput[iOut], pTS->anOutput[iOut], aMerge, nMerge, 0
113875
- );
113876
-
113877
- if( iOut>0 ) sqlite3_free(aMerge);
113878
- sqlite3_free(pTS->aaOutput[iOut]);
113879
- pTS->aaOutput[iOut] = 0;
113880
-
113881
- aMerge = aNew;
113882
- nMerge = nNew;
113883
- if( (iOut+1)==SizeofArray(pTS->aaOutput) ){
113884
- pTS->aaOutput[iOut] = aMerge;
113885
- pTS->anOutput[iOut] = nMerge;
113886
- }
113887
- }
113888
- }
113889
- return SQLITE_OK;
113890
-}
113891
-
113892
-static int fts3DeferredTermSelect(
113893
- Fts3DeferredToken *pToken, /* Phrase token */
113894
- int isTermPos, /* True to include positions */
113895
- int *pnOut, /* OUT: Size of list */
113896
- char **ppOut /* OUT: Body of list */
113897
-){
113898
- char *aSource;
113899
- int nSource;
113900
-
113901
- aSource = sqlite3Fts3DeferredDoclist(pToken, &nSource);
113902
- if( !aSource ){
113903
- *pnOut = 0;
113904
- *ppOut = 0;
113905
- }else if( isTermPos ){
113906
- *ppOut = sqlite3_malloc(nSource);
113907
- if( !*ppOut ) return SQLITE_NOMEM;
113908
- memcpy(*ppOut, aSource, nSource);
113909
- *pnOut = nSource;
113910
- }else{
113911
- sqlite3_int64 docid;
113912
- *pnOut = sqlite3Fts3GetVarint(aSource, &docid);
113913
- *ppOut = sqlite3_malloc(*pnOut);
113914
- if( !*ppOut ) return SQLITE_NOMEM;
113915
- sqlite3Fts3PutVarint(*ppOut, docid);
113916
- }
113917
-
113918
- return SQLITE_OK;
113919
-}
113920
-
113921
-SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
115131
+ }else{
115132
+ char *aNew;
115133
+ int nNew;
115134
+
115135
+ int rc = fts3DoclistOrMerge(p->bDescIdx, aMerge, nMerge,
115136
+ pTS->aaOutput[iOut], pTS->anOutput[iOut], &aNew, &nNew
115137
+ );
115138
+ if( rc!=SQLITE_OK ){
115139
+ if( aMerge!=aDoclist ) sqlite3_free(aMerge);
115140
+ return rc;
115141
+ }
115142
+
115143
+ if( aMerge!=aDoclist ) sqlite3_free(aMerge);
115144
+ sqlite3_free(pTS->aaOutput[iOut]);
115145
+ pTS->aaOutput[iOut] = 0;
115146
+
115147
+ aMerge = aNew;
115148
+ nMerge = nNew;
115149
+ if( (iOut+1)==SizeofArray(pTS->aaOutput) ){
115150
+ pTS->aaOutput[iOut] = aMerge;
115151
+ pTS->anOutput[iOut] = nMerge;
115152
+ }
115153
+ }
115154
+ }
115155
+ }
115156
+ return SQLITE_OK;
115157
+}
115158
+
115159
+/*
115160
+** Append SegReader object pNew to the end of the pCsr->apSegment[] array.
115161
+*/
115162
+static int fts3SegReaderCursorAppend(
115163
+ Fts3MultiSegReader *pCsr,
115164
+ Fts3SegReader *pNew
115165
+){
115166
+ if( (pCsr->nSegment%16)==0 ){
115167
+ Fts3SegReader **apNew;
115168
+ int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*);
115169
+ apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte);
115170
+ if( !apNew ){
115171
+ sqlite3Fts3SegReaderFree(pNew);
115172
+ return SQLITE_NOMEM;
115173
+ }
115174
+ pCsr->apSegment = apNew;
115175
+ }
115176
+ pCsr->apSegment[pCsr->nSegment++] = pNew;
115177
+ return SQLITE_OK;
115178
+}
115179
+
115180
+/*
115181
+** Add seg-reader objects to the Fts3MultiSegReader object passed as the
115182
+** 8th argument.
115183
+**
115184
+** This function returns SQLITE_OK if successful, or an SQLite error code
115185
+** otherwise.
115186
+*/
115187
+static int fts3SegReaderCursor(
113922115188
 Fts3Table *p, /* FTS3 table handle */
115189
+ int iIndex, /* Index to search (from 0 to p->nIndex-1) */
113923115190
 int iLevel, /* Level of segments to scan */
113924115191
 const char *zTerm, /* Term to query for */
113925115192
 int nTerm, /* Size of zTerm in bytes */
113926115193
 int isPrefix, /* True for a prefix search */
113927115194
 int isScan, /* True to scan from zTerm to EOF */
113928
- Fts3SegReaderCursor *pCsr /* Cursor object to populate */
115195
+ Fts3MultiSegReader *pCsr /* Cursor object to populate */
113929115196
 ){
113930
- int rc = SQLITE_OK;
113931
- int rc2;
113932
- int iAge = 0;
113933
- sqlite3_stmt *pStmt = 0;
113934
- Fts3SegReader *pPending = 0;
113935
-
113936
- assert( iLevel==FTS3_SEGCURSOR_ALL
113937
- || iLevel==FTS3_SEGCURSOR_PENDING
113938
- || iLevel>=0
113939
- );
113940
- assert( FTS3_SEGCURSOR_PENDING<0 );
113941
- assert( FTS3_SEGCURSOR_ALL<0 );
113942
- assert( iLevel==FTS3_SEGCURSOR_ALL || (zTerm==0 && isPrefix==1) );
113943
- assert( isPrefix==0 || isScan==0 );
113944
-
113945
-
113946
- memset(pCsr, 0, sizeof(Fts3SegReaderCursor));
113947
-
113948
- /* If iLevel is less than 0, include a seg-reader for the pending-terms. */
113949
- assert( isScan==0 || fts3HashCount(&p->pendingTerms)==0 );
113950
- if( iLevel<0 && isScan==0 ){
113951
- rc = sqlite3Fts3SegReaderPending(p, zTerm, nTerm, isPrefix, &pPending);
113952
- if( rc==SQLITE_OK && pPending ){
113953
- int nByte = (sizeof(Fts3SegReader *) * 16);
113954
- pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc(nByte);
113955
- if( pCsr->apSegment==0 ){
113956
- rc = SQLITE_NOMEM;
113957
- }else{
113958
- pCsr->apSegment[0] = pPending;
113959
- pCsr->nSegment = 1;
113960
- pPending = 0;
113961
- }
115197
+ int rc = SQLITE_OK; /* Error code */
115198
+ sqlite3_stmt *pStmt = 0; /* Statement to iterate through segments */
115199
+ int rc2; /* Result of sqlite3_reset() */
115200
+
115201
+ /* If iLevel is less than 0 and this is not a scan, include a seg-reader
115202
+ ** for the pending-terms. If this is a scan, then this call must be being
115203
+ ** made by an fts4aux module, not an FTS table. In this case calling
115204
+ ** Fts3SegReaderPending might segfault, as the data structures used by
115205
+ ** fts4aux are not completely populated. So it's easiest to filter these
115206
+ ** calls out here. */
115207
+ if( iLevel<0 && p->aIndex ){
115208
+ Fts3SegReader *pSeg = 0;
115209
+ rc = sqlite3Fts3SegReaderPending(p, iIndex, zTerm, nTerm, isPrefix, &pSeg);
115210
+ if( rc==SQLITE_OK && pSeg ){
115211
+ rc = fts3SegReaderCursorAppend(pCsr, pSeg);
113962115212
 }
113963115213
 }
113964115214
113965115215
 if( iLevel!=FTS3_SEGCURSOR_PENDING ){
113966115216
 if( rc==SQLITE_OK ){
113967
- rc = sqlite3Fts3AllSegdirs(p, iLevel, &pStmt);
115217
+ rc = sqlite3Fts3AllSegdirs(p, iIndex, iLevel, &pStmt);
113968115218
 }
115219
+
113969115220
 while( rc==SQLITE_OK && SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
115221
+ Fts3SegReader *pSeg = 0;
113970115222
113971115223
 /* Read the values returned by the SELECT into local variables. */
113972115224
 sqlite3_int64 iStartBlock = sqlite3_column_int64(pStmt, 1);
113973115225
 sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2);
113974115226
 sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3);
113975115227
 int nRoot = sqlite3_column_bytes(pStmt, 4);
113976115228
 char const *zRoot = sqlite3_column_blob(pStmt, 4);
113977115229
113978
- /* If nSegment is a multiple of 16 the array needs to be extended. */
113979
- if( (pCsr->nSegment%16)==0 ){
113980
- Fts3SegReader **apNew;
113981
- int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*);
113982
- apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte);
113983
- if( !apNew ){
113984
- rc = SQLITE_NOMEM;
113985
- goto finished;
113986
- }
113987
- pCsr->apSegment = apNew;
113988
- }
113989
-
113990115230
 /* If zTerm is not NULL, and this segment is not stored entirely on its
113991115231
 ** root node, the range of leaves scanned can be reduced. Do this. */
113992115232
 if( iStartBlock && zTerm ){
113993115233
 sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0);
113994115234
 rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi);
113995115235
 if( rc!=SQLITE_OK ) goto finished;
113996115236
 if( isPrefix==0 && isScan==0 ) iLeavesEndBlock = iStartBlock;
113997115237
 }
113998115238
113999
- rc = sqlite3Fts3SegReaderNew(iAge, iStartBlock, iLeavesEndBlock,
114000
- iEndBlock, zRoot, nRoot, &pCsr->apSegment[pCsr->nSegment]
115239
+ rc = sqlite3Fts3SegReaderNew(pCsr->nSegment+1,
115240
+ iStartBlock, iLeavesEndBlock, iEndBlock, zRoot, nRoot, &pSeg
114001115241
 );
114002115242
 if( rc!=SQLITE_OK ) goto finished;
114003
- pCsr->nSegment++;
114004
- iAge++;
115243
+ rc = fts3SegReaderCursorAppend(pCsr, pSeg);
114005115244
 }
114006115245
 }
114007115246
114008115247
 finished:
114009115248
 rc2 = sqlite3_reset(pStmt);
114010115249
 if( rc==SQLITE_DONE ) rc = rc2;
114011
- sqlite3Fts3SegReaderFree(pPending);
114012115250
114013115251
 return rc;
114014115252
 }
114015115253
115254
+/*
115255
+** Set up a cursor object for iterating through a full-text index or a
115256
+** single level therein.
115257
+*/
115258
+SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
115259
+ Fts3Table *p, /* FTS3 table handle */
115260
+ int iIndex, /* Index to search (from 0 to p->nIndex-1) */
115261
+ int iLevel, /* Level of segments to scan */
115262
+ const char *zTerm, /* Term to query for */
115263
+ int nTerm, /* Size of zTerm in bytes */
115264
+ int isPrefix, /* True for a prefix search */
115265
+ int isScan, /* True to scan from zTerm to EOF */
115266
+ Fts3MultiSegReader *pCsr /* Cursor object to populate */
115267
+){
115268
+ assert( iIndex>=0 && iIndex<p->nIndex );
115269
+ assert( iLevel==FTS3_SEGCURSOR_ALL
115270
+ || iLevel==FTS3_SEGCURSOR_PENDING
115271
+ || iLevel>=0
115272
+ );
115273
+ assert( iLevel<FTS3_SEGDIR_MAXLEVEL );
115274
+ assert( FTS3_SEGCURSOR_ALL<0 && FTS3_SEGCURSOR_PENDING<0 );
115275
+ assert( isPrefix==0 || isScan==0 );
114016115276
115277
+ /* "isScan" is only set to true by the ft4aux module, an ordinary
115278
+ ** full-text tables. */
115279
+ assert( isScan==0 || p->aIndex==0 );
115280
+
115281
+ memset(pCsr, 0, sizeof(Fts3MultiSegReader));
115282
+
115283
+ return fts3SegReaderCursor(
115284
+ p, iIndex, iLevel, zTerm, nTerm, isPrefix, isScan, pCsr
115285
+ );
115286
+}
115287
+
115288
+/*
115289
+** In addition to its current configuration, have the Fts3MultiSegReader
115290
+** passed as the 4th argument also scan the doclist for term zTerm/nTerm.
115291
+**
115292
+** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
115293
+*/
115294
+static int fts3SegReaderCursorAddZero(
115295
+ Fts3Table *p, /* FTS virtual table handle */
115296
+ const char *zTerm, /* Term to scan doclist of */
115297
+ int nTerm, /* Number of bytes in zTerm */
115298
+ Fts3MultiSegReader *pCsr /* Fts3MultiSegReader to modify */
115299
+){
115300
+ return fts3SegReaderCursor(p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0,pCsr);
115301
+}
115302
+
115303
+/*
115304
+** Open an Fts3MultiSegReader to scan the doclist for term zTerm/nTerm. Or,
115305
+** if isPrefix is true, to scan the doclist for all terms for which
115306
+** zTerm/nTerm is a prefix. If successful, return SQLITE_OK and write
115307
+** a pointer to the new Fts3MultiSegReader to *ppSegcsr. Otherwise, return
115308
+** an SQLite error code.
115309
+**
115310
+** It is the responsibility of the caller to free this object by eventually
115311
+** passing it to fts3SegReaderCursorFree()
115312
+**
115313
+** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
115314
+** Output parameter *ppSegcsr is set to 0 if an error occurs.
115315
+*/
114017115316
 static int fts3TermSegReaderCursor(
114018115317
 Fts3Cursor *pCsr, /* Virtual table cursor handle */
114019115318
 const char *zTerm, /* Term to query for */
114020115319
 int nTerm, /* Size of zTerm in bytes */
114021115320
 int isPrefix, /* True for a prefix search */
114022
- Fts3SegReaderCursor **ppSegcsr /* OUT: Allocated seg-reader cursor */
115321
+ Fts3MultiSegReader **ppSegcsr /* OUT: Allocated seg-reader cursor */
114023115322
 ){
114024
- Fts3SegReaderCursor *pSegcsr; /* Object to allocate and return */
115323
+ Fts3MultiSegReader *pSegcsr; /* Object to allocate and return */
114025115324
 int rc = SQLITE_NOMEM; /* Return code */
114026115325
114027
- pSegcsr = sqlite3_malloc(sizeof(Fts3SegReaderCursor));
115326
+ pSegcsr = sqlite3_malloc(sizeof(Fts3MultiSegReader));
114028115327
 if( pSegcsr ){
115328
+ int i;
115329
+ int bFound = 0; /* True once an index has been found */
114029115330
 Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
114030
- int i;
114031
- int nCost = 0;
114032
- rc = sqlite3Fts3SegReaderCursor(
114033
- p, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr);
114034
-
114035
- for(i=0; rc==SQLITE_OK && i<pSegcsr->nSegment; i++){
114036
- rc = sqlite3Fts3SegReaderCost(pCsr, pSegcsr->apSegment[i], &nCost);
114037
- }
114038
- pSegcsr->nCost = nCost;
115331
+
115332
+ if( isPrefix ){
115333
+ for(i=1; bFound==0 && i<p->nIndex; i++){
115334
+ if( p->aIndex[i].nPrefix==nTerm ){
115335
+ bFound = 1;
115336
+ rc = sqlite3Fts3SegReaderCursor(
115337
+ p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0, pSegcsr);
115338
+ pSegcsr->bLookup = 1;
115339
+ }
115340
+ }
115341
+
115342
+ for(i=1; bFound==0 && i<p->nIndex; i++){
115343
+ if( p->aIndex[i].nPrefix==nTerm+1 ){
115344
+ bFound = 1;
115345
+ rc = sqlite3Fts3SegReaderCursor(
115346
+ p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 1, 0, pSegcsr
115347
+ );
115348
+ if( rc==SQLITE_OK ){
115349
+ rc = fts3SegReaderCursorAddZero(p, zTerm, nTerm, pSegcsr);
115350
+ }
115351
+ }
115352
+ }
115353
+ }
115354
+
115355
+ if( bFound==0 ){
115356
+ rc = sqlite3Fts3SegReaderCursor(
115357
+ p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr
115358
+ );
115359
+ pSegcsr->bLookup = !isPrefix;
115360
+ }
114039115361
 }
114040115362
114041115363
 *ppSegcsr = pSegcsr;
114042115364
 return rc;
114043115365
 }
114044115366
114045
-static void fts3SegReaderCursorFree(Fts3SegReaderCursor *pSegcsr){
115367
+/*
115368
+** Free an Fts3MultiSegReader allocated by fts3TermSegReaderCursor().
115369
+*/
115370
+static void fts3SegReaderCursorFree(Fts3MultiSegReader *pSegcsr){
114046115371
 sqlite3Fts3SegReaderFinish(pSegcsr);
114047115372
 sqlite3_free(pSegcsr);
114048115373
 }
114049115374
114050115375
 /*
114051115376
 ** This function retreives the doclist for the specified term (or term
114052
-** prefix) from the database.
114053
-**
114054
-** The returned doclist may be in one of two formats, depending on the
114055
-** value of parameter isReqPos. If isReqPos is zero, then the doclist is
114056
-** a sorted list of delta-compressed docids (a bare doclist). If isReqPos
114057
-** is non-zero, then the returned list is in the same format as is stored
114058
-** in the database without the found length specifier at the start of on-disk
114059
-** doclists.
115377
+** prefix) from the database.
114060115378
 */
114061115379
 static int fts3TermSelect(
114062115380
 Fts3Table *p, /* Virtual table handle */
114063115381
 Fts3PhraseToken *pTok, /* Token to query for */
114064115382
 int iColumn, /* Column to query (or -ve for all columns) */
114065
- int isReqPos, /* True to include position lists in output */
114066115383
 int *pnOut, /* OUT: Size of buffer at *ppOut */
114067115384
 char **ppOut /* OUT: Malloced result buffer */
114068115385
 ){
114069115386
 int rc; /* Return code */
114070
- Fts3SegReaderCursor *pSegcsr; /* Seg-reader cursor for this term */
114071
- TermSelect tsc; /* Context object for fts3TermSelectCb() */
115387
+ Fts3MultiSegReader *pSegcsr; /* Seg-reader cursor for this term */
115388
+ TermSelect tsc; /* Object for pair-wise doclist merging */
114072115389
 Fts3SegFilter filter; /* Segment term filter configuration */
114073115390
114074115391
 pSegcsr = pTok->pSegcsr;
114075115392
 memset(&tsc, 0, sizeof(TermSelect));
114076
- tsc.isReqPos = isReqPos;
114077115393
114078
- filter.flags = FTS3_SEGMENT_IGNORE_EMPTY
115394
+ filter.flags = FTS3_SEGMENT_IGNORE_EMPTY | FTS3_SEGMENT_REQUIRE_POS
114079115395
 | (pTok->isPrefix ? FTS3_SEGMENT_PREFIX : 0)
114080
- | (isReqPos ? FTS3_SEGMENT_REQUIRE_POS : 0)
114081115396
 | (iColumn<p->nColumn ? FTS3_SEGMENT_COLUMN_FILTER : 0);
114082115397
 filter.iCol = iColumn;
114083115398
 filter.zTerm = pTok->z;
114084115399
 filter.nTerm = pTok->n;
114085115400
114086115401
 rc = sqlite3Fts3SegReaderStart(p, pSegcsr, &filter);
114087115402
 while( SQLITE_OK==rc
114088115403
 && SQLITE_ROW==(rc = sqlite3Fts3SegReaderStep(p, pSegcsr))
114089115404
 ){
114090
- rc = fts3TermSelectCb(p, (void *)&tsc,
114091
- pSegcsr->zTerm, pSegcsr->nTerm, pSegcsr->aDoclist, pSegcsr->nDoclist
114092
- );
115405
+ rc = fts3TermSelectMerge(p, &tsc, pSegcsr->aDoclist, pSegcsr->nDoclist);
114093115406
 }
114094115407
114095115408
 if( rc==SQLITE_OK ){
114096
- rc = fts3TermSelectMerge(&tsc);
115409
+ rc = fts3TermSelectFinishMerge(p, &tsc);
114097115410
 }
114098115411
 if( rc==SQLITE_OK ){
114099115412
 *ppOut = tsc.aaOutput[0];
114100115413
 *pnOut = tsc.anOutput[0];
114101115414
 }else{
@@ -114117,686 +115430,23 @@
114117115430
 ** If the isPoslist argument is true, then it is assumed that the doclist
114118115431
 ** contains a position-list following each docid. Otherwise, it is assumed
114119115432
 ** that the doclist is simply a list of docids stored as delta encoded
114120115433
 ** varints.
114121115434
 */
114122
-static int fts3DoclistCountDocids(int isPoslist, char *aList, int nList){
115435
+static int fts3DoclistCountDocids(char *aList, int nList){
114123115436
 int nDoc = 0; /* Return value */
114124115437
 if( aList ){
114125115438
 char *aEnd = &aList[nList]; /* Pointer to one byte after EOF */
114126115439
 char *p = aList; /* Cursor */
114127
- if( !isPoslist ){
114128
- /* The number of docids in the list is the same as the number of
114129
- ** varints. In FTS3 a varint consists of a single byte with the 0x80
114130
- ** bit cleared and zero or more bytes with the 0x80 bit set. So to
114131
- ** count the varints in the buffer, just count the number of bytes
114132
- ** with the 0x80 bit clear. */
114133
- while( p<aEnd ) nDoc += (((*p++)&0x80)==0);
114134
- }else{
114135
- while( p<aEnd ){
114136
- nDoc++;
114137
- while( (*p++)&0x80 ); /* Skip docid varint */
114138
- fts3PoslistCopy(0, &p); /* Skip over position list */
114139
- }
114140
- }
114141
- }
114142
-
114143
- return nDoc;
114144
-}
114145
-
114146
-/*
114147
-** Call sqlite3Fts3DeferToken() for each token in the expression pExpr.
114148
-*/
114149
-static int fts3DeferExpression(Fts3Cursor *pCsr, Fts3Expr *pExpr){
114150
- int rc = SQLITE_OK;
114151
- if( pExpr ){
114152
- rc = fts3DeferExpression(pCsr, pExpr->pLeft);
114153
- if( rc==SQLITE_OK ){
114154
- rc = fts3DeferExpression(pCsr, pExpr->pRight);
114155
- }
114156
- if( pExpr->eType==FTSQUERY_PHRASE ){
114157
- int iCol = pExpr->pPhrase->iColumn;
114158
- int i;
114159
- for(i=0; rc==SQLITE_OK && i<pExpr->pPhrase->nToken; i++){
114160
- Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i];
114161
- if( pToken->pDeferred==0 ){
114162
- rc = sqlite3Fts3DeferToken(pCsr, pToken, iCol);
114163
- }
114164
- }
114165
- }
114166
- }
114167
- return rc;
114168
-}
114169
-
114170
-/*
114171
-** This function removes the position information from a doclist. When
114172
-** called, buffer aList (size *pnList bytes) contains a doclist that includes
114173
-** position information. This function removes the position information so
114174
-** that aList contains only docids, and adjusts *pnList to reflect the new
114175
-** (possibly reduced) size of the doclist.
114176
-*/
114177
-static void fts3DoclistStripPositions(
114178
- char *aList, /* IN/OUT: Buffer containing doclist */
114179
- int *pnList /* IN/OUT: Size of doclist in bytes */
114180
-){
114181
- if( aList ){
114182
- char *aEnd = &aList[*pnList]; /* Pointer to one byte after EOF */
114183
- char *p = aList; /* Input cursor */
114184
- char *pOut = aList; /* Output cursor */
114185
-
114186
- while( p<aEnd ){
114187
- sqlite3_int64 delta;
114188
- p += sqlite3Fts3GetVarint(p, &delta);
114189
- fts3PoslistCopy(0, &p);
114190
- pOut += sqlite3Fts3PutVarint(pOut, delta);
114191
- }
114192
-
114193
- *pnList = (int)(pOut - aList);
114194
- }
114195
-}
114196
-
114197
-/*
114198
-** Return a DocList corresponding to the phrase *pPhrase.
114199
-**
114200
-** If this function returns SQLITE_OK, but *pnOut is set to a negative value,
114201
-** then no tokens in the phrase were looked up in the full-text index. This
114202
-** is only possible when this function is called from within xFilter(). The
114203
-** caller should assume that all documents match the phrase. The actual
114204
-** filtering will take place in xNext().
114205
-*/
114206
-static int fts3PhraseSelect(
114207
- Fts3Cursor *pCsr, /* Virtual table cursor handle */
114208
- Fts3Phrase *pPhrase, /* Phrase to return a doclist for */
114209
- int isReqPos, /* True if output should contain positions */
114210
- char **paOut, /* OUT: Pointer to malloc'd result buffer */
114211
- int *pnOut /* OUT: Size of buffer at *paOut */
114212
-){
114213
- char *pOut = 0;
114214
- int nOut = 0;
114215
- int rc = SQLITE_OK;
114216
- int ii;
114217
- int iCol = pPhrase->iColumn;
114218
- int isTermPos = (pPhrase->nToken>1 || isReqPos);
114219
- Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
114220
- int isFirst = 1;
114221
-
114222
- int iPrevTok = 0;
114223
- int nDoc = 0;
114224
-
114225
- /* If this is an xFilter() evaluation, create a segment-reader for each
114226
- ** phrase token. Or, if this is an xNext() or snippet/offsets/matchinfo
114227
- ** evaluation, only create segment-readers if there are no Fts3DeferredToken
114228
- ** objects attached to the phrase-tokens.
114229
- */
114230
- for(ii=0; ii<pPhrase->nToken; ii++){
114231
- Fts3PhraseToken *pTok = &pPhrase->aToken[ii];
114232
- if( pTok->pSegcsr==0 ){
114233
- if( (pCsr->eEvalmode==FTS3_EVAL_FILTER)
114234
- || (pCsr->eEvalmode==FTS3_EVAL_NEXT && pCsr->pDeferred==0)
114235
- || (pCsr->eEvalmode==FTS3_EVAL_MATCHINFO && pTok->bFulltext)
114236
- ){
114237
- rc = fts3TermSegReaderCursor(
114238
- pCsr, pTok->z, pTok->n, pTok->isPrefix, &pTok->pSegcsr
114239
- );
114240
- if( rc!=SQLITE_OK ) return rc;
114241
- }
114242
- }
114243
- }
114244
-
114245
- for(ii=0; ii<pPhrase->nToken; ii++){
114246
- Fts3PhraseToken *pTok; /* Token to find doclist for */
114247
- int iTok = 0; /* The token being queried this iteration */
114248
- char *pList = 0; /* Pointer to token doclist */
114249
- int nList = 0; /* Size of buffer at pList */
114250
-
114251
- /* Select a token to process. If this is an xFilter() call, then tokens
114252
- ** are processed in order from least to most costly. Otherwise, tokens
114253
- ** are processed in the order in which they occur in the phrase.
114254
- */
114255
- if( pCsr->eEvalmode==FTS3_EVAL_MATCHINFO ){
114256
- assert( isReqPos );
114257
- iTok = ii;
114258
- pTok = &pPhrase->aToken[iTok];
114259
- if( pTok->bFulltext==0 ) continue;
114260
- }else if( pCsr->eEvalmode==FTS3_EVAL_NEXT || isReqPos ){
114261
- iTok = ii;
114262
- pTok = &pPhrase->aToken[iTok];
114263
- }else{
114264
- int nMinCost = 0x7FFFFFFF;
114265
- int jj;
114266
-
114267
- /* Find the remaining token with the lowest cost. */
114268
- for(jj=0; jj<pPhrase->nToken; jj++){
114269
- Fts3SegReaderCursor *pSegcsr = pPhrase->aToken[jj].pSegcsr;
114270
- if( pSegcsr && pSegcsr->nCost<nMinCost ){
114271
- iTok = jj;
114272
- nMinCost = pSegcsr->nCost;
114273
- }
114274
- }
114275
- pTok = &pPhrase->aToken[iTok];
114276
-
114277
- /* This branch is taken if it is determined that loading the doclist
114278
- ** for the next token would require more IO than loading all documents
114279
- ** currently identified by doclist pOut/nOut. No further doclists will
114280
- ** be loaded from the full-text index for this phrase.
114281
- */
114282
- if( nMinCost>nDoc && ii>0 ){
114283
- rc = fts3DeferExpression(pCsr, pCsr->pExpr);
114284
- break;
114285
- }
114286
- }
114287
-
114288
- if( pCsr->eEvalmode==FTS3_EVAL_NEXT && pTok->pDeferred ){
114289
- rc = fts3DeferredTermSelect(pTok->pDeferred, isTermPos, &nList, &pList);
114290
- }else{
114291
- if( pTok->pSegcsr ){
114292
- rc = fts3TermSelect(p, pTok, iCol, isTermPos, &nList, &pList);
114293
- }
114294
- pTok->bFulltext = 1;
114295
- }
114296
- assert( rc!=SQLITE_OK || pCsr->eEvalmode || pTok->pSegcsr==0 );
114297
- if( rc!=SQLITE_OK ) break;
114298
-
114299
- if( isFirst ){
114300
- pOut = pList;
114301
- nOut = nList;
114302
- if( pCsr->eEvalmode==FTS3_EVAL_FILTER && pPhrase->nToken>1 ){
114303
- nDoc = fts3DoclistCountDocids(1, pOut, nOut);
114304
- }
114305
- isFirst = 0;
114306
- iPrevTok = iTok;
114307
- }else{
114308
- /* Merge the new term list and the current output. */
114309
- char *aLeft, *aRight;
114310
- int nLeft, nRight;
114311
- int nDist;
114312
- int mt;
114313
-
114314
- /* If this is the final token of the phrase, and positions were not
114315
- ** requested by the caller, use MERGE_PHRASE instead of POS_PHRASE.
114316
- ** This drops the position information from the output list.
114317
- */
114318
- mt = MERGE_POS_PHRASE;
114319
- if( ii==pPhrase->nToken-1 && !isReqPos ) mt = MERGE_PHRASE;
114320
-
114321
- assert( iPrevTok!=iTok );
114322
- if( iPrevTok<iTok ){
114323
- aLeft = pOut;
114324
- nLeft = nOut;
114325
- aRight = pList;
114326
- nRight = nList;
114327
- nDist = iTok-iPrevTok;
114328
- iPrevTok = iTok;
114329
- }else{
114330
- aRight = pOut;
114331
- nRight = nOut;
114332
- aLeft = pList;
114333
- nLeft = nList;
114334
- nDist = iPrevTok-iTok;
114335
- }
114336
- pOut = aRight;
114337
- fts3DoclistMerge(
114338
- mt, nDist, 0, pOut, &nOut, aLeft, nLeft, aRight, nRight, &nDoc
114339
- );
114340
- sqlite3_free(aLeft);
114341
- }
114342
- assert( nOut==0 || pOut!=0 );
114343
- }
114344
-
114345
- if( rc==SQLITE_OK ){
114346
- if( ii!=pPhrase->nToken ){
114347
- assert( pCsr->eEvalmode==FTS3_EVAL_FILTER && isReqPos==0 );
114348
- fts3DoclistStripPositions(pOut, &nOut);
114349
- }
114350
- *paOut = pOut;
114351
- *pnOut = nOut;
114352
- }else{
114353
- sqlite3_free(pOut);
114354
- }
114355
- return rc;
114356
-}
114357
-
114358
-/*
114359
-** This function merges two doclists according to the requirements of a
114360
-** NEAR operator.
114361
-**
114362
-** Both input doclists must include position information. The output doclist
114363
-** includes position information if the first argument to this function
114364
-** is MERGE_POS_NEAR, or does not if it is MERGE_NEAR.
114365
-*/
114366
-static int fts3NearMerge(
114367
- int mergetype, /* MERGE_POS_NEAR or MERGE_NEAR */
114368
- int nNear, /* Parameter to NEAR operator */
114369
- int nTokenLeft, /* Number of tokens in LHS phrase arg */
114370
- char *aLeft, /* Doclist for LHS (incl. positions) */
114371
- int nLeft, /* Size of LHS doclist in bytes */
114372
- int nTokenRight, /* As nTokenLeft */
114373
- char *aRight, /* As aLeft */
114374
- int nRight, /* As nRight */
114375
- char **paOut, /* OUT: Results of merge (malloced) */
114376
- int *pnOut /* OUT: Sized of output buffer */
114377
-){
114378
- char *aOut; /* Buffer to write output doclist to */
114379
- int rc; /* Return code */
114380
-
114381
- assert( mergetype==MERGE_POS_NEAR || MERGE_NEAR );
114382
-
114383
- aOut = sqlite3_malloc(nLeft+nRight+1);
114384
- if( aOut==0 ){
114385
- rc = SQLITE_NOMEM;
114386
- }else{
114387
- rc = fts3DoclistMerge(mergetype, nNear+nTokenRight, nNear+nTokenLeft,
114388
- aOut, pnOut, aLeft, nLeft, aRight, nRight, 0
114389
- );
114390
- if( rc!=SQLITE_OK ){
114391
- sqlite3_free(aOut);
114392
- aOut = 0;
114393
- }
114394
- }
114395
-
114396
- *paOut = aOut;
114397
- return rc;
114398
-}
114399
-
114400
-/*
114401
-** This function is used as part of the processing for the snippet() and
114402
-** offsets() functions.
114403
-**
114404
-** Both pLeft and pRight are expression nodes of type FTSQUERY_PHRASE. Both
114405
-** have their respective doclists (including position information) loaded
114406
-** in Fts3Expr.aDoclist/nDoclist. This function removes all entries from
114407
-** each doclist that are not within nNear tokens of a corresponding entry
114408
-** in the other doclist.
114409
-*/
114410
-SQLITE_PRIVATE int sqlite3Fts3ExprNearTrim(Fts3Expr *pLeft, Fts3Expr *pRight, int nNear){
114411
- int rc; /* Return code */
114412
-
114413
- assert( pLeft->eType==FTSQUERY_PHRASE );
114414
- assert( pRight->eType==FTSQUERY_PHRASE );
114415
- assert( pLeft->isLoaded && pRight->isLoaded );
114416
-
114417
- if( pLeft->aDoclist==0 || pRight->aDoclist==0 ){
114418
- sqlite3_free(pLeft->aDoclist);
114419
- sqlite3_free(pRight->aDoclist);
114420
- pRight->aDoclist = 0;
114421
- pLeft->aDoclist = 0;
114422
- rc = SQLITE_OK;
114423
- }else{
114424
- char *aOut; /* Buffer in which to assemble new doclist */
114425
- int nOut; /* Size of buffer aOut in bytes */
114426
-
114427
- rc = fts3NearMerge(MERGE_POS_NEAR, nNear,
114428
- pLeft->pPhrase->nToken, pLeft->aDoclist, pLeft->nDoclist,
114429
- pRight->pPhrase->nToken, pRight->aDoclist, pRight->nDoclist,
114430
- &aOut, &nOut
114431
- );
114432
- if( rc!=SQLITE_OK ) return rc;
114433
- sqlite3_free(pRight->aDoclist);
114434
- pRight->aDoclist = aOut;
114435
- pRight->nDoclist = nOut;
114436
-
114437
- rc = fts3NearMerge(MERGE_POS_NEAR, nNear,
114438
- pRight->pPhrase->nToken, pRight->aDoclist, pRight->nDoclist,
114439
- pLeft->pPhrase->nToken, pLeft->aDoclist, pLeft->nDoclist,
114440
- &aOut, &nOut
114441
- );
114442
- sqlite3_free(pLeft->aDoclist);
114443
- pLeft->aDoclist = aOut;
114444
- pLeft->nDoclist = nOut;
114445
- }
114446
- return rc;
114447
-}
114448
-
114449
-
114450
-/*
114451
-** Allocate an Fts3SegReaderArray for each token in the expression pExpr.
114452
-** The allocated objects are stored in the Fts3PhraseToken.pArray member
114453
-** variables of each token structure.
114454
-*/
114455
-static int fts3ExprAllocateSegReaders(
114456
- Fts3Cursor *pCsr, /* FTS3 table */
114457
- Fts3Expr *pExpr, /* Expression to create seg-readers for */
114458
- int *pnExpr /* OUT: Number of AND'd expressions */
114459
-){
114460
- int rc = SQLITE_OK; /* Return code */
114461
-
114462
- assert( pCsr->eEvalmode==FTS3_EVAL_FILTER );
114463
- if( pnExpr && pExpr->eType!=FTSQUERY_AND ){
114464
- (*pnExpr)++;
114465
- pnExpr = 0;
114466
- }
114467
-
114468
- if( pExpr->eType==FTSQUERY_PHRASE ){
114469
- Fts3Phrase *pPhrase = pExpr->pPhrase;
114470
- int ii;
114471
-
114472
- for(ii=0; rc==SQLITE_OK && ii<pPhrase->nToken; ii++){
114473
- Fts3PhraseToken *pTok = &pPhrase->aToken[ii];
114474
- if( pTok->pSegcsr==0 ){
114475
- rc = fts3TermSegReaderCursor(
114476
- pCsr, pTok->z, pTok->n, pTok->isPrefix, &pTok->pSegcsr
114477
- );
114478
- }
114479
- }
114480
- }else{
114481
- rc = fts3ExprAllocateSegReaders(pCsr, pExpr->pLeft, pnExpr);
114482
- if( rc==SQLITE_OK ){
114483
- rc = fts3ExprAllocateSegReaders(pCsr, pExpr->pRight, pnExpr);
114484
- }
114485
- }
114486
- return rc;
114487
-}
114488
-
114489
-/*
114490
-** Free the Fts3SegReaderArray objects associated with each token in the
114491
-** expression pExpr. In other words, this function frees the resources
114492
-** allocated by fts3ExprAllocateSegReaders().
114493
-*/
114494
-static void fts3ExprFreeSegReaders(Fts3Expr *pExpr){
114495
- if( pExpr ){
114496
- Fts3Phrase *pPhrase = pExpr->pPhrase;
114497
- if( pPhrase ){
114498
- int kk;
114499
- for(kk=0; kk<pPhrase->nToken; kk++){
114500
- fts3SegReaderCursorFree(pPhrase->aToken[kk].pSegcsr);
114501
- pPhrase->aToken[kk].pSegcsr = 0;
114502
- }
114503
- }
114504
- fts3ExprFreeSegReaders(pExpr->pLeft);
114505
- fts3ExprFreeSegReaders(pExpr->pRight);
114506
- }
114507
-}
114508
-
114509
-/*
114510
-** Return the sum of the costs of all tokens in the expression pExpr. This
114511
-** function must be called after Fts3SegReaderArrays have been allocated
114512
-** for all tokens using fts3ExprAllocateSegReaders().
114513
-*/
114514
-static int fts3ExprCost(Fts3Expr *pExpr){
114515
- int nCost; /* Return value */
114516
- if( pExpr->eType==FTSQUERY_PHRASE ){
114517
- Fts3Phrase *pPhrase = pExpr->pPhrase;
114518
- int ii;
114519
- nCost = 0;
114520
- for(ii=0; ii<pPhrase->nToken; ii++){
114521
- Fts3SegReaderCursor *pSegcsr = pPhrase->aToken[ii].pSegcsr;
114522
- if( pSegcsr ) nCost += pSegcsr->nCost;
114523
- }
114524
- }else{
114525
- nCost = fts3ExprCost(pExpr->pLeft) + fts3ExprCost(pExpr->pRight);
114526
- }
114527
- return nCost;
114528
-}
114529
-
114530
-/*
114531
-** The following is a helper function (and type) for fts3EvalExpr(). It
114532
-** must be called after Fts3SegReaders have been allocated for every token
114533
-** in the expression. See the context it is called from in fts3EvalExpr()
114534
-** for further explanation.
114535
-*/
114536
-typedef struct ExprAndCost ExprAndCost;
114537
-struct ExprAndCost {
114538
- Fts3Expr *pExpr;
114539
- int nCost;
114540
-};
114541
-static void fts3ExprAssignCosts(
114542
- Fts3Expr *pExpr, /* Expression to create seg-readers for */
114543
- ExprAndCost **ppExprCost /* OUT: Write to *ppExprCost */
114544
-){
114545
- if( pExpr->eType==FTSQUERY_AND ){
114546
- fts3ExprAssignCosts(pExpr->pLeft, ppExprCost);
114547
- fts3ExprAssignCosts(pExpr->pRight, ppExprCost);
114548
- }else{
114549
- (*ppExprCost)->pExpr = pExpr;
114550
- (*ppExprCost)->nCost = fts3ExprCost(pExpr);
114551
- (*ppExprCost)++;
114552
- }
114553
-}
114554
-
114555
-/*
114556
-** Evaluate the full-text expression pExpr against FTS3 table pTab. Store
114557
-** the resulting doclist in *paOut and *pnOut. This routine mallocs for
114558
-** the space needed to store the output. The caller is responsible for
114559
-** freeing the space when it has finished.
114560
-**
114561
-** This function is called in two distinct contexts:
114562
-**
114563
-** * From within the virtual table xFilter() method. In this case, the
114564
-** output doclist contains entries for all rows in the table, based on
114565
-** data read from the full-text index.
114566
-**
114567
-** In this case, if the query expression contains one or more tokens that
114568
-** are very common, then the returned doclist may contain a superset of
114569
-** the documents that actually match the expression.
114570
-**
114571
-** * From within the virtual table xNext() method. This call is only made
114572
-** if the call from within xFilter() found that there were very common
114573
-** tokens in the query expression and did return a superset of the
114574
-** matching documents. In this case the returned doclist contains only
114575
-** entries that correspond to the current row of the table. Instead of
114576
-** reading the data for each token from the full-text index, the data is
114577
-** already available in-memory in the Fts3PhraseToken.pDeferred structures.
114578
-** See fts3EvalDeferred() for how it gets there.
114579
-**
114580
-** In the first case above, Fts3Cursor.doDeferred==0. In the second (if it is
114581
-** required) Fts3Cursor.doDeferred==1.
114582
-**
114583
-** If the SQLite invokes the snippet(), offsets() or matchinfo() function
114584
-** as part of a SELECT on an FTS3 table, this function is called on each
114585
-** individual phrase expression in the query. If there were very common tokens
114586
-** found in the xFilter() call, then this function is called once for phrase
114587
-** for each row visited, and the returned doclist contains entries for the
114588
-** current row only. Otherwise, if there were no very common tokens, then this
114589
-** function is called once only for each phrase in the query and the returned
114590
-** doclist contains entries for all rows of the table.
114591
-**
114592
-** Fts3Cursor.doDeferred==1 when this function is called on phrases as a
114593
-** result of a snippet(), offsets() or matchinfo() invocation.
114594
-*/
114595
-static int fts3EvalExpr(
114596
- Fts3Cursor *p, /* Virtual table cursor handle */
114597
- Fts3Expr *pExpr, /* Parsed fts3 expression */
114598
- char **paOut, /* OUT: Pointer to malloc'd result buffer */
114599
- int *pnOut, /* OUT: Size of buffer at *paOut */
114600
- int isReqPos /* Require positions in output buffer */
114601
-){
114602
- int rc = SQLITE_OK; /* Return code */
114603
-
114604
- /* Zero the output parameters. */
114605
- *paOut = 0;
114606
- *pnOut = 0;
114607
-
114608
- if( pExpr ){
114609
- assert( pExpr->eType==FTSQUERY_NEAR || pExpr->eType==FTSQUERY_OR
114610
- || pExpr->eType==FTSQUERY_AND || pExpr->eType==FTSQUERY_NOT
114611
- || pExpr->eType==FTSQUERY_PHRASE
114612
- );
114613
- assert( pExpr->eType==FTSQUERY_PHRASE || isReqPos==0 );
114614
-
114615
- if( pExpr->eType==FTSQUERY_PHRASE ){
114616
- rc = fts3PhraseSelect(p, pExpr->pPhrase,
114617
- isReqPos || (pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR),
114618
- paOut, pnOut
114619
- );
114620
- fts3ExprFreeSegReaders(pExpr);
114621
- }else if( p->eEvalmode==FTS3_EVAL_FILTER && pExpr->eType==FTSQUERY_AND ){
114622
- ExprAndCost *aExpr = 0; /* Array of AND'd expressions and costs */
114623
- int nExpr = 0; /* Size of aExpr[] */
114624
- char *aRet = 0; /* Doclist to return to caller */
114625
- int nRet = 0; /* Length of aRet[] in bytes */
114626
- int nDoc = 0x7FFFFFFF;
114627
-
114628
- assert( !isReqPos );
114629
-
114630
- rc = fts3ExprAllocateSegReaders(p, pExpr, &nExpr);
114631
- if( rc==SQLITE_OK ){
114632
- assert( nExpr>1 );
114633
- aExpr = sqlite3_malloc(sizeof(ExprAndCost) * nExpr);
114634
- if( !aExpr ) rc = SQLITE_NOMEM;
114635
- }
114636
- if( rc==SQLITE_OK ){
114637
- int ii; /* Used to iterate through expressions */
114638
-
114639
- fts3ExprAssignCosts(pExpr, &aExpr);
114640
- aExpr -= nExpr;
114641
- for(ii=0; ii<nExpr; ii++){
114642
- char *aNew;
114643
- int nNew;
114644
- int jj;
114645
- ExprAndCost *pBest = 0;
114646
-
114647
- for(jj=0; jj<nExpr; jj++){
114648
- ExprAndCost *pCand = &aExpr[jj];
114649
- if( pCand->pExpr && (pBest==0 || pCand->nCost<pBest->nCost) ){
114650
- pBest = pCand;
114651
- }
114652
- }
114653
-
114654
- if( pBest->nCost>nDoc ){
114655
- rc = fts3DeferExpression(p, p->pExpr);
114656
- break;
114657
- }else{
114658
- rc = fts3EvalExpr(p, pBest->pExpr, &aNew, &nNew, 0);
114659
- if( rc!=SQLITE_OK ) break;
114660
- pBest->pExpr = 0;
114661
- if( ii==0 ){
114662
- aRet = aNew;
114663
- nRet = nNew;
114664
- nDoc = fts3DoclistCountDocids(0, aRet, nRet);
114665
- }else{
114666
- fts3DoclistMerge(
114667
- MERGE_AND, 0, 0, aRet, &nRet, aRet, nRet, aNew, nNew, &nDoc
114668
- );
114669
- sqlite3_free(aNew);
114670
- }
114671
- }
114672
- }
114673
- }
114674
-
114675
- if( rc==SQLITE_OK ){
114676
- *paOut = aRet;
114677
- *pnOut = nRet;
114678
- }else{
114679
- assert( *paOut==0 );
114680
- sqlite3_free(aRet);
114681
- }
114682
- sqlite3_free(aExpr);
114683
- fts3ExprFreeSegReaders(pExpr);
114684
-
114685
- }else{
114686
- char *aLeft;
114687
- char *aRight;
114688
- int nLeft;
114689
- int nRight;
114690
-
114691
- assert( pExpr->eType==FTSQUERY_NEAR
114692
- || pExpr->eType==FTSQUERY_OR
114693
- || pExpr->eType==FTSQUERY_NOT
114694
- || (pExpr->eType==FTSQUERY_AND && p->eEvalmode==FTS3_EVAL_NEXT)
114695
- );
114696
-
114697
- if( 0==(rc = fts3EvalExpr(p, pExpr->pRight, &aRight, &nRight, isReqPos))
114698
- && 0==(rc = fts3EvalExpr(p, pExpr->pLeft, &aLeft, &nLeft, isReqPos))
114699
- ){
114700
- switch( pExpr->eType ){
114701
- case FTSQUERY_NEAR: {
114702
- Fts3Expr *pLeft;
114703
- Fts3Expr *pRight;
114704
- int mergetype = MERGE_NEAR;
114705
- if( pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR ){
114706
- mergetype = MERGE_POS_NEAR;
114707
- }
114708
- pLeft = pExpr->pLeft;
114709
- while( pLeft->eType==FTSQUERY_NEAR ){
114710
- pLeft=pLeft->pRight;
114711
- }
114712
- pRight = pExpr->pRight;
114713
- assert( pRight->eType==FTSQUERY_PHRASE );
114714
- assert( pLeft->eType==FTSQUERY_PHRASE );
114715
-
114716
- rc = fts3NearMerge(mergetype, pExpr->nNear,
114717
- pLeft->pPhrase->nToken, aLeft, nLeft,
114718
- pRight->pPhrase->nToken, aRight, nRight,
114719
- paOut, pnOut
114720
- );
114721
- sqlite3_free(aLeft);
114722
- break;
114723
- }
114724
-
114725
- case FTSQUERY_OR: {
114726
- /* Allocate a buffer for the output. The maximum size is the
114727
- ** sum of the sizes of the two input buffers. The +1 term is
114728
- ** so that a buffer of zero bytes is never allocated - this can
114729
- ** cause fts3DoclistMerge() to incorrectly return SQLITE_NOMEM.
114730
- */
114731
- char *aBuffer = sqlite3_malloc(nRight+nLeft+1);
114732
- rc = fts3DoclistMerge(MERGE_OR, 0, 0, aBuffer, pnOut,
114733
- aLeft, nLeft, aRight, nRight, 0
114734
- );
114735
- *paOut = aBuffer;
114736
- sqlite3_free(aLeft);
114737
- break;
114738
- }
114739
-
114740
- default: {
114741
- assert( FTSQUERY_NOT==MERGE_NOT && FTSQUERY_AND==MERGE_AND );
114742
- fts3DoclistMerge(pExpr->eType, 0, 0, aLeft, pnOut,
114743
- aLeft, nLeft, aRight, nRight, 0
114744
- );
114745
- *paOut = aLeft;
114746
- break;
114747
- }
114748
- }
114749
- }
114750
- sqlite3_free(aRight);
114751
- }
114752
- }
114753
-
114754
- assert( rc==SQLITE_OK || *paOut==0 );
114755
- return rc;
114756
-}
114757
-
114758
-/*
114759
-** This function is called from within xNext() for each row visited by
114760
-** an FTS3 query. If evaluating the FTS3 query expression within xFilter()
114761
-** was able to determine the exact set of matching rows, this function sets
114762
-** *pbRes to true and returns SQLITE_IO immediately.
114763
-**
114764
-** Otherwise, if evaluating the query expression within xFilter() returned a
114765
-** superset of the matching documents instead of an exact set (this happens
114766
-** when the query includes very common tokens and it is deemed too expensive to
114767
-** load their doclists from disk), this function tests if the current row
114768
-** really does match the FTS3 query.
114769
-**
114770
-** If an error occurs, an SQLite error code is returned. Otherwise, SQLITE_OK
114771
-** is returned and *pbRes is set to true if the current row matches the
114772
-** FTS3 query (and should be included in the results returned to SQLite), or
114773
-** false otherwise.
114774
-*/
114775
-static int fts3EvalDeferred(
114776
- Fts3Cursor *pCsr, /* FTS3 cursor pointing at row to test */
114777
- int *pbRes /* OUT: Set to true if row is a match */
114778
-){
114779
- int rc = SQLITE_OK;
114780
- if( pCsr->pDeferred==0 ){
114781
- *pbRes = 1;
114782
- }else{
114783
- rc = fts3CursorSeek(0, pCsr);
114784
- if( rc==SQLITE_OK ){
114785
- sqlite3Fts3FreeDeferredDoclists(pCsr);
114786
- rc = sqlite3Fts3CacheDeferredDoclists(pCsr);
114787
- }
114788
- if( rc==SQLITE_OK ){
114789
- char *a = 0;
114790
- int n = 0;
114791
- rc = fts3EvalExpr(pCsr, pCsr->pExpr, &a, &n, 0);
114792
- assert( n>=0 );
114793
- *pbRes = (n>0);
114794
- sqlite3_free(a);
114795
- }
114796
- }
114797
- return rc;
115440
+ while( p<aEnd ){
115441
+ nDoc++;
115442
+ while( (*p++)&0x80 ); /* Skip docid varint */
115443
+ fts3PoslistCopy(0, &p); /* Skip over position list */
115444
+ }
115445
+ }
115446
+
115447
+ return nDoc;
114798115448
 }
114799115449
114800115450
 /*
114801115451
 ** Advance the cursor to the next row in the %_content table that
114802115452
 ** matches the search criteria. For a MATCH search, this will be
@@ -114807,43 +115457,24 @@
114807115457
 ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned
114808115458
 ** even if we reach end-of-file. The fts3EofMethod() will be called
114809115459
 ** subsequently to determine whether or not an EOF was hit.
114810115460
 */
114811115461
 static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){
114812
- int res;
114813
- int rc = SQLITE_OK; /* Return code */
115462
+ int rc;
114814115463
 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
114815
-
114816
- pCsr->eEvalmode = FTS3_EVAL_NEXT;
114817
- do {
114818
- if( pCsr->aDoclist==0 ){
114819
- if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){
114820
- pCsr->isEof = 1;
114821
- rc = sqlite3_reset(pCsr->pStmt);
114822
- break;
114823
- }
115464
+ if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){
115465
+ if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){
115466
+ pCsr->isEof = 1;
115467
+ rc = sqlite3_reset(pCsr->pStmt);
115468
+ }else{
114824115469
 pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0);
114825
- }else{
114826
- if( pCsr->desc==0 ){
114827
- if( pCsr->pNextId>=&pCsr->aDoclist[pCsr->nDoclist] ){
114828
- pCsr->isEof = 1;
114829
- break;
114830
- }
114831
- fts3GetDeltaVarint(&pCsr->pNextId, &pCsr->iPrevId);
114832
- }else{
114833
- fts3GetReverseDeltaVarint(&pCsr->pNextId,pCsr->aDoclist,&pCsr->iPrevId);
114834
- if( pCsr->pNextId<=pCsr->aDoclist ){
114835
- pCsr->isEof = 1;
114836
- break;
114837
- }
114838
- }
114839
- sqlite3_reset(pCsr->pStmt);
114840
- pCsr->isRequireSeek = 1;
114841
- pCsr->isMatchinfoNeeded = 1;
114842
- }
114843
- }while( SQLITE_OK==(rc = fts3EvalDeferred(pCsr, &res)) && res==0 );
114844
-
115470
+ rc = SQLITE_OK;
115471
+ }
115472
+ }else{
115473
+ rc = fts3EvalNext((Fts3Cursor *)pCursor);
115474
+ }
115475
+ assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
114845115476
 return rc;
114846115477
 }
114847115478
114848115479
 /*
114849115480
 ** This is the xFilter interface for the virtual table. See
@@ -114866,15 +115497,11 @@
114866115497
 int idxNum, /* Strategy index */
114867115498
 const char *idxStr, /* Unused */
114868115499
 int nVal, /* Number of elements in apVal */
114869115500
 sqlite3_value **apVal /* Arguments for the indexing scheme */
114870115501
 ){
114871
- const char *azSql[] = {
114872
- "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?", /* non-full-scan */
114873
- "SELECT %s FROM %Q.'%q_content' AS x ORDER BY docid %s", /* full-scan */
114874
- };
114875
- int rc; /* Return code */
115502
+ int rc;
114876115503
 char *zSql; /* SQL statement used to access %_content */
114877115504
 Fts3Table *p = (Fts3Table *)pCursor->pVtab;
114878115505
 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
114879115506
114880115507
 UNUSED_PARAMETER(idxStr);
@@ -114889,10 +115516,17 @@
114889115516
 sqlite3_finalize(pCsr->pStmt);
114890115517
 sqlite3_free(pCsr->aDoclist);
114891115518
 sqlite3Fts3ExprFree(pCsr->pExpr);
114892115519
 memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor));
114893115520
115521
+ if( idxStr ){
115522
+ pCsr->bDesc = (idxStr[0]=='D');
115523
+ }else{
115524
+ pCsr->bDesc = p->bDescIdx;
115525
+ }
115526
+ pCsr->eSearch = (i16)idxNum;
115527
+
114894115528
 if( idxNum!=FTS3_DOCID_SEARCH && idxNum!=FTS3_FULLSCAN_SEARCH ){
114895115529
 int iCol = idxNum-FTS3_FULLTEXT_SEARCH;
114896115530
 const char *zQuery = (const char *)sqlite3_value_text(apVal[0]);
114897115531
114898115532
 if( zQuery==0 && sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
@@ -114902,20 +115536,21 @@
114902115536
 rc = sqlite3Fts3ExprParse(p->pTokenizer, p->azColumn, p->nColumn,
114903115537
 iCol, zQuery, -1, &pCsr->pExpr
114904115538
 );
114905115539
 if( rc!=SQLITE_OK ){
114906115540
 if( rc==SQLITE_ERROR ){
114907
- p->base.zErrMsg = sqlite3_mprintf("malformed MATCH expression: [%s]",
114908
- zQuery);
115541
+ static const char *zErr = "malformed MATCH expression: [%s]";
115542
+ p->base.zErrMsg = sqlite3_mprintf(zErr, zQuery);
114909115543
 }
114910115544
 return rc;
114911115545
 }
114912115546
114913115547
 rc = sqlite3Fts3ReadLock(p);
114914115548
 if( rc!=SQLITE_OK ) return rc;
114915115549
114916
- rc = fts3EvalExpr(pCsr, pCsr->pExpr, &pCsr->aDoclist, &pCsr->nDoclist, 0);
115550
+ rc = fts3EvalStart(pCsr);
115551
+
114917115552
 sqlite3Fts3SegmentsClose(p);
114918115553
 if( rc!=SQLITE_OK ) return rc;
114919115554
 pCsr->pNextId = pCsr->aDoclist;
114920115555
 pCsr->iPrevId = 0;
114921115556
 }
@@ -114923,41 +115558,28 @@
114923115558
 /* Compile a SELECT statement for this cursor. For a full-table-scan, the
114924115559
 ** statement loops through all rows of the %_content table. For a
114925115560
 ** full-text query or docid lookup, the statement retrieves a single
114926115561
 ** row by docid.
114927115562
 */
114928
- zSql = (char *)azSql[idxNum==FTS3_FULLSCAN_SEARCH];
114929
- zSql = sqlite3_mprintf(
114930
- zSql, p->zReadExprlist, p->zDb, p->zName, (idxStr ? idxStr : "ASC")
114931
- );
114932
- if( !zSql ){
114933
- rc = SQLITE_NOMEM;
115563
+ if( idxNum==FTS3_FULLSCAN_SEARCH ){
115564
+ const char *zSort = (pCsr->bDesc ? "DESC" : "ASC");
115565
+ const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x ORDER BY docid %s";
115566
+ zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName, zSort);
114934115567
 }else{
114935
- rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0);
114936
- sqlite3_free(zSql);
115568
+ const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?";
115569
+ zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName);
114937115570
 }
114938
- if( rc==SQLITE_OK && idxNum==FTS3_DOCID_SEARCH ){
115571
+ if( !zSql ) return SQLITE_NOMEM;
115572
+ rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0);
115573
+ sqlite3_free(zSql);
115574
+ if( rc!=SQLITE_OK ) return rc;
115575
+
115576
+ if( idxNum==FTS3_DOCID_SEARCH ){
114939115577
 rc = sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]);
114940
- }
114941
- pCsr->eSearch = (i16)idxNum;
114942
-
114943
- assert( pCsr->desc==0 );
114944
- if( rc!=SQLITE_OK ) return rc;
114945
- if( rc==SQLITE_OK && pCsr->nDoclist>0 && idxStr && idxStr[0]=='D' ){
114946
- sqlite3_int64 iDocid = 0;
114947
- char *csr = pCsr->aDoclist;
114948
- while( csr<&pCsr->aDoclist[pCsr->nDoclist] ){
114949
- fts3GetDeltaVarint(&csr, &iDocid);
114950
- }
114951
- pCsr->pNextId = csr;
114952
- pCsr->iPrevId = iDocid;
114953
- pCsr->desc = 1;
114954
- pCsr->isRequireSeek = 1;
114955
- pCsr->isMatchinfoNeeded = 1;
114956
- pCsr->eEvalmode = FTS3_EVAL_NEXT;
114957
- return SQLITE_OK;
114958
- }
115578
+ if( rc!=SQLITE_OK ) return rc;
115579
+ }
115580
+
114959115581
 return fts3NextMethod(pCursor);
114960115582
 }
114961115583
114962115584
 /*
114963115585
 ** This is the xEof method of the virtual table. SQLite calls this
@@ -114973,20 +115595,11 @@
114973115595
 ** exposes %_content.docid as the rowid for the virtual table. The
114974115596
 ** rowid should be written to *pRowid.
114975115597
 */
114976115598
 static int fts3RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
114977115599
 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor;
114978
- if( pCsr->aDoclist ){
114979
- *pRowid = pCsr->iPrevId;
114980
- }else{
114981
- /* This branch runs if the query is implemented using a full-table scan
114982
- ** (not using the full-text index). In this case grab the rowid from the
114983
- ** SELECT statement.
114984
- */
114985
- assert( pCsr->isRequireSeek==0 );
114986
- *pRowid = sqlite3_column_int64(pCsr->pStmt, 0);
114987
- }
115600
+ *pRowid = pCsr->iPrevId;
114988115601
 return SQLITE_OK;
114989115602
 }
114990115603
114991115604
 /*
114992115605
 ** This is the xColumn method, called by SQLite to request a value from
@@ -114995,11 +115608,11 @@
114995115608
 static int fts3ColumnMethod(
114996115609
 sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */
114997115610
 sqlite3_context *pContext, /* Context for sqlite3_result_xxx() calls */
114998115611
 int iCol /* Index of column to read value from */
114999115612
 ){
115000
- int rc; /* Return Code */
115613
+ int rc = SQLITE_OK; /* Return Code */
115001115614
 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor;
115002115615
 Fts3Table *p = (Fts3Table *)pCursor->pVtab;
115003115616
115004115617
 /* The column value supplied by SQLite must be in range. */
115005115618
 assert( iCol>=0 && iCol<=p->nColumn+1 );
@@ -115006,25 +115619,24 @@
115006115619
115007115620
 if( iCol==p->nColumn+1 ){
115008115621
 /* This call is a request for the "docid" column. Since "docid" is an
115009115622
 ** alias for "rowid", use the xRowid() method to obtain the value.
115010115623
 */
115011
- sqlite3_int64 iRowid;
115012
- rc = fts3RowidMethod(pCursor, &iRowid);
115013
- sqlite3_result_int64(pContext, iRowid);
115624
+ sqlite3_result_int64(pContext, pCsr->iPrevId);
115014115625
 }else if( iCol==p->nColumn ){
115015115626
 /* The extra column whose name is the same as the table.
115016115627
 ** Return a blob which is a pointer to the cursor.
115017115628
 */
115018115629
 sqlite3_result_blob(pContext, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT);
115019
- rc = SQLITE_OK;
115020115630
 }else{
115021115631
 rc = fts3CursorSeek(0, pCsr);
115022115632
 if( rc==SQLITE_OK ){
115023115633
 sqlite3_result_value(pContext, sqlite3_column_value(pCsr->pStmt, iCol+1));
115024115634
 }
115025115635
 }
115636
+
115637
+ assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
115026115638
 return rc;
115027115639
 }
115028115640
115029115641
 /*
115030115642
 ** This function is the implementation of the xUpdate callback used by
@@ -115052,12 +115664,13 @@
115052115664
115053115665
 /*
115054115666
 ** Implementation of xBegin() method. This is a no-op.
115055115667
 */
115056115668
 static int fts3BeginMethod(sqlite3_vtab *pVtab){
115057
- UNUSED_PARAMETER(pVtab);
115058115669
 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115670
+ UNUSED_PARAMETER(pVtab);
115671
+ assert( p->pSegments==0 );
115059115672
 assert( p->nPendingData==0 );
115060115673
 assert( p->inTransaction!=1 );
115061115674
 TESTONLY( p->inTransaction = 1 );
115062115675
 TESTONLY( p->mxSavepoint = -1; );
115063115676
 return SQLITE_OK;
@@ -115067,14 +115680,15 @@
115067115680
 ** Implementation of xCommit() method. This is a no-op. The contents of
115068115681
 ** the pending-terms hash-table have already been flushed into the database
115069115682
 ** by fts3SyncMethod().
115070115683
 */
115071115684
 static int fts3CommitMethod(sqlite3_vtab *pVtab){
115072
- UNUSED_PARAMETER(pVtab);
115073115685
 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115686
+ UNUSED_PARAMETER(pVtab);
115074115687
 assert( p->nPendingData==0 );
115075115688
 assert( p->inTransaction!=0 );
115689
+ assert( p->pSegments==0 );
115076115690
 TESTONLY( p->inTransaction = 0 );
115077115691
 TESTONLY( p->mxSavepoint = -1; );
115078115692
 return SQLITE_OK;
115079115693
 }
115080115694
@@ -115088,132 +115702,30 @@
115088115702
 assert( p->inTransaction!=0 );
115089115703
 TESTONLY( p->inTransaction = 0 );
115090115704
 TESTONLY( p->mxSavepoint = -1; );
115091115705
 return SQLITE_OK;
115092115706
 }
115093
-
115094
-/*
115095
-** Load the doclist associated with expression pExpr to pExpr->aDoclist.
115096
-** The loaded doclist contains positions as well as the document ids.
115097
-** This is used by the matchinfo(), snippet() and offsets() auxillary
115098
-** functions.
115099
-*/
115100
-SQLITE_PRIVATE int sqlite3Fts3ExprLoadDoclist(Fts3Cursor *pCsr, Fts3Expr *pExpr){
115101
- int rc;
115102
- assert( pExpr->eType==FTSQUERY_PHRASE && pExpr->pPhrase );
115103
- assert( pCsr->eEvalmode==FTS3_EVAL_NEXT );
115104
- rc = fts3EvalExpr(pCsr, pExpr, &pExpr->aDoclist, &pExpr->nDoclist, 1);
115105
- return rc;
115106
-}
115107
-
115108
-SQLITE_PRIVATE int sqlite3Fts3ExprLoadFtDoclist(
115109
- Fts3Cursor *pCsr,
115110
- Fts3Expr *pExpr,
115111
- char **paDoclist,
115112
- int *pnDoclist
115113
-){
115114
- int rc;
115115
- assert( pCsr->eEvalmode==FTS3_EVAL_NEXT );
115116
- assert( pExpr->eType==FTSQUERY_PHRASE && pExpr->pPhrase );
115117
- pCsr->eEvalmode = FTS3_EVAL_MATCHINFO;
115118
- rc = fts3EvalExpr(pCsr, pExpr, paDoclist, pnDoclist, 1);
115119
- pCsr->eEvalmode = FTS3_EVAL_NEXT;
115120
- return rc;
115121
-}
115122
-
115123115707
115124115708
 /*
115125115709
 ** When called, *ppPoslist must point to the byte immediately following the
115126115710
 ** end of a position-list. i.e. ( (*ppPoslist)[-1]==POS_END ). This function
115127115711
 ** moves *ppPoslist so that it instead points to the first byte of the
115128115712
 ** same position list.
115129115713
 */
115130115714
 static void fts3ReversePoslist(char *pStart, char **ppPoslist){
115131
- char *p = &(*ppPoslist)[-3];
115132
- char c = p[1];
115715
+ char *p = &(*ppPoslist)[-2];
115716
+ char c;
115717
+
115718
+ while( p>pStart && (c=*p--)==0 );
115133115719
 while( p>pStart && (*p & 0x80) | c ){
115134115720
 c = *p--;
115135115721
 }
115136115722
 if( p>pStart ){ p = &p[2]; }
115137115723
 while( *p++&0x80 );
115138115724
 *ppPoslist = p;
115139115725
 }
115140115726
115141
-
115142
-/*
115143
-** After ExprLoadDoclist() (see above) has been called, this function is
115144
-** used to iterate/search through the position lists that make up the doclist
115145
-** stored in pExpr->aDoclist.
115146
-*/
115147
-SQLITE_PRIVATE char *sqlite3Fts3FindPositions(
115148
- Fts3Cursor *pCursor, /* Associate FTS3 cursor */
115149
- Fts3Expr *pExpr, /* Access this expressions doclist */
115150
- sqlite3_int64 iDocid, /* Docid associated with requested pos-list */
115151
- int iCol /* Column of requested pos-list */
115152
-){
115153
- assert( pExpr->isLoaded );
115154
- if( pExpr->aDoclist ){
115155
- char *pEnd = &pExpr->aDoclist[pExpr->nDoclist];
115156
- char *pCsr;
115157
-
115158
- if( pExpr->pCurrent==0 ){
115159
- if( pCursor->desc==0 ){
115160
- pExpr->pCurrent = pExpr->aDoclist;
115161
- pExpr->iCurrent = 0;
115162
- fts3GetDeltaVarint(&pExpr->pCurrent, &pExpr->iCurrent);
115163
- }else{
115164
- pCsr = pExpr->aDoclist;
115165
- while( pCsr<pEnd ){
115166
- fts3GetDeltaVarint(&pCsr, &pExpr->iCurrent);
115167
- fts3PoslistCopy(0, &pCsr);
115168
- }
115169
- fts3ReversePoslist(pExpr->aDoclist, &pCsr);
115170
- pExpr->pCurrent = pCsr;
115171
- }
115172
- }
115173
- pCsr = pExpr->pCurrent;
115174
- assert( pCsr );
115175
-
115176
- while( (pCursor->desc==0 && pCsr<pEnd)
115177
- || (pCursor->desc && pCsr>pExpr->aDoclist)
115178
- ){
115179
- if( pCursor->desc==0 && pExpr->iCurrent<iDocid ){
115180
- fts3PoslistCopy(0, &pCsr);
115181
- if( pCsr<pEnd ){
115182
- fts3GetDeltaVarint(&pCsr, &pExpr->iCurrent);
115183
- }
115184
- pExpr->pCurrent = pCsr;
115185
- }else if( pCursor->desc && pExpr->iCurrent>iDocid ){
115186
- fts3GetReverseDeltaVarint(&pCsr, pExpr->aDoclist, &pExpr->iCurrent);
115187
- fts3ReversePoslist(pExpr->aDoclist, &pCsr);
115188
- pExpr->pCurrent = pCsr;
115189
- }else{
115190
- if( pExpr->iCurrent==iDocid ){
115191
- int iThis = 0;
115192
- if( iCol<0 ){
115193
- /* If iCol is negative, return a pointer to the start of the
115194
- ** position-list (instead of a pointer to the start of a list
115195
- ** of offsets associated with a specific column).
115196
- */
115197
- return pCsr;
115198
- }
115199
- while( iThis<iCol ){
115200
- fts3ColumnlistCopy(0, &pCsr);
115201
- if( *pCsr==0x00 ) return 0;
115202
- pCsr++;
115203
- pCsr += sqlite3Fts3GetVarint32(pCsr, &iThis);
115204
- }
115205
- if( iCol==iThis && (*pCsr&0xFE) ) return pCsr;
115206
- }
115207
- return 0;
115208
- }
115209
- }
115210
- }
115211
-
115212
- return 0;
115213
-}
115214
-
115215115727
 /*
115216115728
 ** Helper function used by the implementation of the overloaded snippet(),
115217115729
 ** offsets() and optimize() SQL functions.
115218115730
 **
115219115731
 ** If the value passed as the third argument is a blob of size
@@ -115440,27 +115952,43 @@
115440115952
 p->zDb, p->zName, zName
115441115953
 );
115442115954
 return rc;
115443115955
 }
115444115956
115957
+/*
115958
+** The xSavepoint() method.
115959
+**
115960
+** Flush the contents of the pending-terms table to disk.
115961
+*/
115445115962
 static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
115446
- Fts3Table *p = (Fts3Table*)pVtab;
115447115963
 UNUSED_PARAMETER(iSavepoint);
115448
- assert( p->inTransaction );
115449
- assert( p->mxSavepoint < iSavepoint );
115450
- TESTONLY( p->mxSavepoint = iSavepoint );
115451
- return sqlite3Fts3PendingTermsFlush(p);
115964
+ assert( ((Fts3Table *)pVtab)->inTransaction );
115965
+ assert( ((Fts3Table *)pVtab)->mxSavepoint < iSavepoint );
115966
+ TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint );
115967
+ return fts3SyncMethod(pVtab);
115452115968
 }
115969
+
115970
+/*
115971
+** The xRelease() method.
115972
+**
115973
+** This is a no-op.
115974
+*/
115453115975
 static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
115454115976
 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115455115977
 UNUSED_PARAMETER(iSavepoint);
115456115978
 UNUSED_PARAMETER(pVtab);
115457115979
 assert( p->inTransaction );
115458115980
 assert( p->mxSavepoint >= iSavepoint );
115459115981
 TESTONLY( p->mxSavepoint = iSavepoint-1 );
115460115982
 return SQLITE_OK;
115461115983
 }
115984
+
115985
+/*
115986
+** The xRollbackTo() method.
115987
+**
115988
+** Discard the contents of the pending terms table.
115989
+*/
115462115990
 static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
115463115991
 Fts3Table *p = (Fts3Table*)pVtab;
115464115992
 UNUSED_PARAMETER(iSavepoint);
115465115993
 assert( p->inTransaction );
115466115994
 assert( p->mxSavepoint >= iSavepoint );
@@ -115605,12 +116133,1553 @@
115605116133
 sqlite3Fts3HashClear(pHash);
115606116134
 sqlite3_free(pHash);
115607116135
 }
115608116136
 return rc;
115609116137
 }
116138
+
116139
+/*
116140
+** Allocate an Fts3MultiSegReader for each token in the expression headed
116141
+** by pExpr.
116142
+**
116143
+** An Fts3SegReader object is a cursor that can seek or scan a range of
116144
+** entries within a single segment b-tree. An Fts3MultiSegReader uses multiple
116145
+** Fts3SegReader objects internally to provide an interface to seek or scan
116146
+** within the union of all segments of a b-tree. Hence the name.
116147
+**
116148
+** If the allocated Fts3MultiSegReader just seeks to a single entry in a
116149
+** segment b-tree (if the term is not a prefix or it is a prefix for which
116150
+** there exists prefix b-tree of the right length) then it may be traversed
116151
+** and merged incrementally. Otherwise, it has to be merged into an in-memory
116152
+** doclist and then traversed.
116153
+*/
116154
+static void fts3EvalAllocateReaders(
116155
+ Fts3Cursor *pCsr, /* FTS cursor handle */
116156
+ Fts3Expr *pExpr, /* Allocate readers for this expression */
116157
+ int *pnToken, /* OUT: Total number of tokens in phrase. */
116158
+ int *pnOr, /* OUT: Total number of OR nodes in expr. */
116159
+ int *pRc /* IN/OUT: Error code */
116160
+){
116161
+ if( pExpr && SQLITE_OK==*pRc ){
116162
+ if( pExpr->eType==FTSQUERY_PHRASE ){
116163
+ int i;
116164
+ int nToken = pExpr->pPhrase->nToken;
116165
+ *pnToken += nToken;
116166
+ for(i=0; i<nToken; i++){
116167
+ Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i];
116168
+ int rc = fts3TermSegReaderCursor(pCsr,
116169
+ pToken->z, pToken->n, pToken->isPrefix, &pToken->pSegcsr
116170
+ );
116171
+ if( rc!=SQLITE_OK ){
116172
+ *pRc = rc;
116173
+ return;
116174
+ }
116175
+ }
116176
+ assert( pExpr->pPhrase->iDoclistToken==0 );
116177
+ pExpr->pPhrase->iDoclistToken = -1;
116178
+ }else{
116179
+ *pnOr += (pExpr->eType==FTSQUERY_OR);
116180
+ fts3EvalAllocateReaders(pCsr, pExpr->pLeft, pnToken, pnOr, pRc);
116181
+ fts3EvalAllocateReaders(pCsr, pExpr->pRight, pnToken, pnOr, pRc);
116182
+ }
116183
+ }
116184
+}
116185
+
116186
+/*
116187
+** Arguments pList/nList contain the doclist for token iToken of phrase p.
116188
+** It is merged into the main doclist stored in p->doclist.aAll/nAll.
116189
+**
116190
+** This function assumes that pList points to a buffer allocated using
116191
+** sqlite3_malloc(). This function takes responsibility for eventually
116192
+** freeing the buffer.
116193
+*/
116194
+static void fts3EvalPhraseMergeToken(
116195
+ Fts3Table *pTab, /* FTS Table pointer */
116196
+ Fts3Phrase *p, /* Phrase to merge pList/nList into */
116197
+ int iToken, /* Token pList/nList corresponds to */
116198
+ char *pList, /* Pointer to doclist */
116199
+ int nList /* Number of bytes in pList */
116200
+){
116201
+ assert( iToken!=p->iDoclistToken );
116202
+
116203
+ if( pList==0 ){
116204
+ sqlite3_free(p->doclist.aAll);
116205
+ p->doclist.aAll = 0;
116206
+ p->doclist.nAll = 0;
116207
+ }
116208
+
116209
+ else if( p->iDoclistToken<0 ){
116210
+ p->doclist.aAll = pList;
116211
+ p->doclist.nAll = nList;
116212
+ }
116213
+
116214
+ else if( p->doclist.aAll==0 ){
116215
+ sqlite3_free(pList);
116216
+ }
116217
+
116218
+ else {
116219
+ char *pLeft;
116220
+ char *pRight;
116221
+ int nLeft;
116222
+ int nRight;
116223
+ int nDiff;
116224
+
116225
+ if( p->iDoclistToken<iToken ){
116226
+ pLeft = p->doclist.aAll;
116227
+ nLeft = p->doclist.nAll;
116228
+ pRight = pList;
116229
+ nRight = nList;
116230
+ nDiff = iToken - p->iDoclistToken;
116231
+ }else{
116232
+ pRight = p->doclist.aAll;
116233
+ nRight = p->doclist.nAll;
116234
+ pLeft = pList;
116235
+ nLeft = nList;
116236
+ nDiff = p->iDoclistToken - iToken;
116237
+ }
116238
+
116239
+ fts3DoclistPhraseMerge(pTab->bDescIdx, nDiff, pLeft, nLeft, pRight,&nRight);
116240
+ sqlite3_free(pLeft);
116241
+ p->doclist.aAll = pRight;
116242
+ p->doclist.nAll = nRight;
116243
+ }
116244
+
116245
+ if( iToken>p->iDoclistToken ) p->iDoclistToken = iToken;
116246
+}
116247
+
116248
+/*
116249
+** Load the doclist for phrase p into p->doclist.aAll/nAll. The loaded doclist
116250
+** does not take deferred tokens into account.
116251
+**
116252
+** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
116253
+*/
116254
+static int fts3EvalPhraseLoad(
116255
+ Fts3Cursor *pCsr, /* FTS Cursor handle */
116256
+ Fts3Phrase *p /* Phrase object */
116257
+){
116258
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116259
+ int iToken;
116260
+ int rc = SQLITE_OK;
116261
+
116262
+ for(iToken=0; rc==SQLITE_OK && iToken<p->nToken; iToken++){
116263
+ Fts3PhraseToken *pToken = &p->aToken[iToken];
116264
+ assert( pToken->pDeferred==0 || pToken->pSegcsr==0 );
116265
+
116266
+ if( pToken->pSegcsr ){
116267
+ int nThis = 0;
116268
+ char *pThis = 0;
116269
+ rc = fts3TermSelect(pTab, pToken, p->iColumn, &nThis, &pThis);
116270
+ if( rc==SQLITE_OK ){
116271
+ fts3EvalPhraseMergeToken(pTab, p, iToken, pThis, nThis);
116272
+ }
116273
+ }
116274
+ assert( pToken->pSegcsr==0 );
116275
+ }
116276
+
116277
+ return rc;
116278
+}
116279
+
116280
+/*
116281
+** This function is called on each phrase after the position lists for
116282
+** any deferred tokens have been loaded into memory. It updates the phrases
116283
+** current position list to include only those positions that are really
116284
+** instances of the phrase (after considering deferred tokens). If this
116285
+** means that the phrase does not appear in the current row, doclist.pList
116286
+** and doclist.nList are both zeroed.
116287
+**
116288
+** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
116289
+*/
116290
+static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
116291
+ int iToken; /* Used to iterate through phrase tokens */
116292
+ int rc = SQLITE_OK; /* Return code */
116293
+ char *aPoslist = 0; /* Position list for deferred tokens */
116294
+ int nPoslist = 0; /* Number of bytes in aPoslist */
116295
+ int iPrev = -1; /* Token number of previous deferred token */
116296
+
116297
+ assert( pPhrase->doclist.bFreeList==0 );
116298
+
116299
+ for(iToken=0; rc==SQLITE_OK && iToken<pPhrase->nToken; iToken++){
116300
+ Fts3PhraseToken *pToken = &pPhrase->aToken[iToken];
116301
+ Fts3DeferredToken *pDeferred = pToken->pDeferred;
116302
+
116303
+ if( pDeferred ){
116304
+ char *pList;
116305
+ int nList;
116306
+ rc = sqlite3Fts3DeferredTokenList(pDeferred, &pList, &nList);
116307
+ if( rc!=SQLITE_OK ) return rc;
116308
+
116309
+ if( pList==0 ){
116310
+ sqlite3_free(aPoslist);
116311
+ pPhrase->doclist.pList = 0;
116312
+ pPhrase->doclist.nList = 0;
116313
+ return SQLITE_OK;
116314
+
116315
+ }else if( aPoslist==0 ){
116316
+ aPoslist = pList;
116317
+ nPoslist = nList;
116318
+
116319
+ }else{
116320
+ char *aOut = pList;
116321
+ char *p1 = aPoslist;
116322
+ char *p2 = aOut;
116323
+
116324
+ assert( iPrev>=0 );
116325
+ fts3PoslistPhraseMerge(&aOut, iToken-iPrev, 0, 1, &p1, &p2);
116326
+ sqlite3_free(aPoslist);
116327
+ aPoslist = pList;
116328
+ nPoslist = aOut - aPoslist;
116329
+ if( nPoslist==0 ){
116330
+ sqlite3_free(aPoslist);
116331
+ pPhrase->doclist.pList = 0;
116332
+ pPhrase->doclist.nList = 0;
116333
+ return SQLITE_OK;
116334
+ }
116335
+ }
116336
+ iPrev = iToken;
116337
+ }
116338
+ }
116339
+
116340
+ if( iPrev>=0 ){
116341
+ int nMaxUndeferred = pPhrase->iDoclistToken;
116342
+ if( nMaxUndeferred<0 ){
116343
+ pPhrase->doclist.pList = aPoslist;
116344
+ pPhrase->doclist.nList = nPoslist;
116345
+ pPhrase->doclist.iDocid = pCsr->iPrevId;
116346
+ pPhrase->doclist.bFreeList = 1;
116347
+ }else{
116348
+ int nDistance;
116349
+ char *p1;
116350
+ char *p2;
116351
+ char *aOut;
116352
+
116353
+ if( nMaxUndeferred>iPrev ){
116354
+ p1 = aPoslist;
116355
+ p2 = pPhrase->doclist.pList;
116356
+ nDistance = nMaxUndeferred - iPrev;
116357
+ }else{
116358
+ p1 = pPhrase->doclist.pList;
116359
+ p2 = aPoslist;
116360
+ nDistance = iPrev - nMaxUndeferred;
116361
+ }
116362
+
116363
+ aOut = (char *)sqlite3_malloc(nPoslist+8);
116364
+ if( !aOut ){
116365
+ sqlite3_free(aPoslist);
116366
+ return SQLITE_NOMEM;
116367
+ }
116368
+
116369
+ pPhrase->doclist.pList = aOut;
116370
+ if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){
116371
+ pPhrase->doclist.bFreeList = 1;
116372
+ pPhrase->doclist.nList = (aOut - pPhrase->doclist.pList);
116373
+ }else{
116374
+ sqlite3_free(aOut);
116375
+ pPhrase->doclist.pList = 0;
116376
+ pPhrase->doclist.nList = 0;
116377
+ }
116378
+ sqlite3_free(aPoslist);
116379
+ }
116380
+ }
116381
+
116382
+ return SQLITE_OK;
116383
+}
116384
+
116385
+/*
116386
+** This function is called for each Fts3Phrase in a full-text query
116387
+** expression to initialize the mechanism for returning rows. Once this
116388
+** function has been called successfully on an Fts3Phrase, it may be
116389
+** used with fts3EvalPhraseNext() to iterate through the matching docids.
116390
+**
116391
+** If parameter bOptOk is true, then the phrase may (or may not) use the
116392
+** incremental loading strategy. Otherwise, the entire doclist is loaded into
116393
+** memory within this call.
116394
+**
116395
+** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
116396
+*/
116397
+static int fts3EvalPhraseStart(Fts3Cursor *pCsr, int bOptOk, Fts3Phrase *p){
116398
+ int rc; /* Error code */
116399
+ Fts3PhraseToken *pFirst = &p->aToken[0];
116400
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116401
+
116402
+ if( pCsr->bDesc==pTab->bDescIdx
116403
+ && bOptOk==1
116404
+ && p->nToken==1
116405
+ && pFirst->pSegcsr
116406
+ && pFirst->pSegcsr->bLookup
116407
+ ){
116408
+ /* Use the incremental approach. */
116409
+ int iCol = (p->iColumn >= pTab->nColumn ? -1 : p->iColumn);
116410
+ rc = sqlite3Fts3MsrIncrStart(
116411
+ pTab, pFirst->pSegcsr, iCol, pFirst->z, pFirst->n);
116412
+ p->bIncr = 1;
116413
+
116414
+ }else{
116415
+ /* Load the full doclist for the phrase into memory. */
116416
+ rc = fts3EvalPhraseLoad(pCsr, p);
116417
+ p->bIncr = 0;
116418
+ }
116419
+
116420
+ assert( rc!=SQLITE_OK || p->nToken<1 || p->aToken[0].pSegcsr==0 || p->bIncr );
116421
+ return rc;
116422
+}
116423
+
116424
+/*
116425
+** This function is used to iterate backwards (from the end to start)
116426
+** through doclists. It is used by this module to iterate through phrase
116427
+** doclists in reverse and by the fts3_write.c module to iterate through
116428
+** pending-terms lists when writing to databases with "order=desc".
116429
+**
116430
+** The doclist may be sorted in ascending (parameter bDescIdx==0) or
116431
+** descending (parameter bDescIdx==1) order of docid. Regardless, this
116432
+** function iterates from the end of the doclist to the beginning.
116433
+*/
116434
+SQLITE_PRIVATE void sqlite3Fts3DoclistPrev(
116435
+ int bDescIdx, /* True if the doclist is desc */
116436
+ char *aDoclist, /* Pointer to entire doclist */
116437
+ int nDoclist, /* Length of aDoclist in bytes */
116438
+ char **ppIter, /* IN/OUT: Iterator pointer */
116439
+ sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */
116440
+ int *pnList, /* IN/OUT: List length pointer */
116441
+ u8 *pbEof /* OUT: End-of-file flag */
116442
+){
116443
+ char *p = *ppIter;
116444
+
116445
+ assert( nDoclist>0 );
116446
+ assert( *pbEof==0 );
116447
+ assert( p || *piDocid==0 );
116448
+ assert( !p || (p>aDoclist && p<&aDoclist[nDoclist]) );
116449
+
116450
+ if( p==0 ){
116451
+ sqlite3_int64 iDocid = 0;
116452
+ char *pNext = 0;
116453
+ char *pDocid = aDoclist;
116454
+ char *pEnd = &aDoclist[nDoclist];
116455
+ int iMul = 1;
116456
+
116457
+ while( pDocid<pEnd ){
116458
+ sqlite3_int64 iDelta;
116459
+ pDocid += sqlite3Fts3GetVarint(pDocid, &iDelta);
116460
+ iDocid += (iMul * iDelta);
116461
+ pNext = pDocid;
116462
+ fts3PoslistCopy(0, &pDocid);
116463
+ while( pDocid<pEnd && *pDocid==0 ) pDocid++;
116464
+ iMul = (bDescIdx ? -1 : 1);
116465
+ }
116466
+
116467
+ *pnList = pEnd - pNext;
116468
+ *ppIter = pNext;
116469
+ *piDocid = iDocid;
116470
+ }else{
116471
+ int iMul = (bDescIdx ? -1 : 1);
116472
+ sqlite3_int64 iDelta;
116473
+ fts3GetReverseVarint(&p, aDoclist, &iDelta);
116474
+ *piDocid -= (iMul * iDelta);
116475
+
116476
+ if( p==aDoclist ){
116477
+ *pbEof = 1;
116478
+ }else{
116479
+ char *pSave = p;
116480
+ fts3ReversePoslist(aDoclist, &p);
116481
+ *pnList = (pSave - p);
116482
+ }
116483
+ *ppIter = p;
116484
+ }
116485
+}
116486
+
116487
+/*
116488
+** Attempt to move the phrase iterator to point to the next matching docid.
116489
+** If an error occurs, return an SQLite error code. Otherwise, return
116490
+** SQLITE_OK.
116491
+**
116492
+** If there is no "next" entry and no error occurs, then *pbEof is set to
116493
+** 1 before returning. Otherwise, if no error occurs and the iterator is
116494
+** successfully advanced, *pbEof is set to 0.
116495
+*/
116496
+static int fts3EvalPhraseNext(
116497
+ Fts3Cursor *pCsr, /* FTS Cursor handle */
116498
+ Fts3Phrase *p, /* Phrase object to advance to next docid */
116499
+ u8 *pbEof /* OUT: Set to 1 if EOF */
116500
+){
116501
+ int rc = SQLITE_OK;
116502
+ Fts3Doclist *pDL = &p->doclist;
116503
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116504
+
116505
+ if( p->bIncr ){
116506
+ assert( p->nToken==1 );
116507
+ assert( pDL->pNextDocid==0 );
116508
+ rc = sqlite3Fts3MsrIncrNext(pTab, p->aToken[0].pSegcsr,
116509
+ &pDL->iDocid, &pDL->pList, &pDL->nList
116510
+ );
116511
+ if( rc==SQLITE_OK && !pDL->pList ){
116512
+ *pbEof = 1;
116513
+ }
116514
+ }else if( pCsr->bDesc!=pTab->bDescIdx && pDL->nAll ){
116515
+ sqlite3Fts3DoclistPrev(pTab->bDescIdx, pDL->aAll, pDL->nAll,
116516
+ &pDL->pNextDocid, &pDL->iDocid, &pDL->nList, pbEof
116517
+ );
116518
+ pDL->pList = pDL->pNextDocid;
116519
+ }else{
116520
+ char *pIter; /* Used to iterate through aAll */
116521
+ char *pEnd = &pDL->aAll[pDL->nAll]; /* 1 byte past end of aAll */
116522
+ if( pDL->pNextDocid ){
116523
+ pIter = pDL->pNextDocid;
116524
+ }else{
116525
+ pIter = pDL->aAll;
116526
+ }
116527
+
116528
+ if( pIter>=pEnd ){
116529
+ /* We have already reached the end of this doclist. EOF. */
116530
+ *pbEof = 1;
116531
+ }else{
116532
+ sqlite3_int64 iDelta;
116533
+ pIter += sqlite3Fts3GetVarint(pIter, &iDelta);
116534
+ if( pTab->bDescIdx==0 || pDL->pNextDocid==0 ){
116535
+ pDL->iDocid += iDelta;
116536
+ }else{
116537
+ pDL->iDocid -= iDelta;
116538
+ }
116539
+ pDL->pList = pIter;
116540
+ fts3PoslistCopy(0, &pIter);
116541
+ pDL->nList = (pIter - pDL->pList);
116542
+
116543
+ /* pIter now points just past the 0x00 that terminates the position-
116544
+ ** list for document pDL->iDocid. However, if this position-list was
116545
+ ** edited in place by fts3EvalNearTrim(), then pIter may not actually
116546
+ ** point to the start of the next docid value. The following line deals
116547
+ ** with this case by advancing pIter past the zero-padding added by
116548
+ ** fts3EvalNearTrim(). */
116549
+ while( pIter<pEnd && *pIter==0 ) pIter++;
116550
+
116551
+ pDL->pNextDocid = pIter;
116552
+ assert( pIter>=&pDL->aAll[pDL->nAll] || *pIter );
116553
+ *pbEof = 0;
116554
+ }
116555
+ }
116556
+
116557
+ return rc;
116558
+}
116559
+
116560
+/*
116561
+**
116562
+** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
116563
+** Otherwise, fts3EvalPhraseStart() is called on all phrases within the
116564
+** expression. Also the Fts3Expr.bDeferred variable is set to true for any
116565
+** expressions for which all descendent tokens are deferred.
116566
+**
116567
+** If parameter bOptOk is zero, then it is guaranteed that the
116568
+** Fts3Phrase.doclist.aAll/nAll variables contain the entire doclist for
116569
+** each phrase in the expression (subject to deferred token processing).
116570
+** Or, if bOptOk is non-zero, then one or more tokens within the expression
116571
+** may be loaded incrementally, meaning doclist.aAll/nAll is not available.
116572
+**
116573
+** If an error occurs within this function, *pRc is set to an SQLite error
116574
+** code before returning.
116575
+*/
116576
+static void fts3EvalStartReaders(
116577
+ Fts3Cursor *pCsr, /* FTS Cursor handle */
116578
+ Fts3Expr *pExpr, /* Expression to initialize phrases in */
116579
+ int bOptOk, /* True to enable incremental loading */
116580
+ int *pRc /* IN/OUT: Error code */
116581
+){
116582
+ if( pExpr && SQLITE_OK==*pRc ){
116583
+ if( pExpr->eType==FTSQUERY_PHRASE ){
116584
+ int i;
116585
+ int nToken = pExpr->pPhrase->nToken;
116586
+ for(i=0; i<nToken; i++){
116587
+ if( pExpr->pPhrase->aToken[i].pDeferred==0 ) break;
116588
+ }
116589
+ pExpr->bDeferred = (i==nToken);
116590
+ *pRc = fts3EvalPhraseStart(pCsr, bOptOk, pExpr->pPhrase);
116591
+ }else{
116592
+ fts3EvalStartReaders(pCsr, pExpr->pLeft, bOptOk, pRc);
116593
+ fts3EvalStartReaders(pCsr, pExpr->pRight, bOptOk, pRc);
116594
+ pExpr->bDeferred = (pExpr->pLeft->bDeferred && pExpr->pRight->bDeferred);
116595
+ }
116596
+ }
116597
+}
116598
+
116599
+/*
116600
+** An array of the following structures is assembled as part of the process
116601
+** of selecting tokens to defer before the query starts executing (as part
116602
+** of the xFilter() method). There is one element in the array for each
116603
+** token in the FTS expression.
116604
+**
116605
+** Tokens are divided into AND/NEAR clusters. All tokens in a cluster belong
116606
+** to phrases that are connected only by AND and NEAR operators (not OR or
116607
+** NOT). When determining tokens to defer, each AND/NEAR cluster is considered
116608
+** separately. The root of a tokens AND/NEAR cluster is stored in
116609
+** Fts3TokenAndCost.pRoot.
116610
+*/
116611
+typedef struct Fts3TokenAndCost Fts3TokenAndCost;
116612
+struct Fts3TokenAndCost {
116613
+ Fts3Phrase *pPhrase; /* The phrase the token belongs to */
116614
+ int iToken; /* Position of token in phrase */
116615
+ Fts3PhraseToken *pToken; /* The token itself */
116616
+ Fts3Expr *pRoot; /* Root of NEAR/AND cluster */
116617
+ int nOvfl; /* Number of overflow pages to load doclist */
116618
+ int iCol; /* The column the token must match */
116619
+};
116620
+
116621
+/*
116622
+** This function is used to populate an allocated Fts3TokenAndCost array.
116623
+**
116624
+** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
116625
+** Otherwise, if an error occurs during execution, *pRc is set to an
116626
+** SQLite error code.
116627
+*/
116628
+static void fts3EvalTokenCosts(
116629
+ Fts3Cursor *pCsr, /* FTS Cursor handle */
116630
+ Fts3Expr *pRoot, /* Root of current AND/NEAR cluster */
116631
+ Fts3Expr *pExpr, /* Expression to consider */
116632
+ Fts3TokenAndCost **ppTC, /* Write new entries to *(*ppTC)++ */
116633
+ Fts3Expr ***ppOr, /* Write new OR root to *(*ppOr)++ */
116634
+ int *pRc /* IN/OUT: Error code */
116635
+){
116636
+ if( *pRc==SQLITE_OK && pExpr ){
116637
+ if( pExpr->eType==FTSQUERY_PHRASE ){
116638
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
116639
+ int i;
116640
+ for(i=0; *pRc==SQLITE_OK && i<pPhrase->nToken; i++){
116641
+ Fts3TokenAndCost *pTC = (*ppTC)++;
116642
+ pTC->pPhrase = pPhrase;
116643
+ pTC->iToken = i;
116644
+ pTC->pRoot = pRoot;
116645
+ pTC->pToken = &pPhrase->aToken[i];
116646
+ pTC->iCol = pPhrase->iColumn;
116647
+ *pRc = sqlite3Fts3MsrOvfl(pCsr, pTC->pToken->pSegcsr, &pTC->nOvfl);
116648
+ }
116649
+ }else if( pExpr->eType!=FTSQUERY_NOT ){
116650
+ if( pExpr->eType==FTSQUERY_OR ){
116651
+ pRoot = pExpr->pLeft;
116652
+ **ppOr = pRoot;
116653
+ (*ppOr)++;
116654
+ }
116655
+ fts3EvalTokenCosts(pCsr, pRoot, pExpr->pLeft, ppTC, ppOr, pRc);
116656
+ if( pExpr->eType==FTSQUERY_OR ){
116657
+ pRoot = pExpr->pRight;
116658
+ **ppOr = pRoot;
116659
+ (*ppOr)++;
116660
+ }
116661
+ fts3EvalTokenCosts(pCsr, pRoot, pExpr->pRight, ppTC, ppOr, pRc);
116662
+ }
116663
+ }
116664
+}
116665
+
116666
+/*
116667
+** Determine the average document (row) size in pages. If successful,
116668
+** write this value to *pnPage and return SQLITE_OK. Otherwise, return
116669
+** an SQLite error code.
116670
+**
116671
+** The average document size in pages is calculated by first calculating
116672
+** determining the average size in bytes, B. If B is less than the amount
116673
+** of data that will fit on a single leaf page of an intkey table in
116674
+** this database, then the average docsize is 1. Otherwise, it is 1 plus
116675
+** the number of overflow pages consumed by a record B bytes in size.
116676
+*/
116677
+static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){
116678
+ if( pCsr->nRowAvg==0 ){
116679
+ /* The average document size, which is required to calculate the cost
116680
+ ** of each doclist, has not yet been determined. Read the required
116681
+ ** data from the %_stat table to calculate it.
116682
+ **
116683
+ ** Entry 0 of the %_stat table is a blob containing (nCol+1) FTS3
116684
+ ** varints, where nCol is the number of columns in the FTS3 table.
116685
+ ** The first varint is the number of documents currently stored in
116686
+ ** the table. The following nCol varints contain the total amount of
116687
+ ** data stored in all rows of each column of the table, from left
116688
+ ** to right.
116689
+ */
116690
+ int rc;
116691
+ Fts3Table *p = (Fts3Table*)pCsr->base.pVtab;
116692
+ sqlite3_stmt *pStmt;
116693
+ sqlite3_int64 nDoc = 0;
116694
+ sqlite3_int64 nByte = 0;
116695
+ const char *pEnd;
116696
+ const char *a;
116697
+
116698
+ rc = sqlite3Fts3SelectDoctotal(p, &pStmt);
116699
+ if( rc!=SQLITE_OK ) return rc;
116700
+ a = sqlite3_column_blob(pStmt, 0);
116701
+ assert( a );
116702
+
116703
+ pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
116704
+ a += sqlite3Fts3GetVarint(a, &nDoc);
116705
+ while( a<pEnd ){
116706
+ a += sqlite3Fts3GetVarint(a, &nByte);
116707
+ }
116708
+ if( nDoc==0 || nByte==0 ){
116709
+ sqlite3_reset(pStmt);
116710
+ return SQLITE_CORRUPT_VTAB;
116711
+ }
116712
+
116713
+ pCsr->nDoc = nDoc;
116714
+ pCsr->nRowAvg = (int)(((nByte / nDoc) + p->nPgsz) / p->nPgsz);
116715
+ assert( pCsr->nRowAvg>0 );
116716
+ rc = sqlite3_reset(pStmt);
116717
+ if( rc!=SQLITE_OK ) return rc;
116718
+ }
116719
+
116720
+ *pnPage = pCsr->nRowAvg;
116721
+ return SQLITE_OK;
116722
+}
116723
+
116724
+/*
116725
+** This function is called to select the tokens (if any) that will be
116726
+** deferred. The array aTC[] has already been populated when this is
116727
+** called.
116728
+**
116729
+** This function is called once for each AND/NEAR cluster in the
116730
+** expression. Each invocation determines which tokens to defer within
116731
+** the cluster with root node pRoot. See comments above the definition
116732
+** of struct Fts3TokenAndCost for more details.
116733
+**
116734
+** If no error occurs, SQLITE_OK is returned and sqlite3Fts3DeferToken()
116735
+** called on each token to defer. Otherwise, an SQLite error code is
116736
+** returned.
116737
+*/
116738
+static int fts3EvalSelectDeferred(
116739
+ Fts3Cursor *pCsr, /* FTS Cursor handle */
116740
+ Fts3Expr *pRoot, /* Consider tokens with this root node */
116741
+ Fts3TokenAndCost *aTC, /* Array of expression tokens and costs */
116742
+ int nTC /* Number of entries in aTC[] */
116743
+){
116744
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116745
+ int nDocSize = 0; /* Number of pages per doc loaded */
116746
+ int rc = SQLITE_OK; /* Return code */
116747
+ int ii; /* Iterator variable for various purposes */
116748
+ int nOvfl = 0; /* Total overflow pages used by doclists */
116749
+ int nToken = 0; /* Total number of tokens in cluster */
116750
+
116751
+ int nMinEst = 0; /* The minimum count for any phrase so far. */
116752
+ int nLoad4 = 1; /* (Phrases that will be loaded)^4. */
116753
+
116754
+ /* Count the tokens in this AND/NEAR cluster. If none of the doclists
116755
+ ** associated with the tokens spill onto overflow pages, or if there is
116756
+ ** only 1 token, exit early. No tokens to defer in this case. */
116757
+ for(ii=0; ii<nTC; ii++){
116758
+ if( aTC[ii].pRoot==pRoot ){
116759
+ nOvfl += aTC[ii].nOvfl;
116760
+ nToken++;
116761
+ }
116762
+ }
116763
+ if( nOvfl==0 || nToken<2 ) return SQLITE_OK;
116764
+
116765
+ /* Obtain the average docsize (in pages). */
116766
+ rc = fts3EvalAverageDocsize(pCsr, &nDocSize);
116767
+ assert( rc!=SQLITE_OK || nDocSize>0 );
116768
+
116769
+
116770
+ /* Iterate through all tokens in this AND/NEAR cluster, in ascending order
116771
+ ** of the number of overflow pages that will be loaded by the pager layer
116772
+ ** to retrieve the entire doclist for the token from the full-text index.
116773
+ ** Load the doclists for tokens that are either:
116774
+ **
116775
+ ** a. The cheapest token in the entire query (i.e. the one visited by the
116776
+ ** first iteration of this loop), or
116777
+ **
116778
+ ** b. Part of a multi-token phrase.
116779
+ **
116780
+ ** After each token doclist is loaded, merge it with the others from the
116781
+ ** same phrase and count the number of documents that the merged doclist
116782
+ ** contains. Set variable "nMinEst" to the smallest number of documents in
116783
+ ** any phrase doclist for which 1 or more token doclists have been loaded.
116784
+ ** Let nOther be the number of other phrases for which it is certain that
116785
+ ** one or more tokens will not be deferred.
116786
+ **
116787
+ ** Then, for each token, defer it if loading the doclist would result in
116788
+ ** loading N or more overflow pages into memory, where N is computed as:
116789
+ **
116790
+ ** (nMinEst + 4^nOther - 1) / (4^nOther)
116791
+ */
116792
+ for(ii=0; ii<nToken && rc==SQLITE_OK; ii++){
116793
+ int iTC; /* Used to iterate through aTC[] array. */
116794
+ Fts3TokenAndCost *pTC = 0; /* Set to cheapest remaining token. */
116795
+
116796
+ /* Set pTC to point to the cheapest remaining token. */
116797
+ for(iTC=0; iTC<nTC; iTC++){
116798
+ if( aTC[iTC].pToken && aTC[iTC].pRoot==pRoot
116799
+ && (!pTC || aTC[iTC].nOvfl<pTC->nOvfl)
116800
+ ){
116801
+ pTC = &aTC[iTC];
116802
+ }
116803
+ }
116804
+ assert( pTC );
116805
+
116806
+ if( ii && pTC->nOvfl>=((nMinEst+(nLoad4/4)-1)/(nLoad4/4))*nDocSize ){
116807
+ /* The number of overflow pages to load for this (and therefore all
116808
+ ** subsequent) tokens is greater than the estimated number of pages
116809
+ ** that will be loaded if all subsequent tokens are deferred.
116810
+ */
116811
+ Fts3PhraseToken *pToken = pTC->pToken;
116812
+ rc = sqlite3Fts3DeferToken(pCsr, pToken, pTC->iCol);
116813
+ fts3SegReaderCursorFree(pToken->pSegcsr);
116814
+ pToken->pSegcsr = 0;
116815
+ }else{
116816
+ nLoad4 = nLoad4*4;
116817
+ if( ii==0 || pTC->pPhrase->nToken>1 ){
116818
+ /* Either this is the cheapest token in the entire query, or it is
116819
+ ** part of a multi-token phrase. Either way, the entire doclist will
116820
+ ** (eventually) be loaded into memory. It may as well be now. */
116821
+ Fts3PhraseToken *pToken = pTC->pToken;
116822
+ int nList = 0;
116823
+ char *pList = 0;
116824
+ rc = fts3TermSelect(pTab, pToken, pTC->iCol, &nList, &pList);
116825
+ assert( rc==SQLITE_OK || pList==0 );
116826
+ if( rc==SQLITE_OK ){
116827
+ int nCount;
116828
+ fts3EvalPhraseMergeToken(pTab, pTC->pPhrase, pTC->iToken,pList,nList);
116829
+ nCount = fts3DoclistCountDocids(
116830
+ pTC->pPhrase->doclist.aAll, pTC->pPhrase->doclist.nAll
116831
+ );
116832
+ if( ii==0 || nCount<nMinEst ) nMinEst = nCount;
116833
+ }
116834
+ }
116835
+ }
116836
+ pTC->pToken = 0;
116837
+ }
116838
+
116839
+ return rc;
116840
+}
116841
+
116842
+/*
116843
+** This function is called from within the xFilter method. It initializes
116844
+** the full-text query currently stored in pCsr->pExpr. To iterate through
116845
+** the results of a query, the caller does:
116846
+**
116847
+** fts3EvalStart(pCsr);
116848
+** while( 1 ){
116849
+** fts3EvalNext(pCsr);
116850
+** if( pCsr->bEof ) break;
116851
+** ... return row pCsr->iPrevId to the caller ...
116852
+** }
116853
+*/
116854
+static int fts3EvalStart(Fts3Cursor *pCsr){
116855
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116856
+ int rc = SQLITE_OK;
116857
+ int nToken = 0;
116858
+ int nOr = 0;
116859
+
116860
+ /* Allocate a MultiSegReader for each token in the expression. */
116861
+ fts3EvalAllocateReaders(pCsr, pCsr->pExpr, &nToken, &nOr, &rc);
116862
+
116863
+ /* Determine which, if any, tokens in the expression should be deferred. */
116864
+ if( rc==SQLITE_OK && nToken>1 && pTab->bHasStat ){
116865
+ Fts3TokenAndCost *aTC;
116866
+ Fts3Expr **apOr;
116867
+ aTC = (Fts3TokenAndCost *)sqlite3_malloc(
116868
+ sizeof(Fts3TokenAndCost) * nToken
116869
+ + sizeof(Fts3Expr *) * nOr * 2
116870
+ );
116871
+ apOr = (Fts3Expr **)&aTC[nToken];
116872
+
116873
+ if( !aTC ){
116874
+ rc = SQLITE_NOMEM;
116875
+ }else{
116876
+ int ii;
116877
+ Fts3TokenAndCost *pTC = aTC;
116878
+ Fts3Expr **ppOr = apOr;
116879
+
116880
+ fts3EvalTokenCosts(pCsr, 0, pCsr->pExpr, &pTC, &ppOr, &rc);
116881
+ nToken = pTC-aTC;
116882
+ nOr = ppOr-apOr;
116883
+
116884
+ if( rc==SQLITE_OK ){
116885
+ rc = fts3EvalSelectDeferred(pCsr, 0, aTC, nToken);
116886
+ for(ii=0; rc==SQLITE_OK && ii<nOr; ii++){
116887
+ rc = fts3EvalSelectDeferred(pCsr, apOr[ii], aTC, nToken);
116888
+ }
116889
+ }
116890
+
116891
+ sqlite3_free(aTC);
116892
+ }
116893
+ }
116894
+
116895
+ fts3EvalStartReaders(pCsr, pCsr->pExpr, 1, &rc);
116896
+ return rc;
116897
+}
116898
+
116899
+/*
116900
+** Invalidate the current position list for phrase pPhrase.
116901
+*/
116902
+static void fts3EvalInvalidatePoslist(Fts3Phrase *pPhrase){
116903
+ if( pPhrase->doclist.bFreeList ){
116904
+ sqlite3_free(pPhrase->doclist.pList);
116905
+ }
116906
+ pPhrase->doclist.pList = 0;
116907
+ pPhrase->doclist.nList = 0;
116908
+ pPhrase->doclist.bFreeList = 0;
116909
+}
116910
+
116911
+/*
116912
+** This function is called to edit the position list associated with
116913
+** the phrase object passed as the fifth argument according to a NEAR
116914
+** condition. For example:
116915
+**
116916
+** abc NEAR/5 "def ghi"
116917
+**
116918
+** Parameter nNear is passed the NEAR distance of the expression (5 in
116919
+** the example above). When this function is called, *paPoslist points to
116920
+** the position list, and *pnToken is the number of phrase tokens in, the
116921
+** phrase on the other side of the NEAR operator to pPhrase. For example,
116922
+** if pPhrase refers to the "def ghi" phrase, then *paPoslist points to
116923
+** the position list associated with phrase "abc".
116924
+**
116925
+** All positions in the pPhrase position list that are not sufficiently
116926
+** close to a position in the *paPoslist position list are removed. If this
116927
+** leaves 0 positions, zero is returned. Otherwise, non-zero.
116928
+**
116929
+** Before returning, *paPoslist is set to point to the position lsit
116930
+** associated with pPhrase. And *pnToken is set to the number of tokens in
116931
+** pPhrase.
116932
+*/
116933
+static int fts3EvalNearTrim(
116934
+ int nNear, /* NEAR distance. As in "NEAR/nNear". */
116935
+ char *aTmp, /* Temporary space to use */
116936
+ char **paPoslist, /* IN/OUT: Position list */
116937
+ int *pnToken, /* IN/OUT: Tokens in phrase of *paPoslist */
116938
+ Fts3Phrase *pPhrase /* The phrase object to trim the doclist of */
116939
+){
116940
+ int nParam1 = nNear + pPhrase->nToken;
116941
+ int nParam2 = nNear + *pnToken;
116942
+ int nNew;
116943
+ char *p2;
116944
+ char *pOut;
116945
+ int res;
116946
+
116947
+ assert( pPhrase->doclist.pList );
116948
+
116949
+ p2 = pOut = pPhrase->doclist.pList;
116950
+ res = fts3PoslistNearMerge(
116951
+ &pOut, aTmp, nParam1, nParam2, paPoslist, &p2
116952
+ );
116953
+ if( res ){
116954
+ nNew = (pOut - pPhrase->doclist.pList) - 1;
116955
+ assert( pPhrase->doclist.pList[nNew]=='\0' );
116956
+ assert( nNew<=pPhrase->doclist.nList && nNew>0 );
116957
+ memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew);
116958
+ pPhrase->doclist.nList = nNew;
116959
+ *paPoslist = pPhrase->doclist.pList;
116960
+ *pnToken = pPhrase->nToken;
116961
+ }
116962
+
116963
+ return res;
116964
+}
116965
+
116966
+/*
116967
+** This function is a no-op if *pRc is other than SQLITE_OK when it is called.
116968
+** Otherwise, it advances the expression passed as the second argument to
116969
+** point to the next matching row in the database. Expressions iterate through
116970
+** matching rows in docid order. Ascending order if Fts3Cursor.bDesc is zero,
116971
+** or descending if it is non-zero.
116972
+**
116973
+** If an error occurs, *pRc is set to an SQLite error code. Otherwise, if
116974
+** successful, the following variables in pExpr are set:
116975
+**
116976
+** Fts3Expr.bEof (non-zero if EOF - there is no next row)
116977
+** Fts3Expr.iDocid (valid if bEof==0. The docid of the next row)
116978
+**
116979
+** If the expression is of type FTSQUERY_PHRASE, and the expression is not
116980
+** at EOF, then the following variables are populated with the position list
116981
+** for the phrase for the visited row:
116982
+**
116983
+** FTs3Expr.pPhrase->doclist.nList (length of pList in bytes)
116984
+** FTs3Expr.pPhrase->doclist.pList (pointer to position list)
116985
+**
116986
+** It says above that this function advances the expression to the next
116987
+** matching row. This is usually true, but there are the following exceptions:
116988
+**
116989
+** 1. Deferred tokens are not taken into account. If a phrase consists
116990
+** entirely of deferred tokens, it is assumed to match every row in
116991
+** the db. In this case the position-list is not populated at all.
116992
+**
116993
+** Or, if a phrase contains one or more deferred tokens and one or
116994
+** more non-deferred tokens, then the expression is advanced to the
116995
+** next possible match, considering only non-deferred tokens. In other
116996
+** words, if the phrase is "A B C", and "B" is deferred, the expression
116997
+** is advanced to the next row that contains an instance of "A * C",
116998
+** where "*" may match any single token. The position list in this case
116999
+** is populated as for "A * C" before returning.
117000
+**
117001
+** 2. NEAR is treated as AND. If the expression is "x NEAR y", it is
117002
+** advanced to point to the next row that matches "x AND y".
117003
+**
117004
+** See fts3EvalTestDeferredAndNear() for details on testing if a row is
117005
+** really a match, taking into account deferred tokens and NEAR operators.
117006
+*/
117007
+static void fts3EvalNextRow(
117008
+ Fts3Cursor *pCsr, /* FTS Cursor handle */
117009
+ Fts3Expr *pExpr, /* Expr. to advance to next matching row */
117010
+ int *pRc /* IN/OUT: Error code */
117011
+){
117012
+ if( *pRc==SQLITE_OK ){
117013
+ int bDescDoclist = pCsr->bDesc; /* Used by DOCID_CMP() macro */
117014
+ assert( pExpr->bEof==0 );
117015
+ pExpr->bStart = 1;
117016
+
117017
+ switch( pExpr->eType ){
117018
+ case FTSQUERY_NEAR:
117019
+ case FTSQUERY_AND: {
117020
+ Fts3Expr *pLeft = pExpr->pLeft;
117021
+ Fts3Expr *pRight = pExpr->pRight;
117022
+ assert( !pLeft->bDeferred || !pRight->bDeferred );
117023
+
117024
+ if( pLeft->bDeferred ){
117025
+ /* LHS is entirely deferred. So we assume it matches every row.
117026
+ ** Advance the RHS iterator to find the next row visited. */
117027
+ fts3EvalNextRow(pCsr, pRight, pRc);
117028
+ pExpr->iDocid = pRight->iDocid;
117029
+ pExpr->bEof = pRight->bEof;
117030
+ }else if( pRight->bDeferred ){
117031
+ /* RHS is entirely deferred. So we assume it matches every row.
117032
+ ** Advance the LHS iterator to find the next row visited. */
117033
+ fts3EvalNextRow(pCsr, pLeft, pRc);
117034
+ pExpr->iDocid = pLeft->iDocid;
117035
+ pExpr->bEof = pLeft->bEof;
117036
+ }else{
117037
+ /* Neither the RHS or LHS are deferred. */
117038
+ fts3EvalNextRow(pCsr, pLeft, pRc);
117039
+ fts3EvalNextRow(pCsr, pRight, pRc);
117040
+ while( !pLeft->bEof && !pRight->bEof && *pRc==SQLITE_OK ){
117041
+ sqlite3_int64 iDiff = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
117042
+ if( iDiff==0 ) break;
117043
+ if( iDiff<0 ){
117044
+ fts3EvalNextRow(pCsr, pLeft, pRc);
117045
+ }else{
117046
+ fts3EvalNextRow(pCsr, pRight, pRc);
117047
+ }
117048
+ }
117049
+ pExpr->iDocid = pLeft->iDocid;
117050
+ pExpr->bEof = (pLeft->bEof || pRight->bEof);
117051
+ }
117052
+ break;
117053
+ }
117054
+
117055
+ case FTSQUERY_OR: {
117056
+ Fts3Expr *pLeft = pExpr->pLeft;
117057
+ Fts3Expr *pRight = pExpr->pRight;
117058
+ sqlite3_int64 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
117059
+
117060
+ assert( pLeft->bStart || pLeft->iDocid==pRight->iDocid );
117061
+ assert( pRight->bStart || pLeft->iDocid==pRight->iDocid );
117062
+
117063
+ if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){
117064
+ fts3EvalNextRow(pCsr, pLeft, pRc);
117065
+ }else if( pLeft->bEof || (pRight->bEof==0 && iCmp>0) ){
117066
+ fts3EvalNextRow(pCsr, pRight, pRc);
117067
+ }else{
117068
+ fts3EvalNextRow(pCsr, pLeft, pRc);
117069
+ fts3EvalNextRow(pCsr, pRight, pRc);
117070
+ }
117071
+
117072
+ pExpr->bEof = (pLeft->bEof && pRight->bEof);
117073
+ iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
117074
+ if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){
117075
+ pExpr->iDocid = pLeft->iDocid;
117076
+ }else{
117077
+ pExpr->iDocid = pRight->iDocid;
117078
+ }
117079
+
117080
+ break;
117081
+ }
117082
+
117083
+ case FTSQUERY_NOT: {
117084
+ Fts3Expr *pLeft = pExpr->pLeft;
117085
+ Fts3Expr *pRight = pExpr->pRight;
117086
+
117087
+ if( pRight->bStart==0 ){
117088
+ fts3EvalNextRow(pCsr, pRight, pRc);
117089
+ assert( *pRc!=SQLITE_OK || pRight->bStart );
117090
+ }
117091
+
117092
+ fts3EvalNextRow(pCsr, pLeft, pRc);
117093
+ if( pLeft->bEof==0 ){
117094
+ while( !*pRc
117095
+ && !pRight->bEof
117096
+ && DOCID_CMP(pLeft->iDocid, pRight->iDocid)>0
117097
+ ){
117098
+ fts3EvalNextRow(pCsr, pRight, pRc);
117099
+ }
117100
+ }
117101
+ pExpr->iDocid = pLeft->iDocid;
117102
+ pExpr->bEof = pLeft->bEof;
117103
+ break;
117104
+ }
117105
+
117106
+ default: {
117107
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
117108
+ fts3EvalInvalidatePoslist(pPhrase);
117109
+ *pRc = fts3EvalPhraseNext(pCsr, pPhrase, &pExpr->bEof);
117110
+ pExpr->iDocid = pPhrase->doclist.iDocid;
117111
+ break;
117112
+ }
117113
+ }
117114
+ }
117115
+}
117116
+
117117
+/*
117118
+** If *pRc is not SQLITE_OK, or if pExpr is not the root node of a NEAR
117119
+** cluster, then this function returns 1 immediately.
117120
+**
117121
+** Otherwise, it checks if the current row really does match the NEAR
117122
+** expression, using the data currently stored in the position lists
117123
+** (Fts3Expr->pPhrase.doclist.pList/nList) for each phrase in the expression.
117124
+**
117125
+** If the current row is a match, the position list associated with each
117126
+** phrase in the NEAR expression is edited in place to contain only those
117127
+** phrase instances sufficiently close to their peers to satisfy all NEAR
117128
+** constraints. In this case it returns 1. If the NEAR expression does not
117129
+** match the current row, 0 is returned. The position lists may or may not
117130
+** be edited if 0 is returned.
117131
+*/
117132
+static int fts3EvalNearTest(Fts3Expr *pExpr, int *pRc){
117133
+ int res = 1;
117134
+
117135
+ /* The following block runs if pExpr is the root of a NEAR query.
117136
+ ** For example, the query:
117137
+ **
117138
+ ** "w" NEAR "x" NEAR "y" NEAR "z"
117139
+ **
117140
+ ** which is represented in tree form as:
117141
+ **
117142
+ ** |
117143
+ ** +--NEAR--+ <-- root of NEAR query
117144
+ ** | |
117145
+ ** +--NEAR--+ "z"
117146
+ ** | |
117147
+ ** +--NEAR--+ "y"
117148
+ ** | |
117149
+ ** "w" "x"
117150
+ **
117151
+ ** The right-hand child of a NEAR node is always a phrase. The
117152
+ ** left-hand child may be either a phrase or a NEAR node. There are
117153
+ ** no exceptions to this - it's the way the parser in fts3_expr.c works.
117154
+ */
117155
+ if( *pRc==SQLITE_OK
117156
+ && pExpr->eType==FTSQUERY_NEAR
117157
+ && pExpr->bEof==0
117158
+ && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR)
117159
+ ){
117160
+ Fts3Expr *p;
117161
+ int nTmp = 0; /* Bytes of temp space */
117162
+ char *aTmp; /* Temp space for PoslistNearMerge() */
117163
+
117164
+ /* Allocate temporary working space. */
117165
+ for(p=pExpr; p->pLeft; p=p->pLeft){
117166
+ nTmp += p->pRight->pPhrase->doclist.nList;
117167
+ }
117168
+ nTmp += p->pPhrase->doclist.nList;
117169
+ aTmp = sqlite3_malloc(nTmp*2);
117170
+ if( !aTmp ){
117171
+ *pRc = SQLITE_NOMEM;
117172
+ res = 0;
117173
+ }else{
117174
+ char *aPoslist = p->pPhrase->doclist.pList;
117175
+ int nToken = p->pPhrase->nToken;
117176
+
117177
+ for(p=p->pParent;res && p && p->eType==FTSQUERY_NEAR; p=p->pParent){
117178
+ Fts3Phrase *pPhrase = p->pRight->pPhrase;
117179
+ int nNear = p->nNear;
117180
+ res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
117181
+ }
117182
+
117183
+ aPoslist = pExpr->pRight->pPhrase->doclist.pList;
117184
+ nToken = pExpr->pRight->pPhrase->nToken;
117185
+ for(p=pExpr->pLeft; p && res; p=p->pLeft){
117186
+ int nNear = p->pParent->nNear;
117187
+ Fts3Phrase *pPhrase = (
117188
+ p->eType==FTSQUERY_NEAR ? p->pRight->pPhrase : p->pPhrase
117189
+ );
117190
+ res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
117191
+ }
117192
+ }
117193
+
117194
+ sqlite3_free(aTmp);
117195
+ }
117196
+
117197
+ return res;
117198
+}
117199
+
117200
+/*
117201
+** This function is a helper function for fts3EvalTestDeferredAndNear().
117202
+** Assuming no error occurs or has occurred, It returns non-zero if the
117203
+** expression passed as the second argument matches the row that pCsr
117204
+** currently points to, or zero if it does not.
117205
+**
117206
+** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
117207
+** If an error occurs during execution of this function, *pRc is set to
117208
+** the appropriate SQLite error code. In this case the returned value is
117209
+** undefined.
117210
+*/
117211
+static int fts3EvalTestExpr(
117212
+ Fts3Cursor *pCsr, /* FTS cursor handle */
117213
+ Fts3Expr *pExpr, /* Expr to test. May or may not be root. */
117214
+ int *pRc /* IN/OUT: Error code */
117215
+){
117216
+ int bHit = 1; /* Return value */
117217
+ if( *pRc==SQLITE_OK ){
117218
+ switch( pExpr->eType ){
117219
+ case FTSQUERY_NEAR:
117220
+ case FTSQUERY_AND:
117221
+ bHit = (
117222
+ fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc)
117223
+ && fts3EvalTestExpr(pCsr, pExpr->pRight, pRc)
117224
+ && fts3EvalNearTest(pExpr, pRc)
117225
+ );
117226
+
117227
+ /* If the NEAR expression does not match any rows, zero the doclist for
117228
+ ** all phrases involved in the NEAR. This is because the snippet(),
117229
+ ** offsets() and matchinfo() functions are not supposed to recognize
117230
+ ** any instances of phrases that are part of unmatched NEAR queries.
117231
+ ** For example if this expression:
117232
+ **
117233
+ ** ... MATCH 'a OR (b NEAR c)'
117234
+ **
117235
+ ** is matched against a row containing:
117236
+ **
117237
+ ** 'a b d e'
117238
+ **
117239
+ ** then any snippet() should ony highlight the "a" term, not the "b"
117240
+ ** (as "b" is part of a non-matching NEAR clause).
117241
+ */
117242
+ if( bHit==0
117243
+ && pExpr->eType==FTSQUERY_NEAR
117244
+ && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR)
117245
+ ){
117246
+ Fts3Expr *p;
117247
+ for(p=pExpr; p->pPhrase==0; p=p->pLeft){
117248
+ if( p->pRight->iDocid==pCsr->iPrevId ){
117249
+ fts3EvalInvalidatePoslist(p->pRight->pPhrase);
117250
+ }
117251
+ }
117252
+ if( p->iDocid==pCsr->iPrevId ){
117253
+ fts3EvalInvalidatePoslist(p->pPhrase);
117254
+ }
117255
+ }
117256
+
117257
+ break;
117258
+
117259
+ case FTSQUERY_OR: {
117260
+ int bHit1 = fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc);
117261
+ int bHit2 = fts3EvalTestExpr(pCsr, pExpr->pRight, pRc);
117262
+ bHit = bHit1 || bHit2;
117263
+ break;
117264
+ }
117265
+
117266
+ case FTSQUERY_NOT:
117267
+ bHit = (
117268
+ fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc)
117269
+ && !fts3EvalTestExpr(pCsr, pExpr->pRight, pRc)
117270
+ );
117271
+ break;
117272
+
117273
+ default: {
117274
+ if( pCsr->pDeferred
117275
+ && (pExpr->iDocid==pCsr->iPrevId || pExpr->bDeferred)
117276
+ ){
117277
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
117278
+ assert( pExpr->bDeferred || pPhrase->doclist.bFreeList==0 );
117279
+ if( pExpr->bDeferred ){
117280
+ fts3EvalInvalidatePoslist(pPhrase);
117281
+ }
117282
+ *pRc = fts3EvalDeferredPhrase(pCsr, pPhrase);
117283
+ bHit = (pPhrase->doclist.pList!=0);
117284
+ pExpr->iDocid = pCsr->iPrevId;
117285
+ }else{
117286
+ bHit = (pExpr->bEof==0 && pExpr->iDocid==pCsr->iPrevId);
117287
+ }
117288
+ break;
117289
+ }
117290
+ }
117291
+ }
117292
+ return bHit;
117293
+}
117294
+
117295
+/*
117296
+** This function is called as the second part of each xNext operation when
117297
+** iterating through the results of a full-text query. At this point the
117298
+** cursor points to a row that matches the query expression, with the
117299
+** following caveats:
117300
+**
117301
+** * Up until this point, "NEAR" operators in the expression have been
117302
+** treated as "AND".
117303
+**
117304
+** * Deferred tokens have not yet been considered.
117305
+**
117306
+** If *pRc is not SQLITE_OK when this function is called, it immediately
117307
+** returns 0. Otherwise, it tests whether or not after considering NEAR
117308
+** operators and deferred tokens the current row is still a match for the
117309
+** expression. It returns 1 if both of the following are true:
117310
+**
117311
+** 1. *pRc is SQLITE_OK when this function returns, and
117312
+**
117313
+** 2. After scanning the current FTS table row for the deferred tokens,
117314
+** it is determined that the row does *not* match the query.
117315
+**
117316
+** Or, if no error occurs and it seems the current row does match the FTS
117317
+** query, return 0.
117318
+*/
117319
+static int fts3EvalTestDeferredAndNear(Fts3Cursor *pCsr, int *pRc){
117320
+ int rc = *pRc;
117321
+ int bMiss = 0;
117322
+ if( rc==SQLITE_OK ){
117323
+
117324
+ /* If there are one or more deferred tokens, load the current row into
117325
+ ** memory and scan it to determine the position list for each deferred
117326
+ ** token. Then, see if this row is really a match, considering deferred
117327
+ ** tokens and NEAR operators (neither of which were taken into account
117328
+ ** earlier, by fts3EvalNextRow()).
117329
+ */
117330
+ if( pCsr->pDeferred ){
117331
+ rc = fts3CursorSeek(0, pCsr);
117332
+ if( rc==SQLITE_OK ){
117333
+ rc = sqlite3Fts3CacheDeferredDoclists(pCsr);
117334
+ }
117335
+ }
117336
+ bMiss = (0==fts3EvalTestExpr(pCsr, pCsr->pExpr, &rc));
117337
+
117338
+ /* Free the position-lists accumulated for each deferred token above. */
117339
+ sqlite3Fts3FreeDeferredDoclists(pCsr);
117340
+ *pRc = rc;
117341
+ }
117342
+ return (rc==SQLITE_OK && bMiss);
117343
+}
117344
+
117345
+/*
117346
+** Advance to the next document that matches the FTS expression in
117347
+** Fts3Cursor.pExpr.
117348
+*/
117349
+static int fts3EvalNext(Fts3Cursor *pCsr){
117350
+ int rc = SQLITE_OK; /* Return Code */
117351
+ Fts3Expr *pExpr = pCsr->pExpr;
117352
+ assert( pCsr->isEof==0 );
117353
+ if( pExpr==0 ){
117354
+ pCsr->isEof = 1;
117355
+ }else{
117356
+ do {
117357
+ if( pCsr->isRequireSeek==0 ){
117358
+ sqlite3_reset(pCsr->pStmt);
117359
+ }
117360
+ assert( sqlite3_data_count(pCsr->pStmt)==0 );
117361
+ fts3EvalNextRow(pCsr, pExpr, &rc);
117362
+ pCsr->isEof = pExpr->bEof;
117363
+ pCsr->isRequireSeek = 1;
117364
+ pCsr->isMatchinfoNeeded = 1;
117365
+ pCsr->iPrevId = pExpr->iDocid;
117366
+ }while( pCsr->isEof==0 && fts3EvalTestDeferredAndNear(pCsr, &rc) );
117367
+ }
117368
+ return rc;
117369
+}
117370
+
117371
+/*
117372
+** Restart interation for expression pExpr so that the next call to
117373
+** fts3EvalNext() visits the first row. Do not allow incremental
117374
+** loading or merging of phrase doclists for this iteration.
117375
+**
117376
+** If *pRc is other than SQLITE_OK when this function is called, it is
117377
+** a no-op. If an error occurs within this function, *pRc is set to an
117378
+** SQLite error code before returning.
117379
+*/
117380
+static void fts3EvalRestart(
117381
+ Fts3Cursor *pCsr,
117382
+ Fts3Expr *pExpr,
117383
+ int *pRc
117384
+){
117385
+ if( pExpr && *pRc==SQLITE_OK ){
117386
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
117387
+
117388
+ if( pPhrase ){
117389
+ fts3EvalInvalidatePoslist(pPhrase);
117390
+ if( pPhrase->bIncr ){
117391
+ assert( pPhrase->nToken==1 );
117392
+ assert( pPhrase->aToken[0].pSegcsr );
117393
+ sqlite3Fts3MsrIncrRestart(pPhrase->aToken[0].pSegcsr);
117394
+ *pRc = fts3EvalPhraseStart(pCsr, 0, pPhrase);
117395
+ }
117396
+
117397
+ pPhrase->doclist.pNextDocid = 0;
117398
+ pPhrase->doclist.iDocid = 0;
117399
+ }
117400
+
117401
+ pExpr->iDocid = 0;
117402
+ pExpr->bEof = 0;
117403
+ pExpr->bStart = 0;
117404
+
117405
+ fts3EvalRestart(pCsr, pExpr->pLeft, pRc);
117406
+ fts3EvalRestart(pCsr, pExpr->pRight, pRc);
117407
+ }
117408
+}
117409
+
117410
+/*
117411
+** After allocating the Fts3Expr.aMI[] array for each phrase in the
117412
+** expression rooted at pExpr, the cursor iterates through all rows matched
117413
+** by pExpr, calling this function for each row. This function increments
117414
+** the values in Fts3Expr.aMI[] according to the position-list currently
117415
+** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase
117416
+** expression nodes.
117417
+*/
117418
+static void fts3EvalUpdateCounts(Fts3Expr *pExpr){
117419
+ if( pExpr ){
117420
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
117421
+ if( pPhrase && pPhrase->doclist.pList ){
117422
+ int iCol = 0;
117423
+ char *p = pPhrase->doclist.pList;
117424
+
117425
+ assert( *p );
117426
+ while( 1 ){
117427
+ u8 c = 0;
117428
+ int iCnt = 0;
117429
+ while( 0xFE & (*p | c) ){
117430
+ if( (c&0x80)==0 ) iCnt++;
117431
+ c = *p++ & 0x80;
117432
+ }
117433
+
117434
+ /* aMI[iCol*3 + 1] = Number of occurrences
117435
+ ** aMI[iCol*3 + 2] = Number of rows containing at least one instance
117436
+ */
117437
+ pExpr->aMI[iCol*3 + 1] += iCnt;
117438
+ pExpr->aMI[iCol*3 + 2] += (iCnt>0);
117439
+ if( *p==0x00 ) break;
117440
+ p++;
117441
+ p += sqlite3Fts3GetVarint32(p, &iCol);
117442
+ }
117443
+ }
117444
+
117445
+ fts3EvalUpdateCounts(pExpr->pLeft);
117446
+ fts3EvalUpdateCounts(pExpr->pRight);
117447
+ }
117448
+}
117449
+
117450
+/*
117451
+** Expression pExpr must be of type FTSQUERY_PHRASE.
117452
+**
117453
+** If it is not already allocated and populated, this function allocates and
117454
+** populates the Fts3Expr.aMI[] array for expression pExpr. If pExpr is part
117455
+** of a NEAR expression, then it also allocates and populates the same array
117456
+** for all other phrases that are part of the NEAR expression.
117457
+**
117458
+** SQLITE_OK is returned if the aMI[] array is successfully allocated and
117459
+** populated. Otherwise, if an error occurs, an SQLite error code is returned.
117460
+*/
117461
+static int fts3EvalGatherStats(
117462
+ Fts3Cursor *pCsr, /* Cursor object */
117463
+ Fts3Expr *pExpr /* FTSQUERY_PHRASE expression */
117464
+){
117465
+ int rc = SQLITE_OK; /* Return code */
117466
+
117467
+ assert( pExpr->eType==FTSQUERY_PHRASE );
117468
+ if( pExpr->aMI==0 ){
117469
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
117470
+ Fts3Expr *pRoot; /* Root of NEAR expression */
117471
+ Fts3Expr *p; /* Iterator used for several purposes */
117472
+
117473
+ sqlite3_int64 iPrevId = pCsr->iPrevId;
117474
+ sqlite3_int64 iDocid;
117475
+ u8 bEof;
117476
+
117477
+ /* Find the root of the NEAR expression */
117478
+ pRoot = pExpr;
117479
+ while( pRoot->pParent && pRoot->pParent->eType==FTSQUERY_NEAR ){
117480
+ pRoot = pRoot->pParent;
117481
+ }
117482
+ iDocid = pRoot->iDocid;
117483
+ bEof = pRoot->bEof;
117484
+ assert( pRoot->bStart );
117485
+
117486
+ /* Allocate space for the aMSI[] array of each FTSQUERY_PHRASE node */
117487
+ for(p=pRoot; p; p=p->pLeft){
117488
+ Fts3Expr *pE = (p->eType==FTSQUERY_PHRASE?p:p->pRight);
117489
+ assert( pE->aMI==0 );
117490
+ pE->aMI = (u32 *)sqlite3_malloc(pTab->nColumn * 3 * sizeof(u32));
117491
+ if( !pE->aMI ) return SQLITE_NOMEM;
117492
+ memset(pE->aMI, 0, pTab->nColumn * 3 * sizeof(u32));
117493
+ }
117494
+
117495
+ fts3EvalRestart(pCsr, pRoot, &rc);
117496
+
117497
+ while( pCsr->isEof==0 && rc==SQLITE_OK ){
117498
+
117499
+ do {
117500
+ /* Ensure the %_content statement is reset. */
117501
+ if( pCsr->isRequireSeek==0 ) sqlite3_reset(pCsr->pStmt);
117502
+ assert( sqlite3_data_count(pCsr->pStmt)==0 );
117503
+
117504
+ /* Advance to the next document */
117505
+ fts3EvalNextRow(pCsr, pRoot, &rc);
117506
+ pCsr->isEof = pRoot->bEof;
117507
+ pCsr->isRequireSeek = 1;
117508
+ pCsr->isMatchinfoNeeded = 1;
117509
+ pCsr->iPrevId = pRoot->iDocid;
117510
+ }while( pCsr->isEof==0
117511
+ && pRoot->eType==FTSQUERY_NEAR
117512
+ && fts3EvalTestDeferredAndNear(pCsr, &rc)
117513
+ );
117514
+
117515
+ if( rc==SQLITE_OK && pCsr->isEof==0 ){
117516
+ fts3EvalUpdateCounts(pRoot);
117517
+ }
117518
+ }
117519
+
117520
+ pCsr->isEof = 0;
117521
+ pCsr->iPrevId = iPrevId;
117522
+
117523
+ if( bEof ){
117524
+ pRoot->bEof = bEof;
117525
+ }else{
117526
+ /* Caution: pRoot may iterate through docids in ascending or descending
117527
+ ** order. For this reason, even though it seems more defensive, the
117528
+ ** do loop can not be written:
117529
+ **
117530
+ ** do {...} while( pRoot->iDocid<iDocid && rc==SQLITE_OK );
117531
+ */
117532
+ fts3EvalRestart(pCsr, pRoot, &rc);
117533
+ do {
117534
+ fts3EvalNextRow(pCsr, pRoot, &rc);
117535
+ assert( pRoot->bEof==0 );
117536
+ }while( pRoot->iDocid!=iDocid && rc==SQLITE_OK );
117537
+ fts3EvalTestDeferredAndNear(pCsr, &rc);
117538
+ }
117539
+ }
117540
+ return rc;
117541
+}
117542
+
117543
+/*
117544
+** This function is used by the matchinfo() module to query a phrase
117545
+** expression node for the following information:
117546
+**
117547
+** 1. The total number of occurrences of the phrase in each column of
117548
+** the FTS table (considering all rows), and
117549
+**
117550
+** 2. For each column, the number of rows in the table for which the
117551
+** column contains at least one instance of the phrase.
117552
+**
117553
+** If no error occurs, SQLITE_OK is returned and the values for each column
117554
+** written into the array aiOut as follows:
117555
+**
117556
+** aiOut[iCol*3 + 1] = Number of occurrences
117557
+** aiOut[iCol*3 + 2] = Number of rows containing at least one instance
117558
+**
117559
+** Caveats:
117560
+**
117561
+** * If a phrase consists entirely of deferred tokens, then all output
117562
+** values are set to the number of documents in the table. In other
117563
+** words we assume that very common tokens occur exactly once in each
117564
+** column of each row of the table.
117565
+**
117566
+** * If a phrase contains some deferred tokens (and some non-deferred
117567
+** tokens), count the potential occurrence identified by considering
117568
+** the non-deferred tokens instead of actual phrase occurrences.
117569
+**
117570
+** * If the phrase is part of a NEAR expression, then only phrase instances
117571
+** that meet the NEAR constraint are included in the counts.
117572
+*/
117573
+SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats(
117574
+ Fts3Cursor *pCsr, /* FTS cursor handle */
117575
+ Fts3Expr *pExpr, /* Phrase expression */
117576
+ u32 *aiOut /* Array to write results into (see above) */
117577
+){
117578
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
117579
+ int rc = SQLITE_OK;
117580
+ int iCol;
117581
+
117582
+ if( pExpr->bDeferred && pExpr->pParent->eType!=FTSQUERY_NEAR ){
117583
+ assert( pCsr->nDoc>0 );
117584
+ for(iCol=0; iCol<pTab->nColumn; iCol++){
117585
+ aiOut[iCol*3 + 1] = (u32)pCsr->nDoc;
117586
+ aiOut[iCol*3 + 2] = (u32)pCsr->nDoc;
117587
+ }
117588
+ }else{
117589
+ rc = fts3EvalGatherStats(pCsr, pExpr);
117590
+ if( rc==SQLITE_OK ){
117591
+ assert( pExpr->aMI );
117592
+ for(iCol=0; iCol<pTab->nColumn; iCol++){
117593
+ aiOut[iCol*3 + 1] = pExpr->aMI[iCol*3 + 1];
117594
+ aiOut[iCol*3 + 2] = pExpr->aMI[iCol*3 + 2];
117595
+ }
117596
+ }
117597
+ }
117598
+
117599
+ return rc;
117600
+}
117601
+
117602
+/*
117603
+** The expression pExpr passed as the second argument to this function
117604
+** must be of type FTSQUERY_PHRASE.
117605
+**
117606
+** The returned value is either NULL or a pointer to a buffer containing
117607
+** a position-list indicating the occurrences of the phrase in column iCol
117608
+** of the current row.
117609
+**
117610
+** More specifically, the returned buffer contains 1 varint for each
117611
+** occurence of the phrase in the column, stored using the normal (delta+2)
117612
+** compression and is terminated by either an 0x01 or 0x00 byte. For example,
117613
+** if the requested column contains "a b X c d X X" and the position-list
117614
+** for 'X' is requested, the buffer returned may contain:
117615
+**
117616
+** 0x04 0x05 0x03 0x01 or 0x04 0x05 0x03 0x00
117617
+**
117618
+** This function works regardless of whether or not the phrase is deferred,
117619
+** incremental, or neither.
117620
+*/
117621
+SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist(
117622
+ Fts3Cursor *pCsr, /* FTS3 cursor object */
117623
+ Fts3Expr *pExpr, /* Phrase to return doclist for */
117624
+ int iCol /* Column to return position list for */
117625
+){
117626
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
117627
+ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
117628
+ char *pIter = pPhrase->doclist.pList;
117629
+ int iThis;
117630
+
117631
+ assert( iCol>=0 && iCol<pTab->nColumn );
117632
+ if( !pIter
117633
+ || pExpr->bEof
117634
+ || pExpr->iDocid!=pCsr->iPrevId
117635
+ || (pPhrase->iColumn<pTab->nColumn && pPhrase->iColumn!=iCol)
117636
+ ){
117637
+ return 0;
117638
+ }
117639
+
117640
+ assert( pPhrase->doclist.nList>0 );
117641
+ if( *pIter==0x01 ){
117642
+ pIter++;
117643
+ pIter += sqlite3Fts3GetVarint32(pIter, &iThis);
117644
+ }else{
117645
+ iThis = 0;
117646
+ }
117647
+ while( iThis<iCol ){
117648
+ fts3ColumnlistCopy(0, &pIter);
117649
+ if( *pIter==0x00 ) return 0;
117650
+ pIter++;
117651
+ pIter += sqlite3Fts3GetVarint32(pIter, &iThis);
117652
+ }
117653
+
117654
+ return ((iCol==iThis)?pIter:0);
117655
+}
117656
+
117657
+/*
117658
+** Free all components of the Fts3Phrase structure that were allocated by
117659
+** the eval module. Specifically, this means to free:
117660
+**
117661
+** * the contents of pPhrase->doclist, and
117662
+** * any Fts3MultiSegReader objects held by phrase tokens.
117663
+*/
117664
+SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *pPhrase){
117665
+ if( pPhrase ){
117666
+ int i;
117667
+ sqlite3_free(pPhrase->doclist.aAll);
117668
+ fts3EvalInvalidatePoslist(pPhrase);
117669
+ memset(&pPhrase->doclist, 0, sizeof(Fts3Doclist));
117670
+ for(i=0; i<pPhrase->nToken; i++){
117671
+ fts3SegReaderCursorFree(pPhrase->aToken[i].pSegcsr);
117672
+ pPhrase->aToken[i].pSegcsr = 0;
117673
+ }
117674
+ }
117675
+}
115610117676
115611117677
 #if !SQLITE_CORE
117678
+/*
117679
+** Initialize API pointer table, if required.
117680
+*/
115612117681
 SQLITE_API int sqlite3_extension_init(
115613117682
 sqlite3 *db,
115614117683
 char **pzErrMsg,
115615117684
 const sqlite3_api_routines *pApi
115616117685
 ){
@@ -115634,11 +117703,10 @@
115634117703
 ** May you share freely, never taking more than you give.
115635117704
 **
115636117705
 ******************************************************************************
115637117706
 **
115638117707
 */
115639
-
115640117708
 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
115641117709
115642117710
115643117711
 typedef struct Fts3auxTable Fts3auxTable;
115644117712
 typedef struct Fts3auxCursor Fts3auxCursor;
@@ -115648,11 +117716,11 @@
115648117716
 Fts3Table *pFts3Tab;
115649117717
 };
115650117718
115651117719
 struct Fts3auxCursor {
115652117720
 sqlite3_vtab_cursor base; /* Base class used by SQLite core */
115653
- Fts3SegReaderCursor csr; /* Must be right after "base" */
117721
+ Fts3MultiSegReader csr; /* Must be right after "base" */
115654117722
 Fts3SegFilter filter;
115655117723
 char *zStop;
115656117724
 int nStop; /* Byte-length of string zStop */
115657117725
 int isEof; /* True if cursor is at EOF */
115658117726
 sqlite3_int64 iRowid; /* Current rowid */
@@ -115716,10 +117784,11 @@
115716117784
115717117785
 p->pFts3Tab = (Fts3Table *)&p[1];
115718117786
 p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1];
115719117787
 p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1];
115720117788
 p->pFts3Tab->db = db;
117789
+ p->pFts3Tab->nIndex = 1;
115721117790
115722117791
 memcpy((char *)p->pFts3Tab->zDb, zDb, nDb);
115723117792
 memcpy((char *)p->pFts3Tab->zName, zFts3, nFts3);
115724117793
 sqlite3Fts3Dequote((char *)p->pFts3Tab->zName);
115725117794
@@ -115996,11 +118065,11 @@
115996118065
 pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iIdx]));
115997118066
 pCsr->nStop = sqlite3_value_bytes(apVal[iIdx]);
115998118067
 if( pCsr->zStop==0 ) return SQLITE_NOMEM;
115999118068
 }
116000118069
116001
- rc = sqlite3Fts3SegReaderCursor(pFts3, FTS3_SEGCURSOR_ALL,
118070
+ rc = sqlite3Fts3SegReaderCursor(pFts3, 0, FTS3_SEGCURSOR_ALL,
116002118071
 pCsr->filter.zTerm, pCsr->filter.nTerm, 0, isScan, &pCsr->csr
116003118072
 );
116004118073
 if( rc==SQLITE_OK ){
116005118074
 rc = sqlite3Fts3SegReaderStart(pFts3, &pCsr->csr, &pCsr->filter);
116006118075
 }
@@ -116175,16 +118244,25 @@
116175118244
 ** Default span for NEAR operators.
116176118245
 */
116177118246
 #define SQLITE_FTS3_DEFAULT_NEAR_PARAM 10
116178118247
116179118248
118249
+/*
118250
+** isNot:
118251
+** This variable is used by function getNextNode(). When getNextNode() is
118252
+** called, it sets ParseContext.isNot to true if the 'next node' is a
118253
+** FTSQUERY_PHRASE with a unary "-" attached to it. i.e. "mysql" in the
118254
+** FTS3 query "sqlite -mysql". Otherwise, ParseContext.isNot is set to
118255
+** zero.
118256
+*/
116180118257
 typedef struct ParseContext ParseContext;
116181118258
 struct ParseContext {
116182118259
 sqlite3_tokenizer *pTokenizer; /* Tokenizer module */
116183118260
 const char **azCol; /* Array of column names for fts3 table */
116184118261
 int nCol; /* Number of entries in azCol[] */
116185118262
 int iDefaultCol; /* Default column to query */
118263
+ int isNot; /* True if getNextNode() sees a unary - */
116186118264
 sqlite3_context *pCtx; /* Write error message here */
116187118265
 int nNest; /* Number of nested brackets */
116188118266
 };
116189118267
116190118268
 /*
@@ -116266,11 +118344,11 @@
116266118344
 if( iEnd<n && z[iEnd]=='*' ){
116267118345
 pRet->pPhrase->aToken[0].isPrefix = 1;
116268118346
 iEnd++;
116269118347
 }
116270118348
 if( !sqlite3_fts3_enable_parentheses && iStart>0 && z[iStart-1]=='-' ){
116271
- pRet->pPhrase->isNot = 1;
118349
+ pParse->isNot = 1;
116272118350
 }
116273118351
 }
116274118352
 nConsumed = iEnd;
116275118353
 }
116276118354
@@ -116318,71 +118396,86 @@
116318118396
 Fts3Expr *p = 0;
116319118397
 sqlite3_tokenizer_cursor *pCursor = 0;
116320118398
 char *zTemp = 0;
116321118399
 int nTemp = 0;
116322118400
118401
+ const int nSpace = sizeof(Fts3Expr) + sizeof(Fts3Phrase);
118402
+ int nToken = 0;
118403
+
118404
+ /* The final Fts3Expr data structure, including the Fts3Phrase,
118405
+ ** Fts3PhraseToken structures token buffers are all stored as a single
118406
+ ** allocation so that the expression can be freed with a single call to
118407
+ ** sqlite3_free(). Setting this up requires a two pass approach.
118408
+ **
118409
+ ** The first pass, in the block below, uses a tokenizer cursor to iterate
118410
+ ** through the tokens in the expression. This pass uses fts3ReallocOrFree()
118411
+ ** to assemble data in two dynamic buffers:
118412
+ **
118413
+ ** Buffer p: Points to the Fts3Expr structure, followed by the Fts3Phrase
118414
+ ** structure, followed by the array of Fts3PhraseToken
118415
+ ** structures. This pass only populates the Fts3PhraseToken array.
118416
+ **
118417
+ ** Buffer zTemp: Contains copies of all tokens.
118418
+ **
118419
+ ** The second pass, in the block that begins "if( rc==SQLITE_DONE )" below,
118420
+ ** appends buffer zTemp to buffer p, and fills in the Fts3Expr and Fts3Phrase
118421
+ ** structures.
118422
+ */
116323118423
 rc = pModule->xOpen(pTokenizer, zInput, nInput, &pCursor);
116324118424
 if( rc==SQLITE_OK ){
116325118425
 int ii;
116326118426
 pCursor->pTokenizer = pTokenizer;
116327118427
 for(ii=0; rc==SQLITE_OK; ii++){
116328
- const char *zToken;
116329
- int nToken, iBegin, iEnd, iPos;
116330
- rc = pModule->xNext(pCursor, &zToken, &nToken, &iBegin, &iEnd, &iPos);
118428
+ const char *zByte;
118429
+ int nByte, iBegin, iEnd, iPos;
118430
+ rc = pModule->xNext(pCursor, &zByte, &nByte, &iBegin, &iEnd, &iPos);
116331118431
 if( rc==SQLITE_OK ){
116332
- int nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase);
116333
- p = fts3ReallocOrFree(p, nByte+ii*sizeof(Fts3PhraseToken));
116334
- zTemp = fts3ReallocOrFree(zTemp, nTemp + nToken);
116335
- if( !p || !zTemp ){
116336
- goto no_mem;
116337
- }
116338
- if( ii==0 ){
116339
- memset(p, 0, nByte);
116340
- p->pPhrase = (Fts3Phrase *)&p[1];
116341
- }
116342
- p->pPhrase = (Fts3Phrase *)&p[1];
116343
- memset(&p->pPhrase->aToken[ii], 0, sizeof(Fts3PhraseToken));
116344
- p->pPhrase->nToken = ii+1;
116345
- p->pPhrase->aToken[ii].n = nToken;
116346
- memcpy(&zTemp[nTemp], zToken, nToken);
116347
- nTemp += nToken;
116348
- if( iEnd<nInput && zInput[iEnd]=='*' ){
116349
- p->pPhrase->aToken[ii].isPrefix = 1;
116350
- }else{
116351
- p->pPhrase->aToken[ii].isPrefix = 0;
116352
- }
118432
+ Fts3PhraseToken *pToken;
118433
+
118434
+ p = fts3ReallocOrFree(p, nSpace + ii*sizeof(Fts3PhraseToken));
118435
+ if( !p ) goto no_mem;
118436
+
118437
+ zTemp = fts3ReallocOrFree(zTemp, nTemp + nByte);
118438
+ if( !zTemp ) goto no_mem;
118439
+
118440
+ assert( nToken==ii );
118441
+ pToken = &((Fts3Phrase *)(&p[1]))->aToken[ii];
118442
+ memset(pToken, 0, sizeof(Fts3PhraseToken));
118443
+
118444
+ memcpy(&zTemp[nTemp], zByte, nByte);
118445
+ nTemp += nByte;
118446
+
118447
+ pToken->n = nByte;
118448
+ pToken->isPrefix = (iEnd<nInput && zInput[iEnd]=='*');
118449
+ nToken = ii+1;
116353118450
 }
116354118451
 }
116355118452
116356118453
 pModule->xClose(pCursor);
116357118454
 pCursor = 0;
116358118455
 }
116359118456
116360118457
 if( rc==SQLITE_DONE ){
116361118458
 int jj;
116362
- char *zNew = NULL;
116363
- int nNew = 0;
116364
- int nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase);
116365
- nByte += (p?(p->pPhrase->nToken-1):0) * sizeof(Fts3PhraseToken);
116366
- p = fts3ReallocOrFree(p, nByte + nTemp);
116367
- if( !p ){
116368
- goto no_mem;
116369
- }
116370
- if( zTemp ){
116371
- zNew = &(((char *)p)[nByte]);
116372
- memcpy(zNew, zTemp, nTemp);
116373
- }else{
116374
- memset(p, 0, nByte+nTemp);
116375
- }
118459
+ char *zBuf = 0;
118460
+
118461
+ p = fts3ReallocOrFree(p, nSpace + nToken*sizeof(Fts3PhraseToken) + nTemp);
118462
+ if( !p ) goto no_mem;
118463
+ memset(p, 0, (char *)&(((Fts3Phrase *)&p[1])->aToken[0])-(char *)p);
118464
+ p->eType = FTSQUERY_PHRASE;
116376118465
 p->pPhrase = (Fts3Phrase *)&p[1];
118466
+ p->pPhrase->iColumn = pParse->iDefaultCol;
118467
+ p->pPhrase->nToken = nToken;
118468
+
118469
+ zBuf = (char *)&p->pPhrase->aToken[nToken];
118470
+ memcpy(zBuf, zTemp, nTemp);
118471
+ sqlite3_free(zTemp);
118472
+
116377118473
 for(jj=0; jj<p->pPhrase->nToken; jj++){
116378
- p->pPhrase->aToken[jj].z = &zNew[nNew];
116379
- nNew += p->pPhrase->aToken[jj].n;
118474
+ p->pPhrase->aToken[jj].z = zBuf;
118475
+ zBuf += p->pPhrase->aToken[jj].n;
116380118476
 }
116381
- sqlite3_free(zTemp);
116382
- p->eType = FTSQUERY_PHRASE;
116383
- p->pPhrase->iColumn = pParse->iDefaultCol;
116384118477
 rc = SQLITE_OK;
116385118478
 }
116386118479
116387118480
 *ppExpr = p;
116388118481
 return rc;
@@ -116434,10 +118527,12 @@
116434118527
 int rc;
116435118528
 Fts3Expr *pRet = 0;
116436118529
116437118530
 const char *zInput = z;
116438118531
 int nInput = n;
118532
+
118533
+ pParse->isNot = 0;
116439118534
116440118535
 /* Skip over any whitespace before checking for a keyword, an open or
116441118536
 ** close bracket, or a quoted string.
116442118537
 */
116443118538
 while( nInput>0 && fts3isspace(*zInput) ){
@@ -116653,11 +118748,11 @@
116653118748
 rc = getNextNode(pParse, zIn, nIn, &p, &nByte);
116654118749
 if( rc==SQLITE_OK ){
116655118750
 int isPhrase;
116656118751
116657118752
 if( !sqlite3_fts3_enable_parentheses
116658
- && p->eType==FTSQUERY_PHRASE && p->pPhrase->isNot
118753
+ && p->eType==FTSQUERY_PHRASE && pParse->isNot
116659118754
 ){
116660118755
 /* Create an implicit NOT operator. */
116661118756
 Fts3Expr *pNot = fts3MallocZero(sizeof(Fts3Expr));
116662118757
 if( !pNot ){
116663118758
 sqlite3Fts3ExprFree(p);
@@ -116671,11 +118766,10 @@
116671118766
 }
116672118767
 pNotBranch = pNot;
116673118768
 p = pPrev;
116674118769
 }else{
116675118770
 int eType = p->eType;
116676
- assert( eType!=FTSQUERY_PHRASE || !p->pPhrase->isNot );
116677118771
 isPhrase = (eType==FTSQUERY_PHRASE || p->pLeft);
116678118772
116679118773
 /* The isRequirePhrase variable is set to true if a phrase or
116680118774
 ** an expression contained in parenthesis is required. If a
116681118775
 ** binary operator (AND, OR, NOT or NEAR) is encounted when
@@ -116834,13 +118928,15 @@
116834118928
 /*
116835118929
 ** Free a parsed fts3 query expression allocated by sqlite3Fts3ExprParse().
116836118930
 */
116837118931
 SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *p){
116838118932
 if( p ){
118933
+ assert( p->eType==FTSQUERY_PHRASE || p->pPhrase==0 );
116839118934
 sqlite3Fts3ExprFree(p->pLeft);
116840118935
 sqlite3Fts3ExprFree(p->pRight);
116841
- sqlite3_free(p->aDoclist);
118936
+ sqlite3Fts3EvalPhraseCleanup(p->pPhrase);
118937
+ sqlite3_free(p->aMI);
116842118938
 sqlite3_free(p);
116843118939
 }
116844118940
 }
116845118941
116846118942
 /****************************************************************************
@@ -116893,11 +118989,11 @@
116893118989
 switch( pExpr->eType ){
116894118990
 case FTSQUERY_PHRASE: {
116895118991
 Fts3Phrase *pPhrase = pExpr->pPhrase;
116896118992
 int i;
116897118993
 zBuf = sqlite3_mprintf(
116898
- "%zPHRASE %d %d", zBuf, pPhrase->iColumn, pPhrase->isNot);
118994
+ "%zPHRASE %d 0", zBuf, pPhrase->iColumn);
116899118995
 for(i=0; zBuf && i<pPhrase->nToken; i++){
116900118996
 zBuf = sqlite3_mprintf("%z %.*s%s", zBuf,
116901118997
 pPhrase->aToken[i].n, pPhrase->aToken[i].z,
116902118998
 (pPhrase->aToken[i].isPrefix?"+":"")
116903118999
 );
@@ -117437,11 +119533,10 @@
117437119533
 **
117438119534
 ** * The FTS3 module is being built into the core of
117439119535
 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
117440119536
 */
117441119537
 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
117442
-
117443119538
117444119539
117445119540
117446119541
 /*
117447119542
 ** Class derived from sqlite3_tokenizer
@@ -118082,14 +120177,10 @@
118082120177
 ** * The FTS3 module is being built into the core of
118083120178
 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
118084120179
 */
118085120180
 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
118086120181
118087
-#ifndef SQLITE_CORE
118088
- SQLITE_EXTENSION_INIT1
118089
-#endif
118090
-
118091120182
118092120183
 /*
118093120184
 ** Implementation of the SQL scalar function for accessing the underlying
118094120185
 ** hash table. This function may be called as follows:
118095120186
 **
@@ -118209,11 +120300,11 @@
118209120300
 sqlite3_tokenizer **ppTok, /* OUT: Tokenizer (if applicable) */
118210120301
 char **pzErr /* OUT: Set to malloced error message */
118211120302
 ){
118212120303
 int rc;
118213120304
 char *z = (char *)zArg;
118214
- int n;
120305
+ int n = 0;
118215120306
 char *zCopy;
118216120307
 char *zEnd; /* Pointer to nul-term of zCopy */
118217120308
 sqlite3_tokenizer_module *m;
118218120309
118219120310
 zCopy = sqlite3_mprintf("%s", zArg);
@@ -118574,11 +120665,10 @@
118574120665
 */
118575120666
 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
118576120667
118577120668
118578120669
118579
-
118580120670
 typedef struct simple_tokenizer {
118581120671
 sqlite3_tokenizer base;
118582120672
 char delim[128]; /* flag ASCII delimiters */
118583120673
 } simple_tokenizer;
118584120674
@@ -118811,18 +120901,44 @@
118811120901
 ** it is always safe to read up to two varints from it without risking an
118812120902
 ** overread, even if the node data is corrupted.
118813120903
 */
118814120904
 #define FTS3_NODE_PADDING (FTS3_VARINT_MAX*2)
118815120905
120906
+/*
120907
+** Under certain circumstances, b-tree nodes (doclists) can be loaded into
120908
+** memory incrementally instead of all at once. This can be a big performance
120909
+** win (reduced IO and CPU) if SQLite stops calling the virtual table xNext()
120910
+** method before retrieving all query results (as may happen, for example,
120911
+** if a query has a LIMIT clause).
120912
+**
120913
+** Incremental loading is used for b-tree nodes FTS3_NODE_CHUNK_THRESHOLD
120914
+** bytes and larger. Nodes are loaded in chunks of FTS3_NODE_CHUNKSIZE bytes.
120915
+** The code is written so that the hard lower-limit for each of these values
120916
+** is 1. Clearly such small values would be inefficient, but can be useful
120917
+** for testing purposes.
120918
+**
120919
+** If this module is built with SQLITE_TEST defined, these constants may
120920
+** be overridden at runtime for testing purposes. File fts3_test.c contains
120921
+** a Tcl interface to read and write the values.
120922
+*/
120923
+#ifdef SQLITE_TEST
120924
+int test_fts3_node_chunksize = (4*1024);
120925
+int test_fts3_node_chunk_threshold = (4*1024)*4;
120926
+# define FTS3_NODE_CHUNKSIZE test_fts3_node_chunksize
120927
+# define FTS3_NODE_CHUNK_THRESHOLD test_fts3_node_chunk_threshold
120928
+#else
120929
+# define FTS3_NODE_CHUNKSIZE (4*1024)
120930
+# define FTS3_NODE_CHUNK_THRESHOLD (FTS3_NODE_CHUNKSIZE*4)
120931
+#endif
120932
+
118816120933
 typedef struct PendingList PendingList;
118817120934
 typedef struct SegmentNode SegmentNode;
118818120935
 typedef struct SegmentWriter SegmentWriter;
118819120936
118820120937
 /*
118821
-** Data structure used while accumulating terms in the pending-terms hash
118822
-** table. The hash table entry maps from term (a string) to a malloc'd
118823
-** instance of this structure.
120938
+** An instance of the following data structure is used to build doclists
120939
+** incrementally. See function fts3PendingListAppend() for details.
118824120940
 */
118825120941
 struct PendingList {
118826120942
 int nData;
118827120943
 char *aData;
118828120944
 int nSpace;
@@ -118849,11 +120965,10 @@
118849120965
 ** of type Fts3SegReader* are also used by code in fts3.c to iterate through
118850120966
 ** terms when querying the full-text index. See functions:
118851120967
 **
118852120968
 ** sqlite3Fts3SegReaderNew()
118853120969
 ** sqlite3Fts3SegReaderFree()
118854
-** sqlite3Fts3SegReaderCost()
118855120970
 ** sqlite3Fts3SegReaderIterate()
118856120971
 **
118857120972
 ** Methods used to manipulate Fts3SegReader structures:
118858120973
 **
118859120974
 ** fts3SegReaderNext()
@@ -118868,10 +120983,13 @@
118868120983
 sqlite3_int64 iEndBlock; /* Rowid of final block in segment (or 0) */
118869120984
 sqlite3_int64 iCurrentBlock; /* Current leaf block (or 0) */
118870120985
118871120986
 char *aNode; /* Pointer to node data (or NULL) */
118872120987
 int nNode; /* Size of buffer at aNode (or 0) */
120988
+ int nPopulate; /* If >0, bytes of buffer aNode[] loaded */
120989
+ sqlite3_blob *pBlob; /* If not NULL, blob handle to read node */
120990
+
118873120991
 Fts3HashElem **ppNextElem;
118874120992
118875120993
 /* Variables set by fts3SegReaderNext(). These may be read directly
118876120994
 ** by the caller. They are valid from the time SegmentReaderNew() returns
118877120995
 ** until SegmentReaderNext() returns something other than SQLITE_OK
@@ -118881,12 +120999,15 @@
118881120999
 char *zTerm; /* Pointer to current term */
118882121000
 int nTermAlloc; /* Allocated size of zTerm buffer */
118883121001
 char *aDoclist; /* Pointer to doclist of current entry */
118884121002
 int nDoclist; /* Size of doclist in current entry */
118885121003
118886
- /* The following variables are used to iterate through the current doclist */
121004
+ /* The following variables are used by fts3SegReaderNextDocid() to iterate
121005
+ ** through the current doclist (aDoclist/nDoclist).
121006
+ */
118887121007
 char *pOffsetList;
121008
+ int nOffsetList; /* For descending pending seg-readers only */
118888121009
 sqlite3_int64 iDocid;
118889121010
 };
118890121011
118891121012
 #define fts3SegReaderIsPending(p) ((p)->ppNextElem!=0)
118892121013
 #define fts3SegReaderIsRootOnly(p) ((p)->aNode==(char *)&(p)[1])
@@ -118920,10 +121041,18 @@
118920121041
 ** within the fts3SegWriterXXX() family of functions described above.
118921121042
 **
118922121043
 ** fts3NodeAddTerm()
118923121044
 ** fts3NodeWrite()
118924121045
 ** fts3NodeFree()
121046
+**
121047
+** When a b+tree is written to the database (either as a result of a merge
121048
+** or the pending-terms table being flushed), leaves are written into the
121049
+** database file as soon as they are completely populated. The interior of
121050
+** the tree is assembled in memory and written out only once all leaves have
121051
+** been populated and stored. This is Ok, as the b+-tree fanout is usually
121052
+** very large, meaning that the interior of the tree consumes relatively
121053
+** little memory.
118925121054
 */
118926121055
 struct SegmentNode {
118927121056
 SegmentNode *pParent; /* Parent node (or NULL for root node) */
118928121057
 SegmentNode *pRight; /* Pointer to right-sibling */
118929121058
 SegmentNode *pLeftmost; /* Pointer to left-most node of this depth */
@@ -118950,21 +121079,26 @@
118950121079
 #define SQL_NEXT_SEGMENT_INDEX 8
118951121080
 #define SQL_INSERT_SEGMENTS 9
118952121081
 #define SQL_NEXT_SEGMENTS_ID 10
118953121082
 #define SQL_INSERT_SEGDIR 11
118954121083
 #define SQL_SELECT_LEVEL 12
118955
-#define SQL_SELECT_ALL_LEVEL 13
121084
+#define SQL_SELECT_LEVEL_RANGE 13
118956121085
 #define SQL_SELECT_LEVEL_COUNT 14
118957
-#define SQL_SELECT_SEGDIR_COUNT_MAX 15
118958
-#define SQL_DELETE_SEGDIR_BY_LEVEL 16
121086
+#define SQL_SELECT_SEGDIR_MAX_LEVEL 15
121087
+#define SQL_DELETE_SEGDIR_LEVEL 16
118959121088
 #define SQL_DELETE_SEGMENTS_RANGE 17
118960121089
 #define SQL_CONTENT_INSERT 18
118961121090
 #define SQL_DELETE_DOCSIZE 19
118962121091
 #define SQL_REPLACE_DOCSIZE 20
118963121092
 #define SQL_SELECT_DOCSIZE 21
118964121093
 #define SQL_SELECT_DOCTOTAL 22
118965121094
 #define SQL_REPLACE_DOCTOTAL 23
121095
+
121096
+#define SQL_SELECT_ALL_PREFIX_LEVEL 24
121097
+#define SQL_DELETE_ALL_TERMS_SEGDIR 25
121098
+
121099
+#define SQL_DELETE_SEGDIR_RANGE 26
118966121100
118967121101
 /*
118968121102
 ** This function is used to obtain an SQLite prepared statement handle
118969121103
 ** for the statement identified by the second argument. If successful,
118970121104
 ** *pp is set to the requested statement handle and SQLITE_OK returned.
@@ -118997,23 +121131,29 @@
118997121131
118998121132
 /* Return segments in order from oldest to newest.*/
118999121133
 /* 12 */ "SELECT idx, start_block, leaves_end_block, end_block, root "
119000121134
 "FROM %Q.'%q_segdir' WHERE level = ? ORDER BY idx ASC",
119001121135
 /* 13 */ "SELECT idx, start_block, leaves_end_block, end_block, root "
119002
- "FROM %Q.'%q_segdir' ORDER BY level DESC, idx ASC",
121136
+ "FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?"
121137
+ "ORDER BY level DESC, idx ASC",
119003121138
119004121139
 /* 14 */ "SELECT count(*) FROM %Q.'%q_segdir' WHERE level = ?",
119005
-/* 15 */ "SELECT count(*), max(level) FROM %Q.'%q_segdir'",
121140
+/* 15 */ "SELECT max(level) FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?",
119006121141
119007121142
 /* 16 */ "DELETE FROM %Q.'%q_segdir' WHERE level = ?",
119008121143
 /* 17 */ "DELETE FROM %Q.'%q_segments' WHERE blockid BETWEEN ? AND ?",
119009121144
 /* 18 */ "INSERT INTO %Q.'%q_content' VALUES(%s)",
119010121145
 /* 19 */ "DELETE FROM %Q.'%q_docsize' WHERE docid = ?",
119011121146
 /* 20 */ "REPLACE INTO %Q.'%q_docsize' VALUES(?,?)",
119012121147
 /* 21 */ "SELECT size FROM %Q.'%q_docsize' WHERE docid=?",
119013121148
 /* 22 */ "SELECT value FROM %Q.'%q_stat' WHERE id=0",
119014121149
 /* 23 */ "REPLACE INTO %Q.'%q_stat' VALUES(0,?)",
121150
+/* 24 */ "",
121151
+/* 25 */ "",
121152
+
121153
+/* 26 */ "DELETE FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?",
121154
+
119015121155
 };
119016121156
 int rc = SQLITE_OK;
119017121157
 sqlite3_stmt *pStmt;
119018121158
119019121159
 assert( SizeofArray(azSql)==SizeofArray(p->aStmt) );
@@ -119165,18 +121305,36 @@
119165121305
 ** 1: start_block
119166121306
 ** 2: leaves_end_block
119167121307
 ** 3: end_block
119168121308
 ** 4: root
119169121309
 */
119170
-SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table *p, int iLevel, sqlite3_stmt **ppStmt){
121310
+SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(
121311
+ Fts3Table *p, /* FTS3 table */
121312
+ int iIndex, /* Index for p->aIndex[] */
121313
+ int iLevel, /* Level to select */
121314
+ sqlite3_stmt **ppStmt /* OUT: Compiled statement */
121315
+){
119171121316
 int rc;
119172121317
 sqlite3_stmt *pStmt = 0;
121318
+
121319
+ assert( iLevel==FTS3_SEGCURSOR_ALL || iLevel>=0 );
121320
+ assert( iLevel<FTS3_SEGDIR_MAXLEVEL );
121321
+ assert( iIndex>=0 && iIndex<p->nIndex );
121322
+
119173121323
 if( iLevel<0 ){
119174
- rc = fts3SqlStmt(p, SQL_SELECT_ALL_LEVEL, &pStmt, 0);
121324
+ /* "SELECT * FROM %_segdir WHERE level BETWEEN ? AND ? ORDER BY ..." */
121325
+ rc = fts3SqlStmt(p, SQL_SELECT_LEVEL_RANGE, &pStmt, 0);
121326
+ if( rc==SQLITE_OK ){
121327
+ sqlite3_bind_int(pStmt, 1, iIndex*FTS3_SEGDIR_MAXLEVEL);
121328
+ sqlite3_bind_int(pStmt, 2, (iIndex+1)*FTS3_SEGDIR_MAXLEVEL-1);
121329
+ }
119175121330
 }else{
121331
+ /* "SELECT * FROM %_segdir WHERE level = ? ORDER BY ..." */
119176121332
 rc = fts3SqlStmt(p, SQL_SELECT_LEVEL, &pStmt, 0);
119177
- if( rc==SQLITE_OK ) sqlite3_bind_int(pStmt, 1, iLevel);
121333
+ if( rc==SQLITE_OK ){
121334
+ sqlite3_bind_int(pStmt, 1, iLevel+iIndex*FTS3_SEGDIR_MAXLEVEL);
121335
+ }
119178121336
 }
119179121337
 *ppStmt = pStmt;
119180121338
 return rc;
119181121339
 }
119182121340
@@ -119286,10 +121444,51 @@
119286121444
 *pp = p;
119287121445
 return 1;
119288121446
 }
119289121447
 return 0;
119290121448
 }
121449
+
121450
+/*
121451
+** Free a PendingList object allocated by fts3PendingListAppend().
121452
+*/
121453
+static void fts3PendingListDelete(PendingList *pList){
121454
+ sqlite3_free(pList);
121455
+}
121456
+
121457
+/*
121458
+** Add an entry to one of the pending-terms hash tables.
121459
+*/
121460
+static int fts3PendingTermsAddOne(
121461
+ Fts3Table *p,
121462
+ int iCol,
121463
+ int iPos,
121464
+ Fts3Hash *pHash, /* Pending terms hash table to add entry to */
121465
+ const char *zToken,
121466
+ int nToken
121467
+){
121468
+ PendingList *pList;
121469
+ int rc = SQLITE_OK;
121470
+
121471
+ pList = (PendingList *)fts3HashFind(pHash, zToken, nToken);
121472
+ if( pList ){
121473
+ p->nPendingData -= (pList->nData + nToken + sizeof(Fts3HashElem));
121474
+ }
121475
+ if( fts3PendingListAppend(&pList, p->iPrevDocid, iCol, iPos, &rc) ){
121476
+ if( pList==fts3HashInsert(pHash, zToken, nToken, pList) ){
121477
+ /* Malloc failed while inserting the new entry. This can only
121478
+ ** happen if there was no previous entry for this token.
121479
+ */
121480
+ assert( 0==fts3HashFind(pHash, zToken, nToken) );
121481
+ sqlite3_free(pList);
121482
+ rc = SQLITE_NOMEM;
121483
+ }
121484
+ }
121485
+ if( rc==SQLITE_OK ){
121486
+ p->nPendingData += (pList->nData + nToken + sizeof(Fts3HashElem));
121487
+ }
121488
+ return rc;
121489
+}
119291121490
119292121491
 /*
119293121492
 ** Tokenize the nul-terminated string zText and add all tokens to the
119294121493
 ** pending-terms hash-table. The docid used is that currently stored in
119295121494
 ** p->iPrevDocid, and the column is specified by argument iCol.
@@ -119335,12 +121534,11 @@
119335121534
119336121535
 xNext = pModule->xNext;
119337121536
 while( SQLITE_OK==rc
119338121537
 && SQLITE_OK==(rc = xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos))
119339121538
 ){
119340
- PendingList *pList;
119341
-
121539
+ int i;
119342121540
 if( iPos>=nWord ) nWord = iPos+1;
119343121541
119344121542
 /* Positions cannot be negative; we use -1 as a terminator internally.
119345121543
 ** Tokens must have a non-zero length.
119346121544
 */
@@ -119347,26 +121545,23 @@
119347121545
 if( iPos<0 || !zToken || nToken<=0 ){
119348121546
 rc = SQLITE_ERROR;
119349121547
 break;
119350121548
 }
119351121549
119352
- pList = (PendingList *)fts3HashFind(&p->pendingTerms, zToken, nToken);
119353
- if( pList ){
119354
- p->nPendingData -= (pList->nData + nToken + sizeof(Fts3HashElem));
119355
- }
119356
- if( fts3PendingListAppend(&pList, p->iPrevDocid, iCol, iPos, &rc) ){
119357
- if( pList==fts3HashInsert(&p->pendingTerms, zToken, nToken, pList) ){
119358
- /* Malloc failed while inserting the new entry. This can only
119359
- ** happen if there was no previous entry for this token.
119360
- */
119361
- assert( 0==fts3HashFind(&p->pendingTerms, zToken, nToken) );
119362
- sqlite3_free(pList);
119363
- rc = SQLITE_NOMEM;
119364
- }
119365
- }
119366
- if( rc==SQLITE_OK ){
119367
- p->nPendingData += (pList->nData + nToken + sizeof(Fts3HashElem));
121550
+ /* Add the term to the terms index */
121551
+ rc = fts3PendingTermsAddOne(
121552
+ p, iCol, iPos, &p->aIndex[0].hPending, zToken, nToken
121553
+ );
121554
+
121555
+ /* Add the term to each of the prefix indexes that it is not too
121556
+ ** short for. */
121557
+ for(i=1; rc==SQLITE_OK && i<p->nIndex; i++){
121558
+ struct Fts3Index *pIndex = &p->aIndex[i];
121559
+ if( nToken<pIndex->nPrefix ) continue;
121560
+ rc = fts3PendingTermsAddOne(
121561
+ p, iCol, iPos, &pIndex->hPending, zToken, pIndex->nPrefix
121562
+ );
119368121563
 }
119369121564
 }
119370121565
119371121566
 pModule->xClose(pCsr);
119372121567
 *pnWord = nWord;
@@ -119392,18 +121587,23 @@
119392121587
 p->iPrevDocid = iDocid;
119393121588
 return SQLITE_OK;
119394121589
 }
119395121590
119396121591
 /*
119397
-** Discard the contents of the pending-terms hash table.
121592
+** Discard the contents of the pending-terms hash tables.
119398121593
 */
119399121594
 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *p){
119400
- Fts3HashElem *pElem;
119401
- for(pElem=fts3HashFirst(&p->pendingTerms); pElem; pElem=fts3HashNext(pElem)){
119402
- sqlite3_free(fts3HashData(pElem));
121595
+ int i;
121596
+ for(i=0; i<p->nIndex; i++){
121597
+ Fts3HashElem *pElem;
121598
+ Fts3Hash *pHash = &p->aIndex[i].hPending;
121599
+ for(pElem=fts3HashFirst(pHash); pElem; pElem=fts3HashNext(pElem)){
121600
+ PendingList *pList = (PendingList *)fts3HashData(pElem);
121601
+ fts3PendingListDelete(pList);
121602
+ }
121603
+ fts3HashClear(pHash);
119403121604
 }
119404
- fts3HashClear(&p->pendingTerms);
119405121605
 p->nPendingData = 0;
119406121606
 }
119407121607
119408121608
 /*
119409121609
 ** This function is called by the xUpdate() method as part of an INSERT
@@ -119555,11 +121755,11 @@
119555121755
119556121756
 /*
119557121757
 ** Forward declaration to account for the circular dependency between
119558121758
 ** functions fts3SegmentMerge() and fts3AllocateSegdirIdx().
119559121759
 */
119560
-static int fts3SegmentMerge(Fts3Table *, int);
121760
+static int fts3SegmentMerge(Fts3Table *, int, int);
119561121761
119562121762
 /*
119563121763
 ** This function allocates a new level iLevel index in the segdir table.
119564121764
 ** Usually, indexes are allocated within a level sequentially starting
119565121765
 ** with 0, so the allocated index is one greater than the value returned
@@ -119572,19 +121772,24 @@
119572121772
 ** allocated index is 0.
119573121773
 **
119574121774
 ** If successful, *piIdx is set to the allocated index slot and SQLITE_OK
119575121775
 ** returned. Otherwise, an SQLite error code is returned.
119576121776
 */
119577
-static int fts3AllocateSegdirIdx(Fts3Table *p, int iLevel, int *piIdx){
121777
+static int fts3AllocateSegdirIdx(
121778
+ Fts3Table *p,
121779
+ int iIndex, /* Index for p->aIndex */
121780
+ int iLevel,
121781
+ int *piIdx
121782
+){
119578121783
 int rc; /* Return Code */
119579121784
 sqlite3_stmt *pNextIdx; /* Query for next idx at level iLevel */
119580121785
 int iNext = 0; /* Result of query pNextIdx */
119581121786
119582121787
 /* Set variable iNext to the next available segdir index at level iLevel. */
119583121788
 rc = fts3SqlStmt(p, SQL_NEXT_SEGMENT_INDEX, &pNextIdx, 0);
119584121789
 if( rc==SQLITE_OK ){
119585
- sqlite3_bind_int(pNextIdx, 1, iLevel);
121790
+ sqlite3_bind_int(pNextIdx, 1, iIndex*FTS3_SEGDIR_MAXLEVEL + iLevel);
119586121791
 if( SQLITE_ROW==sqlite3_step(pNextIdx) ){
119587121792
 iNext = sqlite3_column_int(pNextIdx, 0);
119588121793
 }
119589121794
 rc = sqlite3_reset(pNextIdx);
119590121795
 }
@@ -119594,11 +121799,11 @@
119594121799
 ** full, merge all segments in level iLevel into a single iLevel+1
119595121800
 ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise,
119596121801
 ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext.
119597121802
 */
119598121803
 if( iNext>=FTS3_MERGE_COUNT ){
119599
- rc = fts3SegmentMerge(p, iLevel);
121804
+ rc = fts3SegmentMerge(p, iIndex, iLevel);
119600121805
 *piIdx = 0;
119601121806
 }else{
119602121807
 *piIdx = iNext;
119603121808
 }
119604121809
 }
@@ -119635,11 +121840,12 @@
119635121840
 */
119636121841
 SQLITE_PRIVATE int sqlite3Fts3ReadBlock(
119637121842
 Fts3Table *p, /* FTS3 table handle */
119638121843
 sqlite3_int64 iBlockid, /* Access the row with blockid=$iBlockid */
119639121844
 char **paBlob, /* OUT: Blob data in malloc'd buffer */
119640
- int *pnBlob /* OUT: Size of blob data */
121845
+ int *pnBlob, /* OUT: Size of blob data */
121846
+ int *pnLoad /* OUT: Bytes actually loaded */
119641121847
 ){
119642121848
 int rc; /* Return code */
119643121849
119644121850
 /* pnBlob must be non-NULL. paBlob may be NULL or non-NULL. */
119645121851
 assert( pnBlob);
@@ -119656,25 +121862,29 @@
119656121862
 );
119657121863
 }
119658121864
119659121865
 if( rc==SQLITE_OK ){
119660121866
 int nByte = sqlite3_blob_bytes(p->pSegments);
121867
+ *pnBlob = nByte;
119661121868
 if( paBlob ){
119662121869
 char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING);
119663121870
 if( !aByte ){
119664121871
 rc = SQLITE_NOMEM;
119665121872
 }else{
121873
+ if( pnLoad && nByte>(FTS3_NODE_CHUNK_THRESHOLD) ){
121874
+ nByte = FTS3_NODE_CHUNKSIZE;
121875
+ *pnLoad = nByte;
121876
+ }
119666121877
 rc = sqlite3_blob_read(p->pSegments, aByte, nByte, 0);
119667121878
 memset(&aByte[nByte], 0, FTS3_NODE_PADDING);
119668121879
 if( rc!=SQLITE_OK ){
119669121880
 sqlite3_free(aByte);
119670121881
 aByte = 0;
119671121882
 }
119672121883
 }
119673121884
 *paBlob = aByte;
119674121885
 }
119675
- *pnBlob = nByte;
119676121886
 }
119677121887
119678121888
 return rc;
119679121889
 }
119680121890
@@ -119684,17 +121894,59 @@
119684121894
 */
119685121895
 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *p){
119686121896
 sqlite3_blob_close(p->pSegments);
119687121897
 p->pSegments = 0;
119688121898
 }
121899
+
121900
+static int fts3SegReaderIncrRead(Fts3SegReader *pReader){
121901
+ int nRead; /* Number of bytes to read */
121902
+ int rc; /* Return code */
121903
+
121904
+ nRead = MIN(pReader->nNode - pReader->nPopulate, FTS3_NODE_CHUNKSIZE);
121905
+ rc = sqlite3_blob_read(
121906
+ pReader->pBlob,
121907
+ &pReader->aNode[pReader->nPopulate],
121908
+ nRead,
121909
+ pReader->nPopulate
121910
+ );
121911
+
121912
+ if( rc==SQLITE_OK ){
121913
+ pReader->nPopulate += nRead;
121914
+ memset(&pReader->aNode[pReader->nPopulate], 0, FTS3_NODE_PADDING);
121915
+ if( pReader->nPopulate==pReader->nNode ){
121916
+ sqlite3_blob_close(pReader->pBlob);
121917
+ pReader->pBlob = 0;
121918
+ pReader->nPopulate = 0;
121919
+ }
121920
+ }
121921
+ return rc;
121922
+}
121923
+
121924
+static int fts3SegReaderRequire(Fts3SegReader *pReader, char *pFrom, int nByte){
121925
+ int rc = SQLITE_OK;
121926
+ assert( !pReader->pBlob
121927
+ || (pFrom>=pReader->aNode && pFrom<&pReader->aNode[pReader->nNode])
121928
+ );
121929
+ while( pReader->pBlob && rc==SQLITE_OK
121930
+ && (pFrom - pReader->aNode + nByte)>pReader->nPopulate
121931
+ ){
121932
+ rc = fts3SegReaderIncrRead(pReader);
121933
+ }
121934
+ return rc;
121935
+}
119689121936
119690121937
 /*
119691121938
 ** Move the iterator passed as the first argument to the next term in the
119692121939
 ** segment. If successful, SQLITE_OK is returned. If there is no next term,
119693121940
 ** SQLITE_DONE. Otherwise, an SQLite error code.
119694121941
 */
119695
-static int fts3SegReaderNext(Fts3Table *p, Fts3SegReader *pReader){
121942
+static int fts3SegReaderNext(
121943
+ Fts3Table *p,
121944
+ Fts3SegReader *pReader,
121945
+ int bIncr
121946
+){
121947
+ int rc; /* Return code of various sub-routines */
119696121948
 char *pNext; /* Cursor variable */
119697121949
 int nPrefix; /* Number of bytes in term prefix */
119698121950
 int nSuffix; /* Number of bytes in term suffix */
119699121951
119700121952
 if( !pReader->aDoclist ){
@@ -119702,11 +121954,10 @@
119702121954
 }else{
119703121955
 pNext = &pReader->aDoclist[pReader->nDoclist];
119704121956
 }
119705121957
119706121958
 if( !pNext || pNext>=&pReader->aNode[pReader->nNode] ){
119707
- int rc; /* Return code from Fts3ReadBlock() */
119708121959
119709121960
 if( fts3SegReaderIsPending(pReader) ){
119710121961
 Fts3HashElem *pElem = *(pReader->ppNextElem);
119711121962
 if( pElem==0 ){
119712121963
 pReader->aNode = 0;
@@ -119722,10 +121973,12 @@
119722121973
 return SQLITE_OK;
119723121974
 }
119724121975
119725121976
 if( !fts3SegReaderIsRootOnly(pReader) ){
119726121977
 sqlite3_free(pReader->aNode);
121978
+ sqlite3_blob_close(pReader->pBlob);
121979
+ pReader->pBlob = 0;
119727121980
 }
119728121981
 pReader->aNode = 0;
119729121982
119730121983
 /* If iCurrentBlock>=iLeafEndBlock, this is an EOF condition. All leaf
119731121984
 ** blocks have already been traversed. */
@@ -119733,19 +121986,29 @@
119733121986
 if( pReader->iCurrentBlock>=pReader->iLeafEndBlock ){
119734121987
 return SQLITE_OK;
119735121988
 }
119736121989
119737121990
 rc = sqlite3Fts3ReadBlock(
119738
- p, ++pReader->iCurrentBlock, &pReader->aNode, &pReader->nNode
121991
+ p, ++pReader->iCurrentBlock, &pReader->aNode, &pReader->nNode,
121992
+ (bIncr ? &pReader->nPopulate : 0)
119739121993
 );
119740121994
 if( rc!=SQLITE_OK ) return rc;
121995
+ assert( pReader->pBlob==0 );
121996
+ if( bIncr && pReader->nPopulate<pReader->nNode ){
121997
+ pReader->pBlob = p->pSegments;
121998
+ p->pSegments = 0;
121999
+ }
119741122000
 pNext = pReader->aNode;
119742122001
 }
122002
+
122003
+ assert( !fts3SegReaderIsPending(pReader) );
122004
+
122005
+ rc = fts3SegReaderRequire(pReader, pNext, FTS3_VARINT_MAX*2);
122006
+ if( rc!=SQLITE_OK ) return rc;
119743122007
119744122008
 /* Because of the FTS3_NODE_PADDING bytes of padding, the following is
119745
- ** safe (no risk of overread) even if the node data is corrupted.
119746
- */
122009
+ ** safe (no risk of overread) even if the node data is corrupted. */
119747122010
 pNext += sqlite3Fts3GetVarint32(pNext, &nPrefix);
119748122011
 pNext += sqlite3Fts3GetVarint32(pNext, &nSuffix);
119749122012
 if( nPrefix<0 || nSuffix<=0
119750122013
 || &pNext[nSuffix]>&pReader->aNode[pReader->nNode]
119751122014
 ){
@@ -119759,10 +122022,14 @@
119759122022
 return SQLITE_NOMEM;
119760122023
 }
119761122024
 pReader->zTerm = zNew;
119762122025
 pReader->nTermAlloc = nNew;
119763122026
 }
122027
+
122028
+ rc = fts3SegReaderRequire(pReader, pNext, nSuffix+FTS3_VARINT_MAX);
122029
+ if( rc!=SQLITE_OK ) return rc;
122030
+
119764122031
 memcpy(&pReader->zTerm[nPrefix], pNext, nSuffix);
119765122032
 pReader->nTerm = nPrefix+nSuffix;
119766122033
 pNext += nSuffix;
119767122034
 pNext += sqlite3Fts3GetVarint32(pNext, &pReader->nDoclist);
119768122035
 pReader->aDoclist = pNext;
@@ -119771,11 +122038,11 @@
119771122038
 /* Check that the doclist does not appear to extend past the end of the
119772122039
 ** b-tree node. And that the final byte of the doclist is 0x00. If either
119773122040
 ** of these statements is untrue, then the data structure is corrupt.
119774122041
 */
119775122042
 if( &pReader->aDoclist[pReader->nDoclist]>&pReader->aNode[pReader->nNode]
119776
- || pReader->aDoclist[pReader->nDoclist-1]
122043
+ || (pReader->nPopulate==0 && pReader->aDoclist[pReader->nDoclist-1])
119777122044
 ){
119778122045
 return SQLITE_CORRUPT_VTAB;
119779122046
 }
119780122047
 return SQLITE_OK;
119781122048
 }
@@ -119782,16 +122049,30 @@
119782122049
119783122050
 /*
119784122051
 ** Set the SegReader to point to the first docid in the doclist associated
119785122052
 ** with the current term.
119786122053
 */
119787
-static void fts3SegReaderFirstDocid(Fts3SegReader *pReader){
119788
- int n;
122054
+static int fts3SegReaderFirstDocid(Fts3Table *pTab, Fts3SegReader *pReader){
122055
+ int rc = SQLITE_OK;
119789122056
 assert( pReader->aDoclist );
119790122057
 assert( !pReader->pOffsetList );
119791
- n = sqlite3Fts3GetVarint(pReader->aDoclist, &pReader->iDocid);
119792
- pReader->pOffsetList = &pReader->aDoclist[n];
122058
+ if( pTab->bDescIdx && fts3SegReaderIsPending(pReader) ){
122059
+ u8 bEof = 0;
122060
+ pReader->iDocid = 0;
122061
+ pReader->nOffsetList = 0;
122062
+ sqlite3Fts3DoclistPrev(0,
122063
+ pReader->aDoclist, pReader->nDoclist, &pReader->pOffsetList,
122064
+ &pReader->iDocid, &pReader->nOffsetList, &bEof
122065
+ );
122066
+ }else{
122067
+ rc = fts3SegReaderRequire(pReader, pReader->aDoclist, FTS3_VARINT_MAX);
122068
+ if( rc==SQLITE_OK ){
122069
+ int n = sqlite3Fts3GetVarint(pReader->aDoclist, &pReader->iDocid);
122070
+ pReader->pOffsetList = &pReader->aDoclist[n];
122071
+ }
122072
+ }
122073
+ return rc;
119793122074
 }
119794122075
119795122076
 /*
119796122077
 ** Advance the SegReader to point to the next docid in the doclist
119797122078
 ** associated with the current term.
@@ -119800,132 +122081,129 @@
119800122081
 ** *ppOffsetList is set to point to the first column-offset list
119801122082
 ** in the doclist entry (i.e. immediately past the docid varint).
119802122083
 ** *pnOffsetList is set to the length of the set of column-offset
119803122084
 ** lists, not including the nul-terminator byte. For example:
119804122085
 */
119805
-static void fts3SegReaderNextDocid(
119806
- Fts3SegReader *pReader,
119807
- char **ppOffsetList,
119808
- int *pnOffsetList
122086
+static int fts3SegReaderNextDocid(
122087
+ Fts3Table *pTab,
122088
+ Fts3SegReader *pReader, /* Reader to advance to next docid */
122089
+ char **ppOffsetList, /* OUT: Pointer to current position-list */
122090
+ int *pnOffsetList /* OUT: Length of *ppOffsetList in bytes */
119809122091
 ){
122092
+ int rc = SQLITE_OK;
119810122093
 char *p = pReader->pOffsetList;
119811122094
 char c = 0;
119812122095
119813
- /* Pointer p currently points at the first byte of an offset list. The
119814
- ** following two lines advance it to point one byte past the end of
119815
- ** the same offset list.
119816
- */
119817
- while( *p | c ) c = *p++ & 0x80;
119818
- p++;
119819
-
119820
- /* If required, populate the output variables with a pointer to and the
119821
- ** size of the previous offset-list.
119822
- */
119823
- if( ppOffsetList ){
119824
- *ppOffsetList = pReader->pOffsetList;
119825
- *pnOffsetList = (int)(p - pReader->pOffsetList - 1);
119826
- }
119827
-
119828
- /* If there are no more entries in the doclist, set pOffsetList to
119829
- ** NULL. Otherwise, set Fts3SegReader.iDocid to the next docid and
119830
- ** Fts3SegReader.pOffsetList to point to the next offset list before
119831
- ** returning.
119832
- */
119833
- if( p>=&pReader->aDoclist[pReader->nDoclist] ){
119834
- pReader->pOffsetList = 0;
122096
+ assert( p );
122097
+
122098
+ if( pTab->bDescIdx && fts3SegReaderIsPending(pReader) ){
122099
+ /* A pending-terms seg-reader for an FTS4 table that uses order=desc.
122100
+ ** Pending-terms doclists are always built up in ascending order, so
122101
+ ** we have to iterate through them backwards here. */
122102
+ u8 bEof = 0;
122103
+ if( ppOffsetList ){
122104
+ *ppOffsetList = pReader->pOffsetList;
122105
+ *pnOffsetList = pReader->nOffsetList - 1;
122106
+ }
122107
+ sqlite3Fts3DoclistPrev(0,
122108
+ pReader->aDoclist, pReader->nDoclist, &p, &pReader->iDocid,
122109
+ &pReader->nOffsetList, &bEof
122110
+ );
122111
+ if( bEof ){
122112
+ pReader->pOffsetList = 0;
122113
+ }else{
122114
+ pReader->pOffsetList = p;
122115
+ }
119835122116
 }else{
119836
- sqlite3_int64 iDelta;
119837
- pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta);
119838
- pReader->iDocid += iDelta;
119839
- }
119840
-}
119841
-
119842
-/*
119843
-** This function is called to estimate the amount of data that will be
119844
-** loaded from the disk If SegReaderIterate() is called on this seg-reader,
119845
-** in units of average document size.
119846
-**
119847
-** This can be used as follows: If the caller has a small doclist that
119848
-** contains references to N documents, and is considering merging it with
119849
-** a large doclist (size X "average documents"), it may opt not to load
119850
-** the large doclist if X>N.
119851
-*/
119852
-SQLITE_PRIVATE int sqlite3Fts3SegReaderCost(
119853
- Fts3Cursor *pCsr, /* FTS3 cursor handle */
119854
- Fts3SegReader *pReader, /* Segment-reader handle */
119855
- int *pnCost /* IN/OUT: Number of bytes read */
122117
+ char *pEnd = &pReader->aDoclist[pReader->nDoclist];
122118
+
122119
+ /* Pointer p currently points at the first byte of an offset list. The
122120
+ ** following block advances it to point one byte past the end of
122121
+ ** the same offset list. */
122122
+ while( 1 ){
122123
+
122124
+ /* The following line of code (and the "p++" below the while() loop) is
122125
+ ** normally all that is required to move pointer p to the desired
122126
+ ** position. The exception is if this node is being loaded from disk
122127
+ ** incrementally and pointer "p" now points to the first byte passed
122128
+ ** the populated part of pReader->aNode[].
122129
+ */
122130
+ while( *p | c ) c = *p++ & 0x80;
122131
+ assert( *p==0 );
122132
+
122133
+ if( pReader->pBlob==0 || p<&pReader->aNode[pReader->nPopulate] ) break;
122134
+ rc = fts3SegReaderIncrRead(pReader);
122135
+ if( rc!=SQLITE_OK ) return rc;
122136
+ }
122137
+ p++;
122138
+
122139
+ /* If required, populate the output variables with a pointer to and the
122140
+ ** size of the previous offset-list.
122141
+ */
122142
+ if( ppOffsetList ){
122143
+ *ppOffsetList = pReader->pOffsetList;
122144
+ *pnOffsetList = (int)(p - pReader->pOffsetList - 1);
122145
+ }
122146
+
122147
+ while( p<pEnd && *p==0 ) p++;
122148
+
122149
+ /* If there are no more entries in the doclist, set pOffsetList to
122150
+ ** NULL. Otherwise, set Fts3SegReader.iDocid to the next docid and
122151
+ ** Fts3SegReader.pOffsetList to point to the next offset list before
122152
+ ** returning.
122153
+ */
122154
+ if( p>=pEnd ){
122155
+ pReader->pOffsetList = 0;
122156
+ }else{
122157
+ rc = fts3SegReaderRequire(pReader, p, FTS3_VARINT_MAX);
122158
+ if( rc==SQLITE_OK ){
122159
+ sqlite3_int64 iDelta;
122160
+ pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta);
122161
+ if( pTab->bDescIdx ){
122162
+ pReader->iDocid -= iDelta;
122163
+ }else{
122164
+ pReader->iDocid += iDelta;
122165
+ }
122166
+ }
122167
+ }
122168
+ }
122169
+
122170
+ return SQLITE_OK;
122171
+}
122172
+
122173
+
122174
+SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(
122175
+ Fts3Cursor *pCsr,
122176
+ Fts3MultiSegReader *pMsr,
122177
+ int *pnOvfl
119856122178
 ){
119857122179
 Fts3Table *p = (Fts3Table*)pCsr->base.pVtab;
119858
- int rc = SQLITE_OK; /* Return code */
119859
- int nCost = 0; /* Cost in bytes to return */
119860
- int pgsz = p->nPgsz; /* Database page size */
119861
-
119862
- /* If this seg-reader is reading the pending-terms table, or if all data
119863
- ** for the segment is stored on the root page of the b-tree, then the cost
119864
- ** is zero. In this case all required data is already in main memory.
119865
- */
119866
- if( p->bHasStat
119867
- && !fts3SegReaderIsPending(pReader)
119868
- && !fts3SegReaderIsRootOnly(pReader)
119869
- ){
119870
- int nBlob = 0;
119871
- sqlite3_int64 iBlock;
119872
-
119873
- if( pCsr->nRowAvg==0 ){
119874
- /* The average document size, which is required to calculate the cost
119875
- ** of each doclist, has not yet been determined. Read the required
119876
- ** data from the %_stat table to calculate it.
119877
- **
119878
- ** Entry 0 of the %_stat table is a blob containing (nCol+1) FTS3
119879
- ** varints, where nCol is the number of columns in the FTS3 table.
119880
- ** The first varint is the number of documents currently stored in
119881
- ** the table. The following nCol varints contain the total amount of
119882
- ** data stored in all rows of each column of the table, from left
119883
- ** to right.
119884
- */
119885
- sqlite3_stmt *pStmt;
119886
- sqlite3_int64 nDoc = 0;
119887
- sqlite3_int64 nByte = 0;
119888
- const char *pEnd;
119889
- const char *a;
119890
-
119891
- rc = sqlite3Fts3SelectDoctotal(p, &pStmt);
119892
- if( rc!=SQLITE_OK ) return rc;
119893
- a = sqlite3_column_blob(pStmt, 0);
119894
- assert( a );
119895
-
119896
- pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
119897
- a += sqlite3Fts3GetVarint(a, &nDoc);
119898
- while( a<pEnd ){
119899
- a += sqlite3Fts3GetVarint(a, &nByte);
119900
- }
119901
- if( nDoc==0 || nByte==0 ){
119902
- sqlite3_reset(pStmt);
119903
- return SQLITE_CORRUPT_VTAB;
119904
- }
119905
-
119906
- pCsr->nRowAvg = (int)(((nByte / nDoc) + pgsz) / pgsz);
119907
- assert( pCsr->nRowAvg>0 );
119908
- rc = sqlite3_reset(pStmt);
119909
- if( rc!=SQLITE_OK ) return rc;
119910
- }
119911
-
119912
- /* Assume that a blob flows over onto overflow pages if it is larger
119913
- ** than (pgsz-35) bytes in size (the file-format documentation
119914
- ** confirms this).
119915
- */
119916
- for(iBlock=pReader->iStartBlock; iBlock<=pReader->iLeafEndBlock; iBlock++){
119917
- rc = sqlite3Fts3ReadBlock(p, iBlock, 0, &nBlob);
119918
- if( rc!=SQLITE_OK ) break;
119919
- if( (nBlob+35)>pgsz ){
119920
- int nOvfl = (nBlob + 34)/pgsz;
119921
- nCost += ((nOvfl + pCsr->nRowAvg - 1)/pCsr->nRowAvg);
119922
- }
119923
- }
119924
- }
119925
-
119926
- *pnCost += nCost;
122180
+ int nOvfl = 0;
122181
+ int ii;
122182
+ int rc = SQLITE_OK;
122183
+ int pgsz = p->nPgsz;
122184
+
122185
+ assert( p->bHasStat );
122186
+ assert( pgsz>0 );
122187
+
122188
+ for(ii=0; rc==SQLITE_OK && ii<pMsr->nSegment; ii++){
122189
+ Fts3SegReader *pReader = pMsr->apSegment[ii];
122190
+ if( !fts3SegReaderIsPending(pReader)
122191
+ && !fts3SegReaderIsRootOnly(pReader)
122192
+ ){
122193
+ sqlite3_int64 jj;
122194
+ for(jj=pReader->iStartBlock; jj<=pReader->iLeafEndBlock; jj++){
122195
+ int nBlob;
122196
+ rc = sqlite3Fts3ReadBlock(p, jj, 0, &nBlob, 0);
122197
+ if( rc!=SQLITE_OK ) break;
122198
+ if( (nBlob+35)>pgsz ){
122199
+ nOvfl += (nBlob + 34)/pgsz;
122200
+ }
122201
+ }
122202
+ }
122203
+ }
122204
+ *pnOvfl = nOvfl;
119927122205
 return rc;
119928122206
 }
119929122207
119930122208
 /*
119931122209
 ** Free all allocations associated with the iterator passed as the
@@ -119934,10 +122212,11 @@
119934122212
 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){
119935122213
 if( pReader && !fts3SegReaderIsPending(pReader) ){
119936122214
 sqlite3_free(pReader->zTerm);
119937122215
 if( !fts3SegReaderIsRootOnly(pReader) ){
119938122216
 sqlite3_free(pReader->aNode);
122217
+ sqlite3_blob_close(pReader->pBlob);
119939122218
 }
119940122219
 }
119941122220
 sqlite3_free(pReader);
119942122221
 }
119943122222
@@ -120010,28 +122289,46 @@
120010122289
 }
120011122290
120012122291
 /*
120013122292
 ** This function is used to allocate an Fts3SegReader that iterates through
120014122293
 ** a subset of the terms stored in the Fts3Table.pendingTerms array.
122294
+**
122295
+** If the isPrefixIter parameter is zero, then the returned SegReader iterates
122296
+** through each term in the pending-terms table. Or, if isPrefixIter is
122297
+** non-zero, it iterates through each term and its prefixes. For example, if
122298
+** the pending terms hash table contains the terms "sqlite", "mysql" and
122299
+** "firebird", then the iterator visits the following 'terms' (in the order
122300
+** shown):
122301
+**
122302
+** f fi fir fire fireb firebi firebir firebird
122303
+** m my mys mysq mysql
122304
+** s sq sql sqli sqlit sqlite
122305
+**
122306
+** Whereas if isPrefixIter is zero, the terms visited are:
122307
+**
122308
+** firebird mysql sqlite
120015122309
 */
120016122310
 SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(
120017122311
 Fts3Table *p, /* Virtual table handle */
122312
+ int iIndex, /* Index for p->aIndex */
120018122313
 const char *zTerm, /* Term to search for */
120019122314
 int nTerm, /* Size of buffer zTerm */
120020
- int isPrefix, /* True for a term-prefix query */
122315
+ int bPrefix, /* True for a prefix iterator */
120021122316
 Fts3SegReader **ppReader /* OUT: SegReader for pending-terms */
120022122317
 ){
120023122318
 Fts3SegReader *pReader = 0; /* Fts3SegReader object to return */
120024122319
 Fts3HashElem **aElem = 0; /* Array of term hash entries to scan */
120025122320
 int nElem = 0; /* Size of array at aElem */
120026122321
 int rc = SQLITE_OK; /* Return Code */
122322
+ Fts3Hash *pHash;
120027122323
120028
- if( isPrefix ){
122324
+ pHash = &p->aIndex[iIndex].hPending;
122325
+ if( bPrefix ){
120029122326
 int nAlloc = 0; /* Size of allocated array at aElem */
120030122327
 Fts3HashElem *pE = 0; /* Iterator variable */
120031122328
120032
- for(pE=fts3HashFirst(&p->pendingTerms); pE; pE=fts3HashNext(pE)){
122329
+ for(pE=fts3HashFirst(pHash); pE; pE=fts3HashNext(pE)){
120033122330
 char *zKey = (char *)fts3HashKey(pE);
120034122331
 int nKey = fts3HashKeysize(pE);
120035122332
 if( nTerm==0 || (nKey>=nTerm && 0==memcmp(zKey, zTerm, nTerm)) ){
120036122333
 if( nElem==nAlloc ){
120037122334
 Fts3HashElem **aElem2;
@@ -120044,10 +122341,11 @@
120044122341
 nElem = 0;
120045122342
 break;
120046122343
 }
120047122344
 aElem = aElem2;
120048122345
 }
122346
+
120049122347
 aElem[nElem++] = pE;
120050122348
 }
120051122349
 }
120052122350
120053122351
 /* If more than one term matches the prefix, sort the Fts3HashElem
@@ -120057,11 +122355,13 @@
120057122355
 if( nElem>1 ){
120058122356
 qsort(aElem, nElem, sizeof(Fts3HashElem *), fts3CompareElemByTerm);
120059122357
 }
120060122358
120061122359
 }else{
120062
- Fts3HashElem *pE = fts3HashFindElem(&p->pendingTerms, zTerm, nTerm);
122360
+ /* The query is a simple term lookup that matches at most one term in
122361
+ ** the index. All that is required is a straight hash-lookup. */
122362
+ Fts3HashElem *pE = fts3HashFindElem(pHash, zTerm, nTerm);
120063122363
 if( pE ){
120064122364
 aElem = &pE;
120065122365
 nElem = 1;
120066122366
 }
120067122367
 }
@@ -120077,11 +122377,11 @@
120077122377
 pReader->ppNextElem = (Fts3HashElem **)&pReader[1];
120078122378
 memcpy(pReader->ppNextElem, aElem, nElem*sizeof(Fts3HashElem *));
120079122379
 }
120080122380
 }
120081122381
120082
- if( isPrefix ){
122382
+ if( bPrefix ){
120083122383
 sqlite3_free(aElem);
120084122384
 }
120085122385
 *ppReader = pReader;
120086122386
 return rc;
120087122387
 }
@@ -120137,10 +122437,22 @@
120137122437
 if( pLhs->iDocid==pRhs->iDocid ){
120138122438
 rc = pRhs->iIdx - pLhs->iIdx;
120139122439
 }else{
120140122440
 rc = (pLhs->iDocid > pRhs->iDocid) ? 1 : -1;
120141122441
 }
122442
+ }
122443
+ assert( pLhs->aNode && pRhs->aNode );
122444
+ return rc;
122445
+}
122446
+static int fts3SegReaderDoclistCmpRev(Fts3SegReader *pLhs, Fts3SegReader *pRhs){
122447
+ int rc = (pLhs->pOffsetList==0)-(pRhs->pOffsetList==0);
122448
+ if( rc==0 ){
122449
+ if( pLhs->iDocid==pRhs->iDocid ){
122450
+ rc = pRhs->iIdx - pLhs->iIdx;
122451
+ }else{
122452
+ rc = (pLhs->iDocid < pRhs->iDocid) ? 1 : -1;
122453
+ }
120142122454
 }
120143122455
 assert( pLhs->aNode && pRhs->aNode );
120144122456
 return rc;
120145122457
 }
120146122458
@@ -120689,25 +123001,34 @@
120689123001
 }
120690123002
 return rc;
120691123003
 }
120692123004
120693123005
 /*
120694
-** Set *pnSegment to the total number of segments in the database. Set
120695
-** *pnMax to the largest segment level in the database (segment levels
120696
-** are stored in the 'level' column of the %_segdir table).
123006
+** Set *pnMax to the largest segment level in the database for the index
123007
+** iIndex.
123008
+**
123009
+** Segment levels are stored in the 'level' column of the %_segdir table.
120697123010
 **
120698123011
 ** Return SQLITE_OK if successful, or an SQLite error code if not.
120699123012
 */
120700
-static int fts3SegmentCountMax(Fts3Table *p, int *pnSegment, int *pnMax){
123013
+static int fts3SegmentMaxLevel(Fts3Table *p, int iIndex, int *pnMax){
120701123014
 sqlite3_stmt *pStmt;
120702123015
 int rc;
123016
+ assert( iIndex>=0 && iIndex<p->nIndex );
120703123017
120704
- rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_COUNT_MAX, &pStmt, 0);
123018
+ /* Set pStmt to the compiled version of:
123019
+ **
123020
+ ** SELECT max(level) FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?
123021
+ **
123022
+ ** (1024 is actually the value of macro FTS3_SEGDIR_PREFIXLEVEL_STR).
123023
+ */
123024
+ rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_MAX_LEVEL, &pStmt, 0);
120705123025
 if( rc!=SQLITE_OK ) return rc;
123026
+ sqlite3_bind_int(pStmt, 1, iIndex*FTS3_SEGDIR_MAXLEVEL);
123027
+ sqlite3_bind_int(pStmt, 2, (iIndex+1)*FTS3_SEGDIR_MAXLEVEL - 1);
120706123028
 if( SQLITE_ROW==sqlite3_step(pStmt) ){
120707
- *pnSegment = sqlite3_column_int(pStmt, 0);
120708
- *pnMax = sqlite3_column_int(pStmt, 1);
123029
+ *pnMax = sqlite3_column_int(pStmt, 0);
120709123030
 }
120710123031
 return sqlite3_reset(pStmt);
120711123032
 }
120712123033
120713123034
 /*
@@ -120724,10 +123045,11 @@
120724123045
 **
120725123046
 ** SQLITE_OK is returned if successful, otherwise an SQLite error code.
120726123047
 */
120727123048
 static int fts3DeleteSegdir(
120728123049
 Fts3Table *p, /* Virtual table handle */
123050
+ int iIndex, /* Index for p->aIndex */
120729123051
 int iLevel, /* Level of %_segdir entries to delete */
120730123052
 Fts3SegReader **apSegment, /* Array of SegReader objects */
120731123053
 int nReader /* Size of array apSegment */
120732123054
 ){
120733123055
 int rc; /* Return Code */
@@ -120746,23 +123068,28 @@
120746123068
 }
120747123069
 if( rc!=SQLITE_OK ){
120748123070
 return rc;
120749123071
 }
120750123072
123073
+ assert( iLevel>=0 || iLevel==FTS3_SEGCURSOR_ALL );
120751123074
 if( iLevel==FTS3_SEGCURSOR_ALL ){
120752
- fts3SqlExec(&rc, p, SQL_DELETE_ALL_SEGDIR, 0);
120753
- }else if( iLevel==FTS3_SEGCURSOR_PENDING ){
120754
- sqlite3Fts3PendingTermsClear(p);
123075
+ rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_RANGE, &pDelete, 0);
123076
+ if( rc==SQLITE_OK ){
123077
+ sqlite3_bind_int(pDelete, 1, iIndex*FTS3_SEGDIR_MAXLEVEL);
123078
+ sqlite3_bind_int(pDelete, 2, (iIndex+1) * FTS3_SEGDIR_MAXLEVEL - 1);
123079
+ }
120755123080
 }else{
120756
- assert( iLevel>=0 );
120757
- rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_BY_LEVEL, &pDelete, 0);
123081
+ rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_LEVEL, &pDelete, 0);
120758123082
 if( rc==SQLITE_OK ){
120759
- sqlite3_bind_int(pDelete, 1, iLevel);
120760
- sqlite3_step(pDelete);
120761
- rc = sqlite3_reset(pDelete);
123083
+ sqlite3_bind_int(pDelete, 1, iIndex*FTS3_SEGDIR_MAXLEVEL + iLevel);
120762123084
 }
120763123085
 }
123086
+
123087
+ if( rc==SQLITE_OK ){
123088
+ sqlite3_step(pDelete);
123089
+ rc = sqlite3_reset(pDelete);
123090
+ }
120764123091
120765123092
 return rc;
120766123093
 }
120767123094
120768123095
 /*
@@ -120806,44 +123133,216 @@
120806123133
120807123134
 *ppList = pList;
120808123135
 *pnList = nList;
120809123136
 }
120810123137
120811
-SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(
123138
+/*
123139
+** Cache data in the Fts3MultiSegReader.aBuffer[] buffer (overwriting any
123140
+** existing data). Grow the buffer if required.
123141
+**
123142
+** If successful, return SQLITE_OK. Otherwise, if an OOM error is encountered
123143
+** trying to resize the buffer, return SQLITE_NOMEM.
123144
+*/
123145
+static int fts3MsrBufferData(
123146
+ Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */
123147
+ char *pList,
123148
+ int nList
123149
+){
123150
+ if( nList>pMsr->nBuffer ){
123151
+ char *pNew;
123152
+ pMsr->nBuffer = nList*2;
123153
+ pNew = (char *)sqlite3_realloc(pMsr->aBuffer, pMsr->nBuffer);
123154
+ if( !pNew ) return SQLITE_NOMEM;
123155
+ pMsr->aBuffer = pNew;
123156
+ }
123157
+
123158
+ memcpy(pMsr->aBuffer, pList, nList);
123159
+ return SQLITE_OK;
123160
+}
123161
+
123162
+SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext(
123163
+ Fts3Table *p, /* Virtual table handle */
123164
+ Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */
123165
+ sqlite3_int64 *piDocid, /* OUT: Docid value */
123166
+ char **paPoslist, /* OUT: Pointer to position list */
123167
+ int *pnPoslist /* OUT: Size of position list in bytes */
123168
+){
123169
+ int nMerge = pMsr->nAdvance;
123170
+ Fts3SegReader **apSegment = pMsr->apSegment;
123171
+ int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = (
123172
+ p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp
123173
+ );
123174
+
123175
+ if( nMerge==0 ){
123176
+ *paPoslist = 0;
123177
+ return SQLITE_OK;
123178
+ }
123179
+
123180
+ while( 1 ){
123181
+ Fts3SegReader *pSeg;
123182
+ pSeg = pMsr->apSegment[0];
123183
+
123184
+ if( pSeg->pOffsetList==0 ){
123185
+ *paPoslist = 0;
123186
+ break;
123187
+ }else{
123188
+ int rc;
123189
+ char *pList;
123190
+ int nList;
123191
+ int j;
123192
+ sqlite3_int64 iDocid = apSegment[0]->iDocid;
123193
+
123194
+ rc = fts3SegReaderNextDocid(p, apSegment[0], &pList, &nList);
123195
+ j = 1;
123196
+ while( rc==SQLITE_OK
123197
+ && j<nMerge
123198
+ && apSegment[j]->pOffsetList
123199
+ && apSegment[j]->iDocid==iDocid
123200
+ ){
123201
+ rc = fts3SegReaderNextDocid(p, apSegment[j], 0, 0);
123202
+ j++;
123203
+ }
123204
+ if( rc!=SQLITE_OK ) return rc;
123205
+ fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp);
123206
+
123207
+ if( pMsr->iColFilter>=0 ){
123208
+ fts3ColumnFilter(pMsr->iColFilter, &pList, &nList);
123209
+ }
123210
+
123211
+ if( nList>0 ){
123212
+ if( fts3SegReaderIsPending(apSegment[0]) ){
123213
+ rc = fts3MsrBufferData(pMsr, pList, nList+1);
123214
+ if( rc!=SQLITE_OK ) return rc;
123215
+ *paPoslist = pMsr->aBuffer;
123216
+ assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 );
123217
+ }else{
123218
+ *paPoslist = pList;
123219
+ }
123220
+ *piDocid = iDocid;
123221
+ *pnPoslist = nList;
123222
+ break;
123223
+ }
123224
+ }
123225
+ }
123226
+
123227
+ return SQLITE_OK;
123228
+}
123229
+
123230
+static int fts3SegReaderStart(
120812123231
 Fts3Table *p, /* Virtual table handle */
120813
- Fts3SegReaderCursor *pCsr, /* Cursor object */
120814
- Fts3SegFilter *pFilter /* Restrictions on range of iteration */
123232
+ Fts3MultiSegReader *pCsr, /* Cursor object */
123233
+ const char *zTerm, /* Term searched for (or NULL) */
123234
+ int nTerm /* Length of zTerm in bytes */
120815123235
 ){
120816123236
 int i;
120817
-
120818
- /* Initialize the cursor object */
120819
- pCsr->pFilter = pFilter;
123237
+ int nSeg = pCsr->nSegment;
120820123238
120821123239
 /* If the Fts3SegFilter defines a specific term (or term prefix) to search
120822123240
 ** for, then advance each segment iterator until it points to a term of
120823123241
 ** equal or greater value than the specified term. This prevents many
120824123242
 ** unnecessary merge/sort operations for the case where single segment
120825123243
 ** b-tree leaf nodes contain more than one term.
120826123244
 */
120827
- for(i=0; i<pCsr->nSegment; i++){
120828
- int nTerm = pFilter->nTerm;
120829
- const char *zTerm = pFilter->zTerm;
123245
+ for(i=0; pCsr->bRestart==0 && i<pCsr->nSegment; i++){
120830123246
 Fts3SegReader *pSeg = pCsr->apSegment[i];
120831123247
 do {
120832
- int rc = fts3SegReaderNext(p, pSeg);
123248
+ int rc = fts3SegReaderNext(p, pSeg, 0);
120833123249
 if( rc!=SQLITE_OK ) return rc;
120834123250
 }while( zTerm && fts3SegReaderTermCmp(pSeg, zTerm, nTerm)<0 );
120835123251
 }
120836
- fts3SegReaderSort(
120837
- pCsr->apSegment, pCsr->nSegment, pCsr->nSegment, fts3SegReaderCmp);
123252
+ fts3SegReaderSort(pCsr->apSegment, nSeg, nSeg, fts3SegReaderCmp);
123253
+
123254
+ return SQLITE_OK;
123255
+}
123256
+
123257
+SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(
123258
+ Fts3Table *p, /* Virtual table handle */
123259
+ Fts3MultiSegReader *pCsr, /* Cursor object */
123260
+ Fts3SegFilter *pFilter /* Restrictions on range of iteration */
123261
+){
123262
+ pCsr->pFilter = pFilter;
123263
+ return fts3SegReaderStart(p, pCsr, pFilter->zTerm, pFilter->nTerm);
123264
+}
123265
+
123266
+SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart(
123267
+ Fts3Table *p, /* Virtual table handle */
123268
+ Fts3MultiSegReader *pCsr, /* Cursor object */
123269
+ int iCol, /* Column to match on. */
123270
+ const char *zTerm, /* Term to iterate through a doclist for */
123271
+ int nTerm /* Number of bytes in zTerm */
123272
+){
123273
+ int i;
123274
+ int rc;
123275
+ int nSegment = pCsr->nSegment;
123276
+ int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = (
123277
+ p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp
123278
+ );
123279
+
123280
+ assert( pCsr->pFilter==0 );
123281
+ assert( zTerm && nTerm>0 );
123282
+
123283
+ /* Advance each segment iterator until it points to the term zTerm/nTerm. */
123284
+ rc = fts3SegReaderStart(p, pCsr, zTerm, nTerm);
123285
+ if( rc!=SQLITE_OK ) return rc;
123286
+
123287
+ /* Determine how many of the segments actually point to zTerm/nTerm. */
123288
+ for(i=0; i<nSegment; i++){
123289
+ Fts3SegReader *pSeg = pCsr->apSegment[i];
123290
+ if( !pSeg->aNode || fts3SegReaderTermCmp(pSeg, zTerm, nTerm) ){
123291
+ break;
123292
+ }
123293
+ }
123294
+ pCsr->nAdvance = i;
123295
+
123296
+ /* Advance each of the segments to point to the first docid. */
123297
+ for(i=0; i<pCsr->nAdvance; i++){
123298
+ rc = fts3SegReaderFirstDocid(p, pCsr->apSegment[i]);
123299
+ if( rc!=SQLITE_OK ) return rc;
123300
+ }
123301
+ fts3SegReaderSort(pCsr->apSegment, i, i, xCmp);
123302
+
123303
+ assert( iCol<0 || iCol<p->nColumn );
123304
+ pCsr->iColFilter = iCol;
123305
+
123306
+ return SQLITE_OK;
123307
+}
123308
+
123309
+/*
123310
+** This function is called on a MultiSegReader that has been started using
123311
+** sqlite3Fts3MsrIncrStart(). One or more calls to MsrIncrNext() may also
123312
+** have been made. Calling this function puts the MultiSegReader in such
123313
+** a state that if the next two calls are:
123314
+**
123315
+** sqlite3Fts3SegReaderStart()
123316
+** sqlite3Fts3SegReaderStep()
123317
+**
123318
+** then the entire doclist for the term is available in
123319
+** MultiSegReader.aDoclist/nDoclist.
123320
+*/
123321
+SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr){
123322
+ int i; /* Used to iterate through segment-readers */
123323
+
123324
+ assert( pCsr->zTerm==0 );
123325
+ assert( pCsr->nTerm==0 );
123326
+ assert( pCsr->aDoclist==0 );
123327
+ assert( pCsr->nDoclist==0 );
123328
+
123329
+ pCsr->nAdvance = 0;
123330
+ pCsr->bRestart = 1;
123331
+ for(i=0; i<pCsr->nSegment; i++){
123332
+ pCsr->apSegment[i]->pOffsetList = 0;
123333
+ pCsr->apSegment[i]->nOffsetList = 0;
123334
+ pCsr->apSegment[i]->iDocid = 0;
123335
+ }
120838123336
120839123337
 return SQLITE_OK;
120840123338
 }
123339
+
120841123340
120842123341
 SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(
120843123342
 Fts3Table *p, /* Virtual table handle */
120844
- Fts3SegReaderCursor *pCsr /* Cursor object */
123343
+ Fts3MultiSegReader *pCsr /* Cursor object */
120845123344
 ){
120846123345
 int rc = SQLITE_OK;
120847123346
120848123347
 int isIgnoreEmpty = (pCsr->pFilter->flags & FTS3_SEGMENT_IGNORE_EMPTY);
120849123348
 int isRequirePos = (pCsr->pFilter->flags & FTS3_SEGMENT_REQUIRE_POS);
@@ -120852,10 +123351,13 @@
120852123351
 int isScan = (pCsr->pFilter->flags & FTS3_SEGMENT_SCAN);
120853123352
120854123353
 Fts3SegReader **apSegment = pCsr->apSegment;
120855123354
 int nSegment = pCsr->nSegment;
120856123355
 Fts3SegFilter *pFilter = pCsr->pFilter;
123356
+ int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = (
123357
+ p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp
123358
+ );
120857123359
120858123360
 if( pCsr->nSegment==0 ) return SQLITE_OK;
120859123361
120860123362
 do {
120861123363
 int nMerge;
@@ -120863,11 +123365,11 @@
120863123365
120864123366
 /* Advance the first pCsr->nAdvance entries in the apSegment[] array
120865123367
 ** forward. Then sort the list in order of current term again.
120866123368
 */
120867123369
 for(i=0; i<pCsr->nAdvance; i++){
120868
- rc = fts3SegReaderNext(p, apSegment[i]);
123370
+ rc = fts3SegReaderNext(p, apSegment[i], 0);
120869123371
 if( rc!=SQLITE_OK ) return rc;
120870123372
 }
120871123373
 fts3SegReaderSort(apSegment, nSegment, pCsr->nAdvance, fts3SegReaderCmp);
120872123374
 pCsr->nAdvance = 0;
120873123375
@@ -120902,69 +123404,87 @@
120902123404
 ){
120903123405
 nMerge++;
120904123406
 }
120905123407
120906123408
 assert( isIgnoreEmpty || (isRequirePos && !isColFilter) );
120907
- if( nMerge==1 && !isIgnoreEmpty ){
120908
- pCsr->aDoclist = apSegment[0]->aDoclist;
123409
+ if( nMerge==1
123410
+ && !isIgnoreEmpty
123411
+ && (p->bDescIdx==0 || fts3SegReaderIsPending(apSegment[0])==0)
123412
+ ){
120909123413
 pCsr->nDoclist = apSegment[0]->nDoclist;
120910
- rc = SQLITE_ROW;
123414
+ if( fts3SegReaderIsPending(apSegment[0]) ){
123415
+ rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, pCsr->nDoclist);
123416
+ pCsr->aDoclist = pCsr->aBuffer;
123417
+ }else{
123418
+ pCsr->aDoclist = apSegment[0]->aDoclist;
123419
+ }
123420
+ if( rc==SQLITE_OK ) rc = SQLITE_ROW;
120911123421
 }else{
120912123422
 int nDoclist = 0; /* Size of doclist */
120913123423
 sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */
120914123424
120915123425
 /* The current term of the first nMerge entries in the array
120916123426
 ** of Fts3SegReader objects is the same. The doclists must be merged
120917123427
 ** and a single term returned with the merged doclist.
120918123428
 */
120919123429
 for(i=0; i<nMerge; i++){
120920
- fts3SegReaderFirstDocid(apSegment[i]);
123430
+ fts3SegReaderFirstDocid(p, apSegment[i]);
120921123431
 }
120922
- fts3SegReaderSort(apSegment, nMerge, nMerge, fts3SegReaderDoclistCmp);
123432
+ fts3SegReaderSort(apSegment, nMerge, nMerge, xCmp);
120923123433
 while( apSegment[0]->pOffsetList ){
120924123434
 int j; /* Number of segments that share a docid */
120925123435
 char *pList;
120926123436
 int nList;
120927123437
 int nByte;
120928123438
 sqlite3_int64 iDocid = apSegment[0]->iDocid;
120929
- fts3SegReaderNextDocid(apSegment[0], &pList, &nList);
123439
+ fts3SegReaderNextDocid(p, apSegment[0], &pList, &nList);
120930123440
 j = 1;
120931123441
 while( j<nMerge
120932123442
 && apSegment[j]->pOffsetList
120933123443
 && apSegment[j]->iDocid==iDocid
120934123444
 ){
120935
- fts3SegReaderNextDocid(apSegment[j], 0, 0);
123445
+ fts3SegReaderNextDocid(p, apSegment[j], 0, 0);
120936123446
 j++;
120937123447
 }
120938123448
120939123449
 if( isColFilter ){
120940123450
 fts3ColumnFilter(pFilter->iCol, &pList, &nList);
120941123451
 }
120942123452
120943123453
 if( !isIgnoreEmpty || nList>0 ){
120944
- nByte = sqlite3Fts3VarintLen(iDocid-iPrev) + (isRequirePos?nList+1:0);
123454
+
123455
+ /* Calculate the 'docid' delta value to write into the merged
123456
+ ** doclist. */
123457
+ sqlite3_int64 iDelta;
123458
+ if( p->bDescIdx && nDoclist>0 ){
123459
+ iDelta = iPrev - iDocid;
123460
+ }else{
123461
+ iDelta = iDocid - iPrev;
123462
+ }
123463
+ assert( iDelta>0 || (nDoclist==0 && iDelta==iDocid) );
123464
+ assert( nDoclist>0 || iDelta==iDocid );
123465
+
123466
+ nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0);
120945123467
 if( nDoclist+nByte>pCsr->nBuffer ){
120946123468
 char *aNew;
120947123469
 pCsr->nBuffer = (nDoclist+nByte)*2;
120948123470
 aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer);
120949123471
 if( !aNew ){
120950123472
 return SQLITE_NOMEM;
120951123473
 }
120952123474
 pCsr->aBuffer = aNew;
120953123475
 }
120954
- nDoclist += sqlite3Fts3PutVarint(
120955
- &pCsr->aBuffer[nDoclist], iDocid-iPrev
120956
- );
123476
+ nDoclist += sqlite3Fts3PutVarint(&pCsr->aBuffer[nDoclist], iDelta);
120957123477
 iPrev = iDocid;
120958123478
 if( isRequirePos ){
120959123479
 memcpy(&pCsr->aBuffer[nDoclist], pList, nList);
120960123480
 nDoclist += nList;
120961123481
 pCsr->aBuffer[nDoclist++] = '\0';
120962123482
 }
120963123483
 }
120964123484
120965
- fts3SegReaderSort(apSegment, nMerge, j, fts3SegReaderDoclistCmp);
123485
+ fts3SegReaderSort(apSegment, nMerge, j, xCmp);
120966123486
 }
120967123487
 if( nDoclist>0 ){
120968123488
 pCsr->aDoclist = pCsr->aBuffer;
120969123489
 pCsr->nDoclist = nDoclist;
120970123490
 rc = SQLITE_ROW;
@@ -120973,13 +123493,14 @@
120973123493
 pCsr->nAdvance = nMerge;
120974123494
 }while( rc==SQLITE_OK );
120975123495
120976123496
 return rc;
120977123497
 }
123498
+
120978123499
120979123500
 SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(
120980
- Fts3SegReaderCursor *pCsr /* Cursor object */
123501
+ Fts3MultiSegReader *pCsr /* Cursor object */
120981123502
 ){
120982123503
 if( pCsr ){
120983123504
 int i;
120984123505
 for(i=0; i<pCsr->nSegment; i++){
120985123506
 sqlite3Fts3SegReaderFree(pCsr->apSegment[i]);
@@ -121002,47 +123523,60 @@
121002123523
 ** If this function is called with iLevel<0, but there is only one
121003123524
 ** segment in the database, SQLITE_DONE is returned immediately.
121004123525
 ** Otherwise, if successful, SQLITE_OK is returned. If an error occurs,
121005123526
 ** an SQLite error code is returned.
121006123527
 */
121007
-static int fts3SegmentMerge(Fts3Table *p, int iLevel){
123528
+static int fts3SegmentMerge(Fts3Table *p, int iIndex, int iLevel){
121008123529
 int rc; /* Return code */
121009123530
 int iIdx = 0; /* Index of new segment */
121010
- int iNewLevel = 0; /* Level to create new segment at */
123531
+ int iNewLevel = 0; /* Level/index to create new segment at */
121011123532
 SegmentWriter *pWriter = 0; /* Used to write the new, merged, segment */
121012123533
 Fts3SegFilter filter; /* Segment term filter condition */
121013
- Fts3SegReaderCursor csr; /* Cursor to iterate through level(s) */
123534
+ Fts3MultiSegReader csr; /* Cursor to iterate through level(s) */
123535
+ int bIgnoreEmpty = 0; /* True to ignore empty segments */
121014123536
121015
- rc = sqlite3Fts3SegReaderCursor(p, iLevel, 0, 0, 1, 0, &csr);
123537
+ assert( iLevel==FTS3_SEGCURSOR_ALL
123538
+ || iLevel==FTS3_SEGCURSOR_PENDING
123539
+ || iLevel>=0
123540
+ );
123541
+ assert( iLevel<FTS3_SEGDIR_MAXLEVEL );
123542
+ assert( iIndex>=0 && iIndex<p->nIndex );
123543
+
123544
+ rc = sqlite3Fts3SegReaderCursor(p, iIndex, iLevel, 0, 0, 1, 0, &csr);
121016123545
 if( rc!=SQLITE_OK || csr.nSegment==0 ) goto finished;
121017123546
121018123547
 if( iLevel==FTS3_SEGCURSOR_ALL ){
121019123548
 /* This call is to merge all segments in the database to a single
121020123549
 ** segment. The level of the new segment is equal to the the numerically
121021
- ** greatest segment level currently present in the database. The index
121022
- ** of the new segment is always 0. */
121023
- int nDummy; /* TODO: Remove this */
123550
+ ** greatest segment level currently present in the database for this
123551
+ ** index. The idx of the new segment is always 0. */
121024123552
 if( csr.nSegment==1 ){
121025123553
 rc = SQLITE_DONE;
121026123554
 goto finished;
121027123555
 }
121028
- rc = fts3SegmentCountMax(p, &nDummy, &iNewLevel);
123556
+ rc = fts3SegmentMaxLevel(p, iIndex, &iNewLevel);
123557
+ bIgnoreEmpty = 1;
123558
+
123559
+ }else if( iLevel==FTS3_SEGCURSOR_PENDING ){
123560
+ iNewLevel = iIndex * FTS3_SEGDIR_MAXLEVEL;
123561
+ rc = fts3AllocateSegdirIdx(p, iIndex, 0, &iIdx);
121029123562
 }else{
121030
- /* This call is to merge all segments at level iLevel. Find the next
123563
+ /* This call is to merge all segments at level iLevel. find the next
121031123564
 ** available segment index at level iLevel+1. The call to
121032123565
 ** fts3AllocateSegdirIdx() will merge the segments at level iLevel+1 to
121033123566
 ** a single iLevel+2 segment if necessary. */
121034
- iNewLevel = iLevel+1;
121035
- rc = fts3AllocateSegdirIdx(p, iNewLevel, &iIdx);
123567
+ rc = fts3AllocateSegdirIdx(p, iIndex, iLevel+1, &iIdx);
123568
+ iNewLevel = iIndex * FTS3_SEGDIR_MAXLEVEL + iLevel+1;
121036123569
 }
121037123570
 if( rc!=SQLITE_OK ) goto finished;
121038123571
 assert( csr.nSegment>0 );
121039
- assert( iNewLevel>=0 );
123572
+ assert( iNewLevel>=(iIndex*FTS3_SEGDIR_MAXLEVEL) );
123573
+ assert( iNewLevel<((iIndex+1)*FTS3_SEGDIR_MAXLEVEL) );
121040123574
121041123575
 memset(&filter, 0, sizeof(Fts3SegFilter));
121042123576
 filter.flags = FTS3_SEGMENT_REQUIRE_POS;
121043
- filter.flags |= (iLevel==FTS3_SEGCURSOR_ALL ? FTS3_SEGMENT_IGNORE_EMPTY : 0);
123577
+ filter.flags |= (bIgnoreEmpty ? FTS3_SEGMENT_IGNORE_EMPTY : 0);
121044123578
121045123579
 rc = sqlite3Fts3SegReaderStart(p, &csr, &filter);
121046123580
 while( SQLITE_OK==rc ){
121047123581
 rc = sqlite3Fts3SegReaderStep(p, &csr);
121048123582
 if( rc!=SQLITE_ROW ) break;
@@ -121050,12 +123584,14 @@
121050123584
 csr.zTerm, csr.nTerm, csr.aDoclist, csr.nDoclist);
121051123585
 }
121052123586
 if( rc!=SQLITE_OK ) goto finished;
121053123587
 assert( pWriter );
121054123588
121055
- rc = fts3DeleteSegdir(p, iLevel, csr.apSegment, csr.nSegment);
121056
- if( rc!=SQLITE_OK ) goto finished;
123589
+ if( iLevel!=FTS3_SEGCURSOR_PENDING ){
123590
+ rc = fts3DeleteSegdir(p, iIndex, iLevel, csr.apSegment, csr.nSegment);
123591
+ if( rc!=SQLITE_OK ) goto finished;
123592
+ }
121057123593
 rc = fts3SegWriterFlush(p, pWriter, iNewLevel, iIdx);
121058123594
121059123595
 finished:
121060123596
 fts3SegWriterFree(pWriter);
121061123597
 sqlite3Fts3SegReaderFinish(&csr);
@@ -121062,14 +123598,21 @@
121062123598
 return rc;
121063123599
 }
121064123600
121065123601
121066123602
 /*
121067
-** Flush the contents of pendingTerms to a level 0 segment.
123603
+** Flush the contents of pendingTerms to level 0 segments.
121068123604
 */
121069123605
 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
121070
- return fts3SegmentMerge(p, FTS3_SEGCURSOR_PENDING);
123606
+ int rc = SQLITE_OK;
123607
+ int i;
123608
+ for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){
123609
+ rc = fts3SegmentMerge(p, i, FTS3_SEGCURSOR_PENDING);
123610
+ if( rc==SQLITE_DONE ) rc = SQLITE_OK;
123611
+ }
123612
+ sqlite3Fts3PendingTermsClear(p);
123613
+ return rc;
121071123614
 }
121072123615
121073123616
 /*
121074123617
 ** Encode N integers as varints into a blob.
121075123618
 */
@@ -121215,10 +123758,27 @@
121215123758
 sqlite3_bind_blob(pStmt, 1, pBlob, nBlob, SQLITE_STATIC);
121216123759
 sqlite3_step(pStmt);
121217123760
 *pRC = sqlite3_reset(pStmt);
121218123761
 sqlite3_free(a);
121219123762
 }
123763
+
123764
+static int fts3DoOptimize(Fts3Table *p, int bReturnDone){
123765
+ int i;
123766
+ int bSeenDone = 0;
123767
+ int rc = SQLITE_OK;
123768
+ for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){
123769
+ rc = fts3SegmentMerge(p, i, FTS3_SEGCURSOR_ALL);
123770
+ if( rc==SQLITE_DONE ){
123771
+ bSeenDone = 1;
123772
+ rc = SQLITE_OK;
123773
+ }
123774
+ }
123775
+ sqlite3Fts3SegmentsClose(p);
123776
+ sqlite3Fts3PendingTermsClear(p);
123777
+
123778
+ return (rc==SQLITE_OK && bReturnDone && bSeenDone) ? SQLITE_DONE : rc;
123779
+}
121220123780
121221123781
 /*
121222123782
 ** Handle a 'special' INSERT of the form:
121223123783
 **
121224123784
 ** "INSERT INTO tbl(tbl) VALUES(<expr>)"
@@ -121232,16 +123792,11 @@
121232123792
 int nVal = sqlite3_value_bytes(pVal);
121233123793
121234123794
 if( !zVal ){
121235123795
 return SQLITE_NOMEM;
121236123796
 }else if( nVal==8 && 0==sqlite3_strnicmp(zVal, "optimize", 8) ){
121237
- rc = fts3SegmentMerge(p, FTS3_SEGCURSOR_ALL);
121238
- if( rc==SQLITE_DONE ){
121239
- rc = SQLITE_OK;
121240
- }else{
121241
- sqlite3Fts3PendingTermsClear(p);
121242
- }
123797
+ rc = fts3DoOptimize(p, 0);
121243123798
 #ifdef SQLITE_TEST
121244123799
 }else if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
121245123800
 p->nNodeSize = atoi(&zVal[9]);
121246123801
 rc = SQLITE_OK;
121247123802
 }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){
@@ -121250,61 +123805,23 @@
121250123805
 #endif
121251123806
 }else{
121252123807
 rc = SQLITE_ERROR;
121253123808
 }
121254123809
121255
- sqlite3Fts3SegmentsClose(p);
121256123810
 return rc;
121257123811
 }
121258123812
121259
-/*
121260
-** Return the deferred doclist associated with deferred token pDeferred.
121261
-** This function assumes that sqlite3Fts3CacheDeferredDoclists() has already
121262
-** been called to allocate and populate the doclist.
121263
-*/
121264
-SQLITE_PRIVATE char *sqlite3Fts3DeferredDoclist(Fts3DeferredToken *pDeferred, int *pnByte){
121265
- if( pDeferred->pList ){
121266
- *pnByte = pDeferred->pList->nData;
121267
- return pDeferred->pList->aData;
121268
- }
121269
- *pnByte = 0;
121270
- return 0;
121271
-}
121272
-
121273
-/*
121274
-** Helper fucntion for FreeDeferredDoclists(). This function removes all
121275
-** references to deferred doclists from within the tree of Fts3Expr
121276
-** structures headed by
121277
-*/
121278
-static void fts3DeferredDoclistClear(Fts3Expr *pExpr){
121279
- if( pExpr ){
121280
- fts3DeferredDoclistClear(pExpr->pLeft);
121281
- fts3DeferredDoclistClear(pExpr->pRight);
121282
- if( pExpr->isLoaded ){
121283
- sqlite3_free(pExpr->aDoclist);
121284
- pExpr->isLoaded = 0;
121285
- pExpr->aDoclist = 0;
121286
- pExpr->nDoclist = 0;
121287
- pExpr->pCurrent = 0;
121288
- pExpr->iCurrent = 0;
121289
- }
121290
- }
121291
-}
121292
-
121293123813
 /*
121294123814
 ** Delete all cached deferred doclists. Deferred doclists are cached
121295123815
 ** (allocated) by the sqlite3Fts3CacheDeferredDoclists() function.
121296123816
 */
121297123817
 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *pCsr){
121298123818
 Fts3DeferredToken *pDef;
121299123819
 for(pDef=pCsr->pDeferred; pDef; pDef=pDef->pNext){
121300
- sqlite3_free(pDef->pList);
123820
+ fts3PendingListDelete(pDef->pList);
121301123821
 pDef->pList = 0;
121302123822
 }
121303
- if( pCsr->pDeferred ){
121304
- fts3DeferredDoclistClear(pCsr->pExpr);
121305
- }
121306123823
 }
121307123824
121308123825
 /*
121309123826
 ** Free all entries in the pCsr->pDeffered list. Entries are added to
121310123827
 ** this list using sqlite3Fts3DeferToken().
@@ -121312,11 +123829,11 @@
121312123829
 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *pCsr){
121313123830
 Fts3DeferredToken *pDef;
121314123831
 Fts3DeferredToken *pNext;
121315123832
 for(pDef=pCsr->pDeferred; pDef; pDef=pNext){
121316123833
 pNext = pDef->pNext;
121317
- sqlite3_free(pDef->pList);
123834
+ fts3PendingListDelete(pDef->pList);
121318123835
 sqlite3_free(pDef);
121319123836
 }
121320123837
 pCsr->pDeferred = 0;
121321123838
 }
121322123839
@@ -121376,10 +123893,37 @@
121376123893
 }
121377123894
 }
121378123895
121379123896
 return rc;
121380123897
 }
123898
+
123899
+SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(
123900
+ Fts3DeferredToken *p,
123901
+ char **ppData,
123902
+ int *pnData
123903
+){
123904
+ char *pRet;
123905
+ int nSkip;
123906
+ sqlite3_int64 dummy;
123907
+
123908
+ *ppData = 0;
123909
+ *pnData = 0;
123910
+
123911
+ if( p->pList==0 ){
123912
+ return SQLITE_OK;
123913
+ }
123914
+
123915
+ pRet = (char *)sqlite3_malloc(p->pList->nData);
123916
+ if( !pRet ) return SQLITE_NOMEM;
123917
+
123918
+ nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy);
123919
+ *pnData = p->pList->nData - nSkip;
123920
+ *ppData = pRet;
123921
+
123922
+ memcpy(pRet, &p->pList->aData[nSkip], *pnData);
123923
+ return SQLITE_OK;
123924
+}
121381123925
121382123926
 /*
121383123927
 ** Add an entry for token pToken to the pCsr->pDeferred list.
121384123928
 */
121385123929
 SQLITE_PRIVATE int sqlite3Fts3DeferToken(
@@ -121451,11 +123995,11 @@
121451123995
 ){
121452123996
 Fts3Table *p = (Fts3Table *)pVtab;
121453123997
 int rc = SQLITE_OK; /* Return Code */
121454123998
 int isRemove = 0; /* True for an UPDATE or DELETE */
121455123999
 sqlite3_int64 iRemove = 0; /* Rowid removed by UPDATE or DELETE */
121456
- u32 *aSzIns; /* Sizes of inserted documents */
124000
+ u32 *aSzIns = 0; /* Sizes of inserted documents */
121457124001
 u32 *aSzDel; /* Sizes of deleted documents */
121458124002
 int nChng = 0; /* Net change in number of documents */
121459124003
 int bInsertDone = 0;
121460124004
121461124005
 assert( p->pSegments==0 );
@@ -121466,16 +124010,20 @@
121466124010
 */
121467124011
 if( nArg>1
121468124012
 && sqlite3_value_type(apVal[0])==SQLITE_NULL
121469124013
 && sqlite3_value_type(apVal[p->nColumn+2])!=SQLITE_NULL
121470124014
 ){
121471
- return fts3SpecialInsert(p, apVal[p->nColumn+2]);
124015
+ rc = fts3SpecialInsert(p, apVal[p->nColumn+2]);
124016
+ goto update_out;
121472124017
 }
121473124018
121474124019
 /* Allocate space to hold the change in document sizes */
121475124020
 aSzIns = sqlite3_malloc( sizeof(aSzIns[0])*(p->nColumn+1)*2 );
121476
- if( aSzIns==0 ) return SQLITE_NOMEM;
124021
+ if( aSzIns==0 ){
124022
+ rc = SQLITE_NOMEM;
124023
+ goto update_out;
124024
+ }
121477124025
 aSzDel = &aSzIns[p->nColumn+1];
121478124026
 memset(aSzIns, 0, sizeof(aSzIns[0])*(p->nColumn+1)*2);
121479124027
121480124028
 /* If this is an INSERT operation, or an UPDATE that modifies the rowid
121481124029
 ** value, then this operation requires constraint handling.
@@ -121521,12 +124069,11 @@
121521124069
 bInsertDone = 1;
121522124070
 }
121523124071
 }
121524124072
 }
121525124073
 if( rc!=SQLITE_OK ){
121526
- sqlite3_free(aSzIns);
121527
- return rc;
124074
+ goto update_out;
121528124075
 }
121529124076
121530124077
 /* If this is a DELETE or UPDATE operation, remove the old record. */
121531124078
 if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
121532124079
 assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER );
@@ -121555,10 +124102,11 @@
121555124102
121556124103
 if( p->bHasStat ){
121557124104
 fts3UpdateDocTotals(&rc, p, aSzIns, aSzDel, nChng);
121558124105
 }
121559124106
124107
+ update_out:
121560124108
 sqlite3_free(aSzIns);
121561124109
 sqlite3Fts3SegmentsClose(p);
121562124110
 return rc;
121563124111
 }
121564124112
@@ -121569,16 +124117,14 @@
121569124117
 */
121570124118
 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *p){
121571124119
 int rc;
121572124120
 rc = sqlite3_exec(p->db, "SAVEPOINT fts3", 0, 0, 0);
121573124121
 if( rc==SQLITE_OK ){
121574
- rc = fts3SegmentMerge(p, FTS3_SEGCURSOR_ALL);
121575
- if( rc==SQLITE_OK ){
121576
- rc = sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
121577
- if( rc==SQLITE_OK ){
121578
- sqlite3Fts3PendingTermsClear(p);
121579
- }
124122
+ rc = fts3DoOptimize(p, 1);
124123
+ if( rc==SQLITE_OK || rc==SQLITE_DONE ){
124124
+ int rc2 = sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
124125
+ if( rc2!=SQLITE_OK ) rc = rc2;
121580124126
 }else{
121581124127
 sqlite3_exec(p->db, "ROLLBACK TO fts3", 0, 0, 0);
121582124128
 sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
121583124129
 }
121584124130
 }
@@ -121763,76 +124309,24 @@
121763124309
 ){
121764124310
 int iPhrase = 0; /* Variable used as the phrase counter */
121765124311
 return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx);
121766124312
 }
121767124313
121768
-/*
121769
-** The argument to this function is always a phrase node. Its doclist
121770
-** (Fts3Expr.aDoclist[]) and the doclists associated with all phrase nodes
121771
-** to the left of this one in the query tree have already been loaded.
121772
-**
121773
-** If this phrase node is part of a series of phrase nodes joined by
121774
-** NEAR operators (and is not the left-most of said series), then elements are
121775
-** removed from the phrases doclist consistent with the NEAR restriction. If
121776
-** required, elements may be removed from the doclists of phrases to the
121777
-** left of this one that are part of the same series of NEAR operator
121778
-** connected phrases.
121779
-**
121780
-** If an OOM error occurs, SQLITE_NOMEM is returned. Otherwise, SQLITE_OK.
121781
-*/
121782
-static int fts3ExprNearTrim(Fts3Expr *pExpr){
121783
- int rc = SQLITE_OK;
121784
- Fts3Expr *pParent = pExpr->pParent;
121785
-
121786
- assert( pExpr->eType==FTSQUERY_PHRASE );
121787
- while( rc==SQLITE_OK
121788
- && pParent
121789
- && pParent->eType==FTSQUERY_NEAR
121790
- && pParent->pRight==pExpr
121791
- ){
121792
- /* This expression (pExpr) is the right-hand-side of a NEAR operator.
121793
- ** Find the expression to the left of the same operator.
121794
- */
121795
- int nNear = pParent->nNear;
121796
- Fts3Expr *pLeft = pParent->pLeft;
121797
-
121798
- if( pLeft->eType!=FTSQUERY_PHRASE ){
121799
- assert( pLeft->eType==FTSQUERY_NEAR );
121800
- assert( pLeft->pRight->eType==FTSQUERY_PHRASE );
121801
- pLeft = pLeft->pRight;
121802
- }
121803
-
121804
- rc = sqlite3Fts3ExprNearTrim(pLeft, pExpr, nNear);
121805
-
121806
- pExpr = pLeft;
121807
- pParent = pExpr->pParent;
121808
- }
121809
-
121810
- return rc;
121811
-}
121812
-
121813124314
 /*
121814124315
 ** This is an fts3ExprIterate() callback used while loading the doclists
121815124316
 ** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
121816124317
 ** fts3ExprLoadDoclists().
121817124318
 */
121818124319
 static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
121819124320
 int rc = SQLITE_OK;
124321
+ Fts3Phrase *pPhrase = pExpr->pPhrase;
121820124322
 LoadDoclistCtx *p = (LoadDoclistCtx *)ctx;
121821124323
121822124324
 UNUSED_PARAMETER(iPhrase);
121823124325
121824124326
 p->nPhrase++;
121825
- p->nToken += pExpr->pPhrase->nToken;
121826
-
121827
- if( pExpr->isLoaded==0 ){
121828
- rc = sqlite3Fts3ExprLoadDoclist(p->pCsr, pExpr);
121829
- pExpr->isLoaded = 1;
121830
- if( rc==SQLITE_OK ){
121831
- rc = fts3ExprNearTrim(pExpr);
121832
- }
121833
- }
124327
+ p->nToken += pPhrase->nToken;
121834124328
121835124329
 return rc;
121836124330
 }
121837124331
121838124332
 /*
@@ -122002,11 +124496,11 @@
122002124496
 SnippetPhrase *pPhrase = &p->aPhrase[iPhrase];
122003124497
 char *pCsr;
122004124498
122005124499
 pPhrase->nToken = pExpr->pPhrase->nToken;
122006124500
122007
- pCsr = sqlite3Fts3FindPositions(p->pCsr, pExpr, p->pCsr->iPrevId, p->iCol);
124501
+ pCsr = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol);
122008124502
 if( pCsr ){
122009124503
 int iFirst = 0;
122010124504
 pPhrase->pList = pCsr;
122011124505
 fts3GetDeltaPosition(&pCsr, &iFirst);
122012124506
 pPhrase->pHead = pCsr;
@@ -122359,30 +124853,10 @@
122359124853
122360124854
 *ppCollist = pEnd;
122361124855
 return nEntry;
122362124856
 }
122363124857
122364
-static void fts3LoadColumnlistCounts(char **pp, u32 *aOut, int isGlobal){
122365
- char *pCsr = *pp;
122366
- while( *pCsr ){
122367
- int nHit;
122368
- sqlite3_int64 iCol = 0;
122369
- if( *pCsr==0x01 ){
122370
- pCsr++;
122371
- pCsr += sqlite3Fts3GetVarint(pCsr, &iCol);
122372
- }
122373
- nHit = fts3ColumnlistCount(&pCsr);
122374
- assert( nHit>0 );
122375
- if( isGlobal ){
122376
- aOut[iCol*3+1]++;
122377
- }
122378
- aOut[iCol*3] += nHit;
122379
- }
122380
- pCsr++;
122381
- *pp = pCsr;
122382
-}
122383
-
122384124858
 /*
122385124859
 ** fts3ExprIterate() callback used to collect the "global" matchinfo stats
122386124860
 ** for a single query.
122387124861
 **
122388124862
 ** fts3ExprIterate() callback to load the 'global' elements of a
@@ -122412,52 +124886,13 @@
122412124886
 Fts3Expr *pExpr, /* Phrase expression node */
122413124887
 int iPhrase, /* Phrase number (numbered from zero) */
122414124888
 void *pCtx /* Pointer to MatchInfo structure */
122415124889
 ){
122416124890
 MatchInfo *p = (MatchInfo *)pCtx;
122417
- Fts3Cursor *pCsr = p->pCursor;
122418
- char *pIter;
122419
- char *pEnd;
122420
- char *pFree = 0;
122421
- u32 *aOut = &p->aMatchinfo[3*iPhrase*p->nCol];
122422
-
122423
- assert( pExpr->isLoaded );
122424
- assert( pExpr->eType==FTSQUERY_PHRASE );
122425
-
122426
- if( pCsr->pDeferred ){
122427
- Fts3Phrase *pPhrase = pExpr->pPhrase;
122428
- int ii;
122429
- for(ii=0; ii<pPhrase->nToken; ii++){
122430
- if( pPhrase->aToken[ii].bFulltext ) break;
122431
- }
122432
- if( ii<pPhrase->nToken ){
122433
- int nFree = 0;
122434
- int rc = sqlite3Fts3ExprLoadFtDoclist(pCsr, pExpr, &pFree, &nFree);
122435
- if( rc!=SQLITE_OK ) return rc;
122436
- pIter = pFree;
122437
- pEnd = &pFree[nFree];
122438
- }else{
122439
- int iCol; /* Column index */
122440
- for(iCol=0; iCol<p->nCol; iCol++){
122441
- aOut[iCol*3 + 1] = (u32)p->nDoc;
122442
- aOut[iCol*3 + 2] = (u32)p->nDoc;
122443
- }
122444
- return SQLITE_OK;
122445
- }
122446
- }else{
122447
- pIter = pExpr->aDoclist;
122448
- pEnd = &pExpr->aDoclist[pExpr->nDoclist];
122449
- }
122450
-
122451
- /* Fill in the global hit count matrix row for this phrase. */
122452
- while( pIter<pEnd ){
122453
- while( *pIter++ & 0x80 ); /* Skip past docid. */
122454
- fts3LoadColumnlistCounts(&pIter, &aOut[1], 1);
122455
- }
122456
-
122457
- sqlite3_free(pFree);
122458
- return SQLITE_OK;
124891
+ return sqlite3Fts3EvalPhraseStats(
124892
+ p->pCursor, pExpr, &p->aMatchinfo[3*iPhrase*p->nCol]
124893
+ );
122459124894
 }
122460124895
122461124896
 /*
122462124897
 ** fts3ExprIterate() callback used to collect the "local" part of the
122463124898
 ** FTS3_MATCHINFO_HITS array. The local stats are those elements of the
@@ -122470,18 +124905,17 @@
122470124905
 ){
122471124906
 MatchInfo *p = (MatchInfo *)pCtx;
122472124907
 int iStart = iPhrase * p->nCol * 3;
122473124908
 int i;
122474124909
122475
- for(i=0; i<p->nCol; i++) p->aMatchinfo[iStart+i*3] = 0;
122476
-
122477
- if( pExpr->aDoclist ){
124910
+ for(i=0; i<p->nCol; i++){
122478124911
 char *pCsr;
122479
-
122480
- pCsr = sqlite3Fts3FindPositions(p->pCursor, pExpr, p->pCursor->iPrevId, -1);
124912
+ pCsr = sqlite3Fts3EvalPhrasePoslist(p->pCursor, pExpr, i);
122481124913
 if( pCsr ){
122482
- fts3LoadColumnlistCounts(&pCsr, &p->aMatchinfo[iStart], 0);
124914
+ p->aMatchinfo[iStart+i*3] = fts3ColumnlistCount(&pCsr);
124915
+ }else{
124916
+ p->aMatchinfo[iStart+i*3] = 0;
122483124917
 }
122484124918
 }
122485124919
122486124920
 return SQLITE_OK;
122487124921
 }
@@ -122563,13 +124997,12 @@
122563124997
 ** values for a matchinfo() FTS3_MATCHINFO_LCS request.
122564124998
 */
122565124999
 typedef struct LcsIterator LcsIterator;
122566125000
 struct LcsIterator {
122567125001
 Fts3Expr *pExpr; /* Pointer to phrase expression */
122568
- char *pRead; /* Cursor used to iterate through aDoclist */
122569125002
 int iPosOffset; /* Tokens count up to end of this phrase */
122570
- int iCol; /* Current column number */
125003
+ char *pRead; /* Cursor used to iterate through aDoclist */
122571125004
 int iPos; /* Current position */
122572125005
 };
122573125006
122574125007
 /*
122575125008
 ** If LcsIterator.iCol is set to the following value, the iterator has
@@ -122596,21 +125029,14 @@
122596125029
 char *pRead = pIter->pRead;
122597125030
 sqlite3_int64 iRead;
122598125031
 int rc = 0;
122599125032
122600125033
 pRead += sqlite3Fts3GetVarint(pRead, &iRead);
122601
- if( iRead==0 ){
122602
- pIter->iCol = LCS_ITERATOR_FINISHED;
125034
+ if( iRead==0 || iRead==1 ){
125035
+ pRead = 0;
122603125036
 rc = 1;
122604125037
 }else{
122605
- if( iRead==1 ){
122606
- pRead += sqlite3Fts3GetVarint(pRead, &iRead);
122607
- pIter->iCol = (int)iRead;
122608
- pIter->iPos = pIter->iPosOffset;
122609
- pRead += sqlite3Fts3GetVarint(pRead, &iRead);
122610
- rc = 1;
122611
- }
122612125038
 pIter->iPos += (int)(iRead-2);
122613125039
 }
122614125040
122615125041
 pIter->pRead = pRead;
122616125042
 return rc;
@@ -122638,46 +125064,38 @@
122638125064
 **/
122639125065
 aIter = sqlite3_malloc(sizeof(LcsIterator) * pCsr->nPhrase);
122640125066
 if( !aIter ) return SQLITE_NOMEM;
122641125067
 memset(aIter, 0, sizeof(LcsIterator) * pCsr->nPhrase);
122642125068
 (void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
125069
+
122643125070
 for(i=0; i<pInfo->nPhrase; i++){
122644125071
 LcsIterator *pIter = &aIter[i];
122645125072
 nToken -= pIter->pExpr->pPhrase->nToken;
122646125073
 pIter->iPosOffset = nToken;
122647
- pIter->pRead = sqlite3Fts3FindPositions(pCsr,pIter->pExpr,pCsr->iPrevId,-1);
122648
- if( pIter->pRead ){
122649
- pIter->iPos = pIter->iPosOffset;
122650
- fts3LcsIteratorAdvance(&aIter[i]);
122651
- }else{
122652
- pIter->iCol = LCS_ITERATOR_FINISHED;
122653
- }
122654125074
 }
122655125075
122656125076
 for(iCol=0; iCol<pInfo->nCol; iCol++){
122657125077
 int nLcs = 0; /* LCS value for this column */
122658125078
 int nLive = 0; /* Number of iterators in aIter not at EOF */
122659125079
122660
- /* Loop through the iterators in aIter[]. Set nLive to the number of
122661
- ** iterators that point to a position-list corresponding to column iCol.
122662
- */
122663125080
 for(i=0; i<pInfo->nPhrase; i++){
122664
- assert( aIter[i].iCol>=iCol );
122665
- if( aIter[i].iCol==iCol ) nLive++;
125081
+ LcsIterator *pIt = &aIter[i];
125082
+ pIt->pRead = sqlite3Fts3EvalPhrasePoslist(pCsr, pIt->pExpr, iCol);
125083
+ if( pIt->pRead ){
125084
+ pIt->iPos = pIt->iPosOffset;
125085
+ fts3LcsIteratorAdvance(&aIter[i]);
125086
+ nLive++;
125087
+ }
122666125088
 }
122667125089
122668
- /* The following loop runs until all iterators in aIter[] have finished
122669
- ** iterating through positions in column iCol. Exactly one of the
122670
- ** iterators is advanced each time the body of the loop is run.
122671
- */
122672125090
 while( nLive>0 ){
122673125091
 LcsIterator *pAdv = 0; /* The iterator to advance by one position */
122674125092
 int nThisLcs = 0; /* LCS for the current iterator positions */
122675125093
122676125094
 for(i=0; i<pInfo->nPhrase; i++){
122677125095
 LcsIterator *pIter = &aIter[i];
122678
- if( iCol!=pIter->iCol ){
125096
+ if( pIter->pRead==0 ){
122679125097
 /* This iterator is already at EOF for this column. */
122680125098
 nThisLcs = 0;
122681125099
 }else{
122682125100
 if( pAdv==0 || pIter->iPos<pAdv->iPos ){
122683125101
 pAdv = pIter;
@@ -122739,11 +125157,11 @@
122739125157
 if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol;
122740125158
 break;
122741125159
122742125160
 case FTS3_MATCHINFO_NDOC:
122743125161
 if( bGlobal ){
122744
- sqlite3_int64 nDoc;
125162
+ sqlite3_int64 nDoc = 0;
122745125163
 rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0);
122746125164
 pInfo->aMatchinfo[0] = (u32)nDoc;
122747125165
 }
122748125166
 break;
122749125167
@@ -123013,11 +125431,11 @@
123013125431
 int iTerm; /* For looping through nTerm phrase terms */
123014125432
 char *pList; /* Pointer to position list for phrase */
123015125433
 int iPos = 0; /* First position in position-list */
123016125434
123017125435
 UNUSED_PARAMETER(iPhrase);
123018
- pList = sqlite3Fts3FindPositions(p->pCsr, pExpr, p->iDocid, p->iCol);
125436
+ pList = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol);
123019125437
 nTerm = pExpr->pPhrase->nToken;
123020125438
 if( pList ){
123021125439
 fts3GetDeltaPosition(&pList, &iPos);
123022125440
 assert( iPos>=0 );
123023125441
 }
@@ -124634,11 +127052,11 @@
124634127052
 */
124635127053
 static float cellArea(Rtree *pRtree, RtreeCell *p){
124636127054
 float area = 1.0;
124637127055
 int ii;
124638127056
 for(ii=0; ii<(pRtree->nDim*2); ii+=2){
124639
- area = area * (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]));
127057
+ area = (float)(area * (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii])));
124640127058
 }
124641127059
 return area;
124642127060
 }
124643127061
124644127062
 /*
@@ -124647,11 +127065,11 @@
124647127065
 */
124648127066
 static float cellMargin(Rtree *pRtree, RtreeCell *p){
124649127067
 float margin = 0.0;
124650127068
 int ii;
124651127069
 for(ii=0; ii<(pRtree->nDim*2); ii+=2){
124652
- margin += (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]));
127070
+ margin += (float)(DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]));
124653127071
 }
124654127072
 return margin;
124655127073
 }
124656127074
124657127075
 /*
@@ -124732,11 +127150,11 @@
124732127150
124733127151
 if( x2<x1 ){
124734127152
 o = 0.0;
124735127153
 break;
124736127154
 }else{
124737
- o = o * (x2-x1);
127155
+ o = o * (float)(x2-x1);
124738127156
 }
124739127157
 }
124740127158
 overlap += o;
124741127159
 }
124742127160
 }
@@ -124751,16 +127169,16 @@
124751127169
 RtreeCell *pInsert,
124752127170
 RtreeCell *aCell,
124753127171
 int nCell,
124754127172
 int iExclude
124755127173
 ){
124756
- float before;
124757
- float after;
127174
+ double before;
127175
+ double after;
124758127176
 before = cellOverlap(pRtree, p, aCell, nCell, iExclude);
124759127177
 cellUnion(pRtree, p, pInsert);
124760127178
 after = cellOverlap(pRtree, p, aCell, nCell, iExclude);
124761
- return after-before;
127179
+ return (float)(after-before);
124762127180
 }
124763127181
 #endif
124764127182
124765127183
124766127184
 /*
@@ -124778,15 +127196,15 @@
124778127196
 RtreeNode *pNode;
124779127197
 rc = nodeAcquire(pRtree, 1, 0, &pNode);
124780127198
124781127199
 for(ii=0; rc==SQLITE_OK && ii<(pRtree->iDepth-iHeight); ii++){
124782127200
 int iCell;
124783
- sqlite3_int64 iBest;
127201
+ sqlite3_int64 iBest = 0;
124784127202
124785
- float fMinGrowth;
124786
- float fMinArea;
124787
- float fMinOverlap;
127203
+ float fMinGrowth = 0.0;
127204
+ float fMinArea = 0.0;
127205
+ float fMinOverlap = 0.0;
124788127206
124789127207
 int nCell = NCELL(pNode);
124790127208
 RtreeCell cell;
124791127209
 RtreeNode *pChild;
124792127210
@@ -125212,13 +127630,13 @@
125212127630
 ){
125213127631
 int **aaSorted;
125214127632
 int *aSpare;
125215127633
 int ii;
125216127634
125217
- int iBestDim;
125218
- int iBestSplit;
125219
- float fBestMargin;
127635
+ int iBestDim = 0;
127636
+ int iBestSplit = 0;
127637
+ float fBestMargin = 0.0;
125220127638
125221127639
 int nByte = (pRtree->nDim+1)*(sizeof(int*)+nCell*sizeof(int));
125222127640
125223127641
 aaSorted = (int **)sqlite3_malloc(nByte);
125224127642
 if( !aaSorted ){
@@ -125236,13 +127654,13 @@
125236127654
 SortByDimension(pRtree, aaSorted[ii], nCell, ii, aCell, aSpare);
125237127655
 }
125238127656
125239127657
 for(ii=0; ii<pRtree->nDim; ii++){
125240127658
 float margin = 0.0;
125241
- float fBestOverlap;
125242
- float fBestArea;
125243
- int iBestLeft;
127659
+ float fBestOverlap = 0.0;
127660
+ float fBestArea = 0.0;
127661
+ int iBestLeft = 0;
125244127662
 int nLeft;
125245127663
125246127664
 for(
125247127665
 nLeft=RTREE_MINCELLS(pRtree);
125248127666
 nLeft<=(nCell-RTREE_MINCELLS(pRtree));
@@ -125553,11 +127971,11 @@
125553127971
 static int deleteCell(Rtree *, RtreeNode *, int, int);
125554127972
125555127973
 static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){
125556127974
 int rc;
125557127975
 int rc2;
125558
- RtreeNode *pParent;
127976
+ RtreeNode *pParent = 0;
125559127977
 int iCell;
125560127978
125561127979
 assert( pNode->nRef==1 );
125562127980
125563127981
 /* Remove the entry in the parent cell. */
@@ -125701,23 +128119,23 @@
125701128119
 }else{
125702128120
 nodeGetCell(pRtree, pNode, ii, &aCell[ii]);
125703128121
 }
125704128122
 aOrder[ii] = ii;
125705128123
 for(iDim=0; iDim<pRtree->nDim; iDim++){
125706
- aCenterCoord[iDim] += DCOORD(aCell[ii].aCoord[iDim*2]);
125707
- aCenterCoord[iDim] += DCOORD(aCell[ii].aCoord[iDim*2+1]);
128124
+ aCenterCoord[iDim] += (float)DCOORD(aCell[ii].aCoord[iDim*2]);
128125
+ aCenterCoord[iDim] += (float)DCOORD(aCell[ii].aCoord[iDim*2+1]);
125708128126
 }
125709128127
 }
125710128128
 for(iDim=0; iDim<pRtree->nDim; iDim++){
125711
- aCenterCoord[iDim] = aCenterCoord[iDim]/((float)nCell*2.0);
128129
+ aCenterCoord[iDim] = (float)(aCenterCoord[iDim]/((float)nCell*2.0));
125712128130
 }
125713128131
125714128132
 for(ii=0; ii<nCell; ii++){
125715128133
 aDistance[ii] = 0.0;
125716128134
 for(iDim=0; iDim<pRtree->nDim; iDim++){
125717
- float coord = DCOORD(aCell[ii].aCoord[iDim*2+1]) -
125718
- DCOORD(aCell[ii].aCoord[iDim*2]);
128135
+ float coord = (float)(DCOORD(aCell[ii].aCoord[iDim*2+1]) -
128136
+ DCOORD(aCell[ii].aCoord[iDim*2]));
125719128137
 aDistance[ii] += (coord-aCenterCoord[iDim])*(coord-aCenterCoord[iDim]);
125720128138
 }
125721128139
 }
125722128140
125723128141
 SortByDistance(aOrder, nCell, aDistance, aSpare);
@@ -125812,14 +128230,14 @@
125812128230
 nodeGetCell(pRtree, pNode, ii, &cell);
125813128231
125814128232
 /* Find a node to store this cell in. pNode->iNode currently contains
125815128233
 ** the height of the sub-tree headed by the cell.
125816128234
 */
125817
- rc = ChooseLeaf(pRtree, &cell, pNode->iNode, &pInsert);
128235
+ rc = ChooseLeaf(pRtree, &cell, (int)pNode->iNode, &pInsert);
125818128236
 if( rc==SQLITE_OK ){
125819128237
 int rc2;
125820
- rc = rtreeInsertCell(pRtree, pInsert, &cell, pNode->iNode);
128238
+ rc = rtreeInsertCell(pRtree, pInsert, &cell, (int)pNode->iNode);
125821128239
 rc2 = nodeRelease(pRtree, pInsert);
125822128240
 if( rc==SQLITE_OK ){
125823128241
 rc = rc2;
125824128242
 }
125825128243
 }
@@ -126204,11 +128622,11 @@
126204128622
 int isCreate /* True for xCreate, false for xConnect */
126205128623
 ){
126206128624
 int rc;
126207128625
 char *zSql;
126208128626
 if( isCreate ){
126209
- int iPageSize;
128627
+ int iPageSize = 0;
126210128628
 zSql = sqlite3_mprintf("PRAGMA %Q.page_size", pRtree->zDb);
126211128629
 rc = getIntFromStmt(db, zSql, &iPageSize);
126212128630
 if( rc==SQLITE_OK ){
126213128631
 pRtree->iNodeSize = iPageSize-64;
126214128632
 if( (4+pRtree->nBytesPerCell*RTREE_MAXCELLS)<pRtree->iNodeSize ){
@@ -127007,14 +129425,11 @@
127007129425
 ** May you find forgiveness for yourself and forgive others.
127008129426
 ** May you share freely, never taking more than you give.
127009129427
 **
127010129428
 *************************************************************************
127011129429
 ** This file implements a tokenizer for fts3 based on the ICU library.
127012
-**
127013
-** $Id: fts3_icu.c,v 1.3 2008/09/01 18:34:20 danielk1977 Exp $
127014129430
 */
127015
-
127016129431
 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
127017129432
 #ifdef SQLITE_ENABLE_ICU
127018129433
127019129434
127020129435
 #include <unicode/ubrk.h>
127021129436
--- src/sqlite3.c
+++ src/sqlite3.c
@@ -1,8 +1,8 @@
1 /******************************************************************************
2 ** This file is an amalgamation of many separate C source files from SQLite
3 ** version 3.7.7. By combining all the individual C code files into this
4 ** single large file, the entire code can be compiled as a single translation
5 ** unit. This allows many compilers to do optimizations that would not be
6 ** possible if the files were compiled separately. Performance improvements
7 ** of 5% or more are commonly seen when SQLite is compiled as a single
8 ** translation unit.
@@ -648,13 +648,13 @@
648 **
649 ** See also: [sqlite3_libversion()],
650 ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
651 ** [sqlite_version()] and [sqlite_source_id()].
652 */
653 #define SQLITE_VERSION "3.7.7"
654 #define SQLITE_VERSION_NUMBER 3007007
655 #define SQLITE_SOURCE_ID "2011-06-03 14:19:10 0206bc6f87bb9393218a380fc5b18039d334a8d8"
656
657 /*
658 ** CAPI3REF: Run-Time Library Version Numbers
659 ** KEYWORDS: sqlite3_version, sqlite3_sourceid
660 **
@@ -851,11 +851,11 @@
851 ** semicolon-separate SQL statements passed into its 2nd argument,
852 ** in the context of the [database connection] passed in as its 1st
853 ** argument. ^If the callback function of the 3rd argument to
854 ** sqlite3_exec() is not NULL, then it is invoked for each result row
855 ** coming out of the evaluated SQL statements. ^The 4th argument to
856 ** to sqlite3_exec() is relayed through to the 1st argument of each
857 ** callback invocation. ^If the callback pointer to sqlite3_exec()
858 ** is NULL, then no callback is ever invoked and result rows are
859 ** ignored.
860 **
861 ** ^If an error occurs while evaluating the SQL statements passed into
@@ -1282,20 +1282,52 @@
1282 ** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most
1283 ** VFSes do not need this signal and should silently ignore this opcode.
1284 ** Applications should not call [sqlite3_file_control()] with this
1285 ** opcode as doing so may disrupt the operation of the specialized VFSes
1286 ** that do require it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1287 */
1288 #define SQLITE_FCNTL_LOCKSTATE 1
1289 #define SQLITE_GET_LOCKPROXYFILE 2
1290 #define SQLITE_SET_LOCKPROXYFILE 3
1291 #define SQLITE_LAST_ERRNO 4
1292 #define SQLITE_FCNTL_SIZE_HINT 5
1293 #define SQLITE_FCNTL_CHUNK_SIZE 6
1294 #define SQLITE_FCNTL_FILE_POINTER 7
1295 #define SQLITE_FCNTL_SYNC_OMITTED 8
1296
 
1297
1298 /*
1299 ** CAPI3REF: Mutex Handle
1300 **
1301 ** The mutex module within SQLite defines [sqlite3_mutex] to be an
@@ -1443,11 +1475,11 @@
1443 ** The xSleep() method causes the calling thread to sleep for at
1444 ** least the number of microseconds given. ^The xCurrentTime()
1445 ** method returns a Julian Day Number for the current date and time as
1446 ** a floating point value.
1447 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian
1448 ** Day Number multipled by 86400000 (the number of milliseconds in
1449 ** a 24-hour day).
1450 ** ^SQLite will use the xCurrentTimeInt64() method to get the current
1451 ** date and time if that method is available (if iVersion is 2 or
1452 ** greater and the function pointer is not NULL) and will fall back
1453 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable.
@@ -1881,11 +1913,11 @@
1881 ** scratch memory beyond what is provided by this configuration option, then
1882 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd>
1883 **
1884 ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt>
1885 ** <dd> ^This option specifies a static memory buffer that SQLite can use for
1886 ** the database page cache with the default page cache implemenation.
1887 ** This configuration should not be used if an application-define page
1888 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option.
1889 ** There are three arguments to this option: A pointer to 8-byte aligned
1890 ** memory, the size of each page buffer (sz), and the number of pages (N).
1891 ** The sz argument should be the size of the largest database page
@@ -2979,16 +3011,16 @@
2979 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
2980 **
2981 ** ^If [URI filename] interpretation is enabled, and the filename argument
2982 ** begins with "file:", then the filename is interpreted as a URI. ^URI
2983 ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is
2984 ** is set in the fourth argument to sqlite3_open_v2(), or if it has
2985 ** been enabled globally using the [SQLITE_CONFIG_URI] option with the
2986 ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.
2987 ** As of SQLite version 3.7.7, URI filename interpretation is turned off
2988 ** by default, but future releases of SQLite might enable URI filename
2989 ** intepretation by default. See "[URI filenames]" for additional
2990 ** information.
2991 **
2992 ** URI filenames are parsed according to RFC 3986. ^If the URI contains an
2993 ** authority, then it must be either an empty string or the string
2994 ** "localhost". ^If the authority is not an empty string or "localhost", an
@@ -3365,11 +3397,11 @@
3365 ** a schema change, on the first [sqlite3_step()] call following any change
3366 ** to the [sqlite3_bind_text | bindings] of that [parameter].
3367 ** ^The specific value of WHERE-clause [parameter] might influence the
3368 ** choice of query plan if the parameter is the left-hand side of a [LIKE]
3369 ** or [GLOB] operator or if the parameter is compared to an indexed column
3370 ** and the [SQLITE_ENABLE_STAT2] compile-time option is enabled.
3371 ** the
3372 ** </li>
3373 ** </ol>
3374 */
3375 SQLITE_API int sqlite3_prepare(
@@ -3803,11 +3835,11 @@
3803 ** [extended result codes] might be returned as well.
3804 **
3805 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the
3806 ** database locks it needs to do its job. ^If the statement is a [COMMIT]
3807 ** or occurs outside of an explicit transaction, then you can retry the
3808 ** statement. If the statement is not a [COMMIT] and occurs within a
3809 ** explicit transaction then you should rollback the transaction before
3810 ** continuing.
3811 **
3812 ** ^[SQLITE_DONE] means that the statement has finished executing
3813 ** successfully. sqlite3_step() should not be called again on this virtual
@@ -4082,11 +4114,11 @@
4082
4083 /*
4084 ** CAPI3REF: Destroy A Prepared Statement Object
4085 **
4086 ** ^The sqlite3_finalize() function is called to delete a [prepared statement].
4087 ** ^If the most recent evaluation of the statement encountered no errors or
4088 ** or if the statement is never been evaluated, then sqlite3_finalize() returns
4089 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then
4090 ** sqlite3_finalize(S) returns the appropriate [error code] or
4091 ** [extended error code].
4092 **
@@ -5996,11 +6028,11 @@
5996 ** versions of these routines, it should at least provide stubs that always
5997 ** return true so that one does not get spurious assertion failures.
5998 **
5999 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then
6000 ** the routine should return 1. This seems counter-intuitive since
6001 ** clearly the mutex cannot be held if it does not exist. But the
6002 ** the reason the mutex does not exist is because the build is not
6003 ** using mutexes. And we do not want the assert() containing the
6004 ** call to sqlite3_mutex_held() to fail, so a non-zero return is
6005 ** the appropriate thing to do. ^The sqlite3_mutex_notheld()
6006 ** interface should also return 1 when given a NULL pointer.
@@ -6119,11 +6151,12 @@
6119 #define SQLITE_TESTCTRL_RESERVE 14
6120 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15
6121 #define SQLITE_TESTCTRL_ISKEYWORD 16
6122 #define SQLITE_TESTCTRL_PGHDRSZ 17
6123 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18
6124 #define SQLITE_TESTCTRL_LAST 18
 
6125
6126 /*
6127 ** CAPI3REF: SQLite Runtime Status
6128 **
6129 ** ^This interface is used to retrieve runtime status information
@@ -6505,11 +6538,11 @@
6505 ** [[the xFetch() page cache methods]]
6506 ** The xFetch() method locates a page in the cache and returns a pointer to
6507 ** the page, or a NULL pointer.
6508 ** A "page", in this context, means a buffer of szPage bytes aligned at an
6509 ** 8-byte boundary. The page to be fetched is determined by the key. ^The
6510 ** mimimum key value is 1. After it has been retrieved using xFetch, the page
6511 ** is considered to be "pinned".
6512 **
6513 ** If the requested page is already in the page cache, then the page cache
6514 ** implementation must return a pointer to the page buffer with its content
6515 ** intact. If the requested page is not already in the cache, then the
@@ -7678,10 +7711,22 @@
7678 ** is 0x00000000ffffffff. But because of quirks of some compilers, we
7679 ** have to specify the value in the less intuitive manner shown:
7680 */
7681 #define SQLITE_MAX_U32 ((((u64)1)<<32)-1)
7682
 
 
 
 
 
 
 
 
 
 
 
 
7683 /*
7684 ** Macros to determine whether the machine is big or little endian,
7685 ** evaluated at runtime.
7686 */
7687 #ifdef SQLITE_AMALGAMATION
@@ -8483,10 +8528,11 @@
8483 SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int);
8484 SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int);
8485 SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int);
8486 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int);
8487 SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp);
 
8488 SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1);
8489 SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2);
8490 SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3);
8491 SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u8 P5);
8492 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr);
@@ -9282,11 +9328,11 @@
9282 ** sqlite3_close().
9283 *
9284 ** A thread must be holding a mutex on the corresponding Btree in order
9285 ** to access Schema content. This implies that the thread must also be
9286 ** holding a mutex on the sqlite3 connection pointer that owns the Btree.
9287 ** For a TEMP Schema, on the connection mutex is required.
9288 */
9289 struct Schema {
9290 int schema_cookie; /* Database schema version number for this file */
9291 int iGeneration; /* Generation counter. Incremented with each change */
9292 Hash tblHash; /* All tables indexed by name */
@@ -9555,10 +9601,12 @@
9555 #define SQLITE_IndexSort 0x04 /* Disable indexes for sorting */
9556 #define SQLITE_IndexSearch 0x08 /* Disable indexes for searching */
9557 #define SQLITE_IndexCover 0x10 /* Disable index covering table */
9558 #define SQLITE_GroupByOrder 0x20 /* Disable GROUPBY cover of ORDERBY */
9559 #define SQLITE_FactorOutConst 0x40 /* Disable factoring out constants */
 
 
9560 #define SQLITE_OptMask 0xff /* Mask of all disablable opts */
9561
9562 /*
9563 ** Possible values for the sqlite.magic field.
9564 ** The numbers are obtained at random and have no special meaning, other
@@ -9876,11 +9924,11 @@
9876 int iPKey; /* If not negative, use aCol[iPKey] as the primary key */
9877 int nCol; /* Number of columns in this table */
9878 Column *aCol; /* Information about each column */
9879 Index *pIndex; /* List of SQL indexes on this table. */
9880 int tnum; /* Root BTree node for this table (see note above) */
9881 unsigned nRowEst; /* Estimated rows in table - from sqlite_stat1 table */
9882 Select *pSelect; /* NULL for tables. Points to definition if a view. */
9883 u16 nRef; /* Number of pointers to this Table */
9884 u8 tabFlags; /* Mask of TF_* values */
9885 u8 keyConf; /* What to do in case of uniqueness conflict on iPKey */
9886 FKey *pFKey; /* Linked list of all foreign keys in this table */
@@ -10075,35 +10123,43 @@
10075 */
10076 struct Index {
10077 char *zName; /* Name of this index */
10078 int nColumn; /* Number of columns in the table used by this index */
10079 int *aiColumn; /* Which columns are used by this index. 1st is 0 */
10080 unsigned *aiRowEst; /* Result of ANALYZE: Est. rows selected by each column */
10081 Table *pTable; /* The SQL table being indexed */
10082 int tnum; /* Page containing root of this index in database file */
10083 u8 onError; /* OE_Abort, OE_Ignore, OE_Replace, or OE_None */
10084 u8 autoIndex; /* True if is automatically created (ex: by UNIQUE) */
10085 u8 bUnordered; /* Use this index for == or IN queries only */
 
10086 char *zColAff; /* String defining the affinity of each column */
10087 Index *pNext; /* The next index associated with the same table */
10088 Schema *pSchema; /* Schema containing this index */
10089 u8 *aSortOrder; /* Array of size Index.nColumn. True==DESC, False==ASC */
10090 char **azColl; /* Array of collation sequence names for index */
10091 IndexSample *aSample; /* Array of SQLITE_INDEX_SAMPLES samples */
 
 
 
10092 };
10093
10094 /*
10095 ** Each sample stored in the sqlite_stat2 table is represented in memory
10096 ** using a structure of this type.
10097 */
10098 struct IndexSample {
10099 union {
10100 char *z; /* Value if eType is SQLITE_TEXT or SQLITE_BLOB */
10101 double r; /* Value if eType is SQLITE_FLOAT or SQLITE_INTEGER */
 
10102 } u;
10103 u8 eType; /* SQLITE_NULL, SQLITE_INTEGER ... etc. */
10104 u8 nByte; /* Size in byte of text or blob. */
 
 
 
10105 };
10106
10107 /*
10108 ** Each token coming out of the lexer is an instance of
10109 ** this structure. Tokens are also used as part of an expression.
@@ -10446,10 +10502,11 @@
10446 Table *pTab; /* An SQL table corresponding to zName */
10447 Select *pSelect; /* A SELECT statement used in place of a table name */
10448 u8 isPopulated; /* Temporary table associated with SELECT is populated */
10449 u8 jointype; /* Type of join between this able and the previous */
10450 u8 notIndexed; /* True if there is a NOT INDEXED clause */
 
10451 #ifndef SQLITE_OMIT_EXPLAIN
10452 u8 iSelectId; /* If pSelect!=0, the id of the sub-select in EQP */
10453 #endif
10454 int iCursor; /* The VDBE cursor number used to access this table */
10455 Expr *pOn; /* The ON clause of a join */
@@ -10565,10 +10622,11 @@
10565 struct WhereInfo {
10566 Parse *pParse; /* Parsing and code generating context */
10567 u16 wctrlFlags; /* Flags originally passed to sqlite3WhereBegin() */
10568 u8 okOnePass; /* Ok to use one-pass algorithm for UPDATE or DELETE */
10569 u8 untestedTerms; /* Not all WHERE terms resolved by outer loop */
 
10570 SrcList *pTabList; /* List of tables in the join */
10571 int iTop; /* The very beginning of the WHERE loop */
10572 int iContinue; /* Jump here to continue with next record */
10573 int iBreak; /* Jump here to break out of the loop */
10574 int nLevel; /* Number of nested loop */
@@ -10576,10 +10634,13 @@
10576 double savedNQueryLoop; /* pParse->nQueryLoop outside the WHERE loop */
10577 double nRowOut; /* Estimated number of output rows */
10578 WhereLevel a[1]; /* Information about each nest loop in WHERE */
10579 };
10580
 
 
 
10581 /*
10582 ** A NameContext defines a context in which to resolve table and column
10583 ** names. The context consists of a list of tables (the pSrcList) field and
10584 ** a list of named expression (pEList). The named expression list may
10585 ** be NULL. The pSrc corresponds to the FROM clause of a SELECT or
@@ -11052,10 +11113,11 @@
11052 int isPCacheInit; /* True after malloc is initialized */
11053 sqlite3_mutex *pInitMutex; /* Mutex used by sqlite3_initialize() */
11054 int nRefInitMutex; /* Number of users of pInitMutex */
11055 void (*xLog)(void*,int,const char*); /* Function for logging */
11056 void *pLogArg; /* First argument to xLog() */
 
11057 };
11058
11059 /*
11060 ** Context pointer passed down through the tree-walk.
11061 */
@@ -11299,10 +11361,11 @@
11299 #else
11300 # define sqlite3ViewGetColumnNames(A,B) 0
11301 #endif
11302
11303 SQLITE_PRIVATE void sqlite3DropTable(Parse*, SrcList*, int, int);
 
11304 SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3*, Table*);
11305 #ifndef SQLITE_OMIT_AUTOINCREMENT
11306 SQLITE_PRIVATE void sqlite3AutoincrementBegin(Parse *pParse);
11307 SQLITE_PRIVATE void sqlite3AutoincrementEnd(Parse *pParse);
11308 #else
@@ -11336,11 +11399,11 @@
11336 #if defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT) && !defined(SQLITE_OMIT_SUBQUERY)
11337 SQLITE_PRIVATE Expr *sqlite3LimitWhere(Parse *, SrcList *, Expr *, ExprList *, Expr *, Expr *, char *);
11338 #endif
11339 SQLITE_PRIVATE void sqlite3DeleteFrom(Parse*, SrcList*, Expr*);
11340 SQLITE_PRIVATE void sqlite3Update(Parse*, SrcList*, ExprList*, Expr*, int);
11341 SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(Parse*, SrcList*, Expr*, ExprList**, u16);
11342 SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo*);
11343 SQLITE_PRIVATE int sqlite3ExprCodeGetColumn(Parse*, Table*, int, int, int);
11344 SQLITE_PRIVATE void sqlite3ExprCodeGetColumnOfTable(Vdbe*, Table*, int, int, int);
11345 SQLITE_PRIVATE void sqlite3ExprCodeMove(Parse*, int, int, int);
11346 SQLITE_PRIVATE void sqlite3ExprCodeCopy(Parse*, int, int, int);
@@ -11479,11 +11542,11 @@
11479 SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8);
11480 SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*);
11481 SQLITE_PRIVATE int sqlite3Atoi(const char*);
11482 SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar);
11483 SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte);
11484 SQLITE_PRIVATE int sqlite3Utf8Read(const u8*, const u8**);
11485
11486 /*
11487 ** Routines to read and write variable-length integers. These used to
11488 ** be defined locally, but now we use the varint routines in the util.c
11489 ** file. Code should use the MACRO forms below, as the Varint32 versions
@@ -11555,11 +11618,11 @@
11555 SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8,
11556 void(*)(void*));
11557 SQLITE_PRIVATE void sqlite3ValueFree(sqlite3_value*);
11558 SQLITE_PRIVATE sqlite3_value *sqlite3ValueNew(sqlite3 *);
11559 SQLITE_PRIVATE char *sqlite3Utf16to8(sqlite3 *, const void*, int, u8);
11560 #ifdef SQLITE_ENABLE_STAT2
11561 SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *, u8, char *, int, int *);
11562 #endif
11563 SQLITE_PRIVATE int sqlite3ValueFromExpr(sqlite3 *, Expr *, u8, u8, sqlite3_value **);
11564 SQLITE_PRIVATE void sqlite3ValueApplyAffinity(sqlite3_value *, u8, u8);
11565 #ifndef SQLITE_AMALGAMATION
@@ -12012,10 +12075,11 @@
12012 0, /* isPCacheInit */
12013 0, /* pInitMutex */
12014 0, /* nRefInitMutex */
12015 0, /* xLog */
12016 0, /* pLogArg */
 
12017 };
12018
12019
12020 /*
12021 ** Hash table for global functions - functions common to all
@@ -12180,10 +12244,13 @@
12180 #ifdef SQLITE_ENABLE_RTREE
12181 "ENABLE_RTREE",
12182 #endif
12183 #ifdef SQLITE_ENABLE_STAT2
12184 "ENABLE_STAT2",
 
 
 
12185 #endif
12186 #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY
12187 "ENABLE_UNLOCK_NOTIFY",
12188 #endif
12189 #ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT
@@ -13162,26 +13229,10 @@
13162 */
13163 #include <time.h>
13164
13165 #ifndef SQLITE_OMIT_DATETIME_FUNCS
13166
13167 /*
13168 ** On recent Windows platforms, the localtime_s() function is available
13169 ** as part of the "Secure CRT". It is essentially equivalent to
13170 ** localtime_r() available under most POSIX platforms, except that the
13171 ** order of the parameters is reversed.
13172 **
13173 ** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx.
13174 **
13175 ** If the user has not indicated to use localtime_r() or localtime_s()
13176 ** already, check for an MSVC build environment that provides
13177 ** localtime_s().
13178 */
13179 #if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \
13180 defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE)
13181 #define HAVE_LOCALTIME_S 1
13182 #endif
13183
13184 /*
13185 ** A structure for holding a single date and time.
13186 */
13187 typedef struct DateTime DateTime;
@@ -13522,20 +13573,90 @@
13522 static void clearYMD_HMS_TZ(DateTime *p){
13523 p->validYMD = 0;
13524 p->validHMS = 0;
13525 p->validTZ = 0;
13526 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13527
13528 #ifndef SQLITE_OMIT_LOCALTIME
13529 /*
13530 ** Compute the difference (in milliseconds)
13531 ** between localtime and UTC (a.k.a. GMT)
13532 ** for the time value p where p is in UTC.
 
 
 
13533 */
13534 static sqlite3_int64 localtimeOffset(DateTime *p){
 
 
 
 
13535 DateTime x, y;
13536 time_t t;
 
 
 
 
 
13537 x = *p;
13538 computeYMD_HMS(&x);
13539 if( x.Y<1971 || x.Y>=2038 ){
13540 x.Y = 2000;
13541 x.M = 1;
@@ -13549,51 +13670,27 @@
13549 }
13550 x.tz = 0;
13551 x.validJD = 0;
13552 computeJD(&x);
13553 t = (time_t)(x.iJD/1000 - 21086676*(i64)10000);
13554 #ifdef HAVE_LOCALTIME_R
13555 {
13556 struct tm sLocal;
13557 localtime_r(&t, &sLocal);
13558 y.Y = sLocal.tm_year + 1900;
13559 y.M = sLocal.tm_mon + 1;
13560 y.D = sLocal.tm_mday;
13561 y.h = sLocal.tm_hour;
13562 y.m = sLocal.tm_min;
13563 y.s = sLocal.tm_sec;
13564 }
13565 #elif defined(HAVE_LOCALTIME_S) && HAVE_LOCALTIME_S
13566 {
13567 struct tm sLocal;
13568 localtime_s(&sLocal, &t);
13569 y.Y = sLocal.tm_year + 1900;
13570 y.M = sLocal.tm_mon + 1;
13571 y.D = sLocal.tm_mday;
13572 y.h = sLocal.tm_hour;
13573 y.m = sLocal.tm_min;
13574 y.s = sLocal.tm_sec;
13575 }
13576 #else
13577 {
13578 struct tm *pTm;
13579 sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER));
13580 pTm = localtime(&t);
13581 y.Y = pTm->tm_year + 1900;
13582 y.M = pTm->tm_mon + 1;
13583 y.D = pTm->tm_mday;
13584 y.h = pTm->tm_hour;
13585 y.m = pTm->tm_min;
13586 y.s = pTm->tm_sec;
13587 sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER));
13588 }
13589 #endif
13590 y.validYMD = 1;
13591 y.validHMS = 1;
13592 y.validJD = 0;
13593 y.validTZ = 0;
13594 computeJD(&y);
 
13595 return y.iJD - x.iJD;
13596 }
13597 #endif /* SQLITE_OMIT_LOCALTIME */
13598
13599 /*
@@ -13613,13 +13710,16 @@
13613 ** weekday N
13614 ** unixepoch
13615 ** localtime
13616 ** utc
13617 **
13618 ** Return 0 on success and 1 if there is any kind of error.
 
 
 
13619 */
13620 static int parseModifier(const char *zMod, DateTime *p){
13621 int rc = 1;
13622 int n;
13623 double r;
13624 char *z, zBuf[30];
13625 z = zBuf;
@@ -13635,13 +13735,12 @@
13635 ** Assuming the current time value is UTC (a.k.a. GMT), shift it to
13636 ** show local time.
13637 */
13638 if( strcmp(z, "localtime")==0 ){
13639 computeJD(p);
13640 p->iJD += localtimeOffset(p);
13641 clearYMD_HMS_TZ(p);
13642 rc = 0;
13643 }
13644 break;
13645 }
13646 #endif
13647 case 'u': {
@@ -13658,15 +13757,16 @@
13658 }
13659 #ifndef SQLITE_OMIT_LOCALTIME
13660 else if( strcmp(z, "utc")==0 ){
13661 sqlite3_int64 c1;
13662 computeJD(p);
13663 c1 = localtimeOffset(p);
13664 p->iJD -= c1;
13665 clearYMD_HMS_TZ(p);
13666 p->iJD += c1 - localtimeOffset(p);
13667 rc = 0;
 
13668 }
13669 #endif
13670 break;
13671 }
13672 case 'w': {
@@ -13843,13 +13943,12 @@
13843 if( !z || parseDateOrTime(context, (char*)z, p) ){
13844 return 1;
13845 }
13846 }
13847 for(i=1; i<argc; i++){
13848 if( (z = sqlite3_value_text(argv[i]))==0 || parseModifier((char*)z, p) ){
13849 return 1;
13850 }
13851 }
13852 return 0;
13853 }
13854
13855
@@ -14343,11 +14442,11 @@
14343 DO_OS_MALLOC_TEST(0);
14344 /* 0x87f3f is a mask of SQLITE_OPEN_ flags that are valid to be passed
14345 ** down into the VFS layer. Some SQLITE_OPEN_ flags (for example,
14346 ** SQLITE_OPEN_FULLMUTEX or SQLITE_OPEN_SHAREDCACHE) are blocked before
14347 ** reaching the VFS. */
14348 rc = pVfs->xOpen(pVfs, zPath, pFile, flags & 0x87f3f, pFlagsOut);
14349 assert( rc==SQLITE_OK || pFile->pMethods==0 );
14350 return rc;
14351 }
14352 SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){
14353 return pVfs->xDelete(pVfs, zPath, dirSync);
@@ -15804,11 +15903,11 @@
15804 ** Free an outstanding memory allocation.
15805 **
15806 ** This function assumes that the necessary mutexes, if any, are
15807 ** already held by the caller. Hence "Unsafe".
15808 */
15809 void memsys3FreeUnsafe(void *pOld){
15810 Mem3Block *p = (Mem3Block*)pOld;
15811 int i;
15812 u32 size, x;
15813 assert( sqlite3_mutex_held(mem3.mutex) );
15814 assert( p>mem3.aPool && p<&mem3.aPool[mem3.nPool] );
@@ -15879,21 +15978,21 @@
15879 }
15880
15881 /*
15882 ** Free memory.
15883 */
15884 void memsys3Free(void *pPrior){
15885 assert( pPrior );
15886 memsys3Enter();
15887 memsys3FreeUnsafe(pPrior);
15888 memsys3Leave();
15889 }
15890
15891 /*
15892 ** Change the size of an existing memory allocation
15893 */
15894 void *memsys3Realloc(void *pPrior, int nBytes){
15895 int nOld;
15896 void *p;
15897 if( pPrior==0 ){
15898 return sqlite3_malloc(nBytes);
15899 }
@@ -20086,11 +20185,11 @@
20086 } \
20087 if( c<0x80 \
20088 || (c&0xFFFFF800)==0xD800 \
20089 || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \
20090 }
20091 SQLITE_PRIVATE int sqlite3Utf8Read(
20092 const unsigned char *zIn, /* First byte of UTF-8 character */
20093 const unsigned char **pzNext /* Write first byte past UTF-8 char here */
20094 ){
20095 unsigned int c;
20096
@@ -20387,11 +20486,11 @@
20387 ** no longer required.
20388 **
20389 ** If a malloc failure occurs, NULL is returned and the db.mallocFailed
20390 ** flag set.
20391 */
20392 #ifdef SQLITE_ENABLE_STAT2
20393 SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *db, u8 enc, char *z, int n, int *pnOut){
20394 Mem m;
20395 memset(&m, 0, sizeof(m));
20396 m.db = db;
20397 sqlite3VdbeMemSetStr(&m, z, n, SQLITE_UTF8, SQLITE_STATIC);
@@ -21633,26 +21732,32 @@
21633 return -x;
21634 }
21635
21636 #ifdef SQLITE_ENABLE_8_3_NAMES
21637 /*
21638 ** If SQLITE_ENABLE_8_3_NAME is set at compile-time and if the database
21639 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
21640 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
21641 ** three characters, then shorten the suffix on z[] to be the last three
21642 ** characters of the original suffix.
 
 
 
21643 **
21644 ** Examples:
21645 **
21646 ** test.db-journal => test.nal
21647 ** test.db-wal => test.wal
21648 ** test.db-shm => test.shm
21649 */
21650 SQLITE_PRIVATE void sqlite3FileSuffix3(const char *zBaseFilename, char *z){
 
21651 const char *zOk;
21652 zOk = sqlite3_uri_parameter(zBaseFilename, "8_3_names");
21653 if( zOk && sqlite3GetBoolean(zOk) ){
 
 
21654 int i, sz;
21655 sz = sqlite3Strlen30(z);
21656 for(i=sz-1; i>0 && z[i]!='/' && z[i]!='.'; i--){}
21657 if( z[i]=='.' && ALWAYS(sz>i+4) ) memcpy(&z[i+1], &z[sz-3], 4);
21658 }
@@ -22182,14 +22287,17 @@
22182 #ifdef MEMORY_DEBUG
22183 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
22184 #endif
22185
22186 #ifdef SQLITE_DEBUG
22187 SQLITE_PRIVATE int sqlite3OSTrace = 0;
22188 #define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
 
 
 
22189 #else
22190 #define OSTRACE(X)
22191 #endif
22192
22193 /*
22194 ** Macros for performance tracing. Normally turned off. Only works
22195 ** on i486 hardware.
@@ -24363,10 +24471,14 @@
24363
24364 #if defined(__APPLE__) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS)
24365 # include <sys/mount.h>
24366 #endif
24367
 
 
 
 
24368 /*
24369 ** Allowed values of unixFile.fsFlags
24370 */
24371 #define SQLITE_FSFLAGS_IS_MSDOS 0x1
24372
@@ -24428,11 +24540,10 @@
24428 typedef struct unixFile unixFile;
24429 struct unixFile {
24430 sqlite3_io_methods const *pMethod; /* Always the first entry */
24431 unixInodeInfo *pInode; /* Info about locks on this inode */
24432 int h; /* The file descriptor */
24433 int dirfd; /* File descriptor for the directory */
24434 unsigned char eFileLock; /* The type of lock held on this fd */
24435 unsigned char ctrlFlags; /* Behavioral bits. UNIXFILE_* flags */
24436 int lastErrno; /* The unix errno from last I/O error */
24437 void *lockingContext; /* Locking style specific state */
24438 UnixUnusedFd *pUnused; /* Pre-allocated UnixUnusedFd */
@@ -24470,12 +24581,14 @@
24470 };
24471
24472 /*
24473 ** Allowed values for the unixFile.ctrlFlags bitmask:
24474 */
24475 #define UNIXFILE_EXCL 0x01 /* Connections from one process only */
24476 #define UNIXFILE_RDONLY 0x02 /* Connection is read only */
 
 
24477
24478 /*
24479 ** Include code that is common to all os_*.c files
24480 */
24481 /************** Include os_common.h in the middle of os_unix.c ***************/
@@ -24510,14 +24623,17 @@
24510 #ifdef MEMORY_DEBUG
24511 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
24512 #endif
24513
24514 #ifdef SQLITE_DEBUG
24515 SQLITE_PRIVATE int sqlite3OSTrace = 0;
24516 #define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
 
 
 
24517 #else
24518 #define OSTRACE(X)
24519 #endif
24520
24521 /*
24522 ** Macros for performance tracing. Normally turned off. Only works
24523 ** on i486 hardware.
@@ -24722,10 +24838,13 @@
24722 */
24723 static int posixOpen(const char *zFile, int flags, int mode){
24724 return open(zFile, flags, mode);
24725 }
24726
 
 
 
24727 /*
24728 ** Many system calls are accessed through pointer-to-functions so that
24729 ** they may be overridden at runtime to facilitate fault injection during
24730 ** testing and sandboxing. The following array holds the names and pointers
24731 ** to all overrideable system calls.
@@ -24818,10 +24937,16 @@
24818 #else
24819 { "fallocate", (sqlite3_syscall_ptr)0, 0 },
24820 #endif
24821 #define osFallocate ((int(*)(int,off_t,off_t))aSyscall[15].pCurrent)
24822
 
 
 
 
 
 
24823 }; /* End of the overrideable system calls */
24824
24825 /*
24826 ** This is the xSetSystemCall() method of sqlite3_vfs for all of the
24827 ** "unix" VFSes. Return SQLITE_OK opon successfully updating the
@@ -25102,11 +25227,13 @@
25102 case EINVAL:
25103 case ENOTCONN:
25104 case ENODEV:
25105 case ENXIO:
25106 case ENOENT:
 
25107 case ESTALE:
 
25108 case ENOSYS:
25109 /* these should force the client to close the file and reconnect */
25110
25111 default:
25112 return sqliteIOErr;
@@ -26172,22 +26299,18 @@
26172 ** even on VxWorks. A mutex will be acquired on VxWorks by the
26173 ** vxworksReleaseFileId() routine.
26174 */
26175 static int closeUnixFile(sqlite3_file *id){
26176 unixFile *pFile = (unixFile*)id;
26177 if( pFile->dirfd>=0 ){
26178 robust_close(pFile, pFile->dirfd, __LINE__);
26179 pFile->dirfd=-1;
26180 }
26181 if( pFile->h>=0 ){
26182 robust_close(pFile, pFile->h, __LINE__);
26183 pFile->h = -1;
26184 }
26185 #if OS_VXWORKS
26186 if( pFile->pId ){
26187 if( pFile->isDelete ){
26188 unlink(pFile->pId->zCanonicalName);
26189 }
26190 vxworksReleaseFileId(pFile->pId);
26191 pFile->pId = 0;
26192 }
26193 #endif
@@ -26368,12 +26491,14 @@
26368 /* If we have any lock, then the lock file already exists. All we have
26369 ** to do is adjust our internal record of the lock level.
26370 */
26371 if( pFile->eFileLock > NO_LOCK ){
26372 pFile->eFileLock = eFileLock;
26373 #if !OS_VXWORKS
26374 /* Always update the timestamp on the old file */
 
 
 
26375 utimes(zLockFile, NULL);
26376 #endif
26377 return SQLITE_OK;
26378 }
26379
@@ -26430,11 +26555,11 @@
26430 return SQLITE_OK;
26431 }
26432
26433 /* To fully unlock the database, delete the lock file */
26434 assert( eFileLock==NO_LOCK );
26435 if( unlink(zLockFile) ){
26436 int rc = 0;
26437 int tErrno = errno;
26438 if( ENOENT != tErrno ){
26439 rc = SQLITE_IOERR_UNLOCK;
26440 }
@@ -27665,10 +27790,54 @@
27665 if( OS_VXWORKS && rc!= -1 ){
27666 rc = 0;
27667 }
27668 return rc;
27669 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27670
27671 /*
27672 ** Make sure all writes to a particular file are committed to disk.
27673 **
27674 ** If dataOnly==0 then both the file itself and its metadata (file
@@ -27706,32 +27875,25 @@
27706 SimulateIOError( rc=1 );
27707 if( rc ){
27708 pFile->lastErrno = errno;
27709 return unixLogError(SQLITE_IOERR_FSYNC, "full_fsync", pFile->zPath);
27710 }
27711 if( pFile->dirfd>=0 ){
27712 OSTRACE(("DIRSYNC %-3d (have_fullfsync=%d fullsync=%d)\n", pFile->dirfd,
 
 
 
 
 
 
27713 HAVE_FULLFSYNC, isFullsync));
27714 #ifndef SQLITE_DISABLE_DIRSYNC
27715 /* The directory sync is only attempted if full_fsync is
27716 ** turned off or unavailable. If a full_fsync occurred above,
27717 ** then the directory sync is superfluous.
27718 */
27719 if( (!HAVE_FULLFSYNC || !isFullsync) && full_fsync(pFile->dirfd,0,0) ){
27720 /*
27721 ** We have received multiple reports of fsync() returning
27722 ** errors when applied to directories on certain file systems.
27723 ** A failed directory sync is not a big deal. So it seems
27724 ** better to ignore the error. Ticket #1657
27725 */
27726 /* pFile->lastErrno = errno; */
27727 /* return SQLITE_IOERR; */
27728 }
27729 #endif
27730 /* Only need to sync once, so close the directory when we are done */
27731 robust_close(pFile, pFile->dirfd, __LINE__);
27732 pFile->dirfd = -1;
27733 }
27734 return rc;
27735 }
27736
27737 /*
@@ -27816,17 +27978,23 @@
27816 ** If the user has configured a chunk-size for this file, it could be
27817 ** that the file needs to be extended at this point. Otherwise, the
27818 ** SQLITE_FCNTL_SIZE_HINT operation is a no-op for Unix.
27819 */
27820 static int fcntlSizeHint(unixFile *pFile, i64 nByte){
27821 if( pFile->szChunk ){
27822 i64 nSize; /* Required file size */
 
27823 struct stat buf; /* Used to hold return values of fstat() */
27824
27825 if( osFstat(pFile->h, &buf) ) return SQLITE_IOERR_FSTAT;
27826
27827 nSize = ((nByte+pFile->szChunk-1) / pFile->szChunk) * pFile->szChunk;
 
 
 
 
 
27828 if( nSize>(i64)buf.st_size ){
27829
27830 #if defined(HAVE_POSIX_FALLOCATE) && HAVE_POSIX_FALLOCATE
27831 /* The code below is handling the return value of osFallocate()
27832 ** correctly. posix_fallocate() is defined to "returns zero on success,
@@ -27865,25 +28033,37 @@
27865
27866 /*
27867 ** Information and control of an open file handle.
27868 */
27869 static int unixFileControl(sqlite3_file *id, int op, void *pArg){
 
27870 switch( op ){
27871 case SQLITE_FCNTL_LOCKSTATE: {
27872 *(int*)pArg = ((unixFile*)id)->eFileLock;
27873 return SQLITE_OK;
27874 }
27875 case SQLITE_LAST_ERRNO: {
27876 *(int*)pArg = ((unixFile*)id)->lastErrno;
27877 return SQLITE_OK;
27878 }
27879 case SQLITE_FCNTL_CHUNK_SIZE: {
27880 ((unixFile*)id)->szChunk = *(int *)pArg;
27881 return SQLITE_OK;
27882 }
27883 case SQLITE_FCNTL_SIZE_HINT: {
27884 return fcntlSizeHint((unixFile *)id, *(i64 *)pArg);
 
 
 
 
 
 
 
 
 
 
 
27885 }
27886 #ifndef NDEBUG
27887 /* The pager calls this method to signal that it has done
27888 ** a rollback and that the database is therefore unchanged and
27889 ** it hence it is OK for the transaction change counter to be
@@ -28095,11 +28275,11 @@
28095 unixShmNode *p = pFd->pInode->pShmNode;
28096 assert( unixMutexHeld() );
28097 if( p && p->nRef==0 ){
28098 int i;
28099 assert( p->pInode==pFd->pInode );
28100 if( p->mutex ) sqlite3_mutex_free(p->mutex);
28101 for(i=0; i<p->nRegion; i++){
28102 if( p->h>=0 ){
28103 munmap(p->apRegion[i], p->szRegion);
28104 }else{
28105 sqlite3_free(p->apRegion[i]);
@@ -28564,11 +28744,11 @@
28564 ** shared-memory file, too */
28565 unixEnterMutex();
28566 assert( pShmNode->nRef>0 );
28567 pShmNode->nRef--;
28568 if( pShmNode->nRef==0 ){
28569 if( deleteFlag && pShmNode->h>=0 ) unlink(pShmNode->zFilename);
28570 unixShmPurge(pDbFd);
28571 }
28572 unixLeaveMutex();
28573
28574 return SQLITE_OK;
@@ -28877,11 +29057,11 @@
28877 ** Initialize the contents of the unixFile structure pointed to by pId.
28878 */
28879 static int fillInUnixFile(
28880 sqlite3_vfs *pVfs, /* Pointer to vfs object */
28881 int h, /* Open file descriptor of file being opened */
28882 int dirfd, /* Directory file descriptor */
28883 sqlite3_file *pId, /* Write to the unixFile structure here */
28884 const char *zFilename, /* Name of the file being opened */
28885 int noLock, /* Omit locking if true */
28886 int isDelete, /* Delete on close if true */
28887 int isReadOnly /* True if the file is opened read-only */
@@ -28908,20 +29088,22 @@
28908 assert( zFilename==0 || zFilename[0]=='/' );
28909 #endif
28910
28911 OSTRACE(("OPEN %-3d %s\n", h, zFilename));
28912 pNew->h = h;
28913 pNew->dirfd = dirfd;
28914 pNew->zPath = zFilename;
28915 if( memcmp(pVfs->zName,"unix-excl",10)==0 ){
28916 pNew->ctrlFlags = UNIXFILE_EXCL;
28917 }else{
28918 pNew->ctrlFlags = 0;
28919 }
28920 if( isReadOnly ){
28921 pNew->ctrlFlags |= UNIXFILE_RDONLY;
28922 }
 
 
 
28923
28924 #if OS_VXWORKS
28925 pNew->pId = vxworksFindFileId(zFilename);
28926 if( pNew->pId==0 ){
28927 noLock = 1;
@@ -29044,56 +29226,24 @@
29044 pNew->lastErrno = 0;
29045 #if OS_VXWORKS
29046 if( rc!=SQLITE_OK ){
29047 if( h>=0 ) robust_close(pNew, h, __LINE__);
29048 h = -1;
29049 unlink(zFilename);
29050 isDelete = 0;
29051 }
29052 pNew->isDelete = isDelete;
29053 #endif
29054 if( rc!=SQLITE_OK ){
29055 if( dirfd>=0 ) robust_close(pNew, dirfd, __LINE__);
29056 if( h>=0 ) robust_close(pNew, h, __LINE__);
29057 }else{
29058 pNew->pMethod = pLockingStyle;
29059 OpenCounter(+1);
29060 }
29061 return rc;
29062 }
29063
29064 /*
29065 ** Open a file descriptor to the directory containing file zFilename.
29066 ** If successful, *pFd is set to the opened file descriptor and
29067 ** SQLITE_OK is returned. If an error occurs, either SQLITE_NOMEM
29068 ** or SQLITE_CANTOPEN is returned and *pFd is set to an undefined
29069 ** value.
29070 **
29071 ** If SQLITE_OK is returned, the caller is responsible for closing
29072 ** the file descriptor *pFd using close().
29073 */
29074 static int openDirectory(const char *zFilename, int *pFd){
29075 int ii;
29076 int fd = -1;
29077 char zDirname[MAX_PATHNAME+1];
29078
29079 sqlite3_snprintf(MAX_PATHNAME, zDirname, "%s", zFilename);
29080 for(ii=(int)strlen(zDirname); ii>1 && zDirname[ii]!='/'; ii--);
29081 if( ii>0 ){
29082 zDirname[ii] = '\0';
29083 fd = robust_open(zDirname, O_RDONLY|O_BINARY, 0);
29084 if( fd>=0 ){
29085 #ifdef FD_CLOEXEC
29086 osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
29087 #endif
29088 OSTRACE(("OPENDIR %-3d %s\n", fd, zDirname));
29089 }
29090 }
29091 *pFd = fd;
29092 return (fd>=0?SQLITE_OK:unixLogError(SQLITE_CANTOPEN_BKPT, "open", zDirname));
29093 }
29094
29095 /*
29096 ** Return the name of a directory in which to put temporary files.
29097 ** If no suitable temporary file directory can be found, return NULL.
29098 */
29099 static const char *unixTempFileDir(void){
@@ -29204,11 +29354,11 @@
29204 ** ignored and -1 is returned. The caller will try to open a new file
29205 ** descriptor on the same path, fail, and return an error to SQLite.
29206 **
29207 ** Even if a subsequent open() call does succeed, the consequences of
29208 ** not searching for a resusable file descriptor are not dire. */
29209 if( 0==stat(zPath, &sStat) ){
29210 unixInodeInfo *pInode;
29211
29212 unixEnterMutex();
29213 pInode = inodeList;
29214 while( pInode && (pInode->fileId.dev!=sStat.st_dev
@@ -29280,11 +29430,11 @@
29280 while( nDb>0 && zPath[nDb]!='-' ) nDb--;
29281 if( nDb==0 ) return SQLITE_OK;
29282 memcpy(zDb, zPath, nDb);
29283 zDb[nDb] = '\0';
29284
29285 if( 0==stat(zDb, &sStat) ){
29286 *pMode = sStat.st_mode & 0777;
29287 }else{
29288 rc = SQLITE_IOERR_FSTAT;
29289 }
29290 }else if( flags & SQLITE_OPEN_DELETEONCLOSE ){
@@ -29322,11 +29472,10 @@
29322 int flags, /* Input flags to control the opening */
29323 int *pOutFlags /* Output flags returned to SQLite core */
29324 ){
29325 unixFile *p = (unixFile *)pFile;
29326 int fd = -1; /* File descriptor returned by open() */
29327 int dirfd = -1; /* Directory file descriptor */
29328 int openFlags = 0; /* Flags to pass to open() */
29329 int eType = flags&0xFFFFFF00; /* Type of file to open */
29330 int noLock; /* True to omit locking primitives */
29331 int rc = SQLITE_OK; /* Function Return Code */
29332
@@ -29341,11 +29490,11 @@
29341
29342 /* If creating a master or main-file journal, this function will open
29343 ** a file-descriptor on the directory too. The first time unixSync()
29344 ** is called the directory file descriptor will be fsync()ed and close()d.
29345 */
29346 int isOpenDirectory = (isCreate && (
29347 eType==SQLITE_OPEN_MASTER_JOURNAL
29348 || eType==SQLITE_OPEN_MAIN_JOURNAL
29349 || eType==SQLITE_OPEN_WAL
29350 ));
29351
@@ -29395,11 +29544,11 @@
29395 }
29396 }
29397 p->pUnused = pUnused;
29398 }else if( !zName ){
29399 /* If zName is NULL, the upper layer is requesting a temp file. */
29400 assert(isDelete && !isOpenDirectory);
29401 rc = unixGetTempname(MAX_PATHNAME+1, zTmpname);
29402 if( rc!=SQLITE_OK ){
29403 return rc;
29404 }
29405 zName = zTmpname;
@@ -29451,32 +29600,19 @@
29451
29452 if( isDelete ){
29453 #if OS_VXWORKS
29454 zPath = zName;
29455 #else
29456 unlink(zName);
29457 #endif
29458 }
29459 #if SQLITE_ENABLE_LOCKING_STYLE
29460 else{
29461 p->openFlags = openFlags;
29462 }
29463 #endif
29464
29465 if( isOpenDirectory ){
29466 rc = openDirectory(zPath, &dirfd);
29467 if( rc!=SQLITE_OK ){
29468 /* It is safe to close fd at this point, because it is guaranteed not
29469 ** to be open on a database file. If it were open on a database file,
29470 ** it would not be safe to close as this would release any locks held
29471 ** on the file by this process. */
29472 assert( eType!=SQLITE_OPEN_MAIN_DB );
29473 robust_close(p, fd, __LINE__);
29474 goto open_finished;
29475 }
29476 }
29477
29478 #ifdef FD_CLOEXEC
29479 osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
29480 #endif
29481
29482 noLock = eType!=SQLITE_OPEN_MAIN_DB;
@@ -29484,11 +29620,10 @@
29484
29485 #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
29486 struct statfs fsInfo;
29487 if( fstatfs(fd, &fsInfo) == -1 ){
29488 ((unixFile*)pFile)->lastErrno = errno;
29489 if( dirfd>=0 ) robust_close(p, dirfd, __LINE__);
29490 robust_close(p, fd, __LINE__);
29491 return SQLITE_IOERR_ACCESS;
29492 }
29493 if (0 == strncmp("msdos", fsInfo.f_fstypename, 5)) {
29494 ((unixFile*)pFile)->fsFlags |= SQLITE_FSFLAGS_IS_MSDOS;
@@ -29516,21 +29651,18 @@
29516 ** then the call to close() will cancel those locks. In practice,
29517 ** we're assuming that statfs() doesn't fail very often. At least
29518 ** not while other file descriptors opened by the same process on
29519 ** the same file are working. */
29520 p->lastErrno = errno;
29521 if( dirfd>=0 ){
29522 robust_close(p, dirfd, __LINE__);
29523 }
29524 robust_close(p, fd, __LINE__);
29525 rc = SQLITE_IOERR_ACCESS;
29526 goto open_finished;
29527 }
29528 useProxy = !(fsInfo.f_flags&MNT_LOCAL);
29529 }
29530 if( useProxy ){
29531 rc = fillInUnixFile(pVfs, fd, dirfd, pFile, zPath, noLock,
29532 isDelete, isReadonly);
29533 if( rc==SQLITE_OK ){
29534 rc = proxyTransformUnixFile((unixFile*)pFile, ":auto:");
29535 if( rc!=SQLITE_OK ){
29536 /* Use unixClose to clean up the resources added in fillInUnixFile
@@ -29544,11 +29676,11 @@
29544 goto open_finished;
29545 }
29546 }
29547 #endif
29548
29549 rc = fillInUnixFile(pVfs, fd, dirfd, pFile, zPath, noLock,
29550 isDelete, isReadonly);
29551 open_finished:
29552 if( rc!=SQLITE_OK ){
29553 sqlite3_free(p->pUnused);
29554 }
@@ -29566,17 +29698,17 @@
29566 int dirSync /* If true, fsync() directory after deleting file */
29567 ){
29568 int rc = SQLITE_OK;
29569 UNUSED_PARAMETER(NotUsed);
29570 SimulateIOError(return SQLITE_IOERR_DELETE);
29571 if( unlink(zPath)==(-1) && errno!=ENOENT ){
29572 return unixLogError(SQLITE_IOERR_DELETE, "unlink", zPath);
29573 }
29574 #ifndef SQLITE_DISABLE_DIRSYNC
29575 if( dirSync ){
29576 int fd;
29577 rc = openDirectory(zPath, &fd);
29578 if( rc==SQLITE_OK ){
29579 #if OS_VXWORKS
29580 if( fsync(fd)==-1 )
29581 #else
29582 if( fsync(fd) )
@@ -29625,11 +29757,11 @@
29625 assert(!"Invalid flags argument");
29626 }
29627 *pResOut = (osAccess(zPath, amode)==0);
29628 if( flags==SQLITE_ACCESS_EXISTS && *pResOut ){
29629 struct stat buf;
29630 if( 0==stat(zPath, &buf) && buf.st_size==0 ){
29631 *pResOut = 0;
29632 }
29633 }
29634 return SQLITE_OK;
29635 }
@@ -30144,11 +30276,10 @@
30144 const char *path, /* path for the new unixFile */
30145 unixFile **ppFile, /* unixFile created and returned by ref */
30146 int islockfile /* if non zero missing dirs will be created */
30147 ) {
30148 int fd = -1;
30149 int dirfd = -1;
30150 unixFile *pNew;
30151 int rc = SQLITE_OK;
30152 int openFlags = O_RDWR | O_CREAT;
30153 sqlite3_vfs dummyVfs;
30154 int terrno = 0;
@@ -30209,11 +30340,11 @@
30209 dummyVfs.zName = "dummy";
30210 pUnused->fd = fd;
30211 pUnused->flags = openFlags;
30212 pNew->pUnused = pUnused;
30213
30214 rc = fillInUnixFile(&dummyVfs, fd, dirfd, (sqlite3_file*)pNew, path, 0, 0, 0);
30215 if( rc==SQLITE_OK ){
30216 *ppFile = pNew;
30217 return SQLITE_OK;
30218 }
30219 end_create_proxy:
@@ -30323,11 +30454,11 @@
30323 conchFile->openFlags = O_RDWR | O_CREAT;
30324
30325 end_breaklock:
30326 if( rc ){
30327 if( fd>=0 ){
30328 unlink(tPath);
30329 robust_close(pFile, fd, __LINE__);
30330 }
30331 fprintf(stderr, "failed to break stale lock on %s, %s\n", cPath, errmsg);
30332 }
30333 return rc;
@@ -31146,11 +31277,11 @@
31146 };
31147 unsigned int i; /* Loop counter */
31148
31149 /* Double-check that the aSyscall[] array has been constructed
31150 ** correctly. See ticket [bb3a86e890c8e96ab] */
31151 assert( ArraySize(aSyscall)==16 );
31152
31153 /* Register all VFSes defined in the aVfs[] array */
31154 for(i=0; i<(sizeof(aVfs)/sizeof(sqlite3_vfs)); i++){
31155 sqlite3_vfs_register(&aVfs[i], i==0);
31156 }
@@ -31263,14 +31394,17 @@
31263 #ifdef MEMORY_DEBUG
31264 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
31265 #endif
31266
31267 #ifdef SQLITE_DEBUG
31268 SQLITE_PRIVATE int sqlite3OSTrace = 0;
31269 #define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
 
 
 
31270 #else
31271 #define OSTRACE(X)
31272 #endif
31273
31274 /*
31275 ** Macros for performance tracing. Normally turned off. Only works
31276 ** on i486 hardware.
@@ -31478,12 +31612,13 @@
31478 typedef struct winFile winFile;
31479 struct winFile {
31480 const sqlite3_io_methods *pMethod; /*** Must be first ***/
31481 sqlite3_vfs *pVfs; /* The VFS used to open this file */
31482 HANDLE h; /* Handle for accessing the file */
31483 unsigned char locktype; /* Type of lock currently held on this file */
31484 short sharedLockByte; /* Randomly chosen byte used as a shared lock */
 
31485 DWORD lastErrno; /* The Windows errno from the last I/O error */
31486 DWORD sectorSize; /* Sector size of the device file is on */
31487 winShm *pShm; /* Instance of shared memory on this file */
31488 const char *zPath; /* Full pathname of this file */
31489 int szChunk; /* Chunk size configured by FCNTL_CHUNK_SIZE */
@@ -31777,10 +31912,58 @@
31777 iLine, iErrno, zFunc, zPath, zMsg
31778 );
31779
31780 return errcode;
31781 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31782
31783 #if SQLITE_OS_WINCE
31784 /*************************************************************************
31785 ** This section contains code for WinCE only.
31786 */
@@ -32195,22 +32378,25 @@
32195 int amt, /* Number of bytes to read */
32196 sqlite3_int64 offset /* Begin reading at this offset */
32197 ){
32198 winFile *pFile = (winFile*)id; /* file handle */
32199 DWORD nRead; /* Number of bytes actually read from file */
 
32200
32201 assert( id!=0 );
32202 SimulateIOError(return SQLITE_IOERR_READ);
32203 OSTRACE(("READ %d lock=%d\n", pFile->h, pFile->locktype));
32204
32205 if( seekWinFile(pFile, offset) ){
32206 return SQLITE_FULL;
32207 }
32208 if( !ReadFile(pFile->h, pBuf, amt, &nRead, 0) ){
 
32209 pFile->lastErrno = GetLastError();
32210 return winLogError(SQLITE_IOERR_READ, "winRead", pFile->zPath);
32211 }
 
32212 if( nRead<(DWORD)amt ){
32213 /* Unread parts of the buffer must be zero-filled */
32214 memset(&((char*)pBuf)[nRead], 0, amt-nRead);
32215 return SQLITE_IOERR_SHORT_READ;
32216 }
@@ -32228,10 +32414,11 @@
32228 int amt, /* Number of bytes to write */
32229 sqlite3_int64 offset /* Offset into the file to begin writing at */
32230 ){
32231 int rc; /* True if error has occured, else false */
32232 winFile *pFile = (winFile*)id; /* File handle */
 
32233
32234 assert( amt>0 );
32235 assert( pFile );
32236 SimulateIOError(return SQLITE_IOERR_WRITE);
32237 SimulateDiskfullError(return SQLITE_FULL);
@@ -32242,11 +32429,16 @@
32242 if( rc==0 ){
32243 u8 *aRem = (u8 *)pBuf; /* Data yet to be written */
32244 int nRem = amt; /* Number of bytes yet to be written */
32245 DWORD nWrite; /* Bytes written by each WriteFile() call */
32246
32247 while( nRem>0 && WriteFile(pFile->h, aRem, nRem, &nWrite, 0) && nWrite>0 ){
 
 
 
 
 
32248 aRem += nWrite;
32249 nRem -= nWrite;
32250 }
32251 if( nRem>0 ){
32252 pFile->lastErrno = GetLastError();
@@ -32253,14 +32445,17 @@
32253 rc = 1;
32254 }
32255 }
32256
32257 if( rc ){
32258 if( pFile->lastErrno==ERROR_HANDLE_DISK_FULL ){
 
32259 return SQLITE_FULL;
32260 }
32261 return winLogError(SQLITE_IOERR_WRITE, "winWrite", pFile->zPath);
 
 
32262 }
32263 return SQLITE_OK;
32264 }
32265
32266 /*
@@ -32650,32 +32845,56 @@
32650
32651 /*
32652 ** Control and query of the open file handle.
32653 */
32654 static int winFileControl(sqlite3_file *id, int op, void *pArg){
 
32655 switch( op ){
32656 case SQLITE_FCNTL_LOCKSTATE: {
32657 *(int*)pArg = ((winFile*)id)->locktype;
32658 return SQLITE_OK;
32659 }
32660 case SQLITE_LAST_ERRNO: {
32661 *(int*)pArg = (int)((winFile*)id)->lastErrno;
32662 return SQLITE_OK;
32663 }
32664 case SQLITE_FCNTL_CHUNK_SIZE: {
32665 ((winFile*)id)->szChunk = *(int *)pArg;
32666 return SQLITE_OK;
32667 }
32668 case SQLITE_FCNTL_SIZE_HINT: {
32669 sqlite3_int64 sz = *(sqlite3_int64*)pArg;
32670 SimulateIOErrorBenign(1);
32671 winTruncate(id, sz);
32672 SimulateIOErrorBenign(0);
32673 return SQLITE_OK;
 
 
 
 
 
 
 
 
 
32674 }
32675 case SQLITE_FCNTL_SYNC_OMITTED: {
32676 return SQLITE_OK;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32677 }
32678 }
32679 return SQLITE_NOTFOUND;
32680 }
32681
@@ -33481,10 +33700,11 @@
33481 int isTemp = 0;
33482 #endif
33483 winFile *pFile = (winFile*)id;
33484 void *zConverted; /* Filename in OS encoding */
33485 const char *zUtf8Name = zName; /* Filename in UTF-8 encoding */
 
33486
33487 /* If argument zPath is a NULL pointer, this function is required to open
33488 ** a temporary file. Use this buffer to store the file name in.
33489 */
33490 char zTmpname[MAX_PATH+1]; /* Buffer used to create temp filename */
@@ -33600,34 +33820,34 @@
33600 #if SQLITE_OS_WINCE
33601 dwFlagsAndAttributes |= FILE_FLAG_RANDOM_ACCESS;
33602 #endif
33603
33604 if( isNT() ){
33605 h = CreateFileW((WCHAR*)zConverted,
33606 dwDesiredAccess,
33607 dwShareMode,
33608 NULL,
33609 dwCreationDisposition,
33610 dwFlagsAndAttributes,
33611 NULL
33612 );
33613 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed.
33614 ** Since the ASCII version of these Windows API do not exist for WINCE,
33615 ** it's important to not reference them for WINCE builds.
33616 */
33617 #if SQLITE_OS_WINCE==0
33618 }else{
33619 h = CreateFileA((char*)zConverted,
33620 dwDesiredAccess,
33621 dwShareMode,
33622 NULL,
33623 dwCreationDisposition,
33624 dwFlagsAndAttributes,
33625 NULL
33626 );
33627 #endif
33628 }
 
 
33629
33630 OSTRACE(("OPEN %d %s 0x%lx %s\n",
33631 h, zName, dwDesiredAccess,
33632 h==INVALID_HANDLE_VALUE ? "failed" : "ok"));
33633
@@ -33690,19 +33910,17 @@
33690 ** file open, we will be unable to delete it. To work around this
33691 ** problem, we delay 100 milliseconds and try to delete again. Up
33692 ** to MX_DELETION_ATTEMPTs deletion attempts are run before giving
33693 ** up and returning an error.
33694 */
33695 #define MX_DELETION_ATTEMPTS 5
33696 static int winDelete(
33697 sqlite3_vfs *pVfs, /* Not used on win32 */
33698 const char *zFilename, /* Name of file to delete */
33699 int syncDir /* Not used on win32 */
33700 ){
33701 int cnt = 0;
33702 DWORD rc;
33703 DWORD error = 0;
33704 void *zConverted;
33705 UNUSED_PARAMETER(pVfs);
33706 UNUSED_PARAMETER(syncDir);
33707
33708 SimulateIOError(return SQLITE_IOERR_DELETE);
@@ -33709,38 +33927,34 @@
33709 zConverted = convertUtf8Filename(zFilename);
33710 if( zConverted==0 ){
33711 return SQLITE_NOMEM;
33712 }
33713 if( isNT() ){
33714 do{
33715 DeleteFileW(zConverted);
33716 }while( ( ((rc = GetFileAttributesW(zConverted)) != INVALID_FILE_ATTRIBUTES)
33717 || ((error = GetLastError()) == ERROR_ACCESS_DENIED))
33718 && (++cnt < MX_DELETION_ATTEMPTS)
33719 && (Sleep(100), 1) );
33720 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed.
33721 ** Since the ASCII version of these Windows API do not exist for WINCE,
33722 ** it's important to not reference them for WINCE builds.
33723 */
33724 #if SQLITE_OS_WINCE==0
33725 }else{
33726 do{
33727 DeleteFileA(zConverted);
33728 }while( ( ((rc = GetFileAttributesA(zConverted)) != INVALID_FILE_ATTRIBUTES)
33729 || ((error = GetLastError()) == ERROR_ACCESS_DENIED))
33730 && (++cnt < MX_DELETION_ATTEMPTS)
33731 && (Sleep(100), 1) );
33732 #endif
33733 }
 
 
 
 
 
33734 free(zConverted);
33735 OSTRACE(("DELETE \"%s\" %s\n", zFilename,
33736 ( (rc==INVALID_FILE_ATTRIBUTES) && (error==ERROR_FILE_NOT_FOUND)) ?
33737 "ok" : "failed" ));
33738
33739 return ( (rc == INVALID_FILE_ATTRIBUTES)
33740 && (error == ERROR_FILE_NOT_FOUND)) ? SQLITE_OK :
33741 winLogError(SQLITE_IOERR_DELETE, "winDelete", zFilename);
33742 }
33743
33744 /*
33745 ** Check the existance and status of a file.
33746 */
@@ -33759,15 +33973,17 @@
33759 zConverted = convertUtf8Filename(zFilename);
33760 if( zConverted==0 ){
33761 return SQLITE_NOMEM;
33762 }
33763 if( isNT() ){
 
33764 WIN32_FILE_ATTRIBUTE_DATA sAttrData;
33765 memset(&sAttrData, 0, sizeof(sAttrData));
33766 if( GetFileAttributesExW((WCHAR*)zConverted,
33767 GetFileExInfoStandard,
33768 &sAttrData) ){
 
33769 /* For an SQLITE_ACCESS_EXISTS query, treat a zero-length file
33770 ** as if it does not exist.
33771 */
33772 if( flags==SQLITE_ACCESS_EXISTS
33773 && sAttrData.nFileSizeHigh==0
@@ -33775,10 +33991,11 @@
33775 attr = INVALID_FILE_ATTRIBUTES;
33776 }else{
33777 attr = sAttrData.dwFileAttributes;
33778 }
33779 }else{
 
33780 if( GetLastError()!=ERROR_FILE_NOT_FOUND ){
33781 winLogError(SQLITE_IOERR_ACCESS, "winAccess", zFilename);
33782 free(zConverted);
33783 return SQLITE_IOERR_ACCESS;
33784 }else{
@@ -33799,11 +34016,12 @@
33799 case SQLITE_ACCESS_READ:
33800 case SQLITE_ACCESS_EXISTS:
33801 rc = attr!=INVALID_FILE_ATTRIBUTES;
33802 break;
33803 case SQLITE_ACCESS_READWRITE:
33804 rc = (attr & FILE_ATTRIBUTE_READONLY)==0;
 
33805 break;
33806 default:
33807 assert(!"Invalid flags argument");
33808 }
33809 *pResOut = rc;
@@ -35805,11 +36023,11 @@
35805 memset(pCache, 0, sz);
35806 if( separateCache ){
35807 pGroup = (PGroup*)&pCache[1];
35808 pGroup->mxPinned = 10;
35809 }else{
35810 pGroup = &pcache1_g.grp;
35811 }
35812 pCache->pGroup = pGroup;
35813 pCache->szPage = szPage;
35814 pCache->bPurgeable = (bPurgeable ? 1 : 0);
35815 if( bPurgeable ){
@@ -45383,17 +45601,19 @@
45383 **
45384 ** The EXCLUSIVE lock is not released before returning.
45385 */
45386 rc = sqlite3OsLock(pWal->pDbFd, SQLITE_LOCK_EXCLUSIVE);
45387 if( rc==SQLITE_OK ){
 
45388 if( pWal->exclusiveMode==WAL_NORMAL_MODE ){
45389 pWal->exclusiveMode = WAL_EXCLUSIVE_MODE;
45390 }
45391 rc = sqlite3WalCheckpoint(
45392 pWal, SQLITE_CHECKPOINT_PASSIVE, 0, 0, sync_flags, nBuf, zBuf, 0, 0
45393 );
45394 if( rc==SQLITE_OK ){
 
45395 isDelete = 1;
45396 }
45397 }
45398
45399 walIndexClose(pWal, isDelete);
@@ -48324,10 +48544,12 @@
48324 **
48325 ** This routine works only for pages that do not contain overflow cells.
48326 */
48327 #define findCell(P,I) \
48328 ((P)->aData + ((P)->maskPage & get2byte(&(P)->aData[(P)->cellOffset+2*(I)])))
 
 
48329
48330 /*
48331 ** This a more complex version of findCell() that works for
48332 ** pages that do contain overflow cells.
48333 */
@@ -51918,11 +52140,11 @@
51918 assert( pCur->apPage[pCur->iPage]->nCell==0 );
51919 return SQLITE_OK;
51920 }
51921 assert( pCur->apPage[0]->intKey || pIdxKey );
51922 for(;;){
51923 int lwr, upr;
51924 Pgno chldPg;
51925 MemPage *pPage = pCur->apPage[pCur->iPage];
51926 int c;
51927
51928 /* pPage->nCell must be greater than zero. If this is the root-page
@@ -51934,18 +52156,18 @@
51934 assert( pPage->nCell>0 );
51935 assert( pPage->intKey==(pIdxKey==0) );
51936 lwr = 0;
51937 upr = pPage->nCell-1;
51938 if( biasRight ){
51939 pCur->aiIdx[pCur->iPage] = (u16)upr;
51940 }else{
51941 pCur->aiIdx[pCur->iPage] = (u16)((upr+lwr)/2);
51942 }
51943 for(;;){
51944 int idx = pCur->aiIdx[pCur->iPage]; /* Index of current cell in pPage */
51945 u8 *pCell; /* Pointer to current cell in pPage */
51946
 
51947 pCur->info.nSize = 0;
51948 pCell = findCell(pPage, idx) + pPage->childPtrSize;
51949 if( pPage->intKey ){
51950 i64 nCellKey;
51951 if( pPage->hasData ){
@@ -52024,11 +52246,11 @@
52024 upr = idx-1;
52025 }
52026 if( lwr>upr ){
52027 break;
52028 }
52029 pCur->aiIdx[pCur->iPage] = (u16)((lwr+upr)/2);
52030 }
52031 assert( lwr==upr+1 );
52032 assert( pPage->isInit );
52033 if( pPage->leaf ){
52034 chldPg = 0;
@@ -52886,13 +53108,13 @@
52886 if( rc ){
52887 *pRC = rc;
52888 return;
52889 }
52890 endPtr = &data[pPage->cellOffset + 2*pPage->nCell - 2];
 
52891 while( ptr<endPtr ){
52892 ptr[0] = ptr[2];
52893 ptr[1] = ptr[3];
52894 ptr += 2;
52895 }
52896 pPage->nCell--;
52897 put2byte(&data[hdr+3], pPage->nCell);
52898 pPage->nFree += 2;
@@ -52929,10 +53151,11 @@
52929 int end; /* First byte past the last cell pointer in data[] */
52930 int ins; /* Index in data[] where new cell pointer is inserted */
52931 int cellOffset; /* Address of first cell pointer in data[] */
52932 u8 *data; /* The content of the whole page */
52933 u8 *ptr; /* Used for moving information around in data[] */
 
52934
52935 int nSkip = (iChild ? 4 : 0);
52936
52937 if( *pRC ) return;
52938
@@ -52979,13 +53202,16 @@
52979 pPage->nFree -= (u16)(2 + sz);
52980 memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip);
52981 if( iChild ){
52982 put4byte(&data[idx], iChild);
52983 }
52984 for(j=end, ptr=&data[j]; j>ins; j-=2, ptr-=2){
52985 ptr[0] = ptr[-2];
52986 ptr[1] = ptr[-1];
 
 
 
52987 }
52988 put2byte(&data[ins], idx);
52989 put2byte(&data[pPage->hdrOffset+3], pPage->nCell);
52990 #ifndef SQLITE_OMIT_AUTOVACUUM
52991 if( pPage->pBt->autoVacuum ){
@@ -53026,14 +53252,15 @@
53026 assert( get2byteNotZero(&data[hdr+5])==nUsable );
53027
53028 pCellptr = &data[pPage->cellOffset + nCell*2];
53029 cellbody = nUsable;
53030 for(i=nCell-1; i>=0; i--){
 
53031 pCellptr -= 2;
53032 cellbody -= aSize[i];
53033 put2byte(pCellptr, cellbody);
53034 memcpy(&data[cellbody], apCell[i], aSize[i]);
53035 }
53036 put2byte(&data[hdr+3], nCell);
53037 put2byte(&data[hdr+5], cellbody);
53038 pPage->nFree -= (nCell*2 + nUsable - cellbody);
53039 pPage->nCell = (u16)nCell;
@@ -53483,16 +53710,28 @@
53483 memcpy(pOld, apOld[i], sizeof(MemPage));
53484 pOld->aData = (void*)&pOld[1];
53485 memcpy(pOld->aData, apOld[i]->aData, pBt->pageSize);
53486
53487 limit = pOld->nCell+pOld->nOverflow;
53488 for(j=0; j<limit; j++){
53489 assert( nCell<nMaxCells );
53490 apCell[nCell] = findOverflowCell(pOld, j);
53491 szCell[nCell] = cellSizePtr(pOld, apCell[nCell]);
53492 nCell++;
53493 }
 
 
 
 
 
 
 
 
 
 
 
 
53494 if( i<nOld-1 && !leafData){
53495 u16 sz = (u16)szNew[i];
53496 u8 *pTemp;
53497 assert( nCell<nMaxCells );
53498 szCell[nCell] = sz;
@@ -57353,15 +57592,15 @@
57353 *ppVal = 0;
57354 return SQLITE_OK;
57355 }
57356 op = pExpr->op;
57357
57358 /* op can only be TK_REGISTER if we have compiled with SQLITE_ENABLE_STAT2.
57359 ** The ifdef here is to enable us to achieve 100% branch test coverage even
57360 ** when SQLITE_ENABLE_STAT2 is omitted.
57361 */
57362 #ifdef SQLITE_ENABLE_STAT2
57363 if( op==TK_REGISTER ) op = pExpr->op2;
57364 #else
57365 if( NEVER(op==TK_REGISTER) ) op = pExpr->op2;
57366 #endif
57367
@@ -57637,17 +57876,10 @@
57637 pOp->p1 = p1;
57638 pOp->p2 = p2;
57639 pOp->p3 = p3;
57640 pOp->p4.p = 0;
57641 pOp->p4type = P4_NOTUSED;
57642 p->expired = 0;
57643 if( op==OP_ParseSchema ){
57644 /* Any program that uses the OP_ParseSchema opcode needs to lock
57645 ** all btrees. */
57646 int j;
57647 for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j);
57648 }
57649 #ifdef SQLITE_DEBUG
57650 pOp->zComment = 0;
57651 if( sqlite3VdbeAddopTrace ) sqlite3VdbePrintOp(0, i, &p->aOp[i]);
57652 #endif
57653 #ifdef VDBE_PROFILE
@@ -57681,10 +57913,24 @@
57681 ){
57682 int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3);
57683 sqlite3VdbeChangeP4(p, addr, zP4, p4type);
57684 return addr;
57685 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57686
57687 /*
57688 ** Add an opcode that includes the p4 value as an integer.
57689 */
57690 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(
@@ -58046,12 +58292,12 @@
58046 /*
58047 ** Change the P2 operand of instruction addr so that it points to
58048 ** the address of the next instruction to be coded.
58049 */
58050 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe *p, int addr){
58051 assert( addr>=0 );
58052 sqlite3VdbeChangeP2(p, addr, p->nOp);
58053 }
58054
58055
58056 /*
58057 ** If the input FuncDef structure is ephemeral, then free it. If
@@ -58979,10 +59225,11 @@
58979 nMem = 10;
58980 }
58981 memset(zCsr, 0, zEnd-zCsr);
58982 zCsr += (zCsr - (u8*)0)&7;
58983 assert( EIGHT_BYTE_ALIGNMENT(zCsr) );
 
58984
58985 /* Memory for registers, parameters, cursor, etc, is allocated in two
58986 ** passes. On the first pass, we try to reuse unused space at the
58987 ** end of the opcode array. If we are unable to satisfy all memory
58988 ** requirements by reusing the opcode array tail, then the second
@@ -61178,10 +61425,18 @@
61178 rc = db->errCode = p->rc;
61179 }
61180 return (rc&db->errMask);
61181 }
61182
 
 
 
 
 
 
 
 
61183 /*
61184 ** This is the top-level implementation of sqlite3_step(). Call
61185 ** sqlite3Step() to do most of the work. If a schema error occurs,
61186 ** call sqlite3Reprepare() and try again.
61187 */
@@ -61196,14 +61451,14 @@
61196 return SQLITE_MISUSE_BKPT;
61197 }
61198 db = v->db;
61199 sqlite3_mutex_enter(db->mutex);
61200 while( (rc = sqlite3Step(v))==SQLITE_SCHEMA
61201 && cnt++ < 5
61202 && (rc2 = rc = sqlite3Reprepare(v))==SQLITE_OK ){
61203 sqlite3_reset(pStmt);
61204 v->expired = 0;
61205 }
61206 if( rc2!=SQLITE_OK && ALWAYS(v->isPrepareV2) && ALWAYS(db->pErr) ){
61207 /* This case occurs after failing to recompile an sql statement.
61208 ** The error message from the SQL compiler has already been loaded
61209 ** into the database handle. This block copies the error message
@@ -63469,11 +63724,11 @@
63469 break;
63470 }
63471
63472 /* Opcode: HaltIfNull P1 P2 P3 P4 *
63473 **
63474 ** Check the value in register P3. If is is NULL then Halt using
63475 ** parameter P1, P2, and P4 as if this were a Halt instruction. If the
63476 ** value in register P3 is not NULL, then this routine is a no-op.
63477 */
63478 case OP_HaltIfNull: { /* in3 */
63479 pIn3 = &aMem[pOp->p3];
@@ -63703,10 +63958,15 @@
63703 assert( memIsValid(pIn1) );
63704 memAboutToChange(p, pOut);
63705 u.ac.zMalloc = pOut->zMalloc;
63706 pOut->zMalloc = 0;
63707 sqlite3VdbeMemMove(pOut, pIn1);
 
 
 
 
 
63708 pIn1->zMalloc = u.ac.zMalloc;
63709 REGISTER_TRACE(u.ac.p2++, pOut);
63710 pIn1++;
63711 pOut++;
63712 }
@@ -64083,10 +64343,20 @@
64083 u.ag.ctx.pColl = pOp[-1].p4.pColl;
64084 }
64085 db->lastRowid = lastRowid;
64086 (*u.ag.ctx.pFunc->xFunc)(&u.ag.ctx, u.ag.n, u.ag.apVal); /* IMP: R-24505-23230 */
64087 lastRowid = db->lastRowid;
 
 
 
 
 
 
 
 
 
 
64088 if( db->mallocFailed ){
64089 /* Even though a malloc() has failed, the implementation of the
64090 ** user function may have called an sqlite3_result_XXX() function
64091 ** to return a value. The following call releases any resources
64092 ** associated with such a value.
@@ -64093,19 +64363,10 @@
64093 */
64094 sqlite3VdbeMemRelease(&u.ag.ctx.s);
64095 goto no_mem;
64096 }
64097
64098 /* If any auxiliary data functions have been called by this user function,
64099 ** immediately call the destructor for any non-static values.
64100 */
64101 if( u.ag.ctx.pVdbeFunc ){
64102 sqlite3VdbeDeleteAuxData(u.ag.ctx.pVdbeFunc, pOp->p1);
64103 pOp->p4.pVdbeFunc = u.ag.ctx.pVdbeFunc;
64104 pOp->p4type = P4_VDBEFUNC;
64105 }
64106
64107 /* If the function returned an error, throw an exception */
64108 if( u.ag.ctx.isError ){
64109 sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.ag.ctx.s));
64110 rc = u.ag.ctx.isError;
64111 }
@@ -64405,11 +64666,11 @@
64405 ** additional information.
64406 **
64407 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
64408 ** true or false and is never NULL. If both operands are NULL then the result
64409 ** of comparison is false. If either operand is NULL then the result is true.
64410 ** If neither operand is NULL the the result is the same as it would be if
64411 ** the SQLITE_NULLEQ flag were omitted from P5.
64412 */
64413 /* Opcode: Eq P1 P2 P3 P4 P5
64414 **
64415 ** This works just like the Lt opcode except that the jump is taken if
@@ -64417,11 +64678,11 @@
64417 ** See the Lt opcode for additional information.
64418 **
64419 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
64420 ** true or false and is never NULL. If both operands are NULL then the result
64421 ** of comparison is true. If either operand is NULL then the result is false.
64422 ** If neither operand is NULL the the result is the same as it would be if
64423 ** the SQLITE_NULLEQ flag were omitted from P5.
64424 */
64425 /* Opcode: Le P1 P2 P3 P4 P5
64426 **
64427 ** This works just like the Lt opcode except that the jump is taken if
@@ -64702,17 +64963,17 @@
64702 break;
64703 }
64704
64705 /* Opcode: If P1 P2 P3 * *
64706 **
64707 ** Jump to P2 if the value in register P1 is true. The value is
64708 ** is considered true if it is numeric and non-zero. If the value
64709 ** in P1 is NULL then take the jump if P3 is true.
64710 */
64711 /* Opcode: IfNot P1 P2 P3 * *
64712 **
64713 ** Jump to P2 if the value in register P1 is False. The value is
64714 ** is considered true if it has a numeric value of zero. If the value
64715 ** in P1 is NULL then take the jump if P3 is true.
64716 */
64717 case OP_If: /* jump, in1 */
64718 case OP_IfNot: { /* jump, in1 */
@@ -65889,11 +66150,11 @@
65889 ** automatically created table with root-page 1 (an BLOB_INTKEY table).
65890 */
65891 if( pOp->p4.pKeyInfo ){
65892 int pgno;
65893 assert( pOp->p4type==P4_KEYINFO );
65894 rc = sqlite3BtreeCreateTable(u.ax.pCx->pBt, &pgno, BTREE_BLOBKEY);
65895 if( rc==SQLITE_OK ){
65896 assert( pgno==MASTER_ROOT+1 );
65897 rc = sqlite3BtreeCursor(u.ax.pCx->pBt, pgno, 1,
65898 (KeyInfo*)pOp->p4.z, u.ax.pCx->pCursor);
65899 u.ax.pCx->pKeyInfo = pOp->p4.pKeyInfo;
@@ -66362,11 +66623,11 @@
66362 break;
66363 }
66364
66365 /* Opcode: NotExists P1 P2 P3 * *
66366 **
66367 ** Use the content of register P3 as a integer key. If a record
66368 ** with that key does not exist in table of P1, then jump to P2.
66369 ** If the record does exist, then fall through. The cursor is left
66370 ** pointing to the record if it exists.
66371 **
66372 ** The difference between this operation and NotFound is that this
@@ -66440,11 +66701,11 @@
66440 ** written to register P2.
66441 **
66442 ** If P3>0 then P3 is a register in the root frame of this VDBE that holds
66443 ** the largest previously generated record number. No new record numbers are
66444 ** allowed to be less than this value. When this value reaches its maximum,
66445 ** a SQLITE_FULL error is generated. The P3 register is updated with the '
66446 ** generated record number. This P3 mechanism is used to help implement the
66447 ** AUTOINCREMENT feature.
66448 */
66449 case OP_NewRowid: { /* out2-prerelease */
66450 #if 0 /* local variables moved into u.be */
@@ -67082,11 +67343,11 @@
67082 break;
67083 }
67084
67085 /* Opcode: IdxInsert P1 P2 P3 * P5
67086 **
67087 ** Register P2 holds a SQL index key made using the
67088 ** MakeRecord instructions. This opcode writes that key
67089 ** into the index P1. Data for the entry is nil.
67090 **
67091 ** P3 is a flag that provides a hint to the b-tree layer that this
67092 ** insert is likely to be an append.
@@ -70928,15 +71189,29 @@
70928 /* Recursively resolve names in all subqueries
70929 */
70930 for(i=0; i<p->pSrc->nSrc; i++){
70931 struct SrcList_item *pItem = &p->pSrc->a[i];
70932 if( pItem->pSelect ){
 
 
70933 const char *zSavedContext = pParse->zAuthContext;
 
 
 
 
 
 
 
 
70934 if( pItem->zName ) pParse->zAuthContext = pItem->zName;
70935 sqlite3ResolveSelectNames(pParse, pItem->pSelect, pOuterNC);
70936 pParse->zAuthContext = zSavedContext;
70937 if( pParse->nErr || db->mallocFailed ) return WRC_Abort;
 
 
 
 
70938 }
70939 }
70940
70941 /* If there are no aggregate functions in the result-set, and no GROUP BY
70942 ** expression, do not allow aggregates in any of the other expressions.
@@ -72040,10 +72315,11 @@
72040 pNewItem->zName = sqlite3DbStrDup(db, pOldItem->zName);
72041 pNewItem->zAlias = sqlite3DbStrDup(db, pOldItem->zAlias);
72042 pNewItem->jointype = pOldItem->jointype;
72043 pNewItem->iCursor = pOldItem->iCursor;
72044 pNewItem->isPopulated = pOldItem->isPopulated;
 
72045 pNewItem->zIndex = sqlite3DbStrDup(db, pOldItem->zIndex);
72046 pNewItem->notIndexed = pOldItem->notIndexed;
72047 pNewItem->pIndex = pOldItem->pIndex;
72048 pTab = pNewItem->pTab = pOldItem->pTab;
72049 if( pTab ){
@@ -75255,18 +75531,18 @@
75255 sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
75256
75257 /* Reload the table, index and permanent trigger schemas. */
75258 zWhere = sqlite3MPrintf(pParse->db, "tbl_name=%Q", zName);
75259 if( !zWhere ) return;
75260 sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC);
75261
75262 #ifndef SQLITE_OMIT_TRIGGER
75263 /* Now, if the table is not stored in the temp database, reload any temp
75264 ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined.
75265 */
75266 if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){
75267 sqlite3VdbeAddOp4(v, OP_ParseSchema, 1, 0, 0, zWhere, P4_DYNAMIC);
75268 }
75269 #endif
75270 }
75271
75272 /*
@@ -75736,10 +76012,100 @@
75736 ** May you find forgiveness for yourself and forgive others.
75737 ** May you share freely, never taking more than you give.
75738 **
75739 *************************************************************************
75740 ** This file contains code associated with the ANALYZE command.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75741 */
75742 #ifndef SQLITE_OMIT_ANALYZE
75743
75744 /*
75745 ** This routine generates code that opens the sqlite_stat1 table for
@@ -75767,12 +76133,18 @@
75767 static const struct {
75768 const char *zName;
75769 const char *zCols;
75770 } aTable[] = {
75771 { "sqlite_stat1", "tbl,idx,stat" },
75772 #ifdef SQLITE_ENABLE_STAT2
75773 { "sqlite_stat2", "tbl,idx,sampleno,sample" },
 
 
 
 
 
 
75774 #endif
75775 };
75776
75777 int aRoot[] = {0, 0};
75778 u8 aCreateTbl[] = {0, 0};
@@ -75784,10 +76156,21 @@
75784 if( v==0 ) return;
75785 assert( sqlite3BtreeHoldsAllMutexes(db) );
75786 assert( sqlite3VdbeDb(v)==db );
75787 pDb = &db->aDb[iDb];
75788
 
 
 
 
 
 
 
 
 
 
 
75789 for(i=0; i<ArraySize(aTable); i++){
75790 const char *zTab = aTable[i].zName;
75791 Table *pStat;
75792 if( (pStat = sqlite3FindTable(db, zTab, pDb->zName))==0 ){
75793 /* The sqlite_stat[12] table does not exist. Create it. Note that a
@@ -75814,17 +76197,238 @@
75814 sqlite3VdbeAddOp2(v, OP_Clear, aRoot[i], iDb);
75815 }
75816 }
75817 }
75818
75819 /* Open the sqlite_stat[12] tables for writing. */
75820 for(i=0; i<ArraySize(aTable); i++){
75821 sqlite3VdbeAddOp3(v, OP_OpenWrite, iStatCur+i, aRoot[i], iDb);
75822 sqlite3VdbeChangeP4(v, -1, (char *)3, P4_INT32);
75823 sqlite3VdbeChangeP5(v, aCreateTbl[i]);
75824 }
75825 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75826
75827 /*
75828 ** Generate code to do an analysis of all indices associated with
75829 ** a single table.
75830 */
@@ -75844,24 +76448,31 @@
75844 int endOfLoop; /* The end of the loop */
75845 int jZeroRows = -1; /* Jump from here if number of rows is zero */
75846 int iDb; /* Index of database containing pTab */
75847 int regTabname = iMem++; /* Register containing table name */
75848 int regIdxname = iMem++; /* Register containing index name */
75849 int regSampleno = iMem++; /* Register containing next sample number */
75850 int regCol = iMem++; /* Content of a column analyzed table */
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
75851 int regRec = iMem++; /* Register holding completed record */
75852 int regTemp = iMem++; /* Temporary use register */
75853 int regRowid = iMem++; /* Rowid for the inserted record */
75854
75855 #ifdef SQLITE_ENABLE_STAT2
75856 int addr = 0; /* Instruction address */
75857 int regTemp2 = iMem++; /* Temporary use register */
75858 int regSamplerecno = iMem++; /* Index of next sample to record */
75859 int regRecno = iMem++; /* Current sample index */
75860 int regLast = iMem++; /* Index of last sample to record */
75861 int regFirst = iMem++; /* Index of first sample to record */
75862 #endif
75863
75864 v = sqlite3GetVdbe(pParse);
75865 if( v==0 || NEVER(pTab==0) ){
75866 return;
75867 }
@@ -75890,17 +76501,22 @@
75890 iIdxCur = pParse->nTab++;
75891 sqlite3VdbeAddOp4(v, OP_String8, 0, regTabname, 0, pTab->zName, 0);
75892 for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
75893 int nCol;
75894 KeyInfo *pKey;
 
 
75895
75896 if( pOnlyIdx && pOnlyIdx!=pIdx ) continue;
 
75897 nCol = pIdx->nColumn;
75898 pKey = sqlite3IndexKeyinfo(pParse, pIdx);
75899 if( iMem+1+(nCol*2)>pParse->nMem ){
75900 pParse->nMem = iMem+1+(nCol*2);
75901 }
 
 
75902
75903 /* Open a cursor to the index to be analyzed. */
75904 assert( iDb==sqlite3SchemaToIndex(db, pIdx->pSchema) );
75905 sqlite3VdbeAddOp4(v, OP_OpenRead, iIdxCur, pIdx->tnum, iDb,
75906 (char *)pKey, P4_KEYINFO_HANDOFF);
@@ -75907,35 +76523,24 @@
75907 VdbeComment((v, "%s", pIdx->zName));
75908
75909 /* Populate the register containing the index name. */
75910 sqlite3VdbeAddOp4(v, OP_String8, 0, regIdxname, 0, pIdx->zName, 0);
75911
75912 #ifdef SQLITE_ENABLE_STAT2
75913
75914 /* If this iteration of the loop is generating code to analyze the
75915 ** first index in the pTab->pIndex list, then register regLast has
75916 ** not been populated. In this case populate it now. */
75917 if( pTab->pIndex==pIdx ){
75918 sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES, regSamplerecno);
75919 sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES*2-1, regTemp);
75920 sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES*2, regTemp2);
75921
75922 sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regLast);
75923 sqlite3VdbeAddOp2(v, OP_Null, 0, regFirst);
75924 addr = sqlite3VdbeAddOp3(v, OP_Lt, regSamplerecno, 0, regLast);
75925 sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regLast, regFirst);
75926 sqlite3VdbeAddOp3(v, OP_Multiply, regLast, regTemp, regLast);
75927 sqlite3VdbeAddOp2(v, OP_AddImm, regLast, SQLITE_INDEX_SAMPLES*2-2);
75928 sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regLast, regLast);
75929 sqlite3VdbeJumpHere(v, addr);
75930 }
75931
75932 /* Zero the regSampleno and regRecno registers. */
75933 sqlite3VdbeAddOp2(v, OP_Integer, 0, regSampleno);
75934 sqlite3VdbeAddOp2(v, OP_Integer, 0, regRecno);
75935 sqlite3VdbeAddOp2(v, OP_Copy, regFirst, regSamplerecno);
75936 #endif
75937
75938 /* The block of memory cells initialized here is used as follows.
75939 **
75940 ** iMem:
75941 ** The total number of rows in the table.
@@ -75961,79 +76566,87 @@
75961 /* Start the analysis loop. This loop runs through all the entries in
75962 ** the index b-tree. */
75963 endOfLoop = sqlite3VdbeMakeLabel(v);
75964 sqlite3VdbeAddOp2(v, OP_Rewind, iIdxCur, endOfLoop);
75965 topOfLoop = sqlite3VdbeCurrentAddr(v);
75966 sqlite3VdbeAddOp2(v, OP_AddImm, iMem, 1);
75967
75968 for(i=0; i<nCol; i++){
75969 CollSeq *pColl;
75970 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regCol);
75971 if( i==0 ){
75972 #ifdef SQLITE_ENABLE_STAT2
75973 /* Check if the record that cursor iIdxCur points to contains a
75974 ** value that should be stored in the sqlite_stat2 table. If so,
75975 ** store it. */
75976 int ne = sqlite3VdbeAddOp3(v, OP_Ne, regRecno, 0, regSamplerecno);
75977 assert( regTabname+1==regIdxname
75978 && regTabname+2==regSampleno
75979 && regTabname+3==regCol
75980 );
75981 sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL);
75982 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 4, regRec, "aaab", 0);
75983 sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur+1, regRowid);
75984 sqlite3VdbeAddOp3(v, OP_Insert, iStatCur+1, regRec, regRowid);
75985
75986 /* Calculate new values for regSamplerecno and regSampleno.
75987 **
75988 ** sampleno = sampleno + 1
75989 ** samplerecno = samplerecno+(remaining records)/(remaining samples)
75990 */
75991 sqlite3VdbeAddOp2(v, OP_AddImm, regSampleno, 1);
75992 sqlite3VdbeAddOp3(v, OP_Subtract, regRecno, regLast, regTemp);
75993 sqlite3VdbeAddOp2(v, OP_AddImm, regTemp, -1);
75994 sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES, regTemp2);
75995 sqlite3VdbeAddOp3(v, OP_Subtract, regSampleno, regTemp2, regTemp2);
75996 sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regTemp, regTemp);
75997 sqlite3VdbeAddOp3(v, OP_Add, regSamplerecno, regTemp, regSamplerecno);
75998
75999 sqlite3VdbeJumpHere(v, ne);
76000 sqlite3VdbeAddOp2(v, OP_AddImm, regRecno, 1);
76001 #endif
76002
76003 /* Always record the very first row */
76004 sqlite3VdbeAddOp1(v, OP_IfNot, iMem+1);
76005 }
76006 assert( pIdx->azColl!=0 );
76007 assert( pIdx->azColl[i]!=0 );
76008 pColl = sqlite3LocateCollSeq(pParse, pIdx->azColl[i]);
76009 sqlite3VdbeAddOp4(v, OP_Ne, regCol, 0, iMem+nCol+i+1,
76010 (char*)pColl, P4_COLLSEQ);
76011 sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
76012 }
76013 if( db->mallocFailed ){
76014 /* If a malloc failure has occurred, then the result of the expression
76015 ** passed as the second argument to the call to sqlite3VdbeJumpHere()
76016 ** below may be negative. Which causes an assert() to fail (or an
76017 ** out-of-bounds write if SQLITE_DEBUG is not defined). */
76018 return;
76019 }
76020 sqlite3VdbeAddOp2(v, OP_Goto, 0, endOfLoop);
76021 for(i=0; i<nCol; i++){
76022 int addr2 = sqlite3VdbeCurrentAddr(v) - (nCol*2);
76023 if( i==0 ){
76024 sqlite3VdbeJumpHere(v, addr2-1); /* Set jump dest for the OP_IfNot */
 
 
 
 
 
 
 
 
 
76025 }
76026 sqlite3VdbeJumpHere(v, addr2); /* Set jump dest for the OP_Ne */
76027 sqlite3VdbeAddOp2(v, OP_AddImm, iMem+i+1, 1);
76028 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, iMem+nCol+i+1);
76029 }
 
76030
76031 /* End of the analysis loop. */
76032 sqlite3VdbeResolveLabel(v, endOfLoop);
 
76033 sqlite3VdbeAddOp2(v, OP_Next, iIdxCur, topOfLoop);
76034 sqlite3VdbeAddOp1(v, OP_Close, iIdxCur);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76035
76036 /* Store the results in sqlite_stat1.
76037 **
76038 ** The result is a single row of the sqlite_stat1 table. The first
76039 ** two columns are the names of the table and index. The third column
@@ -76049,50 +76662,51 @@
76049 **
76050 ** If K==0 then no entry is made into the sqlite_stat1 table.
76051 ** If K>0 then it is always the case the D>0 so division by zero
76052 ** is never possible.
76053 */
76054 sqlite3VdbeAddOp2(v, OP_SCopy, iMem, regSampleno);
76055 if( jZeroRows<0 ){
76056 jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, iMem);
76057 }
76058 for(i=0; i<nCol; i++){
76059 sqlite3VdbeAddOp4(v, OP_String8, 0, regTemp, 0, " ", 0);
76060 sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regSampleno, regSampleno);
76061 sqlite3VdbeAddOp3(v, OP_Add, iMem, iMem+i+1, regTemp);
76062 sqlite3VdbeAddOp2(v, OP_AddImm, regTemp, -1);
76063 sqlite3VdbeAddOp3(v, OP_Divide, iMem+i+1, regTemp, regTemp);
76064 sqlite3VdbeAddOp1(v, OP_ToInt, regTemp);
76065 sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regSampleno, regSampleno);
76066 }
76067 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0);
76068 sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regRowid);
76069 sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regRowid);
76070 sqlite3VdbeChangeP5(v, OPFLAG_APPEND);
76071 }
76072
76073 /* If the table has no indices, create a single sqlite_stat1 entry
76074 ** containing NULL as the index name and the row count as the content.
76075 */
76076 if( pTab->pIndex==0 ){
76077 sqlite3VdbeAddOp3(v, OP_OpenRead, iIdxCur, pTab->tnum, iDb);
76078 VdbeComment((v, "%s", pTab->zName));
76079 sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regSampleno);
76080 sqlite3VdbeAddOp1(v, OP_Close, iIdxCur);
76081 jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, regSampleno);
76082 }else{
76083 sqlite3VdbeJumpHere(v, jZeroRows);
76084 jZeroRows = sqlite3VdbeAddOp0(v, OP_Goto);
76085 }
76086 sqlite3VdbeAddOp2(v, OP_Null, 0, regIdxname);
76087 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0);
76088 sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regRowid);
76089 sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regRowid);
76090 sqlite3VdbeChangeP5(v, OPFLAG_APPEND);
76091 if( pParse->nMem<regRec ) pParse->nMem = regRec;
76092 sqlite3VdbeJumpHere(v, jZeroRows);
76093 }
 
76094
76095 /*
76096 ** Generate code that will cause the most recent index analysis to
76097 ** be loaded into internal hash tables where is can be used.
76098 */
@@ -76113,11 +76727,11 @@
76113 int iStatCur;
76114 int iMem;
76115
76116 sqlite3BeginWriteOperation(pParse, 0, iDb);
76117 iStatCur = pParse->nTab;
76118 pParse->nTab += 2;
76119 openStatTable(pParse, iDb, iStatCur, 0, 0);
76120 iMem = pParse->nMem+1;
76121 assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
76122 for(k=sqliteHashFirst(&pSchema->tblHash); k; k=sqliteHashNext(k)){
76123 Table *pTab = (Table*)sqliteHashData(k);
@@ -76138,11 +76752,11 @@
76138 assert( pTab!=0 );
76139 assert( sqlite3BtreeHoldsAllMutexes(pParse->db) );
76140 iDb = sqlite3SchemaToIndex(pParse->db, pTab->pSchema);
76141 sqlite3BeginWriteOperation(pParse, 0, iDb);
76142 iStatCur = pParse->nTab;
76143 pParse->nTab += 2;
76144 if( pOnlyIdx ){
76145 openStatTable(pParse, iDb, iStatCur, pOnlyIdx->zName, "idx");
76146 }else{
76147 openStatTable(pParse, iDb, iStatCur, pTab->zName, "tbl");
76148 }
@@ -76243,11 +76857,11 @@
76243 static int analysisLoader(void *pData, int argc, char **argv, char **NotUsed){
76244 analysisInfo *pInfo = (analysisInfo*)pData;
76245 Index *pIndex;
76246 Table *pTable;
76247 int i, c, n;
76248 unsigned int v;
76249 const char *z;
76250
76251 assert( argc==3 );
76252 UNUSED_PARAMETER2(NotUsed, argc);
76253
@@ -76286,40 +76900,172 @@
76286 /*
76287 ** If the Index.aSample variable is not NULL, delete the aSample[] array
76288 ** and its contents.
76289 */
76290 SQLITE_PRIVATE void sqlite3DeleteIndexSamples(sqlite3 *db, Index *pIdx){
76291 #ifdef SQLITE_ENABLE_STAT2
76292 if( pIdx->aSample ){
76293 int j;
76294 for(j=0; j<SQLITE_INDEX_SAMPLES; j++){
76295 IndexSample *p = &pIdx->aSample[j];
76296 if( p->eType==SQLITE_TEXT || p->eType==SQLITE_BLOB ){
76297 sqlite3DbFree(db, p->u.z);
76298 }
76299 }
76300 sqlite3DbFree(db, pIdx->aSample);
76301 }
 
 
 
76302 #else
76303 UNUSED_PARAMETER(db);
76304 UNUSED_PARAMETER(pIdx);
76305 #endif
76306 }
76307
 
76308 /*
76309 ** Load the content of the sqlite_stat1 and sqlite_stat2 tables. The
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76310 ** contents of sqlite_stat1 are used to populate the Index.aiRowEst[]
76311 ** arrays. The contents of sqlite_stat2 are used to populate the
76312 ** Index.aSample[] arrays.
76313 **
76314 ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR
76315 ** is returned. In this case, even if SQLITE_ENABLE_STAT2 was defined
76316 ** during compilation and the sqlite_stat2 table is present, no data is
76317 ** read from it.
76318 **
76319 ** If SQLITE_ENABLE_STAT2 was defined during compilation and the
76320 ** sqlite_stat2 table is not present in the database, SQLITE_ERROR is
76321 ** returned. However, in this case, data is read from the sqlite_stat1
76322 ** table (if it is present) before returning.
76323 **
76324 ** If an OOM error occurs, this function always sets db->mallocFailed.
76325 ** This means if the caller does not care about other errors, the return
@@ -76337,12 +77083,14 @@
76337 /* Clear any prior statistics */
76338 assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
76339 for(i=sqliteHashFirst(&db->aDb[iDb].pSchema->idxHash);i;i=sqliteHashNext(i)){
76340 Index *pIdx = sqliteHashData(i);
76341 sqlite3DefaultRowEst(pIdx);
 
76342 sqlite3DeleteIndexSamples(db, pIdx);
76343 pIdx->aSample = 0;
 
76344 }
76345
76346 /* Check to make sure the sqlite_stat1 table exists */
76347 sInfo.db = db;
76348 sInfo.zDatabase = db->aDb[iDb].zName;
@@ -76350,91 +77098,23 @@
76350 return SQLITE_ERROR;
76351 }
76352
76353 /* Load new statistics out of the sqlite_stat1 table */
76354 zSql = sqlite3MPrintf(db,
76355 "SELECT tbl, idx, stat FROM %Q.sqlite_stat1", sInfo.zDatabase);
76356 if( zSql==0 ){
76357 rc = SQLITE_NOMEM;
76358 }else{
76359 rc = sqlite3_exec(db, zSql, analysisLoader, &sInfo, 0);
76360 sqlite3DbFree(db, zSql);
76361 }
76362
76363
76364 /* Load the statistics from the sqlite_stat2 table. */
76365 #ifdef SQLITE_ENABLE_STAT2
76366 if( rc==SQLITE_OK && !sqlite3FindTable(db, "sqlite_stat2", sInfo.zDatabase) ){
76367 rc = SQLITE_ERROR;
76368 }
76369 if( rc==SQLITE_OK ){
76370 sqlite3_stmt *pStmt = 0;
76371
76372 zSql = sqlite3MPrintf(db,
76373 "SELECT idx,sampleno,sample FROM %Q.sqlite_stat2", sInfo.zDatabase);
76374 if( !zSql ){
76375 rc = SQLITE_NOMEM;
76376 }else{
76377 rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
76378 sqlite3DbFree(db, zSql);
76379 }
76380
76381 if( rc==SQLITE_OK ){
76382 while( sqlite3_step(pStmt)==SQLITE_ROW ){
76383 char *zIndex; /* Index name */
76384 Index *pIdx; /* Pointer to the index object */
76385
76386 zIndex = (char *)sqlite3_column_text(pStmt, 0);
76387 pIdx = zIndex ? sqlite3FindIndex(db, zIndex, sInfo.zDatabase) : 0;
76388 if( pIdx ){
76389 int iSample = sqlite3_column_int(pStmt, 1);
76390 if( iSample<SQLITE_INDEX_SAMPLES && iSample>=0 ){
76391 int eType = sqlite3_column_type(pStmt, 2);
76392
76393 if( pIdx->aSample==0 ){
76394 static const int sz = sizeof(IndexSample)*SQLITE_INDEX_SAMPLES;
76395 pIdx->aSample = (IndexSample *)sqlite3DbMallocRaw(0, sz);
76396 if( pIdx->aSample==0 ){
76397 db->mallocFailed = 1;
76398 break;
76399 }
76400 memset(pIdx->aSample, 0, sz);
76401 }
76402
76403 assert( pIdx->aSample );
76404 {
76405 IndexSample *pSample = &pIdx->aSample[iSample];
76406 pSample->eType = (u8)eType;
76407 if( eType==SQLITE_INTEGER || eType==SQLITE_FLOAT ){
76408 pSample->u.r = sqlite3_column_double(pStmt, 2);
76409 }else if( eType==SQLITE_TEXT || eType==SQLITE_BLOB ){
76410 const char *z = (const char *)(
76411 (eType==SQLITE_BLOB) ?
76412 sqlite3_column_blob(pStmt, 2):
76413 sqlite3_column_text(pStmt, 2)
76414 );
76415 int n = sqlite3_column_bytes(pStmt, 2);
76416 if( n>24 ){
76417 n = 24;
76418 }
76419 pSample->nByte = (u8)n;
76420 if( n < 1){
76421 pSample->u.z = 0;
76422 }else{
76423 pSample->u.z = sqlite3DbStrNDup(0, z, n);
76424 if( pSample->u.z==0 ){
76425 db->mallocFailed = 1;
76426 break;
76427 }
76428 }
76429 }
76430 }
76431 }
76432 }
76433 }
76434 rc = sqlite3_finalize(pStmt);
76435 }
76436 }
76437 #endif
76438
76439 if( rc==SQLITE_NOMEM ){
76440 db->mallocFailed = 1;
@@ -78875,12 +79555,12 @@
78875 }
78876 }
78877 #endif
78878
78879 /* Reparse everything to update our internal data structures */
78880 sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0,
78881 sqlite3MPrintf(db, "tbl_name='%q'",p->zName), P4_DYNAMIC);
78882 }
78883
78884
78885 /* Add the table to the in-memory representation of the database.
78886 */
@@ -79235,10 +79915,108 @@
79235 iDestroyed = iLargest;
79236 }
79237 }
79238 #endif
79239 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
79240
79241 /*
79242 ** This routine is called to do the work of a DROP TABLE statement.
79243 ** pName is the name of the table to be dropped.
79244 */
@@ -79304,11 +80082,11 @@
79304 if( sqlite3AuthCheck(pParse, SQLITE_DELETE, pTab->zName, 0, zDb) ){
79305 goto exit_drop_table;
79306 }
79307 }
79308 #endif
79309 if( sqlite3StrNICmp(pTab->zName, "sqlite_", 7)==0 ){
79310 sqlite3ErrorMsg(pParse, "table %s may not be dropped", pTab->zName);
79311 goto exit_drop_table;
79312 }
79313
79314 #ifndef SQLITE_OMIT_VIEW
@@ -79328,79 +80106,15 @@
79328 /* Generate code to remove the table from the master table
79329 ** on disk.
79330 */
79331 v = sqlite3GetVdbe(pParse);
79332 if( v ){
79333 Trigger *pTrigger;
79334 Db *pDb = &db->aDb[iDb];
79335 sqlite3BeginWriteOperation(pParse, 1, iDb);
79336
79337 #ifndef SQLITE_OMIT_VIRTUALTABLE
79338 if( IsVirtual(pTab) ){
79339 sqlite3VdbeAddOp0(v, OP_VBegin);
79340 }
79341 #endif
79342 sqlite3FkDropTable(pParse, pName, pTab);
79343
79344 /* Drop all triggers associated with the table being dropped. Code
79345 ** is generated to remove entries from sqlite_master and/or
79346 ** sqlite_temp_master if required.
79347 */
79348 pTrigger = sqlite3TriggerList(pParse, pTab);
79349 while( pTrigger ){
79350 assert( pTrigger->pSchema==pTab->pSchema ||
79351 pTrigger->pSchema==db->aDb[1].pSchema );
79352 sqlite3DropTriggerPtr(pParse, pTrigger);
79353 pTrigger = pTrigger->pNext;
79354 }
79355
79356 #ifndef SQLITE_OMIT_AUTOINCREMENT
79357 /* Remove any entries of the sqlite_sequence table associated with
79358 ** the table being dropped. This is done before the table is dropped
79359 ** at the btree level, in case the sqlite_sequence table needs to
79360 ** move as a result of the drop (can happen in auto-vacuum mode).
79361 */
79362 if( pTab->tabFlags & TF_Autoincrement ){
79363 sqlite3NestedParse(pParse,
79364 "DELETE FROM %s.sqlite_sequence WHERE name=%Q",
79365 pDb->zName, pTab->zName
79366 );
79367 }
79368 #endif
79369
79370 /* Drop all SQLITE_MASTER table and index entries that refer to the
79371 ** table. The program name loops through the master table and deletes
79372 ** every row that refers to a table of the same name as the one being
79373 ** dropped. Triggers are handled seperately because a trigger can be
79374 ** created in the temp database that refers to a table in another
79375 ** database.
79376 */
79377 sqlite3NestedParse(pParse,
79378 "DELETE FROM %Q.%s WHERE tbl_name=%Q and type!='trigger'",
79379 pDb->zName, SCHEMA_TABLE(iDb), pTab->zName);
79380
79381 /* Drop any statistics from the sqlite_stat1 table, if it exists */
79382 if( sqlite3FindTable(db, "sqlite_stat1", db->aDb[iDb].zName) ){
79383 sqlite3NestedParse(pParse,
79384 "DELETE FROM %Q.sqlite_stat1 WHERE tbl=%Q", pDb->zName, pTab->zName
79385 );
79386 }
79387
79388 if( !isView && !IsVirtual(pTab) ){
79389 destroyTable(pParse, pTab);
79390 }
79391
79392 /* Remove the table entry from SQLite's internal schema and modify
79393 ** the schema cookie.
79394 */
79395 if( IsVirtual(pTab) ){
79396 sqlite3VdbeAddOp4(v, OP_VDestroy, iDb, 0, 0, pTab->zName, 0);
79397 }
79398 sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
79399 sqlite3ChangeCookie(pParse, iDb);
79400 }
79401 sqliteViewResetAll(db, iDb);
79402
79403 exit_drop_table:
79404 sqlite3SrcListDelete(db, pName);
79405 }
79406
@@ -79843,24 +80557,24 @@
79843 */
79844 nName = sqlite3Strlen30(zName);
79845 nCol = pList->nExpr;
79846 pIndex = sqlite3DbMallocZero(db,
79847 sizeof(Index) + /* Index structure */
 
79848 sizeof(int)*nCol + /* Index.aiColumn */
79849 sizeof(int)*(nCol+1) + /* Index.aiRowEst */
79850 sizeof(char *)*nCol + /* Index.azColl */
79851 sizeof(u8)*nCol + /* Index.aSortOrder */
79852 nName + 1 + /* Index.zName */
79853 nExtra /* Collation sequence names */
79854 );
79855 if( db->mallocFailed ){
79856 goto exit_create_index;
79857 }
79858 pIndex->azColl = (char**)(&pIndex[1]);
 
79859 pIndex->aiColumn = (int *)(&pIndex->azColl[nCol]);
79860 pIndex->aiRowEst = (unsigned *)(&pIndex->aiColumn[nCol]);
79861 pIndex->aSortOrder = (u8 *)(&pIndex->aiRowEst[nCol+1]);
79862 pIndex->zName = (char *)(&pIndex->aSortOrder[nCol]);
79863 zExtra = (char *)(&pIndex->zName[nName+1]);
79864 memcpy(pIndex->zName, zName, nName+1);
79865 pIndex->pTable = pTab;
79866 pIndex->nColumn = pList->nExpr;
@@ -80047,11 +80761,11 @@
80047 if( pStart ){
80048 assert( pEnd!=0 );
80049 /* A named index with an explicit CREATE INDEX statement */
80050 zStmt = sqlite3MPrintf(db, "CREATE%s INDEX %.*s",
80051 onError==OE_None ? "" : " UNIQUE",
80052 pEnd->z - pName->z + 1,
80053 pName->z);
80054 }else{
80055 /* An automatic index created by a PRIMARY KEY or UNIQUE constraint */
80056 /* zStmt = sqlite3MPrintf(""); */
80057 zStmt = 0;
@@ -80073,13 +80787,12 @@
80073 ** to invalidate all pre-compiled statements.
80074 */
80075 if( pTblName ){
80076 sqlite3RefillIndex(pParse, pIndex, iMem);
80077 sqlite3ChangeCookie(pParse, iDb);
80078 sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0,
80079 sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName),
80080 P4_DYNAMIC);
80081 sqlite3VdbeAddOp1(v, OP_Expire, 0);
80082 }
80083 }
80084
80085 /* When adding an index to the list of indices for a table, make
@@ -80134,13 +80847,13 @@
80134 ** Apart from that, we have little to go on besides intuition as to
80135 ** how aiRowEst[] should be initialized. The numbers generated here
80136 ** are based on typical values found in actual indices.
80137 */
80138 SQLITE_PRIVATE void sqlite3DefaultRowEst(Index *pIdx){
80139 unsigned *a = pIdx->aiRowEst;
80140 int i;
80141 unsigned n;
80142 assert( a!=0 );
80143 a[0] = pIdx->pTable->nRowEst;
80144 if( a[0]<10 ) a[0] = 10;
80145 n = 10;
80146 for(i=1; i<=pIdx->nColumn; i++){
@@ -80206,19 +80919,13 @@
80206 v = sqlite3GetVdbe(pParse);
80207 if( v ){
80208 sqlite3BeginWriteOperation(pParse, 1, iDb);
80209 sqlite3NestedParse(pParse,
80210 "DELETE FROM %Q.%s WHERE name=%Q AND type='index'",
80211 db->aDb[iDb].zName, SCHEMA_TABLE(iDb),
80212 pIndex->zName
80213 );
80214 if( sqlite3FindTable(db, "sqlite_stat1", db->aDb[iDb].zName) ){
80215 sqlite3NestedParse(pParse,
80216 "DELETE FROM %Q.sqlite_stat1 WHERE idx=%Q",
80217 db->aDb[iDb].zName, pIndex->zName
80218 );
80219 }
80220 sqlite3ChangeCookie(pParse, iDb);
80221 destroyRootPage(pParse, pIndex->tnum, iDb);
80222 sqlite3VdbeAddOp4(v, OP_DropIndex, iDb, 0, 0, pIndex->zName, 0);
80223 }
80224
@@ -81843,11 +82550,13 @@
81843 int regRowid; /* Actual register containing rowids */
81844
81845 /* Collect rowids of every row to be deleted.
81846 */
81847 sqlite3VdbeAddOp2(v, OP_Null, 0, iRowSet);
81848 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere,0,WHERE_DUPLICATES_OK);
 
 
81849 if( pWInfo==0 ) goto delete_from_cleanup;
81850 regRowid = sqlite3ExprCodeGetColumn(pParse, pTab, -1, iCur, iRowid);
81851 sqlite3VdbeAddOp2(v, OP_RowSetAdd, iRowSet, regRowid);
81852 if( db->flags & SQLITE_CountRows ){
81853 sqlite3VdbeAddOp2(v, OP_AddImm, memCnt, 1);
@@ -82108,12 +82817,18 @@
82108 sqlite3VdbeAddOp3(v, OP_Column, iCur, idx, regBase+j);
82109 sqlite3ColumnDefault(v, pTab, idx, -1);
82110 }
82111 }
82112 if( doMakeRec ){
 
 
 
 
 
 
82113 sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol+1, regOut);
82114 sqlite3VdbeChangeP4(v, -1, sqlite3IndexAffinityStr(v, pIdx), P4_TRANSIENT);
82115 }
82116 sqlite3ReleaseTempRange(pParse, regBase, nCol+1);
82117 return regBase;
82118 }
82119
@@ -82621,14 +83336,14 @@
82621 ** character is exactly one byte in size. Also, all characters are
82622 ** able to participate in upper-case-to-lower-case mappings in EBCDIC
82623 ** whereas only characters less than 0x80 do in ASCII.
82624 */
82625 #if defined(SQLITE_EBCDIC)
82626 # define sqlite3Utf8Read(A,C) (*(A++))
82627 # define GlogUpperToLower(A) A = sqlite3UpperToLower[A]
82628 #else
82629 # define GlogUpperToLower(A) if( A<0x80 ){ A = sqlite3UpperToLower[A]; }
82630 #endif
82631
82632 static const struct compareInfo globInfo = { '*', '?', '[', 0 };
82633 /* The correct SQL-92 behavior is for the LIKE operator to ignore
82634 ** case. Thus 'a' LIKE 'A' would be true. */
@@ -82667,13 +83382,13 @@
82667 */
82668 static int patternCompare(
82669 const u8 *zPattern, /* The glob pattern */
82670 const u8 *zString, /* The string to compare against the glob */
82671 const struct compareInfo *pInfo, /* Information about how to do the compare */
82672 const int esc /* The escape character */
82673 ){
82674 int c, c2;
82675 int invert;
82676 int seen;
82677 u8 matchOne = pInfo->matchOne;
82678 u8 matchAll = pInfo->matchAll;
82679 u8 matchSet = pInfo->matchSet;
@@ -82723,11 +83438,11 @@
82723 }else if( !prevEscape && c==matchOne ){
82724 if( sqlite3Utf8Read(zString, &zString)==0 ){
82725 return 0;
82726 }
82727 }else if( c==matchSet ){
82728 int prior_c = 0;
82729 assert( esc==0 ); /* This only occurs for GLOB, not LIKE */
82730 seen = 0;
82731 invert = 0;
82732 c = sqlite3Utf8Read(zString, &zString);
82733 if( c==0 ) return 0;
@@ -82799,11 +83514,11 @@
82799 sqlite3_context *context,
82800 int argc,
82801 sqlite3_value **argv
82802 ){
82803 const unsigned char *zA, *zB;
82804 int escape = 0;
82805 int nPat;
82806 sqlite3 *db = sqlite3_context_db_handle(context);
82807
82808 zB = sqlite3_value_text(argv[0]);
82809 zA = sqlite3_value_text(argv[1]);
@@ -84110,17 +84825,29 @@
84110 }
84111
84112 /* If the parent table is the same as the child table, and we are about
84113 ** to increment the constraint-counter (i.e. this is an INSERT operation),
84114 ** then check if the row being inserted matches itself. If so, do not
84115 ** increment the constraint-counter. */
 
 
 
 
 
 
84116 if( pTab==pFKey->pFrom && nIncr==1 ){
84117 int iJump = sqlite3VdbeCurrentAddr(v) + nCol + 1;
84118 for(i=0; i<nCol; i++){
84119 int iChild = aiCol[i]+1+regData;
84120 int iParent = pIdx->aiColumn[i]+1+regData;
 
 
 
 
 
84121 sqlite3VdbeAddOp3(v, OP_Ne, iChild, iJump, iParent);
 
84122 }
84123 sqlite3VdbeAddOp2(v, OP_Goto, 0, iOk);
84124 }
84125
84126 sqlite3VdbeAddOp3(v, OP_MakeRecord, regTemp, nCol, regRec);
@@ -84272,11 +84999,11 @@
84272
84273 /* Create VDBE to loop through the entries in pSrc that match the WHERE
84274 ** clause. If the constraint is not deferred, throw an exception for
84275 ** each row found. Otherwise, for deferred constraints, increment the
84276 ** deferred constraint counter by nIncr for each row selected. */
84277 pWInfo = sqlite3WhereBegin(pParse, pSrc, pWhere, 0, 0);
84278 if( nIncr>0 && pFKey->isDeferred==0 ){
84279 sqlite3ParseToplevel(pParse)->mayAbort = 1;
84280 }
84281 sqlite3VdbeAddOp2(v, OP_FkCounter, pFKey->isDeferred, nIncr);
84282 if( pWInfo ){
@@ -87139,10 +87866,13 @@
87139 int (*strnicmp)(const char*,const char*,int);
87140 int (*unlock_notify)(sqlite3*,void(*)(void**,int),void*);
87141 int (*wal_autocheckpoint)(sqlite3*,int);
87142 int (*wal_checkpoint)(sqlite3*,const char*);
87143 void *(*wal_hook)(sqlite3*,int(*)(void*,sqlite3*,const char*,int),void*);
 
 
 
87144 };
87145
87146 /*
87147 ** The following macros redefine the API routines so that they are
87148 ** redirected throught the global sqlite3_api structure.
@@ -87339,10 +88069,13 @@
87339 #define sqlite3_strnicmp sqlite3_api->strnicmp
87340 #define sqlite3_unlock_notify sqlite3_api->unlock_notify
87341 #define sqlite3_wal_autocheckpoint sqlite3_api->wal_autocheckpoint
87342 #define sqlite3_wal_checkpoint sqlite3_api->wal_checkpoint
87343 #define sqlite3_wal_hook sqlite3_api->wal_hook
 
 
 
87344 #endif /* SQLITE_CORE */
87345
87346 #define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api = 0;
87347 #define SQLITE_EXTENSION_INIT2(v) sqlite3_api = v;
87348
@@ -87413,10 +88146,12 @@
87413
87414 #ifdef SQLITE_OMIT_VIRTUALTABLE
87415 # define sqlite3_create_module 0
87416 # define sqlite3_create_module_v2 0
87417 # define sqlite3_declare_vtab 0
 
 
87418 #endif
87419
87420 #ifdef SQLITE_OMIT_SHARED_CACHE
87421 # define sqlite3_enable_shared_cache 0
87422 #endif
@@ -87436,10 +88171,11 @@
87436 #define sqlite3_blob_bytes 0
87437 #define sqlite3_blob_close 0
87438 #define sqlite3_blob_open 0
87439 #define sqlite3_blob_read 0
87440 #define sqlite3_blob_write 0
 
87441 #endif
87442
87443 /*
87444 ** The following structure contains pointers to all SQLite API routines.
87445 ** A pointer to this structure is passed into extensions when they are
@@ -87701,10 +88437,13 @@
87701 #else
87702 0,
87703 0,
87704 0,
87705 #endif
 
 
 
87706 };
87707
87708 /*
87709 ** Attempt to load an SQLite extension library contained in the file
87710 ** zFile. The entry point is zProc. zProc may be 0 in which case a
@@ -94090,10 +94829,11 @@
94090 Expr *pHaving; /* The HAVING clause. May be NULL */
94091 int isDistinct; /* True if the DISTINCT keyword is present */
94092 int distinct; /* Table to use for the distinct set */
94093 int rc = 1; /* Value to return from this function */
94094 int addrSortIndex; /* Address of an OP_OpenEphemeral instruction */
 
94095 AggInfo sAggInfo; /* Information used by aggregate queries */
94096 int iEnd; /* Address of the end of the query */
94097 sqlite3 *db; /* The database connection */
94098
94099 #ifndef SQLITE_OMIT_EXPLAIN
@@ -94216,20 +94956,10 @@
94216 explainSetInteger(pParse->iSelectId, iRestoreSelectId);
94217 return rc;
94218 }
94219 #endif
94220
94221 /* If possible, rewrite the query to use GROUP BY instead of DISTINCT.
94222 ** GROUP BY might use an index, DISTINCT never does.
94223 */
94224 assert( p->pGroupBy==0 || (p->selFlags & SF_Aggregate)!=0 );
94225 if( (p->selFlags & (SF_Distinct|SF_Aggregate))==SF_Distinct ){
94226 p->pGroupBy = sqlite3ExprListDup(db, p->pEList, 0);
94227 pGroupBy = p->pGroupBy;
94228 p->selFlags &= ~SF_Distinct;
94229 }
94230
94231 /* If there is both a GROUP BY and an ORDER BY clause and they are
94232 ** identical, then disable the ORDER BY clause since the GROUP BY
94233 ** will cause elements to come out in the correct order. This is
94234 ** an optimization - the correct answer should result regardless.
94235 ** Use the SQLITE_GroupByOrder flag with SQLITE_TESTCTRL_OPTIMIZER
@@ -94237,10 +94967,34 @@
94237 */
94238 if( sqlite3ExprListCompare(p->pGroupBy, pOrderBy)==0
94239 && (db->flags & SQLITE_GroupByOrder)==0 ){
94240 pOrderBy = 0;
94241 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
94242
94243 /* If there is an ORDER BY clause, then this sorting
94244 ** index might end up being unused if the data can be
94245 ** extracted in pre-sorted order. If that is the case, then the
94246 ** OP_OpenEphemeral instruction will be changed to an OP_Noop once
@@ -94273,26 +95027,25 @@
94273
94274 /* Open a virtual index to use for the distinct set.
94275 */
94276 if( p->selFlags & SF_Distinct ){
94277 KeyInfo *pKeyInfo;
94278 assert( isAgg || pGroupBy );
94279 distinct = pParse->nTab++;
94280 pKeyInfo = keyInfoFromExprList(pParse, p->pEList);
94281 sqlite3VdbeAddOp4(v, OP_OpenEphemeral, distinct, 0, 0,
94282 (char*)pKeyInfo, P4_KEYINFO_HANDOFF);
94283 sqlite3VdbeChangeP5(v, BTREE_UNORDERED);
94284 }else{
94285 distinct = -1;
94286 }
94287
94288 /* Aggregate and non-aggregate queries are handled differently */
94289 if( !isAgg && pGroupBy==0 ){
94290 /* This case is for non-aggregate queries
94291 ** Begin the database scan
94292 */
94293 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pOrderBy, 0);
94294 if( pWInfo==0 ) goto select_end;
94295 if( pWInfo->nRowOut < p->nSelectRow ) p->nSelectRow = pWInfo->nRowOut;
94296
94297 /* If sorting index that was created by a prior OP_OpenEphemeral
94298 ** instruction ended up not being needed, then change the OP_OpenEphemeral
@@ -94301,14 +95054,56 @@
94301 if( addrSortIndex>=0 && pOrderBy==0 ){
94302 sqlite3VdbeChangeToNoop(v, addrSortIndex, 1);
94303 p->addrOpenEphm[2] = -1;
94304 }
94305
94306 /* Use the standard inner loop
94307 */
94308 assert(!isDistinct);
94309 selectInnerLoop(pParse, p, pEList, 0, 0, pOrderBy, -1, pDest,
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
94310 pWInfo->iContinue, pWInfo->iBreak);
94311
94312 /* End the database scan loop.
94313 */
94314 sqlite3WhereEnd(pWInfo);
@@ -94414,11 +95209,11 @@
94414 ** This might involve two separate loops with an OP_Sort in between, or
94415 ** it might be a single loop that uses an index to extract information
94416 ** in the right order to begin with.
94417 */
94418 sqlite3VdbeAddOp2(v, OP_Gosub, regReset, addrReset);
94419 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pGroupBy, 0);
94420 if( pWInfo==0 ) goto select_end;
94421 if( pGroupBy==0 ){
94422 /* The optimizer is able to deliver rows in group by order so
94423 ** we do not have to sort. The OP_OpenEphemeral table will be
94424 ** cancelled later because we still need to use the pKeyInfo
@@ -94676,11 +95471,11 @@
94676 /* This case runs if the aggregate has no GROUP BY clause. The
94677 ** processing is much simpler since there is only a single row
94678 ** of output.
94679 */
94680 resetAccumulator(pParse, &sAggInfo);
94681 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pMinMax, flag);
94682 if( pWInfo==0 ){
94683 sqlite3ExprListDelete(db, pDel);
94684 goto select_end;
94685 }
94686 updateAccumulator(pParse, &sAggInfo);
@@ -95152,19 +95947,32 @@
95152 iDb = sqlite3TwoPartName(pParse, pName1, pName2, &pName);
95153 if( iDb<0 ){
95154 goto trigger_cleanup;
95155 }
95156 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
95157
95158 /* If the trigger name was unqualified, and the table is a temp table,
95159 ** then set iDb to 1 to create the trigger in the temporary database.
95160 ** If sqlite3SrcListLookup() returns 0, indicating the table does not
95161 ** exist, the error is caught by the block below.
95162 */
95163 if( !pTableName || db->mallocFailed ){
95164 goto trigger_cleanup;
95165 }
95166 pTab = sqlite3SrcListLookup(pParse, pTableName);
95167 if( db->init.busy==0 && pName2->n==0 && pTab
95168 && pTab->pSchema==db->aDb[1].pSchema ){
95169 iDb = 1;
95170 }
@@ -95336,13 +96144,12 @@
95336 "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')",
95337 db->aDb[iDb].zName, SCHEMA_TABLE(iDb), zName,
95338 pTrig->table, z);
95339 sqlite3DbFree(db, z);
95340 sqlite3ChangeCookie(pParse, iDb);
95341 sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, sqlite3MPrintf(
95342 db, "type='trigger' AND name='%q'", zName), P4_DYNAMIC
95343 );
95344 }
95345
95346 if( db->init.busy ){
95347 Trigger *pLink = pTrig;
95348 Hash *pHash = &db->aDb[iDb].pSchema->trigHash;
@@ -96392,11 +97199,11 @@
96392 aRegIdx = sqlite3DbMallocRaw(db, sizeof(Index*) * nIdx );
96393 if( aRegIdx==0 ) goto update_cleanup;
96394 }
96395 for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){
96396 int reg;
96397 if( chngRowid ){
96398 reg = ++pParse->nMem;
96399 }else{
96400 reg = 0;
96401 for(i=0; i<pIdx->nColumn; i++){
96402 if( aXRef[pIdx->aiColumn[i]]>=0 ){
@@ -96459,11 +97266,13 @@
96459 }
96460
96461 /* Begin the database scan
96462 */
96463 sqlite3VdbeAddOp2(v, OP_Null, 0, regOldRowid);
96464 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere,0, WHERE_ONEPASS_DESIRED);
 
 
96465 if( pWInfo==0 ) goto update_cleanup;
96466 okOnePass = pWInfo->okOnePass;
96467
96468 /* Remember the rowid of every item to be updated.
96469 */
@@ -97547,11 +98356,11 @@
97547 v = sqlite3GetVdbe(pParse);
97548 sqlite3ChangeCookie(pParse, iDb);
97549
97550 sqlite3VdbeAddOp2(v, OP_Expire, 0, 0);
97551 zWhere = sqlite3MPrintf(db, "name='%q' AND type='table'", pTab->zName);
97552 sqlite3VdbeAddOp4(v, OP_ParseSchema, iDb, 0, 0, zWhere, P4_DYNAMIC);
97553 sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0,
97554 pTab->zName, sqlite3Strlen30(pTab->zName) + 1);
97555 }
97556
97557 /* If we are rereading the sqlite_master table create the in-memory
@@ -98350,11 +99159,11 @@
98350 #define TERM_CODED 0x04 /* This term is already coded */
98351 #define TERM_COPIED 0x08 /* Has a child */
98352 #define TERM_ORINFO 0x10 /* Need to free the WhereTerm.u.pOrInfo object */
98353 #define TERM_ANDINFO 0x20 /* Need to free the WhereTerm.u.pAndInfo obj */
98354 #define TERM_OR_OK 0x40 /* Used during OR-clause processing */
98355 #ifdef SQLITE_ENABLE_STAT2
98356 # define TERM_VNULL 0x80 /* Manufactured x>NULL or x<=NULL term */
98357 #else
98358 # define TERM_VNULL 0x00 /* Disabled if not using stat2 */
98359 #endif
98360
@@ -98485,10 +99294,11 @@
98485 #define WHERE_REVERSE 0x02000000 /* Scan in reverse order */
98486 #define WHERE_UNIQUE 0x04000000 /* Selects no more than one row */
98487 #define WHERE_VIRTUALTABLE 0x08000000 /* Use virtual-table processing */
98488 #define WHERE_MULTI_OR 0x10000000 /* OR using multiple indices */
98489 #define WHERE_TEMP_INDEX 0x20000000 /* Uses an ephemeral index */
 
98490
98491 /*
98492 ** Initialize a preallocated WhereClause structure.
98493 */
98494 static void whereClauseInit(
@@ -99563,11 +100373,11 @@
99563 pNewTerm->prereqAll = pTerm->prereqAll;
99564 }
99565 }
99566 #endif /* SQLITE_OMIT_VIRTUALTABLE */
99567
99568 #ifdef SQLITE_ENABLE_STAT2
99569 /* When sqlite_stat2 histogram data is available an operator of the
99570 ** form "x IS NOT NULL" can sometimes be evaluated more efficiently
99571 ** as "x>NULL" if x is not an INTEGER PRIMARY KEY. So construct a
99572 ** virtual term of that form.
99573 **
@@ -99602,11 +100412,11 @@
99602 pTerm->nChild = 1;
99603 pTerm->wtFlags |= TERM_COPIED;
99604 pNewTerm->prereqAll = pTerm->prereqAll;
99605 }
99606 }
99607 #endif /* SQLITE_ENABLE_STAT2 */
99608
99609 /* Prevent ON clause terms of a LEFT JOIN from being used to drive
99610 ** an index for tables to the left of the join.
99611 */
99612 pTerm->prereqRight |= extraRight;
@@ -99629,10 +100439,166 @@
99629 }
99630 }
99631 return 0;
99632 }
99633
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99634
99635 /*
99636 ** This routine decides if pIdx can be used to satisfy the ORDER BY
99637 ** clause. If it can, it returns 1. If pIdx cannot satisfy the
99638 ** ORDER BY clause, this routine returns 0.
@@ -99665,11 +100631,14 @@
99665 int sortOrder = 0; /* XOR of index and ORDER BY sort direction */
99666 int nTerm; /* Number of ORDER BY terms */
99667 struct ExprList_item *pTerm; /* A term of the ORDER BY clause */
99668 sqlite3 *db = pParse->db;
99669
99670 assert( pOrderBy!=0 );
 
 
 
99671 nTerm = pOrderBy->nExpr;
99672 assert( nTerm>0 );
99673
99674 /* Argument pIdx must either point to a 'real' named index structure,
99675 ** or an index structure allocated on the stack by bestBtreeIndex() to
@@ -99978,10 +100947,14 @@
99978 double costTempIdx; /* per-query cost of the transient index */
99979 WhereTerm *pTerm; /* A single term of the WHERE clause */
99980 WhereTerm *pWCEnd; /* End of pWC->a[] */
99981 Table *pTable; /* Table tht might be indexed */
99982
 
 
 
 
99983 if( (pParse->db->flags & SQLITE_AutoIndex)==0 ){
99984 /* Automatic indices are disabled at run-time */
99985 return;
99986 }
99987 if( (pCost->plan.wsFlags & WHERE_NOT_FULLSCAN)!=0 ){
@@ -99989,10 +100962,14 @@
99989 return;
99990 }
99991 if( pSrc->notIndexed ){
99992 /* The NOT INDEXED clause appears in the SQL. */
99993 return;
 
 
 
 
99994 }
99995
99996 assert( pParse->nQueryLoop >= (double)1 );
99997 pTable = pSrc->pTab;
99998 nTableRow = pTable->nRowEst;
@@ -100206,10 +101183,11 @@
100206 if( pTerm->leftCursor != pSrc->iCursor ) continue;
100207 assert( (pTerm->eOperator&(pTerm->eOperator-1))==0 );
100208 testcase( pTerm->eOperator==WO_IN );
100209 testcase( pTerm->eOperator==WO_ISNULL );
100210 if( pTerm->eOperator & (WO_IN|WO_ISNULL) ) continue;
 
100211 nTerm++;
100212 }
100213
100214 /* If the ORDER BY clause contains only columns in the current
100215 ** virtual table then allocate space for the aOrderBy part of
@@ -100256,10 +101234,11 @@
100256 if( pTerm->leftCursor != pSrc->iCursor ) continue;
100257 assert( (pTerm->eOperator&(pTerm->eOperator-1))==0 );
100258 testcase( pTerm->eOperator==WO_IN );
100259 testcase( pTerm->eOperator==WO_ISNULL );
100260 if( pTerm->eOperator & (WO_IN|WO_ISNULL) ) continue;
 
100261 pIdxCons[j].iColumn = pTerm->u.leftColumn;
100262 pIdxCons[j].iTermOffset = i;
100263 pIdxCons[j].op = (u8)pTerm->eOperator;
100264 /* The direct assignment in the previous line is possible only because
100265 ** the WO_ and SQLITE_INDEX_CONSTRAINT_ codes are identical. The
@@ -100482,71 +101461,89 @@
100482 */
100483 bestOrClauseIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost);
100484 }
100485 #endif /* SQLITE_OMIT_VIRTUALTABLE */
100486
 
100487 /*
100488 ** Argument pIdx is a pointer to an index structure that has an array of
100489 ** SQLITE_INDEX_SAMPLES evenly spaced samples of the first indexed column
100490 ** stored in Index.aSample. These samples divide the domain of values stored
100491 ** the index into (SQLITE_INDEX_SAMPLES+1) regions.
100492 ** Region 0 contains all values less than the first sample value. Region
100493 ** 1 contains values between the first and second samples. Region 2 contains
100494 ** values between samples 2 and 3. And so on. Region SQLITE_INDEX_SAMPLES
100495 ** contains values larger than the last sample.
100496 **
100497 ** If the index contains many duplicates of a single value, then it is
100498 ** possible that two or more adjacent samples can hold the same value.
100499 ** When that is the case, the smallest possible region code is returned
100500 ** when roundUp is false and the largest possible region code is returned
100501 ** when roundUp is true.
100502 **
100503 ** If successful, this function determines which of the regions value
100504 ** pVal lies in, sets *piRegion to the region index (a value between 0
100505 ** and SQLITE_INDEX_SAMPLES+1, inclusive) and returns SQLITE_OK.
100506 ** Or, if an OOM occurs while converting text values between encodings,
100507 ** SQLITE_NOMEM is returned and *piRegion is undefined.
100508 */
100509 #ifdef SQLITE_ENABLE_STAT2
100510 static int whereRangeRegion(
100511 Parse *pParse, /* Database connection */
100512 Index *pIdx, /* Index to consider domain of */
100513 sqlite3_value *pVal, /* Value to consider */
100514 int roundUp, /* Return largest valid region if true */
100515 int *piRegion /* OUT: Region of domain in which value lies */
100516 ){
 
 
 
 
 
 
 
100517 assert( roundUp==0 || roundUp==1 );
100518 if( ALWAYS(pVal) ){
100519 IndexSample *aSample = pIdx->aSample;
100520 int i = 0;
100521 int eType = sqlite3_value_type(pVal);
100522
100523 if( eType==SQLITE_INTEGER || eType==SQLITE_FLOAT ){
100524 double r = sqlite3_value_double(pVal);
100525 for(i=0; i<SQLITE_INDEX_SAMPLES; i++){
100526 if( aSample[i].eType==SQLITE_NULL ) continue;
100527 if( aSample[i].eType>=SQLITE_TEXT ) break;
100528 if( roundUp ){
100529 if( aSample[i].u.r>r ) break;
100530 }else{
100531 if( aSample[i].u.r>=r ) break;
100532 }
100533 }
100534 }else if( eType==SQLITE_NULL ){
100535 i = 0;
100536 if( roundUp ){
100537 while( i<SQLITE_INDEX_SAMPLES && aSample[i].eType==SQLITE_NULL ) i++;
100538 }
100539 }else{
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100540 sqlite3 *db = pParse->db;
100541 CollSeq *pColl;
100542 const u8 *z;
100543 int n;
100544
100545 /* pVal comes from sqlite3ValueFromExpr() so the type cannot be NULL */
100546 assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB );
100547
100548 if( eType==SQLITE_BLOB ){
100549 z = (const u8 *)sqlite3_value_blob(pVal);
100550 pColl = db->pDfltColl;
100551 assert( pColl->enc==SQLITE_UTF8 );
100552 }else{
@@ -100561,16 +101558,16 @@
100561 return SQLITE_NOMEM;
100562 }
100563 assert( z && pColl && pColl->xCmp );
100564 }
100565 n = sqlite3ValueBytes(pVal, pColl->enc);
100566
100567 for(i=0; i<SQLITE_INDEX_SAMPLES; i++){
100568 int c;
100569 int eSampletype = aSample[i].eType;
100570 if( eSampletype==SQLITE_NULL || eSampletype<eType ) continue;
100571 if( (eSampletype!=eType) ) break;
100572 #ifndef SQLITE_OMIT_UTF16
100573 if( pColl->enc!=SQLITE_UTF8 ){
100574 int nSample;
100575 char *zSample = sqlite3Utf8to16(
100576 db, pColl->enc, aSample[i].u.z, aSample[i].nByte, &nSample
@@ -100584,20 +101581,52 @@
100584 }else
100585 #endif
100586 {
100587 c = pColl->xCmp(pColl->pUser, aSample[i].nByte, aSample[i].u.z, n, z);
100588 }
100589 if( c-roundUp>=0 ) break;
 
 
 
100590 }
100591 }
 
100592
100593 assert( i>=0 && i<=SQLITE_INDEX_SAMPLES );
100594 *piRegion = i;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
100595 }
100596 return SQLITE_OK;
100597 }
100598 #endif /* #ifdef SQLITE_ENABLE_STAT2 */
100599
100600 /*
100601 ** If expression pExpr represents a literal value, set *pp to point to
100602 ** an sqlite3_value structure containing the same value, with affinity
100603 ** aff applied to it, before returning. It is the responsibility of the
@@ -100611,11 +101640,11 @@
100611 **
100612 ** If neither of the above apply, set *pp to NULL.
100613 **
100614 ** If an error occurs, return an error code. Otherwise, SQLITE_OK.
100615 */
100616 #ifdef SQLITE_ENABLE_STAT2
100617 static int valueFromExpr(
100618 Parse *pParse,
100619 Expr *pExpr,
100620 u8 aff,
100621 sqlite3_value **pp
@@ -100659,106 +101688,92 @@
100659 **
100660 ** ... FROM t1 WHERE a > ? AND a < ? ...
100661 **
100662 ** then nEq should be passed 0.
100663 **
100664 ** The returned value is an integer between 1 and 100, inclusive. A return
100665 ** value of 1 indicates that the proposed range scan is expected to visit
100666 ** approximately 1/100th (1%) of the rows selected by the nEq equality
100667 ** constraints (if any). A return value of 100 indicates that it is expected
100668 ** that the range scan will visit every row (100%) selected by the equality
100669 ** constraints.
100670 **
100671 ** In the absence of sqlite_stat2 ANALYZE data, each range inequality
100672 ** reduces the search space by 3/4ths. Hence a single constraint (x>?)
100673 ** results in a return of 25 and a range constraint (x>? AND x<?) results
100674 ** in a return of 6.
100675 */
100676 static int whereRangeScanEst(
100677 Parse *pParse, /* Parsing & code generating context */
100678 Index *p, /* The index containing the range-compared column; "x" */
100679 int nEq, /* index into p->aCol[] of the range-compared column */
100680 WhereTerm *pLower, /* Lower bound on the range. ex: "x>123" Might be NULL */
100681 WhereTerm *pUpper, /* Upper bound on the range. ex: "x<455" Might be NULL */
100682 int *piEst /* OUT: Return value */
100683 ){
100684 int rc = SQLITE_OK;
100685
100686 #ifdef SQLITE_ENABLE_STAT2
100687
100688 if( nEq==0 && p->aSample ){
100689 sqlite3_value *pLowerVal = 0;
100690 sqlite3_value *pUpperVal = 0;
100691 int iEst;
100692 int iLower = 0;
100693 int iUpper = SQLITE_INDEX_SAMPLES;
100694 int roundUpUpper = 0;
100695 int roundUpLower = 0;
100696 u8 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
100697
100698 if( pLower ){
100699 Expr *pExpr = pLower->pExpr->pRight;
100700 rc = valueFromExpr(pParse, pExpr, aff, &pLowerVal);
100701 assert( pLower->eOperator==WO_GT || pLower->eOperator==WO_GE );
100702 roundUpLower = (pLower->eOperator==WO_GT) ?1:0;
 
 
 
 
 
 
100703 }
100704 if( rc==SQLITE_OK && pUpper ){
100705 Expr *pExpr = pUpper->pExpr->pRight;
100706 rc = valueFromExpr(pParse, pExpr, aff, &pUpperVal);
100707 assert( pUpper->eOperator==WO_LT || pUpper->eOperator==WO_LE );
100708 roundUpUpper = (pUpper->eOperator==WO_LE) ?1:0;
100709 }
100710
100711 if( rc!=SQLITE_OK || (pLowerVal==0 && pUpperVal==0) ){
100712 sqlite3ValueFree(pLowerVal);
100713 sqlite3ValueFree(pUpperVal);
100714 goto range_est_fallback;
100715 }else if( pLowerVal==0 ){
100716 rc = whereRangeRegion(pParse, p, pUpperVal, roundUpUpper, &iUpper);
100717 if( pLower ) iLower = iUpper/2;
100718 }else if( pUpperVal==0 ){
100719 rc = whereRangeRegion(pParse, p, pLowerVal, roundUpLower, &iLower);
100720 if( pUpper ) iUpper = (iLower + SQLITE_INDEX_SAMPLES + 1)/2;
100721 }else{
100722 rc = whereRangeRegion(pParse, p, pUpperVal, roundUpUpper, &iUpper);
100723 if( rc==SQLITE_OK ){
100724 rc = whereRangeRegion(pParse, p, pLowerVal, roundUpLower, &iLower);
100725 }
100726 }
100727 WHERETRACE(("range scan regions: %d..%d\n", iLower, iUpper));
100728
100729 iEst = iUpper - iLower;
100730 testcase( iEst==SQLITE_INDEX_SAMPLES );
100731 assert( iEst<=SQLITE_INDEX_SAMPLES );
100732 if( iEst<1 ){
100733 *piEst = 50/SQLITE_INDEX_SAMPLES;
100734 }else{
100735 *piEst = (iEst*100)/SQLITE_INDEX_SAMPLES;
100736 }
100737 sqlite3ValueFree(pLowerVal);
100738 sqlite3ValueFree(pUpperVal);
100739 return rc;
100740 }
100741 range_est_fallback:
100742 #else
100743 UNUSED_PARAMETER(pParse);
100744 UNUSED_PARAMETER(p);
100745 UNUSED_PARAMETER(nEq);
100746 #endif
100747 assert( pLower || pUpper );
100748 *piEst = 100;
100749 if( pLower && (pLower->wtFlags & TERM_VNULL)==0 ) *piEst /= 4;
100750 if( pUpper ) *piEst /= 4;
100751 return rc;
100752 }
100753
100754 #ifdef SQLITE_ENABLE_STAT2
100755 /*
100756 ** Estimate the number of rows that will be returned based on
100757 ** an equality constraint x=VALUE and where that VALUE occurs in
100758 ** the histogram data. This only works when x is the left-most
100759 ** column of an index and sqlite_stat2 histogram data is available
100760 ** for that index. When pExpr==NULL that means the constraint is
100761 ** "x IS NULL" instead of "x=VALUE".
100762 **
100763 ** Write the estimated row count into *pnRow and return SQLITE_OK.
100764 ** If unable to make an estimate, leave *pnRow unchanged and return
@@ -100774,14 +101789,13 @@
100774 Index *p, /* The index whose left-most column is pTerm */
100775 Expr *pExpr, /* Expression for VALUE in the x=VALUE constraint */
100776 double *pnRow /* Write the revised row estimate here */
100777 ){
100778 sqlite3_value *pRhs = 0; /* VALUE on right-hand side of pTerm */
100779 int iLower, iUpper; /* Range of histogram regions containing pRhs */
100780 u8 aff; /* Column affinity */
100781 int rc; /* Subfunction return code */
100782 double nRowEst; /* New estimate of the number of rows */
100783
100784 assert( p->aSample!=0 );
100785 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
100786 if( pExpr ){
100787 rc = valueFromExpr(pParse, pExpr, aff, &pRhs);
@@ -100788,30 +101802,22 @@
100788 if( rc ) goto whereEqualScanEst_cancel;
100789 }else{
100790 pRhs = sqlite3ValueNew(pParse->db);
100791 }
100792 if( pRhs==0 ) return SQLITE_NOTFOUND;
100793 rc = whereRangeRegion(pParse, p, pRhs, 0, &iLower);
100794 if( rc ) goto whereEqualScanEst_cancel;
100795 rc = whereRangeRegion(pParse, p, pRhs, 1, &iUpper);
100796 if( rc ) goto whereEqualScanEst_cancel;
100797 WHERETRACE(("equality scan regions: %d..%d\n", iLower, iUpper));
100798 if( iLower>=iUpper ){
100799 nRowEst = p->aiRowEst[0]/(SQLITE_INDEX_SAMPLES*2);
100800 if( nRowEst<*pnRow ) *pnRow = nRowEst;
100801 }else{
100802 nRowEst = (iUpper-iLower)*p->aiRowEst[0]/SQLITE_INDEX_SAMPLES;
100803 *pnRow = nRowEst;
100804 }
100805
100806 whereEqualScanEst_cancel:
100807 sqlite3ValueFree(pRhs);
100808 return rc;
100809 }
100810 #endif /* defined(SQLITE_ENABLE_STAT2) */
100811
100812 #ifdef SQLITE_ENABLE_STAT2
100813 /*
100814 ** Estimate the number of rows that will be returned based on
100815 ** an IN constraint where the right-hand side of the IN operator
100816 ** is a list of values. Example:
100817 **
@@ -100830,64 +101836,29 @@
100830 Parse *pParse, /* Parsing & code generating context */
100831 Index *p, /* The index whose left-most column is pTerm */
100832 ExprList *pList, /* The value list on the RHS of "x IN (v1,v2,v3,...)" */
100833 double *pnRow /* Write the revised row estimate here */
100834 ){
100835 sqlite3_value *pVal = 0; /* One value from list */
100836 int iLower, iUpper; /* Range of histogram regions containing pRhs */
100837 u8 aff; /* Column affinity */
100838 int rc = SQLITE_OK; /* Subfunction return code */
100839 double nRowEst; /* New estimate of the number of rows */
100840 int nSpan = 0; /* Number of histogram regions spanned */
100841 int nSingle = 0; /* Histogram regions hit by a single value */
100842 int nNotFound = 0; /* Count of values that are not constants */
100843 int i; /* Loop counter */
100844 u8 aSpan[SQLITE_INDEX_SAMPLES+1]; /* Histogram regions that are spanned */
100845 u8 aSingle[SQLITE_INDEX_SAMPLES+1]; /* Histogram regions hit once */
100846
100847 assert( p->aSample!=0 );
100848 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
100849 memset(aSpan, 0, sizeof(aSpan));
100850 memset(aSingle, 0, sizeof(aSingle));
100851 for(i=0; i<pList->nExpr; i++){
100852 sqlite3ValueFree(pVal);
100853 rc = valueFromExpr(pParse, pList->a[i].pExpr, aff, &pVal);
100854 if( rc ) break;
100855 if( pVal==0 || sqlite3_value_type(pVal)==SQLITE_NULL ){
100856 nNotFound++;
100857 continue;
100858 }
100859 rc = whereRangeRegion(pParse, p, pVal, 0, &iLower);
100860 if( rc ) break;
100861 rc = whereRangeRegion(pParse, p, pVal, 1, &iUpper);
100862 if( rc ) break;
100863 if( iLower>=iUpper ){
100864 aSingle[iLower] = 1;
100865 }else{
100866 assert( iLower>=0 && iUpper<=SQLITE_INDEX_SAMPLES );
100867 while( iLower<iUpper ) aSpan[iLower++] = 1;
100868 }
100869 }
100870 if( rc==SQLITE_OK ){
100871 for(i=nSpan=0; i<=SQLITE_INDEX_SAMPLES; i++){
100872 if( aSpan[i] ){
100873 nSpan++;
100874 }else if( aSingle[i] ){
100875 nSingle++;
100876 }
100877 }
100878 nRowEst = (nSpan*2+nSingle)*p->aiRowEst[0]/(2*SQLITE_INDEX_SAMPLES)
100879 + nNotFound*p->aiRowEst[1];
100880 if( nRowEst > p->aiRowEst[0] ) nRowEst = p->aiRowEst[0];
100881 *pnRow = nRowEst;
100882 WHERETRACE(("IN row estimate: nSpan=%d, nSingle=%d, nNotFound=%d, est=%g\n",
100883 nSpan, nSingle, nNotFound, nRowEst));
100884 }
100885 sqlite3ValueFree(pVal);
100886 return rc;
100887 }
100888 #endif /* defined(SQLITE_ENABLE_STAT2) */
100889
100890
100891 /*
100892 ** Find the best query plan for accessing a particular table. Write the
100893 ** best query plan and its cost into the WhereCost object supplied as the
@@ -100921,19 +101892,20 @@
100921 WhereClause *pWC, /* The WHERE clause */
100922 struct SrcList_item *pSrc, /* The FROM clause term to search */
100923 Bitmask notReady, /* Mask of cursors not available for indexing */
100924 Bitmask notValid, /* Cursors not available for any purpose */
100925 ExprList *pOrderBy, /* The ORDER BY clause */
 
100926 WhereCost *pCost /* Lowest cost query plan */
100927 ){
100928 int iCur = pSrc->iCursor; /* The cursor of the table to be accessed */
100929 Index *pProbe; /* An index we are evaluating */
100930 Index *pIdx; /* Copy of pProbe, or zero for IPK index */
100931 int eqTermMask; /* Current mask of valid equality operators */
100932 int idxEqTermMask; /* Index mask of valid equality operators */
100933 Index sPk; /* A fake index object for the primary key */
100934 unsigned int aiRowEstPk[2]; /* The aiRowEst[] value for the sPk index */
100935 int aiColumnPk = -1; /* The aColumn[] value for the sPk index */
100936 int wsFlagMask; /* Allowed flags in pCost->plan.wsFlag */
100937
100938 /* Initialize the cost to a worst-case value */
100939 memset(pCost, 0, sizeof(*pCost));
@@ -100984,11 +101956,11 @@
100984 }
100985
100986 /* Loop over all indices looking for the best one to use
100987 */
100988 for(; pProbe; pIdx=pProbe=pProbe->pNext){
100989 const unsigned int * const aiRowEst = pProbe->aiRowEst;
100990 double cost; /* Cost of using pProbe */
100991 double nRow; /* Estimated number of rows in result set */
100992 double log10N; /* base-10 logarithm of nRow (inexact) */
100993 int rev; /* True to scan in reverse order */
100994 int wsFlags = 0;
@@ -101027,18 +101999,16 @@
101027 ** Set to true if there was at least one "x IN (SELECT ...)" term used
101028 ** in determining the value of nInMul. Note that the RHS of the
101029 ** IN operator must be a SELECT, not a value list, for this variable
101030 ** to be true.
101031 **
101032 ** estBound:
101033 ** An estimate on the amount of the table that must be searched. A
101034 ** value of 100 means the entire table is searched. Range constraints
101035 ** might reduce this to a value less than 100 to indicate that only
101036 ** a fraction of the table needs searching. In the absence of
101037 ** sqlite_stat2 ANALYZE data, a single inequality reduces the search
101038 ** space to 1/4rd its original size. So an x>? constraint reduces
101039 ** estBound to 25. Two constraints (x>? AND x<?) reduce estBound to 6.
101040 **
101041 ** bSort:
101042 ** Boolean. True if there is an ORDER BY clause that will require an
101043 ** external sort (i.e. scanning the index being evaluated will not
101044 ** correctly order records).
@@ -101059,16 +102029,17 @@
101059 ** SELECT a, b, c FROM tbl WHERE a = 1;
101060 */
101061 int nEq; /* Number of == or IN terms matching index */
101062 int bInEst = 0; /* True if "x IN (SELECT...)" seen */
101063 int nInMul = 1; /* Number of distinct equalities to lookup */
101064 int estBound = 100; /* Estimated reduction in search space */
101065 int nBound = 0; /* Number of range constraints seen */
101066 int bSort = 0; /* True if external sort required */
 
101067 int bLookup = 0; /* True if not a covering index */
101068 WhereTerm *pTerm; /* A single term of the WHERE clause */
101069 #ifdef SQLITE_ENABLE_STAT2
101070 WhereTerm *pFirstTerm = 0; /* First term matching the index */
101071 #endif
101072
101073 /* Determine the values of nEq and nInMul */
101074 for(nEq=0; nEq<pProbe->nColumn; nEq++){
@@ -101088,23 +102059,23 @@
101088 nInMul *= pExpr->x.pList->nExpr;
101089 }
101090 }else if( pTerm->eOperator & WO_ISNULL ){
101091 wsFlags |= WHERE_COLUMN_NULL;
101092 }
101093 #ifdef SQLITE_ENABLE_STAT2
101094 if( nEq==0 && pProbe->aSample ) pFirstTerm = pTerm;
101095 #endif
101096 used |= pTerm->prereqRight;
101097 }
101098
101099 /* Determine the value of estBound. */
101100 if( nEq<pProbe->nColumn && pProbe->bUnordered==0 ){
101101 int j = pProbe->aiColumn[nEq];
101102 if( findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE|WO_GT|WO_GE, pIdx) ){
101103 WhereTerm *pTop = findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE, pIdx);
101104 WhereTerm *pBtm = findTerm(pWC, iCur, j, notReady, WO_GT|WO_GE, pIdx);
101105 whereRangeScanEst(pParse, pProbe, nEq, pBtm, pTop, &estBound);
101106 if( pTop ){
101107 nBound = 1;
101108 wsFlags |= WHERE_TOP_LIMIT;
101109 used |= pTop->prereqRight;
101110 }
@@ -101125,21 +102096,24 @@
101125
101126 /* If there is an ORDER BY clause and the index being considered will
101127 ** naturally scan rows in the required order, set the appropriate flags
101128 ** in wsFlags. Otherwise, if there is an ORDER BY clause but the index
101129 ** will scan rows in a different order, set the bSort variable. */
101130 if( pOrderBy ){
101131 if( (wsFlags & WHERE_COLUMN_IN)==0
101132 && pProbe->bUnordered==0
101133 && isSortingIndex(pParse, pWC->pMaskSet, pProbe, iCur, pOrderBy,
101134 nEq, wsFlags, &rev)
101135 ){
101136 wsFlags |= WHERE_ROWID_RANGE|WHERE_COLUMN_RANGE|WHERE_ORDERBY;
101137 wsFlags |= (rev ? WHERE_REVERSE : 0);
101138 }else{
101139 bSort = 1;
101140 }
 
 
 
101141 }
101142
101143 /* If currently calculating the cost of using an index (not the IPK
101144 ** index), determine if all required column data may be obtained without
101145 ** using the main table (i.e. if the index is a covering
@@ -101169,31 +102143,32 @@
101169 if( bInEst && nRow*2>aiRowEst[0] ){
101170 nRow = aiRowEst[0]/2;
101171 nInMul = (int)(nRow / aiRowEst[nEq]);
101172 }
101173
101174 #ifdef SQLITE_ENABLE_STAT2
101175 /* If the constraint is of the form x=VALUE and histogram
 
101176 ** data is available for column x, then it might be possible
101177 ** to get a better estimate on the number of rows based on
101178 ** VALUE and how common that value is according to the histogram.
101179 */
101180 if( nRow>(double)1 && nEq==1 && pFirstTerm!=0 ){
101181 if( pFirstTerm->eOperator & (WO_EQ|WO_ISNULL) ){
101182 testcase( pFirstTerm->eOperator==WO_EQ );
101183 testcase( pFirstTerm->eOperator==WO_ISNULL );
101184 whereEqualScanEst(pParse, pProbe, pFirstTerm->pExpr->pRight, &nRow);
101185 }else if( pFirstTerm->eOperator==WO_IN && bInEst==0 ){
101186 whereInScanEst(pParse, pProbe, pFirstTerm->pExpr->x.pList, &nRow);
101187 }
101188 }
101189 #endif /* SQLITE_ENABLE_STAT2 */
101190
101191 /* Adjust the number of output rows and downward to reflect rows
101192 ** that are excluded by range constraints.
101193 */
101194 nRow = (nRow * (double)estBound) / (double)100;
101195 if( nRow<1 ) nRow = 1;
101196
101197 /* Experiments run on real SQLite databases show that the time needed
101198 ** to do a binary search to locate a row in a table or index is roughly
101199 ** log10(N) times the time to move from one row to the next row within
@@ -101252,10 +102227,13 @@
101252 ** difference and select C of 3.0.
101253 */
101254 if( bSort ){
101255 cost += nRow*estLog(nRow)*3;
101256 }
 
 
 
101257
101258 /**** Cost of using this index has now been computed ****/
101259
101260 /* If there are additional constraints on this table that cannot
101261 ** be used with the current index, but which might lower the number
@@ -101315,14 +102293,14 @@
101315 if( nRow<2 ) nRow = 2;
101316 }
101317
101318
101319 WHERETRACE((
101320 "%s(%s): nEq=%d nInMul=%d estBound=%d bSort=%d bLookup=%d wsFlags=0x%x\n"
101321 " notReady=0x%llx log10N=%.1f nRow=%.1f cost=%.1f used=0x%llx\n",
101322 pSrc->pTab->zName, (pIdx ? pIdx->zName : "ipk"),
101323 nEq, nInMul, estBound, bSort, bLookup, wsFlags,
101324 notReady, log10N, nRow, cost, used
101325 ));
101326
101327 /* If this index is the best we have seen so far, then record this
101328 ** index and its cost in the pCost structure.
@@ -101397,11 +102375,11 @@
101397 }
101398 sqlite3DbFree(pParse->db, p);
101399 }else
101400 #endif
101401 {
101402 bestBtreeIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost);
101403 }
101404 }
101405
101406 /*
101407 ** Disable a term in the WHERE clause. Except, do not disable the term
@@ -102359,11 +103337,11 @@
102359 for(ii=0; ii<pOrWc->nTerm; ii++){
102360 WhereTerm *pOrTerm = &pOrWc->a[ii];
102361 if( pOrTerm->leftCursor==iCur || pOrTerm->eOperator==WO_AND ){
102362 WhereInfo *pSubWInfo; /* Info for single OR-term scan */
102363 /* Loop through table entries that match term pOrTerm. */
102364 pSubWInfo = sqlite3WhereBegin(pParse, pOrTab, pOrTerm->pExpr, 0,
102365 WHERE_OMIT_OPEN | WHERE_OMIT_CLOSE |
102366 WHERE_FORCE_TABLE | WHERE_ONETABLE_ONLY);
102367 if( pSubWInfo ){
102368 explainOneScan(
102369 pParse, pOrTab, &pSubWInfo->a[0], iLevel, pLevel->iFrom, 0
@@ -102600,10 +103578,11 @@
102600 SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
102601 Parse *pParse, /* The parser context */
102602 SrcList *pTabList, /* A list of all tables to be scanned */
102603 Expr *pWhere, /* The WHERE clause */
102604 ExprList **ppOrderBy, /* An ORDER BY clause, or NULL */
 
102605 u16 wctrlFlags /* One of the WHERE_* flags defined in sqliteInt.h */
102606 ){
102607 int i; /* Loop counter */
102608 int nByteWInfo; /* Num. bytes allocated for WhereInfo struct */
102609 int nTabList; /* Number of elements in pTabList */
@@ -102659,10 +103638,14 @@
102659 pWInfo->iBreak = sqlite3VdbeMakeLabel(v);
102660 pWInfo->pWC = pWC = (WhereClause *)&((u8 *)pWInfo)[nByteWInfo];
102661 pWInfo->wctrlFlags = wctrlFlags;
102662 pWInfo->savedNQueryLoop = pParse->nQueryLoop;
102663 pMaskSet = (WhereMaskSet*)&pWC[1];
 
 
 
 
102664
102665 /* Split the WHERE clause into separate subexpressions where each
102666 ** subexpression is separated by an AND operator.
102667 */
102668 initMaskSet(pMaskSet);
@@ -102726,10 +103709,19 @@
102726 */
102727 exprAnalyzeAll(pTabList, pWC);
102728 if( db->mallocFailed ){
102729 goto whereBeginError;
102730 }
 
 
 
 
 
 
 
 
 
102731
102732 /* Chose the best index to use for each table in the FROM clause.
102733 **
102734 ** This loop fills in the following fields:
102735 **
@@ -102810,10 +103802,11 @@
102810 Bitmask mask; /* Mask of tables not yet ready */
102811 for(j=iFrom, pTabItem=&pTabList->a[j]; j<nTabList; j++, pTabItem++){
102812 int doNotReorder; /* True if this table should not be reordered */
102813 WhereCost sCost; /* Cost information from best[Virtual]Index() */
102814 ExprList *pOrderBy; /* ORDER BY clause for index to optimize */
 
102815
102816 doNotReorder = (pTabItem->jointype & (JT_LEFT|JT_CROSS))!=0;
102817 if( j!=iFrom && doNotReorder ) break;
102818 m = getMask(pMaskSet, pTabItem->iCursor);
102819 if( (m & notReady)==0 ){
@@ -102820,10 +103813,11 @@
102820 if( j==iFrom ) iFrom++;
102821 continue;
102822 }
102823 mask = (isOptimal ? m : notReady);
102824 pOrderBy = ((i==0 && ppOrderBy )?*ppOrderBy:0);
 
102825 if( pTabItem->pIndex==0 ) nUnconstrained++;
102826
102827 WHERETRACE(("=== trying table %d with isOptimal=%d ===\n",
102828 j, isOptimal));
102829 assert( pTabItem->pTab );
@@ -102834,11 +103828,11 @@
102834 &sCost, pp);
102835 }else
102836 #endif
102837 {
102838 bestBtreeIndex(pParse, pWC, pTabItem, mask, notReady, pOrderBy,
102839 &sCost);
102840 }
102841 assert( isOptimal || (sCost.used&notReady)==0 );
102842
102843 /* If an INDEXED BY clause is present, then the plan must use that
102844 ** index if it uses any index at all */
@@ -102894,10 +103888,14 @@
102894 WHERETRACE(("*** Optimizer selects table %d for loop %d"
102895 " with cost=%g and nRow=%g\n",
102896 bestJ, pLevel-pWInfo->a, bestPlan.rCost, bestPlan.plan.nRow));
102897 if( (bestPlan.plan.wsFlags & WHERE_ORDERBY)!=0 ){
102898 *ppOrderBy = 0;
 
 
 
 
102899 }
102900 andFlags &= bestPlan.plan.wsFlags;
102901 pLevel->plan = bestPlan.plan;
102902 testcase( bestPlan.plan.wsFlags & WHERE_INDEXED );
102903 testcase( bestPlan.plan.wsFlags & WHERE_TEMP_INDEX );
@@ -107274,17 +108272,16 @@
107274 #ifndef SQLITE_OMIT_BLOB_LITERAL
107275 case 'x': case 'X': {
107276 testcase( z[0]=='x' ); testcase( z[0]=='X' );
107277 if( z[1]=='\'' ){
107278 *tokenType = TK_BLOB;
107279 for(i=2; (c=z[i])!=0 && c!='\''; i++){
107280 if( !sqlite3Isxdigit(c) ){
107281 *tokenType = TK_ILLEGAL;
107282 }
107283 }
107284 if( i%2 || !c ) *tokenType = TK_ILLEGAL;
107285 if( c ) i++;
107286 return i;
107287 }
107288 /* Otherwise fall through to the next case */
107289 }
107290 #endif
@@ -110726,10 +111723,21 @@
110726 if( sz ) *ppNew = sqlite3ScratchMalloc(sz);
110727 sqlite3ScratchFree(pFree);
110728 break;
110729 }
110730
 
 
 
 
 
 
 
 
 
 
 
110731 }
110732 va_end(ap);
110733 #endif /* SQLITE_OMIT_BUILTIN_TEST */
110734 return rc;
110735 }
@@ -111383,16 +112391,10 @@
111383 ** TODO(shess) Provide a VACUUM type operation to clear out all
111384 ** deletions and duplications. This would basically be a forced merge
111385 ** into a single segment.
111386 */
111387
111388 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
111389
111390 #if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE)
111391 # define SQLITE_CORE 1
111392 #endif
111393
111394 /************** Include fts3Int.h in the middle of fts3.c ********************/
111395 /************** Begin file fts3Int.h *****************************************/
111396 /*
111397 ** 2009 Nov 12
111398 **
@@ -111404,18 +112406,33 @@
111404 ** May you share freely, never taking more than you give.
111405 **
111406 ******************************************************************************
111407 **
111408 */
111409
111410 #ifndef _FTSINT_H
111411 #define _FTSINT_H
111412
111413 #if !defined(NDEBUG) && !defined(SQLITE_DEBUG)
111414 # define NDEBUG 1
111415 #endif
111416
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111417 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/
111418 /************** Begin file fts3_tokenizer.h **********************************/
111419 /*
111420 ** 2006 July 10
111421 **
@@ -111710,16 +112727,39 @@
111710 ** similar macro called ArraySize(). Use a different name to avoid
111711 ** a collision when building an amalgamation with built-in FTS3.
111712 */
111713 #define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0])))
111714
 
 
 
 
 
111715 /*
111716 ** Maximum length of a varint encoded integer. The varint format is different
111717 ** from that used by SQLite, so the maximum length is 10, not 9.
111718 */
111719 #define FTS3_VARINT_MAX 10
111720
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
111721 /*
111722 ** The testcase() macro is only used by the amalgamation. If undefined,
111723 ** make it a no-op.
111724 */
111725 #ifndef testcase
@@ -111787,14 +112827,15 @@
111787 typedef struct Fts3Cursor Fts3Cursor;
111788 typedef struct Fts3Expr Fts3Expr;
111789 typedef struct Fts3Phrase Fts3Phrase;
111790 typedef struct Fts3PhraseToken Fts3PhraseToken;
111791
 
111792 typedef struct Fts3SegFilter Fts3SegFilter;
111793 typedef struct Fts3DeferredToken Fts3DeferredToken;
111794 typedef struct Fts3SegReader Fts3SegReader;
111795 typedef struct Fts3SegReaderCursor Fts3SegReaderCursor;
111796
111797 /*
111798 ** A connection to a fulltext index is an instance of the following
111799 ** structure. The xCreate and xConnect methods create an instance
111800 ** of this structure and xDestroy and xDisconnect free that instance.
@@ -111811,33 +112852,45 @@
111811 sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */
111812
111813 /* Precompiled statements used by the implementation. Each of these
111814 ** statements is run and reset within a single virtual table API call.
111815 */
111816 sqlite3_stmt *aStmt[24];
111817
111818 char *zReadExprlist;
111819 char *zWriteExprlist;
111820
111821 int nNodeSize; /* Soft limit for node size */
111822 u8 bHasStat; /* True if %_stat table exists */
111823 u8 bHasDocsize; /* True if %_docsize table exists */
 
111824 int nPgsz; /* Page size for host database */
111825 char *zSegmentsTbl; /* Name of %_segments table */
111826 sqlite3_blob *pSegments; /* Blob handle open on %_segments table */
111827
111828 /* The following hash table is used to buffer pending index updates during
 
 
111829 ** transactions. Variable nPendingData estimates the memory size of the
111830 ** pending data, including hash table overhead, but not malloc overhead.
111831 ** When nPendingData exceeds nMaxPendingData, the buffer is flushed
111832 ** automatically. Variable iPrevDocid is the docid of the most recently
111833 ** inserted record.
 
 
 
 
 
111834 */
111835 int nMaxPendingData;
111836 int nPendingData;
111837 sqlite_int64 iPrevDocid;
111838 Fts3Hash pendingTerms;
 
 
 
 
111839
111840 #if defined(SQLITE_DEBUG)
111841 /* State variables used for validating that the transaction control
111842 ** methods of the virtual table are called at appropriate times. These
111843 ** values do not contribution to the FTS computation; they are used for
@@ -111864,13 +112917,14 @@
111864 Fts3DeferredToken *pDeferred; /* Deferred search tokens, if any */
111865 sqlite3_int64 iPrevId; /* Previous id read from aDoclist */
111866 char *pNextId; /* Pointer into the body of aDoclist */
111867 char *aDoclist; /* List of docids for full-text queries */
111868 int nDoclist; /* Size of buffer at aDoclist */
111869 int desc; /* True to sort in descending order */
111870 int eEvalmode; /* An FTS3_EVAL_XX constant */
111871 int nRowAvg; /* Average size of database rows, in pages */
 
111872
111873 int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */
111874 u32 *aMatchinfo; /* Information about most recent match */
111875 int nMatchinfo; /* Number of elements in aMatchinfo[] */
111876 char *zMatchinfo; /* Matchinfo specification */
@@ -111897,66 +112951,90 @@
111897 */
111898 #define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */
111899 #define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */
111900 #define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */
111901
 
 
 
 
 
 
 
 
 
 
 
 
111902 /*
111903 ** A "phrase" is a sequence of one or more tokens that must match in
111904 ** sequence. A single token is the base case and the most common case.
111905 ** For a sequence of tokens contained in double-quotes (i.e. "one two three")
111906 ** nToken will be the number of tokens in the string.
111907 **
111908 ** The nDocMatch and nMatch variables contain data that may be used by the
111909 ** matchinfo() function. They are populated when the full-text index is
111910 ** queried for hits on the phrase. If one or more tokens in the phrase
111911 ** are deferred, the nDocMatch and nMatch variables are populated based
111912 ** on the assumption that the
111913 */
111914 struct Fts3PhraseToken {
111915 char *z; /* Text of the token */
111916 int n; /* Number of bytes in buffer z */
111917 int isPrefix; /* True if token ends with a "*" character */
111918 int bFulltext; /* True if full-text index was used */
111919 Fts3SegReaderCursor *pSegcsr; /* Segment-reader for this token */
 
 
111920 Fts3DeferredToken *pDeferred; /* Deferred token object for this token */
 
111921 };
111922
111923 struct Fts3Phrase {
111924 /* Variables populated by fts3_expr.c when parsing a MATCH expression */
 
 
 
 
 
 
 
111925 int nToken; /* Number of tokens in the phrase */
111926 int iColumn; /* Index of column this phrase must match */
111927 int isNot; /* Phrase prefixed by unary not (-) operator */
111928 Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */
111929 };
111930
111931 /*
111932 ** A tree of these objects forms the RHS of a MATCH operator.
111933 **
111934 ** If Fts3Expr.eType is either FTSQUERY_NEAR or FTSQUERY_PHRASE and isLoaded
111935 ** is true, then aDoclist points to a malloced buffer, size nDoclist bytes,
111936 ** containing the results of the NEAR or phrase query in FTS3 doclist
111937 ** format. As usual, the initial "Length" field found in doclists stored
111938 ** on disk is omitted from this buffer.
111939 **
111940 ** Variable pCurrent always points to the start of a docid field within
111941 ** aDoclist. Since the doclist is usually scanned in docid order, this can
111942 ** be used to accelerate seeking to the required docid within the doclist.
 
 
 
 
 
 
 
 
111943 */
111944 struct Fts3Expr {
111945 int eType; /* One of the FTSQUERY_XXX values defined below */
111946 int nNear; /* Valid if eType==FTSQUERY_NEAR */
111947 Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */
111948 Fts3Expr *pLeft; /* Left operand */
111949 Fts3Expr *pRight; /* Right operand */
111950 Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */
111951
111952 int isLoaded; /* True if aDoclist/nDoclist are initialized. */
111953 char *aDoclist; /* Buffer containing doclist */
111954 int nDoclist; /* Size of aDoclist in bytes */
 
 
111955
111956 sqlite3_int64 iCurrent;
111957 char *pCurrent;
111958 };
111959
111960 /*
111961 ** Candidate values for Fts3Query.eType. Note that the order of the first
111962 ** four values is in order of precedence when parsing expressions. For
@@ -111980,35 +113058,36 @@
111980 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *);
111981 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *);
111982 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *);
111983 SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(int, sqlite3_int64,
111984 sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**);
111985 SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(Fts3Table*,const char*,int,int,Fts3SegReader**);
 
111986 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *);
111987 SQLITE_PRIVATE int sqlite3Fts3SegReaderCost(Fts3Cursor *, Fts3SegReader *, int *);
111988 SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, sqlite3_stmt **);
111989 SQLITE_PRIVATE int sqlite3Fts3ReadLock(Fts3Table *);
111990 SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*);
111991
111992 SQLITE_PRIVATE int sqlite3Fts3SelectDoctotal(Fts3Table *, sqlite3_stmt **);
111993 SQLITE_PRIVATE int sqlite3Fts3SelectDocsize(Fts3Table *, sqlite3_int64, sqlite3_stmt **);
111994
111995 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *);
111996 SQLITE_PRIVATE int sqlite3Fts3DeferToken(Fts3Cursor *, Fts3PhraseToken *, int);
111997 SQLITE_PRIVATE int sqlite3Fts3CacheDeferredDoclists(Fts3Cursor *);
111998 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *);
111999 SQLITE_PRIVATE char *sqlite3Fts3DeferredDoclist(Fts3DeferredToken *, int *);
112000 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *);
112001
112002 #define FTS3_SEGCURSOR_PENDING -1
112003 #define FTS3_SEGCURSOR_ALL -2
 
112004
112005 SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3SegReaderCursor*, Fts3SegFilter*);
112006 SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3SegReaderCursor *);
112007 SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3SegReaderCursor *);
 
112008 SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
112009 Fts3Table *, int, const char *, int, int, int, Fts3SegReaderCursor *);
112010
112011 /* Flags allowed as part of the 4th argument to SegmentReaderIterate() */
112012 #define FTS3_SEGMENT_REQUIRE_POS 0x00000001
112013 #define FTS3_SEGMENT_IGNORE_EMPTY 0x00000002
112014 #define FTS3_SEGMENT_COLUMN_FILTER 0x00000004
@@ -112021,21 +113100,25 @@
112021 int nTerm;
112022 int iCol;
112023 int flags;
112024 };
112025
112026 struct Fts3SegReaderCursor {
112027 /* Used internally by sqlite3Fts3SegReaderXXX() calls */
112028 Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */
112029 int nSegment; /* Size of apSegment array */
112030 int nAdvance; /* How many seg-readers to advance */
112031 Fts3SegFilter *pFilter; /* Pointer to filter object */
112032 char *aBuffer; /* Buffer to merge doclists in */
112033 int nBuffer; /* Allocated size of aBuffer[] in bytes */
112034
112035 /* Cost of running this iterator. Used by fts3.c only. */
112036 int nCost;
 
 
 
 
112037
112038 /* Output values. Valid only after Fts3SegReaderStep() returns SQLITE_ROW. */
112039 char *zTerm; /* Pointer to term buffer */
112040 int nTerm; /* Size of zTerm in bytes */
112041 char *aDoclist; /* Pointer to doclist buffer */
@@ -112046,15 +113129,13 @@
112046 SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *, sqlite3_int64);
112047 SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *, sqlite_int64 *);
112048 SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *, int *);
112049 SQLITE_PRIVATE int sqlite3Fts3VarintLen(sqlite3_uint64);
112050 SQLITE_PRIVATE void sqlite3Fts3Dequote(char *);
 
112051
112052 SQLITE_PRIVATE char *sqlite3Fts3FindPositions(Fts3Cursor *, Fts3Expr *, sqlite3_int64, int);
112053 SQLITE_PRIVATE int sqlite3Fts3ExprLoadDoclist(Fts3Cursor *, Fts3Expr *);
112054 SQLITE_PRIVATE int sqlite3Fts3ExprLoadFtDoclist(Fts3Cursor *, Fts3Expr *, char **, int *);
112055 SQLITE_PRIVATE int sqlite3Fts3ExprNearTrim(Fts3Expr *, Fts3Expr *, int);
112056
112057 /* fts3_tokenizer.c */
112058 SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *, int *);
112059 SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *);
112060 SQLITE_PRIVATE int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *,
@@ -112080,20 +113161,43 @@
112080 #endif
112081
112082 /* fts3_aux.c */
112083 SQLITE_PRIVATE int sqlite3Fts3InitAux(sqlite3 *db);
112084
 
 
 
 
 
 
 
 
 
 
 
 
 
112085 #endif /* _FTSINT_H */
112086
112087 /************** End of fts3Int.h *********************************************/
112088 /************** Continuing where we left off in fts3.c ***********************/
 
 
 
 
 
112089
112090
112091 #ifndef SQLITE_CORE
112092 SQLITE_EXTENSION_INIT1
112093 #endif
112094
 
 
 
 
 
112095 /*
112096 ** Write a 64-bit variable-length integer to memory starting at p[0].
112097 ** The length of data written will be between 1 and FTS3_VARINT_MAX bytes.
112098 ** The number of bytes written is returned.
112099 */
@@ -112200,16 +113304,16 @@
112200
112201 /*
112202 ** When this function is called, *pp points to the first byte following a
112203 ** varint that is part of a doclist (or position-list, or any other list
112204 ** of varints). This function moves *pp to point to the start of that varint,
112205 ** and decrements the value stored in *pVal by the varint value.
112206 **
112207 ** Argument pStart points to the first byte of the doclist that the
112208 ** varint is part of.
112209 */
112210 static void fts3GetReverseDeltaVarint(
112211 char **pp,
112212 char *pStart,
112213 sqlite3_int64 *pVal
112214 ){
112215 sqlite3_int64 iVal;
@@ -112221,25 +113325,11 @@
112221 for(p = (*pp)-2; p>=pStart && *p&0x80; p--);
112222 p++;
112223 *pp = p;
112224
112225 sqlite3Fts3GetVarint(p, &iVal);
112226 *pVal -= iVal;
112227 }
112228
112229 /*
112230 ** As long as *pp has not reached its end (pEnd), then do the same
112231 ** as fts3GetDeltaVarint(): read a single varint and add it to *pVal.
112232 ** But if we have reached the end of the varint, just set *pp=0 and
112233 ** leave *pVal unchanged.
112234 */
112235 static void fts3GetDeltaVarint2(char **pp, char *pEnd, sqlite3_int64 *pVal){
112236 if( *pp>=pEnd ){
112237 *pp = 0;
112238 }else{
112239 fts3GetDeltaVarint(pp, pVal);
112240 }
112241 }
112242
112243 /*
112244 ** The xDisconnect() virtual table method.
112245 */
@@ -112436,10 +113526,13 @@
112436 rc = sqlite3_prepare(p->db, zSql, -1, &pStmt, 0);
112437 if( rc==SQLITE_OK ){
112438 sqlite3_step(pStmt);
112439 p->nPgsz = sqlite3_column_int(pStmt, 0);
112440 rc = sqlite3_finalize(pStmt);
 
 
 
112441 }
112442 }
112443 assert( p->nPgsz>0 || rc!=SQLITE_OK );
112444 sqlite3_free(zSql);
112445 *pRc = rc;
@@ -112608,10 +113701,91 @@
112608 fts3Appendf(pRc, &zRet, ",%s(?)", zFunction);
112609 }
112610 sqlite3_free(zFree);
112611 return zRet;
112612 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112613
112614 /*
112615 ** This function is the implementation of both the xConnect and xCreate
112616 ** methods of the FTS3 virtual table.
112617 **
@@ -112641,16 +113815,22 @@
112641 int nCol = 0; /* Number of columns in the FTS table */
112642 char *zCsr; /* Space for holding column names */
112643 int nDb; /* Bytes required to hold database name */
112644 int nName; /* Bytes required to hold table name */
112645 int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */
112646 int bNoDocsize = 0; /* True to omit %_docsize table */
112647 const char **aCol; /* Array of column names */
112648 sqlite3_tokenizer *pTokenizer = 0; /* Tokenizer for this table */
112649
112650 char *zCompress = 0;
112651 char *zUncompress = 0;
 
 
 
 
 
 
 
112652
112653 assert( strlen(argv[0])==4 );
112654 assert( (sqlite3_strnicmp(argv[0], "fts4", 4)==0 && isFts4)
112655 || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4)
112656 );
@@ -112687,32 +113867,76 @@
112687 rc = sqlite3Fts3InitTokenizer(pHash, &z[9], &pTokenizer, pzErr);
112688 }
112689
112690 /* Check if it is an FTS4 special argument. */
112691 else if( isFts4 && fts3IsSpecialColumn(z, &nKey, &zVal) ){
 
 
 
 
 
 
 
 
 
 
 
 
 
112692 if( !zVal ){
112693 rc = SQLITE_NOMEM;
112694 goto fts3_init_out;
112695 }
112696 if( nKey==9 && 0==sqlite3_strnicmp(z, "matchinfo", 9) ){
112697 if( strlen(zVal)==4 && 0==sqlite3_strnicmp(zVal, "fts3", 4) ){
112698 bNoDocsize = 1;
112699 }else{
112700 *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal);
112701 rc = SQLITE_ERROR;
112702 }
112703 }else if( nKey==8 && 0==sqlite3_strnicmp(z, "compress", 8) ){
112704 zCompress = zVal;
112705 zVal = 0;
112706 }else if( nKey==10 && 0==sqlite3_strnicmp(z, "uncompress", 10) ){
112707 zUncompress = zVal;
112708 zVal = 0;
112709 }else{
112710 *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z);
112711 rc = SQLITE_ERROR;
112712 }
112713 sqlite3_free(zVal);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112714 }
112715
112716 /* Otherwise, the argument is a column name. */
112717 else {
112718 nString += (int)(strlen(z) + 1);
@@ -112732,14 +113956,21 @@
112732 rc = sqlite3Fts3InitTokenizer(pHash, "simple", &pTokenizer, pzErr);
112733 if( rc!=SQLITE_OK ) goto fts3_init_out;
112734 }
112735 assert( pTokenizer );
112736
 
 
 
 
 
 
112737
112738 /* Allocate and populate the Fts3Table structure. */
112739 nByte = sizeof(Fts3Table) + /* Fts3Table */
112740 nCol * sizeof(char *) + /* azColumn */
 
112741 nName + /* zName */
112742 nDb + /* zDb */
112743 nString; /* Space for azColumn strings */
112744 p = (Fts3Table*)sqlite3_malloc(nByte);
112745 if( p==0 ){
@@ -112750,20 +113981,26 @@
112750 p->db = db;
112751 p->nColumn = nCol;
112752 p->nPendingData = 0;
112753 p->azColumn = (char **)&p[1];
112754 p->pTokenizer = pTokenizer;
112755 p->nNodeSize = 1000;
112756 p->nMaxPendingData = FTS3_MAX_PENDING_DATA;
112757 p->bHasDocsize = (isFts4 && bNoDocsize==0);
112758 p->bHasStat = isFts4;
 
112759 TESTONLY( p->inTransaction = -1 );
112760 TESTONLY( p->mxSavepoint = -1 );
112761 fts3HashInit(&p->pendingTerms, FTS3_HASH_STRING, 1);
 
 
 
 
 
 
112762
112763 /* Fill in the zName and zDb fields of the vtab structure. */
112764 zCsr = (char *)&p->azColumn[nCol];
112765 p->zName = zCsr;
112766 memcpy(zCsr, argv[2], nName);
112767 zCsr += nName;
112768 p->zDb = zCsr;
112769 memcpy(zCsr, argv[1], nDb);
@@ -112770,11 +114007,11 @@
112770 zCsr += nDb;
112771
112772 /* Fill in the azColumn array */
112773 for(iCol=0; iCol<nCol; iCol++){
112774 char *z;
112775 int n;
112776 z = (char *)sqlite3Fts3NextToken(aCol[iCol], &n);
112777 memcpy(zCsr, z, n);
112778 zCsr[n] = '\0';
112779 sqlite3Fts3Dequote(zCsr);
112780 p->azColumn[iCol] = zCsr;
@@ -112797,19 +114034,20 @@
112797 if( isCreate ){
112798 rc = fts3CreateTables(p);
112799 }
112800
112801 /* Figure out the page-size for the database. This is required in order to
112802 ** estimate the cost of loading large doclists from the database (see
112803 ** function sqlite3Fts3SegReaderCost() for details).
112804 */
112805 fts3DatabasePageSize(&rc, p);
 
112806
112807 /* Declare the table schema to SQLite. */
112808 fts3DeclareVtab(&rc, p);
112809
112810 fts3_init_out:
 
 
112811 sqlite3_free(zCompress);
112812 sqlite3_free(zUncompress);
112813 sqlite3_free((void *)aCol);
112814 if( rc!=SQLITE_OK ){
112815 if( p ){
@@ -112816,10 +114054,11 @@
112816 fts3DisconnectMethod((sqlite3_vtab *)p);
112817 }else if( pTokenizer ){
112818 pTokenizer->pModule->xDestroy(pTokenizer);
112819 }
112820 }else{
 
112821 *ppVTab = &p->base;
112822 }
112823 return rc;
112824 }
112825
@@ -112913,14 +114152,15 @@
112913 if( pOrder->desc ){
112914 pInfo->idxStr = "DESC";
112915 }else{
112916 pInfo->idxStr = "ASC";
112917 }
 
112918 }
112919 pInfo->orderByConsumed = 1;
112920 }
112921
 
112922 return SQLITE_OK;
112923 }
112924
112925 /*
112926 ** Implementation of xOpen method.
@@ -112952,10 +114192,11 @@
112952 sqlite3_finalize(pCsr->pStmt);
112953 sqlite3Fts3ExprFree(pCsr->pExpr);
112954 sqlite3Fts3FreeDeferredTokens(pCsr);
112955 sqlite3_free(pCsr->aDoclist);
112956 sqlite3_free(pCsr->aMatchinfo);
 
112957 sqlite3_free(pCsr);
112958 return SQLITE_OK;
112959 }
112960
112961 /*
@@ -112963,12 +114204,12 @@
112963 ** of the %_content table that contains the last match. Return
112964 ** SQLITE_OK on success.
112965 */
112966 static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){
112967 if( pCsr->isRequireSeek ){
112968 pCsr->isRequireSeek = 0;
112969 sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId);
 
112970 if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){
112971 return SQLITE_OK;
112972 }else{
112973 int rc = sqlite3_reset(pCsr->pStmt);
112974 if( rc==SQLITE_OK ){
@@ -113145,21 +114386,21 @@
113145 if( rc==SQLITE_OK && iHeight>1 ){
113146 char *zBlob = 0; /* Blob read from %_segments table */
113147 int nBlob; /* Size of zBlob in bytes */
113148
113149 if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){
113150 rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob);
113151 if( rc==SQLITE_OK ){
113152 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, 0);
113153 }
113154 sqlite3_free(zBlob);
113155 piLeaf = 0;
113156 zBlob = 0;
113157 }
113158
113159 if( rc==SQLITE_OK ){
113160 rc = sqlite3Fts3ReadBlock(p, piLeaf ? *piLeaf : *piLeaf2, &zBlob, &nBlob);
113161 }
113162 if( rc==SQLITE_OK ){
113163 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2);
113164 }
113165 sqlite3_free(zBlob);
@@ -113393,12 +114634,10 @@
113393 *pp1 = p1 + 1;
113394 *pp2 = p2 + 1;
113395 }
113396
113397 /*
113398 ** nToken==1 searches for adjacent positions.
113399 **
113400 ** This function is used to merge two position lists into one. When it is
113401 ** called, *pp1 and *pp2 must both point to position lists. A position-list is
113402 ** the part of a doclist that follows each document id. For example, if a row
113403 ** contains:
113404 **
@@ -113414,10 +114653,12 @@
113414 ** If isSaveLeft is 0, an entry is added to the output position list for
113415 ** each position in *pp2 for which there exists one or more positions in
113416 ** *pp1 so that (pos(*pp2)>pos(*pp1) && pos(*pp2)-pos(*pp1)<=nToken). i.e.
113417 ** when the *pp1 token appears before the *pp2 token, but not more than nToken
113418 ** slots before it.
 
 
113419 */
113420 static int fts3PoslistPhraseMerge(
113421 char **pp, /* IN/OUT: Preallocated output buffer */
113422 int nToken, /* Maximum difference in token positions */
113423 int isSaveLeft, /* Save the left position */
@@ -113531,11 +114772,23 @@
113531 *pp = p;
113532 return 1;
113533 }
113534
113535 /*
113536 ** Merge two position-lists as required by the NEAR operator.
 
 
 
 
 
 
 
 
 
 
 
 
113537 */
113538 static int fts3PoslistNearMerge(
113539 char **pp, /* Output buffer */
113540 char *aTmp, /* Temporary buffer space */
113541 int nRight, /* Maximum difference in token positions */
@@ -113544,230 +114797,248 @@
113544 char **pp2 /* IN/OUT: Right input list */
113545 ){
113546 char *p1 = *pp1;
113547 char *p2 = *pp2;
113548
113549 if( !pp ){
113550 if( fts3PoslistPhraseMerge(0, nRight, 0, 0, pp1, pp2) ) return 1;
113551 *pp1 = p1;
113552 *pp2 = p2;
113553 return fts3PoslistPhraseMerge(0, nLeft, 0, 0, pp2, pp1);
113554 }else{
113555 char *pTmp1 = aTmp;
113556 char *pTmp2;
113557 char *aTmp2;
113558 int res = 1;
113559
113560 fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2);
113561 aTmp2 = pTmp2 = pTmp1;
113562 *pp1 = p1;
113563 *pp2 = p2;
113564 fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1);
113565 if( pTmp1!=aTmp && pTmp2!=aTmp2 ){
113566 fts3PoslistMerge(pp, &aTmp, &aTmp2);
113567 }else if( pTmp1!=aTmp ){
113568 fts3PoslistCopy(pp, &aTmp);
113569 }else if( pTmp2!=aTmp2 ){
113570 fts3PoslistCopy(pp, &aTmp2);
113571 }else{
113572 res = 0;
113573 }
113574
113575 return res;
113576 }
113577 }
113578
113579 /*
113580 ** Values that may be used as the first parameter to fts3DoclistMerge().
113581 */
113582 #define MERGE_NOT 2 /* D + D -> D */
113583 #define MERGE_AND 3 /* D + D -> D */
113584 #define MERGE_OR 4 /* D + D -> D */
113585 #define MERGE_POS_OR 5 /* P + P -> P */
113586 #define MERGE_PHRASE 6 /* P + P -> D */
113587 #define MERGE_POS_PHRASE 7 /* P + P -> P */
113588 #define MERGE_NEAR 8 /* P + P -> D */
113589 #define MERGE_POS_NEAR 9 /* P + P -> P */
113590
113591 /*
113592 ** Merge the two doclists passed in buffer a1 (size n1 bytes) and a2
113593 ** (size n2 bytes). The output is written to pre-allocated buffer aBuffer,
113594 ** which is guaranteed to be large enough to hold the results. The number
113595 ** of bytes written to aBuffer is stored in *pnBuffer before returning.
113596 **
113597 ** If successful, SQLITE_OK is returned. Otherwise, if a malloc error
113598 ** occurs while allocating a temporary buffer as part of the merge operation,
113599 ** SQLITE_NOMEM is returned.
113600 */
113601 static int fts3DoclistMerge(
113602 int mergetype, /* One of the MERGE_XXX constants */
113603 int nParam1, /* Used by MERGE_NEAR and MERGE_POS_NEAR */
113604 int nParam2, /* Used by MERGE_NEAR and MERGE_POS_NEAR */
113605 char *aBuffer, /* Pre-allocated output buffer */
113606 int *pnBuffer, /* OUT: Bytes written to aBuffer */
113607 char *a1, /* Buffer containing first doclist */
113608 int n1, /* Size of buffer a1 */
113609 char *a2, /* Buffer containing second doclist */
113610 int n2, /* Size of buffer a2 */
113611 int *pnDoc /* OUT: Number of docids in output */
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
113612 ){
113613 sqlite3_int64 i1 = 0;
113614 sqlite3_int64 i2 = 0;
113615 sqlite3_int64 iPrev = 0;
113616
113617 char *p = aBuffer;
113618 char *p1 = a1;
113619 char *p2 = a2;
113620 char *pEnd1 = &a1[n1];
113621 char *pEnd2 = &a2[n2];
113622 int nDoc = 0;
113623
113624 assert( mergetype==MERGE_OR || mergetype==MERGE_POS_OR
113625 || mergetype==MERGE_AND || mergetype==MERGE_NOT
113626 || mergetype==MERGE_PHRASE || mergetype==MERGE_POS_PHRASE
113627 || mergetype==MERGE_NEAR || mergetype==MERGE_POS_NEAR
113628 );
113629
113630 if( !aBuffer ){
113631 *pnBuffer = 0;
113632 return SQLITE_NOMEM;
113633 }
113634
113635 /* Read the first docid from each doclist */
113636 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113637 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113638
113639 switch( mergetype ){
113640 case MERGE_OR:
113641 case MERGE_POS_OR:
113642 while( p1 || p2 ){
113643 if( p2 && p1 && i1==i2 ){
113644 fts3PutDeltaVarint(&p, &iPrev, i1);
113645 if( mergetype==MERGE_POS_OR ) fts3PoslistMerge(&p, &p1, &p2);
113646 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113647 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113648 }else if( !p2 || (p1 && i1<i2) ){
113649 fts3PutDeltaVarint(&p, &iPrev, i1);
113650 if( mergetype==MERGE_POS_OR ) fts3PoslistCopy(&p, &p1);
113651 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113652 }else{
113653 fts3PutDeltaVarint(&p, &iPrev, i2);
113654 if( mergetype==MERGE_POS_OR ) fts3PoslistCopy(&p, &p2);
113655 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113656 }
113657 }
113658 break;
113659
113660 case MERGE_AND:
113661 while( p1 && p2 ){
113662 if( i1==i2 ){
113663 fts3PutDeltaVarint(&p, &iPrev, i1);
113664 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113665 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113666 nDoc++;
113667 }else if( i1<i2 ){
113668 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113669 }else{
113670 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113671 }
113672 }
113673 break;
113674
113675 case MERGE_NOT:
113676 while( p1 ){
113677 if( p2 && i1==i2 ){
113678 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113679 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113680 }else if( !p2 || i1<i2 ){
113681 fts3PutDeltaVarint(&p, &iPrev, i1);
113682 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113683 }else{
113684 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113685 }
113686 }
113687 break;
113688
113689 case MERGE_POS_PHRASE:
113690 case MERGE_PHRASE: {
113691 char **ppPos = (mergetype==MERGE_PHRASE ? 0 : &p);
113692 while( p1 && p2 ){
113693 if( i1==i2 ){
113694 char *pSave = p;
113695 sqlite3_int64 iPrevSave = iPrev;
113696 fts3PutDeltaVarint(&p, &iPrev, i1);
113697 if( 0==fts3PoslistPhraseMerge(ppPos, nParam1, 0, 1, &p1, &p2) ){
113698 p = pSave;
113699 iPrev = iPrevSave;
113700 }else{
113701 nDoc++;
113702 }
113703 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113704 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113705 }else if( i1<i2 ){
113706 fts3PoslistCopy(0, &p1);
113707 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113708 }else{
113709 fts3PoslistCopy(0, &p2);
113710 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113711 }
113712 }
113713 break;
113714 }
113715
113716 default: assert( mergetype==MERGE_POS_NEAR || mergetype==MERGE_NEAR ); {
113717 char *aTmp = 0;
113718 char **ppPos = 0;
113719
113720 if( mergetype==MERGE_POS_NEAR ){
113721 ppPos = &p;
113722 aTmp = sqlite3_malloc(2*(n1+n2+1));
113723 if( !aTmp ){
113724 return SQLITE_NOMEM;
113725 }
113726 }
113727
113728 while( p1 && p2 ){
113729 if( i1==i2 ){
113730 char *pSave = p;
113731 sqlite3_int64 iPrevSave = iPrev;
113732 fts3PutDeltaVarint(&p, &iPrev, i1);
113733
113734 if( !fts3PoslistNearMerge(ppPos, aTmp, nParam1, nParam2, &p1, &p2) ){
113735 iPrev = iPrevSave;
113736 p = pSave;
113737 }
113738
113739 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113740 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113741 }else if( i1<i2 ){
113742 fts3PoslistCopy(0, &p1);
113743 fts3GetDeltaVarint2(&p1, pEnd1, &i1);
113744 }else{
113745 fts3PoslistCopy(0, &p2);
113746 fts3GetDeltaVarint2(&p2, pEnd2, &i2);
113747 }
113748 }
113749 sqlite3_free(aTmp);
113750 break;
113751 }
113752 }
113753
113754 if( pnDoc ) *pnDoc = nDoc;
113755 *pnBuffer = (int)(p-aBuffer);
113756 return SQLITE_OK;
113757 }
113758
113759 /*
113760 ** A pointer to an instance of this structure is used as the context
113761 ** argument to sqlite3Fts3SegReaderIterate()
113762 */
113763 typedef struct TermSelect TermSelect;
113764 struct TermSelect {
113765 int isReqPos;
113766 char *aaOutput[16]; /* Malloc'd output buffer */
113767 int anOutput[16]; /* Size of output in bytes */
113768 };
113769
113770 /*
113771 ** Merge all doclists in the TermSelect.aaOutput[] array into a single
113772 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all
113773 ** other doclists (except the aaOutput[0] one) and return SQLITE_OK.
@@ -113774,12 +115045,11 @@
113774 **
113775 ** If an OOM error occurs, return SQLITE_NOMEM. In this case it is
113776 ** the responsibility of the caller to free any doclists left in the
113777 ** TermSelect.aaOutput[] array.
113778 */
113779 static int fts3TermSelectMerge(TermSelect *pTS){
113780 int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR);
113781 char *aOut = 0;
113782 int nOut = 0;
113783 int i;
113784
113785 /* Loop through the doclists in the aaOutput[] array. Merge them all
@@ -113790,19 +115060,21 @@
113790 if( !aOut ){
113791 aOut = pTS->aaOutput[i];
113792 nOut = pTS->anOutput[i];
113793 pTS->aaOutput[i] = 0;
113794 }else{
113795 int nNew = nOut + pTS->anOutput[i];
113796 char *aNew = sqlite3_malloc(nNew);
113797 if( !aNew ){
 
 
 
 
113798 sqlite3_free(aOut);
113799 return SQLITE_NOMEM;
113800 }
113801 fts3DoclistMerge(mergetype, 0, 0,
113802 aNew, &nNew, pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, 0
113803 );
113804 sqlite3_free(pTS->aaOutput[i]);
113805 sqlite3_free(aOut);
113806 pTS->aaOutput[i] = 0;
113807 aOut = aNew;
113808 nOut = nNew;
@@ -113814,288 +115086,329 @@
113814 pTS->anOutput[0] = nOut;
113815 return SQLITE_OK;
113816 }
113817
113818 /*
113819 ** This function is used as the sqlite3Fts3SegReaderIterate() callback when
113820 ** querying the full-text index for a doclist associated with a term or
113821 ** term-prefix.
 
 
 
 
 
 
 
 
 
113822 */
113823 static int fts3TermSelectCb(
113824 Fts3Table *p, /* Virtual table object */
113825 void *pContext, /* Pointer to TermSelect structure */
113826 char *zTerm,
113827 int nTerm,
113828 char *aDoclist,
113829 int nDoclist
113830 ){
113831 TermSelect *pTS = (TermSelect *)pContext;
113832
113833 UNUSED_PARAMETER(p);
113834 UNUSED_PARAMETER(zTerm);
113835 UNUSED_PARAMETER(nTerm);
113836
113837 if( pTS->aaOutput[0]==0 ){
113838 /* If this is the first term selected, copy the doclist to the output
113839 ** buffer using memcpy(). TODO: Add a way to transfer control of the
113840 ** aDoclist buffer from the caller so as to avoid the memcpy().
113841 */
113842 pTS->aaOutput[0] = sqlite3_malloc(nDoclist);
113843 pTS->anOutput[0] = nDoclist;
113844 if( pTS->aaOutput[0] ){
113845 memcpy(pTS->aaOutput[0], aDoclist, nDoclist);
113846 }else{
113847 return SQLITE_NOMEM;
113848 }
113849 }else{
113850 int mergetype = (pTS->isReqPos ? MERGE_POS_OR : MERGE_OR);
113851 char *aMerge = aDoclist;
113852 int nMerge = nDoclist;
113853 int iOut;
113854
113855 for(iOut=0; iOut<SizeofArray(pTS->aaOutput); iOut++){
113856 char *aNew;
113857 int nNew;
113858 if( pTS->aaOutput[iOut]==0 ){
113859 assert( iOut>0 );
113860 pTS->aaOutput[iOut] = aMerge;
113861 pTS->anOutput[iOut] = nMerge;
113862 break;
113863 }
113864
113865 nNew = nMerge + pTS->anOutput[iOut];
113866 aNew = sqlite3_malloc(nNew);
113867 if( !aNew ){
113868 if( aMerge!=aDoclist ){
113869 sqlite3_free(aMerge);
113870 }
113871 return SQLITE_NOMEM;
113872 }
113873 fts3DoclistMerge(mergetype, 0, 0, aNew, &nNew,
113874 pTS->aaOutput[iOut], pTS->anOutput[iOut], aMerge, nMerge, 0
113875 );
113876
113877 if( iOut>0 ) sqlite3_free(aMerge);
113878 sqlite3_free(pTS->aaOutput[iOut]);
113879 pTS->aaOutput[iOut] = 0;
113880
113881 aMerge = aNew;
113882 nMerge = nNew;
113883 if( (iOut+1)==SizeofArray(pTS->aaOutput) ){
113884 pTS->aaOutput[iOut] = aMerge;
113885 pTS->anOutput[iOut] = nMerge;
113886 }
113887 }
113888 }
113889 return SQLITE_OK;
113890 }
113891
113892 static int fts3DeferredTermSelect(
113893 Fts3DeferredToken *pToken, /* Phrase token */
113894 int isTermPos, /* True to include positions */
113895 int *pnOut, /* OUT: Size of list */
113896 char **ppOut /* OUT: Body of list */
113897 ){
113898 char *aSource;
113899 int nSource;
113900
113901 aSource = sqlite3Fts3DeferredDoclist(pToken, &nSource);
113902 if( !aSource ){
113903 *pnOut = 0;
113904 *ppOut = 0;
113905 }else if( isTermPos ){
113906 *ppOut = sqlite3_malloc(nSource);
113907 if( !*ppOut ) return SQLITE_NOMEM;
113908 memcpy(*ppOut, aSource, nSource);
113909 *pnOut = nSource;
113910 }else{
113911 sqlite3_int64 docid;
113912 *pnOut = sqlite3Fts3GetVarint(aSource, &docid);
113913 *ppOut = sqlite3_malloc(*pnOut);
113914 if( !*ppOut ) return SQLITE_NOMEM;
113915 sqlite3Fts3PutVarint(*ppOut, docid);
113916 }
113917
113918 return SQLITE_OK;
113919 }
113920
113921 SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
113922 Fts3Table *p, /* FTS3 table handle */
 
113923 int iLevel, /* Level of segments to scan */
113924 const char *zTerm, /* Term to query for */
113925 int nTerm, /* Size of zTerm in bytes */
113926 int isPrefix, /* True for a prefix search */
113927 int isScan, /* True to scan from zTerm to EOF */
113928 Fts3SegReaderCursor *pCsr /* Cursor object to populate */
113929 ){
113930 int rc = SQLITE_OK;
113931 int rc2;
113932 int iAge = 0;
113933 sqlite3_stmt *pStmt = 0;
113934 Fts3SegReader *pPending = 0;
113935
113936 assert( iLevel==FTS3_SEGCURSOR_ALL
113937 || iLevel==FTS3_SEGCURSOR_PENDING
113938 || iLevel>=0
113939 );
113940 assert( FTS3_SEGCURSOR_PENDING<0 );
113941 assert( FTS3_SEGCURSOR_ALL<0 );
113942 assert( iLevel==FTS3_SEGCURSOR_ALL || (zTerm==0 && isPrefix==1) );
113943 assert( isPrefix==0 || isScan==0 );
113944
113945
113946 memset(pCsr, 0, sizeof(Fts3SegReaderCursor));
113947
113948 /* If iLevel is less than 0, include a seg-reader for the pending-terms. */
113949 assert( isScan==0 || fts3HashCount(&p->pendingTerms)==0 );
113950 if( iLevel<0 && isScan==0 ){
113951 rc = sqlite3Fts3SegReaderPending(p, zTerm, nTerm, isPrefix, &pPending);
113952 if( rc==SQLITE_OK && pPending ){
113953 int nByte = (sizeof(Fts3SegReader *) * 16);
113954 pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc(nByte);
113955 if( pCsr->apSegment==0 ){
113956 rc = SQLITE_NOMEM;
113957 }else{
113958 pCsr->apSegment[0] = pPending;
113959 pCsr->nSegment = 1;
113960 pPending = 0;
113961 }
113962 }
113963 }
113964
113965 if( iLevel!=FTS3_SEGCURSOR_PENDING ){
113966 if( rc==SQLITE_OK ){
113967 rc = sqlite3Fts3AllSegdirs(p, iLevel, &pStmt);
113968 }
 
113969 while( rc==SQLITE_OK && SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
 
113970
113971 /* Read the values returned by the SELECT into local variables. */
113972 sqlite3_int64 iStartBlock = sqlite3_column_int64(pStmt, 1);
113973 sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2);
113974 sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3);
113975 int nRoot = sqlite3_column_bytes(pStmt, 4);
113976 char const *zRoot = sqlite3_column_blob(pStmt, 4);
113977
113978 /* If nSegment is a multiple of 16 the array needs to be extended. */
113979 if( (pCsr->nSegment%16)==0 ){
113980 Fts3SegReader **apNew;
113981 int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*);
113982 apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte);
113983 if( !apNew ){
113984 rc = SQLITE_NOMEM;
113985 goto finished;
113986 }
113987 pCsr->apSegment = apNew;
113988 }
113989
113990 /* If zTerm is not NULL, and this segment is not stored entirely on its
113991 ** root node, the range of leaves scanned can be reduced. Do this. */
113992 if( iStartBlock && zTerm ){
113993 sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0);
113994 rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi);
113995 if( rc!=SQLITE_OK ) goto finished;
113996 if( isPrefix==0 && isScan==0 ) iLeavesEndBlock = iStartBlock;
113997 }
113998
113999 rc = sqlite3Fts3SegReaderNew(iAge, iStartBlock, iLeavesEndBlock,
114000 iEndBlock, zRoot, nRoot, &pCsr->apSegment[pCsr->nSegment]
114001 );
114002 if( rc!=SQLITE_OK ) goto finished;
114003 pCsr->nSegment++;
114004 iAge++;
114005 }
114006 }
114007
114008 finished:
114009 rc2 = sqlite3_reset(pStmt);
114010 if( rc==SQLITE_DONE ) rc = rc2;
114011 sqlite3Fts3SegReaderFree(pPending);
114012
114013 return rc;
114014 }
114015
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114016
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114017 static int fts3TermSegReaderCursor(
114018 Fts3Cursor *pCsr, /* Virtual table cursor handle */
114019 const char *zTerm, /* Term to query for */
114020 int nTerm, /* Size of zTerm in bytes */
114021 int isPrefix, /* True for a prefix search */
114022 Fts3SegReaderCursor **ppSegcsr /* OUT: Allocated seg-reader cursor */
114023 ){
114024 Fts3SegReaderCursor *pSegcsr; /* Object to allocate and return */
114025 int rc = SQLITE_NOMEM; /* Return code */
114026
114027 pSegcsr = sqlite3_malloc(sizeof(Fts3SegReaderCursor));
114028 if( pSegcsr ){
 
 
114029 Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
114030 int i;
114031 int nCost = 0;
114032 rc = sqlite3Fts3SegReaderCursor(
114033 p, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr);
114034
114035 for(i=0; rc==SQLITE_OK && i<pSegcsr->nSegment; i++){
114036 rc = sqlite3Fts3SegReaderCost(pCsr, pSegcsr->apSegment[i], &nCost);
114037 }
114038 pSegcsr->nCost = nCost;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114039 }
114040
114041 *ppSegcsr = pSegcsr;
114042 return rc;
114043 }
114044
114045 static void fts3SegReaderCursorFree(Fts3SegReaderCursor *pSegcsr){
 
 
 
114046 sqlite3Fts3SegReaderFinish(pSegcsr);
114047 sqlite3_free(pSegcsr);
114048 }
114049
114050 /*
114051 ** This function retreives the doclist for the specified term (or term
114052 ** prefix) from the database.
114053 **
114054 ** The returned doclist may be in one of two formats, depending on the
114055 ** value of parameter isReqPos. If isReqPos is zero, then the doclist is
114056 ** a sorted list of delta-compressed docids (a bare doclist). If isReqPos
114057 ** is non-zero, then the returned list is in the same format as is stored
114058 ** in the database without the found length specifier at the start of on-disk
114059 ** doclists.
114060 */
114061 static int fts3TermSelect(
114062 Fts3Table *p, /* Virtual table handle */
114063 Fts3PhraseToken *pTok, /* Token to query for */
114064 int iColumn, /* Column to query (or -ve for all columns) */
114065 int isReqPos, /* True to include position lists in output */
114066 int *pnOut, /* OUT: Size of buffer at *ppOut */
114067 char **ppOut /* OUT: Malloced result buffer */
114068 ){
114069 int rc; /* Return code */
114070 Fts3SegReaderCursor *pSegcsr; /* Seg-reader cursor for this term */
114071 TermSelect tsc; /* Context object for fts3TermSelectCb() */
114072 Fts3SegFilter filter; /* Segment term filter configuration */
114073
114074 pSegcsr = pTok->pSegcsr;
114075 memset(&tsc, 0, sizeof(TermSelect));
114076 tsc.isReqPos = isReqPos;
114077
114078 filter.flags = FTS3_SEGMENT_IGNORE_EMPTY
114079 | (pTok->isPrefix ? FTS3_SEGMENT_PREFIX : 0)
114080 | (isReqPos ? FTS3_SEGMENT_REQUIRE_POS : 0)
114081 | (iColumn<p->nColumn ? FTS3_SEGMENT_COLUMN_FILTER : 0);
114082 filter.iCol = iColumn;
114083 filter.zTerm = pTok->z;
114084 filter.nTerm = pTok->n;
114085
114086 rc = sqlite3Fts3SegReaderStart(p, pSegcsr, &filter);
114087 while( SQLITE_OK==rc
114088 && SQLITE_ROW==(rc = sqlite3Fts3SegReaderStep(p, pSegcsr))
114089 ){
114090 rc = fts3TermSelectCb(p, (void *)&tsc,
114091 pSegcsr->zTerm, pSegcsr->nTerm, pSegcsr->aDoclist, pSegcsr->nDoclist
114092 );
114093 }
114094
114095 if( rc==SQLITE_OK ){
114096 rc = fts3TermSelectMerge(&tsc);
114097 }
114098 if( rc==SQLITE_OK ){
114099 *ppOut = tsc.aaOutput[0];
114100 *pnOut = tsc.anOutput[0];
114101 }else{
@@ -114117,686 +115430,23 @@
114117 ** If the isPoslist argument is true, then it is assumed that the doclist
114118 ** contains a position-list following each docid. Otherwise, it is assumed
114119 ** that the doclist is simply a list of docids stored as delta encoded
114120 ** varints.
114121 */
114122 static int fts3DoclistCountDocids(int isPoslist, char *aList, int nList){
114123 int nDoc = 0; /* Return value */
114124 if( aList ){
114125 char *aEnd = &aList[nList]; /* Pointer to one byte after EOF */
114126 char *p = aList; /* Cursor */
114127 if( !isPoslist ){
114128 /* The number of docids in the list is the same as the number of
114129 ** varints. In FTS3 a varint consists of a single byte with the 0x80
114130 ** bit cleared and zero or more bytes with the 0x80 bit set. So to
114131 ** count the varints in the buffer, just count the number of bytes
114132 ** with the 0x80 bit clear. */
114133 while( p<aEnd ) nDoc += (((*p++)&0x80)==0);
114134 }else{
114135 while( p<aEnd ){
114136 nDoc++;
114137 while( (*p++)&0x80 ); /* Skip docid varint */
114138 fts3PoslistCopy(0, &p); /* Skip over position list */
114139 }
114140 }
114141 }
114142
114143 return nDoc;
114144 }
114145
114146 /*
114147 ** Call sqlite3Fts3DeferToken() for each token in the expression pExpr.
114148 */
114149 static int fts3DeferExpression(Fts3Cursor *pCsr, Fts3Expr *pExpr){
114150 int rc = SQLITE_OK;
114151 if( pExpr ){
114152 rc = fts3DeferExpression(pCsr, pExpr->pLeft);
114153 if( rc==SQLITE_OK ){
114154 rc = fts3DeferExpression(pCsr, pExpr->pRight);
114155 }
114156 if( pExpr->eType==FTSQUERY_PHRASE ){
114157 int iCol = pExpr->pPhrase->iColumn;
114158 int i;
114159 for(i=0; rc==SQLITE_OK && i<pExpr->pPhrase->nToken; i++){
114160 Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i];
114161 if( pToken->pDeferred==0 ){
114162 rc = sqlite3Fts3DeferToken(pCsr, pToken, iCol);
114163 }
114164 }
114165 }
114166 }
114167 return rc;
114168 }
114169
114170 /*
114171 ** This function removes the position information from a doclist. When
114172 ** called, buffer aList (size *pnList bytes) contains a doclist that includes
114173 ** position information. This function removes the position information so
114174 ** that aList contains only docids, and adjusts *pnList to reflect the new
114175 ** (possibly reduced) size of the doclist.
114176 */
114177 static void fts3DoclistStripPositions(
114178 char *aList, /* IN/OUT: Buffer containing doclist */
114179 int *pnList /* IN/OUT: Size of doclist in bytes */
114180 ){
114181 if( aList ){
114182 char *aEnd = &aList[*pnList]; /* Pointer to one byte after EOF */
114183 char *p = aList; /* Input cursor */
114184 char *pOut = aList; /* Output cursor */
114185
114186 while( p<aEnd ){
114187 sqlite3_int64 delta;
114188 p += sqlite3Fts3GetVarint(p, &delta);
114189 fts3PoslistCopy(0, &p);
114190 pOut += sqlite3Fts3PutVarint(pOut, delta);
114191 }
114192
114193 *pnList = (int)(pOut - aList);
114194 }
114195 }
114196
114197 /*
114198 ** Return a DocList corresponding to the phrase *pPhrase.
114199 **
114200 ** If this function returns SQLITE_OK, but *pnOut is set to a negative value,
114201 ** then no tokens in the phrase were looked up in the full-text index. This
114202 ** is only possible when this function is called from within xFilter(). The
114203 ** caller should assume that all documents match the phrase. The actual
114204 ** filtering will take place in xNext().
114205 */
114206 static int fts3PhraseSelect(
114207 Fts3Cursor *pCsr, /* Virtual table cursor handle */
114208 Fts3Phrase *pPhrase, /* Phrase to return a doclist for */
114209 int isReqPos, /* True if output should contain positions */
114210 char **paOut, /* OUT: Pointer to malloc'd result buffer */
114211 int *pnOut /* OUT: Size of buffer at *paOut */
114212 ){
114213 char *pOut = 0;
114214 int nOut = 0;
114215 int rc = SQLITE_OK;
114216 int ii;
114217 int iCol = pPhrase->iColumn;
114218 int isTermPos = (pPhrase->nToken>1 || isReqPos);
114219 Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
114220 int isFirst = 1;
114221
114222 int iPrevTok = 0;
114223 int nDoc = 0;
114224
114225 /* If this is an xFilter() evaluation, create a segment-reader for each
114226 ** phrase token. Or, if this is an xNext() or snippet/offsets/matchinfo
114227 ** evaluation, only create segment-readers if there are no Fts3DeferredToken
114228 ** objects attached to the phrase-tokens.
114229 */
114230 for(ii=0; ii<pPhrase->nToken; ii++){
114231 Fts3PhraseToken *pTok = &pPhrase->aToken[ii];
114232 if( pTok->pSegcsr==0 ){
114233 if( (pCsr->eEvalmode==FTS3_EVAL_FILTER)
114234 || (pCsr->eEvalmode==FTS3_EVAL_NEXT && pCsr->pDeferred==0)
114235 || (pCsr->eEvalmode==FTS3_EVAL_MATCHINFO && pTok->bFulltext)
114236 ){
114237 rc = fts3TermSegReaderCursor(
114238 pCsr, pTok->z, pTok->n, pTok->isPrefix, &pTok->pSegcsr
114239 );
114240 if( rc!=SQLITE_OK ) return rc;
114241 }
114242 }
114243 }
114244
114245 for(ii=0; ii<pPhrase->nToken; ii++){
114246 Fts3PhraseToken *pTok; /* Token to find doclist for */
114247 int iTok = 0; /* The token being queried this iteration */
114248 char *pList = 0; /* Pointer to token doclist */
114249 int nList = 0; /* Size of buffer at pList */
114250
114251 /* Select a token to process. If this is an xFilter() call, then tokens
114252 ** are processed in order from least to most costly. Otherwise, tokens
114253 ** are processed in the order in which they occur in the phrase.
114254 */
114255 if( pCsr->eEvalmode==FTS3_EVAL_MATCHINFO ){
114256 assert( isReqPos );
114257 iTok = ii;
114258 pTok = &pPhrase->aToken[iTok];
114259 if( pTok->bFulltext==0 ) continue;
114260 }else if( pCsr->eEvalmode==FTS3_EVAL_NEXT || isReqPos ){
114261 iTok = ii;
114262 pTok = &pPhrase->aToken[iTok];
114263 }else{
114264 int nMinCost = 0x7FFFFFFF;
114265 int jj;
114266
114267 /* Find the remaining token with the lowest cost. */
114268 for(jj=0; jj<pPhrase->nToken; jj++){
114269 Fts3SegReaderCursor *pSegcsr = pPhrase->aToken[jj].pSegcsr;
114270 if( pSegcsr && pSegcsr->nCost<nMinCost ){
114271 iTok = jj;
114272 nMinCost = pSegcsr->nCost;
114273 }
114274 }
114275 pTok = &pPhrase->aToken[iTok];
114276
114277 /* This branch is taken if it is determined that loading the doclist
114278 ** for the next token would require more IO than loading all documents
114279 ** currently identified by doclist pOut/nOut. No further doclists will
114280 ** be loaded from the full-text index for this phrase.
114281 */
114282 if( nMinCost>nDoc && ii>0 ){
114283 rc = fts3DeferExpression(pCsr, pCsr->pExpr);
114284 break;
114285 }
114286 }
114287
114288 if( pCsr->eEvalmode==FTS3_EVAL_NEXT && pTok->pDeferred ){
114289 rc = fts3DeferredTermSelect(pTok->pDeferred, isTermPos, &nList, &pList);
114290 }else{
114291 if( pTok->pSegcsr ){
114292 rc = fts3TermSelect(p, pTok, iCol, isTermPos, &nList, &pList);
114293 }
114294 pTok->bFulltext = 1;
114295 }
114296 assert( rc!=SQLITE_OK || pCsr->eEvalmode || pTok->pSegcsr==0 );
114297 if( rc!=SQLITE_OK ) break;
114298
114299 if( isFirst ){
114300 pOut = pList;
114301 nOut = nList;
114302 if( pCsr->eEvalmode==FTS3_EVAL_FILTER && pPhrase->nToken>1 ){
114303 nDoc = fts3DoclistCountDocids(1, pOut, nOut);
114304 }
114305 isFirst = 0;
114306 iPrevTok = iTok;
114307 }else{
114308 /* Merge the new term list and the current output. */
114309 char *aLeft, *aRight;
114310 int nLeft, nRight;
114311 int nDist;
114312 int mt;
114313
114314 /* If this is the final token of the phrase, and positions were not
114315 ** requested by the caller, use MERGE_PHRASE instead of POS_PHRASE.
114316 ** This drops the position information from the output list.
114317 */
114318 mt = MERGE_POS_PHRASE;
114319 if( ii==pPhrase->nToken-1 && !isReqPos ) mt = MERGE_PHRASE;
114320
114321 assert( iPrevTok!=iTok );
114322 if( iPrevTok<iTok ){
114323 aLeft = pOut;
114324 nLeft = nOut;
114325 aRight = pList;
114326 nRight = nList;
114327 nDist = iTok-iPrevTok;
114328 iPrevTok = iTok;
114329 }else{
114330 aRight = pOut;
114331 nRight = nOut;
114332 aLeft = pList;
114333 nLeft = nList;
114334 nDist = iPrevTok-iTok;
114335 }
114336 pOut = aRight;
114337 fts3DoclistMerge(
114338 mt, nDist, 0, pOut, &nOut, aLeft, nLeft, aRight, nRight, &nDoc
114339 );
114340 sqlite3_free(aLeft);
114341 }
114342 assert( nOut==0 || pOut!=0 );
114343 }
114344
114345 if( rc==SQLITE_OK ){
114346 if( ii!=pPhrase->nToken ){
114347 assert( pCsr->eEvalmode==FTS3_EVAL_FILTER && isReqPos==0 );
114348 fts3DoclistStripPositions(pOut, &nOut);
114349 }
114350 *paOut = pOut;
114351 *pnOut = nOut;
114352 }else{
114353 sqlite3_free(pOut);
114354 }
114355 return rc;
114356 }
114357
114358 /*
114359 ** This function merges two doclists according to the requirements of a
114360 ** NEAR operator.
114361 **
114362 ** Both input doclists must include position information. The output doclist
114363 ** includes position information if the first argument to this function
114364 ** is MERGE_POS_NEAR, or does not if it is MERGE_NEAR.
114365 */
114366 static int fts3NearMerge(
114367 int mergetype, /* MERGE_POS_NEAR or MERGE_NEAR */
114368 int nNear, /* Parameter to NEAR operator */
114369 int nTokenLeft, /* Number of tokens in LHS phrase arg */
114370 char *aLeft, /* Doclist for LHS (incl. positions) */
114371 int nLeft, /* Size of LHS doclist in bytes */
114372 int nTokenRight, /* As nTokenLeft */
114373 char *aRight, /* As aLeft */
114374 int nRight, /* As nRight */
114375 char **paOut, /* OUT: Results of merge (malloced) */
114376 int *pnOut /* OUT: Sized of output buffer */
114377 ){
114378 char *aOut; /* Buffer to write output doclist to */
114379 int rc; /* Return code */
114380
114381 assert( mergetype==MERGE_POS_NEAR || MERGE_NEAR );
114382
114383 aOut = sqlite3_malloc(nLeft+nRight+1);
114384 if( aOut==0 ){
114385 rc = SQLITE_NOMEM;
114386 }else{
114387 rc = fts3DoclistMerge(mergetype, nNear+nTokenRight, nNear+nTokenLeft,
114388 aOut, pnOut, aLeft, nLeft, aRight, nRight, 0
114389 );
114390 if( rc!=SQLITE_OK ){
114391 sqlite3_free(aOut);
114392 aOut = 0;
114393 }
114394 }
114395
114396 *paOut = aOut;
114397 return rc;
114398 }
114399
114400 /*
114401 ** This function is used as part of the processing for the snippet() and
114402 ** offsets() functions.
114403 **
114404 ** Both pLeft and pRight are expression nodes of type FTSQUERY_PHRASE. Both
114405 ** have their respective doclists (including position information) loaded
114406 ** in Fts3Expr.aDoclist/nDoclist. This function removes all entries from
114407 ** each doclist that are not within nNear tokens of a corresponding entry
114408 ** in the other doclist.
114409 */
114410 SQLITE_PRIVATE int sqlite3Fts3ExprNearTrim(Fts3Expr *pLeft, Fts3Expr *pRight, int nNear){
114411 int rc; /* Return code */
114412
114413 assert( pLeft->eType==FTSQUERY_PHRASE );
114414 assert( pRight->eType==FTSQUERY_PHRASE );
114415 assert( pLeft->isLoaded && pRight->isLoaded );
114416
114417 if( pLeft->aDoclist==0 || pRight->aDoclist==0 ){
114418 sqlite3_free(pLeft->aDoclist);
114419 sqlite3_free(pRight->aDoclist);
114420 pRight->aDoclist = 0;
114421 pLeft->aDoclist = 0;
114422 rc = SQLITE_OK;
114423 }else{
114424 char *aOut; /* Buffer in which to assemble new doclist */
114425 int nOut; /* Size of buffer aOut in bytes */
114426
114427 rc = fts3NearMerge(MERGE_POS_NEAR, nNear,
114428 pLeft->pPhrase->nToken, pLeft->aDoclist, pLeft->nDoclist,
114429 pRight->pPhrase->nToken, pRight->aDoclist, pRight->nDoclist,
114430 &aOut, &nOut
114431 );
114432 if( rc!=SQLITE_OK ) return rc;
114433 sqlite3_free(pRight->aDoclist);
114434 pRight->aDoclist = aOut;
114435 pRight->nDoclist = nOut;
114436
114437 rc = fts3NearMerge(MERGE_POS_NEAR, nNear,
114438 pRight->pPhrase->nToken, pRight->aDoclist, pRight->nDoclist,
114439 pLeft->pPhrase->nToken, pLeft->aDoclist, pLeft->nDoclist,
114440 &aOut, &nOut
114441 );
114442 sqlite3_free(pLeft->aDoclist);
114443 pLeft->aDoclist = aOut;
114444 pLeft->nDoclist = nOut;
114445 }
114446 return rc;
114447 }
114448
114449
114450 /*
114451 ** Allocate an Fts3SegReaderArray for each token in the expression pExpr.
114452 ** The allocated objects are stored in the Fts3PhraseToken.pArray member
114453 ** variables of each token structure.
114454 */
114455 static int fts3ExprAllocateSegReaders(
114456 Fts3Cursor *pCsr, /* FTS3 table */
114457 Fts3Expr *pExpr, /* Expression to create seg-readers for */
114458 int *pnExpr /* OUT: Number of AND'd expressions */
114459 ){
114460 int rc = SQLITE_OK; /* Return code */
114461
114462 assert( pCsr->eEvalmode==FTS3_EVAL_FILTER );
114463 if( pnExpr && pExpr->eType!=FTSQUERY_AND ){
114464 (*pnExpr)++;
114465 pnExpr = 0;
114466 }
114467
114468 if( pExpr->eType==FTSQUERY_PHRASE ){
114469 Fts3Phrase *pPhrase = pExpr->pPhrase;
114470 int ii;
114471
114472 for(ii=0; rc==SQLITE_OK && ii<pPhrase->nToken; ii++){
114473 Fts3PhraseToken *pTok = &pPhrase->aToken[ii];
114474 if( pTok->pSegcsr==0 ){
114475 rc = fts3TermSegReaderCursor(
114476 pCsr, pTok->z, pTok->n, pTok->isPrefix, &pTok->pSegcsr
114477 );
114478 }
114479 }
114480 }else{
114481 rc = fts3ExprAllocateSegReaders(pCsr, pExpr->pLeft, pnExpr);
114482 if( rc==SQLITE_OK ){
114483 rc = fts3ExprAllocateSegReaders(pCsr, pExpr->pRight, pnExpr);
114484 }
114485 }
114486 return rc;
114487 }
114488
114489 /*
114490 ** Free the Fts3SegReaderArray objects associated with each token in the
114491 ** expression pExpr. In other words, this function frees the resources
114492 ** allocated by fts3ExprAllocateSegReaders().
114493 */
114494 static void fts3ExprFreeSegReaders(Fts3Expr *pExpr){
114495 if( pExpr ){
114496 Fts3Phrase *pPhrase = pExpr->pPhrase;
114497 if( pPhrase ){
114498 int kk;
114499 for(kk=0; kk<pPhrase->nToken; kk++){
114500 fts3SegReaderCursorFree(pPhrase->aToken[kk].pSegcsr);
114501 pPhrase->aToken[kk].pSegcsr = 0;
114502 }
114503 }
114504 fts3ExprFreeSegReaders(pExpr->pLeft);
114505 fts3ExprFreeSegReaders(pExpr->pRight);
114506 }
114507 }
114508
114509 /*
114510 ** Return the sum of the costs of all tokens in the expression pExpr. This
114511 ** function must be called after Fts3SegReaderArrays have been allocated
114512 ** for all tokens using fts3ExprAllocateSegReaders().
114513 */
114514 static int fts3ExprCost(Fts3Expr *pExpr){
114515 int nCost; /* Return value */
114516 if( pExpr->eType==FTSQUERY_PHRASE ){
114517 Fts3Phrase *pPhrase = pExpr->pPhrase;
114518 int ii;
114519 nCost = 0;
114520 for(ii=0; ii<pPhrase->nToken; ii++){
114521 Fts3SegReaderCursor *pSegcsr = pPhrase->aToken[ii].pSegcsr;
114522 if( pSegcsr ) nCost += pSegcsr->nCost;
114523 }
114524 }else{
114525 nCost = fts3ExprCost(pExpr->pLeft) + fts3ExprCost(pExpr->pRight);
114526 }
114527 return nCost;
114528 }
114529
114530 /*
114531 ** The following is a helper function (and type) for fts3EvalExpr(). It
114532 ** must be called after Fts3SegReaders have been allocated for every token
114533 ** in the expression. See the context it is called from in fts3EvalExpr()
114534 ** for further explanation.
114535 */
114536 typedef struct ExprAndCost ExprAndCost;
114537 struct ExprAndCost {
114538 Fts3Expr *pExpr;
114539 int nCost;
114540 };
114541 static void fts3ExprAssignCosts(
114542 Fts3Expr *pExpr, /* Expression to create seg-readers for */
114543 ExprAndCost **ppExprCost /* OUT: Write to *ppExprCost */
114544 ){
114545 if( pExpr->eType==FTSQUERY_AND ){
114546 fts3ExprAssignCosts(pExpr->pLeft, ppExprCost);
114547 fts3ExprAssignCosts(pExpr->pRight, ppExprCost);
114548 }else{
114549 (*ppExprCost)->pExpr = pExpr;
114550 (*ppExprCost)->nCost = fts3ExprCost(pExpr);
114551 (*ppExprCost)++;
114552 }
114553 }
114554
114555 /*
114556 ** Evaluate the full-text expression pExpr against FTS3 table pTab. Store
114557 ** the resulting doclist in *paOut and *pnOut. This routine mallocs for
114558 ** the space needed to store the output. The caller is responsible for
114559 ** freeing the space when it has finished.
114560 **
114561 ** This function is called in two distinct contexts:
114562 **
114563 ** * From within the virtual table xFilter() method. In this case, the
114564 ** output doclist contains entries for all rows in the table, based on
114565 ** data read from the full-text index.
114566 **
114567 ** In this case, if the query expression contains one or more tokens that
114568 ** are very common, then the returned doclist may contain a superset of
114569 ** the documents that actually match the expression.
114570 **
114571 ** * From within the virtual table xNext() method. This call is only made
114572 ** if the call from within xFilter() found that there were very common
114573 ** tokens in the query expression and did return a superset of the
114574 ** matching documents. In this case the returned doclist contains only
114575 ** entries that correspond to the current row of the table. Instead of
114576 ** reading the data for each token from the full-text index, the data is
114577 ** already available in-memory in the Fts3PhraseToken.pDeferred structures.
114578 ** See fts3EvalDeferred() for how it gets there.
114579 **
114580 ** In the first case above, Fts3Cursor.doDeferred==0. In the second (if it is
114581 ** required) Fts3Cursor.doDeferred==1.
114582 **
114583 ** If the SQLite invokes the snippet(), offsets() or matchinfo() function
114584 ** as part of a SELECT on an FTS3 table, this function is called on each
114585 ** individual phrase expression in the query. If there were very common tokens
114586 ** found in the xFilter() call, then this function is called once for phrase
114587 ** for each row visited, and the returned doclist contains entries for the
114588 ** current row only. Otherwise, if there were no very common tokens, then this
114589 ** function is called once only for each phrase in the query and the returned
114590 ** doclist contains entries for all rows of the table.
114591 **
114592 ** Fts3Cursor.doDeferred==1 when this function is called on phrases as a
114593 ** result of a snippet(), offsets() or matchinfo() invocation.
114594 */
114595 static int fts3EvalExpr(
114596 Fts3Cursor *p, /* Virtual table cursor handle */
114597 Fts3Expr *pExpr, /* Parsed fts3 expression */
114598 char **paOut, /* OUT: Pointer to malloc'd result buffer */
114599 int *pnOut, /* OUT: Size of buffer at *paOut */
114600 int isReqPos /* Require positions in output buffer */
114601 ){
114602 int rc = SQLITE_OK; /* Return code */
114603
114604 /* Zero the output parameters. */
114605 *paOut = 0;
114606 *pnOut = 0;
114607
114608 if( pExpr ){
114609 assert( pExpr->eType==FTSQUERY_NEAR || pExpr->eType==FTSQUERY_OR
114610 || pExpr->eType==FTSQUERY_AND || pExpr->eType==FTSQUERY_NOT
114611 || pExpr->eType==FTSQUERY_PHRASE
114612 );
114613 assert( pExpr->eType==FTSQUERY_PHRASE || isReqPos==0 );
114614
114615 if( pExpr->eType==FTSQUERY_PHRASE ){
114616 rc = fts3PhraseSelect(p, pExpr->pPhrase,
114617 isReqPos || (pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR),
114618 paOut, pnOut
114619 );
114620 fts3ExprFreeSegReaders(pExpr);
114621 }else if( p->eEvalmode==FTS3_EVAL_FILTER && pExpr->eType==FTSQUERY_AND ){
114622 ExprAndCost *aExpr = 0; /* Array of AND'd expressions and costs */
114623 int nExpr = 0; /* Size of aExpr[] */
114624 char *aRet = 0; /* Doclist to return to caller */
114625 int nRet = 0; /* Length of aRet[] in bytes */
114626 int nDoc = 0x7FFFFFFF;
114627
114628 assert( !isReqPos );
114629
114630 rc = fts3ExprAllocateSegReaders(p, pExpr, &nExpr);
114631 if( rc==SQLITE_OK ){
114632 assert( nExpr>1 );
114633 aExpr = sqlite3_malloc(sizeof(ExprAndCost) * nExpr);
114634 if( !aExpr ) rc = SQLITE_NOMEM;
114635 }
114636 if( rc==SQLITE_OK ){
114637 int ii; /* Used to iterate through expressions */
114638
114639 fts3ExprAssignCosts(pExpr, &aExpr);
114640 aExpr -= nExpr;
114641 for(ii=0; ii<nExpr; ii++){
114642 char *aNew;
114643 int nNew;
114644 int jj;
114645 ExprAndCost *pBest = 0;
114646
114647 for(jj=0; jj<nExpr; jj++){
114648 ExprAndCost *pCand = &aExpr[jj];
114649 if( pCand->pExpr && (pBest==0 || pCand->nCost<pBest->nCost) ){
114650 pBest = pCand;
114651 }
114652 }
114653
114654 if( pBest->nCost>nDoc ){
114655 rc = fts3DeferExpression(p, p->pExpr);
114656 break;
114657 }else{
114658 rc = fts3EvalExpr(p, pBest->pExpr, &aNew, &nNew, 0);
114659 if( rc!=SQLITE_OK ) break;
114660 pBest->pExpr = 0;
114661 if( ii==0 ){
114662 aRet = aNew;
114663 nRet = nNew;
114664 nDoc = fts3DoclistCountDocids(0, aRet, nRet);
114665 }else{
114666 fts3DoclistMerge(
114667 MERGE_AND, 0, 0, aRet, &nRet, aRet, nRet, aNew, nNew, &nDoc
114668 );
114669 sqlite3_free(aNew);
114670 }
114671 }
114672 }
114673 }
114674
114675 if( rc==SQLITE_OK ){
114676 *paOut = aRet;
114677 *pnOut = nRet;
114678 }else{
114679 assert( *paOut==0 );
114680 sqlite3_free(aRet);
114681 }
114682 sqlite3_free(aExpr);
114683 fts3ExprFreeSegReaders(pExpr);
114684
114685 }else{
114686 char *aLeft;
114687 char *aRight;
114688 int nLeft;
114689 int nRight;
114690
114691 assert( pExpr->eType==FTSQUERY_NEAR
114692 || pExpr->eType==FTSQUERY_OR
114693 || pExpr->eType==FTSQUERY_NOT
114694 || (pExpr->eType==FTSQUERY_AND && p->eEvalmode==FTS3_EVAL_NEXT)
114695 );
114696
114697 if( 0==(rc = fts3EvalExpr(p, pExpr->pRight, &aRight, &nRight, isReqPos))
114698 && 0==(rc = fts3EvalExpr(p, pExpr->pLeft, &aLeft, &nLeft, isReqPos))
114699 ){
114700 switch( pExpr->eType ){
114701 case FTSQUERY_NEAR: {
114702 Fts3Expr *pLeft;
114703 Fts3Expr *pRight;
114704 int mergetype = MERGE_NEAR;
114705 if( pExpr->pParent && pExpr->pParent->eType==FTSQUERY_NEAR ){
114706 mergetype = MERGE_POS_NEAR;
114707 }
114708 pLeft = pExpr->pLeft;
114709 while( pLeft->eType==FTSQUERY_NEAR ){
114710 pLeft=pLeft->pRight;
114711 }
114712 pRight = pExpr->pRight;
114713 assert( pRight->eType==FTSQUERY_PHRASE );
114714 assert( pLeft->eType==FTSQUERY_PHRASE );
114715
114716 rc = fts3NearMerge(mergetype, pExpr->nNear,
114717 pLeft->pPhrase->nToken, aLeft, nLeft,
114718 pRight->pPhrase->nToken, aRight, nRight,
114719 paOut, pnOut
114720 );
114721 sqlite3_free(aLeft);
114722 break;
114723 }
114724
114725 case FTSQUERY_OR: {
114726 /* Allocate a buffer for the output. The maximum size is the
114727 ** sum of the sizes of the two input buffers. The +1 term is
114728 ** so that a buffer of zero bytes is never allocated - this can
114729 ** cause fts3DoclistMerge() to incorrectly return SQLITE_NOMEM.
114730 */
114731 char *aBuffer = sqlite3_malloc(nRight+nLeft+1);
114732 rc = fts3DoclistMerge(MERGE_OR, 0, 0, aBuffer, pnOut,
114733 aLeft, nLeft, aRight, nRight, 0
114734 );
114735 *paOut = aBuffer;
114736 sqlite3_free(aLeft);
114737 break;
114738 }
114739
114740 default: {
114741 assert( FTSQUERY_NOT==MERGE_NOT && FTSQUERY_AND==MERGE_AND );
114742 fts3DoclistMerge(pExpr->eType, 0, 0, aLeft, pnOut,
114743 aLeft, nLeft, aRight, nRight, 0
114744 );
114745 *paOut = aLeft;
114746 break;
114747 }
114748 }
114749 }
114750 sqlite3_free(aRight);
114751 }
114752 }
114753
114754 assert( rc==SQLITE_OK || *paOut==0 );
114755 return rc;
114756 }
114757
114758 /*
114759 ** This function is called from within xNext() for each row visited by
114760 ** an FTS3 query. If evaluating the FTS3 query expression within xFilter()
114761 ** was able to determine the exact set of matching rows, this function sets
114762 ** *pbRes to true and returns SQLITE_IO immediately.
114763 **
114764 ** Otherwise, if evaluating the query expression within xFilter() returned a
114765 ** superset of the matching documents instead of an exact set (this happens
114766 ** when the query includes very common tokens and it is deemed too expensive to
114767 ** load their doclists from disk), this function tests if the current row
114768 ** really does match the FTS3 query.
114769 **
114770 ** If an error occurs, an SQLite error code is returned. Otherwise, SQLITE_OK
114771 ** is returned and *pbRes is set to true if the current row matches the
114772 ** FTS3 query (and should be included in the results returned to SQLite), or
114773 ** false otherwise.
114774 */
114775 static int fts3EvalDeferred(
114776 Fts3Cursor *pCsr, /* FTS3 cursor pointing at row to test */
114777 int *pbRes /* OUT: Set to true if row is a match */
114778 ){
114779 int rc = SQLITE_OK;
114780 if( pCsr->pDeferred==0 ){
114781 *pbRes = 1;
114782 }else{
114783 rc = fts3CursorSeek(0, pCsr);
114784 if( rc==SQLITE_OK ){
114785 sqlite3Fts3FreeDeferredDoclists(pCsr);
114786 rc = sqlite3Fts3CacheDeferredDoclists(pCsr);
114787 }
114788 if( rc==SQLITE_OK ){
114789 char *a = 0;
114790 int n = 0;
114791 rc = fts3EvalExpr(pCsr, pCsr->pExpr, &a, &n, 0);
114792 assert( n>=0 );
114793 *pbRes = (n>0);
114794 sqlite3_free(a);
114795 }
114796 }
114797 return rc;
114798 }
114799
114800 /*
114801 ** Advance the cursor to the next row in the %_content table that
114802 ** matches the search criteria. For a MATCH search, this will be
@@ -114807,43 +115457,24 @@
114807 ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned
114808 ** even if we reach end-of-file. The fts3EofMethod() will be called
114809 ** subsequently to determine whether or not an EOF was hit.
114810 */
114811 static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){
114812 int res;
114813 int rc = SQLITE_OK; /* Return code */
114814 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
114815
114816 pCsr->eEvalmode = FTS3_EVAL_NEXT;
114817 do {
114818 if( pCsr->aDoclist==0 ){
114819 if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){
114820 pCsr->isEof = 1;
114821 rc = sqlite3_reset(pCsr->pStmt);
114822 break;
114823 }
114824 pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0);
114825 }else{
114826 if( pCsr->desc==0 ){
114827 if( pCsr->pNextId>=&pCsr->aDoclist[pCsr->nDoclist] ){
114828 pCsr->isEof = 1;
114829 break;
114830 }
114831 fts3GetDeltaVarint(&pCsr->pNextId, &pCsr->iPrevId);
114832 }else{
114833 fts3GetReverseDeltaVarint(&pCsr->pNextId,pCsr->aDoclist,&pCsr->iPrevId);
114834 if( pCsr->pNextId<=pCsr->aDoclist ){
114835 pCsr->isEof = 1;
114836 break;
114837 }
114838 }
114839 sqlite3_reset(pCsr->pStmt);
114840 pCsr->isRequireSeek = 1;
114841 pCsr->isMatchinfoNeeded = 1;
114842 }
114843 }while( SQLITE_OK==(rc = fts3EvalDeferred(pCsr, &res)) && res==0 );
114844
114845 return rc;
114846 }
114847
114848 /*
114849 ** This is the xFilter interface for the virtual table. See
@@ -114866,15 +115497,11 @@
114866 int idxNum, /* Strategy index */
114867 const char *idxStr, /* Unused */
114868 int nVal, /* Number of elements in apVal */
114869 sqlite3_value **apVal /* Arguments for the indexing scheme */
114870 ){
114871 const char *azSql[] = {
114872 "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?", /* non-full-scan */
114873 "SELECT %s FROM %Q.'%q_content' AS x ORDER BY docid %s", /* full-scan */
114874 };
114875 int rc; /* Return code */
114876 char *zSql; /* SQL statement used to access %_content */
114877 Fts3Table *p = (Fts3Table *)pCursor->pVtab;
114878 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
114879
114880 UNUSED_PARAMETER(idxStr);
@@ -114889,10 +115516,17 @@
114889 sqlite3_finalize(pCsr->pStmt);
114890 sqlite3_free(pCsr->aDoclist);
114891 sqlite3Fts3ExprFree(pCsr->pExpr);
114892 memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor));
114893
 
 
 
 
 
 
 
114894 if( idxNum!=FTS3_DOCID_SEARCH && idxNum!=FTS3_FULLSCAN_SEARCH ){
114895 int iCol = idxNum-FTS3_FULLTEXT_SEARCH;
114896 const char *zQuery = (const char *)sqlite3_value_text(apVal[0]);
114897
114898 if( zQuery==0 && sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
@@ -114902,20 +115536,21 @@
114902 rc = sqlite3Fts3ExprParse(p->pTokenizer, p->azColumn, p->nColumn,
114903 iCol, zQuery, -1, &pCsr->pExpr
114904 );
114905 if( rc!=SQLITE_OK ){
114906 if( rc==SQLITE_ERROR ){
114907 p->base.zErrMsg = sqlite3_mprintf("malformed MATCH expression: [%s]",
114908 zQuery);
114909 }
114910 return rc;
114911 }
114912
114913 rc = sqlite3Fts3ReadLock(p);
114914 if( rc!=SQLITE_OK ) return rc;
114915
114916 rc = fts3EvalExpr(pCsr, pCsr->pExpr, &pCsr->aDoclist, &pCsr->nDoclist, 0);
 
114917 sqlite3Fts3SegmentsClose(p);
114918 if( rc!=SQLITE_OK ) return rc;
114919 pCsr->pNextId = pCsr->aDoclist;
114920 pCsr->iPrevId = 0;
114921 }
@@ -114923,41 +115558,28 @@
114923 /* Compile a SELECT statement for this cursor. For a full-table-scan, the
114924 ** statement loops through all rows of the %_content table. For a
114925 ** full-text query or docid lookup, the statement retrieves a single
114926 ** row by docid.
114927 */
114928 zSql = (char *)azSql[idxNum==FTS3_FULLSCAN_SEARCH];
114929 zSql = sqlite3_mprintf(
114930 zSql, p->zReadExprlist, p->zDb, p->zName, (idxStr ? idxStr : "ASC")
114931 );
114932 if( !zSql ){
114933 rc = SQLITE_NOMEM;
114934 }else{
114935 rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0);
114936 sqlite3_free(zSql);
114937 }
114938 if( rc==SQLITE_OK && idxNum==FTS3_DOCID_SEARCH ){
 
 
 
 
 
114939 rc = sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]);
114940 }
114941 pCsr->eSearch = (i16)idxNum;
114942
114943 assert( pCsr->desc==0 );
114944 if( rc!=SQLITE_OK ) return rc;
114945 if( rc==SQLITE_OK && pCsr->nDoclist>0 && idxStr && idxStr[0]=='D' ){
114946 sqlite3_int64 iDocid = 0;
114947 char *csr = pCsr->aDoclist;
114948 while( csr<&pCsr->aDoclist[pCsr->nDoclist] ){
114949 fts3GetDeltaVarint(&csr, &iDocid);
114950 }
114951 pCsr->pNextId = csr;
114952 pCsr->iPrevId = iDocid;
114953 pCsr->desc = 1;
114954 pCsr->isRequireSeek = 1;
114955 pCsr->isMatchinfoNeeded = 1;
114956 pCsr->eEvalmode = FTS3_EVAL_NEXT;
114957 return SQLITE_OK;
114958 }
114959 return fts3NextMethod(pCursor);
114960 }
114961
114962 /*
114963 ** This is the xEof method of the virtual table. SQLite calls this
@@ -114973,20 +115595,11 @@
114973 ** exposes %_content.docid as the rowid for the virtual table. The
114974 ** rowid should be written to *pRowid.
114975 */
114976 static int fts3RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
114977 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor;
114978 if( pCsr->aDoclist ){
114979 *pRowid = pCsr->iPrevId;
114980 }else{
114981 /* This branch runs if the query is implemented using a full-table scan
114982 ** (not using the full-text index). In this case grab the rowid from the
114983 ** SELECT statement.
114984 */
114985 assert( pCsr->isRequireSeek==0 );
114986 *pRowid = sqlite3_column_int64(pCsr->pStmt, 0);
114987 }
114988 return SQLITE_OK;
114989 }
114990
114991 /*
114992 ** This is the xColumn method, called by SQLite to request a value from
@@ -114995,11 +115608,11 @@
114995 static int fts3ColumnMethod(
114996 sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */
114997 sqlite3_context *pContext, /* Context for sqlite3_result_xxx() calls */
114998 int iCol /* Index of column to read value from */
114999 ){
115000 int rc; /* Return Code */
115001 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor;
115002 Fts3Table *p = (Fts3Table *)pCursor->pVtab;
115003
115004 /* The column value supplied by SQLite must be in range. */
115005 assert( iCol>=0 && iCol<=p->nColumn+1 );
@@ -115006,25 +115619,24 @@
115006
115007 if( iCol==p->nColumn+1 ){
115008 /* This call is a request for the "docid" column. Since "docid" is an
115009 ** alias for "rowid", use the xRowid() method to obtain the value.
115010 */
115011 sqlite3_int64 iRowid;
115012 rc = fts3RowidMethod(pCursor, &iRowid);
115013 sqlite3_result_int64(pContext, iRowid);
115014 }else if( iCol==p->nColumn ){
115015 /* The extra column whose name is the same as the table.
115016 ** Return a blob which is a pointer to the cursor.
115017 */
115018 sqlite3_result_blob(pContext, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT);
115019 rc = SQLITE_OK;
115020 }else{
115021 rc = fts3CursorSeek(0, pCsr);
115022 if( rc==SQLITE_OK ){
115023 sqlite3_result_value(pContext, sqlite3_column_value(pCsr->pStmt, iCol+1));
115024 }
115025 }
 
 
115026 return rc;
115027 }
115028
115029 /*
115030 ** This function is the implementation of the xUpdate callback used by
@@ -115052,12 +115664,13 @@
115052
115053 /*
115054 ** Implementation of xBegin() method. This is a no-op.
115055 */
115056 static int fts3BeginMethod(sqlite3_vtab *pVtab){
115057 UNUSED_PARAMETER(pVtab);
115058 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
 
 
115059 assert( p->nPendingData==0 );
115060 assert( p->inTransaction!=1 );
115061 TESTONLY( p->inTransaction = 1 );
115062 TESTONLY( p->mxSavepoint = -1; );
115063 return SQLITE_OK;
@@ -115067,14 +115680,15 @@
115067 ** Implementation of xCommit() method. This is a no-op. The contents of
115068 ** the pending-terms hash-table have already been flushed into the database
115069 ** by fts3SyncMethod().
115070 */
115071 static int fts3CommitMethod(sqlite3_vtab *pVtab){
115072 UNUSED_PARAMETER(pVtab);
115073 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
 
115074 assert( p->nPendingData==0 );
115075 assert( p->inTransaction!=0 );
 
115076 TESTONLY( p->inTransaction = 0 );
115077 TESTONLY( p->mxSavepoint = -1; );
115078 return SQLITE_OK;
115079 }
115080
@@ -115088,132 +115702,30 @@
115088 assert( p->inTransaction!=0 );
115089 TESTONLY( p->inTransaction = 0 );
115090 TESTONLY( p->mxSavepoint = -1; );
115091 return SQLITE_OK;
115092 }
115093
115094 /*
115095 ** Load the doclist associated with expression pExpr to pExpr->aDoclist.
115096 ** The loaded doclist contains positions as well as the document ids.
115097 ** This is used by the matchinfo(), snippet() and offsets() auxillary
115098 ** functions.
115099 */
115100 SQLITE_PRIVATE int sqlite3Fts3ExprLoadDoclist(Fts3Cursor *pCsr, Fts3Expr *pExpr){
115101 int rc;
115102 assert( pExpr->eType==FTSQUERY_PHRASE && pExpr->pPhrase );
115103 assert( pCsr->eEvalmode==FTS3_EVAL_NEXT );
115104 rc = fts3EvalExpr(pCsr, pExpr, &pExpr->aDoclist, &pExpr->nDoclist, 1);
115105 return rc;
115106 }
115107
115108 SQLITE_PRIVATE int sqlite3Fts3ExprLoadFtDoclist(
115109 Fts3Cursor *pCsr,
115110 Fts3Expr *pExpr,
115111 char **paDoclist,
115112 int *pnDoclist
115113 ){
115114 int rc;
115115 assert( pCsr->eEvalmode==FTS3_EVAL_NEXT );
115116 assert( pExpr->eType==FTSQUERY_PHRASE && pExpr->pPhrase );
115117 pCsr->eEvalmode = FTS3_EVAL_MATCHINFO;
115118 rc = fts3EvalExpr(pCsr, pExpr, paDoclist, pnDoclist, 1);
115119 pCsr->eEvalmode = FTS3_EVAL_NEXT;
115120 return rc;
115121 }
115122
115123
115124 /*
115125 ** When called, *ppPoslist must point to the byte immediately following the
115126 ** end of a position-list. i.e. ( (*ppPoslist)[-1]==POS_END ). This function
115127 ** moves *ppPoslist so that it instead points to the first byte of the
115128 ** same position list.
115129 */
115130 static void fts3ReversePoslist(char *pStart, char **ppPoslist){
115131 char *p = &(*ppPoslist)[-3];
115132 char c = p[1];
 
 
115133 while( p>pStart && (*p & 0x80) | c ){
115134 c = *p--;
115135 }
115136 if( p>pStart ){ p = &p[2]; }
115137 while( *p++&0x80 );
115138 *ppPoslist = p;
115139 }
115140
115141
115142 /*
115143 ** After ExprLoadDoclist() (see above) has been called, this function is
115144 ** used to iterate/search through the position lists that make up the doclist
115145 ** stored in pExpr->aDoclist.
115146 */
115147 SQLITE_PRIVATE char *sqlite3Fts3FindPositions(
115148 Fts3Cursor *pCursor, /* Associate FTS3 cursor */
115149 Fts3Expr *pExpr, /* Access this expressions doclist */
115150 sqlite3_int64 iDocid, /* Docid associated with requested pos-list */
115151 int iCol /* Column of requested pos-list */
115152 ){
115153 assert( pExpr->isLoaded );
115154 if( pExpr->aDoclist ){
115155 char *pEnd = &pExpr->aDoclist[pExpr->nDoclist];
115156 char *pCsr;
115157
115158 if( pExpr->pCurrent==0 ){
115159 if( pCursor->desc==0 ){
115160 pExpr->pCurrent = pExpr->aDoclist;
115161 pExpr->iCurrent = 0;
115162 fts3GetDeltaVarint(&pExpr->pCurrent, &pExpr->iCurrent);
115163 }else{
115164 pCsr = pExpr->aDoclist;
115165 while( pCsr<pEnd ){
115166 fts3GetDeltaVarint(&pCsr, &pExpr->iCurrent);
115167 fts3PoslistCopy(0, &pCsr);
115168 }
115169 fts3ReversePoslist(pExpr->aDoclist, &pCsr);
115170 pExpr->pCurrent = pCsr;
115171 }
115172 }
115173 pCsr = pExpr->pCurrent;
115174 assert( pCsr );
115175
115176 while( (pCursor->desc==0 && pCsr<pEnd)
115177 || (pCursor->desc && pCsr>pExpr->aDoclist)
115178 ){
115179 if( pCursor->desc==0 && pExpr->iCurrent<iDocid ){
115180 fts3PoslistCopy(0, &pCsr);
115181 if( pCsr<pEnd ){
115182 fts3GetDeltaVarint(&pCsr, &pExpr->iCurrent);
115183 }
115184 pExpr->pCurrent = pCsr;
115185 }else if( pCursor->desc && pExpr->iCurrent>iDocid ){
115186 fts3GetReverseDeltaVarint(&pCsr, pExpr->aDoclist, &pExpr->iCurrent);
115187 fts3ReversePoslist(pExpr->aDoclist, &pCsr);
115188 pExpr->pCurrent = pCsr;
115189 }else{
115190 if( pExpr->iCurrent==iDocid ){
115191 int iThis = 0;
115192 if( iCol<0 ){
115193 /* If iCol is negative, return a pointer to the start of the
115194 ** position-list (instead of a pointer to the start of a list
115195 ** of offsets associated with a specific column).
115196 */
115197 return pCsr;
115198 }
115199 while( iThis<iCol ){
115200 fts3ColumnlistCopy(0, &pCsr);
115201 if( *pCsr==0x00 ) return 0;
115202 pCsr++;
115203 pCsr += sqlite3Fts3GetVarint32(pCsr, &iThis);
115204 }
115205 if( iCol==iThis && (*pCsr&0xFE) ) return pCsr;
115206 }
115207 return 0;
115208 }
115209 }
115210 }
115211
115212 return 0;
115213 }
115214
115215 /*
115216 ** Helper function used by the implementation of the overloaded snippet(),
115217 ** offsets() and optimize() SQL functions.
115218 **
115219 ** If the value passed as the third argument is a blob of size
@@ -115440,27 +115952,43 @@
115440 p->zDb, p->zName, zName
115441 );
115442 return rc;
115443 }
115444
 
 
 
 
 
115445 static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
115446 Fts3Table *p = (Fts3Table*)pVtab;
115447 UNUSED_PARAMETER(iSavepoint);
115448 assert( p->inTransaction );
115449 assert( p->mxSavepoint < iSavepoint );
115450 TESTONLY( p->mxSavepoint = iSavepoint );
115451 return sqlite3Fts3PendingTermsFlush(p);
115452 }
 
 
 
 
 
 
115453 static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
115454 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115455 UNUSED_PARAMETER(iSavepoint);
115456 UNUSED_PARAMETER(pVtab);
115457 assert( p->inTransaction );
115458 assert( p->mxSavepoint >= iSavepoint );
115459 TESTONLY( p->mxSavepoint = iSavepoint-1 );
115460 return SQLITE_OK;
115461 }
 
 
 
 
 
 
115462 static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
115463 Fts3Table *p = (Fts3Table*)pVtab;
115464 UNUSED_PARAMETER(iSavepoint);
115465 assert( p->inTransaction );
115466 assert( p->mxSavepoint >= iSavepoint );
@@ -115605,12 +116133,1553 @@
115605 sqlite3Fts3HashClear(pHash);
115606 sqlite3_free(pHash);
115607 }
115608 return rc;
115609 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115610
115611 #if !SQLITE_CORE
 
 
 
115612 SQLITE_API int sqlite3_extension_init(
115613 sqlite3 *db,
115614 char **pzErrMsg,
115615 const sqlite3_api_routines *pApi
115616 ){
@@ -115634,11 +117703,10 @@
115634 ** May you share freely, never taking more than you give.
115635 **
115636 ******************************************************************************
115637 **
115638 */
115639
115640 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
115641
115642
115643 typedef struct Fts3auxTable Fts3auxTable;
115644 typedef struct Fts3auxCursor Fts3auxCursor;
@@ -115648,11 +117716,11 @@
115648 Fts3Table *pFts3Tab;
115649 };
115650
115651 struct Fts3auxCursor {
115652 sqlite3_vtab_cursor base; /* Base class used by SQLite core */
115653 Fts3SegReaderCursor csr; /* Must be right after "base" */
115654 Fts3SegFilter filter;
115655 char *zStop;
115656 int nStop; /* Byte-length of string zStop */
115657 int isEof; /* True if cursor is at EOF */
115658 sqlite3_int64 iRowid; /* Current rowid */
@@ -115716,10 +117784,11 @@
115716
115717 p->pFts3Tab = (Fts3Table *)&p[1];
115718 p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1];
115719 p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1];
115720 p->pFts3Tab->db = db;
 
115721
115722 memcpy((char *)p->pFts3Tab->zDb, zDb, nDb);
115723 memcpy((char *)p->pFts3Tab->zName, zFts3, nFts3);
115724 sqlite3Fts3Dequote((char *)p->pFts3Tab->zName);
115725
@@ -115996,11 +118065,11 @@
115996 pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iIdx]));
115997 pCsr->nStop = sqlite3_value_bytes(apVal[iIdx]);
115998 if( pCsr->zStop==0 ) return SQLITE_NOMEM;
115999 }
116000
116001 rc = sqlite3Fts3SegReaderCursor(pFts3, FTS3_SEGCURSOR_ALL,
116002 pCsr->filter.zTerm, pCsr->filter.nTerm, 0, isScan, &pCsr->csr
116003 );
116004 if( rc==SQLITE_OK ){
116005 rc = sqlite3Fts3SegReaderStart(pFts3, &pCsr->csr, &pCsr->filter);
116006 }
@@ -116175,16 +118244,25 @@
116175 ** Default span for NEAR operators.
116176 */
116177 #define SQLITE_FTS3_DEFAULT_NEAR_PARAM 10
116178
116179
 
 
 
 
 
 
 
 
116180 typedef struct ParseContext ParseContext;
116181 struct ParseContext {
116182 sqlite3_tokenizer *pTokenizer; /* Tokenizer module */
116183 const char **azCol; /* Array of column names for fts3 table */
116184 int nCol; /* Number of entries in azCol[] */
116185 int iDefaultCol; /* Default column to query */
 
116186 sqlite3_context *pCtx; /* Write error message here */
116187 int nNest; /* Number of nested brackets */
116188 };
116189
116190 /*
@@ -116266,11 +118344,11 @@
116266 if( iEnd<n && z[iEnd]=='*' ){
116267 pRet->pPhrase->aToken[0].isPrefix = 1;
116268 iEnd++;
116269 }
116270 if( !sqlite3_fts3_enable_parentheses && iStart>0 && z[iStart-1]=='-' ){
116271 pRet->pPhrase->isNot = 1;
116272 }
116273 }
116274 nConsumed = iEnd;
116275 }
116276
@@ -116318,71 +118396,86 @@
116318 Fts3Expr *p = 0;
116319 sqlite3_tokenizer_cursor *pCursor = 0;
116320 char *zTemp = 0;
116321 int nTemp = 0;
116322
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116323 rc = pModule->xOpen(pTokenizer, zInput, nInput, &pCursor);
116324 if( rc==SQLITE_OK ){
116325 int ii;
116326 pCursor->pTokenizer = pTokenizer;
116327 for(ii=0; rc==SQLITE_OK; ii++){
116328 const char *zToken;
116329 int nToken, iBegin, iEnd, iPos;
116330 rc = pModule->xNext(pCursor, &zToken, &nToken, &iBegin, &iEnd, &iPos);
116331 if( rc==SQLITE_OK ){
116332 int nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase);
116333 p = fts3ReallocOrFree(p, nByte+ii*sizeof(Fts3PhraseToken));
116334 zTemp = fts3ReallocOrFree(zTemp, nTemp + nToken);
116335 if( !p || !zTemp ){
116336 goto no_mem;
116337 }
116338 if( ii==0 ){
116339 memset(p, 0, nByte);
116340 p->pPhrase = (Fts3Phrase *)&p[1];
116341 }
116342 p->pPhrase = (Fts3Phrase *)&p[1];
116343 memset(&p->pPhrase->aToken[ii], 0, sizeof(Fts3PhraseToken));
116344 p->pPhrase->nToken = ii+1;
116345 p->pPhrase->aToken[ii].n = nToken;
116346 memcpy(&zTemp[nTemp], zToken, nToken);
116347 nTemp += nToken;
116348 if( iEnd<nInput && zInput[iEnd]=='*' ){
116349 p->pPhrase->aToken[ii].isPrefix = 1;
116350 }else{
116351 p->pPhrase->aToken[ii].isPrefix = 0;
116352 }
116353 }
116354 }
116355
116356 pModule->xClose(pCursor);
116357 pCursor = 0;
116358 }
116359
116360 if( rc==SQLITE_DONE ){
116361 int jj;
116362 char *zNew = NULL;
116363 int nNew = 0;
116364 int nByte = sizeof(Fts3Expr) + sizeof(Fts3Phrase);
116365 nByte += (p?(p->pPhrase->nToken-1):0) * sizeof(Fts3PhraseToken);
116366 p = fts3ReallocOrFree(p, nByte + nTemp);
116367 if( !p ){
116368 goto no_mem;
116369 }
116370 if( zTemp ){
116371 zNew = &(((char *)p)[nByte]);
116372 memcpy(zNew, zTemp, nTemp);
116373 }else{
116374 memset(p, 0, nByte+nTemp);
116375 }
116376 p->pPhrase = (Fts3Phrase *)&p[1];
 
 
 
 
 
 
 
116377 for(jj=0; jj<p->pPhrase->nToken; jj++){
116378 p->pPhrase->aToken[jj].z = &zNew[nNew];
116379 nNew += p->pPhrase->aToken[jj].n;
116380 }
116381 sqlite3_free(zTemp);
116382 p->eType = FTSQUERY_PHRASE;
116383 p->pPhrase->iColumn = pParse->iDefaultCol;
116384 rc = SQLITE_OK;
116385 }
116386
116387 *ppExpr = p;
116388 return rc;
@@ -116434,10 +118527,12 @@
116434 int rc;
116435 Fts3Expr *pRet = 0;
116436
116437 const char *zInput = z;
116438 int nInput = n;
 
 
116439
116440 /* Skip over any whitespace before checking for a keyword, an open or
116441 ** close bracket, or a quoted string.
116442 */
116443 while( nInput>0 && fts3isspace(*zInput) ){
@@ -116653,11 +118748,11 @@
116653 rc = getNextNode(pParse, zIn, nIn, &p, &nByte);
116654 if( rc==SQLITE_OK ){
116655 int isPhrase;
116656
116657 if( !sqlite3_fts3_enable_parentheses
116658 && p->eType==FTSQUERY_PHRASE && p->pPhrase->isNot
116659 ){
116660 /* Create an implicit NOT operator. */
116661 Fts3Expr *pNot = fts3MallocZero(sizeof(Fts3Expr));
116662 if( !pNot ){
116663 sqlite3Fts3ExprFree(p);
@@ -116671,11 +118766,10 @@
116671 }
116672 pNotBranch = pNot;
116673 p = pPrev;
116674 }else{
116675 int eType = p->eType;
116676 assert( eType!=FTSQUERY_PHRASE || !p->pPhrase->isNot );
116677 isPhrase = (eType==FTSQUERY_PHRASE || p->pLeft);
116678
116679 /* The isRequirePhrase variable is set to true if a phrase or
116680 ** an expression contained in parenthesis is required. If a
116681 ** binary operator (AND, OR, NOT or NEAR) is encounted when
@@ -116834,13 +118928,15 @@
116834 /*
116835 ** Free a parsed fts3 query expression allocated by sqlite3Fts3ExprParse().
116836 */
116837 SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *p){
116838 if( p ){
 
116839 sqlite3Fts3ExprFree(p->pLeft);
116840 sqlite3Fts3ExprFree(p->pRight);
116841 sqlite3_free(p->aDoclist);
 
116842 sqlite3_free(p);
116843 }
116844 }
116845
116846 /****************************************************************************
@@ -116893,11 +118989,11 @@
116893 switch( pExpr->eType ){
116894 case FTSQUERY_PHRASE: {
116895 Fts3Phrase *pPhrase = pExpr->pPhrase;
116896 int i;
116897 zBuf = sqlite3_mprintf(
116898 "%zPHRASE %d %d", zBuf, pPhrase->iColumn, pPhrase->isNot);
116899 for(i=0; zBuf && i<pPhrase->nToken; i++){
116900 zBuf = sqlite3_mprintf("%z %.*s%s", zBuf,
116901 pPhrase->aToken[i].n, pPhrase->aToken[i].z,
116902 (pPhrase->aToken[i].isPrefix?"+":"")
116903 );
@@ -117437,11 +119533,10 @@
117437 **
117438 ** * The FTS3 module is being built into the core of
117439 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
117440 */
117441 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
117442
117443
117444
117445
117446 /*
117447 ** Class derived from sqlite3_tokenizer
@@ -118082,14 +120177,10 @@
118082 ** * The FTS3 module is being built into the core of
118083 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
118084 */
118085 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
118086
118087 #ifndef SQLITE_CORE
118088 SQLITE_EXTENSION_INIT1
118089 #endif
118090
118091
118092 /*
118093 ** Implementation of the SQL scalar function for accessing the underlying
118094 ** hash table. This function may be called as follows:
118095 **
@@ -118209,11 +120300,11 @@
118209 sqlite3_tokenizer **ppTok, /* OUT: Tokenizer (if applicable) */
118210 char **pzErr /* OUT: Set to malloced error message */
118211 ){
118212 int rc;
118213 char *z = (char *)zArg;
118214 int n;
118215 char *zCopy;
118216 char *zEnd; /* Pointer to nul-term of zCopy */
118217 sqlite3_tokenizer_module *m;
118218
118219 zCopy = sqlite3_mprintf("%s", zArg);
@@ -118574,11 +120665,10 @@
118574 */
118575 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
118576
118577
118578
118579
118580 typedef struct simple_tokenizer {
118581 sqlite3_tokenizer base;
118582 char delim[128]; /* flag ASCII delimiters */
118583 } simple_tokenizer;
118584
@@ -118811,18 +120901,44 @@
118811 ** it is always safe to read up to two varints from it without risking an
118812 ** overread, even if the node data is corrupted.
118813 */
118814 #define FTS3_NODE_PADDING (FTS3_VARINT_MAX*2)
118815
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
118816 typedef struct PendingList PendingList;
118817 typedef struct SegmentNode SegmentNode;
118818 typedef struct SegmentWriter SegmentWriter;
118819
118820 /*
118821 ** Data structure used while accumulating terms in the pending-terms hash
118822 ** table. The hash table entry maps from term (a string) to a malloc'd
118823 ** instance of this structure.
118824 */
118825 struct PendingList {
118826 int nData;
118827 char *aData;
118828 int nSpace;
@@ -118849,11 +120965,10 @@
118849 ** of type Fts3SegReader* are also used by code in fts3.c to iterate through
118850 ** terms when querying the full-text index. See functions:
118851 **
118852 ** sqlite3Fts3SegReaderNew()
118853 ** sqlite3Fts3SegReaderFree()
118854 ** sqlite3Fts3SegReaderCost()
118855 ** sqlite3Fts3SegReaderIterate()
118856 **
118857 ** Methods used to manipulate Fts3SegReader structures:
118858 **
118859 ** fts3SegReaderNext()
@@ -118868,10 +120983,13 @@
118868 sqlite3_int64 iEndBlock; /* Rowid of final block in segment (or 0) */
118869 sqlite3_int64 iCurrentBlock; /* Current leaf block (or 0) */
118870
118871 char *aNode; /* Pointer to node data (or NULL) */
118872 int nNode; /* Size of buffer at aNode (or 0) */
 
 
 
118873 Fts3HashElem **ppNextElem;
118874
118875 /* Variables set by fts3SegReaderNext(). These may be read directly
118876 ** by the caller. They are valid from the time SegmentReaderNew() returns
118877 ** until SegmentReaderNext() returns something other than SQLITE_OK
@@ -118881,12 +120999,15 @@
118881 char *zTerm; /* Pointer to current term */
118882 int nTermAlloc; /* Allocated size of zTerm buffer */
118883 char *aDoclist; /* Pointer to doclist of current entry */
118884 int nDoclist; /* Size of doclist in current entry */
118885
118886 /* The following variables are used to iterate through the current doclist */
 
 
118887 char *pOffsetList;
 
118888 sqlite3_int64 iDocid;
118889 };
118890
118891 #define fts3SegReaderIsPending(p) ((p)->ppNextElem!=0)
118892 #define fts3SegReaderIsRootOnly(p) ((p)->aNode==(char *)&(p)[1])
@@ -118920,10 +121041,18 @@
118920 ** within the fts3SegWriterXXX() family of functions described above.
118921 **
118922 ** fts3NodeAddTerm()
118923 ** fts3NodeWrite()
118924 ** fts3NodeFree()
 
 
 
 
 
 
 
 
118925 */
118926 struct SegmentNode {
118927 SegmentNode *pParent; /* Parent node (or NULL for root node) */
118928 SegmentNode *pRight; /* Pointer to right-sibling */
118929 SegmentNode *pLeftmost; /* Pointer to left-most node of this depth */
@@ -118950,21 +121079,26 @@
118950 #define SQL_NEXT_SEGMENT_INDEX 8
118951 #define SQL_INSERT_SEGMENTS 9
118952 #define SQL_NEXT_SEGMENTS_ID 10
118953 #define SQL_INSERT_SEGDIR 11
118954 #define SQL_SELECT_LEVEL 12
118955 #define SQL_SELECT_ALL_LEVEL 13
118956 #define SQL_SELECT_LEVEL_COUNT 14
118957 #define SQL_SELECT_SEGDIR_COUNT_MAX 15
118958 #define SQL_DELETE_SEGDIR_BY_LEVEL 16
118959 #define SQL_DELETE_SEGMENTS_RANGE 17
118960 #define SQL_CONTENT_INSERT 18
118961 #define SQL_DELETE_DOCSIZE 19
118962 #define SQL_REPLACE_DOCSIZE 20
118963 #define SQL_SELECT_DOCSIZE 21
118964 #define SQL_SELECT_DOCTOTAL 22
118965 #define SQL_REPLACE_DOCTOTAL 23
 
 
 
 
 
118966
118967 /*
118968 ** This function is used to obtain an SQLite prepared statement handle
118969 ** for the statement identified by the second argument. If successful,
118970 ** *pp is set to the requested statement handle and SQLITE_OK returned.
@@ -118997,23 +121131,29 @@
118997
118998 /* Return segments in order from oldest to newest.*/
118999 /* 12 */ "SELECT idx, start_block, leaves_end_block, end_block, root "
119000 "FROM %Q.'%q_segdir' WHERE level = ? ORDER BY idx ASC",
119001 /* 13 */ "SELECT idx, start_block, leaves_end_block, end_block, root "
119002 "FROM %Q.'%q_segdir' ORDER BY level DESC, idx ASC",
 
119003
119004 /* 14 */ "SELECT count(*) FROM %Q.'%q_segdir' WHERE level = ?",
119005 /* 15 */ "SELECT count(*), max(level) FROM %Q.'%q_segdir'",
119006
119007 /* 16 */ "DELETE FROM %Q.'%q_segdir' WHERE level = ?",
119008 /* 17 */ "DELETE FROM %Q.'%q_segments' WHERE blockid BETWEEN ? AND ?",
119009 /* 18 */ "INSERT INTO %Q.'%q_content' VALUES(%s)",
119010 /* 19 */ "DELETE FROM %Q.'%q_docsize' WHERE docid = ?",
119011 /* 20 */ "REPLACE INTO %Q.'%q_docsize' VALUES(?,?)",
119012 /* 21 */ "SELECT size FROM %Q.'%q_docsize' WHERE docid=?",
119013 /* 22 */ "SELECT value FROM %Q.'%q_stat' WHERE id=0",
119014 /* 23 */ "REPLACE INTO %Q.'%q_stat' VALUES(0,?)",
 
 
 
 
 
119015 };
119016 int rc = SQLITE_OK;
119017 sqlite3_stmt *pStmt;
119018
119019 assert( SizeofArray(azSql)==SizeofArray(p->aStmt) );
@@ -119165,18 +121305,36 @@
119165 ** 1: start_block
119166 ** 2: leaves_end_block
119167 ** 3: end_block
119168 ** 4: root
119169 */
119170 SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table *p, int iLevel, sqlite3_stmt **ppStmt){
 
 
 
 
 
119171 int rc;
119172 sqlite3_stmt *pStmt = 0;
 
 
 
 
 
119173 if( iLevel<0 ){
119174 rc = fts3SqlStmt(p, SQL_SELECT_ALL_LEVEL, &pStmt, 0);
 
 
 
 
 
119175 }else{
 
119176 rc = fts3SqlStmt(p, SQL_SELECT_LEVEL, &pStmt, 0);
119177 if( rc==SQLITE_OK ) sqlite3_bind_int(pStmt, 1, iLevel);
 
 
119178 }
119179 *ppStmt = pStmt;
119180 return rc;
119181 }
119182
@@ -119286,10 +121444,51 @@
119286 *pp = p;
119287 return 1;
119288 }
119289 return 0;
119290 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119291
119292 /*
119293 ** Tokenize the nul-terminated string zText and add all tokens to the
119294 ** pending-terms hash-table. The docid used is that currently stored in
119295 ** p->iPrevDocid, and the column is specified by argument iCol.
@@ -119335,12 +121534,11 @@
119335
119336 xNext = pModule->xNext;
119337 while( SQLITE_OK==rc
119338 && SQLITE_OK==(rc = xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos))
119339 ){
119340 PendingList *pList;
119341
119342 if( iPos>=nWord ) nWord = iPos+1;
119343
119344 /* Positions cannot be negative; we use -1 as a terminator internally.
119345 ** Tokens must have a non-zero length.
119346 */
@@ -119347,26 +121545,23 @@
119347 if( iPos<0 || !zToken || nToken<=0 ){
119348 rc = SQLITE_ERROR;
119349 break;
119350 }
119351
119352 pList = (PendingList *)fts3HashFind(&p->pendingTerms, zToken, nToken);
119353 if( pList ){
119354 p->nPendingData -= (pList->nData + nToken + sizeof(Fts3HashElem));
119355 }
119356 if( fts3PendingListAppend(&pList, p->iPrevDocid, iCol, iPos, &rc) ){
119357 if( pList==fts3HashInsert(&p->pendingTerms, zToken, nToken, pList) ){
119358 /* Malloc failed while inserting the new entry. This can only
119359 ** happen if there was no previous entry for this token.
119360 */
119361 assert( 0==fts3HashFind(&p->pendingTerms, zToken, nToken) );
119362 sqlite3_free(pList);
119363 rc = SQLITE_NOMEM;
119364 }
119365 }
119366 if( rc==SQLITE_OK ){
119367 p->nPendingData += (pList->nData + nToken + sizeof(Fts3HashElem));
119368 }
119369 }
119370
119371 pModule->xClose(pCsr);
119372 *pnWord = nWord;
@@ -119392,18 +121587,23 @@
119392 p->iPrevDocid = iDocid;
119393 return SQLITE_OK;
119394 }
119395
119396 /*
119397 ** Discard the contents of the pending-terms hash table.
119398 */
119399 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *p){
119400 Fts3HashElem *pElem;
119401 for(pElem=fts3HashFirst(&p->pendingTerms); pElem; pElem=fts3HashNext(pElem)){
119402 sqlite3_free(fts3HashData(pElem));
 
 
 
 
 
 
119403 }
119404 fts3HashClear(&p->pendingTerms);
119405 p->nPendingData = 0;
119406 }
119407
119408 /*
119409 ** This function is called by the xUpdate() method as part of an INSERT
@@ -119555,11 +121755,11 @@
119555
119556 /*
119557 ** Forward declaration to account for the circular dependency between
119558 ** functions fts3SegmentMerge() and fts3AllocateSegdirIdx().
119559 */
119560 static int fts3SegmentMerge(Fts3Table *, int);
119561
119562 /*
119563 ** This function allocates a new level iLevel index in the segdir table.
119564 ** Usually, indexes are allocated within a level sequentially starting
119565 ** with 0, so the allocated index is one greater than the value returned
@@ -119572,19 +121772,24 @@
119572 ** allocated index is 0.
119573 **
119574 ** If successful, *piIdx is set to the allocated index slot and SQLITE_OK
119575 ** returned. Otherwise, an SQLite error code is returned.
119576 */
119577 static int fts3AllocateSegdirIdx(Fts3Table *p, int iLevel, int *piIdx){
 
 
 
 
 
119578 int rc; /* Return Code */
119579 sqlite3_stmt *pNextIdx; /* Query for next idx at level iLevel */
119580 int iNext = 0; /* Result of query pNextIdx */
119581
119582 /* Set variable iNext to the next available segdir index at level iLevel. */
119583 rc = fts3SqlStmt(p, SQL_NEXT_SEGMENT_INDEX, &pNextIdx, 0);
119584 if( rc==SQLITE_OK ){
119585 sqlite3_bind_int(pNextIdx, 1, iLevel);
119586 if( SQLITE_ROW==sqlite3_step(pNextIdx) ){
119587 iNext = sqlite3_column_int(pNextIdx, 0);
119588 }
119589 rc = sqlite3_reset(pNextIdx);
119590 }
@@ -119594,11 +121799,11 @@
119594 ** full, merge all segments in level iLevel into a single iLevel+1
119595 ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise,
119596 ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext.
119597 */
119598 if( iNext>=FTS3_MERGE_COUNT ){
119599 rc = fts3SegmentMerge(p, iLevel);
119600 *piIdx = 0;
119601 }else{
119602 *piIdx = iNext;
119603 }
119604 }
@@ -119635,11 +121840,12 @@
119635 */
119636 SQLITE_PRIVATE int sqlite3Fts3ReadBlock(
119637 Fts3Table *p, /* FTS3 table handle */
119638 sqlite3_int64 iBlockid, /* Access the row with blockid=$iBlockid */
119639 char **paBlob, /* OUT: Blob data in malloc'd buffer */
119640 int *pnBlob /* OUT: Size of blob data */
 
119641 ){
119642 int rc; /* Return code */
119643
119644 /* pnBlob must be non-NULL. paBlob may be NULL or non-NULL. */
119645 assert( pnBlob);
@@ -119656,25 +121862,29 @@
119656 );
119657 }
119658
119659 if( rc==SQLITE_OK ){
119660 int nByte = sqlite3_blob_bytes(p->pSegments);
 
119661 if( paBlob ){
119662 char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING);
119663 if( !aByte ){
119664 rc = SQLITE_NOMEM;
119665 }else{
 
 
 
 
119666 rc = sqlite3_blob_read(p->pSegments, aByte, nByte, 0);
119667 memset(&aByte[nByte], 0, FTS3_NODE_PADDING);
119668 if( rc!=SQLITE_OK ){
119669 sqlite3_free(aByte);
119670 aByte = 0;
119671 }
119672 }
119673 *paBlob = aByte;
119674 }
119675 *pnBlob = nByte;
119676 }
119677
119678 return rc;
119679 }
119680
@@ -119684,17 +121894,59 @@
119684 */
119685 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *p){
119686 sqlite3_blob_close(p->pSegments);
119687 p->pSegments = 0;
119688 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119689
119690 /*
119691 ** Move the iterator passed as the first argument to the next term in the
119692 ** segment. If successful, SQLITE_OK is returned. If there is no next term,
119693 ** SQLITE_DONE. Otherwise, an SQLite error code.
119694 */
119695 static int fts3SegReaderNext(Fts3Table *p, Fts3SegReader *pReader){
 
 
 
 
 
119696 char *pNext; /* Cursor variable */
119697 int nPrefix; /* Number of bytes in term prefix */
119698 int nSuffix; /* Number of bytes in term suffix */
119699
119700 if( !pReader->aDoclist ){
@@ -119702,11 +121954,10 @@
119702 }else{
119703 pNext = &pReader->aDoclist[pReader->nDoclist];
119704 }
119705
119706 if( !pNext || pNext>=&pReader->aNode[pReader->nNode] ){
119707 int rc; /* Return code from Fts3ReadBlock() */
119708
119709 if( fts3SegReaderIsPending(pReader) ){
119710 Fts3HashElem *pElem = *(pReader->ppNextElem);
119711 if( pElem==0 ){
119712 pReader->aNode = 0;
@@ -119722,10 +121973,12 @@
119722 return SQLITE_OK;
119723 }
119724
119725 if( !fts3SegReaderIsRootOnly(pReader) ){
119726 sqlite3_free(pReader->aNode);
 
 
119727 }
119728 pReader->aNode = 0;
119729
119730 /* If iCurrentBlock>=iLeafEndBlock, this is an EOF condition. All leaf
119731 ** blocks have already been traversed. */
@@ -119733,19 +121986,29 @@
119733 if( pReader->iCurrentBlock>=pReader->iLeafEndBlock ){
119734 return SQLITE_OK;
119735 }
119736
119737 rc = sqlite3Fts3ReadBlock(
119738 p, ++pReader->iCurrentBlock, &pReader->aNode, &pReader->nNode
 
119739 );
119740 if( rc!=SQLITE_OK ) return rc;
 
 
 
 
 
119741 pNext = pReader->aNode;
119742 }
 
 
 
 
 
119743
119744 /* Because of the FTS3_NODE_PADDING bytes of padding, the following is
119745 ** safe (no risk of overread) even if the node data is corrupted.
119746 */
119747 pNext += sqlite3Fts3GetVarint32(pNext, &nPrefix);
119748 pNext += sqlite3Fts3GetVarint32(pNext, &nSuffix);
119749 if( nPrefix<0 || nSuffix<=0
119750 || &pNext[nSuffix]>&pReader->aNode[pReader->nNode]
119751 ){
@@ -119759,10 +122022,14 @@
119759 return SQLITE_NOMEM;
119760 }
119761 pReader->zTerm = zNew;
119762 pReader->nTermAlloc = nNew;
119763 }
 
 
 
 
119764 memcpy(&pReader->zTerm[nPrefix], pNext, nSuffix);
119765 pReader->nTerm = nPrefix+nSuffix;
119766 pNext += nSuffix;
119767 pNext += sqlite3Fts3GetVarint32(pNext, &pReader->nDoclist);
119768 pReader->aDoclist = pNext;
@@ -119771,11 +122038,11 @@
119771 /* Check that the doclist does not appear to extend past the end of the
119772 ** b-tree node. And that the final byte of the doclist is 0x00. If either
119773 ** of these statements is untrue, then the data structure is corrupt.
119774 */
119775 if( &pReader->aDoclist[pReader->nDoclist]>&pReader->aNode[pReader->nNode]
119776 || pReader->aDoclist[pReader->nDoclist-1]
119777 ){
119778 return SQLITE_CORRUPT_VTAB;
119779 }
119780 return SQLITE_OK;
119781 }
@@ -119782,16 +122049,30 @@
119782
119783 /*
119784 ** Set the SegReader to point to the first docid in the doclist associated
119785 ** with the current term.
119786 */
119787 static void fts3SegReaderFirstDocid(Fts3SegReader *pReader){
119788 int n;
119789 assert( pReader->aDoclist );
119790 assert( !pReader->pOffsetList );
119791 n = sqlite3Fts3GetVarint(pReader->aDoclist, &pReader->iDocid);
119792 pReader->pOffsetList = &pReader->aDoclist[n];
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119793 }
119794
119795 /*
119796 ** Advance the SegReader to point to the next docid in the doclist
119797 ** associated with the current term.
@@ -119800,132 +122081,129 @@
119800 ** *ppOffsetList is set to point to the first column-offset list
119801 ** in the doclist entry (i.e. immediately past the docid varint).
119802 ** *pnOffsetList is set to the length of the set of column-offset
119803 ** lists, not including the nul-terminator byte. For example:
119804 */
119805 static void fts3SegReaderNextDocid(
119806 Fts3SegReader *pReader,
119807 char **ppOffsetList,
119808 int *pnOffsetList
 
119809 ){
 
119810 char *p = pReader->pOffsetList;
119811 char c = 0;
119812
119813 /* Pointer p currently points at the first byte of an offset list. The
119814 ** following two lines advance it to point one byte past the end of
119815 ** the same offset list.
119816 */
119817 while( *p | c ) c = *p++ & 0x80;
119818 p++;
119819
119820 /* If required, populate the output variables with a pointer to and the
119821 ** size of the previous offset-list.
119822 */
119823 if( ppOffsetList ){
119824 *ppOffsetList = pReader->pOffsetList;
119825 *pnOffsetList = (int)(p - pReader->pOffsetList - 1);
119826 }
119827
119828 /* If there are no more entries in the doclist, set pOffsetList to
119829 ** NULL. Otherwise, set Fts3SegReader.iDocid to the next docid and
119830 ** Fts3SegReader.pOffsetList to point to the next offset list before
119831 ** returning.
119832 */
119833 if( p>=&pReader->aDoclist[pReader->nDoclist] ){
119834 pReader->pOffsetList = 0;
119835 }else{
119836 sqlite3_int64 iDelta;
119837 pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta);
119838 pReader->iDocid += iDelta;
119839 }
119840 }
119841
119842 /*
119843 ** This function is called to estimate the amount of data that will be
119844 ** loaded from the disk If SegReaderIterate() is called on this seg-reader,
119845 ** in units of average document size.
119846 **
119847 ** This can be used as follows: If the caller has a small doclist that
119848 ** contains references to N documents, and is considering merging it with
119849 ** a large doclist (size X "average documents"), it may opt not to load
119850 ** the large doclist if X>N.
119851 */
119852 SQLITE_PRIVATE int sqlite3Fts3SegReaderCost(
119853 Fts3Cursor *pCsr, /* FTS3 cursor handle */
119854 Fts3SegReader *pReader, /* Segment-reader handle */
119855 int *pnCost /* IN/OUT: Number of bytes read */
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119856 ){
119857 Fts3Table *p = (Fts3Table*)pCsr->base.pVtab;
119858 int rc = SQLITE_OK; /* Return code */
119859 int nCost = 0; /* Cost in bytes to return */
119860 int pgsz = p->nPgsz; /* Database page size */
119861
119862 /* If this seg-reader is reading the pending-terms table, or if all data
119863 ** for the segment is stored on the root page of the b-tree, then the cost
119864 ** is zero. In this case all required data is already in main memory.
119865 */
119866 if( p->bHasStat
119867 && !fts3SegReaderIsPending(pReader)
119868 && !fts3SegReaderIsRootOnly(pReader)
119869 ){
119870 int nBlob = 0;
119871 sqlite3_int64 iBlock;
119872
119873 if( pCsr->nRowAvg==0 ){
119874 /* The average document size, which is required to calculate the cost
119875 ** of each doclist, has not yet been determined. Read the required
119876 ** data from the %_stat table to calculate it.
119877 **
119878 ** Entry 0 of the %_stat table is a blob containing (nCol+1) FTS3
119879 ** varints, where nCol is the number of columns in the FTS3 table.
119880 ** The first varint is the number of documents currently stored in
119881 ** the table. The following nCol varints contain the total amount of
119882 ** data stored in all rows of each column of the table, from left
119883 ** to right.
119884 */
119885 sqlite3_stmt *pStmt;
119886 sqlite3_int64 nDoc = 0;
119887 sqlite3_int64 nByte = 0;
119888 const char *pEnd;
119889 const char *a;
119890
119891 rc = sqlite3Fts3SelectDoctotal(p, &pStmt);
119892 if( rc!=SQLITE_OK ) return rc;
119893 a = sqlite3_column_blob(pStmt, 0);
119894 assert( a );
119895
119896 pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
119897 a += sqlite3Fts3GetVarint(a, &nDoc);
119898 while( a<pEnd ){
119899 a += sqlite3Fts3GetVarint(a, &nByte);
119900 }
119901 if( nDoc==0 || nByte==0 ){
119902 sqlite3_reset(pStmt);
119903 return SQLITE_CORRUPT_VTAB;
119904 }
119905
119906 pCsr->nRowAvg = (int)(((nByte / nDoc) + pgsz) / pgsz);
119907 assert( pCsr->nRowAvg>0 );
119908 rc = sqlite3_reset(pStmt);
119909 if( rc!=SQLITE_OK ) return rc;
119910 }
119911
119912 /* Assume that a blob flows over onto overflow pages if it is larger
119913 ** than (pgsz-35) bytes in size (the file-format documentation
119914 ** confirms this).
119915 */
119916 for(iBlock=pReader->iStartBlock; iBlock<=pReader->iLeafEndBlock; iBlock++){
119917 rc = sqlite3Fts3ReadBlock(p, iBlock, 0, &nBlob);
119918 if( rc!=SQLITE_OK ) break;
119919 if( (nBlob+35)>pgsz ){
119920 int nOvfl = (nBlob + 34)/pgsz;
119921 nCost += ((nOvfl + pCsr->nRowAvg - 1)/pCsr->nRowAvg);
119922 }
119923 }
119924 }
119925
119926 *pnCost += nCost;
119927 return rc;
119928 }
119929
119930 /*
119931 ** Free all allocations associated with the iterator passed as the
@@ -119934,10 +122212,11 @@
119934 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){
119935 if( pReader && !fts3SegReaderIsPending(pReader) ){
119936 sqlite3_free(pReader->zTerm);
119937 if( !fts3SegReaderIsRootOnly(pReader) ){
119938 sqlite3_free(pReader->aNode);
 
119939 }
119940 }
119941 sqlite3_free(pReader);
119942 }
119943
@@ -120010,28 +122289,46 @@
120010 }
120011
120012 /*
120013 ** This function is used to allocate an Fts3SegReader that iterates through
120014 ** a subset of the terms stored in the Fts3Table.pendingTerms array.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
120015 */
120016 SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(
120017 Fts3Table *p, /* Virtual table handle */
 
120018 const char *zTerm, /* Term to search for */
120019 int nTerm, /* Size of buffer zTerm */
120020 int isPrefix, /* True for a term-prefix query */
120021 Fts3SegReader **ppReader /* OUT: SegReader for pending-terms */
120022 ){
120023 Fts3SegReader *pReader = 0; /* Fts3SegReader object to return */
120024 Fts3HashElem **aElem = 0; /* Array of term hash entries to scan */
120025 int nElem = 0; /* Size of array at aElem */
120026 int rc = SQLITE_OK; /* Return Code */
 
120027
120028 if( isPrefix ){
 
120029 int nAlloc = 0; /* Size of allocated array at aElem */
120030 Fts3HashElem *pE = 0; /* Iterator variable */
120031
120032 for(pE=fts3HashFirst(&p->pendingTerms); pE; pE=fts3HashNext(pE)){
120033 char *zKey = (char *)fts3HashKey(pE);
120034 int nKey = fts3HashKeysize(pE);
120035 if( nTerm==0 || (nKey>=nTerm && 0==memcmp(zKey, zTerm, nTerm)) ){
120036 if( nElem==nAlloc ){
120037 Fts3HashElem **aElem2;
@@ -120044,10 +122341,11 @@
120044 nElem = 0;
120045 break;
120046 }
120047 aElem = aElem2;
120048 }
 
120049 aElem[nElem++] = pE;
120050 }
120051 }
120052
120053 /* If more than one term matches the prefix, sort the Fts3HashElem
@@ -120057,11 +122355,13 @@
120057 if( nElem>1 ){
120058 qsort(aElem, nElem, sizeof(Fts3HashElem *), fts3CompareElemByTerm);
120059 }
120060
120061 }else{
120062 Fts3HashElem *pE = fts3HashFindElem(&p->pendingTerms, zTerm, nTerm);
 
 
120063 if( pE ){
120064 aElem = &pE;
120065 nElem = 1;
120066 }
120067 }
@@ -120077,11 +122377,11 @@
120077 pReader->ppNextElem = (Fts3HashElem **)&pReader[1];
120078 memcpy(pReader->ppNextElem, aElem, nElem*sizeof(Fts3HashElem *));
120079 }
120080 }
120081
120082 if( isPrefix ){
120083 sqlite3_free(aElem);
120084 }
120085 *ppReader = pReader;
120086 return rc;
120087 }
@@ -120137,10 +122437,22 @@
120137 if( pLhs->iDocid==pRhs->iDocid ){
120138 rc = pRhs->iIdx - pLhs->iIdx;
120139 }else{
120140 rc = (pLhs->iDocid > pRhs->iDocid) ? 1 : -1;
120141 }
 
 
 
 
 
 
 
 
 
 
 
 
120142 }
120143 assert( pLhs->aNode && pRhs->aNode );
120144 return rc;
120145 }
120146
@@ -120689,25 +123001,34 @@
120689 }
120690 return rc;
120691 }
120692
120693 /*
120694 ** Set *pnSegment to the total number of segments in the database. Set
120695 ** *pnMax to the largest segment level in the database (segment levels
120696 ** are stored in the 'level' column of the %_segdir table).
 
120697 **
120698 ** Return SQLITE_OK if successful, or an SQLite error code if not.
120699 */
120700 static int fts3SegmentCountMax(Fts3Table *p, int *pnSegment, int *pnMax){
120701 sqlite3_stmt *pStmt;
120702 int rc;
 
120703
120704 rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_COUNT_MAX, &pStmt, 0);
 
 
 
 
 
 
120705 if( rc!=SQLITE_OK ) return rc;
 
 
120706 if( SQLITE_ROW==sqlite3_step(pStmt) ){
120707 *pnSegment = sqlite3_column_int(pStmt, 0);
120708 *pnMax = sqlite3_column_int(pStmt, 1);
120709 }
120710 return sqlite3_reset(pStmt);
120711 }
120712
120713 /*
@@ -120724,10 +123045,11 @@
120724 **
120725 ** SQLITE_OK is returned if successful, otherwise an SQLite error code.
120726 */
120727 static int fts3DeleteSegdir(
120728 Fts3Table *p, /* Virtual table handle */
 
120729 int iLevel, /* Level of %_segdir entries to delete */
120730 Fts3SegReader **apSegment, /* Array of SegReader objects */
120731 int nReader /* Size of array apSegment */
120732 ){
120733 int rc; /* Return Code */
@@ -120746,23 +123068,28 @@
120746 }
120747 if( rc!=SQLITE_OK ){
120748 return rc;
120749 }
120750
 
120751 if( iLevel==FTS3_SEGCURSOR_ALL ){
120752 fts3SqlExec(&rc, p, SQL_DELETE_ALL_SEGDIR, 0);
120753 }else if( iLevel==FTS3_SEGCURSOR_PENDING ){
120754 sqlite3Fts3PendingTermsClear(p);
 
 
120755 }else{
120756 assert( iLevel>=0 );
120757 rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_BY_LEVEL, &pDelete, 0);
120758 if( rc==SQLITE_OK ){
120759 sqlite3_bind_int(pDelete, 1, iLevel);
120760 sqlite3_step(pDelete);
120761 rc = sqlite3_reset(pDelete);
120762 }
120763 }
 
 
 
 
 
120764
120765 return rc;
120766 }
120767
120768 /*
@@ -120806,44 +123133,216 @@
120806
120807 *ppList = pList;
120808 *pnList = nList;
120809 }
120810
120811 SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
120812 Fts3Table *p, /* Virtual table handle */
120813 Fts3SegReaderCursor *pCsr, /* Cursor object */
120814 Fts3SegFilter *pFilter /* Restrictions on range of iteration */
 
120815 ){
120816 int i;
120817
120818 /* Initialize the cursor object */
120819 pCsr->pFilter = pFilter;
120820
120821 /* If the Fts3SegFilter defines a specific term (or term prefix) to search
120822 ** for, then advance each segment iterator until it points to a term of
120823 ** equal or greater value than the specified term. This prevents many
120824 ** unnecessary merge/sort operations for the case where single segment
120825 ** b-tree leaf nodes contain more than one term.
120826 */
120827 for(i=0; i<pCsr->nSegment; i++){
120828 int nTerm = pFilter->nTerm;
120829 const char *zTerm = pFilter->zTerm;
120830 Fts3SegReader *pSeg = pCsr->apSegment[i];
120831 do {
120832 int rc = fts3SegReaderNext(p, pSeg);
120833 if( rc!=SQLITE_OK ) return rc;
120834 }while( zTerm && fts3SegReaderTermCmp(pSeg, zTerm, nTerm)<0 );
120835 }
120836 fts3SegReaderSort(
120837 pCsr->apSegment, pCsr->nSegment, pCsr->nSegment, fts3SegReaderCmp);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
120838
120839 return SQLITE_OK;
120840 }
 
120841
120842 SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(
120843 Fts3Table *p, /* Virtual table handle */
120844 Fts3SegReaderCursor *pCsr /* Cursor object */
120845 ){
120846 int rc = SQLITE_OK;
120847
120848 int isIgnoreEmpty = (pCsr->pFilter->flags & FTS3_SEGMENT_IGNORE_EMPTY);
120849 int isRequirePos = (pCsr->pFilter->flags & FTS3_SEGMENT_REQUIRE_POS);
@@ -120852,10 +123351,13 @@
120852 int isScan = (pCsr->pFilter->flags & FTS3_SEGMENT_SCAN);
120853
120854 Fts3SegReader **apSegment = pCsr->apSegment;
120855 int nSegment = pCsr->nSegment;
120856 Fts3SegFilter *pFilter = pCsr->pFilter;
 
 
 
120857
120858 if( pCsr->nSegment==0 ) return SQLITE_OK;
120859
120860 do {
120861 int nMerge;
@@ -120863,11 +123365,11 @@
120863
120864 /* Advance the first pCsr->nAdvance entries in the apSegment[] array
120865 ** forward. Then sort the list in order of current term again.
120866 */
120867 for(i=0; i<pCsr->nAdvance; i++){
120868 rc = fts3SegReaderNext(p, apSegment[i]);
120869 if( rc!=SQLITE_OK ) return rc;
120870 }
120871 fts3SegReaderSort(apSegment, nSegment, pCsr->nAdvance, fts3SegReaderCmp);
120872 pCsr->nAdvance = 0;
120873
@@ -120902,69 +123404,87 @@
120902 ){
120903 nMerge++;
120904 }
120905
120906 assert( isIgnoreEmpty || (isRequirePos && !isColFilter) );
120907 if( nMerge==1 && !isIgnoreEmpty ){
120908 pCsr->aDoclist = apSegment[0]->aDoclist;
 
 
120909 pCsr->nDoclist = apSegment[0]->nDoclist;
120910 rc = SQLITE_ROW;
 
 
 
 
 
 
120911 }else{
120912 int nDoclist = 0; /* Size of doclist */
120913 sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */
120914
120915 /* The current term of the first nMerge entries in the array
120916 ** of Fts3SegReader objects is the same. The doclists must be merged
120917 ** and a single term returned with the merged doclist.
120918 */
120919 for(i=0; i<nMerge; i++){
120920 fts3SegReaderFirstDocid(apSegment[i]);
120921 }
120922 fts3SegReaderSort(apSegment, nMerge, nMerge, fts3SegReaderDoclistCmp);
120923 while( apSegment[0]->pOffsetList ){
120924 int j; /* Number of segments that share a docid */
120925 char *pList;
120926 int nList;
120927 int nByte;
120928 sqlite3_int64 iDocid = apSegment[0]->iDocid;
120929 fts3SegReaderNextDocid(apSegment[0], &pList, &nList);
120930 j = 1;
120931 while( j<nMerge
120932 && apSegment[j]->pOffsetList
120933 && apSegment[j]->iDocid==iDocid
120934 ){
120935 fts3SegReaderNextDocid(apSegment[j], 0, 0);
120936 j++;
120937 }
120938
120939 if( isColFilter ){
120940 fts3ColumnFilter(pFilter->iCol, &pList, &nList);
120941 }
120942
120943 if( !isIgnoreEmpty || nList>0 ){
120944 nByte = sqlite3Fts3VarintLen(iDocid-iPrev) + (isRequirePos?nList+1:0);
 
 
 
 
 
 
 
 
 
 
 
 
120945 if( nDoclist+nByte>pCsr->nBuffer ){
120946 char *aNew;
120947 pCsr->nBuffer = (nDoclist+nByte)*2;
120948 aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer);
120949 if( !aNew ){
120950 return SQLITE_NOMEM;
120951 }
120952 pCsr->aBuffer = aNew;
120953 }
120954 nDoclist += sqlite3Fts3PutVarint(
120955 &pCsr->aBuffer[nDoclist], iDocid-iPrev
120956 );
120957 iPrev = iDocid;
120958 if( isRequirePos ){
120959 memcpy(&pCsr->aBuffer[nDoclist], pList, nList);
120960 nDoclist += nList;
120961 pCsr->aBuffer[nDoclist++] = '\0';
120962 }
120963 }
120964
120965 fts3SegReaderSort(apSegment, nMerge, j, fts3SegReaderDoclistCmp);
120966 }
120967 if( nDoclist>0 ){
120968 pCsr->aDoclist = pCsr->aBuffer;
120969 pCsr->nDoclist = nDoclist;
120970 rc = SQLITE_ROW;
@@ -120973,13 +123493,14 @@
120973 pCsr->nAdvance = nMerge;
120974 }while( rc==SQLITE_OK );
120975
120976 return rc;
120977 }
 
120978
120979 SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(
120980 Fts3SegReaderCursor *pCsr /* Cursor object */
120981 ){
120982 if( pCsr ){
120983 int i;
120984 for(i=0; i<pCsr->nSegment; i++){
120985 sqlite3Fts3SegReaderFree(pCsr->apSegment[i]);
@@ -121002,47 +123523,60 @@
121002 ** If this function is called with iLevel<0, but there is only one
121003 ** segment in the database, SQLITE_DONE is returned immediately.
121004 ** Otherwise, if successful, SQLITE_OK is returned. If an error occurs,
121005 ** an SQLite error code is returned.
121006 */
121007 static int fts3SegmentMerge(Fts3Table *p, int iLevel){
121008 int rc; /* Return code */
121009 int iIdx = 0; /* Index of new segment */
121010 int iNewLevel = 0; /* Level to create new segment at */
121011 SegmentWriter *pWriter = 0; /* Used to write the new, merged, segment */
121012 Fts3SegFilter filter; /* Segment term filter condition */
121013 Fts3SegReaderCursor csr; /* Cursor to iterate through level(s) */
 
121014
121015 rc = sqlite3Fts3SegReaderCursor(p, iLevel, 0, 0, 1, 0, &csr);
 
 
 
 
 
 
 
121016 if( rc!=SQLITE_OK || csr.nSegment==0 ) goto finished;
121017
121018 if( iLevel==FTS3_SEGCURSOR_ALL ){
121019 /* This call is to merge all segments in the database to a single
121020 ** segment. The level of the new segment is equal to the the numerically
121021 ** greatest segment level currently present in the database. The index
121022 ** of the new segment is always 0. */
121023 int nDummy; /* TODO: Remove this */
121024 if( csr.nSegment==1 ){
121025 rc = SQLITE_DONE;
121026 goto finished;
121027 }
121028 rc = fts3SegmentCountMax(p, &nDummy, &iNewLevel);
 
 
 
 
 
121029 }else{
121030 /* This call is to merge all segments at level iLevel. Find the next
121031 ** available segment index at level iLevel+1. The call to
121032 ** fts3AllocateSegdirIdx() will merge the segments at level iLevel+1 to
121033 ** a single iLevel+2 segment if necessary. */
121034 iNewLevel = iLevel+1;
121035 rc = fts3AllocateSegdirIdx(p, iNewLevel, &iIdx);
121036 }
121037 if( rc!=SQLITE_OK ) goto finished;
121038 assert( csr.nSegment>0 );
121039 assert( iNewLevel>=0 );
 
121040
121041 memset(&filter, 0, sizeof(Fts3SegFilter));
121042 filter.flags = FTS3_SEGMENT_REQUIRE_POS;
121043 filter.flags |= (iLevel==FTS3_SEGCURSOR_ALL ? FTS3_SEGMENT_IGNORE_EMPTY : 0);
121044
121045 rc = sqlite3Fts3SegReaderStart(p, &csr, &filter);
121046 while( SQLITE_OK==rc ){
121047 rc = sqlite3Fts3SegReaderStep(p, &csr);
121048 if( rc!=SQLITE_ROW ) break;
@@ -121050,12 +123584,14 @@
121050 csr.zTerm, csr.nTerm, csr.aDoclist, csr.nDoclist);
121051 }
121052 if( rc!=SQLITE_OK ) goto finished;
121053 assert( pWriter );
121054
121055 rc = fts3DeleteSegdir(p, iLevel, csr.apSegment, csr.nSegment);
121056 if( rc!=SQLITE_OK ) goto finished;
 
 
121057 rc = fts3SegWriterFlush(p, pWriter, iNewLevel, iIdx);
121058
121059 finished:
121060 fts3SegWriterFree(pWriter);
121061 sqlite3Fts3SegReaderFinish(&csr);
@@ -121062,14 +123598,21 @@
121062 return rc;
121063 }
121064
121065
121066 /*
121067 ** Flush the contents of pendingTerms to a level 0 segment.
121068 */
121069 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
121070 return fts3SegmentMerge(p, FTS3_SEGCURSOR_PENDING);
 
 
 
 
 
 
 
121071 }
121072
121073 /*
121074 ** Encode N integers as varints into a blob.
121075 */
@@ -121215,10 +123758,27 @@
121215 sqlite3_bind_blob(pStmt, 1, pBlob, nBlob, SQLITE_STATIC);
121216 sqlite3_step(pStmt);
121217 *pRC = sqlite3_reset(pStmt);
121218 sqlite3_free(a);
121219 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
121220
121221 /*
121222 ** Handle a 'special' INSERT of the form:
121223 **
121224 ** "INSERT INTO tbl(tbl) VALUES(<expr>)"
@@ -121232,16 +123792,11 @@
121232 int nVal = sqlite3_value_bytes(pVal);
121233
121234 if( !zVal ){
121235 return SQLITE_NOMEM;
121236 }else if( nVal==8 && 0==sqlite3_strnicmp(zVal, "optimize", 8) ){
121237 rc = fts3SegmentMerge(p, FTS3_SEGCURSOR_ALL);
121238 if( rc==SQLITE_DONE ){
121239 rc = SQLITE_OK;
121240 }else{
121241 sqlite3Fts3PendingTermsClear(p);
121242 }
121243 #ifdef SQLITE_TEST
121244 }else if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
121245 p->nNodeSize = atoi(&zVal[9]);
121246 rc = SQLITE_OK;
121247 }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){
@@ -121250,61 +123805,23 @@
121250 #endif
121251 }else{
121252 rc = SQLITE_ERROR;
121253 }
121254
121255 sqlite3Fts3SegmentsClose(p);
121256 return rc;
121257 }
121258
121259 /*
121260 ** Return the deferred doclist associated with deferred token pDeferred.
121261 ** This function assumes that sqlite3Fts3CacheDeferredDoclists() has already
121262 ** been called to allocate and populate the doclist.
121263 */
121264 SQLITE_PRIVATE char *sqlite3Fts3DeferredDoclist(Fts3DeferredToken *pDeferred, int *pnByte){
121265 if( pDeferred->pList ){
121266 *pnByte = pDeferred->pList->nData;
121267 return pDeferred->pList->aData;
121268 }
121269 *pnByte = 0;
121270 return 0;
121271 }
121272
121273 /*
121274 ** Helper fucntion for FreeDeferredDoclists(). This function removes all
121275 ** references to deferred doclists from within the tree of Fts3Expr
121276 ** structures headed by
121277 */
121278 static void fts3DeferredDoclistClear(Fts3Expr *pExpr){
121279 if( pExpr ){
121280 fts3DeferredDoclistClear(pExpr->pLeft);
121281 fts3DeferredDoclistClear(pExpr->pRight);
121282 if( pExpr->isLoaded ){
121283 sqlite3_free(pExpr->aDoclist);
121284 pExpr->isLoaded = 0;
121285 pExpr->aDoclist = 0;
121286 pExpr->nDoclist = 0;
121287 pExpr->pCurrent = 0;
121288 pExpr->iCurrent = 0;
121289 }
121290 }
121291 }
121292
121293 /*
121294 ** Delete all cached deferred doclists. Deferred doclists are cached
121295 ** (allocated) by the sqlite3Fts3CacheDeferredDoclists() function.
121296 */
121297 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *pCsr){
121298 Fts3DeferredToken *pDef;
121299 for(pDef=pCsr->pDeferred; pDef; pDef=pDef->pNext){
121300 sqlite3_free(pDef->pList);
121301 pDef->pList = 0;
121302 }
121303 if( pCsr->pDeferred ){
121304 fts3DeferredDoclistClear(pCsr->pExpr);
121305 }
121306 }
121307
121308 /*
121309 ** Free all entries in the pCsr->pDeffered list. Entries are added to
121310 ** this list using sqlite3Fts3DeferToken().
@@ -121312,11 +123829,11 @@
121312 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *pCsr){
121313 Fts3DeferredToken *pDef;
121314 Fts3DeferredToken *pNext;
121315 for(pDef=pCsr->pDeferred; pDef; pDef=pNext){
121316 pNext = pDef->pNext;
121317 sqlite3_free(pDef->pList);
121318 sqlite3_free(pDef);
121319 }
121320 pCsr->pDeferred = 0;
121321 }
121322
@@ -121376,10 +123893,37 @@
121376 }
121377 }
121378
121379 return rc;
121380 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
121381
121382 /*
121383 ** Add an entry for token pToken to the pCsr->pDeferred list.
121384 */
121385 SQLITE_PRIVATE int sqlite3Fts3DeferToken(
@@ -121451,11 +123995,11 @@
121451 ){
121452 Fts3Table *p = (Fts3Table *)pVtab;
121453 int rc = SQLITE_OK; /* Return Code */
121454 int isRemove = 0; /* True for an UPDATE or DELETE */
121455 sqlite3_int64 iRemove = 0; /* Rowid removed by UPDATE or DELETE */
121456 u32 *aSzIns; /* Sizes of inserted documents */
121457 u32 *aSzDel; /* Sizes of deleted documents */
121458 int nChng = 0; /* Net change in number of documents */
121459 int bInsertDone = 0;
121460
121461 assert( p->pSegments==0 );
@@ -121466,16 +124010,20 @@
121466 */
121467 if( nArg>1
121468 && sqlite3_value_type(apVal[0])==SQLITE_NULL
121469 && sqlite3_value_type(apVal[p->nColumn+2])!=SQLITE_NULL
121470 ){
121471 return fts3SpecialInsert(p, apVal[p->nColumn+2]);
 
121472 }
121473
121474 /* Allocate space to hold the change in document sizes */
121475 aSzIns = sqlite3_malloc( sizeof(aSzIns[0])*(p->nColumn+1)*2 );
121476 if( aSzIns==0 ) return SQLITE_NOMEM;
 
 
 
121477 aSzDel = &aSzIns[p->nColumn+1];
121478 memset(aSzIns, 0, sizeof(aSzIns[0])*(p->nColumn+1)*2);
121479
121480 /* If this is an INSERT operation, or an UPDATE that modifies the rowid
121481 ** value, then this operation requires constraint handling.
@@ -121521,12 +124069,11 @@
121521 bInsertDone = 1;
121522 }
121523 }
121524 }
121525 if( rc!=SQLITE_OK ){
121526 sqlite3_free(aSzIns);
121527 return rc;
121528 }
121529
121530 /* If this is a DELETE or UPDATE operation, remove the old record. */
121531 if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
121532 assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER );
@@ -121555,10 +124102,11 @@
121555
121556 if( p->bHasStat ){
121557 fts3UpdateDocTotals(&rc, p, aSzIns, aSzDel, nChng);
121558 }
121559
 
121560 sqlite3_free(aSzIns);
121561 sqlite3Fts3SegmentsClose(p);
121562 return rc;
121563 }
121564
@@ -121569,16 +124117,14 @@
121569 */
121570 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *p){
121571 int rc;
121572 rc = sqlite3_exec(p->db, "SAVEPOINT fts3", 0, 0, 0);
121573 if( rc==SQLITE_OK ){
121574 rc = fts3SegmentMerge(p, FTS3_SEGCURSOR_ALL);
121575 if( rc==SQLITE_OK ){
121576 rc = sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
121577 if( rc==SQLITE_OK ){
121578 sqlite3Fts3PendingTermsClear(p);
121579 }
121580 }else{
121581 sqlite3_exec(p->db, "ROLLBACK TO fts3", 0, 0, 0);
121582 sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
121583 }
121584 }
@@ -121763,76 +124309,24 @@
121763 ){
121764 int iPhrase = 0; /* Variable used as the phrase counter */
121765 return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx);
121766 }
121767
121768 /*
121769 ** The argument to this function is always a phrase node. Its doclist
121770 ** (Fts3Expr.aDoclist[]) and the doclists associated with all phrase nodes
121771 ** to the left of this one in the query tree have already been loaded.
121772 **
121773 ** If this phrase node is part of a series of phrase nodes joined by
121774 ** NEAR operators (and is not the left-most of said series), then elements are
121775 ** removed from the phrases doclist consistent with the NEAR restriction. If
121776 ** required, elements may be removed from the doclists of phrases to the
121777 ** left of this one that are part of the same series of NEAR operator
121778 ** connected phrases.
121779 **
121780 ** If an OOM error occurs, SQLITE_NOMEM is returned. Otherwise, SQLITE_OK.
121781 */
121782 static int fts3ExprNearTrim(Fts3Expr *pExpr){
121783 int rc = SQLITE_OK;
121784 Fts3Expr *pParent = pExpr->pParent;
121785
121786 assert( pExpr->eType==FTSQUERY_PHRASE );
121787 while( rc==SQLITE_OK
121788 && pParent
121789 && pParent->eType==FTSQUERY_NEAR
121790 && pParent->pRight==pExpr
121791 ){
121792 /* This expression (pExpr) is the right-hand-side of a NEAR operator.
121793 ** Find the expression to the left of the same operator.
121794 */
121795 int nNear = pParent->nNear;
121796 Fts3Expr *pLeft = pParent->pLeft;
121797
121798 if( pLeft->eType!=FTSQUERY_PHRASE ){
121799 assert( pLeft->eType==FTSQUERY_NEAR );
121800 assert( pLeft->pRight->eType==FTSQUERY_PHRASE );
121801 pLeft = pLeft->pRight;
121802 }
121803
121804 rc = sqlite3Fts3ExprNearTrim(pLeft, pExpr, nNear);
121805
121806 pExpr = pLeft;
121807 pParent = pExpr->pParent;
121808 }
121809
121810 return rc;
121811 }
121812
121813 /*
121814 ** This is an fts3ExprIterate() callback used while loading the doclists
121815 ** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
121816 ** fts3ExprLoadDoclists().
121817 */
121818 static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
121819 int rc = SQLITE_OK;
 
121820 LoadDoclistCtx *p = (LoadDoclistCtx *)ctx;
121821
121822 UNUSED_PARAMETER(iPhrase);
121823
121824 p->nPhrase++;
121825 p->nToken += pExpr->pPhrase->nToken;
121826
121827 if( pExpr->isLoaded==0 ){
121828 rc = sqlite3Fts3ExprLoadDoclist(p->pCsr, pExpr);
121829 pExpr->isLoaded = 1;
121830 if( rc==SQLITE_OK ){
121831 rc = fts3ExprNearTrim(pExpr);
121832 }
121833 }
121834
121835 return rc;
121836 }
121837
121838 /*
@@ -122002,11 +124496,11 @@
122002 SnippetPhrase *pPhrase = &p->aPhrase[iPhrase];
122003 char *pCsr;
122004
122005 pPhrase->nToken = pExpr->pPhrase->nToken;
122006
122007 pCsr = sqlite3Fts3FindPositions(p->pCsr, pExpr, p->pCsr->iPrevId, p->iCol);
122008 if( pCsr ){
122009 int iFirst = 0;
122010 pPhrase->pList = pCsr;
122011 fts3GetDeltaPosition(&pCsr, &iFirst);
122012 pPhrase->pHead = pCsr;
@@ -122359,30 +124853,10 @@
122359
122360 *ppCollist = pEnd;
122361 return nEntry;
122362 }
122363
122364 static void fts3LoadColumnlistCounts(char **pp, u32 *aOut, int isGlobal){
122365 char *pCsr = *pp;
122366 while( *pCsr ){
122367 int nHit;
122368 sqlite3_int64 iCol = 0;
122369 if( *pCsr==0x01 ){
122370 pCsr++;
122371 pCsr += sqlite3Fts3GetVarint(pCsr, &iCol);
122372 }
122373 nHit = fts3ColumnlistCount(&pCsr);
122374 assert( nHit>0 );
122375 if( isGlobal ){
122376 aOut[iCol*3+1]++;
122377 }
122378 aOut[iCol*3] += nHit;
122379 }
122380 pCsr++;
122381 *pp = pCsr;
122382 }
122383
122384 /*
122385 ** fts3ExprIterate() callback used to collect the "global" matchinfo stats
122386 ** for a single query.
122387 **
122388 ** fts3ExprIterate() callback to load the 'global' elements of a
@@ -122412,52 +124886,13 @@
122412 Fts3Expr *pExpr, /* Phrase expression node */
122413 int iPhrase, /* Phrase number (numbered from zero) */
122414 void *pCtx /* Pointer to MatchInfo structure */
122415 ){
122416 MatchInfo *p = (MatchInfo *)pCtx;
122417 Fts3Cursor *pCsr = p->pCursor;
122418 char *pIter;
122419 char *pEnd;
122420 char *pFree = 0;
122421 u32 *aOut = &p->aMatchinfo[3*iPhrase*p->nCol];
122422
122423 assert( pExpr->isLoaded );
122424 assert( pExpr->eType==FTSQUERY_PHRASE );
122425
122426 if( pCsr->pDeferred ){
122427 Fts3Phrase *pPhrase = pExpr->pPhrase;
122428 int ii;
122429 for(ii=0; ii<pPhrase->nToken; ii++){
122430 if( pPhrase->aToken[ii].bFulltext ) break;
122431 }
122432 if( ii<pPhrase->nToken ){
122433 int nFree = 0;
122434 int rc = sqlite3Fts3ExprLoadFtDoclist(pCsr, pExpr, &pFree, &nFree);
122435 if( rc!=SQLITE_OK ) return rc;
122436 pIter = pFree;
122437 pEnd = &pFree[nFree];
122438 }else{
122439 int iCol; /* Column index */
122440 for(iCol=0; iCol<p->nCol; iCol++){
122441 aOut[iCol*3 + 1] = (u32)p->nDoc;
122442 aOut[iCol*3 + 2] = (u32)p->nDoc;
122443 }
122444 return SQLITE_OK;
122445 }
122446 }else{
122447 pIter = pExpr->aDoclist;
122448 pEnd = &pExpr->aDoclist[pExpr->nDoclist];
122449 }
122450
122451 /* Fill in the global hit count matrix row for this phrase. */
122452 while( pIter<pEnd ){
122453 while( *pIter++ & 0x80 ); /* Skip past docid. */
122454 fts3LoadColumnlistCounts(&pIter, &aOut[1], 1);
122455 }
122456
122457 sqlite3_free(pFree);
122458 return SQLITE_OK;
122459 }
122460
122461 /*
122462 ** fts3ExprIterate() callback used to collect the "local" part of the
122463 ** FTS3_MATCHINFO_HITS array. The local stats are those elements of the
@@ -122470,18 +124905,17 @@
122470 ){
122471 MatchInfo *p = (MatchInfo *)pCtx;
122472 int iStart = iPhrase * p->nCol * 3;
122473 int i;
122474
122475 for(i=0; i<p->nCol; i++) p->aMatchinfo[iStart+i*3] = 0;
122476
122477 if( pExpr->aDoclist ){
122478 char *pCsr;
122479
122480 pCsr = sqlite3Fts3FindPositions(p->pCursor, pExpr, p->pCursor->iPrevId, -1);
122481 if( pCsr ){
122482 fts3LoadColumnlistCounts(&pCsr, &p->aMatchinfo[iStart], 0);
 
 
122483 }
122484 }
122485
122486 return SQLITE_OK;
122487 }
@@ -122563,13 +124997,12 @@
122563 ** values for a matchinfo() FTS3_MATCHINFO_LCS request.
122564 */
122565 typedef struct LcsIterator LcsIterator;
122566 struct LcsIterator {
122567 Fts3Expr *pExpr; /* Pointer to phrase expression */
122568 char *pRead; /* Cursor used to iterate through aDoclist */
122569 int iPosOffset; /* Tokens count up to end of this phrase */
122570 int iCol; /* Current column number */
122571 int iPos; /* Current position */
122572 };
122573
122574 /*
122575 ** If LcsIterator.iCol is set to the following value, the iterator has
@@ -122596,21 +125029,14 @@
122596 char *pRead = pIter->pRead;
122597 sqlite3_int64 iRead;
122598 int rc = 0;
122599
122600 pRead += sqlite3Fts3GetVarint(pRead, &iRead);
122601 if( iRead==0 ){
122602 pIter->iCol = LCS_ITERATOR_FINISHED;
122603 rc = 1;
122604 }else{
122605 if( iRead==1 ){
122606 pRead += sqlite3Fts3GetVarint(pRead, &iRead);
122607 pIter->iCol = (int)iRead;
122608 pIter->iPos = pIter->iPosOffset;
122609 pRead += sqlite3Fts3GetVarint(pRead, &iRead);
122610 rc = 1;
122611 }
122612 pIter->iPos += (int)(iRead-2);
122613 }
122614
122615 pIter->pRead = pRead;
122616 return rc;
@@ -122638,46 +125064,38 @@
122638 **/
122639 aIter = sqlite3_malloc(sizeof(LcsIterator) * pCsr->nPhrase);
122640 if( !aIter ) return SQLITE_NOMEM;
122641 memset(aIter, 0, sizeof(LcsIterator) * pCsr->nPhrase);
122642 (void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
 
122643 for(i=0; i<pInfo->nPhrase; i++){
122644 LcsIterator *pIter = &aIter[i];
122645 nToken -= pIter->pExpr->pPhrase->nToken;
122646 pIter->iPosOffset = nToken;
122647 pIter->pRead = sqlite3Fts3FindPositions(pCsr,pIter->pExpr,pCsr->iPrevId,-1);
122648 if( pIter->pRead ){
122649 pIter->iPos = pIter->iPosOffset;
122650 fts3LcsIteratorAdvance(&aIter[i]);
122651 }else{
122652 pIter->iCol = LCS_ITERATOR_FINISHED;
122653 }
122654 }
122655
122656 for(iCol=0; iCol<pInfo->nCol; iCol++){
122657 int nLcs = 0; /* LCS value for this column */
122658 int nLive = 0; /* Number of iterators in aIter not at EOF */
122659
122660 /* Loop through the iterators in aIter[]. Set nLive to the number of
122661 ** iterators that point to a position-list corresponding to column iCol.
122662 */
122663 for(i=0; i<pInfo->nPhrase; i++){
122664 assert( aIter[i].iCol>=iCol );
122665 if( aIter[i].iCol==iCol ) nLive++;
 
 
 
 
 
122666 }
122667
122668 /* The following loop runs until all iterators in aIter[] have finished
122669 ** iterating through positions in column iCol. Exactly one of the
122670 ** iterators is advanced each time the body of the loop is run.
122671 */
122672 while( nLive>0 ){
122673 LcsIterator *pAdv = 0; /* The iterator to advance by one position */
122674 int nThisLcs = 0; /* LCS for the current iterator positions */
122675
122676 for(i=0; i<pInfo->nPhrase; i++){
122677 LcsIterator *pIter = &aIter[i];
122678 if( iCol!=pIter->iCol ){
122679 /* This iterator is already at EOF for this column. */
122680 nThisLcs = 0;
122681 }else{
122682 if( pAdv==0 || pIter->iPos<pAdv->iPos ){
122683 pAdv = pIter;
@@ -122739,11 +125157,11 @@
122739 if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol;
122740 break;
122741
122742 case FTS3_MATCHINFO_NDOC:
122743 if( bGlobal ){
122744 sqlite3_int64 nDoc;
122745 rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0);
122746 pInfo->aMatchinfo[0] = (u32)nDoc;
122747 }
122748 break;
122749
@@ -123013,11 +125431,11 @@
123013 int iTerm; /* For looping through nTerm phrase terms */
123014 char *pList; /* Pointer to position list for phrase */
123015 int iPos = 0; /* First position in position-list */
123016
123017 UNUSED_PARAMETER(iPhrase);
123018 pList = sqlite3Fts3FindPositions(p->pCsr, pExpr, p->iDocid, p->iCol);
123019 nTerm = pExpr->pPhrase->nToken;
123020 if( pList ){
123021 fts3GetDeltaPosition(&pList, &iPos);
123022 assert( iPos>=0 );
123023 }
@@ -124634,11 +127052,11 @@
124634 */
124635 static float cellArea(Rtree *pRtree, RtreeCell *p){
124636 float area = 1.0;
124637 int ii;
124638 for(ii=0; ii<(pRtree->nDim*2); ii+=2){
124639 area = area * (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]));
124640 }
124641 return area;
124642 }
124643
124644 /*
@@ -124647,11 +127065,11 @@
124647 */
124648 static float cellMargin(Rtree *pRtree, RtreeCell *p){
124649 float margin = 0.0;
124650 int ii;
124651 for(ii=0; ii<(pRtree->nDim*2); ii+=2){
124652 margin += (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]));
124653 }
124654 return margin;
124655 }
124656
124657 /*
@@ -124732,11 +127150,11 @@
124732
124733 if( x2<x1 ){
124734 o = 0.0;
124735 break;
124736 }else{
124737 o = o * (x2-x1);
124738 }
124739 }
124740 overlap += o;
124741 }
124742 }
@@ -124751,16 +127169,16 @@
124751 RtreeCell *pInsert,
124752 RtreeCell *aCell,
124753 int nCell,
124754 int iExclude
124755 ){
124756 float before;
124757 float after;
124758 before = cellOverlap(pRtree, p, aCell, nCell, iExclude);
124759 cellUnion(pRtree, p, pInsert);
124760 after = cellOverlap(pRtree, p, aCell, nCell, iExclude);
124761 return after-before;
124762 }
124763 #endif
124764
124765
124766 /*
@@ -124778,15 +127196,15 @@
124778 RtreeNode *pNode;
124779 rc = nodeAcquire(pRtree, 1, 0, &pNode);
124780
124781 for(ii=0; rc==SQLITE_OK && ii<(pRtree->iDepth-iHeight); ii++){
124782 int iCell;
124783 sqlite3_int64 iBest;
124784
124785 float fMinGrowth;
124786 float fMinArea;
124787 float fMinOverlap;
124788
124789 int nCell = NCELL(pNode);
124790 RtreeCell cell;
124791 RtreeNode *pChild;
124792
@@ -125212,13 +127630,13 @@
125212 ){
125213 int **aaSorted;
125214 int *aSpare;
125215 int ii;
125216
125217 int iBestDim;
125218 int iBestSplit;
125219 float fBestMargin;
125220
125221 int nByte = (pRtree->nDim+1)*(sizeof(int*)+nCell*sizeof(int));
125222
125223 aaSorted = (int **)sqlite3_malloc(nByte);
125224 if( !aaSorted ){
@@ -125236,13 +127654,13 @@
125236 SortByDimension(pRtree, aaSorted[ii], nCell, ii, aCell, aSpare);
125237 }
125238
125239 for(ii=0; ii<pRtree->nDim; ii++){
125240 float margin = 0.0;
125241 float fBestOverlap;
125242 float fBestArea;
125243 int iBestLeft;
125244 int nLeft;
125245
125246 for(
125247 nLeft=RTREE_MINCELLS(pRtree);
125248 nLeft<=(nCell-RTREE_MINCELLS(pRtree));
@@ -125553,11 +127971,11 @@
125553 static int deleteCell(Rtree *, RtreeNode *, int, int);
125554
125555 static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){
125556 int rc;
125557 int rc2;
125558 RtreeNode *pParent;
125559 int iCell;
125560
125561 assert( pNode->nRef==1 );
125562
125563 /* Remove the entry in the parent cell. */
@@ -125701,23 +128119,23 @@
125701 }else{
125702 nodeGetCell(pRtree, pNode, ii, &aCell[ii]);
125703 }
125704 aOrder[ii] = ii;
125705 for(iDim=0; iDim<pRtree->nDim; iDim++){
125706 aCenterCoord[iDim] += DCOORD(aCell[ii].aCoord[iDim*2]);
125707 aCenterCoord[iDim] += DCOORD(aCell[ii].aCoord[iDim*2+1]);
125708 }
125709 }
125710 for(iDim=0; iDim<pRtree->nDim; iDim++){
125711 aCenterCoord[iDim] = aCenterCoord[iDim]/((float)nCell*2.0);
125712 }
125713
125714 for(ii=0; ii<nCell; ii++){
125715 aDistance[ii] = 0.0;
125716 for(iDim=0; iDim<pRtree->nDim; iDim++){
125717 float coord = DCOORD(aCell[ii].aCoord[iDim*2+1]) -
125718 DCOORD(aCell[ii].aCoord[iDim*2]);
125719 aDistance[ii] += (coord-aCenterCoord[iDim])*(coord-aCenterCoord[iDim]);
125720 }
125721 }
125722
125723 SortByDistance(aOrder, nCell, aDistance, aSpare);
@@ -125812,14 +128230,14 @@
125812 nodeGetCell(pRtree, pNode, ii, &cell);
125813
125814 /* Find a node to store this cell in. pNode->iNode currently contains
125815 ** the height of the sub-tree headed by the cell.
125816 */
125817 rc = ChooseLeaf(pRtree, &cell, pNode->iNode, &pInsert);
125818 if( rc==SQLITE_OK ){
125819 int rc2;
125820 rc = rtreeInsertCell(pRtree, pInsert, &cell, pNode->iNode);
125821 rc2 = nodeRelease(pRtree, pInsert);
125822 if( rc==SQLITE_OK ){
125823 rc = rc2;
125824 }
125825 }
@@ -126204,11 +128622,11 @@
126204 int isCreate /* True for xCreate, false for xConnect */
126205 ){
126206 int rc;
126207 char *zSql;
126208 if( isCreate ){
126209 int iPageSize;
126210 zSql = sqlite3_mprintf("PRAGMA %Q.page_size", pRtree->zDb);
126211 rc = getIntFromStmt(db, zSql, &iPageSize);
126212 if( rc==SQLITE_OK ){
126213 pRtree->iNodeSize = iPageSize-64;
126214 if( (4+pRtree->nBytesPerCell*RTREE_MAXCELLS)<pRtree->iNodeSize ){
@@ -127007,14 +129425,11 @@
127007 ** May you find forgiveness for yourself and forgive others.
127008 ** May you share freely, never taking more than you give.
127009 **
127010 *************************************************************************
127011 ** This file implements a tokenizer for fts3 based on the ICU library.
127012 **
127013 ** $Id: fts3_icu.c,v 1.3 2008/09/01 18:34:20 danielk1977 Exp $
127014 */
127015
127016 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
127017 #ifdef SQLITE_ENABLE_ICU
127018
127019
127020 #include <unicode/ubrk.h>
127021
--- src/sqlite3.c
+++ src/sqlite3.c
@@ -1,8 +1,8 @@
1 /******************************************************************************
2 ** This file is an amalgamation of many separate C source files from SQLite
3 ** version 3.7.8. By combining all the individual C code files into this
4 ** single large file, the entire code can be compiled as a single translation
5 ** unit. This allows many compilers to do optimizations that would not be
6 ** possible if the files were compiled separately. Performance improvements
7 ** of 5% or more are commonly seen when SQLite is compiled as a single
8 ** translation unit.
@@ -648,13 +648,13 @@
648 **
649 ** See also: [sqlite3_libversion()],
650 ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
651 ** [sqlite_version()] and [sqlite_source_id()].
652 */
653 #define SQLITE_VERSION "3.7.8"
654 #define SQLITE_VERSION_NUMBER 3007008
655 #define SQLITE_SOURCE_ID "2011-08-16 02:07:04 9650d7962804d61f56cac944ff9bb2c7bc111957"
656
657 /*
658 ** CAPI3REF: Run-Time Library Version Numbers
659 ** KEYWORDS: sqlite3_version, sqlite3_sourceid
660 **
@@ -851,11 +851,11 @@
851 ** semicolon-separate SQL statements passed into its 2nd argument,
852 ** in the context of the [database connection] passed in as its 1st
853 ** argument. ^If the callback function of the 3rd argument to
854 ** sqlite3_exec() is not NULL, then it is invoked for each result row
855 ** coming out of the evaluated SQL statements. ^The 4th argument to
856 ** sqlite3_exec() is relayed through to the 1st argument of each
857 ** callback invocation. ^If the callback pointer to sqlite3_exec()
858 ** is NULL, then no callback is ever invoked and result rows are
859 ** ignored.
860 **
861 ** ^If an error occurs while evaluating the SQL statements passed into
@@ -1282,20 +1282,52 @@
1282 ** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most
1283 ** VFSes do not need this signal and should silently ignore this opcode.
1284 ** Applications should not call [sqlite3_file_control()] with this
1285 ** opcode as doing so may disrupt the operation of the specialized VFSes
1286 ** that do require it.
1287 **
1288 ** ^The [SQLITE_FCNTL_WIN32_AV_RETRY] opcode is used to configure automatic
1289 ** retry counts and intervals for certain disk I/O operations for the
1290 ** windows [VFS] in order to work to provide robustness against
1291 ** anti-virus programs. By default, the windows VFS will retry file read,
1292 ** file write, and file delete opertions up to 10 times, with a delay
1293 ** of 25 milliseconds before the first retry and with the delay increasing
1294 ** by an additional 25 milliseconds with each subsequent retry. This
1295 ** opcode allows those to values (10 retries and 25 milliseconds of delay)
1296 ** to be adjusted. The values are changed for all database connections
1297 ** within the same process. The argument is a pointer to an array of two
1298 ** integers where the first integer i the new retry count and the second
1299 ** integer is the delay. If either integer is negative, then the setting
1300 ** is not changed but instead the prior value of that setting is written
1301 ** into the array entry, allowing the current retry settings to be
1302 ** interrogated. The zDbName parameter is ignored.
1303 **
1304 ** ^The [SQLITE_FCNTL_PERSIST_WAL] opcode is used to set or query the
1305 ** persistent [WAL | Write AHead Log] setting. By default, the auxiliary
1306 ** write ahead log and shared memory files used for transaction control
1307 ** are automatically deleted when the latest connection to the database
1308 ** closes. Setting persistent WAL mode causes those files to persist after
1309 ** close. Persisting the files is useful when other processes that do not
1310 ** have write permission on the directory containing the database file want
1311 ** to read the database file, as the WAL and shared memory files must exist
1312 ** in order for the database to be readable. The fourth parameter to
1313 ** [sqlite3_file_control()] for this opcode should be a pointer to an integer.
1314 ** That integer is 0 to disable persistent WAL mode or 1 to enable persistent
1315 ** WAL mode. If the integer is -1, then it is overwritten with the current
1316 ** WAL persistence setting.
1317 **
1318 */
1319 #define SQLITE_FCNTL_LOCKSTATE 1
1320 #define SQLITE_GET_LOCKPROXYFILE 2
1321 #define SQLITE_SET_LOCKPROXYFILE 3
1322 #define SQLITE_LAST_ERRNO 4
1323 #define SQLITE_FCNTL_SIZE_HINT 5
1324 #define SQLITE_FCNTL_CHUNK_SIZE 6
1325 #define SQLITE_FCNTL_FILE_POINTER 7
1326 #define SQLITE_FCNTL_SYNC_OMITTED 8
1327 #define SQLITE_FCNTL_WIN32_AV_RETRY 9
1328 #define SQLITE_FCNTL_PERSIST_WAL 10
1329
1330 /*
1331 ** CAPI3REF: Mutex Handle
1332 **
1333 ** The mutex module within SQLite defines [sqlite3_mutex] to be an
@@ -1443,11 +1475,11 @@
1475 ** The xSleep() method causes the calling thread to sleep for at
1476 ** least the number of microseconds given. ^The xCurrentTime()
1477 ** method returns a Julian Day Number for the current date and time as
1478 ** a floating point value.
1479 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian
1480 ** Day Number multiplied by 86400000 (the number of milliseconds in
1481 ** a 24-hour day).
1482 ** ^SQLite will use the xCurrentTimeInt64() method to get the current
1483 ** date and time if that method is available (if iVersion is 2 or
1484 ** greater and the function pointer is not NULL) and will fall back
1485 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable.
@@ -1881,11 +1913,11 @@
1913 ** scratch memory beyond what is provided by this configuration option, then
1914 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd>
1915 **
1916 ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt>
1917 ** <dd> ^This option specifies a static memory buffer that SQLite can use for
1918 ** the database page cache with the default page cache implementation.
1919 ** This configuration should not be used if an application-define page
1920 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option.
1921 ** There are three arguments to this option: A pointer to 8-byte aligned
1922 ** memory, the size of each page buffer (sz), and the number of pages (N).
1923 ** The sz argument should be the size of the largest database page
@@ -2979,16 +3011,16 @@
3011 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
3012 **
3013 ** ^If [URI filename] interpretation is enabled, and the filename argument
3014 ** begins with "file:", then the filename is interpreted as a URI. ^URI
3015 ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is
3016 ** set in the fourth argument to sqlite3_open_v2(), or if it has
3017 ** been enabled globally using the [SQLITE_CONFIG_URI] option with the
3018 ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.
3019 ** As of SQLite version 3.7.7, URI filename interpretation is turned off
3020 ** by default, but future releases of SQLite might enable URI filename
3021 ** interpretation by default. See "[URI filenames]" for additional
3022 ** information.
3023 **
3024 ** URI filenames are parsed according to RFC 3986. ^If the URI contains an
3025 ** authority, then it must be either an empty string or the string
3026 ** "localhost". ^If the authority is not an empty string or "localhost", an
@@ -3365,11 +3397,11 @@
3397 ** a schema change, on the first [sqlite3_step()] call following any change
3398 ** to the [sqlite3_bind_text | bindings] of that [parameter].
3399 ** ^The specific value of WHERE-clause [parameter] might influence the
3400 ** choice of query plan if the parameter is the left-hand side of a [LIKE]
3401 ** or [GLOB] operator or if the parameter is compared to an indexed column
3402 ** and the [SQLITE_ENABLE_STAT3] compile-time option is enabled.
3403 ** the
3404 ** </li>
3405 ** </ol>
3406 */
3407 SQLITE_API int sqlite3_prepare(
@@ -3803,11 +3835,11 @@
3835 ** [extended result codes] might be returned as well.
3836 **
3837 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the
3838 ** database locks it needs to do its job. ^If the statement is a [COMMIT]
3839 ** or occurs outside of an explicit transaction, then you can retry the
3840 ** statement. If the statement is not a [COMMIT] and occurs within an
3841 ** explicit transaction then you should rollback the transaction before
3842 ** continuing.
3843 **
3844 ** ^[SQLITE_DONE] means that the statement has finished executing
3845 ** successfully. sqlite3_step() should not be called again on this virtual
@@ -4082,11 +4114,11 @@
4114
4115 /*
4116 ** CAPI3REF: Destroy A Prepared Statement Object
4117 **
4118 ** ^The sqlite3_finalize() function is called to delete a [prepared statement].
4119 ** ^If the most recent evaluation of the statement encountered no errors
4120 ** or if the statement is never been evaluated, then sqlite3_finalize() returns
4121 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then
4122 ** sqlite3_finalize(S) returns the appropriate [error code] or
4123 ** [extended error code].
4124 **
@@ -5996,11 +6028,11 @@
6028 ** versions of these routines, it should at least provide stubs that always
6029 ** return true so that one does not get spurious assertion failures.
6030 **
6031 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then
6032 ** the routine should return 1. This seems counter-intuitive since
6033 ** clearly the mutex cannot be held if it does not exist. But
6034 ** the reason the mutex does not exist is because the build is not
6035 ** using mutexes. And we do not want the assert() containing the
6036 ** call to sqlite3_mutex_held() to fail, so a non-zero return is
6037 ** the appropriate thing to do. ^The sqlite3_mutex_notheld()
6038 ** interface should also return 1 when given a NULL pointer.
@@ -6119,11 +6151,12 @@
6151 #define SQLITE_TESTCTRL_RESERVE 14
6152 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15
6153 #define SQLITE_TESTCTRL_ISKEYWORD 16
6154 #define SQLITE_TESTCTRL_PGHDRSZ 17
6155 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18
6156 #define SQLITE_TESTCTRL_LOCALTIME_FAULT 19
6157 #define SQLITE_TESTCTRL_LAST 19
6158
6159 /*
6160 ** CAPI3REF: SQLite Runtime Status
6161 **
6162 ** ^This interface is used to retrieve runtime status information
@@ -6505,11 +6538,11 @@
6538 ** [[the xFetch() page cache methods]]
6539 ** The xFetch() method locates a page in the cache and returns a pointer to
6540 ** the page, or a NULL pointer.
6541 ** A "page", in this context, means a buffer of szPage bytes aligned at an
6542 ** 8-byte boundary. The page to be fetched is determined by the key. ^The
6543 ** minimum key value is 1. After it has been retrieved using xFetch, the page
6544 ** is considered to be "pinned".
6545 **
6546 ** If the requested page is already in the page cache, then the page cache
6547 ** implementation must return a pointer to the page buffer with its content
6548 ** intact. If the requested page is not already in the cache, then the
@@ -7678,10 +7711,22 @@
7711 ** is 0x00000000ffffffff. But because of quirks of some compilers, we
7712 ** have to specify the value in the less intuitive manner shown:
7713 */
7714 #define SQLITE_MAX_U32 ((((u64)1)<<32)-1)
7715
7716 /*
7717 ** The datatype used to store estimates of the number of rows in a
7718 ** table or index. This is an unsigned integer type. For 99.9% of
7719 ** the world, a 32-bit integer is sufficient. But a 64-bit integer
7720 ** can be used at compile-time if desired.
7721 */
7722 #ifdef SQLITE_64BIT_STATS
7723 typedef u64 tRowcnt; /* 64-bit only if requested at compile-time */
7724 #else
7725 typedef u32 tRowcnt; /* 32-bit is the default */
7726 #endif
7727
7728 /*
7729 ** Macros to determine whether the machine is big or little endian,
7730 ** evaluated at runtime.
7731 */
7732 #ifdef SQLITE_AMALGAMATION
@@ -8483,10 +8528,11 @@
8528 SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int);
8529 SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int);
8530 SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int);
8531 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int);
8532 SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp);
8533 SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe*,int,char*);
8534 SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1);
8535 SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2);
8536 SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3);
8537 SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u8 P5);
8538 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr);
@@ -9282,11 +9328,11 @@
9328 ** sqlite3_close().
9329 *
9330 ** A thread must be holding a mutex on the corresponding Btree in order
9331 ** to access Schema content. This implies that the thread must also be
9332 ** holding a mutex on the sqlite3 connection pointer that owns the Btree.
9333 ** For a TEMP Schema, only the connection mutex is required.
9334 */
9335 struct Schema {
9336 int schema_cookie; /* Database schema version number for this file */
9337 int iGeneration; /* Generation counter. Incremented with each change */
9338 Hash tblHash; /* All tables indexed by name */
@@ -9555,10 +9601,12 @@
9601 #define SQLITE_IndexSort 0x04 /* Disable indexes for sorting */
9602 #define SQLITE_IndexSearch 0x08 /* Disable indexes for searching */
9603 #define SQLITE_IndexCover 0x10 /* Disable index covering table */
9604 #define SQLITE_GroupByOrder 0x20 /* Disable GROUPBY cover of ORDERBY */
9605 #define SQLITE_FactorOutConst 0x40 /* Disable factoring out constants */
9606 #define SQLITE_IdxRealAsInt 0x80 /* Store REAL as INT in indices */
9607 #define SQLITE_DistinctOpt 0x80 /* DISTINCT using indexes */
9608 #define SQLITE_OptMask 0xff /* Mask of all disablable opts */
9609
9610 /*
9611 ** Possible values for the sqlite.magic field.
9612 ** The numbers are obtained at random and have no special meaning, other
@@ -9876,11 +9924,11 @@
9924 int iPKey; /* If not negative, use aCol[iPKey] as the primary key */
9925 int nCol; /* Number of columns in this table */
9926 Column *aCol; /* Information about each column */
9927 Index *pIndex; /* List of SQL indexes on this table. */
9928 int tnum; /* Root BTree node for this table (see note above) */
9929 tRowcnt nRowEst; /* Estimated rows in table - from sqlite_stat1 table */
9930 Select *pSelect; /* NULL for tables. Points to definition if a view. */
9931 u16 nRef; /* Number of pointers to this Table */
9932 u8 tabFlags; /* Mask of TF_* values */
9933 u8 keyConf; /* What to do in case of uniqueness conflict on iPKey */
9934 FKey *pFKey; /* Linked list of all foreign keys in this table */
@@ -10075,35 +10123,43 @@
10123 */
10124 struct Index {
10125 char *zName; /* Name of this index */
10126 int nColumn; /* Number of columns in the table used by this index */
10127 int *aiColumn; /* Which columns are used by this index. 1st is 0 */
10128 tRowcnt *aiRowEst; /* Result of ANALYZE: Est. rows selected by each column */
10129 Table *pTable; /* The SQL table being indexed */
10130 int tnum; /* Page containing root of this index in database file */
10131 u8 onError; /* OE_Abort, OE_Ignore, OE_Replace, or OE_None */
10132 u8 autoIndex; /* True if is automatically created (ex: by UNIQUE) */
10133 u8 bUnordered; /* Use this index for == or IN queries only */
10134 u8 nSample; /* Number of elements in aSample[] */
10135 char *zColAff; /* String defining the affinity of each column */
10136 Index *pNext; /* The next index associated with the same table */
10137 Schema *pSchema; /* Schema containing this index */
10138 u8 *aSortOrder; /* Array of size Index.nColumn. True==DESC, False==ASC */
10139 char **azColl; /* Array of collation sequence names for index */
10140 #ifdef SQLITE_ENABLE_STAT3
10141 tRowcnt avgEq; /* Average nEq value for key values not in aSample */
10142 IndexSample *aSample; /* Samples of the left-most key */
10143 #endif
10144 };
10145
10146 /*
10147 ** Each sample stored in the sqlite_stat2 table is represented in memory
10148 ** using a structure of this type.
10149 */
10150 struct IndexSample {
10151 union {
10152 char *z; /* Value if eType is SQLITE_TEXT or SQLITE_BLOB */
10153 double r; /* Value if eType is SQLITE_FLOAT */
10154 i64 i; /* Value if eType is SQLITE_INTEGER */
10155 } u;
10156 u8 eType; /* SQLITE_NULL, SQLITE_INTEGER ... etc. */
10157 u16 nByte; /* Size in byte of text or blob. */
10158 tRowcnt nEq; /* Est. number of rows where the key equals this sample */
10159 tRowcnt nLt; /* Est. number of rows where key is less than this sample */
10160 tRowcnt nDLt; /* Est. number of distinct keys less than this sample */
10161 };
10162
10163 /*
10164 ** Each token coming out of the lexer is an instance of
10165 ** this structure. Tokens are also used as part of an expression.
@@ -10446,10 +10502,11 @@
10502 Table *pTab; /* An SQL table corresponding to zName */
10503 Select *pSelect; /* A SELECT statement used in place of a table name */
10504 u8 isPopulated; /* Temporary table associated with SELECT is populated */
10505 u8 jointype; /* Type of join between this able and the previous */
10506 u8 notIndexed; /* True if there is a NOT INDEXED clause */
10507 u8 isCorrelated; /* True if sub-query is correlated */
10508 #ifndef SQLITE_OMIT_EXPLAIN
10509 u8 iSelectId; /* If pSelect!=0, the id of the sub-select in EQP */
10510 #endif
10511 int iCursor; /* The VDBE cursor number used to access this table */
10512 Expr *pOn; /* The ON clause of a join */
@@ -10565,10 +10622,11 @@
10622 struct WhereInfo {
10623 Parse *pParse; /* Parsing and code generating context */
10624 u16 wctrlFlags; /* Flags originally passed to sqlite3WhereBegin() */
10625 u8 okOnePass; /* Ok to use one-pass algorithm for UPDATE or DELETE */
10626 u8 untestedTerms; /* Not all WHERE terms resolved by outer loop */
10627 u8 eDistinct;
10628 SrcList *pTabList; /* List of tables in the join */
10629 int iTop; /* The very beginning of the WHERE loop */
10630 int iContinue; /* Jump here to continue with next record */
10631 int iBreak; /* Jump here to break out of the loop */
10632 int nLevel; /* Number of nested loop */
@@ -10576,10 +10634,13 @@
10634 double savedNQueryLoop; /* pParse->nQueryLoop outside the WHERE loop */
10635 double nRowOut; /* Estimated number of output rows */
10636 WhereLevel a[1]; /* Information about each nest loop in WHERE */
10637 };
10638
10639 #define WHERE_DISTINCT_UNIQUE 1
10640 #define WHERE_DISTINCT_ORDERED 2
10641
10642 /*
10643 ** A NameContext defines a context in which to resolve table and column
10644 ** names. The context consists of a list of tables (the pSrcList) field and
10645 ** a list of named expression (pEList). The named expression list may
10646 ** be NULL. The pSrc corresponds to the FROM clause of a SELECT or
@@ -11052,10 +11113,11 @@
11113 int isPCacheInit; /* True after malloc is initialized */
11114 sqlite3_mutex *pInitMutex; /* Mutex used by sqlite3_initialize() */
11115 int nRefInitMutex; /* Number of users of pInitMutex */
11116 void (*xLog)(void*,int,const char*); /* Function for logging */
11117 void *pLogArg; /* First argument to xLog() */
11118 int bLocaltimeFault; /* True to fail localtime() calls */
11119 };
11120
11121 /*
11122 ** Context pointer passed down through the tree-walk.
11123 */
@@ -11299,10 +11361,11 @@
11361 #else
11362 # define sqlite3ViewGetColumnNames(A,B) 0
11363 #endif
11364
11365 SQLITE_PRIVATE void sqlite3DropTable(Parse*, SrcList*, int, int);
11366 SQLITE_PRIVATE void sqlite3CodeDropTable(Parse*, Table*, int, int);
11367 SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3*, Table*);
11368 #ifndef SQLITE_OMIT_AUTOINCREMENT
11369 SQLITE_PRIVATE void sqlite3AutoincrementBegin(Parse *pParse);
11370 SQLITE_PRIVATE void sqlite3AutoincrementEnd(Parse *pParse);
11371 #else
@@ -11336,11 +11399,11 @@
11399 #if defined(SQLITE_ENABLE_UPDATE_DELETE_LIMIT) && !defined(SQLITE_OMIT_SUBQUERY)
11400 SQLITE_PRIVATE Expr *sqlite3LimitWhere(Parse *, SrcList *, Expr *, ExprList *, Expr *, Expr *, char *);
11401 #endif
11402 SQLITE_PRIVATE void sqlite3DeleteFrom(Parse*, SrcList*, Expr*);
11403 SQLITE_PRIVATE void sqlite3Update(Parse*, SrcList*, ExprList*, Expr*, int);
11404 SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(Parse*, SrcList*, Expr*, ExprList**,ExprList*,u16);
11405 SQLITE_PRIVATE void sqlite3WhereEnd(WhereInfo*);
11406 SQLITE_PRIVATE int sqlite3ExprCodeGetColumn(Parse*, Table*, int, int, int);
11407 SQLITE_PRIVATE void sqlite3ExprCodeGetColumnOfTable(Vdbe*, Table*, int, int, int);
11408 SQLITE_PRIVATE void sqlite3ExprCodeMove(Parse*, int, int, int);
11409 SQLITE_PRIVATE void sqlite3ExprCodeCopy(Parse*, int, int, int);
@@ -11479,11 +11542,11 @@
11542 SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8);
11543 SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*);
11544 SQLITE_PRIVATE int sqlite3Atoi(const char*);
11545 SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar);
11546 SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte);
11547 SQLITE_PRIVATE u32 sqlite3Utf8Read(const u8*, const u8**);
11548
11549 /*
11550 ** Routines to read and write variable-length integers. These used to
11551 ** be defined locally, but now we use the varint routines in the util.c
11552 ** file. Code should use the MACRO forms below, as the Varint32 versions
@@ -11555,11 +11618,11 @@
11618 SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8,
11619 void(*)(void*));
11620 SQLITE_PRIVATE void sqlite3ValueFree(sqlite3_value*);
11621 SQLITE_PRIVATE sqlite3_value *sqlite3ValueNew(sqlite3 *);
11622 SQLITE_PRIVATE char *sqlite3Utf16to8(sqlite3 *, const void*, int, u8);
11623 #ifdef SQLITE_ENABLE_STAT3
11624 SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *, u8, char *, int, int *);
11625 #endif
11626 SQLITE_PRIVATE int sqlite3ValueFromExpr(sqlite3 *, Expr *, u8, u8, sqlite3_value **);
11627 SQLITE_PRIVATE void sqlite3ValueApplyAffinity(sqlite3_value *, u8, u8);
11628 #ifndef SQLITE_AMALGAMATION
@@ -12012,10 +12075,11 @@
12075 0, /* isPCacheInit */
12076 0, /* pInitMutex */
12077 0, /* nRefInitMutex */
12078 0, /* xLog */
12079 0, /* pLogArg */
12080 0, /* bLocaltimeFault */
12081 };
12082
12083
12084 /*
12085 ** Hash table for global functions - functions common to all
@@ -12180,10 +12244,13 @@
12244 #ifdef SQLITE_ENABLE_RTREE
12245 "ENABLE_RTREE",
12246 #endif
12247 #ifdef SQLITE_ENABLE_STAT2
12248 "ENABLE_STAT2",
12249 #endif
12250 #ifdef SQLITE_ENABLE_STAT3
12251 "ENABLE_STAT3",
12252 #endif
12253 #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY
12254 "ENABLE_UNLOCK_NOTIFY",
12255 #endif
12256 #ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT
@@ -13162,26 +13229,10 @@
13229 */
13230 #include <time.h>
13231
13232 #ifndef SQLITE_OMIT_DATETIME_FUNCS
13233
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13234
13235 /*
13236 ** A structure for holding a single date and time.
13237 */
13238 typedef struct DateTime DateTime;
@@ -13522,20 +13573,90 @@
13573 static void clearYMD_HMS_TZ(DateTime *p){
13574 p->validYMD = 0;
13575 p->validHMS = 0;
13576 p->validTZ = 0;
13577 }
13578
13579 /*
13580 ** On recent Windows platforms, the localtime_s() function is available
13581 ** as part of the "Secure CRT". It is essentially equivalent to
13582 ** localtime_r() available under most POSIX platforms, except that the
13583 ** order of the parameters is reversed.
13584 **
13585 ** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx.
13586 **
13587 ** If the user has not indicated to use localtime_r() or localtime_s()
13588 ** already, check for an MSVC build environment that provides
13589 ** localtime_s().
13590 */
13591 #if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \
13592 defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE)
13593 #define HAVE_LOCALTIME_S 1
13594 #endif
13595
13596 #ifndef SQLITE_OMIT_LOCALTIME
13597 /*
13598 ** The following routine implements the rough equivalent of localtime_r()
13599 ** using whatever operating-system specific localtime facility that
13600 ** is available. This routine returns 0 on success and
13601 ** non-zero on any kind of error.
13602 **
13603 ** If the sqlite3GlobalConfig.bLocaltimeFault variable is true then this
13604 ** routine will always fail.
13605 */
13606 static int osLocaltime(time_t *t, struct tm *pTm){
13607 int rc;
13608 #if (!defined(HAVE_LOCALTIME_R) || !HAVE_LOCALTIME_R) \
13609 && (!defined(HAVE_LOCALTIME_S) || !HAVE_LOCALTIME_S)
13610 struct tm *pX;
13611 #if SQLITE_THREADSAFE>0
13612 sqlite3_mutex *mutex = sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER);
13613 #endif
13614 sqlite3_mutex_enter(mutex);
13615 pX = localtime(t);
13616 #ifndef SQLITE_OMIT_BUILTIN_TEST
13617 if( sqlite3GlobalConfig.bLocaltimeFault ) pX = 0;
13618 #endif
13619 if( pX ) *pTm = *pX;
13620 sqlite3_mutex_leave(mutex);
13621 rc = pX==0;
13622 #else
13623 #ifndef SQLITE_OMIT_BUILTIN_TEST
13624 if( sqlite3GlobalConfig.bLocaltimeFault ) return 1;
13625 #endif
13626 #if defined(HAVE_LOCALTIME_R) && HAVE_LOCALTIME_R
13627 rc = localtime_r(t, pTm)==0;
13628 #else
13629 rc = localtime_s(pTm, t);
13630 #endif /* HAVE_LOCALTIME_R */
13631 #endif /* HAVE_LOCALTIME_R || HAVE_LOCALTIME_S */
13632 return rc;
13633 }
13634 #endif /* SQLITE_OMIT_LOCALTIME */
13635
13636
13637 #ifndef SQLITE_OMIT_LOCALTIME
13638 /*
13639 ** Compute the difference (in milliseconds) between localtime and UTC
13640 ** (a.k.a. GMT) for the time value p where p is in UTC. If no error occurs,
13641 ** return this value and set *pRc to SQLITE_OK.
13642 **
13643 ** Or, if an error does occur, set *pRc to SQLITE_ERROR. The returned value
13644 ** is undefined in this case.
13645 */
13646 static sqlite3_int64 localtimeOffset(
13647 DateTime *p, /* Date at which to calculate offset */
13648 sqlite3_context *pCtx, /* Write error here if one occurs */
13649 int *pRc /* OUT: Error code. SQLITE_OK or ERROR */
13650 ){
13651 DateTime x, y;
13652 time_t t;
13653 struct tm sLocal;
13654
13655 /* Initialize the contents of sLocal to avoid a compiler warning. */
13656 memset(&sLocal, 0, sizeof(sLocal));
13657
13658 x = *p;
13659 computeYMD_HMS(&x);
13660 if( x.Y<1971 || x.Y>=2038 ){
13661 x.Y = 2000;
13662 x.M = 1;
@@ -13549,51 +13670,27 @@
13670 }
13671 x.tz = 0;
13672 x.validJD = 0;
13673 computeJD(&x);
13674 t = (time_t)(x.iJD/1000 - 21086676*(i64)10000);
13675 if( osLocaltime(&t, &sLocal) ){
13676 sqlite3_result_error(pCtx, "local time unavailable", -1);
13677 *pRc = SQLITE_ERROR;
13678 return 0;
13679 }
13680 y.Y = sLocal.tm_year + 1900;
13681 y.M = sLocal.tm_mon + 1;
13682 y.D = sLocal.tm_mday;
13683 y.h = sLocal.tm_hour;
13684 y.m = sLocal.tm_min;
13685 y.s = sLocal.tm_sec;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13686 y.validYMD = 1;
13687 y.validHMS = 1;
13688 y.validJD = 0;
13689 y.validTZ = 0;
13690 computeJD(&y);
13691 *pRc = SQLITE_OK;
13692 return y.iJD - x.iJD;
13693 }
13694 #endif /* SQLITE_OMIT_LOCALTIME */
13695
13696 /*
@@ -13613,13 +13710,16 @@
13710 ** weekday N
13711 ** unixepoch
13712 ** localtime
13713 ** utc
13714 **
13715 ** Return 0 on success and 1 if there is any kind of error. If the error
13716 ** is in a system call (i.e. localtime()), then an error message is written
13717 ** to context pCtx. If the error is an unrecognized modifier, no error is
13718 ** written to pCtx.
13719 */
13720 static int parseModifier(sqlite3_context *pCtx, const char *zMod, DateTime *p){
13721 int rc = 1;
13722 int n;
13723 double r;
13724 char *z, zBuf[30];
13725 z = zBuf;
@@ -13635,13 +13735,12 @@
13735 ** Assuming the current time value is UTC (a.k.a. GMT), shift it to
13736 ** show local time.
13737 */
13738 if( strcmp(z, "localtime")==0 ){
13739 computeJD(p);
13740 p->iJD += localtimeOffset(p, pCtx, &rc);
13741 clearYMD_HMS_TZ(p);
 
13742 }
13743 break;
13744 }
13745 #endif
13746 case 'u': {
@@ -13658,15 +13757,16 @@
13757 }
13758 #ifndef SQLITE_OMIT_LOCALTIME
13759 else if( strcmp(z, "utc")==0 ){
13760 sqlite3_int64 c1;
13761 computeJD(p);
13762 c1 = localtimeOffset(p, pCtx, &rc);
13763 if( rc==SQLITE_OK ){
13764 p->iJD -= c1;
13765 clearYMD_HMS_TZ(p);
13766 p->iJD += c1 - localtimeOffset(p, pCtx, &rc);
13767 }
13768 }
13769 #endif
13770 break;
13771 }
13772 case 'w': {
@@ -13843,13 +13943,12 @@
13943 if( !z || parseDateOrTime(context, (char*)z, p) ){
13944 return 1;
13945 }
13946 }
13947 for(i=1; i<argc; i++){
13948 z = sqlite3_value_text(argv[i]);
13949 if( z==0 || parseModifier(context, (char*)z, p) ) return 1;
 
13950 }
13951 return 0;
13952 }
13953
13954
@@ -14343,11 +14442,11 @@
14442 DO_OS_MALLOC_TEST(0);
14443 /* 0x87f3f is a mask of SQLITE_OPEN_ flags that are valid to be passed
14444 ** down into the VFS layer. Some SQLITE_OPEN_ flags (for example,
14445 ** SQLITE_OPEN_FULLMUTEX or SQLITE_OPEN_SHAREDCACHE) are blocked before
14446 ** reaching the VFS. */
14447 rc = pVfs->xOpen(pVfs, zPath, pFile, flags & 0x87f7f, pFlagsOut);
14448 assert( rc==SQLITE_OK || pFile->pMethods==0 );
14449 return rc;
14450 }
14451 SQLITE_PRIVATE int sqlite3OsDelete(sqlite3_vfs *pVfs, const char *zPath, int dirSync){
14452 return pVfs->xDelete(pVfs, zPath, dirSync);
@@ -15804,11 +15903,11 @@
15903 ** Free an outstanding memory allocation.
15904 **
15905 ** This function assumes that the necessary mutexes, if any, are
15906 ** already held by the caller. Hence "Unsafe".
15907 */
15908 static void memsys3FreeUnsafe(void *pOld){
15909 Mem3Block *p = (Mem3Block*)pOld;
15910 int i;
15911 u32 size, x;
15912 assert( sqlite3_mutex_held(mem3.mutex) );
15913 assert( p>mem3.aPool && p<&mem3.aPool[mem3.nPool] );
@@ -15879,21 +15978,21 @@
15978 }
15979
15980 /*
15981 ** Free memory.
15982 */
15983 static void memsys3Free(void *pPrior){
15984 assert( pPrior );
15985 memsys3Enter();
15986 memsys3FreeUnsafe(pPrior);
15987 memsys3Leave();
15988 }
15989
15990 /*
15991 ** Change the size of an existing memory allocation
15992 */
15993 static void *memsys3Realloc(void *pPrior, int nBytes){
15994 int nOld;
15995 void *p;
15996 if( pPrior==0 ){
15997 return sqlite3_malloc(nBytes);
15998 }
@@ -20086,11 +20185,11 @@
20185 } \
20186 if( c<0x80 \
20187 || (c&0xFFFFF800)==0xD800 \
20188 || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \
20189 }
20190 SQLITE_PRIVATE u32 sqlite3Utf8Read(
20191 const unsigned char *zIn, /* First byte of UTF-8 character */
20192 const unsigned char **pzNext /* Write first byte past UTF-8 char here */
20193 ){
20194 unsigned int c;
20195
@@ -20387,11 +20486,11 @@
20486 ** no longer required.
20487 **
20488 ** If a malloc failure occurs, NULL is returned and the db.mallocFailed
20489 ** flag set.
20490 */
20491 #ifdef SQLITE_ENABLE_STAT3
20492 SQLITE_PRIVATE char *sqlite3Utf8to16(sqlite3 *db, u8 enc, char *z, int n, int *pnOut){
20493 Mem m;
20494 memset(&m, 0, sizeof(m));
20495 m.db = db;
20496 sqlite3VdbeMemSetStr(&m, z, n, SQLITE_UTF8, SQLITE_STATIC);
@@ -21633,26 +21732,32 @@
21732 return -x;
21733 }
21734
21735 #ifdef SQLITE_ENABLE_8_3_NAMES
21736 /*
21737 ** If SQLITE_ENABLE_8_3_NAMES is set at compile-time and if the database
21738 ** filename in zBaseFilename is a URI with the "8_3_names=1" parameter and
21739 ** if filename in z[] has a suffix (a.k.a. "extension") that is longer than
21740 ** three characters, then shorten the suffix on z[] to be the last three
21741 ** characters of the original suffix.
21742 **
21743 ** If SQLITE_ENABLE_8_3_NAMES is set to 2 at compile-time, then always
21744 ** do the suffix shortening regardless of URI parameter.
21745 **
21746 ** Examples:
21747 **
21748 ** test.db-journal => test.nal
21749 ** test.db-wal => test.wal
21750 ** test.db-shm => test.shm
21751 */
21752 SQLITE_PRIVATE void sqlite3FileSuffix3(const char *zBaseFilename, char *z){
21753 #if SQLITE_ENABLE_8_3_NAMES<2
21754 const char *zOk;
21755 zOk = sqlite3_uri_parameter(zBaseFilename, "8_3_names");
21756 if( zOk && sqlite3GetBoolean(zOk) )
21757 #endif
21758 {
21759 int i, sz;
21760 sz = sqlite3Strlen30(z);
21761 for(i=sz-1; i>0 && z[i]!='/' && z[i]!='.'; i--){}
21762 if( z[i]=='.' && ALWAYS(sz>i+4) ) memcpy(&z[i+1], &z[sz-3], 4);
21763 }
@@ -22182,14 +22287,17 @@
22287 #ifdef MEMORY_DEBUG
22288 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
22289 #endif
22290
22291 #ifdef SQLITE_DEBUG
22292 # ifndef SQLITE_DEBUG_OS_TRACE
22293 # define SQLITE_DEBUG_OS_TRACE 0
22294 # endif
22295 int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
22296 # define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
22297 #else
22298 # define OSTRACE(X)
22299 #endif
22300
22301 /*
22302 ** Macros for performance tracing. Normally turned off. Only works
22303 ** on i486 hardware.
@@ -24363,10 +24471,14 @@
24471
24472 #if defined(__APPLE__) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS)
24473 # include <sys/mount.h>
24474 #endif
24475
24476 #ifdef HAVE_UTIME
24477 # include <utime.h>
24478 #endif
24479
24480 /*
24481 ** Allowed values of unixFile.fsFlags
24482 */
24483 #define SQLITE_FSFLAGS_IS_MSDOS 0x1
24484
@@ -24428,11 +24540,10 @@
24540 typedef struct unixFile unixFile;
24541 struct unixFile {
24542 sqlite3_io_methods const *pMethod; /* Always the first entry */
24543 unixInodeInfo *pInode; /* Info about locks on this inode */
24544 int h; /* The file descriptor */
 
24545 unsigned char eFileLock; /* The type of lock held on this fd */
24546 unsigned char ctrlFlags; /* Behavioral bits. UNIXFILE_* flags */
24547 int lastErrno; /* The unix errno from last I/O error */
24548 void *lockingContext; /* Locking style specific state */
24549 UnixUnusedFd *pUnused; /* Pre-allocated UnixUnusedFd */
@@ -24470,12 +24581,14 @@
24581 };
24582
24583 /*
24584 ** Allowed values for the unixFile.ctrlFlags bitmask:
24585 */
24586 #define UNIXFILE_EXCL 0x01 /* Connections from one process only */
24587 #define UNIXFILE_RDONLY 0x02 /* Connection is read only */
24588 #define UNIXFILE_PERSIST_WAL 0x04 /* Persistent WAL mode */
24589 #define UNIXFILE_DIRSYNC 0x08 /* Directory sync needed */
24590
24591 /*
24592 ** Include code that is common to all os_*.c files
24593 */
24594 /************** Include os_common.h in the middle of os_unix.c ***************/
@@ -24510,14 +24623,17 @@
24623 #ifdef MEMORY_DEBUG
24624 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
24625 #endif
24626
24627 #ifdef SQLITE_DEBUG
24628 # ifndef SQLITE_DEBUG_OS_TRACE
24629 # define SQLITE_DEBUG_OS_TRACE 0
24630 # endif
24631 int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
24632 # define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
24633 #else
24634 # define OSTRACE(X)
24635 #endif
24636
24637 /*
24638 ** Macros for performance tracing. Normally turned off. Only works
24639 ** on i486 hardware.
@@ -24722,10 +24838,13 @@
24838 */
24839 static int posixOpen(const char *zFile, int flags, int mode){
24840 return open(zFile, flags, mode);
24841 }
24842
24843 /* Forward reference */
24844 static int openDirectory(const char*, int*);
24845
24846 /*
24847 ** Many system calls are accessed through pointer-to-functions so that
24848 ** they may be overridden at runtime to facilitate fault injection during
24849 ** testing and sandboxing. The following array holds the names and pointers
24850 ** to all overrideable system calls.
@@ -24818,10 +24937,16 @@
24937 #else
24938 { "fallocate", (sqlite3_syscall_ptr)0, 0 },
24939 #endif
24940 #define osFallocate ((int(*)(int,off_t,off_t))aSyscall[15].pCurrent)
24941
24942 { "unlink", (sqlite3_syscall_ptr)unlink, 0 },
24943 #define osUnlink ((int(*)(const char*))aSyscall[16].pCurrent)
24944
24945 { "openDirectory", (sqlite3_syscall_ptr)openDirectory, 0 },
24946 #define osOpenDirectory ((int(*)(const char*,int*))aSyscall[17].pCurrent)
24947
24948 }; /* End of the overrideable system calls */
24949
24950 /*
24951 ** This is the xSetSystemCall() method of sqlite3_vfs for all of the
24952 ** "unix" VFSes. Return SQLITE_OK opon successfully updating the
@@ -25102,11 +25227,13 @@
25227 case EINVAL:
25228 case ENOTCONN:
25229 case ENODEV:
25230 case ENXIO:
25231 case ENOENT:
25232 #ifdef ESTALE /* ESTALE is not defined on Interix systems */
25233 case ESTALE:
25234 #endif
25235 case ENOSYS:
25236 /* these should force the client to close the file and reconnect */
25237
25238 default:
25239 return sqliteIOErr;
@@ -26172,22 +26299,18 @@
26299 ** even on VxWorks. A mutex will be acquired on VxWorks by the
26300 ** vxworksReleaseFileId() routine.
26301 */
26302 static int closeUnixFile(sqlite3_file *id){
26303 unixFile *pFile = (unixFile*)id;
 
 
 
 
26304 if( pFile->h>=0 ){
26305 robust_close(pFile, pFile->h, __LINE__);
26306 pFile->h = -1;
26307 }
26308 #if OS_VXWORKS
26309 if( pFile->pId ){
26310 if( pFile->isDelete ){
26311 osUnlink(pFile->pId->zCanonicalName);
26312 }
26313 vxworksReleaseFileId(pFile->pId);
26314 pFile->pId = 0;
26315 }
26316 #endif
@@ -26368,12 +26491,14 @@
26491 /* If we have any lock, then the lock file already exists. All we have
26492 ** to do is adjust our internal record of the lock level.
26493 */
26494 if( pFile->eFileLock > NO_LOCK ){
26495 pFile->eFileLock = eFileLock;
 
26496 /* Always update the timestamp on the old file */
26497 #ifdef HAVE_UTIME
26498 utime(zLockFile, NULL);
26499 #else
26500 utimes(zLockFile, NULL);
26501 #endif
26502 return SQLITE_OK;
26503 }
26504
@@ -26430,11 +26555,11 @@
26555 return SQLITE_OK;
26556 }
26557
26558 /* To fully unlock the database, delete the lock file */
26559 assert( eFileLock==NO_LOCK );
26560 if( osUnlink(zLockFile) ){
26561 int rc = 0;
26562 int tErrno = errno;
26563 if( ENOENT != tErrno ){
26564 rc = SQLITE_IOERR_UNLOCK;
26565 }
@@ -27665,10 +27790,54 @@
27790 if( OS_VXWORKS && rc!= -1 ){
27791 rc = 0;
27792 }
27793 return rc;
27794 }
27795
27796 /*
27797 ** Open a file descriptor to the directory containing file zFilename.
27798 ** If successful, *pFd is set to the opened file descriptor and
27799 ** SQLITE_OK is returned. If an error occurs, either SQLITE_NOMEM
27800 ** or SQLITE_CANTOPEN is returned and *pFd is set to an undefined
27801 ** value.
27802 **
27803 ** The directory file descriptor is used for only one thing - to
27804 ** fsync() a directory to make sure file creation and deletion events
27805 ** are flushed to disk. Such fsyncs are not needed on newer
27806 ** journaling filesystems, but are required on older filesystems.
27807 **
27808 ** This routine can be overridden using the xSetSysCall interface.
27809 ** The ability to override this routine was added in support of the
27810 ** chromium sandbox. Opening a directory is a security risk (we are
27811 ** told) so making it overrideable allows the chromium sandbox to
27812 ** replace this routine with a harmless no-op. To make this routine
27813 ** a no-op, replace it with a stub that returns SQLITE_OK but leaves
27814 ** *pFd set to a negative number.
27815 **
27816 ** If SQLITE_OK is returned, the caller is responsible for closing
27817 ** the file descriptor *pFd using close().
27818 */
27819 static int openDirectory(const char *zFilename, int *pFd){
27820 int ii;
27821 int fd = -1;
27822 char zDirname[MAX_PATHNAME+1];
27823
27824 sqlite3_snprintf(MAX_PATHNAME, zDirname, "%s", zFilename);
27825 for(ii=(int)strlen(zDirname); ii>1 && zDirname[ii]!='/'; ii--);
27826 if( ii>0 ){
27827 zDirname[ii] = '\0';
27828 fd = robust_open(zDirname, O_RDONLY|O_BINARY, 0);
27829 if( fd>=0 ){
27830 #ifdef FD_CLOEXEC
27831 osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
27832 #endif
27833 OSTRACE(("OPENDIR %-3d %s\n", fd, zDirname));
27834 }
27835 }
27836 *pFd = fd;
27837 return (fd>=0?SQLITE_OK:unixLogError(SQLITE_CANTOPEN_BKPT, "open", zDirname));
27838 }
27839
27840 /*
27841 ** Make sure all writes to a particular file are committed to disk.
27842 **
27843 ** If dataOnly==0 then both the file itself and its metadata (file
@@ -27706,32 +27875,25 @@
27875 SimulateIOError( rc=1 );
27876 if( rc ){
27877 pFile->lastErrno = errno;
27878 return unixLogError(SQLITE_IOERR_FSYNC, "full_fsync", pFile->zPath);
27879 }
27880
27881 /* Also fsync the directory containing the file if the DIRSYNC flag
27882 ** is set. This is a one-time occurrance. Many systems (examples: AIX)
27883 ** are unable to fsync a directory, so ignore errors on the fsync.
27884 */
27885 if( pFile->ctrlFlags & UNIXFILE_DIRSYNC ){
27886 int dirfd;
27887 OSTRACE(("DIRSYNC %s (have_fullfsync=%d fullsync=%d)\n", pFile->zPath,
27888 HAVE_FULLFSYNC, isFullsync));
27889 rc = osOpenDirectory(pFile->zPath, &dirfd);
27890 if( rc==SQLITE_OK && dirfd>=0 ){
27891 full_fsync(dirfd, 0, 0);
27892 robust_close(pFile, dirfd, __LINE__);
27893 }
27894 pFile->ctrlFlags &= ~UNIXFILE_DIRSYNC;
 
 
 
 
 
 
 
 
 
 
 
 
 
27895 }
27896 return rc;
27897 }
27898
27899 /*
@@ -27816,17 +27978,23 @@
27978 ** If the user has configured a chunk-size for this file, it could be
27979 ** that the file needs to be extended at this point. Otherwise, the
27980 ** SQLITE_FCNTL_SIZE_HINT operation is a no-op for Unix.
27981 */
27982 static int fcntlSizeHint(unixFile *pFile, i64 nByte){
27983 { /* preserve indentation of removed "if" */
27984 i64 nSize; /* Required file size */
27985 i64 szChunk; /* Chunk size */
27986 struct stat buf; /* Used to hold return values of fstat() */
27987
27988 if( osFstat(pFile->h, &buf) ) return SQLITE_IOERR_FSTAT;
27989
27990 szChunk = pFile->szChunk;
27991 if( szChunk==0 ){
27992 nSize = nByte;
27993 }else{
27994 nSize = ((nByte+szChunk-1) / szChunk) * szChunk;
27995 }
27996 if( nSize>(i64)buf.st_size ){
27997
27998 #if defined(HAVE_POSIX_FALLOCATE) && HAVE_POSIX_FALLOCATE
27999 /* The code below is handling the return value of osFallocate()
28000 ** correctly. posix_fallocate() is defined to "returns zero on success,
@@ -27865,25 +28033,37 @@
28033
28034 /*
28035 ** Information and control of an open file handle.
28036 */
28037 static int unixFileControl(sqlite3_file *id, int op, void *pArg){
28038 unixFile *pFile = (unixFile*)id;
28039 switch( op ){
28040 case SQLITE_FCNTL_LOCKSTATE: {
28041 *(int*)pArg = pFile->eFileLock;
28042 return SQLITE_OK;
28043 }
28044 case SQLITE_LAST_ERRNO: {
28045 *(int*)pArg = pFile->lastErrno;
28046 return SQLITE_OK;
28047 }
28048 case SQLITE_FCNTL_CHUNK_SIZE: {
28049 pFile->szChunk = *(int *)pArg;
28050 return SQLITE_OK;
28051 }
28052 case SQLITE_FCNTL_SIZE_HINT: {
28053 return fcntlSizeHint(pFile, *(i64 *)pArg);
28054 }
28055 case SQLITE_FCNTL_PERSIST_WAL: {
28056 int bPersist = *(int*)pArg;
28057 if( bPersist<0 ){
28058 *(int*)pArg = (pFile->ctrlFlags & UNIXFILE_PERSIST_WAL)!=0;
28059 }else if( bPersist==0 ){
28060 pFile->ctrlFlags &= ~UNIXFILE_PERSIST_WAL;
28061 }else{
28062 pFile->ctrlFlags |= UNIXFILE_PERSIST_WAL;
28063 }
28064 return SQLITE_OK;
28065 }
28066 #ifndef NDEBUG
28067 /* The pager calls this method to signal that it has done
28068 ** a rollback and that the database is therefore unchanged and
28069 ** it hence it is OK for the transaction change counter to be
@@ -28095,11 +28275,11 @@
28275 unixShmNode *p = pFd->pInode->pShmNode;
28276 assert( unixMutexHeld() );
28277 if( p && p->nRef==0 ){
28278 int i;
28279 assert( p->pInode==pFd->pInode );
28280 sqlite3_mutex_free(p->mutex);
28281 for(i=0; i<p->nRegion; i++){
28282 if( p->h>=0 ){
28283 munmap(p->apRegion[i], p->szRegion);
28284 }else{
28285 sqlite3_free(p->apRegion[i]);
@@ -28564,11 +28744,11 @@
28744 ** shared-memory file, too */
28745 unixEnterMutex();
28746 assert( pShmNode->nRef>0 );
28747 pShmNode->nRef--;
28748 if( pShmNode->nRef==0 ){
28749 if( deleteFlag && pShmNode->h>=0 ) osUnlink(pShmNode->zFilename);
28750 unixShmPurge(pDbFd);
28751 }
28752 unixLeaveMutex();
28753
28754 return SQLITE_OK;
@@ -28877,11 +29057,11 @@
29057 ** Initialize the contents of the unixFile structure pointed to by pId.
29058 */
29059 static int fillInUnixFile(
29060 sqlite3_vfs *pVfs, /* Pointer to vfs object */
29061 int h, /* Open file descriptor of file being opened */
29062 int syncDir, /* True to sync directory on first sync */
29063 sqlite3_file *pId, /* Write to the unixFile structure here */
29064 const char *zFilename, /* Name of the file being opened */
29065 int noLock, /* Omit locking if true */
29066 int isDelete, /* Delete on close if true */
29067 int isReadOnly /* True if the file is opened read-only */
@@ -28908,20 +29088,22 @@
29088 assert( zFilename==0 || zFilename[0]=='/' );
29089 #endif
29090
29091 OSTRACE(("OPEN %-3d %s\n", h, zFilename));
29092 pNew->h = h;
 
29093 pNew->zPath = zFilename;
29094 if( memcmp(pVfs->zName,"unix-excl",10)==0 ){
29095 pNew->ctrlFlags = UNIXFILE_EXCL;
29096 }else{
29097 pNew->ctrlFlags = 0;
29098 }
29099 if( isReadOnly ){
29100 pNew->ctrlFlags |= UNIXFILE_RDONLY;
29101 }
29102 if( syncDir ){
29103 pNew->ctrlFlags |= UNIXFILE_DIRSYNC;
29104 }
29105
29106 #if OS_VXWORKS
29107 pNew->pId = vxworksFindFileId(zFilename);
29108 if( pNew->pId==0 ){
29109 noLock = 1;
@@ -29044,56 +29226,24 @@
29226 pNew->lastErrno = 0;
29227 #if OS_VXWORKS
29228 if( rc!=SQLITE_OK ){
29229 if( h>=0 ) robust_close(pNew, h, __LINE__);
29230 h = -1;
29231 osUnlink(zFilename);
29232 isDelete = 0;
29233 }
29234 pNew->isDelete = isDelete;
29235 #endif
29236 if( rc!=SQLITE_OK ){
 
29237 if( h>=0 ) robust_close(pNew, h, __LINE__);
29238 }else{
29239 pNew->pMethod = pLockingStyle;
29240 OpenCounter(+1);
29241 }
29242 return rc;
29243 }
29244
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29245 /*
29246 ** Return the name of a directory in which to put temporary files.
29247 ** If no suitable temporary file directory can be found, return NULL.
29248 */
29249 static const char *unixTempFileDir(void){
@@ -29204,11 +29354,11 @@
29354 ** ignored and -1 is returned. The caller will try to open a new file
29355 ** descriptor on the same path, fail, and return an error to SQLite.
29356 **
29357 ** Even if a subsequent open() call does succeed, the consequences of
29358 ** not searching for a resusable file descriptor are not dire. */
29359 if( 0==osStat(zPath, &sStat) ){
29360 unixInodeInfo *pInode;
29361
29362 unixEnterMutex();
29363 pInode = inodeList;
29364 while( pInode && (pInode->fileId.dev!=sStat.st_dev
@@ -29280,11 +29430,11 @@
29430 while( nDb>0 && zPath[nDb]!='-' ) nDb--;
29431 if( nDb==0 ) return SQLITE_OK;
29432 memcpy(zDb, zPath, nDb);
29433 zDb[nDb] = '\0';
29434
29435 if( 0==osStat(zDb, &sStat) ){
29436 *pMode = sStat.st_mode & 0777;
29437 }else{
29438 rc = SQLITE_IOERR_FSTAT;
29439 }
29440 }else if( flags & SQLITE_OPEN_DELETEONCLOSE ){
@@ -29322,11 +29472,10 @@
29472 int flags, /* Input flags to control the opening */
29473 int *pOutFlags /* Output flags returned to SQLite core */
29474 ){
29475 unixFile *p = (unixFile *)pFile;
29476 int fd = -1; /* File descriptor returned by open() */
 
29477 int openFlags = 0; /* Flags to pass to open() */
29478 int eType = flags&0xFFFFFF00; /* Type of file to open */
29479 int noLock; /* True to omit locking primitives */
29480 int rc = SQLITE_OK; /* Function Return Code */
29481
@@ -29341,11 +29490,11 @@
29490
29491 /* If creating a master or main-file journal, this function will open
29492 ** a file-descriptor on the directory too. The first time unixSync()
29493 ** is called the directory file descriptor will be fsync()ed and close()d.
29494 */
29495 int syncDir = (isCreate && (
29496 eType==SQLITE_OPEN_MASTER_JOURNAL
29497 || eType==SQLITE_OPEN_MAIN_JOURNAL
29498 || eType==SQLITE_OPEN_WAL
29499 ));
29500
@@ -29395,11 +29544,11 @@
29544 }
29545 }
29546 p->pUnused = pUnused;
29547 }else if( !zName ){
29548 /* If zName is NULL, the upper layer is requesting a temp file. */
29549 assert(isDelete && !syncDir);
29550 rc = unixGetTempname(MAX_PATHNAME+1, zTmpname);
29551 if( rc!=SQLITE_OK ){
29552 return rc;
29553 }
29554 zName = zTmpname;
@@ -29451,32 +29600,19 @@
29600
29601 if( isDelete ){
29602 #if OS_VXWORKS
29603 zPath = zName;
29604 #else
29605 osUnlink(zName);
29606 #endif
29607 }
29608 #if SQLITE_ENABLE_LOCKING_STYLE
29609 else{
29610 p->openFlags = openFlags;
29611 }
29612 #endif
29613
 
 
 
 
 
 
 
 
 
 
 
 
 
29614 #ifdef FD_CLOEXEC
29615 osFcntl(fd, F_SETFD, osFcntl(fd, F_GETFD, 0) | FD_CLOEXEC);
29616 #endif
29617
29618 noLock = eType!=SQLITE_OPEN_MAIN_DB;
@@ -29484,11 +29620,10 @@
29620
29621 #if defined(__APPLE__) || SQLITE_ENABLE_LOCKING_STYLE
29622 struct statfs fsInfo;
29623 if( fstatfs(fd, &fsInfo) == -1 ){
29624 ((unixFile*)pFile)->lastErrno = errno;
 
29625 robust_close(p, fd, __LINE__);
29626 return SQLITE_IOERR_ACCESS;
29627 }
29628 if (0 == strncmp("msdos", fsInfo.f_fstypename, 5)) {
29629 ((unixFile*)pFile)->fsFlags |= SQLITE_FSFLAGS_IS_MSDOS;
@@ -29516,21 +29651,18 @@
29651 ** then the call to close() will cancel those locks. In practice,
29652 ** we're assuming that statfs() doesn't fail very often. At least
29653 ** not while other file descriptors opened by the same process on
29654 ** the same file are working. */
29655 p->lastErrno = errno;
 
 
 
29656 robust_close(p, fd, __LINE__);
29657 rc = SQLITE_IOERR_ACCESS;
29658 goto open_finished;
29659 }
29660 useProxy = !(fsInfo.f_flags&MNT_LOCAL);
29661 }
29662 if( useProxy ){
29663 rc = fillInUnixFile(pVfs, fd, syncDir, pFile, zPath, noLock,
29664 isDelete, isReadonly);
29665 if( rc==SQLITE_OK ){
29666 rc = proxyTransformUnixFile((unixFile*)pFile, ":auto:");
29667 if( rc!=SQLITE_OK ){
29668 /* Use unixClose to clean up the resources added in fillInUnixFile
@@ -29544,11 +29676,11 @@
29676 goto open_finished;
29677 }
29678 }
29679 #endif
29680
29681 rc = fillInUnixFile(pVfs, fd, syncDir, pFile, zPath, noLock,
29682 isDelete, isReadonly);
29683 open_finished:
29684 if( rc!=SQLITE_OK ){
29685 sqlite3_free(p->pUnused);
29686 }
@@ -29566,17 +29698,17 @@
29698 int dirSync /* If true, fsync() directory after deleting file */
29699 ){
29700 int rc = SQLITE_OK;
29701 UNUSED_PARAMETER(NotUsed);
29702 SimulateIOError(return SQLITE_IOERR_DELETE);
29703 if( osUnlink(zPath)==(-1) && errno!=ENOENT ){
29704 return unixLogError(SQLITE_IOERR_DELETE, "unlink", zPath);
29705 }
29706 #ifndef SQLITE_DISABLE_DIRSYNC
29707 if( dirSync ){
29708 int fd;
29709 rc = osOpenDirectory(zPath, &fd);
29710 if( rc==SQLITE_OK ){
29711 #if OS_VXWORKS
29712 if( fsync(fd)==-1 )
29713 #else
29714 if( fsync(fd) )
@@ -29625,11 +29757,11 @@
29757 assert(!"Invalid flags argument");
29758 }
29759 *pResOut = (osAccess(zPath, amode)==0);
29760 if( flags==SQLITE_ACCESS_EXISTS && *pResOut ){
29761 struct stat buf;
29762 if( 0==osStat(zPath, &buf) && buf.st_size==0 ){
29763 *pResOut = 0;
29764 }
29765 }
29766 return SQLITE_OK;
29767 }
@@ -30144,11 +30276,10 @@
30276 const char *path, /* path for the new unixFile */
30277 unixFile **ppFile, /* unixFile created and returned by ref */
30278 int islockfile /* if non zero missing dirs will be created */
30279 ) {
30280 int fd = -1;
 
30281 unixFile *pNew;
30282 int rc = SQLITE_OK;
30283 int openFlags = O_RDWR | O_CREAT;
30284 sqlite3_vfs dummyVfs;
30285 int terrno = 0;
@@ -30209,11 +30340,11 @@
30340 dummyVfs.zName = "dummy";
30341 pUnused->fd = fd;
30342 pUnused->flags = openFlags;
30343 pNew->pUnused = pUnused;
30344
30345 rc = fillInUnixFile(&dummyVfs, fd, 0, (sqlite3_file*)pNew, path, 0, 0, 0);
30346 if( rc==SQLITE_OK ){
30347 *ppFile = pNew;
30348 return SQLITE_OK;
30349 }
30350 end_create_proxy:
@@ -30323,11 +30454,11 @@
30454 conchFile->openFlags = O_RDWR | O_CREAT;
30455
30456 end_breaklock:
30457 if( rc ){
30458 if( fd>=0 ){
30459 osUnlink(tPath);
30460 robust_close(pFile, fd, __LINE__);
30461 }
30462 fprintf(stderr, "failed to break stale lock on %s, %s\n", cPath, errmsg);
30463 }
30464 return rc;
@@ -31146,11 +31277,11 @@
31277 };
31278 unsigned int i; /* Loop counter */
31279
31280 /* Double-check that the aSyscall[] array has been constructed
31281 ** correctly. See ticket [bb3a86e890c8e96ab] */
31282 assert( ArraySize(aSyscall)==18 );
31283
31284 /* Register all VFSes defined in the aVfs[] array */
31285 for(i=0; i<(sizeof(aVfs)/sizeof(sqlite3_vfs)); i++){
31286 sqlite3_vfs_register(&aVfs[i], i==0);
31287 }
@@ -31263,14 +31394,17 @@
31394 #ifdef MEMORY_DEBUG
31395 # error "The MEMORY_DEBUG macro is obsolete. Use SQLITE_DEBUG instead."
31396 #endif
31397
31398 #ifdef SQLITE_DEBUG
31399 # ifndef SQLITE_DEBUG_OS_TRACE
31400 # define SQLITE_DEBUG_OS_TRACE 0
31401 # endif
31402 int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
31403 # define OSTRACE(X) if( sqlite3OSTrace ) sqlite3DebugPrintf X
31404 #else
31405 # define OSTRACE(X)
31406 #endif
31407
31408 /*
31409 ** Macros for performance tracing. Normally turned off. Only works
31410 ** on i486 hardware.
@@ -31478,12 +31612,13 @@
31612 typedef struct winFile winFile;
31613 struct winFile {
31614 const sqlite3_io_methods *pMethod; /*** Must be first ***/
31615 sqlite3_vfs *pVfs; /* The VFS used to open this file */
31616 HANDLE h; /* Handle for accessing the file */
31617 u8 locktype; /* Type of lock currently held on this file */
31618 short sharedLockByte; /* Randomly chosen byte used as a shared lock */
31619 u8 bPersistWal; /* True to persist WAL files */
31620 DWORD lastErrno; /* The Windows errno from the last I/O error */
31621 DWORD sectorSize; /* Sector size of the device file is on */
31622 winShm *pShm; /* Instance of shared memory on this file */
31623 const char *zPath; /* Full pathname of this file */
31624 int szChunk; /* Chunk size configured by FCNTL_CHUNK_SIZE */
@@ -31777,10 +31912,58 @@
31912 iLine, iErrno, zFunc, zPath, zMsg
31913 );
31914
31915 return errcode;
31916 }
31917
31918 /*
31919 ** The number of times that a ReadFile(), WriteFile(), and DeleteFile()
31920 ** will be retried following a locking error - probably caused by
31921 ** antivirus software. Also the initial delay before the first retry.
31922 ** The delay increases linearly with each retry.
31923 */
31924 #ifndef SQLITE_WIN32_IOERR_RETRY
31925 # define SQLITE_WIN32_IOERR_RETRY 10
31926 #endif
31927 #ifndef SQLITE_WIN32_IOERR_RETRY_DELAY
31928 # define SQLITE_WIN32_IOERR_RETRY_DELAY 25
31929 #endif
31930 static int win32IoerrRetry = SQLITE_WIN32_IOERR_RETRY;
31931 static int win32IoerrRetryDelay = SQLITE_WIN32_IOERR_RETRY_DELAY;
31932
31933 /*
31934 ** If a ReadFile() or WriteFile() error occurs, invoke this routine
31935 ** to see if it should be retried. Return TRUE to retry. Return FALSE
31936 ** to give up with an error.
31937 */
31938 static int retryIoerr(int *pnRetry){
31939 DWORD e;
31940 if( *pnRetry>=win32IoerrRetry ){
31941 return 0;
31942 }
31943 e = GetLastError();
31944 if( e==ERROR_ACCESS_DENIED ||
31945 e==ERROR_LOCK_VIOLATION ||
31946 e==ERROR_SHARING_VIOLATION ){
31947 Sleep(win32IoerrRetryDelay*(1+*pnRetry));
31948 ++*pnRetry;
31949 return 1;
31950 }
31951 return 0;
31952 }
31953
31954 /*
31955 ** Log a I/O error retry episode.
31956 */
31957 static void logIoerr(int nRetry){
31958 if( nRetry ){
31959 sqlite3_log(SQLITE_IOERR,
31960 "delayed %dms for lock/sharing conflict",
31961 win32IoerrRetryDelay*nRetry*(nRetry+1)/2
31962 );
31963 }
31964 }
31965
31966 #if SQLITE_OS_WINCE
31967 /*************************************************************************
31968 ** This section contains code for WinCE only.
31969 */
@@ -32195,22 +32378,25 @@
32378 int amt, /* Number of bytes to read */
32379 sqlite3_int64 offset /* Begin reading at this offset */
32380 ){
32381 winFile *pFile = (winFile*)id; /* file handle */
32382 DWORD nRead; /* Number of bytes actually read from file */
32383 int nRetry = 0; /* Number of retrys */
32384
32385 assert( id!=0 );
32386 SimulateIOError(return SQLITE_IOERR_READ);
32387 OSTRACE(("READ %d lock=%d\n", pFile->h, pFile->locktype));
32388
32389 if( seekWinFile(pFile, offset) ){
32390 return SQLITE_FULL;
32391 }
32392 while( !ReadFile(pFile->h, pBuf, amt, &nRead, 0) ){
32393 if( retryIoerr(&nRetry) ) continue;
32394 pFile->lastErrno = GetLastError();
32395 return winLogError(SQLITE_IOERR_READ, "winRead", pFile->zPath);
32396 }
32397 logIoerr(nRetry);
32398 if( nRead<(DWORD)amt ){
32399 /* Unread parts of the buffer must be zero-filled */
32400 memset(&((char*)pBuf)[nRead], 0, amt-nRead);
32401 return SQLITE_IOERR_SHORT_READ;
32402 }
@@ -32228,10 +32414,11 @@
32414 int amt, /* Number of bytes to write */
32415 sqlite3_int64 offset /* Offset into the file to begin writing at */
32416 ){
32417 int rc; /* True if error has occured, else false */
32418 winFile *pFile = (winFile*)id; /* File handle */
32419 int nRetry = 0; /* Number of retries */
32420
32421 assert( amt>0 );
32422 assert( pFile );
32423 SimulateIOError(return SQLITE_IOERR_WRITE);
32424 SimulateDiskfullError(return SQLITE_FULL);
@@ -32242,11 +32429,16 @@
32429 if( rc==0 ){
32430 u8 *aRem = (u8 *)pBuf; /* Data yet to be written */
32431 int nRem = amt; /* Number of bytes yet to be written */
32432 DWORD nWrite; /* Bytes written by each WriteFile() call */
32433
32434 while( nRem>0 ){
32435 if( !WriteFile(pFile->h, aRem, nRem, &nWrite, 0) ){
32436 if( retryIoerr(&nRetry) ) continue;
32437 break;
32438 }
32439 if( nWrite<=0 ) break;
32440 aRem += nWrite;
32441 nRem -= nWrite;
32442 }
32443 if( nRem>0 ){
32444 pFile->lastErrno = GetLastError();
@@ -32253,14 +32445,17 @@
32445 rc = 1;
32446 }
32447 }
32448
32449 if( rc ){
32450 if( ( pFile->lastErrno==ERROR_HANDLE_DISK_FULL )
32451 || ( pFile->lastErrno==ERROR_DISK_FULL )){
32452 return SQLITE_FULL;
32453 }
32454 return winLogError(SQLITE_IOERR_WRITE, "winWrite", pFile->zPath);
32455 }else{
32456 logIoerr(nRetry);
32457 }
32458 return SQLITE_OK;
32459 }
32460
32461 /*
@@ -32650,32 +32845,56 @@
32845
32846 /*
32847 ** Control and query of the open file handle.
32848 */
32849 static int winFileControl(sqlite3_file *id, int op, void *pArg){
32850 winFile *pFile = (winFile*)id;
32851 switch( op ){
32852 case SQLITE_FCNTL_LOCKSTATE: {
32853 *(int*)pArg = pFile->locktype;
32854 return SQLITE_OK;
32855 }
32856 case SQLITE_LAST_ERRNO: {
32857 *(int*)pArg = (int)pFile->lastErrno;
32858 return SQLITE_OK;
32859 }
32860 case SQLITE_FCNTL_CHUNK_SIZE: {
32861 pFile->szChunk = *(int *)pArg;
32862 return SQLITE_OK;
32863 }
32864 case SQLITE_FCNTL_SIZE_HINT: {
32865 sqlite3_int64 sz = *(sqlite3_int64*)pArg;
32866 SimulateIOErrorBenign(1);
32867 winTruncate(id, sz);
32868 SimulateIOErrorBenign(0);
32869 return SQLITE_OK;
32870 }
32871 case SQLITE_FCNTL_PERSIST_WAL: {
32872 int bPersist = *(int*)pArg;
32873 if( bPersist<0 ){
32874 *(int*)pArg = pFile->bPersistWal;
32875 }else{
32876 pFile->bPersistWal = bPersist!=0;
32877 }
32878 return SQLITE_OK;
32879 }
32880 case SQLITE_FCNTL_SYNC_OMITTED: {
32881 return SQLITE_OK;
32882 }
32883 case SQLITE_FCNTL_WIN32_AV_RETRY: {
32884 int *a = (int*)pArg;
32885 if( a[0]>0 ){
32886 win32IoerrRetry = a[0];
32887 }else{
32888 a[0] = win32IoerrRetry;
32889 }
32890 if( a[1]>0 ){
32891 win32IoerrRetryDelay = a[1];
32892 }else{
32893 a[1] = win32IoerrRetryDelay;
32894 }
32895 return SQLITE_OK;
32896 }
32897 }
32898 return SQLITE_NOTFOUND;
32899 }
32900
@@ -33481,10 +33700,11 @@
33700 int isTemp = 0;
33701 #endif
33702 winFile *pFile = (winFile*)id;
33703 void *zConverted; /* Filename in OS encoding */
33704 const char *zUtf8Name = zName; /* Filename in UTF-8 encoding */
33705 int cnt = 0;
33706
33707 /* If argument zPath is a NULL pointer, this function is required to open
33708 ** a temporary file. Use this buffer to store the file name in.
33709 */
33710 char zTmpname[MAX_PATH+1]; /* Buffer used to create temp filename */
@@ -33600,34 +33820,34 @@
33820 #if SQLITE_OS_WINCE
33821 dwFlagsAndAttributes |= FILE_FLAG_RANDOM_ACCESS;
33822 #endif
33823
33824 if( isNT() ){
33825 while( (h = CreateFileW((WCHAR*)zConverted,
33826 dwDesiredAccess,
33827 dwShareMode, NULL,
33828 dwCreationDisposition,
33829 dwFlagsAndAttributes,
33830 NULL))==INVALID_HANDLE_VALUE &&
33831 retryIoerr(&cnt) ){}
 
33832 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed.
33833 ** Since the ASCII version of these Windows API do not exist for WINCE,
33834 ** it's important to not reference them for WINCE builds.
33835 */
33836 #if SQLITE_OS_WINCE==0
33837 }else{
33838 while( (h = CreateFileA((char*)zConverted,
33839 dwDesiredAccess,
33840 dwShareMode, NULL,
33841 dwCreationDisposition,
33842 dwFlagsAndAttributes,
33843 NULL))==INVALID_HANDLE_VALUE &&
33844 retryIoerr(&cnt) ){}
 
33845 #endif
33846 }
33847
33848 logIoerr(cnt);
33849
33850 OSTRACE(("OPEN %d %s 0x%lx %s\n",
33851 h, zName, dwDesiredAccess,
33852 h==INVALID_HANDLE_VALUE ? "failed" : "ok"));
33853
@@ -33690,19 +33910,17 @@
33910 ** file open, we will be unable to delete it. To work around this
33911 ** problem, we delay 100 milliseconds and try to delete again. Up
33912 ** to MX_DELETION_ATTEMPTs deletion attempts are run before giving
33913 ** up and returning an error.
33914 */
 
33915 static int winDelete(
33916 sqlite3_vfs *pVfs, /* Not used on win32 */
33917 const char *zFilename, /* Name of file to delete */
33918 int syncDir /* Not used on win32 */
33919 ){
33920 int cnt = 0;
33921 int rc;
 
33922 void *zConverted;
33923 UNUSED_PARAMETER(pVfs);
33924 UNUSED_PARAMETER(syncDir);
33925
33926 SimulateIOError(return SQLITE_IOERR_DELETE);
@@ -33709,38 +33927,34 @@
33927 zConverted = convertUtf8Filename(zFilename);
33928 if( zConverted==0 ){
33929 return SQLITE_NOMEM;
33930 }
33931 if( isNT() ){
33932 rc = 1;
33933 while( GetFileAttributesW(zConverted)!=INVALID_FILE_ATTRIBUTES &&
33934 (rc = DeleteFileW(zConverted))==0 && retryIoerr(&cnt) ){}
33935 rc = rc ? SQLITE_OK : SQLITE_ERROR;
 
 
33936 /* isNT() is 1 if SQLITE_OS_WINCE==1, so this else is never executed.
33937 ** Since the ASCII version of these Windows API do not exist for WINCE,
33938 ** it's important to not reference them for WINCE builds.
33939 */
33940 #if SQLITE_OS_WINCE==0
33941 }else{
33942 rc = 1;
33943 while( GetFileAttributesA(zConverted)!=INVALID_FILE_ATTRIBUTES &&
33944 (rc = DeleteFileA(zConverted))==0 && retryIoerr(&cnt) ){}
33945 rc = rc ? SQLITE_OK : SQLITE_ERROR;
 
 
33946 #endif
33947 }
33948 if( rc ){
33949 rc = winLogError(SQLITE_IOERR_DELETE, "winDelete", zFilename);
33950 }else{
33951 logIoerr(cnt);
33952 }
33953 free(zConverted);
33954 OSTRACE(("DELETE \"%s\" %s\n", zFilename, (rc ? "failed" : "ok" )));
33955 return rc;
 
 
 
 
 
33956 }
33957
33958 /*
33959 ** Check the existance and status of a file.
33960 */
@@ -33759,15 +33973,17 @@
33973 zConverted = convertUtf8Filename(zFilename);
33974 if( zConverted==0 ){
33975 return SQLITE_NOMEM;
33976 }
33977 if( isNT() ){
33978 int cnt = 0;
33979 WIN32_FILE_ATTRIBUTE_DATA sAttrData;
33980 memset(&sAttrData, 0, sizeof(sAttrData));
33981 while( !(rc = GetFileAttributesExW((WCHAR*)zConverted,
33982 GetFileExInfoStandard,
33983 &sAttrData)) && retryIoerr(&cnt) ){}
33984 if( rc ){
33985 /* For an SQLITE_ACCESS_EXISTS query, treat a zero-length file
33986 ** as if it does not exist.
33987 */
33988 if( flags==SQLITE_ACCESS_EXISTS
33989 && sAttrData.nFileSizeHigh==0
@@ -33775,10 +33991,11 @@
33991 attr = INVALID_FILE_ATTRIBUTES;
33992 }else{
33993 attr = sAttrData.dwFileAttributes;
33994 }
33995 }else{
33996 logIoerr(cnt);
33997 if( GetLastError()!=ERROR_FILE_NOT_FOUND ){
33998 winLogError(SQLITE_IOERR_ACCESS, "winAccess", zFilename);
33999 free(zConverted);
34000 return SQLITE_IOERR_ACCESS;
34001 }else{
@@ -33799,11 +34016,12 @@
34016 case SQLITE_ACCESS_READ:
34017 case SQLITE_ACCESS_EXISTS:
34018 rc = attr!=INVALID_FILE_ATTRIBUTES;
34019 break;
34020 case SQLITE_ACCESS_READWRITE:
34021 rc = attr!=INVALID_FILE_ATTRIBUTES &&
34022 (attr & FILE_ATTRIBUTE_READONLY)==0;
34023 break;
34024 default:
34025 assert(!"Invalid flags argument");
34026 }
34027 *pResOut = rc;
@@ -35805,11 +36023,11 @@
36023 memset(pCache, 0, sz);
36024 if( separateCache ){
36025 pGroup = (PGroup*)&pCache[1];
36026 pGroup->mxPinned = 10;
36027 }else{
36028 pGroup = &pcache1.grp;
36029 }
36030 pCache->pGroup = pGroup;
36031 pCache->szPage = szPage;
36032 pCache->bPurgeable = (bPurgeable ? 1 : 0);
36033 if( bPurgeable ){
@@ -45383,17 +45601,19 @@
45601 **
45602 ** The EXCLUSIVE lock is not released before returning.
45603 */
45604 rc = sqlite3OsLock(pWal->pDbFd, SQLITE_LOCK_EXCLUSIVE);
45605 if( rc==SQLITE_OK ){
45606 int bPersistWal = -1;
45607 if( pWal->exclusiveMode==WAL_NORMAL_MODE ){
45608 pWal->exclusiveMode = WAL_EXCLUSIVE_MODE;
45609 }
45610 rc = sqlite3WalCheckpoint(
45611 pWal, SQLITE_CHECKPOINT_PASSIVE, 0, 0, sync_flags, nBuf, zBuf, 0, 0
45612 );
45613 sqlite3OsFileControl(pWal->pDbFd, SQLITE_FCNTL_PERSIST_WAL, &bPersistWal);
45614 if( rc==SQLITE_OK && bPersistWal!=1 ){
45615 isDelete = 1;
45616 }
45617 }
45618
45619 walIndexClose(pWal, isDelete);
@@ -48324,10 +48544,12 @@
48544 **
48545 ** This routine works only for pages that do not contain overflow cells.
48546 */
48547 #define findCell(P,I) \
48548 ((P)->aData + ((P)->maskPage & get2byte(&(P)->aData[(P)->cellOffset+2*(I)])))
48549 #define findCellv2(D,M,O,I) (D+(M&get2byte(D+(O+2*(I)))))
48550
48551
48552 /*
48553 ** This a more complex version of findCell() that works for
48554 ** pages that do contain overflow cells.
48555 */
@@ -51918,11 +52140,11 @@
52140 assert( pCur->apPage[pCur->iPage]->nCell==0 );
52141 return SQLITE_OK;
52142 }
52143 assert( pCur->apPage[0]->intKey || pIdxKey );
52144 for(;;){
52145 int lwr, upr, idx;
52146 Pgno chldPg;
52147 MemPage *pPage = pCur->apPage[pCur->iPage];
52148 int c;
52149
52150 /* pPage->nCell must be greater than zero. If this is the root-page
@@ -51934,18 +52156,18 @@
52156 assert( pPage->nCell>0 );
52157 assert( pPage->intKey==(pIdxKey==0) );
52158 lwr = 0;
52159 upr = pPage->nCell-1;
52160 if( biasRight ){
52161 pCur->aiIdx[pCur->iPage] = (u16)(idx = upr);
52162 }else{
52163 pCur->aiIdx[pCur->iPage] = (u16)(idx = (upr+lwr)/2);
52164 }
52165 for(;;){
 
52166 u8 *pCell; /* Pointer to current cell in pPage */
52167
52168 assert( idx==pCur->aiIdx[pCur->iPage] );
52169 pCur->info.nSize = 0;
52170 pCell = findCell(pPage, idx) + pPage->childPtrSize;
52171 if( pPage->intKey ){
52172 i64 nCellKey;
52173 if( pPage->hasData ){
@@ -52024,11 +52246,11 @@
52246 upr = idx-1;
52247 }
52248 if( lwr>upr ){
52249 break;
52250 }
52251 pCur->aiIdx[pCur->iPage] = (u16)(idx = (lwr+upr)/2);
52252 }
52253 assert( lwr==upr+1 );
52254 assert( pPage->isInit );
52255 if( pPage->leaf ){
52256 chldPg = 0;
@@ -52886,13 +53108,13 @@
53108 if( rc ){
53109 *pRC = rc;
53110 return;
53111 }
53112 endPtr = &data[pPage->cellOffset + 2*pPage->nCell - 2];
53113 assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */
53114 while( ptr<endPtr ){
53115 *(u16*)ptr = *(u16*)&ptr[2];
 
53116 ptr += 2;
53117 }
53118 pPage->nCell--;
53119 put2byte(&data[hdr+3], pPage->nCell);
53120 pPage->nFree += 2;
@@ -52929,10 +53151,11 @@
53151 int end; /* First byte past the last cell pointer in data[] */
53152 int ins; /* Index in data[] where new cell pointer is inserted */
53153 int cellOffset; /* Address of first cell pointer in data[] */
53154 u8 *data; /* The content of the whole page */
53155 u8 *ptr; /* Used for moving information around in data[] */
53156 u8 *endPtr; /* End of the loop */
53157
53158 int nSkip = (iChild ? 4 : 0);
53159
53160 if( *pRC ) return;
53161
@@ -52979,13 +53202,16 @@
53202 pPage->nFree -= (u16)(2 + sz);
53203 memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip);
53204 if( iChild ){
53205 put4byte(&data[idx], iChild);
53206 }
53207 ptr = &data[end];
53208 endPtr = &data[ins];
53209 assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */
53210 while( ptr>endPtr ){
53211 *(u16*)ptr = *(u16*)&ptr[-2];
53212 ptr -= 2;
53213 }
53214 put2byte(&data[ins], idx);
53215 put2byte(&data[pPage->hdrOffset+3], pPage->nCell);
53216 #ifndef SQLITE_OMIT_AUTOVACUUM
53217 if( pPage->pBt->autoVacuum ){
@@ -53026,14 +53252,15 @@
53252 assert( get2byteNotZero(&data[hdr+5])==nUsable );
53253
53254 pCellptr = &data[pPage->cellOffset + nCell*2];
53255 cellbody = nUsable;
53256 for(i=nCell-1; i>=0; i--){
53257 u16 sz = aSize[i];
53258 pCellptr -= 2;
53259 cellbody -= sz;
53260 put2byte(pCellptr, cellbody);
53261 memcpy(&data[cellbody], apCell[i], sz);
53262 }
53263 put2byte(&data[hdr+3], nCell);
53264 put2byte(&data[hdr+5], cellbody);
53265 pPage->nFree -= (nCell*2 + nUsable - cellbody);
53266 pPage->nCell = (u16)nCell;
@@ -53483,16 +53710,28 @@
53710 memcpy(pOld, apOld[i], sizeof(MemPage));
53711 pOld->aData = (void*)&pOld[1];
53712 memcpy(pOld->aData, apOld[i]->aData, pBt->pageSize);
53713
53714 limit = pOld->nCell+pOld->nOverflow;
53715 if( pOld->nOverflow>0 ){
53716 for(j=0; j<limit; j++){
53717 assert( nCell<nMaxCells );
53718 apCell[nCell] = findOverflowCell(pOld, j);
53719 szCell[nCell] = cellSizePtr(pOld, apCell[nCell]);
53720 nCell++;
53721 }
53722 }else{
53723 u8 *aData = pOld->aData;
53724 u16 maskPage = pOld->maskPage;
53725 u16 cellOffset = pOld->cellOffset;
53726 for(j=0; j<limit; j++){
53727 assert( nCell<nMaxCells );
53728 apCell[nCell] = findCellv2(aData, maskPage, cellOffset, j);
53729 szCell[nCell] = cellSizePtr(pOld, apCell[nCell]);
53730 nCell++;
53731 }
53732 }
53733 if( i<nOld-1 && !leafData){
53734 u16 sz = (u16)szNew[i];
53735 u8 *pTemp;
53736 assert( nCell<nMaxCells );
53737 szCell[nCell] = sz;
@@ -57353,15 +57592,15 @@
57592 *ppVal = 0;
57593 return SQLITE_OK;
57594 }
57595 op = pExpr->op;
57596
57597 /* op can only be TK_REGISTER if we have compiled with SQLITE_ENABLE_STAT3.
57598 ** The ifdef here is to enable us to achieve 100% branch test coverage even
57599 ** when SQLITE_ENABLE_STAT3 is omitted.
57600 */
57601 #ifdef SQLITE_ENABLE_STAT3
57602 if( op==TK_REGISTER ) op = pExpr->op2;
57603 #else
57604 if( NEVER(op==TK_REGISTER) ) op = pExpr->op2;
57605 #endif
57606
@@ -57637,17 +57876,10 @@
57876 pOp->p1 = p1;
57877 pOp->p2 = p2;
57878 pOp->p3 = p3;
57879 pOp->p4.p = 0;
57880 pOp->p4type = P4_NOTUSED;
 
 
 
 
 
 
 
57881 #ifdef SQLITE_DEBUG
57882 pOp->zComment = 0;
57883 if( sqlite3VdbeAddopTrace ) sqlite3VdbePrintOp(0, i, &p->aOp[i]);
57884 #endif
57885 #ifdef VDBE_PROFILE
@@ -57681,10 +57913,24 @@
57913 ){
57914 int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3);
57915 sqlite3VdbeChangeP4(p, addr, zP4, p4type);
57916 return addr;
57917 }
57918
57919 /*
57920 ** Add an OP_ParseSchema opcode. This routine is broken out from
57921 ** sqlite3VdbeAddOp4() since it needs to also local all btrees.
57922 **
57923 ** The zWhere string must have been obtained from sqlite3_malloc().
57924 ** This routine will take ownership of the allocated memory.
57925 */
57926 SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe *p, int iDb, char *zWhere){
57927 int j;
57928 int addr = sqlite3VdbeAddOp3(p, OP_ParseSchema, iDb, 0, 0);
57929 sqlite3VdbeChangeP4(p, addr, zWhere, P4_DYNAMIC);
57930 for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j);
57931 }
57932
57933 /*
57934 ** Add an opcode that includes the p4 value as an integer.
57935 */
57936 SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(
@@ -58046,12 +58292,12 @@
58292 /*
58293 ** Change the P2 operand of instruction addr so that it points to
58294 ** the address of the next instruction to be coded.
58295 */
58296 SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe *p, int addr){
58297 assert( addr>=0 || p->db->mallocFailed );
58298 if( addr>=0 ) sqlite3VdbeChangeP2(p, addr, p->nOp);
58299 }
58300
58301
58302 /*
58303 ** If the input FuncDef structure is ephemeral, then free it. If
@@ -58979,10 +59225,11 @@
59225 nMem = 10;
59226 }
59227 memset(zCsr, 0, zEnd-zCsr);
59228 zCsr += (zCsr - (u8*)0)&7;
59229 assert( EIGHT_BYTE_ALIGNMENT(zCsr) );
59230 p->expired = 0;
59231
59232 /* Memory for registers, parameters, cursor, etc, is allocated in two
59233 ** passes. On the first pass, we try to reuse unused space at the
59234 ** end of the opcode array. If we are unable to satisfy all memory
59235 ** requirements by reusing the opcode array tail, then the second
@@ -61178,10 +61425,18 @@
61425 rc = db->errCode = p->rc;
61426 }
61427 return (rc&db->errMask);
61428 }
61429
61430 /*
61431 ** The maximum number of times that a statement will try to reparse
61432 ** itself before giving up and returning SQLITE_SCHEMA.
61433 */
61434 #ifndef SQLITE_MAX_SCHEMA_RETRY
61435 # define SQLITE_MAX_SCHEMA_RETRY 5
61436 #endif
61437
61438 /*
61439 ** This is the top-level implementation of sqlite3_step(). Call
61440 ** sqlite3Step() to do most of the work. If a schema error occurs,
61441 ** call sqlite3Reprepare() and try again.
61442 */
@@ -61196,14 +61451,14 @@
61451 return SQLITE_MISUSE_BKPT;
61452 }
61453 db = v->db;
61454 sqlite3_mutex_enter(db->mutex);
61455 while( (rc = sqlite3Step(v))==SQLITE_SCHEMA
61456 && cnt++ < SQLITE_MAX_SCHEMA_RETRY
61457 && (rc2 = rc = sqlite3Reprepare(v))==SQLITE_OK ){
61458 sqlite3_reset(pStmt);
61459 assert( v->expired==0 );
61460 }
61461 if( rc2!=SQLITE_OK && ALWAYS(v->isPrepareV2) && ALWAYS(db->pErr) ){
61462 /* This case occurs after failing to recompile an sql statement.
61463 ** The error message from the SQL compiler has already been loaded
61464 ** into the database handle. This block copies the error message
@@ -63469,11 +63724,11 @@
63724 break;
63725 }
63726
63727 /* Opcode: HaltIfNull P1 P2 P3 P4 *
63728 **
63729 ** Check the value in register P3. If it is NULL then Halt using
63730 ** parameter P1, P2, and P4 as if this were a Halt instruction. If the
63731 ** value in register P3 is not NULL, then this routine is a no-op.
63732 */
63733 case OP_HaltIfNull: { /* in3 */
63734 pIn3 = &aMem[pOp->p3];
@@ -63703,10 +63958,15 @@
63958 assert( memIsValid(pIn1) );
63959 memAboutToChange(p, pOut);
63960 u.ac.zMalloc = pOut->zMalloc;
63961 pOut->zMalloc = 0;
63962 sqlite3VdbeMemMove(pOut, pIn1);
63963 #ifdef SQLITE_DEBUG
63964 if( pOut->pScopyFrom>=&aMem[u.ac.p1] && pOut->pScopyFrom<&aMem[u.ac.p1+pOp->p3] ){
63965 pOut->pScopyFrom += u.ac.p1 - pOp->p2;
63966 }
63967 #endif
63968 pIn1->zMalloc = u.ac.zMalloc;
63969 REGISTER_TRACE(u.ac.p2++, pOut);
63970 pIn1++;
63971 pOut++;
63972 }
@@ -64083,10 +64343,20 @@
64343 u.ag.ctx.pColl = pOp[-1].p4.pColl;
64344 }
64345 db->lastRowid = lastRowid;
64346 (*u.ag.ctx.pFunc->xFunc)(&u.ag.ctx, u.ag.n, u.ag.apVal); /* IMP: R-24505-23230 */
64347 lastRowid = db->lastRowid;
64348
64349 /* If any auxiliary data functions have been called by this user function,
64350 ** immediately call the destructor for any non-static values.
64351 */
64352 if( u.ag.ctx.pVdbeFunc ){
64353 sqlite3VdbeDeleteAuxData(u.ag.ctx.pVdbeFunc, pOp->p1);
64354 pOp->p4.pVdbeFunc = u.ag.ctx.pVdbeFunc;
64355 pOp->p4type = P4_VDBEFUNC;
64356 }
64357
64358 if( db->mallocFailed ){
64359 /* Even though a malloc() has failed, the implementation of the
64360 ** user function may have called an sqlite3_result_XXX() function
64361 ** to return a value. The following call releases any resources
64362 ** associated with such a value.
@@ -64093,19 +64363,10 @@
64363 */
64364 sqlite3VdbeMemRelease(&u.ag.ctx.s);
64365 goto no_mem;
64366 }
64367
 
 
 
 
 
 
 
 
 
64368 /* If the function returned an error, throw an exception */
64369 if( u.ag.ctx.isError ){
64370 sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.ag.ctx.s));
64371 rc = u.ag.ctx.isError;
64372 }
@@ -64405,11 +64666,11 @@
64666 ** additional information.
64667 **
64668 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
64669 ** true or false and is never NULL. If both operands are NULL then the result
64670 ** of comparison is false. If either operand is NULL then the result is true.
64671 ** If neither operand is NULL the result is the same as it would be if
64672 ** the SQLITE_NULLEQ flag were omitted from P5.
64673 */
64674 /* Opcode: Eq P1 P2 P3 P4 P5
64675 **
64676 ** This works just like the Lt opcode except that the jump is taken if
@@ -64417,11 +64678,11 @@
64678 ** See the Lt opcode for additional information.
64679 **
64680 ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either
64681 ** true or false and is never NULL. If both operands are NULL then the result
64682 ** of comparison is true. If either operand is NULL then the result is false.
64683 ** If neither operand is NULL the result is the same as it would be if
64684 ** the SQLITE_NULLEQ flag were omitted from P5.
64685 */
64686 /* Opcode: Le P1 P2 P3 P4 P5
64687 **
64688 ** This works just like the Lt opcode except that the jump is taken if
@@ -64702,17 +64963,17 @@
64963 break;
64964 }
64965
64966 /* Opcode: If P1 P2 P3 * *
64967 **
64968 ** Jump to P2 if the value in register P1 is true. The value
64969 ** is considered true if it is numeric and non-zero. If the value
64970 ** in P1 is NULL then take the jump if P3 is true.
64971 */
64972 /* Opcode: IfNot P1 P2 P3 * *
64973 **
64974 ** Jump to P2 if the value in register P1 is False. The value
64975 ** is considered true if it has a numeric value of zero. If the value
64976 ** in P1 is NULL then take the jump if P3 is true.
64977 */
64978 case OP_If: /* jump, in1 */
64979 case OP_IfNot: { /* jump, in1 */
@@ -65889,11 +66150,11 @@
66150 ** automatically created table with root-page 1 (an BLOB_INTKEY table).
66151 */
66152 if( pOp->p4.pKeyInfo ){
66153 int pgno;
66154 assert( pOp->p4type==P4_KEYINFO );
66155 rc = sqlite3BtreeCreateTable(u.ax.pCx->pBt, &pgno, BTREE_BLOBKEY | pOp->p5);
66156 if( rc==SQLITE_OK ){
66157 assert( pgno==MASTER_ROOT+1 );
66158 rc = sqlite3BtreeCursor(u.ax.pCx->pBt, pgno, 1,
66159 (KeyInfo*)pOp->p4.z, u.ax.pCx->pCursor);
66160 u.ax.pCx->pKeyInfo = pOp->p4.pKeyInfo;
@@ -66362,11 +66623,11 @@
66623 break;
66624 }
66625
66626 /* Opcode: NotExists P1 P2 P3 * *
66627 **
66628 ** Use the content of register P3 as an integer key. If a record
66629 ** with that key does not exist in table of P1, then jump to P2.
66630 ** If the record does exist, then fall through. The cursor is left
66631 ** pointing to the record if it exists.
66632 **
66633 ** The difference between this operation and NotFound is that this
@@ -66440,11 +66701,11 @@
66701 ** written to register P2.
66702 **
66703 ** If P3>0 then P3 is a register in the root frame of this VDBE that holds
66704 ** the largest previously generated record number. No new record numbers are
66705 ** allowed to be less than this value. When this value reaches its maximum,
66706 ** an SQLITE_FULL error is generated. The P3 register is updated with the '
66707 ** generated record number. This P3 mechanism is used to help implement the
66708 ** AUTOINCREMENT feature.
66709 */
66710 case OP_NewRowid: { /* out2-prerelease */
66711 #if 0 /* local variables moved into u.be */
@@ -67082,11 +67343,11 @@
67343 break;
67344 }
67345
67346 /* Opcode: IdxInsert P1 P2 P3 * P5
67347 **
67348 ** Register P2 holds an SQL index key made using the
67349 ** MakeRecord instructions. This opcode writes that key
67350 ** into the index P1. Data for the entry is nil.
67351 **
67352 ** P3 is a flag that provides a hint to the b-tree layer that this
67353 ** insert is likely to be an append.
@@ -70928,15 +71189,29 @@
71189 /* Recursively resolve names in all subqueries
71190 */
71191 for(i=0; i<p->pSrc->nSrc; i++){
71192 struct SrcList_item *pItem = &p->pSrc->a[i];
71193 if( pItem->pSelect ){
71194 NameContext *pNC; /* Used to iterate name contexts */
71195 int nRef = 0; /* Refcount for pOuterNC and outer contexts */
71196 const char *zSavedContext = pParse->zAuthContext;
71197
71198 /* Count the total number of references to pOuterNC and all of its
71199 ** parent contexts. After resolving references to expressions in
71200 ** pItem->pSelect, check if this value has changed. If so, then
71201 ** SELECT statement pItem->pSelect must be correlated. Set the
71202 ** pItem->isCorrelated flag if this is the case. */
71203 for(pNC=pOuterNC; pNC; pNC=pNC->pNext) nRef += pNC->nRef;
71204
71205 if( pItem->zName ) pParse->zAuthContext = pItem->zName;
71206 sqlite3ResolveSelectNames(pParse, pItem->pSelect, pOuterNC);
71207 pParse->zAuthContext = zSavedContext;
71208 if( pParse->nErr || db->mallocFailed ) return WRC_Abort;
71209
71210 for(pNC=pOuterNC; pNC; pNC=pNC->pNext) nRef -= pNC->nRef;
71211 assert( pItem->isCorrelated==0 && nRef<=0 );
71212 pItem->isCorrelated = (nRef!=0);
71213 }
71214 }
71215
71216 /* If there are no aggregate functions in the result-set, and no GROUP BY
71217 ** expression, do not allow aggregates in any of the other expressions.
@@ -72040,10 +72315,11 @@
72315 pNewItem->zName = sqlite3DbStrDup(db, pOldItem->zName);
72316 pNewItem->zAlias = sqlite3DbStrDup(db, pOldItem->zAlias);
72317 pNewItem->jointype = pOldItem->jointype;
72318 pNewItem->iCursor = pOldItem->iCursor;
72319 pNewItem->isPopulated = pOldItem->isPopulated;
72320 pNewItem->isCorrelated = pOldItem->isCorrelated;
72321 pNewItem->zIndex = sqlite3DbStrDup(db, pOldItem->zIndex);
72322 pNewItem->notIndexed = pOldItem->notIndexed;
72323 pNewItem->pIndex = pOldItem->pIndex;
72324 pTab = pNewItem->pTab = pOldItem->pTab;
72325 if( pTab ){
@@ -75255,18 +75531,18 @@
75531 sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
75532
75533 /* Reload the table, index and permanent trigger schemas. */
75534 zWhere = sqlite3MPrintf(pParse->db, "tbl_name=%Q", zName);
75535 if( !zWhere ) return;
75536 sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere);
75537
75538 #ifndef SQLITE_OMIT_TRIGGER
75539 /* Now, if the table is not stored in the temp database, reload any temp
75540 ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined.
75541 */
75542 if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){
75543 sqlite3VdbeAddParseSchemaOp(v, 1, zWhere);
75544 }
75545 #endif
75546 }
75547
75548 /*
@@ -75736,10 +76012,100 @@
76012 ** May you find forgiveness for yourself and forgive others.
76013 ** May you share freely, never taking more than you give.
76014 **
76015 *************************************************************************
76016 ** This file contains code associated with the ANALYZE command.
76017 **
76018 ** The ANALYZE command gather statistics about the content of tables
76019 ** and indices. These statistics are made available to the query planner
76020 ** to help it make better decisions about how to perform queries.
76021 **
76022 ** The following system tables are or have been supported:
76023 **
76024 ** CREATE TABLE sqlite_stat1(tbl, idx, stat);
76025 ** CREATE TABLE sqlite_stat2(tbl, idx, sampleno, sample);
76026 ** CREATE TABLE sqlite_stat3(tbl, idx, nEq, nLt, nDLt, sample);
76027 **
76028 ** Additional tables might be added in future releases of SQLite.
76029 ** The sqlite_stat2 table is not created or used unless the SQLite version
76030 ** is between 3.6.18 and 3.7.7, inclusive, and unless SQLite is compiled
76031 ** with SQLITE_ENABLE_STAT2. The sqlite_stat2 table is deprecated.
76032 ** The sqlite_stat2 table is superceded by sqlite_stat3, which is only
76033 ** created and used by SQLite versions after 2011-08-09 with
76034 ** SQLITE_ENABLE_STAT3 defined. The fucntionality of sqlite_stat3
76035 ** is a superset of sqlite_stat2.
76036 **
76037 ** Format of sqlite_stat1:
76038 **
76039 ** There is normally one row per index, with the index identified by the
76040 ** name in the idx column. The tbl column is the name of the table to
76041 ** which the index belongs. In each such row, the stat column will be
76042 ** a string consisting of a list of integers. The first integer in this
76043 ** list is the number of rows in the index and in the table. The second
76044 ** integer is the average number of rows in the index that have the same
76045 ** value in the first column of the index. The third integer is the average
76046 ** number of rows in the index that have the same value for the first two
76047 ** columns. The N-th integer (for N>1) is the average number of rows in
76048 ** the index which have the same value for the first N-1 columns. For
76049 ** a K-column index, there will be K+1 integers in the stat column. If
76050 ** the index is unique, then the last integer will be 1.
76051 **
76052 ** The list of integers in the stat column can optionally be followed
76053 ** by the keyword "unordered". The "unordered" keyword, if it is present,
76054 ** must be separated from the last integer by a single space. If the
76055 ** "unordered" keyword is present, then the query planner assumes that
76056 ** the index is unordered and will not use the index for a range query.
76057 **
76058 ** If the sqlite_stat1.idx column is NULL, then the sqlite_stat1.stat
76059 ** column contains a single integer which is the (estimated) number of
76060 ** rows in the table identified by sqlite_stat1.tbl.
76061 **
76062 ** Format of sqlite_stat2:
76063 **
76064 ** The sqlite_stat2 is only created and is only used if SQLite is compiled
76065 ** with SQLITE_ENABLE_STAT2 and if the SQLite version number is between
76066 ** 3.6.18 and 3.7.7. The "stat2" table contains additional information
76067 ** about the distribution of keys within an index. The index is identified by
76068 ** the "idx" column and the "tbl" column is the name of the table to which
76069 ** the index belongs. There are usually 10 rows in the sqlite_stat2
76070 ** table for each index.
76071 **
76072 ** The sqlite_stat2 entries for an index that have sampleno between 0 and 9
76073 ** inclusive are samples of the left-most key value in the index taken at
76074 ** evenly spaced points along the index. Let the number of samples be S
76075 ** (10 in the standard build) and let C be the number of rows in the index.
76076 ** Then the sampled rows are given by:
76077 **
76078 ** rownumber = (i*C*2 + C)/(S*2)
76079 **
76080 ** For i between 0 and S-1. Conceptually, the index space is divided into
76081 ** S uniform buckets and the samples are the middle row from each bucket.
76082 **
76083 ** The format for sqlite_stat2 is recorded here for legacy reference. This
76084 ** version of SQLite does not support sqlite_stat2. It neither reads nor
76085 ** writes the sqlite_stat2 table. This version of SQLite only supports
76086 ** sqlite_stat3.
76087 **
76088 ** Format for sqlite_stat3:
76089 **
76090 ** The sqlite_stat3 is an enhancement to sqlite_stat2. A new name is
76091 ** used to avoid compatibility problems.
76092 **
76093 ** The format of the sqlite_stat3 table is similar to the format for
76094 ** the sqlite_stat2 table, with the following changes: (1)
76095 ** The sampleno column is removed. (2) Every sample has nEq, nLt, and nDLt
76096 ** columns which hold the approximate number of rows in the table that
76097 ** exactly match the sample, the approximate number of rows with values
76098 ** less than the sample, and the approximate number of distinct key values
76099 ** less than the sample, respectively. (3) The number of samples can vary
76100 ** from one table to the next; the sample count does not have to be
76101 ** exactly 10 as it is with sqlite_stat2.
76102 **
76103 ** The ANALYZE command will typically generate sqlite_stat3 tables
76104 ** that contain between 10 and 40 samples which are distributed across
76105 ** the key space, though not uniformly, and which include samples with
76106 ** largest possible nEq values.
76107 */
76108 #ifndef SQLITE_OMIT_ANALYZE
76109
76110 /*
76111 ** This routine generates code that opens the sqlite_stat1 table for
@@ -75767,12 +76133,18 @@
76133 static const struct {
76134 const char *zName;
76135 const char *zCols;
76136 } aTable[] = {
76137 { "sqlite_stat1", "tbl,idx,stat" },
76138 #ifdef SQLITE_ENABLE_STAT3
76139 { "sqlite_stat3", "tbl,idx,neq,nlt,ndlt,sample" },
76140 #endif
76141 };
76142 static const char *azToDrop[] = {
76143 "sqlite_stat2",
76144 #ifndef SQLITE_ENABLE_STAT3
76145 "sqlite_stat3",
76146 #endif
76147 };
76148
76149 int aRoot[] = {0, 0};
76150 u8 aCreateTbl[] = {0, 0};
@@ -75784,10 +76156,21 @@
76156 if( v==0 ) return;
76157 assert( sqlite3BtreeHoldsAllMutexes(db) );
76158 assert( sqlite3VdbeDb(v)==db );
76159 pDb = &db->aDb[iDb];
76160
76161 /* Drop all statistics tables that this version of SQLite does not
76162 ** understand.
76163 */
76164 for(i=0; i<ArraySize(azToDrop); i++){
76165 Table *pTab = sqlite3FindTable(db, azToDrop[i], pDb->zName);
76166 if( pTab ) sqlite3CodeDropTable(pParse, pTab, iDb, 0);
76167 }
76168
76169 /* Create new statistic tables if they do not exist, or clear them
76170 ** if they do already exist.
76171 */
76172 for(i=0; i<ArraySize(aTable); i++){
76173 const char *zTab = aTable[i].zName;
76174 Table *pStat;
76175 if( (pStat = sqlite3FindTable(db, zTab, pDb->zName))==0 ){
76176 /* The sqlite_stat[12] table does not exist. Create it. Note that a
@@ -75814,17 +76197,238 @@
76197 sqlite3VdbeAddOp2(v, OP_Clear, aRoot[i], iDb);
76198 }
76199 }
76200 }
76201
76202 /* Open the sqlite_stat[13] tables for writing. */
76203 for(i=0; i<ArraySize(aTable); i++){
76204 sqlite3VdbeAddOp3(v, OP_OpenWrite, iStatCur+i, aRoot[i], iDb);
76205 sqlite3VdbeChangeP4(v, -1, (char *)3, P4_INT32);
76206 sqlite3VdbeChangeP5(v, aCreateTbl[i]);
76207 }
76208 }
76209
76210 /*
76211 ** Recommended number of samples for sqlite_stat3
76212 */
76213 #ifndef SQLITE_STAT3_SAMPLES
76214 # define SQLITE_STAT3_SAMPLES 24
76215 #endif
76216
76217 /*
76218 ** Three SQL functions - stat3_init(), stat3_push(), and stat3_pop() -
76219 ** share an instance of the following structure to hold their state
76220 ** information.
76221 */
76222 typedef struct Stat3Accum Stat3Accum;
76223 struct Stat3Accum {
76224 tRowcnt nRow; /* Number of rows in the entire table */
76225 tRowcnt nPSample; /* How often to do a periodic sample */
76226 int iMin; /* Index of entry with minimum nEq and hash */
76227 int mxSample; /* Maximum number of samples to accumulate */
76228 int nSample; /* Current number of samples */
76229 u32 iPrn; /* Pseudo-random number used for sampling */
76230 struct Stat3Sample {
76231 i64 iRowid; /* Rowid in main table of the key */
76232 tRowcnt nEq; /* sqlite_stat3.nEq */
76233 tRowcnt nLt; /* sqlite_stat3.nLt */
76234 tRowcnt nDLt; /* sqlite_stat3.nDLt */
76235 u8 isPSample; /* True if a periodic sample */
76236 u32 iHash; /* Tiebreaker hash */
76237 } *a; /* An array of samples */
76238 };
76239
76240 #ifdef SQLITE_ENABLE_STAT3
76241 /*
76242 ** Implementation of the stat3_init(C,S) SQL function. The two parameters
76243 ** are the number of rows in the table or index (C) and the number of samples
76244 ** to accumulate (S).
76245 **
76246 ** This routine allocates the Stat3Accum object.
76247 **
76248 ** The return value is the Stat3Accum object (P).
76249 */
76250 static void stat3Init(
76251 sqlite3_context *context,
76252 int argc,
76253 sqlite3_value **argv
76254 ){
76255 Stat3Accum *p;
76256 tRowcnt nRow;
76257 int mxSample;
76258 int n;
76259
76260 UNUSED_PARAMETER(argc);
76261 nRow = (tRowcnt)sqlite3_value_int64(argv[0]);
76262 mxSample = sqlite3_value_int(argv[1]);
76263 n = sizeof(*p) + sizeof(p->a[0])*mxSample;
76264 p = sqlite3_malloc( n );
76265 if( p==0 ){
76266 sqlite3_result_error_nomem(context);
76267 return;
76268 }
76269 memset(p, 0, n);
76270 p->a = (struct Stat3Sample*)&p[1];
76271 p->nRow = nRow;
76272 p->mxSample = mxSample;
76273 p->nPSample = p->nRow/(mxSample/3+1) + 1;
76274 sqlite3_randomness(sizeof(p->iPrn), &p->iPrn);
76275 sqlite3_result_blob(context, p, sizeof(p), sqlite3_free);
76276 }
76277 static const FuncDef stat3InitFuncdef = {
76278 2, /* nArg */
76279 SQLITE_UTF8, /* iPrefEnc */
76280 0, /* flags */
76281 0, /* pUserData */
76282 0, /* pNext */
76283 stat3Init, /* xFunc */
76284 0, /* xStep */
76285 0, /* xFinalize */
76286 "stat3_init", /* zName */
76287 0, /* pHash */
76288 0 /* pDestructor */
76289 };
76290
76291
76292 /*
76293 ** Implementation of the stat3_push(nEq,nLt,nDLt,rowid,P) SQL function. The
76294 ** arguments describe a single key instance. This routine makes the
76295 ** decision about whether or not to retain this key for the sqlite_stat3
76296 ** table.
76297 **
76298 ** The return value is NULL.
76299 */
76300 static void stat3Push(
76301 sqlite3_context *context,
76302 int argc,
76303 sqlite3_value **argv
76304 ){
76305 Stat3Accum *p = (Stat3Accum*)sqlite3_value_blob(argv[4]);
76306 tRowcnt nEq = sqlite3_value_int64(argv[0]);
76307 tRowcnt nLt = sqlite3_value_int64(argv[1]);
76308 tRowcnt nDLt = sqlite3_value_int64(argv[2]);
76309 i64 rowid = sqlite3_value_int64(argv[3]);
76310 u8 isPSample = 0;
76311 u8 doInsert = 0;
76312 int iMin = p->iMin;
76313 struct Stat3Sample *pSample;
76314 int i;
76315 u32 h;
76316
76317 UNUSED_PARAMETER(context);
76318 UNUSED_PARAMETER(argc);
76319 if( nEq==0 ) return;
76320 h = p->iPrn = p->iPrn*1103515245 + 12345;
76321 if( (nLt/p->nPSample)!=((nEq+nLt)/p->nPSample) ){
76322 doInsert = isPSample = 1;
76323 }else if( p->nSample<p->mxSample ){
76324 doInsert = 1;
76325 }else{
76326 if( nEq>p->a[iMin].nEq || (nEq==p->a[iMin].nEq && h>p->a[iMin].iHash) ){
76327 doInsert = 1;
76328 }
76329 }
76330 if( !doInsert ) return;
76331 if( p->nSample==p->mxSample ){
76332 if( iMin<p->nSample ){
76333 memcpy(&p->a[iMin], &p->a[iMin+1], sizeof(p->a[0])*(p->nSample-iMin));
76334 }
76335 pSample = &p->a[p->nSample-1];
76336 }else{
76337 pSample = &p->a[p->nSample++];
76338 }
76339 pSample->iRowid = rowid;
76340 pSample->nEq = nEq;
76341 pSample->nLt = nLt;
76342 pSample->nDLt = nDLt;
76343 pSample->iHash = h;
76344 pSample->isPSample = isPSample;
76345
76346 /* Find the new minimum */
76347 if( p->nSample==p->mxSample ){
76348 pSample = p->a;
76349 i = 0;
76350 while( pSample->isPSample ){
76351 i++;
76352 pSample++;
76353 assert( i<p->nSample );
76354 }
76355 nEq = pSample->nEq;
76356 h = pSample->iHash;
76357 iMin = i;
76358 for(i++, pSample++; i<p->nSample; i++, pSample++){
76359 if( pSample->isPSample ) continue;
76360 if( pSample->nEq<nEq
76361 || (pSample->nEq==nEq && pSample->iHash<h)
76362 ){
76363 iMin = i;
76364 nEq = pSample->nEq;
76365 h = pSample->iHash;
76366 }
76367 }
76368 p->iMin = iMin;
76369 }
76370 }
76371 static const FuncDef stat3PushFuncdef = {
76372 5, /* nArg */
76373 SQLITE_UTF8, /* iPrefEnc */
76374 0, /* flags */
76375 0, /* pUserData */
76376 0, /* pNext */
76377 stat3Push, /* xFunc */
76378 0, /* xStep */
76379 0, /* xFinalize */
76380 "stat3_push", /* zName */
76381 0, /* pHash */
76382 0 /* pDestructor */
76383 };
76384
76385 /*
76386 ** Implementation of the stat3_get(P,N,...) SQL function. This routine is
76387 ** used to query the results. Content is returned for the Nth sqlite_stat3
76388 ** row where N is between 0 and S-1 and S is the number of samples. The
76389 ** value returned depends on the number of arguments.
76390 **
76391 ** argc==2 result: rowid
76392 ** argc==3 result: nEq
76393 ** argc==4 result: nLt
76394 ** argc==5 result: nDLt
76395 */
76396 static void stat3Get(
76397 sqlite3_context *context,
76398 int argc,
76399 sqlite3_value **argv
76400 ){
76401 int n = sqlite3_value_int(argv[1]);
76402 Stat3Accum *p = (Stat3Accum*)sqlite3_value_blob(argv[0]);
76403
76404 assert( p!=0 );
76405 if( p->nSample<=n ) return;
76406 switch( argc ){
76407 case 2: sqlite3_result_int64(context, p->a[n].iRowid); break;
76408 case 3: sqlite3_result_int64(context, p->a[n].nEq); break;
76409 case 4: sqlite3_result_int64(context, p->a[n].nLt); break;
76410 case 5: sqlite3_result_int64(context, p->a[n].nDLt); break;
76411 }
76412 }
76413 static const FuncDef stat3GetFuncdef = {
76414 -1, /* nArg */
76415 SQLITE_UTF8, /* iPrefEnc */
76416 0, /* flags */
76417 0, /* pUserData */
76418 0, /* pNext */
76419 stat3Get, /* xFunc */
76420 0, /* xStep */
76421 0, /* xFinalize */
76422 "stat3_get", /* zName */
76423 0, /* pHash */
76424 0 /* pDestructor */
76425 };
76426 #endif /* SQLITE_ENABLE_STAT3 */
76427
76428
76429
76430
76431 /*
76432 ** Generate code to do an analysis of all indices associated with
76433 ** a single table.
76434 */
@@ -75844,24 +76448,31 @@
76448 int endOfLoop; /* The end of the loop */
76449 int jZeroRows = -1; /* Jump from here if number of rows is zero */
76450 int iDb; /* Index of database containing pTab */
76451 int regTabname = iMem++; /* Register containing table name */
76452 int regIdxname = iMem++; /* Register containing index name */
76453 int regStat1 = iMem++; /* The stat column of sqlite_stat1 */
76454 #ifdef SQLITE_ENABLE_STAT3
76455 int regNumEq = regStat1; /* Number of instances. Same as regStat1 */
76456 int regNumLt = iMem++; /* Number of keys less than regSample */
76457 int regNumDLt = iMem++; /* Number of distinct keys less than regSample */
76458 int regSample = iMem++; /* The next sample value */
76459 int regRowid = regSample; /* Rowid of a sample */
76460 int regAccum = iMem++; /* Register to hold Stat3Accum object */
76461 int regLoop = iMem++; /* Loop counter */
76462 int regCount = iMem++; /* Number of rows in the table or index */
76463 int regTemp1 = iMem++; /* Intermediate register */
76464 int regTemp2 = iMem++; /* Intermediate register */
76465 int once = 1; /* One-time initialization */
76466 int shortJump = 0; /* Instruction address */
76467 int iTabCur = pParse->nTab++; /* Table cursor */
76468 #endif
76469 int regCol = iMem++; /* Content of a column in analyzed table */
76470 int regRec = iMem++; /* Register holding completed record */
76471 int regTemp = iMem++; /* Temporary use register */
76472 int regNewRowid = iMem++; /* Rowid for the inserted record */
76473
 
 
 
 
 
 
 
 
76474
76475 v = sqlite3GetVdbe(pParse);
76476 if( v==0 || NEVER(pTab==0) ){
76477 return;
76478 }
@@ -75890,17 +76501,22 @@
76501 iIdxCur = pParse->nTab++;
76502 sqlite3VdbeAddOp4(v, OP_String8, 0, regTabname, 0, pTab->zName, 0);
76503 for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
76504 int nCol;
76505 KeyInfo *pKey;
76506 int addrIfNot = 0; /* address of OP_IfNot */
76507 int *aChngAddr; /* Array of jump instruction addresses */
76508
76509 if( pOnlyIdx && pOnlyIdx!=pIdx ) continue;
76510 VdbeNoopComment((v, "Begin analysis of %s", pIdx->zName));
76511 nCol = pIdx->nColumn;
76512 pKey = sqlite3IndexKeyinfo(pParse, pIdx);
76513 if( iMem+1+(nCol*2)>pParse->nMem ){
76514 pParse->nMem = iMem+1+(nCol*2);
76515 }
76516 aChngAddr = sqlite3DbMallocRaw(db, sizeof(int)*pIdx->nColumn);
76517 if( aChngAddr==0 ) continue;
76518
76519 /* Open a cursor to the index to be analyzed. */
76520 assert( iDb==sqlite3SchemaToIndex(db, pIdx->pSchema) );
76521 sqlite3VdbeAddOp4(v, OP_OpenRead, iIdxCur, pIdx->tnum, iDb,
76522 (char *)pKey, P4_KEYINFO_HANDOFF);
@@ -75907,35 +76523,24 @@
76523 VdbeComment((v, "%s", pIdx->zName));
76524
76525 /* Populate the register containing the index name. */
76526 sqlite3VdbeAddOp4(v, OP_String8, 0, regIdxname, 0, pIdx->zName, 0);
76527
76528 #ifdef SQLITE_ENABLE_STAT3
76529 if( once ){
76530 once = 0;
76531 sqlite3OpenTable(pParse, iTabCur, iDb, pTab, OP_OpenRead);
76532 }
76533 sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regCount);
76534 sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_STAT3_SAMPLES, regTemp1);
76535 sqlite3VdbeAddOp2(v, OP_Integer, 0, regNumEq);
76536 sqlite3VdbeAddOp2(v, OP_Integer, 0, regNumLt);
76537 sqlite3VdbeAddOp2(v, OP_Integer, -1, regNumDLt);
76538 sqlite3VdbeAddOp4(v, OP_Function, 1, regCount, regAccum,
76539 (char*)&stat3InitFuncdef, P4_FUNCDEF);
76540 sqlite3VdbeChangeP5(v, 2);
76541 #endif /* SQLITE_ENABLE_STAT3 */
 
 
 
 
 
 
 
 
 
 
 
76542
76543 /* The block of memory cells initialized here is used as follows.
76544 **
76545 ** iMem:
76546 ** The total number of rows in the table.
@@ -75961,79 +76566,87 @@
76566 /* Start the analysis loop. This loop runs through all the entries in
76567 ** the index b-tree. */
76568 endOfLoop = sqlite3VdbeMakeLabel(v);
76569 sqlite3VdbeAddOp2(v, OP_Rewind, iIdxCur, endOfLoop);
76570 topOfLoop = sqlite3VdbeCurrentAddr(v);
76571 sqlite3VdbeAddOp2(v, OP_AddImm, iMem, 1); /* Increment row counter */
76572
76573 for(i=0; i<nCol; i++){
76574 CollSeq *pColl;
76575 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regCol);
76576 if( i==0 ){
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76577 /* Always record the very first row */
76578 addrIfNot = sqlite3VdbeAddOp1(v, OP_IfNot, iMem+1);
76579 }
76580 assert( pIdx->azColl!=0 );
76581 assert( pIdx->azColl[i]!=0 );
76582 pColl = sqlite3LocateCollSeq(pParse, pIdx->azColl[i]);
76583 aChngAddr[i] = sqlite3VdbeAddOp4(v, OP_Ne, regCol, 0, iMem+nCol+i+1,
76584 (char*)pColl, P4_COLLSEQ);
76585 sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
76586 VdbeComment((v, "jump if column %d changed", i));
76587 #ifdef SQLITE_ENABLE_STAT3
76588 if( i==0 ){
76589 sqlite3VdbeAddOp2(v, OP_AddImm, regNumEq, 1);
76590 VdbeComment((v, "incr repeat count"));
76591 }
76592 #endif
76593 }
76594 sqlite3VdbeAddOp2(v, OP_Goto, 0, endOfLoop);
76595 for(i=0; i<nCol; i++){
76596 sqlite3VdbeJumpHere(v, aChngAddr[i]); /* Set jump dest for the OP_Ne */
76597 if( i==0 ){
76598 sqlite3VdbeJumpHere(v, addrIfNot); /* Jump dest for OP_IfNot */
76599 #ifdef SQLITE_ENABLE_STAT3
76600 sqlite3VdbeAddOp4(v, OP_Function, 1, regNumEq, regTemp2,
76601 (char*)&stat3PushFuncdef, P4_FUNCDEF);
76602 sqlite3VdbeChangeP5(v, 5);
76603 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, pIdx->nColumn, regRowid);
76604 sqlite3VdbeAddOp3(v, OP_Add, regNumEq, regNumLt, regNumLt);
76605 sqlite3VdbeAddOp2(v, OP_AddImm, regNumDLt, 1);
76606 sqlite3VdbeAddOp2(v, OP_Integer, 1, regNumEq);
76607 #endif
76608 }
 
76609 sqlite3VdbeAddOp2(v, OP_AddImm, iMem+i+1, 1);
76610 sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, iMem+nCol+i+1);
76611 }
76612 sqlite3DbFree(db, aChngAddr);
76613
76614 /* Always jump here after updating the iMem+1...iMem+1+nCol counters */
76615 sqlite3VdbeResolveLabel(v, endOfLoop);
76616
76617 sqlite3VdbeAddOp2(v, OP_Next, iIdxCur, topOfLoop);
76618 sqlite3VdbeAddOp1(v, OP_Close, iIdxCur);
76619 #ifdef SQLITE_ENABLE_STAT3
76620 sqlite3VdbeAddOp4(v, OP_Function, 1, regNumEq, regTemp2,
76621 (char*)&stat3PushFuncdef, P4_FUNCDEF);
76622 sqlite3VdbeChangeP5(v, 5);
76623 sqlite3VdbeAddOp2(v, OP_Integer, -1, regLoop);
76624 shortJump =
76625 sqlite3VdbeAddOp2(v, OP_AddImm, regLoop, 1);
76626 sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regTemp1,
76627 (char*)&stat3GetFuncdef, P4_FUNCDEF);
76628 sqlite3VdbeChangeP5(v, 2);
76629 sqlite3VdbeAddOp1(v, OP_IsNull, regTemp1);
76630 sqlite3VdbeAddOp3(v, OP_NotExists, iTabCur, shortJump, regTemp1);
76631 sqlite3VdbeAddOp3(v, OP_Column, iTabCur, pIdx->aiColumn[0], regSample);
76632 sqlite3ColumnDefault(v, pTab, pIdx->aiColumn[0], regSample);
76633 sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumEq,
76634 (char*)&stat3GetFuncdef, P4_FUNCDEF);
76635 sqlite3VdbeChangeP5(v, 3);
76636 sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumLt,
76637 (char*)&stat3GetFuncdef, P4_FUNCDEF);
76638 sqlite3VdbeChangeP5(v, 4);
76639 sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumDLt,
76640 (char*)&stat3GetFuncdef, P4_FUNCDEF);
76641 sqlite3VdbeChangeP5(v, 5);
76642 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 6, regRec, "bbbbbb", 0);
76643 sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur+1, regNewRowid);
76644 sqlite3VdbeAddOp3(v, OP_Insert, iStatCur+1, regRec, regNewRowid);
76645 sqlite3VdbeAddOp2(v, OP_Goto, 0, shortJump);
76646 sqlite3VdbeJumpHere(v, shortJump+2);
76647 #endif
76648
76649 /* Store the results in sqlite_stat1.
76650 **
76651 ** The result is a single row of the sqlite_stat1 table. The first
76652 ** two columns are the names of the table and index. The third column
@@ -76049,50 +76662,51 @@
76662 **
76663 ** If K==0 then no entry is made into the sqlite_stat1 table.
76664 ** If K>0 then it is always the case the D>0 so division by zero
76665 ** is never possible.
76666 */
76667 sqlite3VdbeAddOp2(v, OP_SCopy, iMem, regStat1);
76668 if( jZeroRows<0 ){
76669 jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, iMem);
76670 }
76671 for(i=0; i<nCol; i++){
76672 sqlite3VdbeAddOp4(v, OP_String8, 0, regTemp, 0, " ", 0);
76673 sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regStat1, regStat1);
76674 sqlite3VdbeAddOp3(v, OP_Add, iMem, iMem+i+1, regTemp);
76675 sqlite3VdbeAddOp2(v, OP_AddImm, regTemp, -1);
76676 sqlite3VdbeAddOp3(v, OP_Divide, iMem+i+1, regTemp, regTemp);
76677 sqlite3VdbeAddOp1(v, OP_ToInt, regTemp);
76678 sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regStat1, regStat1);
76679 }
76680 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0);
76681 sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regNewRowid);
76682 sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regNewRowid);
76683 sqlite3VdbeChangeP5(v, OPFLAG_APPEND);
76684 }
76685
76686 /* If the table has no indices, create a single sqlite_stat1 entry
76687 ** containing NULL as the index name and the row count as the content.
76688 */
76689 if( pTab->pIndex==0 ){
76690 sqlite3VdbeAddOp3(v, OP_OpenRead, iIdxCur, pTab->tnum, iDb);
76691 VdbeComment((v, "%s", pTab->zName));
76692 sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regStat1);
76693 sqlite3VdbeAddOp1(v, OP_Close, iIdxCur);
76694 jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, regStat1);
76695 }else{
76696 sqlite3VdbeJumpHere(v, jZeroRows);
76697 jZeroRows = sqlite3VdbeAddOp0(v, OP_Goto);
76698 }
76699 sqlite3VdbeAddOp2(v, OP_Null, 0, regIdxname);
76700 sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0);
76701 sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regNewRowid);
76702 sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regNewRowid);
76703 sqlite3VdbeChangeP5(v, OPFLAG_APPEND);
76704 if( pParse->nMem<regRec ) pParse->nMem = regRec;
76705 sqlite3VdbeJumpHere(v, jZeroRows);
76706 }
76707
76708
76709 /*
76710 ** Generate code that will cause the most recent index analysis to
76711 ** be loaded into internal hash tables where is can be used.
76712 */
@@ -76113,11 +76727,11 @@
76727 int iStatCur;
76728 int iMem;
76729
76730 sqlite3BeginWriteOperation(pParse, 0, iDb);
76731 iStatCur = pParse->nTab;
76732 pParse->nTab += 3;
76733 openStatTable(pParse, iDb, iStatCur, 0, 0);
76734 iMem = pParse->nMem+1;
76735 assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
76736 for(k=sqliteHashFirst(&pSchema->tblHash); k; k=sqliteHashNext(k)){
76737 Table *pTab = (Table*)sqliteHashData(k);
@@ -76138,11 +76752,11 @@
76752 assert( pTab!=0 );
76753 assert( sqlite3BtreeHoldsAllMutexes(pParse->db) );
76754 iDb = sqlite3SchemaToIndex(pParse->db, pTab->pSchema);
76755 sqlite3BeginWriteOperation(pParse, 0, iDb);
76756 iStatCur = pParse->nTab;
76757 pParse->nTab += 3;
76758 if( pOnlyIdx ){
76759 openStatTable(pParse, iDb, iStatCur, pOnlyIdx->zName, "idx");
76760 }else{
76761 openStatTable(pParse, iDb, iStatCur, pTab->zName, "tbl");
76762 }
@@ -76243,11 +76857,11 @@
76857 static int analysisLoader(void *pData, int argc, char **argv, char **NotUsed){
76858 analysisInfo *pInfo = (analysisInfo*)pData;
76859 Index *pIndex;
76860 Table *pTable;
76861 int i, c, n;
76862 tRowcnt v;
76863 const char *z;
76864
76865 assert( argc==3 );
76866 UNUSED_PARAMETER2(NotUsed, argc);
76867
@@ -76286,40 +76900,172 @@
76900 /*
76901 ** If the Index.aSample variable is not NULL, delete the aSample[] array
76902 ** and its contents.
76903 */
76904 SQLITE_PRIVATE void sqlite3DeleteIndexSamples(sqlite3 *db, Index *pIdx){
76905 #ifdef SQLITE_ENABLE_STAT3
76906 if( pIdx->aSample ){
76907 int j;
76908 for(j=0; j<pIdx->nSample; j++){
76909 IndexSample *p = &pIdx->aSample[j];
76910 if( p->eType==SQLITE_TEXT || p->eType==SQLITE_BLOB ){
76911 sqlite3_free(p->u.z);
76912 }
76913 }
76914 sqlite3_free(pIdx->aSample);
76915 }
76916 UNUSED_PARAMETER(db);
76917 pIdx->nSample = 0;
76918 pIdx->aSample = 0;
76919 #else
76920 UNUSED_PARAMETER(db);
76921 UNUSED_PARAMETER(pIdx);
76922 #endif
76923 }
76924
76925 #ifdef SQLITE_ENABLE_STAT3
76926 /*
76927 ** Load content from the sqlite_stat3 table into the Index.aSample[]
76928 ** arrays of all indices.
76929 */
76930 static int loadStat3(sqlite3 *db, const char *zDb){
76931 int rc; /* Result codes from subroutines */
76932 sqlite3_stmt *pStmt = 0; /* An SQL statement being run */
76933 char *zSql; /* Text of the SQL statement */
76934 Index *pPrevIdx = 0; /* Previous index in the loop */
76935 int idx = 0; /* slot in pIdx->aSample[] for next sample */
76936 int eType; /* Datatype of a sample */
76937 IndexSample *pSample; /* A slot in pIdx->aSample[] */
76938
76939 if( !sqlite3FindTable(db, "sqlite_stat3", zDb) ){
76940 return SQLITE_OK;
76941 }
76942
76943 zSql = sqlite3MPrintf(db,
76944 "SELECT idx,count(*) FROM %Q.sqlite_stat3"
76945 " GROUP BY idx", zDb);
76946 if( !zSql ){
76947 return SQLITE_NOMEM;
76948 }
76949 rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
76950 sqlite3DbFree(db, zSql);
76951 if( rc ) return rc;
76952
76953 while( sqlite3_step(pStmt)==SQLITE_ROW ){
76954 char *zIndex; /* Index name */
76955 Index *pIdx; /* Pointer to the index object */
76956 int nSample; /* Number of samples */
76957
76958 zIndex = (char *)sqlite3_column_text(pStmt, 0);
76959 if( zIndex==0 ) continue;
76960 nSample = sqlite3_column_int(pStmt, 1);
76961 if( nSample>255 ) continue;
76962 pIdx = sqlite3FindIndex(db, zIndex, zDb);
76963 if( pIdx==0 ) continue;
76964 assert( pIdx->nSample==0 );
76965 pIdx->nSample = (u8)nSample;
76966 pIdx->aSample = sqlite3MallocZero( nSample*sizeof(IndexSample) );
76967 pIdx->avgEq = pIdx->aiRowEst[1];
76968 if( pIdx->aSample==0 ){
76969 db->mallocFailed = 1;
76970 sqlite3_finalize(pStmt);
76971 return SQLITE_NOMEM;
76972 }
76973 }
76974 sqlite3_finalize(pStmt);
76975
76976 zSql = sqlite3MPrintf(db,
76977 "SELECT idx,neq,nlt,ndlt,sample FROM %Q.sqlite_stat3", zDb);
76978 if( !zSql ){
76979 return SQLITE_NOMEM;
76980 }
76981 rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
76982 sqlite3DbFree(db, zSql);
76983 if( rc ) return rc;
76984
76985 while( sqlite3_step(pStmt)==SQLITE_ROW ){
76986 char *zIndex; /* Index name */
76987 Index *pIdx; /* Pointer to the index object */
76988 int i; /* Loop counter */
76989 tRowcnt sumEq; /* Sum of the nEq values */
76990
76991 zIndex = (char *)sqlite3_column_text(pStmt, 0);
76992 if( zIndex==0 ) continue;
76993 pIdx = sqlite3FindIndex(db, zIndex, zDb);
76994 if( pIdx==0 ) continue;
76995 if( pIdx==pPrevIdx ){
76996 idx++;
76997 }else{
76998 pPrevIdx = pIdx;
76999 idx = 0;
77000 }
77001 assert( idx<pIdx->nSample );
77002 pSample = &pIdx->aSample[idx];
77003 pSample->nEq = (tRowcnt)sqlite3_column_int64(pStmt, 1);
77004 pSample->nLt = (tRowcnt)sqlite3_column_int64(pStmt, 2);
77005 pSample->nDLt = (tRowcnt)sqlite3_column_int64(pStmt, 3);
77006 if( idx==pIdx->nSample-1 ){
77007 if( pSample->nDLt>0 ){
77008 for(i=0, sumEq=0; i<=idx-1; i++) sumEq += pIdx->aSample[i].nEq;
77009 pIdx->avgEq = (pSample->nLt - sumEq)/pSample->nDLt;
77010 }
77011 if( pIdx->avgEq<=0 ) pIdx->avgEq = 1;
77012 }
77013 eType = sqlite3_column_type(pStmt, 4);
77014 pSample->eType = (u8)eType;
77015 switch( eType ){
77016 case SQLITE_INTEGER: {
77017 pSample->u.i = sqlite3_column_int64(pStmt, 4);
77018 break;
77019 }
77020 case SQLITE_FLOAT: {
77021 pSample->u.r = sqlite3_column_double(pStmt, 4);
77022 break;
77023 }
77024 case SQLITE_NULL: {
77025 break;
77026 }
77027 default: assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB ); {
77028 const char *z = (const char *)(
77029 (eType==SQLITE_BLOB) ?
77030 sqlite3_column_blob(pStmt, 4):
77031 sqlite3_column_text(pStmt, 4)
77032 );
77033 int n = sqlite3_column_bytes(pStmt, 4);
77034 if( n>0xffff ) n = 0xffff;
77035 pSample->nByte = (u16)n;
77036 if( n < 1){
77037 pSample->u.z = 0;
77038 }else{
77039 pSample->u.z = sqlite3Malloc(n);
77040 if( pSample->u.z==0 ){
77041 db->mallocFailed = 1;
77042 sqlite3_finalize(pStmt);
77043 return SQLITE_NOMEM;
77044 }
77045 memcpy(pSample->u.z, z, n);
77046 }
77047 }
77048 }
77049 }
77050 return sqlite3_finalize(pStmt);
77051 }
77052 #endif /* SQLITE_ENABLE_STAT3 */
77053
77054 /*
77055 ** Load the content of the sqlite_stat1 and sqlite_stat3 tables. The
77056 ** contents of sqlite_stat1 are used to populate the Index.aiRowEst[]
77057 ** arrays. The contents of sqlite_stat3 are used to populate the
77058 ** Index.aSample[] arrays.
77059 **
77060 ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR
77061 ** is returned. In this case, even if SQLITE_ENABLE_STAT3 was defined
77062 ** during compilation and the sqlite_stat3 table is present, no data is
77063 ** read from it.
77064 **
77065 ** If SQLITE_ENABLE_STAT3 was defined during compilation and the
77066 ** sqlite_stat3 table is not present in the database, SQLITE_ERROR is
77067 ** returned. However, in this case, data is read from the sqlite_stat1
77068 ** table (if it is present) before returning.
77069 **
77070 ** If an OOM error occurs, this function always sets db->mallocFailed.
77071 ** This means if the caller does not care about other errors, the return
@@ -76337,12 +77083,14 @@
77083 /* Clear any prior statistics */
77084 assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
77085 for(i=sqliteHashFirst(&db->aDb[iDb].pSchema->idxHash);i;i=sqliteHashNext(i)){
77086 Index *pIdx = sqliteHashData(i);
77087 sqlite3DefaultRowEst(pIdx);
77088 #ifdef SQLITE_ENABLE_STAT3
77089 sqlite3DeleteIndexSamples(db, pIdx);
77090 pIdx->aSample = 0;
77091 #endif
77092 }
77093
77094 /* Check to make sure the sqlite_stat1 table exists */
77095 sInfo.db = db;
77096 sInfo.zDatabase = db->aDb[iDb].zName;
@@ -76350,91 +77098,23 @@
77098 return SQLITE_ERROR;
77099 }
77100
77101 /* Load new statistics out of the sqlite_stat1 table */
77102 zSql = sqlite3MPrintf(db,
77103 "SELECT tbl,idx,stat FROM %Q.sqlite_stat1", sInfo.zDatabase);
77104 if( zSql==0 ){
77105 rc = SQLITE_NOMEM;
77106 }else{
77107 rc = sqlite3_exec(db, zSql, analysisLoader, &sInfo, 0);
77108 sqlite3DbFree(db, zSql);
77109 }
77110
77111
77112 /* Load the statistics from the sqlite_stat3 table. */
77113 #ifdef SQLITE_ENABLE_STAT3
77114 if( rc==SQLITE_OK ){
77115 rc = loadStat3(db, sInfo.zDatabase);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
77116 }
77117 #endif
77118
77119 if( rc==SQLITE_NOMEM ){
77120 db->mallocFailed = 1;
@@ -78875,12 +79555,12 @@
79555 }
79556 }
79557 #endif
79558
79559 /* Reparse everything to update our internal data structures */
79560 sqlite3VdbeAddParseSchemaOp(v, iDb,
79561 sqlite3MPrintf(db, "tbl_name='%q'", p->zName));
79562 }
79563
79564
79565 /* Add the table to the in-memory representation of the database.
79566 */
@@ -79235,10 +79915,108 @@
79915 iDestroyed = iLargest;
79916 }
79917 }
79918 #endif
79919 }
79920
79921 /*
79922 ** Remove entries from the sqlite_stat1 and sqlite_stat2 tables
79923 ** after a DROP INDEX or DROP TABLE command.
79924 */
79925 static void sqlite3ClearStatTables(
79926 Parse *pParse, /* The parsing context */
79927 int iDb, /* The database number */
79928 const char *zType, /* "idx" or "tbl" */
79929 const char *zName /* Name of index or table */
79930 ){
79931 static const char *azStatTab[] = {
79932 "sqlite_stat1",
79933 "sqlite_stat2",
79934 "sqlite_stat3",
79935 };
79936 int i;
79937 const char *zDbName = pParse->db->aDb[iDb].zName;
79938 for(i=0; i<ArraySize(azStatTab); i++){
79939 if( sqlite3FindTable(pParse->db, azStatTab[i], zDbName) ){
79940 sqlite3NestedParse(pParse,
79941 "DELETE FROM %Q.%s WHERE %s=%Q",
79942 zDbName, azStatTab[i], zType, zName
79943 );
79944 }
79945 }
79946 }
79947
79948 /*
79949 ** Generate code to drop a table.
79950 */
79951 SQLITE_PRIVATE void sqlite3CodeDropTable(Parse *pParse, Table *pTab, int iDb, int isView){
79952 Vdbe *v;
79953 sqlite3 *db = pParse->db;
79954 Trigger *pTrigger;
79955 Db *pDb = &db->aDb[iDb];
79956
79957 v = sqlite3GetVdbe(pParse);
79958 assert( v!=0 );
79959 sqlite3BeginWriteOperation(pParse, 1, iDb);
79960
79961 #ifndef SQLITE_OMIT_VIRTUALTABLE
79962 if( IsVirtual(pTab) ){
79963 sqlite3VdbeAddOp0(v, OP_VBegin);
79964 }
79965 #endif
79966
79967 /* Drop all triggers associated with the table being dropped. Code
79968 ** is generated to remove entries from sqlite_master and/or
79969 ** sqlite_temp_master if required.
79970 */
79971 pTrigger = sqlite3TriggerList(pParse, pTab);
79972 while( pTrigger ){
79973 assert( pTrigger->pSchema==pTab->pSchema ||
79974 pTrigger->pSchema==db->aDb[1].pSchema );
79975 sqlite3DropTriggerPtr(pParse, pTrigger);
79976 pTrigger = pTrigger->pNext;
79977 }
79978
79979 #ifndef SQLITE_OMIT_AUTOINCREMENT
79980 /* Remove any entries of the sqlite_sequence table associated with
79981 ** the table being dropped. This is done before the table is dropped
79982 ** at the btree level, in case the sqlite_sequence table needs to
79983 ** move as a result of the drop (can happen in auto-vacuum mode).
79984 */
79985 if( pTab->tabFlags & TF_Autoincrement ){
79986 sqlite3NestedParse(pParse,
79987 "DELETE FROM %Q.sqlite_sequence WHERE name=%Q",
79988 pDb->zName, pTab->zName
79989 );
79990 }
79991 #endif
79992
79993 /* Drop all SQLITE_MASTER table and index entries that refer to the
79994 ** table. The program name loops through the master table and deletes
79995 ** every row that refers to a table of the same name as the one being
79996 ** dropped. Triggers are handled seperately because a trigger can be
79997 ** created in the temp database that refers to a table in another
79998 ** database.
79999 */
80000 sqlite3NestedParse(pParse,
80001 "DELETE FROM %Q.%s WHERE tbl_name=%Q and type!='trigger'",
80002 pDb->zName, SCHEMA_TABLE(iDb), pTab->zName);
80003 if( !isView && !IsVirtual(pTab) ){
80004 destroyTable(pParse, pTab);
80005 }
80006
80007 /* Remove the table entry from SQLite's internal schema and modify
80008 ** the schema cookie.
80009 */
80010 if( IsVirtual(pTab) ){
80011 sqlite3VdbeAddOp4(v, OP_VDestroy, iDb, 0, 0, pTab->zName, 0);
80012 }
80013 sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0);
80014 sqlite3ChangeCookie(pParse, iDb);
80015 sqliteViewResetAll(db, iDb);
80016
80017 }
80018
80019 /*
80020 ** This routine is called to do the work of a DROP TABLE statement.
80021 ** pName is the name of the table to be dropped.
80022 */
@@ -79304,11 +80082,11 @@
80082 if( sqlite3AuthCheck(pParse, SQLITE_DELETE, pTab->zName, 0, zDb) ){
80083 goto exit_drop_table;
80084 }
80085 }
80086 #endif
80087 if( !pParse->nested && sqlite3StrNICmp(pTab->zName, "sqlite_", 7)==0 ){
80088 sqlite3ErrorMsg(pParse, "table %s may not be dropped", pTab->zName);
80089 goto exit_drop_table;
80090 }
80091
80092 #ifndef SQLITE_OMIT_VIEW
@@ -79328,79 +80106,15 @@
80106 /* Generate code to remove the table from the master table
80107 ** on disk.
80108 */
80109 v = sqlite3GetVdbe(pParse);
80110 if( v ){
 
 
80111 sqlite3BeginWriteOperation(pParse, 1, iDb);
 
 
 
 
 
 
80112 sqlite3FkDropTable(pParse, pName, pTab);
80113 sqlite3CodeDropTable(pParse, pTab, iDb, isView);
80114 sqlite3ClearStatTables(pParse, iDb, "tbl", pTab->zName);
80115 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
80116
80117 exit_drop_table:
80118 sqlite3SrcListDelete(db, pName);
80119 }
80120
@@ -79843,24 +80557,24 @@
80557 */
80558 nName = sqlite3Strlen30(zName);
80559 nCol = pList->nExpr;
80560 pIndex = sqlite3DbMallocZero(db,
80561 sizeof(Index) + /* Index structure */
80562 sizeof(tRowcnt)*(nCol+1) + /* Index.aiRowEst */
80563 sizeof(int)*nCol + /* Index.aiColumn */
 
80564 sizeof(char *)*nCol + /* Index.azColl */
80565 sizeof(u8)*nCol + /* Index.aSortOrder */
80566 nName + 1 + /* Index.zName */
80567 nExtra /* Collation sequence names */
80568 );
80569 if( db->mallocFailed ){
80570 goto exit_create_index;
80571 }
80572 pIndex->aiRowEst = (tRowcnt*)(&pIndex[1]);
80573 pIndex->azColl = (char**)(&pIndex->aiRowEst[nCol+1]);
80574 pIndex->aiColumn = (int *)(&pIndex->azColl[nCol]);
80575 pIndex->aSortOrder = (u8 *)(&pIndex->aiColumn[nCol]);
 
80576 pIndex->zName = (char *)(&pIndex->aSortOrder[nCol]);
80577 zExtra = (char *)(&pIndex->zName[nName+1]);
80578 memcpy(pIndex->zName, zName, nName+1);
80579 pIndex->pTable = pTab;
80580 pIndex->nColumn = pList->nExpr;
@@ -80047,11 +80761,11 @@
80761 if( pStart ){
80762 assert( pEnd!=0 );
80763 /* A named index with an explicit CREATE INDEX statement */
80764 zStmt = sqlite3MPrintf(db, "CREATE%s INDEX %.*s",
80765 onError==OE_None ? "" : " UNIQUE",
80766 (int)(pEnd->z - pName->z) + 1,
80767 pName->z);
80768 }else{
80769 /* An automatic index created by a PRIMARY KEY or UNIQUE constraint */
80770 /* zStmt = sqlite3MPrintf(""); */
80771 zStmt = 0;
@@ -80073,13 +80787,12 @@
80787 ** to invalidate all pre-compiled statements.
80788 */
80789 if( pTblName ){
80790 sqlite3RefillIndex(pParse, pIndex, iMem);
80791 sqlite3ChangeCookie(pParse, iDb);
80792 sqlite3VdbeAddParseSchemaOp(v, iDb,
80793 sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName));
 
80794 sqlite3VdbeAddOp1(v, OP_Expire, 0);
80795 }
80796 }
80797
80798 /* When adding an index to the list of indices for a table, make
@@ -80134,13 +80847,13 @@
80847 ** Apart from that, we have little to go on besides intuition as to
80848 ** how aiRowEst[] should be initialized. The numbers generated here
80849 ** are based on typical values found in actual indices.
80850 */
80851 SQLITE_PRIVATE void sqlite3DefaultRowEst(Index *pIdx){
80852 tRowcnt *a = pIdx->aiRowEst;
80853 int i;
80854 tRowcnt n;
80855 assert( a!=0 );
80856 a[0] = pIdx->pTable->nRowEst;
80857 if( a[0]<10 ) a[0] = 10;
80858 n = 10;
80859 for(i=1; i<=pIdx->nColumn; i++){
@@ -80206,19 +80919,13 @@
80919 v = sqlite3GetVdbe(pParse);
80920 if( v ){
80921 sqlite3BeginWriteOperation(pParse, 1, iDb);
80922 sqlite3NestedParse(pParse,
80923 "DELETE FROM %Q.%s WHERE name=%Q AND type='index'",
80924 db->aDb[iDb].zName, SCHEMA_TABLE(iDb), pIndex->zName
80925 );
80926 sqlite3ClearStatTables(pParse, iDb, "idx", pIndex->zName);
 
 
 
 
 
 
80927 sqlite3ChangeCookie(pParse, iDb);
80928 destroyRootPage(pParse, pIndex->tnum, iDb);
80929 sqlite3VdbeAddOp4(v, OP_DropIndex, iDb, 0, 0, pIndex->zName, 0);
80930 }
80931
@@ -81843,11 +82550,13 @@
82550 int regRowid; /* Actual register containing rowids */
82551
82552 /* Collect rowids of every row to be deleted.
82553 */
82554 sqlite3VdbeAddOp2(v, OP_Null, 0, iRowSet);
82555 pWInfo = sqlite3WhereBegin(
82556 pParse, pTabList, pWhere, 0, 0, WHERE_DUPLICATES_OK
82557 );
82558 if( pWInfo==0 ) goto delete_from_cleanup;
82559 regRowid = sqlite3ExprCodeGetColumn(pParse, pTab, -1, iCur, iRowid);
82560 sqlite3VdbeAddOp2(v, OP_RowSetAdd, iRowSet, regRowid);
82561 if( db->flags & SQLITE_CountRows ){
82562 sqlite3VdbeAddOp2(v, OP_AddImm, memCnt, 1);
@@ -82108,12 +82817,18 @@
82817 sqlite3VdbeAddOp3(v, OP_Column, iCur, idx, regBase+j);
82818 sqlite3ColumnDefault(v, pTab, idx, -1);
82819 }
82820 }
82821 if( doMakeRec ){
82822 const char *zAff;
82823 if( pTab->pSelect || (pParse->db->flags & SQLITE_IdxRealAsInt)!=0 ){
82824 zAff = 0;
82825 }else{
82826 zAff = sqlite3IndexAffinityStr(v, pIdx);
82827 }
82828 sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol+1, regOut);
82829 sqlite3VdbeChangeP4(v, -1, zAff, P4_TRANSIENT);
82830 }
82831 sqlite3ReleaseTempRange(pParse, regBase, nCol+1);
82832 return regBase;
82833 }
82834
@@ -82621,14 +83336,14 @@
83336 ** character is exactly one byte in size. Also, all characters are
83337 ** able to participate in upper-case-to-lower-case mappings in EBCDIC
83338 ** whereas only characters less than 0x80 do in ASCII.
83339 */
83340 #if defined(SQLITE_EBCDIC)
83341 # define sqlite3Utf8Read(A,C) (*(A++))
83342 # define GlogUpperToLower(A) A = sqlite3UpperToLower[A]
83343 #else
83344 # define GlogUpperToLower(A) if( !((A)&~0x7f) ){ A = sqlite3UpperToLower[A]; }
83345 #endif
83346
83347 static const struct compareInfo globInfo = { '*', '?', '[', 0 };
83348 /* The correct SQL-92 behavior is for the LIKE operator to ignore
83349 ** case. Thus 'a' LIKE 'A' would be true. */
@@ -82667,13 +83382,13 @@
83382 */
83383 static int patternCompare(
83384 const u8 *zPattern, /* The glob pattern */
83385 const u8 *zString, /* The string to compare against the glob */
83386 const struct compareInfo *pInfo, /* Information about how to do the compare */
83387 u32 esc /* The escape character */
83388 ){
83389 u32 c, c2;
83390 int invert;
83391 int seen;
83392 u8 matchOne = pInfo->matchOne;
83393 u8 matchAll = pInfo->matchAll;
83394 u8 matchSet = pInfo->matchSet;
@@ -82723,11 +83438,11 @@
83438 }else if( !prevEscape && c==matchOne ){
83439 if( sqlite3Utf8Read(zString, &zString)==0 ){
83440 return 0;
83441 }
83442 }else if( c==matchSet ){
83443 u32 prior_c = 0;
83444 assert( esc==0 ); /* This only occurs for GLOB, not LIKE */
83445 seen = 0;
83446 invert = 0;
83447 c = sqlite3Utf8Read(zString, &zString);
83448 if( c==0 ) return 0;
@@ -82799,11 +83514,11 @@
83514 sqlite3_context *context,
83515 int argc,
83516 sqlite3_value **argv
83517 ){
83518 const unsigned char *zA, *zB;
83519 u32 escape = 0;
83520 int nPat;
83521 sqlite3 *db = sqlite3_context_db_handle(context);
83522
83523 zB = sqlite3_value_text(argv[0]);
83524 zA = sqlite3_value_text(argv[1]);
@@ -84110,17 +84825,29 @@
84825 }
84826
84827 /* If the parent table is the same as the child table, and we are about
84828 ** to increment the constraint-counter (i.e. this is an INSERT operation),
84829 ** then check if the row being inserted matches itself. If so, do not
84830 ** increment the constraint-counter.
84831 **
84832 ** If any of the parent-key values are NULL, then the row cannot match
84833 ** itself. So set JUMPIFNULL to make sure we do the OP_Found if any
84834 ** of the parent-key values are NULL (at this point it is known that
84835 ** none of the child key values are).
84836 */
84837 if( pTab==pFKey->pFrom && nIncr==1 ){
84838 int iJump = sqlite3VdbeCurrentAddr(v) + nCol + 1;
84839 for(i=0; i<nCol; i++){
84840 int iChild = aiCol[i]+1+regData;
84841 int iParent = pIdx->aiColumn[i]+1+regData;
84842 assert( aiCol[i]!=pTab->iPKey );
84843 if( pIdx->aiColumn[i]==pTab->iPKey ){
84844 /* The parent key is a composite key that includes the IPK column */
84845 iParent = regData;
84846 }
84847 sqlite3VdbeAddOp3(v, OP_Ne, iChild, iJump, iParent);
84848 sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL);
84849 }
84850 sqlite3VdbeAddOp2(v, OP_Goto, 0, iOk);
84851 }
84852
84853 sqlite3VdbeAddOp3(v, OP_MakeRecord, regTemp, nCol, regRec);
@@ -84272,11 +84999,11 @@
84999
85000 /* Create VDBE to loop through the entries in pSrc that match the WHERE
85001 ** clause. If the constraint is not deferred, throw an exception for
85002 ** each row found. Otherwise, for deferred constraints, increment the
85003 ** deferred constraint counter by nIncr for each row selected. */
85004 pWInfo = sqlite3WhereBegin(pParse, pSrc, pWhere, 0, 0, 0);
85005 if( nIncr>0 && pFKey->isDeferred==0 ){
85006 sqlite3ParseToplevel(pParse)->mayAbort = 1;
85007 }
85008 sqlite3VdbeAddOp2(v, OP_FkCounter, pFKey->isDeferred, nIncr);
85009 if( pWInfo ){
@@ -87139,10 +87866,13 @@
87866 int (*strnicmp)(const char*,const char*,int);
87867 int (*unlock_notify)(sqlite3*,void(*)(void**,int),void*);
87868 int (*wal_autocheckpoint)(sqlite3*,int);
87869 int (*wal_checkpoint)(sqlite3*,const char*);
87870 void *(*wal_hook)(sqlite3*,int(*)(void*,sqlite3*,const char*,int),void*);
87871 int (*blob_reopen)(sqlite3_blob*,sqlite3_int64);
87872 int (*vtab_config)(sqlite3*,int op,...);
87873 int (*vtab_on_conflict)(sqlite3*);
87874 };
87875
87876 /*
87877 ** The following macros redefine the API routines so that they are
87878 ** redirected throught the global sqlite3_api structure.
@@ -87339,10 +88069,13 @@
88069 #define sqlite3_strnicmp sqlite3_api->strnicmp
88070 #define sqlite3_unlock_notify sqlite3_api->unlock_notify
88071 #define sqlite3_wal_autocheckpoint sqlite3_api->wal_autocheckpoint
88072 #define sqlite3_wal_checkpoint sqlite3_api->wal_checkpoint
88073 #define sqlite3_wal_hook sqlite3_api->wal_hook
88074 #define sqlite3_blob_reopen sqlite3_api->blob_reopen
88075 #define sqlite3_vtab_config sqlite3_api->vtab_config
88076 #define sqlite3_vtab_on_conflict sqlite3_api->vtab_on_conflict
88077 #endif /* SQLITE_CORE */
88078
88079 #define SQLITE_EXTENSION_INIT1 const sqlite3_api_routines *sqlite3_api = 0;
88080 #define SQLITE_EXTENSION_INIT2(v) sqlite3_api = v;
88081
@@ -87413,10 +88146,12 @@
88146
88147 #ifdef SQLITE_OMIT_VIRTUALTABLE
88148 # define sqlite3_create_module 0
88149 # define sqlite3_create_module_v2 0
88150 # define sqlite3_declare_vtab 0
88151 # define sqlite3_vtab_config 0
88152 # define sqlite3_vtab_on_conflict 0
88153 #endif
88154
88155 #ifdef SQLITE_OMIT_SHARED_CACHE
88156 # define sqlite3_enable_shared_cache 0
88157 #endif
@@ -87436,10 +88171,11 @@
88171 #define sqlite3_blob_bytes 0
88172 #define sqlite3_blob_close 0
88173 #define sqlite3_blob_open 0
88174 #define sqlite3_blob_read 0
88175 #define sqlite3_blob_write 0
88176 #define sqlite3_blob_reopen 0
88177 #endif
88178
88179 /*
88180 ** The following structure contains pointers to all SQLite API routines.
88181 ** A pointer to this structure is passed into extensions when they are
@@ -87701,10 +88437,13 @@
88437 #else
88438 0,
88439 0,
88440 0,
88441 #endif
88442 sqlite3_blob_reopen,
88443 sqlite3_vtab_config,
88444 sqlite3_vtab_on_conflict,
88445 };
88446
88447 /*
88448 ** Attempt to load an SQLite extension library contained in the file
88449 ** zFile. The entry point is zProc. zProc may be 0 in which case a
@@ -94090,10 +94829,11 @@
94829 Expr *pHaving; /* The HAVING clause. May be NULL */
94830 int isDistinct; /* True if the DISTINCT keyword is present */
94831 int distinct; /* Table to use for the distinct set */
94832 int rc = 1; /* Value to return from this function */
94833 int addrSortIndex; /* Address of an OP_OpenEphemeral instruction */
94834 int addrDistinctIndex; /* Address of an OP_OpenEphemeral instruction */
94835 AggInfo sAggInfo; /* Information used by aggregate queries */
94836 int iEnd; /* Address of the end of the query */
94837 sqlite3 *db; /* The database connection */
94838
94839 #ifndef SQLITE_OMIT_EXPLAIN
@@ -94216,20 +94956,10 @@
94956 explainSetInteger(pParse->iSelectId, iRestoreSelectId);
94957 return rc;
94958 }
94959 #endif
94960
 
 
 
 
 
 
 
 
 
 
94961 /* If there is both a GROUP BY and an ORDER BY clause and they are
94962 ** identical, then disable the ORDER BY clause since the GROUP BY
94963 ** will cause elements to come out in the correct order. This is
94964 ** an optimization - the correct answer should result regardless.
94965 ** Use the SQLITE_GroupByOrder flag with SQLITE_TESTCTRL_OPTIMIZER
@@ -94237,10 +94967,34 @@
94967 */
94968 if( sqlite3ExprListCompare(p->pGroupBy, pOrderBy)==0
94969 && (db->flags & SQLITE_GroupByOrder)==0 ){
94970 pOrderBy = 0;
94971 }
94972
94973 /* If the query is DISTINCT with an ORDER BY but is not an aggregate, and
94974 ** if the select-list is the same as the ORDER BY list, then this query
94975 ** can be rewritten as a GROUP BY. In other words, this:
94976 **
94977 ** SELECT DISTINCT xyz FROM ... ORDER BY xyz
94978 **
94979 ** is transformed to:
94980 **
94981 ** SELECT xyz FROM ... GROUP BY xyz
94982 **
94983 ** The second form is preferred as a single index (or temp-table) may be
94984 ** used for both the ORDER BY and DISTINCT processing. As originally
94985 ** written the query must use a temp-table for at least one of the ORDER
94986 ** BY and DISTINCT, and an index or separate temp-table for the other.
94987 */
94988 if( (p->selFlags & (SF_Distinct|SF_Aggregate))==SF_Distinct
94989 && sqlite3ExprListCompare(pOrderBy, p->pEList)==0
94990 ){
94991 p->selFlags &= ~SF_Distinct;
94992 p->pGroupBy = sqlite3ExprListDup(db, p->pEList, 0);
94993 pGroupBy = p->pGroupBy;
94994 pOrderBy = 0;
94995 }
94996
94997 /* If there is an ORDER BY clause, then this sorting
94998 ** index might end up being unused if the data can be
94999 ** extracted in pre-sorted order. If that is the case, then the
95000 ** OP_OpenEphemeral instruction will be changed to an OP_Noop once
@@ -94273,26 +95027,25 @@
95027
95028 /* Open a virtual index to use for the distinct set.
95029 */
95030 if( p->selFlags & SF_Distinct ){
95031 KeyInfo *pKeyInfo;
 
95032 distinct = pParse->nTab++;
95033 pKeyInfo = keyInfoFromExprList(pParse, p->pEList);
95034 addrDistinctIndex = sqlite3VdbeAddOp4(v, OP_OpenEphemeral, distinct, 0, 0,
95035 (char*)pKeyInfo, P4_KEYINFO_HANDOFF);
95036 sqlite3VdbeChangeP5(v, BTREE_UNORDERED);
95037 }else{
95038 distinct = addrDistinctIndex = -1;
95039 }
95040
95041 /* Aggregate and non-aggregate queries are handled differently */
95042 if( !isAgg && pGroupBy==0 ){
95043 ExprList *pDist = (isDistinct ? p->pEList : 0);
95044
95045 /* Begin the database scan. */
95046 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pOrderBy, pDist, 0);
95047 if( pWInfo==0 ) goto select_end;
95048 if( pWInfo->nRowOut < p->nSelectRow ) p->nSelectRow = pWInfo->nRowOut;
95049
95050 /* If sorting index that was created by a prior OP_OpenEphemeral
95051 ** instruction ended up not being needed, then change the OP_OpenEphemeral
@@ -94301,14 +95054,56 @@
95054 if( addrSortIndex>=0 && pOrderBy==0 ){
95055 sqlite3VdbeChangeToNoop(v, addrSortIndex, 1);
95056 p->addrOpenEphm[2] = -1;
95057 }
95058
95059 if( pWInfo->eDistinct ){
95060 VdbeOp *pOp; /* No longer required OpenEphemeral instr. */
95061
95062 assert( addrDistinctIndex>0 );
95063 pOp = sqlite3VdbeGetOp(v, addrDistinctIndex);
95064
95065 assert( isDistinct );
95066 assert( pWInfo->eDistinct==WHERE_DISTINCT_ORDERED
95067 || pWInfo->eDistinct==WHERE_DISTINCT_UNIQUE
95068 );
95069 distinct = -1;
95070 if( pWInfo->eDistinct==WHERE_DISTINCT_ORDERED ){
95071 int iJump;
95072 int iExpr;
95073 int iFlag = ++pParse->nMem;
95074 int iBase = pParse->nMem+1;
95075 int iBase2 = iBase + pEList->nExpr;
95076 pParse->nMem += (pEList->nExpr*2);
95077
95078 /* Change the OP_OpenEphemeral coded earlier to an OP_Integer. The
95079 ** OP_Integer initializes the "first row" flag. */
95080 pOp->opcode = OP_Integer;
95081 pOp->p1 = 1;
95082 pOp->p2 = iFlag;
95083
95084 sqlite3ExprCodeExprList(pParse, pEList, iBase, 1);
95085 iJump = sqlite3VdbeCurrentAddr(v) + 1 + pEList->nExpr + 1 + 1;
95086 sqlite3VdbeAddOp2(v, OP_If, iFlag, iJump-1);
95087 for(iExpr=0; iExpr<pEList->nExpr; iExpr++){
95088 CollSeq *pColl = sqlite3ExprCollSeq(pParse, pEList->a[iExpr].pExpr);
95089 sqlite3VdbeAddOp3(v, OP_Ne, iBase+iExpr, iJump, iBase2+iExpr);
95090 sqlite3VdbeChangeP4(v, -1, (const char *)pColl, P4_COLLSEQ);
95091 sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
95092 }
95093 sqlite3VdbeAddOp2(v, OP_Goto, 0, pWInfo->iContinue);
95094
95095 sqlite3VdbeAddOp2(v, OP_Integer, 0, iFlag);
95096 assert( sqlite3VdbeCurrentAddr(v)==iJump );
95097 sqlite3VdbeAddOp3(v, OP_Move, iBase, iBase2, pEList->nExpr);
95098 }else{
95099 pOp->opcode = OP_Noop;
95100 }
95101 }
95102
95103 /* Use the standard inner loop. */
95104 selectInnerLoop(pParse, p, pEList, 0, 0, pOrderBy, distinct, pDest,
95105 pWInfo->iContinue, pWInfo->iBreak);
95106
95107 /* End the database scan loop.
95108 */
95109 sqlite3WhereEnd(pWInfo);
@@ -94414,11 +95209,11 @@
95209 ** This might involve two separate loops with an OP_Sort in between, or
95210 ** it might be a single loop that uses an index to extract information
95211 ** in the right order to begin with.
95212 */
95213 sqlite3VdbeAddOp2(v, OP_Gosub, regReset, addrReset);
95214 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pGroupBy, 0, 0);
95215 if( pWInfo==0 ) goto select_end;
95216 if( pGroupBy==0 ){
95217 /* The optimizer is able to deliver rows in group by order so
95218 ** we do not have to sort. The OP_OpenEphemeral table will be
95219 ** cancelled later because we still need to use the pKeyInfo
@@ -94676,11 +95471,11 @@
95471 /* This case runs if the aggregate has no GROUP BY clause. The
95472 ** processing is much simpler since there is only a single row
95473 ** of output.
95474 */
95475 resetAccumulator(pParse, &sAggInfo);
95476 pWInfo = sqlite3WhereBegin(pParse, pTabList, pWhere, &pMinMax, 0, flag);
95477 if( pWInfo==0 ){
95478 sqlite3ExprListDelete(db, pDel);
95479 goto select_end;
95480 }
95481 updateAccumulator(pParse, &sAggInfo);
@@ -95152,19 +95947,32 @@
95947 iDb = sqlite3TwoPartName(pParse, pName1, pName2, &pName);
95948 if( iDb<0 ){
95949 goto trigger_cleanup;
95950 }
95951 }
95952 if( !pTableName || db->mallocFailed ){
95953 goto trigger_cleanup;
95954 }
95955
95956 /* A long-standing parser bug is that this syntax was allowed:
95957 **
95958 ** CREATE TRIGGER attached.demo AFTER INSERT ON attached.tab ....
95959 ** ^^^^^^^^
95960 **
95961 ** To maintain backwards compatibility, ignore the database
95962 ** name on pTableName if we are reparsing our of SQLITE_MASTER.
95963 */
95964 if( db->init.busy && iDb!=1 ){
95965 sqlite3DbFree(db, pTableName->a[0].zDatabase);
95966 pTableName->a[0].zDatabase = 0;
95967 }
95968
95969 /* If the trigger name was unqualified, and the table is a temp table,
95970 ** then set iDb to 1 to create the trigger in the temporary database.
95971 ** If sqlite3SrcListLookup() returns 0, indicating the table does not
95972 ** exist, the error is caught by the block below.
95973 */
 
 
 
95974 pTab = sqlite3SrcListLookup(pParse, pTableName);
95975 if( db->init.busy==0 && pName2->n==0 && pTab
95976 && pTab->pSchema==db->aDb[1].pSchema ){
95977 iDb = 1;
95978 }
@@ -95336,13 +96144,12 @@
96144 "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')",
96145 db->aDb[iDb].zName, SCHEMA_TABLE(iDb), zName,
96146 pTrig->table, z);
96147 sqlite3DbFree(db, z);
96148 sqlite3ChangeCookie(pParse, iDb);
96149 sqlite3VdbeAddParseSchemaOp(v, iDb,
96150 sqlite3MPrintf(db, "type='trigger' AND name='%q'", zName));
 
96151 }
96152
96153 if( db->init.busy ){
96154 Trigger *pLink = pTrig;
96155 Hash *pHash = &db->aDb[iDb].pSchema->trigHash;
@@ -96392,11 +97199,11 @@
97199 aRegIdx = sqlite3DbMallocRaw(db, sizeof(Index*) * nIdx );
97200 if( aRegIdx==0 ) goto update_cleanup;
97201 }
97202 for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){
97203 int reg;
97204 if( hasFK || chngRowid ){
97205 reg = ++pParse->nMem;
97206 }else{
97207 reg = 0;
97208 for(i=0; i<pIdx->nColumn; i++){
97209 if( aXRef[pIdx->aiColumn[i]]>=0 ){
@@ -96459,11 +97266,13 @@
97266 }
97267
97268 /* Begin the database scan
97269 */
97270 sqlite3VdbeAddOp2(v, OP_Null, 0, regOldRowid);
97271 pWInfo = sqlite3WhereBegin(
97272 pParse, pTabList, pWhere, 0, 0, WHERE_ONEPASS_DESIRED
97273 );
97274 if( pWInfo==0 ) goto update_cleanup;
97275 okOnePass = pWInfo->okOnePass;
97276
97277 /* Remember the rowid of every item to be updated.
97278 */
@@ -97547,11 +98356,11 @@
98356 v = sqlite3GetVdbe(pParse);
98357 sqlite3ChangeCookie(pParse, iDb);
98358
98359 sqlite3VdbeAddOp2(v, OP_Expire, 0, 0);
98360 zWhere = sqlite3MPrintf(db, "name='%q' AND type='table'", pTab->zName);
98361 sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere);
98362 sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0,
98363 pTab->zName, sqlite3Strlen30(pTab->zName) + 1);
98364 }
98365
98366 /* If we are rereading the sqlite_master table create the in-memory
@@ -98350,11 +99159,11 @@
99159 #define TERM_CODED 0x04 /* This term is already coded */
99160 #define TERM_COPIED 0x08 /* Has a child */
99161 #define TERM_ORINFO 0x10 /* Need to free the WhereTerm.u.pOrInfo object */
99162 #define TERM_ANDINFO 0x20 /* Need to free the WhereTerm.u.pAndInfo obj */
99163 #define TERM_OR_OK 0x40 /* Used during OR-clause processing */
99164 #ifdef SQLITE_ENABLE_STAT3
99165 # define TERM_VNULL 0x80 /* Manufactured x>NULL or x<=NULL term */
99166 #else
99167 # define TERM_VNULL 0x00 /* Disabled if not using stat2 */
99168 #endif
99169
@@ -98485,10 +99294,11 @@
99294 #define WHERE_REVERSE 0x02000000 /* Scan in reverse order */
99295 #define WHERE_UNIQUE 0x04000000 /* Selects no more than one row */
99296 #define WHERE_VIRTUALTABLE 0x08000000 /* Use virtual-table processing */
99297 #define WHERE_MULTI_OR 0x10000000 /* OR using multiple indices */
99298 #define WHERE_TEMP_INDEX 0x20000000 /* Uses an ephemeral index */
99299 #define WHERE_DISTINCT 0x40000000 /* Correct order for DISTINCT */
99300
99301 /*
99302 ** Initialize a preallocated WhereClause structure.
99303 */
99304 static void whereClauseInit(
@@ -99563,11 +100373,11 @@
100373 pNewTerm->prereqAll = pTerm->prereqAll;
100374 }
100375 }
100376 #endif /* SQLITE_OMIT_VIRTUALTABLE */
100377
100378 #ifdef SQLITE_ENABLE_STAT3
100379 /* When sqlite_stat2 histogram data is available an operator of the
100380 ** form "x IS NOT NULL" can sometimes be evaluated more efficiently
100381 ** as "x>NULL" if x is not an INTEGER PRIMARY KEY. So construct a
100382 ** virtual term of that form.
100383 **
@@ -99602,11 +100412,11 @@
100412 pTerm->nChild = 1;
100413 pTerm->wtFlags |= TERM_COPIED;
100414 pNewTerm->prereqAll = pTerm->prereqAll;
100415 }
100416 }
100417 #endif /* SQLITE_ENABLE_STAT */
100418
100419 /* Prevent ON clause terms of a LEFT JOIN from being used to drive
100420 ** an index for tables to the left of the join.
100421 */
100422 pTerm->prereqRight |= extraRight;
@@ -99629,10 +100439,166 @@
100439 }
100440 }
100441 return 0;
100442 }
100443
100444 /*
100445 ** This function searches the expression list passed as the second argument
100446 ** for an expression of type TK_COLUMN that refers to the same column and
100447 ** uses the same collation sequence as the iCol'th column of index pIdx.
100448 ** Argument iBase is the cursor number used for the table that pIdx refers
100449 ** to.
100450 **
100451 ** If such an expression is found, its index in pList->a[] is returned. If
100452 ** no expression is found, -1 is returned.
100453 */
100454 static int findIndexCol(
100455 Parse *pParse, /* Parse context */
100456 ExprList *pList, /* Expression list to search */
100457 int iBase, /* Cursor for table associated with pIdx */
100458 Index *pIdx, /* Index to match column of */
100459 int iCol /* Column of index to match */
100460 ){
100461 int i;
100462 const char *zColl = pIdx->azColl[iCol];
100463
100464 for(i=0; i<pList->nExpr; i++){
100465 Expr *p = pList->a[i].pExpr;
100466 if( p->op==TK_COLUMN
100467 && p->iColumn==pIdx->aiColumn[iCol]
100468 && p->iTable==iBase
100469 ){
100470 CollSeq *pColl = sqlite3ExprCollSeq(pParse, p);
100471 if( ALWAYS(pColl) && 0==sqlite3StrICmp(pColl->zName, zColl) ){
100472 return i;
100473 }
100474 }
100475 }
100476
100477 return -1;
100478 }
100479
100480 /*
100481 ** This routine determines if pIdx can be used to assist in processing a
100482 ** DISTINCT qualifier. In other words, it tests whether or not using this
100483 ** index for the outer loop guarantees that rows with equal values for
100484 ** all expressions in the pDistinct list are delivered grouped together.
100485 **
100486 ** For example, the query
100487 **
100488 ** SELECT DISTINCT a, b, c FROM tbl WHERE a = ?
100489 **
100490 ** can benefit from any index on columns "b" and "c".
100491 */
100492 static int isDistinctIndex(
100493 Parse *pParse, /* Parsing context */
100494 WhereClause *pWC, /* The WHERE clause */
100495 Index *pIdx, /* The index being considered */
100496 int base, /* Cursor number for the table pIdx is on */
100497 ExprList *pDistinct, /* The DISTINCT expressions */
100498 int nEqCol /* Number of index columns with == */
100499 ){
100500 Bitmask mask = 0; /* Mask of unaccounted for pDistinct exprs */
100501 int i; /* Iterator variable */
100502
100503 if( pIdx->zName==0 || pDistinct==0 || pDistinct->nExpr>=BMS ) return 0;
100504 testcase( pDistinct->nExpr==BMS-1 );
100505
100506 /* Loop through all the expressions in the distinct list. If any of them
100507 ** are not simple column references, return early. Otherwise, test if the
100508 ** WHERE clause contains a "col=X" clause. If it does, the expression
100509 ** can be ignored. If it does not, and the column does not belong to the
100510 ** same table as index pIdx, return early. Finally, if there is no
100511 ** matching "col=X" expression and the column is on the same table as pIdx,
100512 ** set the corresponding bit in variable mask.
100513 */
100514 for(i=0; i<pDistinct->nExpr; i++){
100515 WhereTerm *pTerm;
100516 Expr *p = pDistinct->a[i].pExpr;
100517 if( p->op!=TK_COLUMN ) return 0;
100518 pTerm = findTerm(pWC, p->iTable, p->iColumn, ~(Bitmask)0, WO_EQ, 0);
100519 if( pTerm ){
100520 Expr *pX = pTerm->pExpr;
100521 CollSeq *p1 = sqlite3BinaryCompareCollSeq(pParse, pX->pLeft, pX->pRight);
100522 CollSeq *p2 = sqlite3ExprCollSeq(pParse, p);
100523 if( p1==p2 ) continue;
100524 }
100525 if( p->iTable!=base ) return 0;
100526 mask |= (((Bitmask)1) << i);
100527 }
100528
100529 for(i=nEqCol; mask && i<pIdx->nColumn; i++){
100530 int iExpr = findIndexCol(pParse, pDistinct, base, pIdx, i);
100531 if( iExpr<0 ) break;
100532 mask &= ~(((Bitmask)1) << iExpr);
100533 }
100534
100535 return (mask==0);
100536 }
100537
100538
100539 /*
100540 ** Return true if the DISTINCT expression-list passed as the third argument
100541 ** is redundant. A DISTINCT list is redundant if the database contains a
100542 ** UNIQUE index that guarantees that the result of the query will be distinct
100543 ** anyway.
100544 */
100545 static int isDistinctRedundant(
100546 Parse *pParse,
100547 SrcList *pTabList,
100548 WhereClause *pWC,
100549 ExprList *pDistinct
100550 ){
100551 Table *pTab;
100552 Index *pIdx;
100553 int i;
100554 int iBase;
100555
100556 /* If there is more than one table or sub-select in the FROM clause of
100557 ** this query, then it will not be possible to show that the DISTINCT
100558 ** clause is redundant. */
100559 if( pTabList->nSrc!=1 ) return 0;
100560 iBase = pTabList->a[0].iCursor;
100561 pTab = pTabList->a[0].pTab;
100562
100563 /* If any of the expressions is an IPK column on table iBase, then return
100564 ** true. Note: The (p->iTable==iBase) part of this test may be false if the
100565 ** current SELECT is a correlated sub-query.
100566 */
100567 for(i=0; i<pDistinct->nExpr; i++){
100568 Expr *p = pDistinct->a[i].pExpr;
100569 if( p->op==TK_COLUMN && p->iTable==iBase && p->iColumn<0 ) return 1;
100570 }
100571
100572 /* Loop through all indices on the table, checking each to see if it makes
100573 ** the DISTINCT qualifier redundant. It does so if:
100574 **
100575 ** 1. The index is itself UNIQUE, and
100576 **
100577 ** 2. All of the columns in the index are either part of the pDistinct
100578 ** list, or else the WHERE clause contains a term of the form "col=X",
100579 ** where X is a constant value. The collation sequences of the
100580 ** comparison and select-list expressions must match those of the index.
100581 */
100582 for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
100583 if( pIdx->onError==OE_None ) continue;
100584 for(i=0; i<pIdx->nColumn; i++){
100585 int iCol = pIdx->aiColumn[i];
100586 if( 0==findTerm(pWC, iBase, iCol, ~(Bitmask)0, WO_EQ, pIdx)
100587 && 0>findIndexCol(pParse, pDistinct, iBase, pIdx, i)
100588 ){
100589 break;
100590 }
100591 }
100592 if( i==pIdx->nColumn ){
100593 /* This index implies that the DISTINCT qualifier is redundant. */
100594 return 1;
100595 }
100596 }
100597
100598 return 0;
100599 }
100600
100601 /*
100602 ** This routine decides if pIdx can be used to satisfy the ORDER BY
100603 ** clause. If it can, it returns 1. If pIdx cannot satisfy the
100604 ** ORDER BY clause, this routine returns 0.
@@ -99665,11 +100631,14 @@
100631 int sortOrder = 0; /* XOR of index and ORDER BY sort direction */
100632 int nTerm; /* Number of ORDER BY terms */
100633 struct ExprList_item *pTerm; /* A term of the ORDER BY clause */
100634 sqlite3 *db = pParse->db;
100635
100636 if( !pOrderBy ) return 0;
100637 if( wsFlags & WHERE_COLUMN_IN ) return 0;
100638 if( pIdx->bUnordered ) return 0;
100639
100640 nTerm = pOrderBy->nExpr;
100641 assert( nTerm>0 );
100642
100643 /* Argument pIdx must either point to a 'real' named index structure,
100644 ** or an index structure allocated on the stack by bestBtreeIndex() to
@@ -99978,10 +100947,14 @@
100947 double costTempIdx; /* per-query cost of the transient index */
100948 WhereTerm *pTerm; /* A single term of the WHERE clause */
100949 WhereTerm *pWCEnd; /* End of pWC->a[] */
100950 Table *pTable; /* Table tht might be indexed */
100951
100952 if( pParse->nQueryLoop<=(double)1 ){
100953 /* There is no point in building an automatic index for a single scan */
100954 return;
100955 }
100956 if( (pParse->db->flags & SQLITE_AutoIndex)==0 ){
100957 /* Automatic indices are disabled at run-time */
100958 return;
100959 }
100960 if( (pCost->plan.wsFlags & WHERE_NOT_FULLSCAN)!=0 ){
@@ -99989,10 +100962,14 @@
100962 return;
100963 }
100964 if( pSrc->notIndexed ){
100965 /* The NOT INDEXED clause appears in the SQL. */
100966 return;
100967 }
100968 if( pSrc->isCorrelated ){
100969 /* The source is a correlated sub-query. No point in indexing it. */
100970 return;
100971 }
100972
100973 assert( pParse->nQueryLoop >= (double)1 );
100974 pTable = pSrc->pTab;
100975 nTableRow = pTable->nRowEst;
@@ -100206,10 +101183,11 @@
101183 if( pTerm->leftCursor != pSrc->iCursor ) continue;
101184 assert( (pTerm->eOperator&(pTerm->eOperator-1))==0 );
101185 testcase( pTerm->eOperator==WO_IN );
101186 testcase( pTerm->eOperator==WO_ISNULL );
101187 if( pTerm->eOperator & (WO_IN|WO_ISNULL) ) continue;
101188 if( pTerm->wtFlags & TERM_VNULL ) continue;
101189 nTerm++;
101190 }
101191
101192 /* If the ORDER BY clause contains only columns in the current
101193 ** virtual table then allocate space for the aOrderBy part of
@@ -100256,10 +101234,11 @@
101234 if( pTerm->leftCursor != pSrc->iCursor ) continue;
101235 assert( (pTerm->eOperator&(pTerm->eOperator-1))==0 );
101236 testcase( pTerm->eOperator==WO_IN );
101237 testcase( pTerm->eOperator==WO_ISNULL );
101238 if( pTerm->eOperator & (WO_IN|WO_ISNULL) ) continue;
101239 if( pTerm->wtFlags & TERM_VNULL ) continue;
101240 pIdxCons[j].iColumn = pTerm->u.leftColumn;
101241 pIdxCons[j].iTermOffset = i;
101242 pIdxCons[j].op = (u8)pTerm->eOperator;
101243 /* The direct assignment in the previous line is possible only because
101244 ** the WO_ and SQLITE_INDEX_CONSTRAINT_ codes are identical. The
@@ -100482,71 +101461,89 @@
101461 */
101462 bestOrClauseIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost);
101463 }
101464 #endif /* SQLITE_OMIT_VIRTUALTABLE */
101465
101466 #ifdef SQLITE_ENABLE_STAT3
101467 /*
101468 ** Estimate the location of a particular key among all keys in an
101469 ** index. Store the results in aStat as follows:
101470 **
101471 ** aStat[0] Est. number of rows less than pVal
101472 ** aStat[1] Est. number of rows equal to pVal
101473 **
101474 ** Return SQLITE_OK on success.
101475 */
101476 static int whereKeyStats(
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101477 Parse *pParse, /* Database connection */
101478 Index *pIdx, /* Index to consider domain of */
101479 sqlite3_value *pVal, /* Value to consider */
101480 int roundUp, /* Round up if true. Round down if false */
101481 tRowcnt *aStat /* OUT: stats written here */
101482 ){
101483 tRowcnt n;
101484 IndexSample *aSample;
101485 int i, eType;
101486 int isEq = 0;
101487 i64 v;
101488 double r, rS;
101489
101490 assert( roundUp==0 || roundUp==1 );
101491 if( pVal==0 ) return SQLITE_ERROR;
101492 n = pIdx->aiRowEst[0];
101493 aSample = pIdx->aSample;
101494 i = 0;
101495 eType = sqlite3_value_type(pVal);
101496
101497 if( eType==SQLITE_INTEGER ){
101498 v = sqlite3_value_int64(pVal);
101499 r = (i64)v;
101500 for(i=0; i<pIdx->nSample; i++){
101501 if( aSample[i].eType==SQLITE_NULL ) continue;
101502 if( aSample[i].eType>=SQLITE_TEXT ) break;
101503 if( aSample[i].eType==SQLITE_INTEGER ){
101504 if( aSample[i].u.i>=v ){
101505 isEq = aSample[i].u.i==v;
101506 break;
101507 }
101508 }else{
101509 assert( aSample[i].eType==SQLITE_FLOAT );
101510 if( aSample[i].u.r>=r ){
101511 isEq = aSample[i].u.r==r;
101512 break;
101513 }
101514 }
101515 }
101516 }else if( eType==SQLITE_FLOAT ){
101517 r = sqlite3_value_double(pVal);
101518 for(i=0; i<pIdx->nSample; i++){
101519 if( aSample[i].eType==SQLITE_NULL ) continue;
101520 if( aSample[i].eType>=SQLITE_TEXT ) break;
101521 if( aSample[i].eType==SQLITE_FLOAT ){
101522 rS = aSample[i].u.r;
101523 }else{
101524 rS = aSample[i].u.i;
101525 }
101526 if( rS>=r ){
101527 isEq = rS==r;
101528 break;
101529 }
101530 }
101531 }else if( eType==SQLITE_NULL ){
101532 i = 0;
101533 if( pIdx->nSample>=1 && aSample[0].eType==SQLITE_NULL ) isEq = 1;
101534 }else{
101535 assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB );
101536 for(i=0; i<pIdx->nSample; i++){
101537 if( aSample[i].eType==SQLITE_TEXT || aSample[i].eType==SQLITE_BLOB ){
101538 break;
101539 }
101540 }
101541 if( i<pIdx->nSample ){
101542 sqlite3 *db = pParse->db;
101543 CollSeq *pColl;
101544 const u8 *z;
 
 
 
 
 
101545 if( eType==SQLITE_BLOB ){
101546 z = (const u8 *)sqlite3_value_blob(pVal);
101547 pColl = db->pDfltColl;
101548 assert( pColl->enc==SQLITE_UTF8 );
101549 }else{
@@ -100561,16 +101558,16 @@
101558 return SQLITE_NOMEM;
101559 }
101560 assert( z && pColl && pColl->xCmp );
101561 }
101562 n = sqlite3ValueBytes(pVal, pColl->enc);
101563
101564 for(; i<pIdx->nSample; i++){
101565 int c;
101566 int eSampletype = aSample[i].eType;
101567 if( eSampletype<eType ) continue;
101568 if( eSampletype!=eType ) break;
101569 #ifndef SQLITE_OMIT_UTF16
101570 if( pColl->enc!=SQLITE_UTF8 ){
101571 int nSample;
101572 char *zSample = sqlite3Utf8to16(
101573 db, pColl->enc, aSample[i].u.z, aSample[i].nByte, &nSample
@@ -100584,20 +101581,52 @@
101581 }else
101582 #endif
101583 {
101584 c = pColl->xCmp(pColl->pUser, aSample[i].nByte, aSample[i].u.z, n, z);
101585 }
101586 if( c>=0 ){
101587 if( c==0 ) isEq = 1;
101588 break;
101589 }
101590 }
101591 }
101592 }
101593
101594 /* At this point, aSample[i] is the first sample that is greater than
101595 ** or equal to pVal. Or if i==pIdx->nSample, then all samples are less
101596 ** than pVal. If aSample[i]==pVal, then isEq==1.
101597 */
101598 if( isEq ){
101599 assert( i<pIdx->nSample );
101600 aStat[0] = aSample[i].nLt;
101601 aStat[1] = aSample[i].nEq;
101602 }else{
101603 tRowcnt iLower, iUpper, iGap;
101604 if( i==0 ){
101605 iLower = 0;
101606 iUpper = aSample[0].nLt;
101607 }else{
101608 iUpper = i>=pIdx->nSample ? n : aSample[i].nLt;
101609 iLower = aSample[i-1].nEq + aSample[i-1].nLt;
101610 }
101611 aStat[1] = pIdx->avgEq;
101612 if( iLower>=iUpper ){
101613 iGap = 0;
101614 }else{
101615 iGap = iUpper - iLower;
101616 if( iGap>=aStat[1]/2 ) iGap -= aStat[1]/2;
101617 }
101618 if( roundUp ){
101619 iGap = (iGap*2)/3;
101620 }else{
101621 iGap = iGap/3;
101622 }
101623 aStat[0] = iLower + iGap;
101624 }
101625 return SQLITE_OK;
101626 }
101627 #endif /* SQLITE_ENABLE_STAT3 */
101628
101629 /*
101630 ** If expression pExpr represents a literal value, set *pp to point to
101631 ** an sqlite3_value structure containing the same value, with affinity
101632 ** aff applied to it, before returning. It is the responsibility of the
@@ -100611,11 +101640,11 @@
101640 **
101641 ** If neither of the above apply, set *pp to NULL.
101642 **
101643 ** If an error occurs, return an error code. Otherwise, SQLITE_OK.
101644 */
101645 #ifdef SQLITE_ENABLE_STAT3
101646 static int valueFromExpr(
101647 Parse *pParse,
101648 Expr *pExpr,
101649 u8 aff,
101650 sqlite3_value **pp
@@ -100659,106 +101688,92 @@
101688 **
101689 ** ... FROM t1 WHERE a > ? AND a < ? ...
101690 **
101691 ** then nEq should be passed 0.
101692 **
101693 ** The returned value is an integer divisor to reduce the estimated
101694 ** search space. A return value of 1 means that range constraints are
101695 ** no help at all. A return value of 2 means range constraints are
101696 ** expected to reduce the search space by half. And so forth...
 
 
101697 **
101698 ** In the absence of sqlite_stat3 ANALYZE data, each range inequality
101699 ** reduces the search space by a factor of 4. Hence a single constraint (x>?)
101700 ** results in a return of 4 and a range constraint (x>? AND x<?) results
101701 ** in a return of 16.
101702 */
101703 static int whereRangeScanEst(
101704 Parse *pParse, /* Parsing & code generating context */
101705 Index *p, /* The index containing the range-compared column; "x" */
101706 int nEq, /* index into p->aCol[] of the range-compared column */
101707 WhereTerm *pLower, /* Lower bound on the range. ex: "x>123" Might be NULL */
101708 WhereTerm *pUpper, /* Upper bound on the range. ex: "x<455" Might be NULL */
101709 double *pRangeDiv /* OUT: Reduce search space by this divisor */
101710 ){
101711 int rc = SQLITE_OK;
101712
101713 #ifdef SQLITE_ENABLE_STAT3
101714
101715 if( nEq==0 && p->nSample ){
101716 sqlite3_value *pRangeVal;
101717 tRowcnt iLower = 0;
101718 tRowcnt iUpper = p->aiRowEst[0];
101719 tRowcnt a[2];
 
 
 
101720 u8 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
101721
101722 if( pLower ){
101723 Expr *pExpr = pLower->pExpr->pRight;
101724 rc = valueFromExpr(pParse, pExpr, aff, &pRangeVal);
101725 assert( pLower->eOperator==WO_GT || pLower->eOperator==WO_GE );
101726 if( rc==SQLITE_OK
101727 && whereKeyStats(pParse, p, pRangeVal, 0, a)==SQLITE_OK
101728 ){
101729 iLower = a[0];
101730 if( pLower->eOperator==WO_GT ) iLower += a[1];
101731 }
101732 sqlite3ValueFree(pRangeVal);
101733 }
101734 if( rc==SQLITE_OK && pUpper ){
101735 Expr *pExpr = pUpper->pExpr->pRight;
101736 rc = valueFromExpr(pParse, pExpr, aff, &pRangeVal);
101737 assert( pUpper->eOperator==WO_LT || pUpper->eOperator==WO_LE );
101738 if( rc==SQLITE_OK
101739 && whereKeyStats(pParse, p, pRangeVal, 1, a)==SQLITE_OK
101740 ){
101741 iUpper = a[0];
101742 if( pUpper->eOperator==WO_LE ) iUpper += a[1];
101743 }
101744 sqlite3ValueFree(pRangeVal);
101745 }
101746 if( rc==SQLITE_OK ){
101747 if( iUpper<=iLower ){
101748 *pRangeDiv = (double)p->aiRowEst[0];
101749 }else{
101750 *pRangeDiv = (double)p->aiRowEst[0]/(double)(iUpper - iLower);
101751 }
101752 WHERETRACE(("range scan regions: %u..%u div=%g\n",
101753 (u32)iLower, (u32)iUpper, *pRangeDiv));
101754 return SQLITE_OK;
101755 }
101756 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101757 #else
101758 UNUSED_PARAMETER(pParse);
101759 UNUSED_PARAMETER(p);
101760 UNUSED_PARAMETER(nEq);
101761 #endif
101762 assert( pLower || pUpper );
101763 *pRangeDiv = (double)1;
101764 if( pLower && (pLower->wtFlags & TERM_VNULL)==0 ) *pRangeDiv *= (double)4;
101765 if( pUpper ) *pRangeDiv *= (double)4;
101766 return rc;
101767 }
101768
101769 #ifdef SQLITE_ENABLE_STAT3
101770 /*
101771 ** Estimate the number of rows that will be returned based on
101772 ** an equality constraint x=VALUE and where that VALUE occurs in
101773 ** the histogram data. This only works when x is the left-most
101774 ** column of an index and sqlite_stat3 histogram data is available
101775 ** for that index. When pExpr==NULL that means the constraint is
101776 ** "x IS NULL" instead of "x=VALUE".
101777 **
101778 ** Write the estimated row count into *pnRow and return SQLITE_OK.
101779 ** If unable to make an estimate, leave *pnRow unchanged and return
@@ -100774,14 +101789,13 @@
101789 Index *p, /* The index whose left-most column is pTerm */
101790 Expr *pExpr, /* Expression for VALUE in the x=VALUE constraint */
101791 double *pnRow /* Write the revised row estimate here */
101792 ){
101793 sqlite3_value *pRhs = 0; /* VALUE on right-hand side of pTerm */
 
101794 u8 aff; /* Column affinity */
101795 int rc; /* Subfunction return code */
101796 tRowcnt a[2]; /* Statistics */
101797
101798 assert( p->aSample!=0 );
101799 aff = p->pTable->aCol[p->aiColumn[0]].affinity;
101800 if( pExpr ){
101801 rc = valueFromExpr(pParse, pExpr, aff, &pRhs);
@@ -100788,30 +101802,22 @@
101802 if( rc ) goto whereEqualScanEst_cancel;
101803 }else{
101804 pRhs = sqlite3ValueNew(pParse->db);
101805 }
101806 if( pRhs==0 ) return SQLITE_NOTFOUND;
101807 rc = whereKeyStats(pParse, p, pRhs, 0, a);
101808 if( rc==SQLITE_OK ){
101809 WHERETRACE(("equality scan regions: %d\n", (int)a[1]));
101810 *pnRow = a[1];
101811 }
 
 
 
 
 
 
 
 
101812 whereEqualScanEst_cancel:
101813 sqlite3ValueFree(pRhs);
101814 return rc;
101815 }
101816 #endif /* defined(SQLITE_ENABLE_STAT3) */
101817
101818 #ifdef SQLITE_ENABLE_STAT3
101819 /*
101820 ** Estimate the number of rows that will be returned based on
101821 ** an IN constraint where the right-hand side of the IN operator
101822 ** is a list of values. Example:
101823 **
@@ -100830,64 +101836,29 @@
101836 Parse *pParse, /* Parsing & code generating context */
101837 Index *p, /* The index whose left-most column is pTerm */
101838 ExprList *pList, /* The value list on the RHS of "x IN (v1,v2,v3,...)" */
101839 double *pnRow /* Write the revised row estimate here */
101840 ){
101841 int rc = SQLITE_OK; /* Subfunction return code */
101842 double nEst; /* Number of rows for a single term */
101843 double nRowEst = (double)0; /* New estimate of the number of rows */
101844 int i; /* Loop counter */
 
 
 
 
 
 
 
101845
101846 assert( p->aSample!=0 );
101847 for(i=0; rc==SQLITE_OK && i<pList->nExpr; i++){
101848 nEst = p->aiRowEst[0];
101849 rc = whereEqualScanEst(pParse, p, pList->a[i].pExpr, &nEst);
101850 nRowEst += nEst;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
101851 }
101852 if( rc==SQLITE_OK ){
 
 
 
 
 
 
 
 
 
101853 if( nRowEst > p->aiRowEst[0] ) nRowEst = p->aiRowEst[0];
101854 *pnRow = nRowEst;
101855 WHERETRACE(("IN row estimate: est=%g\n", nRowEst));
 
101856 }
 
101857 return rc;
101858 }
101859 #endif /* defined(SQLITE_ENABLE_STAT3) */
101860
101861
101862 /*
101863 ** Find the best query plan for accessing a particular table. Write the
101864 ** best query plan and its cost into the WhereCost object supplied as the
@@ -100921,19 +101892,20 @@
101892 WhereClause *pWC, /* The WHERE clause */
101893 struct SrcList_item *pSrc, /* The FROM clause term to search */
101894 Bitmask notReady, /* Mask of cursors not available for indexing */
101895 Bitmask notValid, /* Cursors not available for any purpose */
101896 ExprList *pOrderBy, /* The ORDER BY clause */
101897 ExprList *pDistinct, /* The select-list if query is DISTINCT */
101898 WhereCost *pCost /* Lowest cost query plan */
101899 ){
101900 int iCur = pSrc->iCursor; /* The cursor of the table to be accessed */
101901 Index *pProbe; /* An index we are evaluating */
101902 Index *pIdx; /* Copy of pProbe, or zero for IPK index */
101903 int eqTermMask; /* Current mask of valid equality operators */
101904 int idxEqTermMask; /* Index mask of valid equality operators */
101905 Index sPk; /* A fake index object for the primary key */
101906 tRowcnt aiRowEstPk[2]; /* The aiRowEst[] value for the sPk index */
101907 int aiColumnPk = -1; /* The aColumn[] value for the sPk index */
101908 int wsFlagMask; /* Allowed flags in pCost->plan.wsFlag */
101909
101910 /* Initialize the cost to a worst-case value */
101911 memset(pCost, 0, sizeof(*pCost));
@@ -100984,11 +101956,11 @@
101956 }
101957
101958 /* Loop over all indices looking for the best one to use
101959 */
101960 for(; pProbe; pIdx=pProbe=pProbe->pNext){
101961 const tRowcnt * const aiRowEst = pProbe->aiRowEst;
101962 double cost; /* Cost of using pProbe */
101963 double nRow; /* Estimated number of rows in result set */
101964 double log10N; /* base-10 logarithm of nRow (inexact) */
101965 int rev; /* True to scan in reverse order */
101966 int wsFlags = 0;
@@ -101027,18 +101999,16 @@
101999 ** Set to true if there was at least one "x IN (SELECT ...)" term used
102000 ** in determining the value of nInMul. Note that the RHS of the
102001 ** IN operator must be a SELECT, not a value list, for this variable
102002 ** to be true.
102003 **
102004 ** rangeDiv:
102005 ** An estimate of a divisor by which to reduce the search space due
102006 ** to inequality constraints. In the absence of sqlite_stat3 ANALYZE
102007 ** data, a single inequality reduces the search space to 1/4rd its
102008 ** original size (rangeDiv==4). Two inequalities reduce the search
102009 ** space to 1/16th of its original size (rangeDiv==16).
 
 
102010 **
102011 ** bSort:
102012 ** Boolean. True if there is an ORDER BY clause that will require an
102013 ** external sort (i.e. scanning the index being evaluated will not
102014 ** correctly order records).
@@ -101059,16 +102029,17 @@
102029 ** SELECT a, b, c FROM tbl WHERE a = 1;
102030 */
102031 int nEq; /* Number of == or IN terms matching index */
102032 int bInEst = 0; /* True if "x IN (SELECT...)" seen */
102033 int nInMul = 1; /* Number of distinct equalities to lookup */
102034 double rangeDiv = (double)1; /* Estimated reduction in search space */
102035 int nBound = 0; /* Number of range constraints seen */
102036 int bSort = !!pOrderBy; /* True if external sort required */
102037 int bDist = !!pDistinct; /* True if index cannot help with DISTINCT */
102038 int bLookup = 0; /* True if not a covering index */
102039 WhereTerm *pTerm; /* A single term of the WHERE clause */
102040 #ifdef SQLITE_ENABLE_STAT3
102041 WhereTerm *pFirstTerm = 0; /* First term matching the index */
102042 #endif
102043
102044 /* Determine the values of nEq and nInMul */
102045 for(nEq=0; nEq<pProbe->nColumn; nEq++){
@@ -101088,23 +102059,23 @@
102059 nInMul *= pExpr->x.pList->nExpr;
102060 }
102061 }else if( pTerm->eOperator & WO_ISNULL ){
102062 wsFlags |= WHERE_COLUMN_NULL;
102063 }
102064 #ifdef SQLITE_ENABLE_STAT3
102065 if( nEq==0 && pProbe->aSample ) pFirstTerm = pTerm;
102066 #endif
102067 used |= pTerm->prereqRight;
102068 }
102069
102070 /* Determine the value of rangeDiv */
102071 if( nEq<pProbe->nColumn && pProbe->bUnordered==0 ){
102072 int j = pProbe->aiColumn[nEq];
102073 if( findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE|WO_GT|WO_GE, pIdx) ){
102074 WhereTerm *pTop = findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE, pIdx);
102075 WhereTerm *pBtm = findTerm(pWC, iCur, j, notReady, WO_GT|WO_GE, pIdx);
102076 whereRangeScanEst(pParse, pProbe, nEq, pBtm, pTop, &rangeDiv);
102077 if( pTop ){
102078 nBound = 1;
102079 wsFlags |= WHERE_TOP_LIMIT;
102080 used |= pTop->prereqRight;
102081 }
@@ -101125,21 +102096,24 @@
102096
102097 /* If there is an ORDER BY clause and the index being considered will
102098 ** naturally scan rows in the required order, set the appropriate flags
102099 ** in wsFlags. Otherwise, if there is an ORDER BY clause but the index
102100 ** will scan rows in a different order, set the bSort variable. */
102101 if( isSortingIndex(
102102 pParse, pWC->pMaskSet, pProbe, iCur, pOrderBy, nEq, wsFlags, &rev)
102103 ){
102104 bSort = 0;
102105 wsFlags |= WHERE_ROWID_RANGE|WHERE_COLUMN_RANGE|WHERE_ORDERBY;
102106 wsFlags |= (rev ? WHERE_REVERSE : 0);
102107 }
102108
102109 /* If there is a DISTINCT qualifier and this index will scan rows in
102110 ** order of the DISTINCT expressions, clear bDist and set the appropriate
102111 ** flags in wsFlags. */
102112 if( isDistinctIndex(pParse, pWC, pProbe, iCur, pDistinct, nEq) ){
102113 bDist = 0;
102114 wsFlags |= WHERE_ROWID_RANGE|WHERE_COLUMN_RANGE|WHERE_DISTINCT;
102115 }
102116
102117 /* If currently calculating the cost of using an index (not the IPK
102118 ** index), determine if all required column data may be obtained without
102119 ** using the main table (i.e. if the index is a covering
@@ -101169,31 +102143,32 @@
102143 if( bInEst && nRow*2>aiRowEst[0] ){
102144 nRow = aiRowEst[0]/2;
102145 nInMul = (int)(nRow / aiRowEst[nEq]);
102146 }
102147
102148 #ifdef SQLITE_ENABLE_STAT3
102149 /* If the constraint is of the form x=VALUE or x IN (E1,E2,...)
102150 ** and we do not think that values of x are unique and if histogram
102151 ** data is available for column x, then it might be possible
102152 ** to get a better estimate on the number of rows based on
102153 ** VALUE and how common that value is according to the histogram.
102154 */
102155 if( nRow>(double)1 && nEq==1 && pFirstTerm!=0 && aiRowEst[1]>1 ){
102156 if( pFirstTerm->eOperator & (WO_EQ|WO_ISNULL) ){
102157 testcase( pFirstTerm->eOperator==WO_EQ );
102158 testcase( pFirstTerm->eOperator==WO_ISNULL );
102159 whereEqualScanEst(pParse, pProbe, pFirstTerm->pExpr->pRight, &nRow);
102160 }else if( pFirstTerm->eOperator==WO_IN && bInEst==0 ){
102161 whereInScanEst(pParse, pProbe, pFirstTerm->pExpr->x.pList, &nRow);
102162 }
102163 }
102164 #endif /* SQLITE_ENABLE_STAT3 */
102165
102166 /* Adjust the number of output rows and downward to reflect rows
102167 ** that are excluded by range constraints.
102168 */
102169 nRow = nRow/rangeDiv;
102170 if( nRow<1 ) nRow = 1;
102171
102172 /* Experiments run on real SQLite databases show that the time needed
102173 ** to do a binary search to locate a row in a table or index is roughly
102174 ** log10(N) times the time to move from one row to the next row within
@@ -101252,10 +102227,13 @@
102227 ** difference and select C of 3.0.
102228 */
102229 if( bSort ){
102230 cost += nRow*estLog(nRow)*3;
102231 }
102232 if( bDist ){
102233 cost += nRow*estLog(nRow)*3;
102234 }
102235
102236 /**** Cost of using this index has now been computed ****/
102237
102238 /* If there are additional constraints on this table that cannot
102239 ** be used with the current index, but which might lower the number
@@ -101315,14 +102293,14 @@
102293 if( nRow<2 ) nRow = 2;
102294 }
102295
102296
102297 WHERETRACE((
102298 "%s(%s): nEq=%d nInMul=%d rangeDiv=%d bSort=%d bLookup=%d wsFlags=0x%x\n"
102299 " notReady=0x%llx log10N=%.1f nRow=%.1f cost=%.1f used=0x%llx\n",
102300 pSrc->pTab->zName, (pIdx ? pIdx->zName : "ipk"),
102301 nEq, nInMul, (int)rangeDiv, bSort, bLookup, wsFlags,
102302 notReady, log10N, nRow, cost, used
102303 ));
102304
102305 /* If this index is the best we have seen so far, then record this
102306 ** index and its cost in the pCost structure.
@@ -101397,11 +102375,11 @@
102375 }
102376 sqlite3DbFree(pParse->db, p);
102377 }else
102378 #endif
102379 {
102380 bestBtreeIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, 0, pCost);
102381 }
102382 }
102383
102384 /*
102385 ** Disable a term in the WHERE clause. Except, do not disable the term
@@ -102359,11 +103337,11 @@
103337 for(ii=0; ii<pOrWc->nTerm; ii++){
103338 WhereTerm *pOrTerm = &pOrWc->a[ii];
103339 if( pOrTerm->leftCursor==iCur || pOrTerm->eOperator==WO_AND ){
103340 WhereInfo *pSubWInfo; /* Info for single OR-term scan */
103341 /* Loop through table entries that match term pOrTerm. */
103342 pSubWInfo = sqlite3WhereBegin(pParse, pOrTab, pOrTerm->pExpr, 0, 0,
103343 WHERE_OMIT_OPEN | WHERE_OMIT_CLOSE |
103344 WHERE_FORCE_TABLE | WHERE_ONETABLE_ONLY);
103345 if( pSubWInfo ){
103346 explainOneScan(
103347 pParse, pOrTab, &pSubWInfo->a[0], iLevel, pLevel->iFrom, 0
@@ -102600,10 +103578,11 @@
103578 SQLITE_PRIVATE WhereInfo *sqlite3WhereBegin(
103579 Parse *pParse, /* The parser context */
103580 SrcList *pTabList, /* A list of all tables to be scanned */
103581 Expr *pWhere, /* The WHERE clause */
103582 ExprList **ppOrderBy, /* An ORDER BY clause, or NULL */
103583 ExprList *pDistinct, /* The select-list for DISTINCT queries - or NULL */
103584 u16 wctrlFlags /* One of the WHERE_* flags defined in sqliteInt.h */
103585 ){
103586 int i; /* Loop counter */
103587 int nByteWInfo; /* Num. bytes allocated for WhereInfo struct */
103588 int nTabList; /* Number of elements in pTabList */
@@ -102659,10 +103638,14 @@
103638 pWInfo->iBreak = sqlite3VdbeMakeLabel(v);
103639 pWInfo->pWC = pWC = (WhereClause *)&((u8 *)pWInfo)[nByteWInfo];
103640 pWInfo->wctrlFlags = wctrlFlags;
103641 pWInfo->savedNQueryLoop = pParse->nQueryLoop;
103642 pMaskSet = (WhereMaskSet*)&pWC[1];
103643
103644 /* Disable the DISTINCT optimization if SQLITE_DistinctOpt is set via
103645 ** sqlite3_test_ctrl(SQLITE_TESTCTRL_OPTIMIZATIONS,...) */
103646 if( db->flags & SQLITE_DistinctOpt ) pDistinct = 0;
103647
103648 /* Split the WHERE clause into separate subexpressions where each
103649 ** subexpression is separated by an AND operator.
103650 */
103651 initMaskSet(pMaskSet);
@@ -102726,10 +103709,19 @@
103709 */
103710 exprAnalyzeAll(pTabList, pWC);
103711 if( db->mallocFailed ){
103712 goto whereBeginError;
103713 }
103714
103715 /* Check if the DISTINCT qualifier, if there is one, is redundant.
103716 ** If it is, then set pDistinct to NULL and WhereInfo.eDistinct to
103717 ** WHERE_DISTINCT_UNIQUE to tell the caller to ignore the DISTINCT.
103718 */
103719 if( pDistinct && isDistinctRedundant(pParse, pTabList, pWC, pDistinct) ){
103720 pDistinct = 0;
103721 pWInfo->eDistinct = WHERE_DISTINCT_UNIQUE;
103722 }
103723
103724 /* Chose the best index to use for each table in the FROM clause.
103725 **
103726 ** This loop fills in the following fields:
103727 **
@@ -102810,10 +103802,11 @@
103802 Bitmask mask; /* Mask of tables not yet ready */
103803 for(j=iFrom, pTabItem=&pTabList->a[j]; j<nTabList; j++, pTabItem++){
103804 int doNotReorder; /* True if this table should not be reordered */
103805 WhereCost sCost; /* Cost information from best[Virtual]Index() */
103806 ExprList *pOrderBy; /* ORDER BY clause for index to optimize */
103807 ExprList *pDist; /* DISTINCT clause for index to optimize */
103808
103809 doNotReorder = (pTabItem->jointype & (JT_LEFT|JT_CROSS))!=0;
103810 if( j!=iFrom && doNotReorder ) break;
103811 m = getMask(pMaskSet, pTabItem->iCursor);
103812 if( (m & notReady)==0 ){
@@ -102820,10 +103813,11 @@
103813 if( j==iFrom ) iFrom++;
103814 continue;
103815 }
103816 mask = (isOptimal ? m : notReady);
103817 pOrderBy = ((i==0 && ppOrderBy )?*ppOrderBy:0);
103818 pDist = (i==0 ? pDistinct : 0);
103819 if( pTabItem->pIndex==0 ) nUnconstrained++;
103820
103821 WHERETRACE(("=== trying table %d with isOptimal=%d ===\n",
103822 j, isOptimal));
103823 assert( pTabItem->pTab );
@@ -102834,11 +103828,11 @@
103828 &sCost, pp);
103829 }else
103830 #endif
103831 {
103832 bestBtreeIndex(pParse, pWC, pTabItem, mask, notReady, pOrderBy,
103833 pDist, &sCost);
103834 }
103835 assert( isOptimal || (sCost.used&notReady)==0 );
103836
103837 /* If an INDEXED BY clause is present, then the plan must use that
103838 ** index if it uses any index at all */
@@ -102894,10 +103888,14 @@
103888 WHERETRACE(("*** Optimizer selects table %d for loop %d"
103889 " with cost=%g and nRow=%g\n",
103890 bestJ, pLevel-pWInfo->a, bestPlan.rCost, bestPlan.plan.nRow));
103891 if( (bestPlan.plan.wsFlags & WHERE_ORDERBY)!=0 ){
103892 *ppOrderBy = 0;
103893 }
103894 if( (bestPlan.plan.wsFlags & WHERE_DISTINCT)!=0 ){
103895 assert( pWInfo->eDistinct==0 );
103896 pWInfo->eDistinct = WHERE_DISTINCT_ORDERED;
103897 }
103898 andFlags &= bestPlan.plan.wsFlags;
103899 pLevel->plan = bestPlan.plan;
103900 testcase( bestPlan.plan.wsFlags & WHERE_INDEXED );
103901 testcase( bestPlan.plan.wsFlags & WHERE_TEMP_INDEX );
@@ -107274,17 +108272,16 @@
108272 #ifndef SQLITE_OMIT_BLOB_LITERAL
108273 case 'x': case 'X': {
108274 testcase( z[0]=='x' ); testcase( z[0]=='X' );
108275 if( z[1]=='\'' ){
108276 *tokenType = TK_BLOB;
108277 for(i=2; sqlite3Isxdigit(z[i]); i++){}
108278 if( z[i]!='\'' || i%2 ){
108279 *tokenType = TK_ILLEGAL;
108280 while( z[i] && z[i]!='\'' ){ i++; }
108281 }
108282 if( z[i] ) i++;
 
108283 return i;
108284 }
108285 /* Otherwise fall through to the next case */
108286 }
108287 #endif
@@ -110726,10 +111723,21 @@
111723 if( sz ) *ppNew = sqlite3ScratchMalloc(sz);
111724 sqlite3ScratchFree(pFree);
111725 break;
111726 }
111727
111728 /* sqlite3_test_control(SQLITE_TESTCTRL_LOCALTIME_FAULT, int onoff);
111729 **
111730 ** If parameter onoff is non-zero, configure the wrappers so that all
111731 ** subsequent calls to localtime() and variants fail. If onoff is zero,
111732 ** undo this setting.
111733 */
111734 case SQLITE_TESTCTRL_LOCALTIME_FAULT: {
111735 sqlite3GlobalConfig.bLocaltimeFault = va_arg(ap, int);
111736 break;
111737 }
111738
111739 }
111740 va_end(ap);
111741 #endif /* SQLITE_OMIT_BUILTIN_TEST */
111742 return rc;
111743 }
@@ -111383,16 +112391,10 @@
112391 ** TODO(shess) Provide a VACUUM type operation to clear out all
112392 ** deletions and duplications. This would basically be a forced merge
112393 ** into a single segment.
112394 */
112395
 
 
 
 
 
 
112396 /************** Include fts3Int.h in the middle of fts3.c ********************/
112397 /************** Begin file fts3Int.h *****************************************/
112398 /*
112399 ** 2009 Nov 12
112400 **
@@ -111404,18 +112406,33 @@
112406 ** May you share freely, never taking more than you give.
112407 **
112408 ******************************************************************************
112409 **
112410 */
 
112411 #ifndef _FTSINT_H
112412 #define _FTSINT_H
112413
112414 #if !defined(NDEBUG) && !defined(SQLITE_DEBUG)
112415 # define NDEBUG 1
112416 #endif
112417
112418 /*
112419 ** FTS4 is really an extension for FTS3. It is enabled using the
112420 ** SQLITE_ENABLE_FTS3 macro. But to avoid confusion we also all
112421 ** the SQLITE_ENABLE_FTS4 macro to serve as an alisse for SQLITE_ENABLE_FTS3.
112422 */
112423 #if defined(SQLITE_ENABLE_FTS4) && !defined(SQLITE_ENABLE_FTS3)
112424 # define SQLITE_ENABLE_FTS3
112425 #endif
112426
112427 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
112428
112429 /* If not building as part of the core, include sqlite3ext.h. */
112430 #ifndef SQLITE_CORE
112431 SQLITE_API extern const sqlite3_api_routines *sqlite3_api;
112432 #endif
112433
112434 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/
112435 /************** Begin file fts3_tokenizer.h **********************************/
112436 /*
112437 ** 2006 July 10
112438 **
@@ -111710,16 +112727,39 @@
112727 ** similar macro called ArraySize(). Use a different name to avoid
112728 ** a collision when building an amalgamation with built-in FTS3.
112729 */
112730 #define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0])))
112731
112732
112733 #ifndef MIN
112734 # define MIN(x,y) ((x)<(y)?(x):(y))
112735 #endif
112736
112737 /*
112738 ** Maximum length of a varint encoded integer. The varint format is different
112739 ** from that used by SQLite, so the maximum length is 10, not 9.
112740 */
112741 #define FTS3_VARINT_MAX 10
112742
112743 /*
112744 ** FTS4 virtual tables may maintain multiple indexes - one index of all terms
112745 ** in the document set and zero or more prefix indexes. All indexes are stored
112746 ** as one or more b+-trees in the %_segments and %_segdir tables.
112747 **
112748 ** It is possible to determine which index a b+-tree belongs to based on the
112749 ** value stored in the "%_segdir.level" column. Given this value L, the index
112750 ** that the b+-tree belongs to is (L<<10). In other words, all b+-trees with
112751 ** level values between 0 and 1023 (inclusive) belong to index 0, all levels
112752 ** between 1024 and 2047 to index 1, and so on.
112753 **
112754 ** It is considered impossible for an index to use more than 1024 levels. In
112755 ** theory though this may happen, but only after at least
112756 ** (FTS3_MERGE_COUNT^1024) separate flushes of the pending-terms tables.
112757 */
112758 #define FTS3_SEGDIR_MAXLEVEL 1024
112759 #define FTS3_SEGDIR_MAXLEVEL_STR "1024"
112760
112761 /*
112762 ** The testcase() macro is only used by the amalgamation. If undefined,
112763 ** make it a no-op.
112764 */
112765 #ifndef testcase
@@ -111787,14 +112827,15 @@
112827 typedef struct Fts3Cursor Fts3Cursor;
112828 typedef struct Fts3Expr Fts3Expr;
112829 typedef struct Fts3Phrase Fts3Phrase;
112830 typedef struct Fts3PhraseToken Fts3PhraseToken;
112831
112832 typedef struct Fts3Doclist Fts3Doclist;
112833 typedef struct Fts3SegFilter Fts3SegFilter;
112834 typedef struct Fts3DeferredToken Fts3DeferredToken;
112835 typedef struct Fts3SegReader Fts3SegReader;
112836 typedef struct Fts3MultiSegReader Fts3MultiSegReader;
112837
112838 /*
112839 ** A connection to a fulltext index is an instance of the following
112840 ** structure. The xCreate and xConnect methods create an instance
112841 ** of this structure and xDestroy and xDisconnect free that instance.
@@ -111811,33 +112852,45 @@
112852 sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */
112853
112854 /* Precompiled statements used by the implementation. Each of these
112855 ** statements is run and reset within a single virtual table API call.
112856 */
112857 sqlite3_stmt *aStmt[27];
112858
112859 char *zReadExprlist;
112860 char *zWriteExprlist;
112861
112862 int nNodeSize; /* Soft limit for node size */
112863 u8 bHasStat; /* True if %_stat table exists */
112864 u8 bHasDocsize; /* True if %_docsize table exists */
112865 u8 bDescIdx; /* True if doclists are in reverse order */
112866 int nPgsz; /* Page size for host database */
112867 char *zSegmentsTbl; /* Name of %_segments table */
112868 sqlite3_blob *pSegments; /* Blob handle open on %_segments table */
112869
112870 /* TODO: Fix the first paragraph of this comment.
112871 **
112872 ** The following hash table is used to buffer pending index updates during
112873 ** transactions. Variable nPendingData estimates the memory size of the
112874 ** pending data, including hash table overhead, but not malloc overhead.
112875 ** When nPendingData exceeds nMaxPendingData, the buffer is flushed
112876 ** automatically. Variable iPrevDocid is the docid of the most recently
112877 ** inserted record.
112878 **
112879 ** A single FTS4 table may have multiple full-text indexes. For each index
112880 ** there is an entry in the aIndex[] array. Index 0 is an index of all the
112881 ** terms that appear in the document set. Each subsequent index in aIndex[]
112882 ** is an index of prefixes of a specific length.
112883 */
112884 int nIndex; /* Size of aIndex[] */
112885 struct Fts3Index {
112886 int nPrefix; /* Prefix length (0 for main terms index) */
112887 Fts3Hash hPending; /* Pending terms table for this index */
112888 } *aIndex;
112889 int nMaxPendingData; /* Max pending data before flush to disk */
112890 int nPendingData; /* Current bytes of pending data */
112891 sqlite_int64 iPrevDocid; /* Docid of most recently inserted document */
112892
112893 #if defined(SQLITE_DEBUG)
112894 /* State variables used for validating that the transaction control
112895 ** methods of the virtual table are called at appropriate times. These
112896 ** values do not contribution to the FTS computation; they are used for
@@ -111864,13 +112917,14 @@
112917 Fts3DeferredToken *pDeferred; /* Deferred search tokens, if any */
112918 sqlite3_int64 iPrevId; /* Previous id read from aDoclist */
112919 char *pNextId; /* Pointer into the body of aDoclist */
112920 char *aDoclist; /* List of docids for full-text queries */
112921 int nDoclist; /* Size of buffer at aDoclist */
112922 u8 bDesc; /* True to sort in descending order */
112923 int eEvalmode; /* An FTS3_EVAL_XX constant */
112924 int nRowAvg; /* Average size of database rows, in pages */
112925 sqlite3_int64 nDoc; /* Documents in table */
112926
112927 int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */
112928 u32 *aMatchinfo; /* Information about most recent match */
112929 int nMatchinfo; /* Number of elements in aMatchinfo[] */
112930 char *zMatchinfo; /* Matchinfo specification */
@@ -111897,66 +112951,90 @@
112951 */
112952 #define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */
112953 #define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */
112954 #define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */
112955
112956
112957 struct Fts3Doclist {
112958 char *aAll; /* Array containing doclist (or NULL) */
112959 int nAll; /* Size of a[] in bytes */
112960 char *pNextDocid; /* Pointer to next docid */
112961
112962 sqlite3_int64 iDocid; /* Current docid (if pList!=0) */
112963 int bFreeList; /* True if pList should be sqlite3_free()d */
112964 char *pList; /* Pointer to position list following iDocid */
112965 int nList; /* Length of position list */
112966 };
112967
112968 /*
112969 ** A "phrase" is a sequence of one or more tokens that must match in
112970 ** sequence. A single token is the base case and the most common case.
112971 ** For a sequence of tokens contained in double-quotes (i.e. "one two three")
112972 ** nToken will be the number of tokens in the string.
 
 
 
 
 
 
112973 */
112974 struct Fts3PhraseToken {
112975 char *z; /* Text of the token */
112976 int n; /* Number of bytes in buffer z */
112977 int isPrefix; /* True if token ends with a "*" character */
112978
112979 /* Variables above this point are populated when the expression is
112980 ** parsed (by code in fts3_expr.c). Below this point the variables are
112981 ** used when evaluating the expression. */
112982 Fts3DeferredToken *pDeferred; /* Deferred token object for this token */
112983 Fts3MultiSegReader *pSegcsr; /* Segment-reader for this token */
112984 };
112985
112986 struct Fts3Phrase {
112987 /* Cache of doclist for this phrase. */
112988 Fts3Doclist doclist;
112989 int bIncr; /* True if doclist is loaded incrementally */
112990 int iDoclistToken;
112991
112992 /* Variables below this point are populated by fts3_expr.c when parsing
112993 ** a MATCH expression. Everything above is part of the evaluation phase.
112994 */
112995 int nToken; /* Number of tokens in the phrase */
112996 int iColumn; /* Index of column this phrase must match */
 
112997 Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */
112998 };
112999
113000 /*
113001 ** A tree of these objects forms the RHS of a MATCH operator.
113002 **
113003 ** If Fts3Expr.eType is FTSQUERY_PHRASE and isLoaded is true, then aDoclist
113004 ** points to a malloced buffer, size nDoclist bytes, containing the results
113005 ** of this phrase query in FTS3 doclist format. As usual, the initial
113006 ** "Length" field found in doclists stored on disk is omitted from this
113007 ** buffer.
113008 **
113009 ** Variable aMI is used only for FTSQUERY_NEAR nodes to store the global
113010 ** matchinfo data. If it is not NULL, it points to an array of size nCol*3,
113011 ** where nCol is the number of columns in the queried FTS table. The array
113012 ** is populated as follows:
113013 **
113014 ** aMI[iCol*3 + 0] = Undefined
113015 ** aMI[iCol*3 + 1] = Number of occurrences
113016 ** aMI[iCol*3 + 2] = Number of rows containing at least one instance
113017 **
113018 ** The aMI array is allocated using sqlite3_malloc(). It should be freed
113019 ** when the expression node is.
113020 */
113021 struct Fts3Expr {
113022 int eType; /* One of the FTSQUERY_XXX values defined below */
113023 int nNear; /* Valid if eType==FTSQUERY_NEAR */
113024 Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */
113025 Fts3Expr *pLeft; /* Left operand */
113026 Fts3Expr *pRight; /* Right operand */
113027 Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */
113028
113029 /* The following are used by the fts3_eval.c module. */
113030 sqlite3_int64 iDocid; /* Current docid */
113031 u8 bEof; /* True this expression is at EOF already */
113032 u8 bStart; /* True if iDocid is valid */
113033 u8 bDeferred; /* True if this expression is entirely deferred */
113034
113035 u32 *aMI;
 
113036 };
113037
113038 /*
113039 ** Candidate values for Fts3Query.eType. Note that the order of the first
113040 ** four values is in order of precedence when parsing expressions. For
@@ -111980,35 +113058,36 @@
113058 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *);
113059 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *);
113060 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *);
113061 SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(int, sqlite3_int64,
113062 sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**);
113063 SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(
113064 Fts3Table*,int,const char*,int,int,Fts3SegReader**);
113065 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *);
113066 SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, int, sqlite3_stmt **);
 
113067 SQLITE_PRIVATE int sqlite3Fts3ReadLock(Fts3Table *);
113068 SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*, int*);
113069
113070 SQLITE_PRIVATE int sqlite3Fts3SelectDoctotal(Fts3Table *, sqlite3_stmt **);
113071 SQLITE_PRIVATE int sqlite3Fts3SelectDocsize(Fts3Table *, sqlite3_int64, sqlite3_stmt **);
113072
113073 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *);
113074 SQLITE_PRIVATE int sqlite3Fts3DeferToken(Fts3Cursor *, Fts3PhraseToken *, int);
113075 SQLITE_PRIVATE int sqlite3Fts3CacheDeferredDoclists(Fts3Cursor *);
113076 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *);
 
113077 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *);
113078
113079 /* Special values interpreted by sqlite3SegReaderCursor() */
113080 #define FTS3_SEGCURSOR_PENDING -1
113081 #define FTS3_SEGCURSOR_ALL -2
113082
113083 SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3MultiSegReader*, Fts3SegFilter*);
113084 SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3MultiSegReader *);
113085 SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3MultiSegReader *);
113086
113087 SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
113088 Fts3Table *, int, int, const char *, int, int, int, Fts3MultiSegReader *);
113089
113090 /* Flags allowed as part of the 4th argument to SegmentReaderIterate() */
113091 #define FTS3_SEGMENT_REQUIRE_POS 0x00000001
113092 #define FTS3_SEGMENT_IGNORE_EMPTY 0x00000002
113093 #define FTS3_SEGMENT_COLUMN_FILTER 0x00000004
@@ -112021,21 +113100,25 @@
113100 int nTerm;
113101 int iCol;
113102 int flags;
113103 };
113104
113105 struct Fts3MultiSegReader {
113106 /* Used internally by sqlite3Fts3SegReaderXXX() calls */
113107 Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */
113108 int nSegment; /* Size of apSegment array */
113109 int nAdvance; /* How many seg-readers to advance */
113110 Fts3SegFilter *pFilter; /* Pointer to filter object */
113111 char *aBuffer; /* Buffer to merge doclists in */
113112 int nBuffer; /* Allocated size of aBuffer[] in bytes */
113113
113114 int iColFilter; /* If >=0, filter for this column */
113115 int bRestart;
113116
113117 /* Used by fts3.c only. */
113118 int nCost; /* Cost of running iterator */
113119 int bLookup; /* True if a lookup of a single entry. */
113120
113121 /* Output values. Valid only after Fts3SegReaderStep() returns SQLITE_ROW. */
113122 char *zTerm; /* Pointer to term buffer */
113123 int nTerm; /* Size of zTerm in bytes */
113124 char *aDoclist; /* Pointer to doclist buffer */
@@ -112046,15 +113129,13 @@
113129 SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *, sqlite3_int64);
113130 SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *, sqlite_int64 *);
113131 SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *, int *);
113132 SQLITE_PRIVATE int sqlite3Fts3VarintLen(sqlite3_uint64);
113133 SQLITE_PRIVATE void sqlite3Fts3Dequote(char *);
113134 SQLITE_PRIVATE void sqlite3Fts3DoclistPrev(int,char*,int,char**,sqlite3_int64*,int*,u8*);
113135
113136 SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *);
 
 
 
113137
113138 /* fts3_tokenizer.c */
113139 SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *, int *);
113140 SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *);
113141 SQLITE_PRIVATE int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *,
@@ -112080,20 +113161,43 @@
113161 #endif
113162
113163 /* fts3_aux.c */
113164 SQLITE_PRIVATE int sqlite3Fts3InitAux(sqlite3 *db);
113165
113166 SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *);
113167
113168 SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart(
113169 Fts3Table*, Fts3MultiSegReader*, int, const char*, int);
113170 SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext(
113171 Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *);
113172 SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol);
113173 SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *);
113174 SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr);
113175
113176 SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(Fts3DeferredToken *, char **, int *);
113177
113178 #endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */
113179 #endif /* _FTSINT_H */
113180
113181 /************** End of fts3Int.h *********************************************/
113182 /************** Continuing where we left off in fts3.c ***********************/
113183 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
113184
113185 #if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE)
113186 # define SQLITE_CORE 1
113187 #endif
113188
113189
113190 #ifndef SQLITE_CORE
113191 SQLITE_EXTENSION_INIT1
113192 #endif
113193
113194 static int fts3EvalNext(Fts3Cursor *pCsr);
113195 static int fts3EvalStart(Fts3Cursor *pCsr);
113196 static int fts3TermSegReaderCursor(
113197 Fts3Cursor *, const char *, int, int, Fts3MultiSegReader **);
113198
113199 /*
113200 ** Write a 64-bit variable-length integer to memory starting at p[0].
113201 ** The length of data written will be between 1 and FTS3_VARINT_MAX bytes.
113202 ** The number of bytes written is returned.
113203 */
@@ -112200,16 +113304,16 @@
113304
113305 /*
113306 ** When this function is called, *pp points to the first byte following a
113307 ** varint that is part of a doclist (or position-list, or any other list
113308 ** of varints). This function moves *pp to point to the start of that varint,
113309 ** and sets *pVal by the varint value.
113310 **
113311 ** Argument pStart points to the first byte of the doclist that the
113312 ** varint is part of.
113313 */
113314 static void fts3GetReverseVarint(
113315 char **pp,
113316 char *pStart,
113317 sqlite3_int64 *pVal
113318 ){
113319 sqlite3_int64 iVal;
@@ -112221,25 +113325,11 @@
113325 for(p = (*pp)-2; p>=pStart && *p&0x80; p--);
113326 p++;
113327 *pp = p;
113328
113329 sqlite3Fts3GetVarint(p, &iVal);
113330 *pVal = iVal;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
113331 }
113332
113333 /*
113334 ** The xDisconnect() virtual table method.
113335 */
@@ -112436,10 +113526,13 @@
113526 rc = sqlite3_prepare(p->db, zSql, -1, &pStmt, 0);
113527 if( rc==SQLITE_OK ){
113528 sqlite3_step(pStmt);
113529 p->nPgsz = sqlite3_column_int(pStmt, 0);
113530 rc = sqlite3_finalize(pStmt);
113531 }else if( rc==SQLITE_AUTH ){
113532 p->nPgsz = 1024;
113533 rc = SQLITE_OK;
113534 }
113535 }
113536 assert( p->nPgsz>0 || rc!=SQLITE_OK );
113537 sqlite3_free(zSql);
113538 *pRc = rc;
@@ -112608,10 +113701,91 @@
113701 fts3Appendf(pRc, &zRet, ",%s(?)", zFunction);
113702 }
113703 sqlite3_free(zFree);
113704 return zRet;
113705 }
113706
113707 /*
113708 ** This function interprets the string at (*pp) as a non-negative integer
113709 ** value. It reads the integer and sets *pnOut to the value read, then
113710 ** sets *pp to point to the byte immediately following the last byte of
113711 ** the integer value.
113712 **
113713 ** Only decimal digits ('0'..'9') may be part of an integer value.
113714 **
113715 ** If *pp does not being with a decimal digit SQLITE_ERROR is returned and
113716 ** the output value undefined. Otherwise SQLITE_OK is returned.
113717 **
113718 ** This function is used when parsing the "prefix=" FTS4 parameter.
113719 */
113720 static int fts3GobbleInt(const char **pp, int *pnOut){
113721 const char *p = *pp; /* Iterator pointer */
113722 int nInt = 0; /* Output value */
113723
113724 for(p=*pp; p[0]>='0' && p[0]<='9'; p++){
113725 nInt = nInt * 10 + (p[0] - '0');
113726 }
113727 if( p==*pp ) return SQLITE_ERROR;
113728 *pnOut = nInt;
113729 *pp = p;
113730 return SQLITE_OK;
113731 }
113732
113733 /*
113734 ** This function is called to allocate an array of Fts3Index structures
113735 ** representing the indexes maintained by the current FTS table. FTS tables
113736 ** always maintain the main "terms" index, but may also maintain one or
113737 ** more "prefix" indexes, depending on the value of the "prefix=" parameter
113738 ** (if any) specified as part of the CREATE VIRTUAL TABLE statement.
113739 **
113740 ** Argument zParam is passed the value of the "prefix=" option if one was
113741 ** specified, or NULL otherwise.
113742 **
113743 ** If no error occurs, SQLITE_OK is returned and *apIndex set to point to
113744 ** the allocated array. *pnIndex is set to the number of elements in the
113745 ** array. If an error does occur, an SQLite error code is returned.
113746 **
113747 ** Regardless of whether or not an error is returned, it is the responsibility
113748 ** of the caller to call sqlite3_free() on the output array to free it.
113749 */
113750 static int fts3PrefixParameter(
113751 const char *zParam, /* ABC in prefix=ABC parameter to parse */
113752 int *pnIndex, /* OUT: size of *apIndex[] array */
113753 struct Fts3Index **apIndex /* OUT: Array of indexes for this table */
113754 ){
113755 struct Fts3Index *aIndex; /* Allocated array */
113756 int nIndex = 1; /* Number of entries in array */
113757
113758 if( zParam && zParam[0] ){
113759 const char *p;
113760 nIndex++;
113761 for(p=zParam; *p; p++){
113762 if( *p==',' ) nIndex++;
113763 }
113764 }
113765
113766 aIndex = sqlite3_malloc(sizeof(struct Fts3Index) * nIndex);
113767 *apIndex = aIndex;
113768 *pnIndex = nIndex;
113769 if( !aIndex ){
113770 return SQLITE_NOMEM;
113771 }
113772
113773 memset(aIndex, 0, sizeof(struct Fts3Index) * nIndex);
113774 if( zParam ){
113775 const char *p = zParam;
113776 int i;
113777 for(i=1; i<nIndex; i++){
113778 int nPrefix;
113779 if( fts3GobbleInt(&p, &nPrefix) ) return SQLITE_ERROR;
113780 aIndex[i].nPrefix = nPrefix;
113781 p++;
113782 }
113783 }
113784
113785 return SQLITE_OK;
113786 }
113787
113788 /*
113789 ** This function is the implementation of both the xConnect and xCreate
113790 ** methods of the FTS3 virtual table.
113791 **
@@ -112641,16 +113815,22 @@
113815 int nCol = 0; /* Number of columns in the FTS table */
113816 char *zCsr; /* Space for holding column names */
113817 int nDb; /* Bytes required to hold database name */
113818 int nName; /* Bytes required to hold table name */
113819 int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */
 
113820 const char **aCol; /* Array of column names */
113821 sqlite3_tokenizer *pTokenizer = 0; /* Tokenizer for this table */
113822
113823 int nIndex; /* Size of aIndex[] array */
113824 struct Fts3Index *aIndex = 0; /* Array of indexes for this table */
113825
113826 /* The results of parsing supported FTS4 key=value options: */
113827 int bNoDocsize = 0; /* True to omit %_docsize table */
113828 int bDescIdx = 0; /* True to store descending indexes */
113829 char *zPrefix = 0; /* Prefix parameter value (or NULL) */
113830 char *zCompress = 0; /* compress=? parameter (or NULL) */
113831 char *zUncompress = 0; /* uncompress=? parameter (or NULL) */
113832
113833 assert( strlen(argv[0])==4 );
113834 assert( (sqlite3_strnicmp(argv[0], "fts4", 4)==0 && isFts4)
113835 || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4)
113836 );
@@ -112687,32 +113867,76 @@
113867 rc = sqlite3Fts3InitTokenizer(pHash, &z[9], &pTokenizer, pzErr);
113868 }
113869
113870 /* Check if it is an FTS4 special argument. */
113871 else if( isFts4 && fts3IsSpecialColumn(z, &nKey, &zVal) ){
113872 struct Fts4Option {
113873 const char *zOpt;
113874 int nOpt;
113875 char **pzVar;
113876 } aFts4Opt[] = {
113877 { "matchinfo", 9, 0 }, /* 0 -> MATCHINFO */
113878 { "prefix", 6, 0 }, /* 1 -> PREFIX */
113879 { "compress", 8, 0 }, /* 2 -> COMPRESS */
113880 { "uncompress", 10, 0 }, /* 3 -> UNCOMPRESS */
113881 { "order", 5, 0 } /* 4 -> ORDER */
113882 };
113883
113884 int iOpt;
113885 if( !zVal ){
113886 rc = SQLITE_NOMEM;
113887 }else{
113888 for(iOpt=0; iOpt<SizeofArray(aFts4Opt); iOpt++){
113889 struct Fts4Option *pOp = &aFts4Opt[iOpt];
113890 if( nKey==pOp->nOpt && !sqlite3_strnicmp(z, pOp->zOpt, pOp->nOpt) ){
113891 break;
113892 }
113893 }
113894 if( iOpt==SizeofArray(aFts4Opt) ){
113895 *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z);
113896 rc = SQLITE_ERROR;
113897 }else{
113898 switch( iOpt ){
113899 case 0: /* MATCHINFO */
113900 if( strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "fts3", 4) ){
113901 *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal);
113902 rc = SQLITE_ERROR;
113903 }
113904 bNoDocsize = 1;
113905 break;
113906
113907 case 1: /* PREFIX */
113908 sqlite3_free(zPrefix);
113909 zPrefix = zVal;
113910 zVal = 0;
113911 break;
113912
113913 case 2: /* COMPRESS */
113914 sqlite3_free(zCompress);
113915 zCompress = zVal;
113916 zVal = 0;
113917 break;
113918
113919 case 3: /* UNCOMPRESS */
113920 sqlite3_free(zUncompress);
113921 zUncompress = zVal;
113922 zVal = 0;
113923 break;
113924
113925 case 4: /* ORDER */
113926 if( (strlen(zVal)!=3 || sqlite3_strnicmp(zVal, "asc", 3))
113927 && (strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "desc", 3))
113928 ){
113929 *pzErr = sqlite3_mprintf("unrecognized order: %s", zVal);
113930 rc = SQLITE_ERROR;
113931 }
113932 bDescIdx = (zVal[0]=='d' || zVal[0]=='D');
113933 break;
113934 }
113935 }
113936 sqlite3_free(zVal);
113937 }
113938 }
113939
113940 /* Otherwise, the argument is a column name. */
113941 else {
113942 nString += (int)(strlen(z) + 1);
@@ -112732,14 +113956,21 @@
113956 rc = sqlite3Fts3InitTokenizer(pHash, "simple", &pTokenizer, pzErr);
113957 if( rc!=SQLITE_OK ) goto fts3_init_out;
113958 }
113959 assert( pTokenizer );
113960
113961 rc = fts3PrefixParameter(zPrefix, &nIndex, &aIndex);
113962 if( rc==SQLITE_ERROR ){
113963 assert( zPrefix );
113964 *pzErr = sqlite3_mprintf("error parsing prefix parameter: %s", zPrefix);
113965 }
113966 if( rc!=SQLITE_OK ) goto fts3_init_out;
113967
113968 /* Allocate and populate the Fts3Table structure. */
113969 nByte = sizeof(Fts3Table) + /* Fts3Table */
113970 nCol * sizeof(char *) + /* azColumn */
113971 nIndex * sizeof(struct Fts3Index) + /* aIndex */
113972 nName + /* zName */
113973 nDb + /* zDb */
113974 nString; /* Space for azColumn strings */
113975 p = (Fts3Table*)sqlite3_malloc(nByte);
113976 if( p==0 ){
@@ -112750,20 +113981,26 @@
113981 p->db = db;
113982 p->nColumn = nCol;
113983 p->nPendingData = 0;
113984 p->azColumn = (char **)&p[1];
113985 p->pTokenizer = pTokenizer;
 
113986 p->nMaxPendingData = FTS3_MAX_PENDING_DATA;
113987 p->bHasDocsize = (isFts4 && bNoDocsize==0);
113988 p->bHasStat = isFts4;
113989 p->bDescIdx = bDescIdx;
113990 TESTONLY( p->inTransaction = -1 );
113991 TESTONLY( p->mxSavepoint = -1 );
113992
113993 p->aIndex = (struct Fts3Index *)&p->azColumn[nCol];
113994 memcpy(p->aIndex, aIndex, sizeof(struct Fts3Index) * nIndex);
113995 p->nIndex = nIndex;
113996 for(i=0; i<nIndex; i++){
113997 fts3HashInit(&p->aIndex[i].hPending, FTS3_HASH_STRING, 1);
113998 }
113999
114000 /* Fill in the zName and zDb fields of the vtab structure. */
114001 zCsr = (char *)&p->aIndex[nIndex];
114002 p->zName = zCsr;
114003 memcpy(zCsr, argv[2], nName);
114004 zCsr += nName;
114005 p->zDb = zCsr;
114006 memcpy(zCsr, argv[1], nDb);
@@ -112770,11 +114007,11 @@
114007 zCsr += nDb;
114008
114009 /* Fill in the azColumn array */
114010 for(iCol=0; iCol<nCol; iCol++){
114011 char *z;
114012 int n = 0;
114013 z = (char *)sqlite3Fts3NextToken(aCol[iCol], &n);
114014 memcpy(zCsr, z, n);
114015 zCsr[n] = '\0';
114016 sqlite3Fts3Dequote(zCsr);
114017 p->azColumn[iCol] = zCsr;
@@ -112797,19 +114034,20 @@
114034 if( isCreate ){
114035 rc = fts3CreateTables(p);
114036 }
114037
114038 /* Figure out the page-size for the database. This is required in order to
114039 ** estimate the cost of loading large doclists from the database. */
 
 
114040 fts3DatabasePageSize(&rc, p);
114041 p->nNodeSize = p->nPgsz-35;
114042
114043 /* Declare the table schema to SQLite. */
114044 fts3DeclareVtab(&rc, p);
114045
114046 fts3_init_out:
114047 sqlite3_free(zPrefix);
114048 sqlite3_free(aIndex);
114049 sqlite3_free(zCompress);
114050 sqlite3_free(zUncompress);
114051 sqlite3_free((void *)aCol);
114052 if( rc!=SQLITE_OK ){
114053 if( p ){
@@ -112816,10 +114054,11 @@
114054 fts3DisconnectMethod((sqlite3_vtab *)p);
114055 }else if( pTokenizer ){
114056 pTokenizer->pModule->xDestroy(pTokenizer);
114057 }
114058 }else{
114059 assert( p->pSegments==0 );
114060 *ppVTab = &p->base;
114061 }
114062 return rc;
114063 }
114064
@@ -112913,14 +114152,15 @@
114152 if( pOrder->desc ){
114153 pInfo->idxStr = "DESC";
114154 }else{
114155 pInfo->idxStr = "ASC";
114156 }
114157 pInfo->orderByConsumed = 1;
114158 }
 
114159 }
114160
114161 assert( p->pSegments==0 );
114162 return SQLITE_OK;
114163 }
114164
114165 /*
114166 ** Implementation of xOpen method.
@@ -112952,10 +114192,11 @@
114192 sqlite3_finalize(pCsr->pStmt);
114193 sqlite3Fts3ExprFree(pCsr->pExpr);
114194 sqlite3Fts3FreeDeferredTokens(pCsr);
114195 sqlite3_free(pCsr->aDoclist);
114196 sqlite3_free(pCsr->aMatchinfo);
114197 assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
114198 sqlite3_free(pCsr);
114199 return SQLITE_OK;
114200 }
114201
114202 /*
@@ -112963,12 +114204,12 @@
114204 ** of the %_content table that contains the last match. Return
114205 ** SQLITE_OK on success.
114206 */
114207 static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){
114208 if( pCsr->isRequireSeek ){
 
114209 sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId);
114210 pCsr->isRequireSeek = 0;
114211 if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){
114212 return SQLITE_OK;
114213 }else{
114214 int rc = sqlite3_reset(pCsr->pStmt);
114215 if( rc==SQLITE_OK ){
@@ -113145,21 +114386,21 @@
114386 if( rc==SQLITE_OK && iHeight>1 ){
114387 char *zBlob = 0; /* Blob read from %_segments table */
114388 int nBlob; /* Size of zBlob in bytes */
114389
114390 if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){
114391 rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob, 0);
114392 if( rc==SQLITE_OK ){
114393 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, 0);
114394 }
114395 sqlite3_free(zBlob);
114396 piLeaf = 0;
114397 zBlob = 0;
114398 }
114399
114400 if( rc==SQLITE_OK ){
114401 rc = sqlite3Fts3ReadBlock(p, piLeaf?*piLeaf:*piLeaf2, &zBlob, &nBlob, 0);
114402 }
114403 if( rc==SQLITE_OK ){
114404 rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2);
114405 }
114406 sqlite3_free(zBlob);
@@ -113393,12 +114634,10 @@
114634 *pp1 = p1 + 1;
114635 *pp2 = p2 + 1;
114636 }
114637
114638 /*
 
 
114639 ** This function is used to merge two position lists into one. When it is
114640 ** called, *pp1 and *pp2 must both point to position lists. A position-list is
114641 ** the part of a doclist that follows each document id. For example, if a row
114642 ** contains:
114643 **
@@ -113414,10 +114653,12 @@
114653 ** If isSaveLeft is 0, an entry is added to the output position list for
114654 ** each position in *pp2 for which there exists one or more positions in
114655 ** *pp1 so that (pos(*pp2)>pos(*pp1) && pos(*pp2)-pos(*pp1)<=nToken). i.e.
114656 ** when the *pp1 token appears before the *pp2 token, but not more than nToken
114657 ** slots before it.
114658 **
114659 ** e.g. nToken==1 searches for adjacent positions.
114660 */
114661 static int fts3PoslistPhraseMerge(
114662 char **pp, /* IN/OUT: Preallocated output buffer */
114663 int nToken, /* Maximum difference in token positions */
114664 int isSaveLeft, /* Save the left position */
@@ -113531,11 +114772,23 @@
114772 *pp = p;
114773 return 1;
114774 }
114775
114776 /*
114777 ** Merge two position-lists as required by the NEAR operator. The argument
114778 ** position lists correspond to the left and right phrases of an expression
114779 ** like:
114780 **
114781 ** "phrase 1" NEAR "phrase number 2"
114782 **
114783 ** Position list *pp1 corresponds to the left-hand side of the NEAR
114784 ** expression and *pp2 to the right. As usual, the indexes in the position
114785 ** lists are the offsets of the last token in each phrase (tokens "1" and "2"
114786 ** in the example above).
114787 **
114788 ** The output position list - written to *pp - is a copy of *pp2 with those
114789 ** entries that are not sufficiently NEAR entries in *pp1 removed.
114790 */
114791 static int fts3PoslistNearMerge(
114792 char **pp, /* Output buffer */
114793 char *aTmp, /* Temporary buffer space */
114794 int nRight, /* Maximum difference in token positions */
@@ -113544,230 +114797,248 @@
114797 char **pp2 /* IN/OUT: Right input list */
114798 ){
114799 char *p1 = *pp1;
114800 char *p2 = *pp2;
114801
114802 char *pTmp1 = aTmp;
114803 char *pTmp2;
114804 char *aTmp2;
114805 int res = 1;
114806
114807 fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2);
114808 aTmp2 = pTmp2 = pTmp1;
114809 *pp1 = p1;
114810 *pp2 = p2;
114811 fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1);
114812 if( pTmp1!=aTmp && pTmp2!=aTmp2 ){
114813 fts3PoslistMerge(pp, &aTmp, &aTmp2);
114814 }else if( pTmp1!=aTmp ){
114815 fts3PoslistCopy(pp, &aTmp);
114816 }else if( pTmp2!=aTmp2 ){
114817 fts3PoslistCopy(pp, &aTmp2);
114818 }else{
114819 res = 0;
114820 }
114821
114822 return res;
114823 }
114824
114825 /*
114826 ** An instance of this function is used to merge together the (potentially
114827 ** large number of) doclists for each term that matches a prefix query.
114828 ** See function fts3TermSelectMerge() for details.
114829 */
114830 typedef struct TermSelect TermSelect;
114831 struct TermSelect {
114832 char *aaOutput[16]; /* Malloc'd output buffers */
114833 int anOutput[16]; /* Size each output buffer in bytes */
114834 };
114835
114836 /*
114837 ** This function is used to read a single varint from a buffer. Parameter
114838 ** pEnd points 1 byte past the end of the buffer. When this function is
114839 ** called, if *pp points to pEnd or greater, then the end of the buffer
114840 ** has been reached. In this case *pp is set to 0 and the function returns.
114841 **
114842 ** If *pp does not point to or past pEnd, then a single varint is read
114843 ** from *pp. *pp is then set to point 1 byte past the end of the read varint.
114844 **
114845 ** If bDescIdx is false, the value read is added to *pVal before returning.
114846 ** If it is true, the value read is subtracted from *pVal before this
114847 ** function returns.
114848 */
114849 static void fts3GetDeltaVarint3(
114850 char **pp, /* IN/OUT: Point to read varint from */
114851 char *pEnd, /* End of buffer */
114852 int bDescIdx, /* True if docids are descending */
114853 sqlite3_int64 *pVal /* IN/OUT: Integer value */
114854 ){
114855 if( *pp>=pEnd ){
114856 *pp = 0;
114857 }else{
114858 sqlite3_int64 iVal;
114859 *pp += sqlite3Fts3GetVarint(*pp, &iVal);
114860 if( bDescIdx ){
114861 *pVal -= iVal;
114862 }else{
114863 *pVal += iVal;
114864 }
114865 }
114866 }
114867
114868 /*
114869 ** This function is used to write a single varint to a buffer. The varint
114870 ** is written to *pp. Before returning, *pp is set to point 1 byte past the
114871 ** end of the value written.
114872 **
114873 ** If *pbFirst is zero when this function is called, the value written to
114874 ** the buffer is that of parameter iVal.
114875 **
114876 ** If *pbFirst is non-zero when this function is called, then the value
114877 ** written is either (iVal-*piPrev) (if bDescIdx is zero) or (*piPrev-iVal)
114878 ** (if bDescIdx is non-zero).
114879 **
114880 ** Before returning, this function always sets *pbFirst to 1 and *piPrev
114881 ** to the value of parameter iVal.
114882 */
114883 static void fts3PutDeltaVarint3(
114884 char **pp, /* IN/OUT: Output pointer */
114885 int bDescIdx, /* True for descending docids */
114886 sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */
114887 int *pbFirst, /* IN/OUT: True after first int written */
114888 sqlite3_int64 iVal /* Write this value to the list */
114889 ){
114890 sqlite3_int64 iWrite;
114891 if( bDescIdx==0 || *pbFirst==0 ){
114892 iWrite = iVal - *piPrev;
114893 }else{
114894 iWrite = *piPrev - iVal;
114895 }
114896 assert( *pbFirst || *piPrev==0 );
114897 assert( *pbFirst==0 || iWrite>0 );
114898 *pp += sqlite3Fts3PutVarint(*pp, iWrite);
114899 *piPrev = iVal;
114900 *pbFirst = 1;
114901 }
114902
114903
114904 /*
114905 ** This macro is used by various functions that merge doclists. The two
114906 ** arguments are 64-bit docid values. If the value of the stack variable
114907 ** bDescDoclist is 0 when this macro is invoked, then it returns (i1-i2).
114908 ** Otherwise, (i2-i1).
114909 **
114910 ** Using this makes it easier to write code that can merge doclists that are
114911 ** sorted in either ascending or descending order.
114912 */
114913 #define DOCID_CMP(i1, i2) ((bDescDoclist?-1:1) * (i1-i2))
114914
114915 /*
114916 ** This function does an "OR" merge of two doclists (output contains all
114917 ** positions contained in either argument doclist). If the docids in the
114918 ** input doclists are sorted in ascending order, parameter bDescDoclist
114919 ** should be false. If they are sorted in ascending order, it should be
114920 ** passed a non-zero value.
114921 **
114922 ** If no error occurs, *paOut is set to point at an sqlite3_malloc'd buffer
114923 ** containing the output doclist and SQLITE_OK is returned. In this case
114924 ** *pnOut is set to the number of bytes in the output doclist.
114925 **
114926 ** If an error occurs, an SQLite error code is returned. The output values
114927 ** are undefined in this case.
114928 */
114929 static int fts3DoclistOrMerge(
114930 int bDescDoclist, /* True if arguments are desc */
114931 char *a1, int n1, /* First doclist */
114932 char *a2, int n2, /* Second doclist */
114933 char **paOut, int *pnOut /* OUT: Malloc'd doclist */
114934 ){
114935 sqlite3_int64 i1 = 0;
114936 sqlite3_int64 i2 = 0;
114937 sqlite3_int64 iPrev = 0;
114938 char *pEnd1 = &a1[n1];
114939 char *pEnd2 = &a2[n2];
114940 char *p1 = a1;
114941 char *p2 = a2;
114942 char *p;
114943 char *aOut;
114944 int bFirstOut = 0;
114945
114946 *paOut = 0;
114947 *pnOut = 0;
114948 aOut = sqlite3_malloc(n1+n2);
114949 if( !aOut ) return SQLITE_NOMEM;
114950
114951 p = aOut;
114952 fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1);
114953 fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2);
114954 while( p1 || p2 ){
114955 sqlite3_int64 iDiff = DOCID_CMP(i1, i2);
114956
114957 if( p2 && p1 && iDiff==0 ){
114958 fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
114959 fts3PoslistMerge(&p, &p1, &p2);
114960 fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
114961 fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
114962 }else if( !p2 || (p1 && iDiff<0) ){
114963 fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
114964 fts3PoslistCopy(&p, &p1);
114965 fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
114966 }else{
114967 fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i2);
114968 fts3PoslistCopy(&p, &p2);
114969 fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
114970 }
114971 }
114972
114973 *paOut = aOut;
114974 *pnOut = (p-aOut);
114975 return SQLITE_OK;
114976 }
114977
114978 /*
114979 ** This function does a "phrase" merge of two doclists. In a phrase merge,
114980 ** the output contains a copy of each position from the right-hand input
114981 ** doclist for which there is a position in the left-hand input doclist
114982 ** exactly nDist tokens before it.
114983 **
114984 ** If the docids in the input doclists are sorted in ascending order,
114985 ** parameter bDescDoclist should be false. If they are sorted in ascending
114986 ** order, it should be passed a non-zero value.
114987 **
114988 ** The right-hand input doclist is overwritten by this function.
114989 */
114990 static void fts3DoclistPhraseMerge(
114991 int bDescDoclist, /* True if arguments are desc */
114992 int nDist, /* Distance from left to right (1=adjacent) */
114993 char *aLeft, int nLeft, /* Left doclist */
114994 char *aRight, int *pnRight /* IN/OUT: Right/output doclist */
114995 ){
114996 sqlite3_int64 i1 = 0;
114997 sqlite3_int64 i2 = 0;
114998 sqlite3_int64 iPrev = 0;
114999 char *pEnd1 = &aLeft[nLeft];
115000 char *pEnd2 = &aRight[*pnRight];
115001 char *p1 = aLeft;
115002 char *p2 = aRight;
115003 char *p;
115004 int bFirstOut = 0;
115005 char *aOut = aRight;
115006
115007 assert( nDist>0 );
115008
115009 p = aOut;
115010 fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1);
115011 fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2);
115012
115013 while( p1 && p2 ){
115014 sqlite3_int64 iDiff = DOCID_CMP(i1, i2);
115015 if( iDiff==0 ){
115016 char *pSave = p;
115017 sqlite3_int64 iPrevSave = iPrev;
115018 int bFirstOutSave = bFirstOut;
115019
115020 fts3PutDeltaVarint3(&p, bDescDoclist, &iPrev, &bFirstOut, i1);
115021 if( 0==fts3PoslistPhraseMerge(&p, nDist, 0, 1, &p1, &p2) ){
115022 p = pSave;
115023 iPrev = iPrevSave;
115024 bFirstOut = bFirstOutSave;
115025 }
115026 fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
115027 fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
115028 }else if( iDiff<0 ){
115029 fts3PoslistCopy(0, &p1);
115030 fts3GetDeltaVarint3(&p1, pEnd1, bDescDoclist, &i1);
115031 }else{
115032 fts3PoslistCopy(0, &p2);
115033 fts3GetDeltaVarint3(&p2, pEnd2, bDescDoclist, &i2);
115034 }
115035 }
115036
115037 *pnRight = p - aOut;
115038 }
115039
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115040
115041 /*
115042 ** Merge all doclists in the TermSelect.aaOutput[] array into a single
115043 ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all
115044 ** other doclists (except the aaOutput[0] one) and return SQLITE_OK.
@@ -113774,12 +115045,11 @@
115045 **
115046 ** If an OOM error occurs, return SQLITE_NOMEM. In this case it is
115047 ** the responsibility of the caller to free any doclists left in the
115048 ** TermSelect.aaOutput[] array.
115049 */
115050 static int fts3TermSelectFinishMerge(Fts3Table *p, TermSelect *pTS){
 
115051 char *aOut = 0;
115052 int nOut = 0;
115053 int i;
115054
115055 /* Loop through the doclists in the aaOutput[] array. Merge them all
@@ -113790,19 +115060,21 @@
115060 if( !aOut ){
115061 aOut = pTS->aaOutput[i];
115062 nOut = pTS->anOutput[i];
115063 pTS->aaOutput[i] = 0;
115064 }else{
115065 int nNew;
115066 char *aNew;
115067
115068 int rc = fts3DoclistOrMerge(p->bDescIdx,
115069 pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, &aNew, &nNew
115070 );
115071 if( rc!=SQLITE_OK ){
115072 sqlite3_free(aOut);
115073 return rc;
115074 }
115075
 
 
115076 sqlite3_free(pTS->aaOutput[i]);
115077 sqlite3_free(aOut);
115078 pTS->aaOutput[i] = 0;
115079 aOut = aNew;
115080 nOut = nNew;
@@ -113814,288 +115086,329 @@
115086 pTS->anOutput[0] = nOut;
115087 return SQLITE_OK;
115088 }
115089
115090 /*
115091 ** Merge the doclist aDoclist/nDoclist into the TermSelect object passed
115092 ** as the first argument. The merge is an "OR" merge (see function
115093 ** fts3DoclistOrMerge() for details).
115094 **
115095 ** This function is called with the doclist for each term that matches
115096 ** a queried prefix. It merges all these doclists into one, the doclist
115097 ** for the specified prefix. Since there can be a very large number of
115098 ** doclists to merge, the merging is done pair-wise using the TermSelect
115099 ** object.
115100 **
115101 ** This function returns SQLITE_OK if the merge is successful, or an
115102 ** SQLite error code (SQLITE_NOMEM) if an error occurs.
115103 */
115104 static int fts3TermSelectMerge(
115105 Fts3Table *p, /* FTS table handle */
115106 TermSelect *pTS, /* TermSelect object to merge into */
115107 char *aDoclist, /* Pointer to doclist */
115108 int nDoclist /* Size of aDoclist in bytes */
 
 
115109 ){
 
 
 
 
 
 
115110 if( pTS->aaOutput[0]==0 ){
115111 /* If this is the first term selected, copy the doclist to the output
115112 ** buffer using memcpy(). */
 
 
115113 pTS->aaOutput[0] = sqlite3_malloc(nDoclist);
115114 pTS->anOutput[0] = nDoclist;
115115 if( pTS->aaOutput[0] ){
115116 memcpy(pTS->aaOutput[0], aDoclist, nDoclist);
115117 }else{
115118 return SQLITE_NOMEM;
115119 }
115120 }else{
 
115121 char *aMerge = aDoclist;
115122 int nMerge = nDoclist;
115123 int iOut;
115124
115125 for(iOut=0; iOut<SizeofArray(pTS->aaOutput); iOut++){
 
 
115126 if( pTS->aaOutput[iOut]==0 ){
115127 assert( iOut>0 );
115128 pTS->aaOutput[iOut] = aMerge;
115129 pTS->anOutput[iOut] = nMerge;
115130 break;
115131 }else{
115132 char *aNew;
115133 int nNew;
115134
115135 int rc = fts3DoclistOrMerge(p->bDescIdx, aMerge, nMerge,
115136 pTS->aaOutput[iOut], pTS->anOutput[iOut], &aNew, &nNew
115137 );
115138 if( rc!=SQLITE_OK ){
115139 if( aMerge!=aDoclist ) sqlite3_free(aMerge);
115140 return rc;
115141 }
115142
115143 if( aMerge!=aDoclist ) sqlite3_free(aMerge);
115144 sqlite3_free(pTS->aaOutput[iOut]);
115145 pTS->aaOutput[iOut] = 0;
115146
115147 aMerge = aNew;
115148 nMerge = nNew;
115149 if( (iOut+1)==SizeofArray(pTS->aaOutput) ){
115150 pTS->aaOutput[iOut] = aMerge;
115151 pTS->anOutput[iOut] = nMerge;
115152 }
115153 }
115154 }
115155 }
115156 return SQLITE_OK;
115157 }
115158
115159 /*
115160 ** Append SegReader object pNew to the end of the pCsr->apSegment[] array.
115161 */
115162 static int fts3SegReaderCursorAppend(
115163 Fts3MultiSegReader *pCsr,
115164 Fts3SegReader *pNew
115165 ){
115166 if( (pCsr->nSegment%16)==0 ){
115167 Fts3SegReader **apNew;
115168 int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*);
115169 apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte);
115170 if( !apNew ){
115171 sqlite3Fts3SegReaderFree(pNew);
115172 return SQLITE_NOMEM;
115173 }
115174 pCsr->apSegment = apNew;
115175 }
115176 pCsr->apSegment[pCsr->nSegment++] = pNew;
115177 return SQLITE_OK;
115178 }
115179
115180 /*
115181 ** Add seg-reader objects to the Fts3MultiSegReader object passed as the
115182 ** 8th argument.
115183 **
115184 ** This function returns SQLITE_OK if successful, or an SQLite error code
115185 ** otherwise.
115186 */
115187 static int fts3SegReaderCursor(
 
 
115188 Fts3Table *p, /* FTS3 table handle */
115189 int iIndex, /* Index to search (from 0 to p->nIndex-1) */
115190 int iLevel, /* Level of segments to scan */
115191 const char *zTerm, /* Term to query for */
115192 int nTerm, /* Size of zTerm in bytes */
115193 int isPrefix, /* True for a prefix search */
115194 int isScan, /* True to scan from zTerm to EOF */
115195 Fts3MultiSegReader *pCsr /* Cursor object to populate */
115196 ){
115197 int rc = SQLITE_OK; /* Error code */
115198 sqlite3_stmt *pStmt = 0; /* Statement to iterate through segments */
115199 int rc2; /* Result of sqlite3_reset() */
115200
115201 /* If iLevel is less than 0 and this is not a scan, include a seg-reader
115202 ** for the pending-terms. If this is a scan, then this call must be being
115203 ** made by an fts4aux module, not an FTS table. In this case calling
115204 ** Fts3SegReaderPending might segfault, as the data structures used by
115205 ** fts4aux are not completely populated. So it's easiest to filter these
115206 ** calls out here. */
115207 if( iLevel<0 && p->aIndex ){
115208 Fts3SegReader *pSeg = 0;
115209 rc = sqlite3Fts3SegReaderPending(p, iIndex, zTerm, nTerm, isPrefix, &pSeg);
115210 if( rc==SQLITE_OK && pSeg ){
115211 rc = fts3SegReaderCursorAppend(pCsr, pSeg);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115212 }
115213 }
115214
115215 if( iLevel!=FTS3_SEGCURSOR_PENDING ){
115216 if( rc==SQLITE_OK ){
115217 rc = sqlite3Fts3AllSegdirs(p, iIndex, iLevel, &pStmt);
115218 }
115219
115220 while( rc==SQLITE_OK && SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
115221 Fts3SegReader *pSeg = 0;
115222
115223 /* Read the values returned by the SELECT into local variables. */
115224 sqlite3_int64 iStartBlock = sqlite3_column_int64(pStmt, 1);
115225 sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2);
115226 sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3);
115227 int nRoot = sqlite3_column_bytes(pStmt, 4);
115228 char const *zRoot = sqlite3_column_blob(pStmt, 4);
115229
 
 
 
 
 
 
 
 
 
 
 
 
115230 /* If zTerm is not NULL, and this segment is not stored entirely on its
115231 ** root node, the range of leaves scanned can be reduced. Do this. */
115232 if( iStartBlock && zTerm ){
115233 sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0);
115234 rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi);
115235 if( rc!=SQLITE_OK ) goto finished;
115236 if( isPrefix==0 && isScan==0 ) iLeavesEndBlock = iStartBlock;
115237 }
115238
115239 rc = sqlite3Fts3SegReaderNew(pCsr->nSegment+1,
115240 iStartBlock, iLeavesEndBlock, iEndBlock, zRoot, nRoot, &pSeg
115241 );
115242 if( rc!=SQLITE_OK ) goto finished;
115243 rc = fts3SegReaderCursorAppend(pCsr, pSeg);
 
115244 }
115245 }
115246
115247 finished:
115248 rc2 = sqlite3_reset(pStmt);
115249 if( rc==SQLITE_DONE ) rc = rc2;
 
115250
115251 return rc;
115252 }
115253
115254 /*
115255 ** Set up a cursor object for iterating through a full-text index or a
115256 ** single level therein.
115257 */
115258 SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor(
115259 Fts3Table *p, /* FTS3 table handle */
115260 int iIndex, /* Index to search (from 0 to p->nIndex-1) */
115261 int iLevel, /* Level of segments to scan */
115262 const char *zTerm, /* Term to query for */
115263 int nTerm, /* Size of zTerm in bytes */
115264 int isPrefix, /* True for a prefix search */
115265 int isScan, /* True to scan from zTerm to EOF */
115266 Fts3MultiSegReader *pCsr /* Cursor object to populate */
115267 ){
115268 assert( iIndex>=0 && iIndex<p->nIndex );
115269 assert( iLevel==FTS3_SEGCURSOR_ALL
115270 || iLevel==FTS3_SEGCURSOR_PENDING
115271 || iLevel>=0
115272 );
115273 assert( iLevel<FTS3_SEGDIR_MAXLEVEL );
115274 assert( FTS3_SEGCURSOR_ALL<0 && FTS3_SEGCURSOR_PENDING<0 );
115275 assert( isPrefix==0 || isScan==0 );
115276
115277 /* "isScan" is only set to true by the ft4aux module, an ordinary
115278 ** full-text tables. */
115279 assert( isScan==0 || p->aIndex==0 );
115280
115281 memset(pCsr, 0, sizeof(Fts3MultiSegReader));
115282
115283 return fts3SegReaderCursor(
115284 p, iIndex, iLevel, zTerm, nTerm, isPrefix, isScan, pCsr
115285 );
115286 }
115287
115288 /*
115289 ** In addition to its current configuration, have the Fts3MultiSegReader
115290 ** passed as the 4th argument also scan the doclist for term zTerm/nTerm.
115291 **
115292 ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
115293 */
115294 static int fts3SegReaderCursorAddZero(
115295 Fts3Table *p, /* FTS virtual table handle */
115296 const char *zTerm, /* Term to scan doclist of */
115297 int nTerm, /* Number of bytes in zTerm */
115298 Fts3MultiSegReader *pCsr /* Fts3MultiSegReader to modify */
115299 ){
115300 return fts3SegReaderCursor(p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0,pCsr);
115301 }
115302
115303 /*
115304 ** Open an Fts3MultiSegReader to scan the doclist for term zTerm/nTerm. Or,
115305 ** if isPrefix is true, to scan the doclist for all terms for which
115306 ** zTerm/nTerm is a prefix. If successful, return SQLITE_OK and write
115307 ** a pointer to the new Fts3MultiSegReader to *ppSegcsr. Otherwise, return
115308 ** an SQLite error code.
115309 **
115310 ** It is the responsibility of the caller to free this object by eventually
115311 ** passing it to fts3SegReaderCursorFree()
115312 **
115313 ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
115314 ** Output parameter *ppSegcsr is set to 0 if an error occurs.
115315 */
115316 static int fts3TermSegReaderCursor(
115317 Fts3Cursor *pCsr, /* Virtual table cursor handle */
115318 const char *zTerm, /* Term to query for */
115319 int nTerm, /* Size of zTerm in bytes */
115320 int isPrefix, /* True for a prefix search */
115321 Fts3MultiSegReader **ppSegcsr /* OUT: Allocated seg-reader cursor */
115322 ){
115323 Fts3MultiSegReader *pSegcsr; /* Object to allocate and return */
115324 int rc = SQLITE_NOMEM; /* Return code */
115325
115326 pSegcsr = sqlite3_malloc(sizeof(Fts3MultiSegReader));
115327 if( pSegcsr ){
115328 int i;
115329 int bFound = 0; /* True once an index has been found */
115330 Fts3Table *p = (Fts3Table *)pCsr->base.pVtab;
115331
115332 if( isPrefix ){
115333 for(i=1; bFound==0 && i<p->nIndex; i++){
115334 if( p->aIndex[i].nPrefix==nTerm ){
115335 bFound = 1;
115336 rc = sqlite3Fts3SegReaderCursor(
115337 p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0, pSegcsr);
115338 pSegcsr->bLookup = 1;
115339 }
115340 }
115341
115342 for(i=1; bFound==0 && i<p->nIndex; i++){
115343 if( p->aIndex[i].nPrefix==nTerm+1 ){
115344 bFound = 1;
115345 rc = sqlite3Fts3SegReaderCursor(
115346 p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 1, 0, pSegcsr
115347 );
115348 if( rc==SQLITE_OK ){
115349 rc = fts3SegReaderCursorAddZero(p, zTerm, nTerm, pSegcsr);
115350 }
115351 }
115352 }
115353 }
115354
115355 if( bFound==0 ){
115356 rc = sqlite3Fts3SegReaderCursor(
115357 p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr
115358 );
115359 pSegcsr->bLookup = !isPrefix;
115360 }
115361 }
115362
115363 *ppSegcsr = pSegcsr;
115364 return rc;
115365 }
115366
115367 /*
115368 ** Free an Fts3MultiSegReader allocated by fts3TermSegReaderCursor().
115369 */
115370 static void fts3SegReaderCursorFree(Fts3MultiSegReader *pSegcsr){
115371 sqlite3Fts3SegReaderFinish(pSegcsr);
115372 sqlite3_free(pSegcsr);
115373 }
115374
115375 /*
115376 ** This function retreives the doclist for the specified term (or term
115377 ** prefix) from the database.
 
 
 
 
 
 
 
115378 */
115379 static int fts3TermSelect(
115380 Fts3Table *p, /* Virtual table handle */
115381 Fts3PhraseToken *pTok, /* Token to query for */
115382 int iColumn, /* Column to query (or -ve for all columns) */
 
115383 int *pnOut, /* OUT: Size of buffer at *ppOut */
115384 char **ppOut /* OUT: Malloced result buffer */
115385 ){
115386 int rc; /* Return code */
115387 Fts3MultiSegReader *pSegcsr; /* Seg-reader cursor for this term */
115388 TermSelect tsc; /* Object for pair-wise doclist merging */
115389 Fts3SegFilter filter; /* Segment term filter configuration */
115390
115391 pSegcsr = pTok->pSegcsr;
115392 memset(&tsc, 0, sizeof(TermSelect));
 
115393
115394 filter.flags = FTS3_SEGMENT_IGNORE_EMPTY | FTS3_SEGMENT_REQUIRE_POS
115395 | (pTok->isPrefix ? FTS3_SEGMENT_PREFIX : 0)
 
115396 | (iColumn<p->nColumn ? FTS3_SEGMENT_COLUMN_FILTER : 0);
115397 filter.iCol = iColumn;
115398 filter.zTerm = pTok->z;
115399 filter.nTerm = pTok->n;
115400
115401 rc = sqlite3Fts3SegReaderStart(p, pSegcsr, &filter);
115402 while( SQLITE_OK==rc
115403 && SQLITE_ROW==(rc = sqlite3Fts3SegReaderStep(p, pSegcsr))
115404 ){
115405 rc = fts3TermSelectMerge(p, &tsc, pSegcsr->aDoclist, pSegcsr->nDoclist);
 
 
115406 }
115407
115408 if( rc==SQLITE_OK ){
115409 rc = fts3TermSelectFinishMerge(p, &tsc);
115410 }
115411 if( rc==SQLITE_OK ){
115412 *ppOut = tsc.aaOutput[0];
115413 *pnOut = tsc.anOutput[0];
115414 }else{
@@ -114117,686 +115430,23 @@
115430 ** If the isPoslist argument is true, then it is assumed that the doclist
115431 ** contains a position-list following each docid. Otherwise, it is assumed
115432 ** that the doclist is simply a list of docids stored as delta encoded
115433 ** varints.
115434 */
115435 static int fts3DoclistCountDocids(char *aList, int nList){
115436 int nDoc = 0; /* Return value */
115437 if( aList ){
115438 char *aEnd = &aList[nList]; /* Pointer to one byte after EOF */
115439 char *p = aList; /* Cursor */
115440 while( p<aEnd ){
115441 nDoc++;
115442 while( (*p++)&0x80 ); /* Skip docid varint */
115443 fts3PoslistCopy(0, &p); /* Skip over position list */
115444 }
115445 }
115446
115447 return nDoc;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115448 }
115449
115450 /*
115451 ** Advance the cursor to the next row in the %_content table that
115452 ** matches the search criteria. For a MATCH search, this will be
@@ -114807,43 +115457,24 @@
115457 ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned
115458 ** even if we reach end-of-file. The fts3EofMethod() will be called
115459 ** subsequently to determine whether or not an EOF was hit.
115460 */
115461 static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){
115462 int rc;
 
115463 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
115464 if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){
115465 if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){
115466 pCsr->isEof = 1;
115467 rc = sqlite3_reset(pCsr->pStmt);
115468 }else{
 
 
 
 
115469 pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0);
115470 rc = SQLITE_OK;
115471 }
115472 }else{
115473 rc = fts3EvalNext((Fts3Cursor *)pCursor);
115474 }
115475 assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115476 return rc;
115477 }
115478
115479 /*
115480 ** This is the xFilter interface for the virtual table. See
@@ -114866,15 +115497,11 @@
115497 int idxNum, /* Strategy index */
115498 const char *idxStr, /* Unused */
115499 int nVal, /* Number of elements in apVal */
115500 sqlite3_value **apVal /* Arguments for the indexing scheme */
115501 ){
115502 int rc;
 
 
 
 
115503 char *zSql; /* SQL statement used to access %_content */
115504 Fts3Table *p = (Fts3Table *)pCursor->pVtab;
115505 Fts3Cursor *pCsr = (Fts3Cursor *)pCursor;
115506
115507 UNUSED_PARAMETER(idxStr);
@@ -114889,10 +115516,17 @@
115516 sqlite3_finalize(pCsr->pStmt);
115517 sqlite3_free(pCsr->aDoclist);
115518 sqlite3Fts3ExprFree(pCsr->pExpr);
115519 memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor));
115520
115521 if( idxStr ){
115522 pCsr->bDesc = (idxStr[0]=='D');
115523 }else{
115524 pCsr->bDesc = p->bDescIdx;
115525 }
115526 pCsr->eSearch = (i16)idxNum;
115527
115528 if( idxNum!=FTS3_DOCID_SEARCH && idxNum!=FTS3_FULLSCAN_SEARCH ){
115529 int iCol = idxNum-FTS3_FULLTEXT_SEARCH;
115530 const char *zQuery = (const char *)sqlite3_value_text(apVal[0]);
115531
115532 if( zQuery==0 && sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
@@ -114902,20 +115536,21 @@
115536 rc = sqlite3Fts3ExprParse(p->pTokenizer, p->azColumn, p->nColumn,
115537 iCol, zQuery, -1, &pCsr->pExpr
115538 );
115539 if( rc!=SQLITE_OK ){
115540 if( rc==SQLITE_ERROR ){
115541 static const char *zErr = "malformed MATCH expression: [%s]";
115542 p->base.zErrMsg = sqlite3_mprintf(zErr, zQuery);
115543 }
115544 return rc;
115545 }
115546
115547 rc = sqlite3Fts3ReadLock(p);
115548 if( rc!=SQLITE_OK ) return rc;
115549
115550 rc = fts3EvalStart(pCsr);
115551
115552 sqlite3Fts3SegmentsClose(p);
115553 if( rc!=SQLITE_OK ) return rc;
115554 pCsr->pNextId = pCsr->aDoclist;
115555 pCsr->iPrevId = 0;
115556 }
@@ -114923,41 +115558,28 @@
115558 /* Compile a SELECT statement for this cursor. For a full-table-scan, the
115559 ** statement loops through all rows of the %_content table. For a
115560 ** full-text query or docid lookup, the statement retrieves a single
115561 ** row by docid.
115562 */
115563 if( idxNum==FTS3_FULLSCAN_SEARCH ){
115564 const char *zSort = (pCsr->bDesc ? "DESC" : "ASC");
115565 const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x ORDER BY docid %s";
115566 zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName, zSort);
 
 
115567 }else{
115568 const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?";
115569 zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName);
115570 }
115571 if( !zSql ) return SQLITE_NOMEM;
115572 rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0);
115573 sqlite3_free(zSql);
115574 if( rc!=SQLITE_OK ) return rc;
115575
115576 if( idxNum==FTS3_DOCID_SEARCH ){
115577 rc = sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]);
115578 if( rc!=SQLITE_OK ) return rc;
115579 }
115580
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115581 return fts3NextMethod(pCursor);
115582 }
115583
115584 /*
115585 ** This is the xEof method of the virtual table. SQLite calls this
@@ -114973,20 +115595,11 @@
115595 ** exposes %_content.docid as the rowid for the virtual table. The
115596 ** rowid should be written to *pRowid.
115597 */
115598 static int fts3RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){
115599 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor;
115600 *pRowid = pCsr->iPrevId;
 
 
 
 
 
 
 
 
 
115601 return SQLITE_OK;
115602 }
115603
115604 /*
115605 ** This is the xColumn method, called by SQLite to request a value from
@@ -114995,11 +115608,11 @@
115608 static int fts3ColumnMethod(
115609 sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */
115610 sqlite3_context *pContext, /* Context for sqlite3_result_xxx() calls */
115611 int iCol /* Index of column to read value from */
115612 ){
115613 int rc = SQLITE_OK; /* Return Code */
115614 Fts3Cursor *pCsr = (Fts3Cursor *) pCursor;
115615 Fts3Table *p = (Fts3Table *)pCursor->pVtab;
115616
115617 /* The column value supplied by SQLite must be in range. */
115618 assert( iCol>=0 && iCol<=p->nColumn+1 );
@@ -115006,25 +115619,24 @@
115619
115620 if( iCol==p->nColumn+1 ){
115621 /* This call is a request for the "docid" column. Since "docid" is an
115622 ** alias for "rowid", use the xRowid() method to obtain the value.
115623 */
115624 sqlite3_result_int64(pContext, pCsr->iPrevId);
 
 
115625 }else if( iCol==p->nColumn ){
115626 /* The extra column whose name is the same as the table.
115627 ** Return a blob which is a pointer to the cursor.
115628 */
115629 sqlite3_result_blob(pContext, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT);
 
115630 }else{
115631 rc = fts3CursorSeek(0, pCsr);
115632 if( rc==SQLITE_OK ){
115633 sqlite3_result_value(pContext, sqlite3_column_value(pCsr->pStmt, iCol+1));
115634 }
115635 }
115636
115637 assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 );
115638 return rc;
115639 }
115640
115641 /*
115642 ** This function is the implementation of the xUpdate callback used by
@@ -115052,12 +115664,13 @@
115664
115665 /*
115666 ** Implementation of xBegin() method. This is a no-op.
115667 */
115668 static int fts3BeginMethod(sqlite3_vtab *pVtab){
 
115669 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115670 UNUSED_PARAMETER(pVtab);
115671 assert( p->pSegments==0 );
115672 assert( p->nPendingData==0 );
115673 assert( p->inTransaction!=1 );
115674 TESTONLY( p->inTransaction = 1 );
115675 TESTONLY( p->mxSavepoint = -1; );
115676 return SQLITE_OK;
@@ -115067,14 +115680,15 @@
115680 ** Implementation of xCommit() method. This is a no-op. The contents of
115681 ** the pending-terms hash-table have already been flushed into the database
115682 ** by fts3SyncMethod().
115683 */
115684 static int fts3CommitMethod(sqlite3_vtab *pVtab){
 
115685 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115686 UNUSED_PARAMETER(pVtab);
115687 assert( p->nPendingData==0 );
115688 assert( p->inTransaction!=0 );
115689 assert( p->pSegments==0 );
115690 TESTONLY( p->inTransaction = 0 );
115691 TESTONLY( p->mxSavepoint = -1; );
115692 return SQLITE_OK;
115693 }
115694
@@ -115088,132 +115702,30 @@
115702 assert( p->inTransaction!=0 );
115703 TESTONLY( p->inTransaction = 0 );
115704 TESTONLY( p->mxSavepoint = -1; );
115705 return SQLITE_OK;
115706 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115707
115708 /*
115709 ** When called, *ppPoslist must point to the byte immediately following the
115710 ** end of a position-list. i.e. ( (*ppPoslist)[-1]==POS_END ). This function
115711 ** moves *ppPoslist so that it instead points to the first byte of the
115712 ** same position list.
115713 */
115714 static void fts3ReversePoslist(char *pStart, char **ppPoslist){
115715 char *p = &(*ppPoslist)[-2];
115716 char c;
115717
115718 while( p>pStart && (c=*p--)==0 );
115719 while( p>pStart && (*p & 0x80) | c ){
115720 c = *p--;
115721 }
115722 if( p>pStart ){ p = &p[2]; }
115723 while( *p++&0x80 );
115724 *ppPoslist = p;
115725 }
115726
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
115727 /*
115728 ** Helper function used by the implementation of the overloaded snippet(),
115729 ** offsets() and optimize() SQL functions.
115730 **
115731 ** If the value passed as the third argument is a blob of size
@@ -115440,27 +115952,43 @@
115952 p->zDb, p->zName, zName
115953 );
115954 return rc;
115955 }
115956
115957 /*
115958 ** The xSavepoint() method.
115959 **
115960 ** Flush the contents of the pending-terms table to disk.
115961 */
115962 static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){
 
115963 UNUSED_PARAMETER(iSavepoint);
115964 assert( ((Fts3Table *)pVtab)->inTransaction );
115965 assert( ((Fts3Table *)pVtab)->mxSavepoint < iSavepoint );
115966 TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint );
115967 return fts3SyncMethod(pVtab);
115968 }
115969
115970 /*
115971 ** The xRelease() method.
115972 **
115973 ** This is a no-op.
115974 */
115975 static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){
115976 TESTONLY( Fts3Table *p = (Fts3Table*)pVtab );
115977 UNUSED_PARAMETER(iSavepoint);
115978 UNUSED_PARAMETER(pVtab);
115979 assert( p->inTransaction );
115980 assert( p->mxSavepoint >= iSavepoint );
115981 TESTONLY( p->mxSavepoint = iSavepoint-1 );
115982 return SQLITE_OK;
115983 }
115984
115985 /*
115986 ** The xRollbackTo() method.
115987 **
115988 ** Discard the contents of the pending terms table.
115989 */
115990 static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){
115991 Fts3Table *p = (Fts3Table*)pVtab;
115992 UNUSED_PARAMETER(iSavepoint);
115993 assert( p->inTransaction );
115994 assert( p->mxSavepoint >= iSavepoint );
@@ -115605,12 +116133,1553 @@
116133 sqlite3Fts3HashClear(pHash);
116134 sqlite3_free(pHash);
116135 }
116136 return rc;
116137 }
116138
116139 /*
116140 ** Allocate an Fts3MultiSegReader for each token in the expression headed
116141 ** by pExpr.
116142 **
116143 ** An Fts3SegReader object is a cursor that can seek or scan a range of
116144 ** entries within a single segment b-tree. An Fts3MultiSegReader uses multiple
116145 ** Fts3SegReader objects internally to provide an interface to seek or scan
116146 ** within the union of all segments of a b-tree. Hence the name.
116147 **
116148 ** If the allocated Fts3MultiSegReader just seeks to a single entry in a
116149 ** segment b-tree (if the term is not a prefix or it is a prefix for which
116150 ** there exists prefix b-tree of the right length) then it may be traversed
116151 ** and merged incrementally. Otherwise, it has to be merged into an in-memory
116152 ** doclist and then traversed.
116153 */
116154 static void fts3EvalAllocateReaders(
116155 Fts3Cursor *pCsr, /* FTS cursor handle */
116156 Fts3Expr *pExpr, /* Allocate readers for this expression */
116157 int *pnToken, /* OUT: Total number of tokens in phrase. */
116158 int *pnOr, /* OUT: Total number of OR nodes in expr. */
116159 int *pRc /* IN/OUT: Error code */
116160 ){
116161 if( pExpr && SQLITE_OK==*pRc ){
116162 if( pExpr->eType==FTSQUERY_PHRASE ){
116163 int i;
116164 int nToken = pExpr->pPhrase->nToken;
116165 *pnToken += nToken;
116166 for(i=0; i<nToken; i++){
116167 Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i];
116168 int rc = fts3TermSegReaderCursor(pCsr,
116169 pToken->z, pToken->n, pToken->isPrefix, &pToken->pSegcsr
116170 );
116171 if( rc!=SQLITE_OK ){
116172 *pRc = rc;
116173 return;
116174 }
116175 }
116176 assert( pExpr->pPhrase->iDoclistToken==0 );
116177 pExpr->pPhrase->iDoclistToken = -1;
116178 }else{
116179 *pnOr += (pExpr->eType==FTSQUERY_OR);
116180 fts3EvalAllocateReaders(pCsr, pExpr->pLeft, pnToken, pnOr, pRc);
116181 fts3EvalAllocateReaders(pCsr, pExpr->pRight, pnToken, pnOr, pRc);
116182 }
116183 }
116184 }
116185
116186 /*
116187 ** Arguments pList/nList contain the doclist for token iToken of phrase p.
116188 ** It is merged into the main doclist stored in p->doclist.aAll/nAll.
116189 **
116190 ** This function assumes that pList points to a buffer allocated using
116191 ** sqlite3_malloc(). This function takes responsibility for eventually
116192 ** freeing the buffer.
116193 */
116194 static void fts3EvalPhraseMergeToken(
116195 Fts3Table *pTab, /* FTS Table pointer */
116196 Fts3Phrase *p, /* Phrase to merge pList/nList into */
116197 int iToken, /* Token pList/nList corresponds to */
116198 char *pList, /* Pointer to doclist */
116199 int nList /* Number of bytes in pList */
116200 ){
116201 assert( iToken!=p->iDoclistToken );
116202
116203 if( pList==0 ){
116204 sqlite3_free(p->doclist.aAll);
116205 p->doclist.aAll = 0;
116206 p->doclist.nAll = 0;
116207 }
116208
116209 else if( p->iDoclistToken<0 ){
116210 p->doclist.aAll = pList;
116211 p->doclist.nAll = nList;
116212 }
116213
116214 else if( p->doclist.aAll==0 ){
116215 sqlite3_free(pList);
116216 }
116217
116218 else {
116219 char *pLeft;
116220 char *pRight;
116221 int nLeft;
116222 int nRight;
116223 int nDiff;
116224
116225 if( p->iDoclistToken<iToken ){
116226 pLeft = p->doclist.aAll;
116227 nLeft = p->doclist.nAll;
116228 pRight = pList;
116229 nRight = nList;
116230 nDiff = iToken - p->iDoclistToken;
116231 }else{
116232 pRight = p->doclist.aAll;
116233 nRight = p->doclist.nAll;
116234 pLeft = pList;
116235 nLeft = nList;
116236 nDiff = p->iDoclistToken - iToken;
116237 }
116238
116239 fts3DoclistPhraseMerge(pTab->bDescIdx, nDiff, pLeft, nLeft, pRight,&nRight);
116240 sqlite3_free(pLeft);
116241 p->doclist.aAll = pRight;
116242 p->doclist.nAll = nRight;
116243 }
116244
116245 if( iToken>p->iDoclistToken ) p->iDoclistToken = iToken;
116246 }
116247
116248 /*
116249 ** Load the doclist for phrase p into p->doclist.aAll/nAll. The loaded doclist
116250 ** does not take deferred tokens into account.
116251 **
116252 ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
116253 */
116254 static int fts3EvalPhraseLoad(
116255 Fts3Cursor *pCsr, /* FTS Cursor handle */
116256 Fts3Phrase *p /* Phrase object */
116257 ){
116258 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116259 int iToken;
116260 int rc = SQLITE_OK;
116261
116262 for(iToken=0; rc==SQLITE_OK && iToken<p->nToken; iToken++){
116263 Fts3PhraseToken *pToken = &p->aToken[iToken];
116264 assert( pToken->pDeferred==0 || pToken->pSegcsr==0 );
116265
116266 if( pToken->pSegcsr ){
116267 int nThis = 0;
116268 char *pThis = 0;
116269 rc = fts3TermSelect(pTab, pToken, p->iColumn, &nThis, &pThis);
116270 if( rc==SQLITE_OK ){
116271 fts3EvalPhraseMergeToken(pTab, p, iToken, pThis, nThis);
116272 }
116273 }
116274 assert( pToken->pSegcsr==0 );
116275 }
116276
116277 return rc;
116278 }
116279
116280 /*
116281 ** This function is called on each phrase after the position lists for
116282 ** any deferred tokens have been loaded into memory. It updates the phrases
116283 ** current position list to include only those positions that are really
116284 ** instances of the phrase (after considering deferred tokens). If this
116285 ** means that the phrase does not appear in the current row, doclist.pList
116286 ** and doclist.nList are both zeroed.
116287 **
116288 ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
116289 */
116290 static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){
116291 int iToken; /* Used to iterate through phrase tokens */
116292 int rc = SQLITE_OK; /* Return code */
116293 char *aPoslist = 0; /* Position list for deferred tokens */
116294 int nPoslist = 0; /* Number of bytes in aPoslist */
116295 int iPrev = -1; /* Token number of previous deferred token */
116296
116297 assert( pPhrase->doclist.bFreeList==0 );
116298
116299 for(iToken=0; rc==SQLITE_OK && iToken<pPhrase->nToken; iToken++){
116300 Fts3PhraseToken *pToken = &pPhrase->aToken[iToken];
116301 Fts3DeferredToken *pDeferred = pToken->pDeferred;
116302
116303 if( pDeferred ){
116304 char *pList;
116305 int nList;
116306 rc = sqlite3Fts3DeferredTokenList(pDeferred, &pList, &nList);
116307 if( rc!=SQLITE_OK ) return rc;
116308
116309 if( pList==0 ){
116310 sqlite3_free(aPoslist);
116311 pPhrase->doclist.pList = 0;
116312 pPhrase->doclist.nList = 0;
116313 return SQLITE_OK;
116314
116315 }else if( aPoslist==0 ){
116316 aPoslist = pList;
116317 nPoslist = nList;
116318
116319 }else{
116320 char *aOut = pList;
116321 char *p1 = aPoslist;
116322 char *p2 = aOut;
116323
116324 assert( iPrev>=0 );
116325 fts3PoslistPhraseMerge(&aOut, iToken-iPrev, 0, 1, &p1, &p2);
116326 sqlite3_free(aPoslist);
116327 aPoslist = pList;
116328 nPoslist = aOut - aPoslist;
116329 if( nPoslist==0 ){
116330 sqlite3_free(aPoslist);
116331 pPhrase->doclist.pList = 0;
116332 pPhrase->doclist.nList = 0;
116333 return SQLITE_OK;
116334 }
116335 }
116336 iPrev = iToken;
116337 }
116338 }
116339
116340 if( iPrev>=0 ){
116341 int nMaxUndeferred = pPhrase->iDoclistToken;
116342 if( nMaxUndeferred<0 ){
116343 pPhrase->doclist.pList = aPoslist;
116344 pPhrase->doclist.nList = nPoslist;
116345 pPhrase->doclist.iDocid = pCsr->iPrevId;
116346 pPhrase->doclist.bFreeList = 1;
116347 }else{
116348 int nDistance;
116349 char *p1;
116350 char *p2;
116351 char *aOut;
116352
116353 if( nMaxUndeferred>iPrev ){
116354 p1 = aPoslist;
116355 p2 = pPhrase->doclist.pList;
116356 nDistance = nMaxUndeferred - iPrev;
116357 }else{
116358 p1 = pPhrase->doclist.pList;
116359 p2 = aPoslist;
116360 nDistance = iPrev - nMaxUndeferred;
116361 }
116362
116363 aOut = (char *)sqlite3_malloc(nPoslist+8);
116364 if( !aOut ){
116365 sqlite3_free(aPoslist);
116366 return SQLITE_NOMEM;
116367 }
116368
116369 pPhrase->doclist.pList = aOut;
116370 if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){
116371 pPhrase->doclist.bFreeList = 1;
116372 pPhrase->doclist.nList = (aOut - pPhrase->doclist.pList);
116373 }else{
116374 sqlite3_free(aOut);
116375 pPhrase->doclist.pList = 0;
116376 pPhrase->doclist.nList = 0;
116377 }
116378 sqlite3_free(aPoslist);
116379 }
116380 }
116381
116382 return SQLITE_OK;
116383 }
116384
116385 /*
116386 ** This function is called for each Fts3Phrase in a full-text query
116387 ** expression to initialize the mechanism for returning rows. Once this
116388 ** function has been called successfully on an Fts3Phrase, it may be
116389 ** used with fts3EvalPhraseNext() to iterate through the matching docids.
116390 **
116391 ** If parameter bOptOk is true, then the phrase may (or may not) use the
116392 ** incremental loading strategy. Otherwise, the entire doclist is loaded into
116393 ** memory within this call.
116394 **
116395 ** SQLITE_OK is returned if no error occurs, otherwise an SQLite error code.
116396 */
116397 static int fts3EvalPhraseStart(Fts3Cursor *pCsr, int bOptOk, Fts3Phrase *p){
116398 int rc; /* Error code */
116399 Fts3PhraseToken *pFirst = &p->aToken[0];
116400 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116401
116402 if( pCsr->bDesc==pTab->bDescIdx
116403 && bOptOk==1
116404 && p->nToken==1
116405 && pFirst->pSegcsr
116406 && pFirst->pSegcsr->bLookup
116407 ){
116408 /* Use the incremental approach. */
116409 int iCol = (p->iColumn >= pTab->nColumn ? -1 : p->iColumn);
116410 rc = sqlite3Fts3MsrIncrStart(
116411 pTab, pFirst->pSegcsr, iCol, pFirst->z, pFirst->n);
116412 p->bIncr = 1;
116413
116414 }else{
116415 /* Load the full doclist for the phrase into memory. */
116416 rc = fts3EvalPhraseLoad(pCsr, p);
116417 p->bIncr = 0;
116418 }
116419
116420 assert( rc!=SQLITE_OK || p->nToken<1 || p->aToken[0].pSegcsr==0 || p->bIncr );
116421 return rc;
116422 }
116423
116424 /*
116425 ** This function is used to iterate backwards (from the end to start)
116426 ** through doclists. It is used by this module to iterate through phrase
116427 ** doclists in reverse and by the fts3_write.c module to iterate through
116428 ** pending-terms lists when writing to databases with "order=desc".
116429 **
116430 ** The doclist may be sorted in ascending (parameter bDescIdx==0) or
116431 ** descending (parameter bDescIdx==1) order of docid. Regardless, this
116432 ** function iterates from the end of the doclist to the beginning.
116433 */
116434 SQLITE_PRIVATE void sqlite3Fts3DoclistPrev(
116435 int bDescIdx, /* True if the doclist is desc */
116436 char *aDoclist, /* Pointer to entire doclist */
116437 int nDoclist, /* Length of aDoclist in bytes */
116438 char **ppIter, /* IN/OUT: Iterator pointer */
116439 sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */
116440 int *pnList, /* IN/OUT: List length pointer */
116441 u8 *pbEof /* OUT: End-of-file flag */
116442 ){
116443 char *p = *ppIter;
116444
116445 assert( nDoclist>0 );
116446 assert( *pbEof==0 );
116447 assert( p || *piDocid==0 );
116448 assert( !p || (p>aDoclist && p<&aDoclist[nDoclist]) );
116449
116450 if( p==0 ){
116451 sqlite3_int64 iDocid = 0;
116452 char *pNext = 0;
116453 char *pDocid = aDoclist;
116454 char *pEnd = &aDoclist[nDoclist];
116455 int iMul = 1;
116456
116457 while( pDocid<pEnd ){
116458 sqlite3_int64 iDelta;
116459 pDocid += sqlite3Fts3GetVarint(pDocid, &iDelta);
116460 iDocid += (iMul * iDelta);
116461 pNext = pDocid;
116462 fts3PoslistCopy(0, &pDocid);
116463 while( pDocid<pEnd && *pDocid==0 ) pDocid++;
116464 iMul = (bDescIdx ? -1 : 1);
116465 }
116466
116467 *pnList = pEnd - pNext;
116468 *ppIter = pNext;
116469 *piDocid = iDocid;
116470 }else{
116471 int iMul = (bDescIdx ? -1 : 1);
116472 sqlite3_int64 iDelta;
116473 fts3GetReverseVarint(&p, aDoclist, &iDelta);
116474 *piDocid -= (iMul * iDelta);
116475
116476 if( p==aDoclist ){
116477 *pbEof = 1;
116478 }else{
116479 char *pSave = p;
116480 fts3ReversePoslist(aDoclist, &p);
116481 *pnList = (pSave - p);
116482 }
116483 *ppIter = p;
116484 }
116485 }
116486
116487 /*
116488 ** Attempt to move the phrase iterator to point to the next matching docid.
116489 ** If an error occurs, return an SQLite error code. Otherwise, return
116490 ** SQLITE_OK.
116491 **
116492 ** If there is no "next" entry and no error occurs, then *pbEof is set to
116493 ** 1 before returning. Otherwise, if no error occurs and the iterator is
116494 ** successfully advanced, *pbEof is set to 0.
116495 */
116496 static int fts3EvalPhraseNext(
116497 Fts3Cursor *pCsr, /* FTS Cursor handle */
116498 Fts3Phrase *p, /* Phrase object to advance to next docid */
116499 u8 *pbEof /* OUT: Set to 1 if EOF */
116500 ){
116501 int rc = SQLITE_OK;
116502 Fts3Doclist *pDL = &p->doclist;
116503 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116504
116505 if( p->bIncr ){
116506 assert( p->nToken==1 );
116507 assert( pDL->pNextDocid==0 );
116508 rc = sqlite3Fts3MsrIncrNext(pTab, p->aToken[0].pSegcsr,
116509 &pDL->iDocid, &pDL->pList, &pDL->nList
116510 );
116511 if( rc==SQLITE_OK && !pDL->pList ){
116512 *pbEof = 1;
116513 }
116514 }else if( pCsr->bDesc!=pTab->bDescIdx && pDL->nAll ){
116515 sqlite3Fts3DoclistPrev(pTab->bDescIdx, pDL->aAll, pDL->nAll,
116516 &pDL->pNextDocid, &pDL->iDocid, &pDL->nList, pbEof
116517 );
116518 pDL->pList = pDL->pNextDocid;
116519 }else{
116520 char *pIter; /* Used to iterate through aAll */
116521 char *pEnd = &pDL->aAll[pDL->nAll]; /* 1 byte past end of aAll */
116522 if( pDL->pNextDocid ){
116523 pIter = pDL->pNextDocid;
116524 }else{
116525 pIter = pDL->aAll;
116526 }
116527
116528 if( pIter>=pEnd ){
116529 /* We have already reached the end of this doclist. EOF. */
116530 *pbEof = 1;
116531 }else{
116532 sqlite3_int64 iDelta;
116533 pIter += sqlite3Fts3GetVarint(pIter, &iDelta);
116534 if( pTab->bDescIdx==0 || pDL->pNextDocid==0 ){
116535 pDL->iDocid += iDelta;
116536 }else{
116537 pDL->iDocid -= iDelta;
116538 }
116539 pDL->pList = pIter;
116540 fts3PoslistCopy(0, &pIter);
116541 pDL->nList = (pIter - pDL->pList);
116542
116543 /* pIter now points just past the 0x00 that terminates the position-
116544 ** list for document pDL->iDocid. However, if this position-list was
116545 ** edited in place by fts3EvalNearTrim(), then pIter may not actually
116546 ** point to the start of the next docid value. The following line deals
116547 ** with this case by advancing pIter past the zero-padding added by
116548 ** fts3EvalNearTrim(). */
116549 while( pIter<pEnd && *pIter==0 ) pIter++;
116550
116551 pDL->pNextDocid = pIter;
116552 assert( pIter>=&pDL->aAll[pDL->nAll] || *pIter );
116553 *pbEof = 0;
116554 }
116555 }
116556
116557 return rc;
116558 }
116559
116560 /*
116561 **
116562 ** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
116563 ** Otherwise, fts3EvalPhraseStart() is called on all phrases within the
116564 ** expression. Also the Fts3Expr.bDeferred variable is set to true for any
116565 ** expressions for which all descendent tokens are deferred.
116566 **
116567 ** If parameter bOptOk is zero, then it is guaranteed that the
116568 ** Fts3Phrase.doclist.aAll/nAll variables contain the entire doclist for
116569 ** each phrase in the expression (subject to deferred token processing).
116570 ** Or, if bOptOk is non-zero, then one or more tokens within the expression
116571 ** may be loaded incrementally, meaning doclist.aAll/nAll is not available.
116572 **
116573 ** If an error occurs within this function, *pRc is set to an SQLite error
116574 ** code before returning.
116575 */
116576 static void fts3EvalStartReaders(
116577 Fts3Cursor *pCsr, /* FTS Cursor handle */
116578 Fts3Expr *pExpr, /* Expression to initialize phrases in */
116579 int bOptOk, /* True to enable incremental loading */
116580 int *pRc /* IN/OUT: Error code */
116581 ){
116582 if( pExpr && SQLITE_OK==*pRc ){
116583 if( pExpr->eType==FTSQUERY_PHRASE ){
116584 int i;
116585 int nToken = pExpr->pPhrase->nToken;
116586 for(i=0; i<nToken; i++){
116587 if( pExpr->pPhrase->aToken[i].pDeferred==0 ) break;
116588 }
116589 pExpr->bDeferred = (i==nToken);
116590 *pRc = fts3EvalPhraseStart(pCsr, bOptOk, pExpr->pPhrase);
116591 }else{
116592 fts3EvalStartReaders(pCsr, pExpr->pLeft, bOptOk, pRc);
116593 fts3EvalStartReaders(pCsr, pExpr->pRight, bOptOk, pRc);
116594 pExpr->bDeferred = (pExpr->pLeft->bDeferred && pExpr->pRight->bDeferred);
116595 }
116596 }
116597 }
116598
116599 /*
116600 ** An array of the following structures is assembled as part of the process
116601 ** of selecting tokens to defer before the query starts executing (as part
116602 ** of the xFilter() method). There is one element in the array for each
116603 ** token in the FTS expression.
116604 **
116605 ** Tokens are divided into AND/NEAR clusters. All tokens in a cluster belong
116606 ** to phrases that are connected only by AND and NEAR operators (not OR or
116607 ** NOT). When determining tokens to defer, each AND/NEAR cluster is considered
116608 ** separately. The root of a tokens AND/NEAR cluster is stored in
116609 ** Fts3TokenAndCost.pRoot.
116610 */
116611 typedef struct Fts3TokenAndCost Fts3TokenAndCost;
116612 struct Fts3TokenAndCost {
116613 Fts3Phrase *pPhrase; /* The phrase the token belongs to */
116614 int iToken; /* Position of token in phrase */
116615 Fts3PhraseToken *pToken; /* The token itself */
116616 Fts3Expr *pRoot; /* Root of NEAR/AND cluster */
116617 int nOvfl; /* Number of overflow pages to load doclist */
116618 int iCol; /* The column the token must match */
116619 };
116620
116621 /*
116622 ** This function is used to populate an allocated Fts3TokenAndCost array.
116623 **
116624 ** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
116625 ** Otherwise, if an error occurs during execution, *pRc is set to an
116626 ** SQLite error code.
116627 */
116628 static void fts3EvalTokenCosts(
116629 Fts3Cursor *pCsr, /* FTS Cursor handle */
116630 Fts3Expr *pRoot, /* Root of current AND/NEAR cluster */
116631 Fts3Expr *pExpr, /* Expression to consider */
116632 Fts3TokenAndCost **ppTC, /* Write new entries to *(*ppTC)++ */
116633 Fts3Expr ***ppOr, /* Write new OR root to *(*ppOr)++ */
116634 int *pRc /* IN/OUT: Error code */
116635 ){
116636 if( *pRc==SQLITE_OK && pExpr ){
116637 if( pExpr->eType==FTSQUERY_PHRASE ){
116638 Fts3Phrase *pPhrase = pExpr->pPhrase;
116639 int i;
116640 for(i=0; *pRc==SQLITE_OK && i<pPhrase->nToken; i++){
116641 Fts3TokenAndCost *pTC = (*ppTC)++;
116642 pTC->pPhrase = pPhrase;
116643 pTC->iToken = i;
116644 pTC->pRoot = pRoot;
116645 pTC->pToken = &pPhrase->aToken[i];
116646 pTC->iCol = pPhrase->iColumn;
116647 *pRc = sqlite3Fts3MsrOvfl(pCsr, pTC->pToken->pSegcsr, &pTC->nOvfl);
116648 }
116649 }else if( pExpr->eType!=FTSQUERY_NOT ){
116650 if( pExpr->eType==FTSQUERY_OR ){
116651 pRoot = pExpr->pLeft;
116652 **ppOr = pRoot;
116653 (*ppOr)++;
116654 }
116655 fts3EvalTokenCosts(pCsr, pRoot, pExpr->pLeft, ppTC, ppOr, pRc);
116656 if( pExpr->eType==FTSQUERY_OR ){
116657 pRoot = pExpr->pRight;
116658 **ppOr = pRoot;
116659 (*ppOr)++;
116660 }
116661 fts3EvalTokenCosts(pCsr, pRoot, pExpr->pRight, ppTC, ppOr, pRc);
116662 }
116663 }
116664 }
116665
116666 /*
116667 ** Determine the average document (row) size in pages. If successful,
116668 ** write this value to *pnPage and return SQLITE_OK. Otherwise, return
116669 ** an SQLite error code.
116670 **
116671 ** The average document size in pages is calculated by first calculating
116672 ** determining the average size in bytes, B. If B is less than the amount
116673 ** of data that will fit on a single leaf page of an intkey table in
116674 ** this database, then the average docsize is 1. Otherwise, it is 1 plus
116675 ** the number of overflow pages consumed by a record B bytes in size.
116676 */
116677 static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){
116678 if( pCsr->nRowAvg==0 ){
116679 /* The average document size, which is required to calculate the cost
116680 ** of each doclist, has not yet been determined. Read the required
116681 ** data from the %_stat table to calculate it.
116682 **
116683 ** Entry 0 of the %_stat table is a blob containing (nCol+1) FTS3
116684 ** varints, where nCol is the number of columns in the FTS3 table.
116685 ** The first varint is the number of documents currently stored in
116686 ** the table. The following nCol varints contain the total amount of
116687 ** data stored in all rows of each column of the table, from left
116688 ** to right.
116689 */
116690 int rc;
116691 Fts3Table *p = (Fts3Table*)pCsr->base.pVtab;
116692 sqlite3_stmt *pStmt;
116693 sqlite3_int64 nDoc = 0;
116694 sqlite3_int64 nByte = 0;
116695 const char *pEnd;
116696 const char *a;
116697
116698 rc = sqlite3Fts3SelectDoctotal(p, &pStmt);
116699 if( rc!=SQLITE_OK ) return rc;
116700 a = sqlite3_column_blob(pStmt, 0);
116701 assert( a );
116702
116703 pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
116704 a += sqlite3Fts3GetVarint(a, &nDoc);
116705 while( a<pEnd ){
116706 a += sqlite3Fts3GetVarint(a, &nByte);
116707 }
116708 if( nDoc==0 || nByte==0 ){
116709 sqlite3_reset(pStmt);
116710 return SQLITE_CORRUPT_VTAB;
116711 }
116712
116713 pCsr->nDoc = nDoc;
116714 pCsr->nRowAvg = (int)(((nByte / nDoc) + p->nPgsz) / p->nPgsz);
116715 assert( pCsr->nRowAvg>0 );
116716 rc = sqlite3_reset(pStmt);
116717 if( rc!=SQLITE_OK ) return rc;
116718 }
116719
116720 *pnPage = pCsr->nRowAvg;
116721 return SQLITE_OK;
116722 }
116723
116724 /*
116725 ** This function is called to select the tokens (if any) that will be
116726 ** deferred. The array aTC[] has already been populated when this is
116727 ** called.
116728 **
116729 ** This function is called once for each AND/NEAR cluster in the
116730 ** expression. Each invocation determines which tokens to defer within
116731 ** the cluster with root node pRoot. See comments above the definition
116732 ** of struct Fts3TokenAndCost for more details.
116733 **
116734 ** If no error occurs, SQLITE_OK is returned and sqlite3Fts3DeferToken()
116735 ** called on each token to defer. Otherwise, an SQLite error code is
116736 ** returned.
116737 */
116738 static int fts3EvalSelectDeferred(
116739 Fts3Cursor *pCsr, /* FTS Cursor handle */
116740 Fts3Expr *pRoot, /* Consider tokens with this root node */
116741 Fts3TokenAndCost *aTC, /* Array of expression tokens and costs */
116742 int nTC /* Number of entries in aTC[] */
116743 ){
116744 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116745 int nDocSize = 0; /* Number of pages per doc loaded */
116746 int rc = SQLITE_OK; /* Return code */
116747 int ii; /* Iterator variable for various purposes */
116748 int nOvfl = 0; /* Total overflow pages used by doclists */
116749 int nToken = 0; /* Total number of tokens in cluster */
116750
116751 int nMinEst = 0; /* The minimum count for any phrase so far. */
116752 int nLoad4 = 1; /* (Phrases that will be loaded)^4. */
116753
116754 /* Count the tokens in this AND/NEAR cluster. If none of the doclists
116755 ** associated with the tokens spill onto overflow pages, or if there is
116756 ** only 1 token, exit early. No tokens to defer in this case. */
116757 for(ii=0; ii<nTC; ii++){
116758 if( aTC[ii].pRoot==pRoot ){
116759 nOvfl += aTC[ii].nOvfl;
116760 nToken++;
116761 }
116762 }
116763 if( nOvfl==0 || nToken<2 ) return SQLITE_OK;
116764
116765 /* Obtain the average docsize (in pages). */
116766 rc = fts3EvalAverageDocsize(pCsr, &nDocSize);
116767 assert( rc!=SQLITE_OK || nDocSize>0 );
116768
116769
116770 /* Iterate through all tokens in this AND/NEAR cluster, in ascending order
116771 ** of the number of overflow pages that will be loaded by the pager layer
116772 ** to retrieve the entire doclist for the token from the full-text index.
116773 ** Load the doclists for tokens that are either:
116774 **
116775 ** a. The cheapest token in the entire query (i.e. the one visited by the
116776 ** first iteration of this loop), or
116777 **
116778 ** b. Part of a multi-token phrase.
116779 **
116780 ** After each token doclist is loaded, merge it with the others from the
116781 ** same phrase and count the number of documents that the merged doclist
116782 ** contains. Set variable "nMinEst" to the smallest number of documents in
116783 ** any phrase doclist for which 1 or more token doclists have been loaded.
116784 ** Let nOther be the number of other phrases for which it is certain that
116785 ** one or more tokens will not be deferred.
116786 **
116787 ** Then, for each token, defer it if loading the doclist would result in
116788 ** loading N or more overflow pages into memory, where N is computed as:
116789 **
116790 ** (nMinEst + 4^nOther - 1) / (4^nOther)
116791 */
116792 for(ii=0; ii<nToken && rc==SQLITE_OK; ii++){
116793 int iTC; /* Used to iterate through aTC[] array. */
116794 Fts3TokenAndCost *pTC = 0; /* Set to cheapest remaining token. */
116795
116796 /* Set pTC to point to the cheapest remaining token. */
116797 for(iTC=0; iTC<nTC; iTC++){
116798 if( aTC[iTC].pToken && aTC[iTC].pRoot==pRoot
116799 && (!pTC || aTC[iTC].nOvfl<pTC->nOvfl)
116800 ){
116801 pTC = &aTC[iTC];
116802 }
116803 }
116804 assert( pTC );
116805
116806 if( ii && pTC->nOvfl>=((nMinEst+(nLoad4/4)-1)/(nLoad4/4))*nDocSize ){
116807 /* The number of overflow pages to load for this (and therefore all
116808 ** subsequent) tokens is greater than the estimated number of pages
116809 ** that will be loaded if all subsequent tokens are deferred.
116810 */
116811 Fts3PhraseToken *pToken = pTC->pToken;
116812 rc = sqlite3Fts3DeferToken(pCsr, pToken, pTC->iCol);
116813 fts3SegReaderCursorFree(pToken->pSegcsr);
116814 pToken->pSegcsr = 0;
116815 }else{
116816 nLoad4 = nLoad4*4;
116817 if( ii==0 || pTC->pPhrase->nToken>1 ){
116818 /* Either this is the cheapest token in the entire query, or it is
116819 ** part of a multi-token phrase. Either way, the entire doclist will
116820 ** (eventually) be loaded into memory. It may as well be now. */
116821 Fts3PhraseToken *pToken = pTC->pToken;
116822 int nList = 0;
116823 char *pList = 0;
116824 rc = fts3TermSelect(pTab, pToken, pTC->iCol, &nList, &pList);
116825 assert( rc==SQLITE_OK || pList==0 );
116826 if( rc==SQLITE_OK ){
116827 int nCount;
116828 fts3EvalPhraseMergeToken(pTab, pTC->pPhrase, pTC->iToken,pList,nList);
116829 nCount = fts3DoclistCountDocids(
116830 pTC->pPhrase->doclist.aAll, pTC->pPhrase->doclist.nAll
116831 );
116832 if( ii==0 || nCount<nMinEst ) nMinEst = nCount;
116833 }
116834 }
116835 }
116836 pTC->pToken = 0;
116837 }
116838
116839 return rc;
116840 }
116841
116842 /*
116843 ** This function is called from within the xFilter method. It initializes
116844 ** the full-text query currently stored in pCsr->pExpr. To iterate through
116845 ** the results of a query, the caller does:
116846 **
116847 ** fts3EvalStart(pCsr);
116848 ** while( 1 ){
116849 ** fts3EvalNext(pCsr);
116850 ** if( pCsr->bEof ) break;
116851 ** ... return row pCsr->iPrevId to the caller ...
116852 ** }
116853 */
116854 static int fts3EvalStart(Fts3Cursor *pCsr){
116855 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
116856 int rc = SQLITE_OK;
116857 int nToken = 0;
116858 int nOr = 0;
116859
116860 /* Allocate a MultiSegReader for each token in the expression. */
116861 fts3EvalAllocateReaders(pCsr, pCsr->pExpr, &nToken, &nOr, &rc);
116862
116863 /* Determine which, if any, tokens in the expression should be deferred. */
116864 if( rc==SQLITE_OK && nToken>1 && pTab->bHasStat ){
116865 Fts3TokenAndCost *aTC;
116866 Fts3Expr **apOr;
116867 aTC = (Fts3TokenAndCost *)sqlite3_malloc(
116868 sizeof(Fts3TokenAndCost) * nToken
116869 + sizeof(Fts3Expr *) * nOr * 2
116870 );
116871 apOr = (Fts3Expr **)&aTC[nToken];
116872
116873 if( !aTC ){
116874 rc = SQLITE_NOMEM;
116875 }else{
116876 int ii;
116877 Fts3TokenAndCost *pTC = aTC;
116878 Fts3Expr **ppOr = apOr;
116879
116880 fts3EvalTokenCosts(pCsr, 0, pCsr->pExpr, &pTC, &ppOr, &rc);
116881 nToken = pTC-aTC;
116882 nOr = ppOr-apOr;
116883
116884 if( rc==SQLITE_OK ){
116885 rc = fts3EvalSelectDeferred(pCsr, 0, aTC, nToken);
116886 for(ii=0; rc==SQLITE_OK && ii<nOr; ii++){
116887 rc = fts3EvalSelectDeferred(pCsr, apOr[ii], aTC, nToken);
116888 }
116889 }
116890
116891 sqlite3_free(aTC);
116892 }
116893 }
116894
116895 fts3EvalStartReaders(pCsr, pCsr->pExpr, 1, &rc);
116896 return rc;
116897 }
116898
116899 /*
116900 ** Invalidate the current position list for phrase pPhrase.
116901 */
116902 static void fts3EvalInvalidatePoslist(Fts3Phrase *pPhrase){
116903 if( pPhrase->doclist.bFreeList ){
116904 sqlite3_free(pPhrase->doclist.pList);
116905 }
116906 pPhrase->doclist.pList = 0;
116907 pPhrase->doclist.nList = 0;
116908 pPhrase->doclist.bFreeList = 0;
116909 }
116910
116911 /*
116912 ** This function is called to edit the position list associated with
116913 ** the phrase object passed as the fifth argument according to a NEAR
116914 ** condition. For example:
116915 **
116916 ** abc NEAR/5 "def ghi"
116917 **
116918 ** Parameter nNear is passed the NEAR distance of the expression (5 in
116919 ** the example above). When this function is called, *paPoslist points to
116920 ** the position list, and *pnToken is the number of phrase tokens in, the
116921 ** phrase on the other side of the NEAR operator to pPhrase. For example,
116922 ** if pPhrase refers to the "def ghi" phrase, then *paPoslist points to
116923 ** the position list associated with phrase "abc".
116924 **
116925 ** All positions in the pPhrase position list that are not sufficiently
116926 ** close to a position in the *paPoslist position list are removed. If this
116927 ** leaves 0 positions, zero is returned. Otherwise, non-zero.
116928 **
116929 ** Before returning, *paPoslist is set to point to the position lsit
116930 ** associated with pPhrase. And *pnToken is set to the number of tokens in
116931 ** pPhrase.
116932 */
116933 static int fts3EvalNearTrim(
116934 int nNear, /* NEAR distance. As in "NEAR/nNear". */
116935 char *aTmp, /* Temporary space to use */
116936 char **paPoslist, /* IN/OUT: Position list */
116937 int *pnToken, /* IN/OUT: Tokens in phrase of *paPoslist */
116938 Fts3Phrase *pPhrase /* The phrase object to trim the doclist of */
116939 ){
116940 int nParam1 = nNear + pPhrase->nToken;
116941 int nParam2 = nNear + *pnToken;
116942 int nNew;
116943 char *p2;
116944 char *pOut;
116945 int res;
116946
116947 assert( pPhrase->doclist.pList );
116948
116949 p2 = pOut = pPhrase->doclist.pList;
116950 res = fts3PoslistNearMerge(
116951 &pOut, aTmp, nParam1, nParam2, paPoslist, &p2
116952 );
116953 if( res ){
116954 nNew = (pOut - pPhrase->doclist.pList) - 1;
116955 assert( pPhrase->doclist.pList[nNew]=='\0' );
116956 assert( nNew<=pPhrase->doclist.nList && nNew>0 );
116957 memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew);
116958 pPhrase->doclist.nList = nNew;
116959 *paPoslist = pPhrase->doclist.pList;
116960 *pnToken = pPhrase->nToken;
116961 }
116962
116963 return res;
116964 }
116965
116966 /*
116967 ** This function is a no-op if *pRc is other than SQLITE_OK when it is called.
116968 ** Otherwise, it advances the expression passed as the second argument to
116969 ** point to the next matching row in the database. Expressions iterate through
116970 ** matching rows in docid order. Ascending order if Fts3Cursor.bDesc is zero,
116971 ** or descending if it is non-zero.
116972 **
116973 ** If an error occurs, *pRc is set to an SQLite error code. Otherwise, if
116974 ** successful, the following variables in pExpr are set:
116975 **
116976 ** Fts3Expr.bEof (non-zero if EOF - there is no next row)
116977 ** Fts3Expr.iDocid (valid if bEof==0. The docid of the next row)
116978 **
116979 ** If the expression is of type FTSQUERY_PHRASE, and the expression is not
116980 ** at EOF, then the following variables are populated with the position list
116981 ** for the phrase for the visited row:
116982 **
116983 ** FTs3Expr.pPhrase->doclist.nList (length of pList in bytes)
116984 ** FTs3Expr.pPhrase->doclist.pList (pointer to position list)
116985 **
116986 ** It says above that this function advances the expression to the next
116987 ** matching row. This is usually true, but there are the following exceptions:
116988 **
116989 ** 1. Deferred tokens are not taken into account. If a phrase consists
116990 ** entirely of deferred tokens, it is assumed to match every row in
116991 ** the db. In this case the position-list is not populated at all.
116992 **
116993 ** Or, if a phrase contains one or more deferred tokens and one or
116994 ** more non-deferred tokens, then the expression is advanced to the
116995 ** next possible match, considering only non-deferred tokens. In other
116996 ** words, if the phrase is "A B C", and "B" is deferred, the expression
116997 ** is advanced to the next row that contains an instance of "A * C",
116998 ** where "*" may match any single token. The position list in this case
116999 ** is populated as for "A * C" before returning.
117000 **
117001 ** 2. NEAR is treated as AND. If the expression is "x NEAR y", it is
117002 ** advanced to point to the next row that matches "x AND y".
117003 **
117004 ** See fts3EvalTestDeferredAndNear() for details on testing if a row is
117005 ** really a match, taking into account deferred tokens and NEAR operators.
117006 */
117007 static void fts3EvalNextRow(
117008 Fts3Cursor *pCsr, /* FTS Cursor handle */
117009 Fts3Expr *pExpr, /* Expr. to advance to next matching row */
117010 int *pRc /* IN/OUT: Error code */
117011 ){
117012 if( *pRc==SQLITE_OK ){
117013 int bDescDoclist = pCsr->bDesc; /* Used by DOCID_CMP() macro */
117014 assert( pExpr->bEof==0 );
117015 pExpr->bStart = 1;
117016
117017 switch( pExpr->eType ){
117018 case FTSQUERY_NEAR:
117019 case FTSQUERY_AND: {
117020 Fts3Expr *pLeft = pExpr->pLeft;
117021 Fts3Expr *pRight = pExpr->pRight;
117022 assert( !pLeft->bDeferred || !pRight->bDeferred );
117023
117024 if( pLeft->bDeferred ){
117025 /* LHS is entirely deferred. So we assume it matches every row.
117026 ** Advance the RHS iterator to find the next row visited. */
117027 fts3EvalNextRow(pCsr, pRight, pRc);
117028 pExpr->iDocid = pRight->iDocid;
117029 pExpr->bEof = pRight->bEof;
117030 }else if( pRight->bDeferred ){
117031 /* RHS is entirely deferred. So we assume it matches every row.
117032 ** Advance the LHS iterator to find the next row visited. */
117033 fts3EvalNextRow(pCsr, pLeft, pRc);
117034 pExpr->iDocid = pLeft->iDocid;
117035 pExpr->bEof = pLeft->bEof;
117036 }else{
117037 /* Neither the RHS or LHS are deferred. */
117038 fts3EvalNextRow(pCsr, pLeft, pRc);
117039 fts3EvalNextRow(pCsr, pRight, pRc);
117040 while( !pLeft->bEof && !pRight->bEof && *pRc==SQLITE_OK ){
117041 sqlite3_int64 iDiff = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
117042 if( iDiff==0 ) break;
117043 if( iDiff<0 ){
117044 fts3EvalNextRow(pCsr, pLeft, pRc);
117045 }else{
117046 fts3EvalNextRow(pCsr, pRight, pRc);
117047 }
117048 }
117049 pExpr->iDocid = pLeft->iDocid;
117050 pExpr->bEof = (pLeft->bEof || pRight->bEof);
117051 }
117052 break;
117053 }
117054
117055 case FTSQUERY_OR: {
117056 Fts3Expr *pLeft = pExpr->pLeft;
117057 Fts3Expr *pRight = pExpr->pRight;
117058 sqlite3_int64 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
117059
117060 assert( pLeft->bStart || pLeft->iDocid==pRight->iDocid );
117061 assert( pRight->bStart || pLeft->iDocid==pRight->iDocid );
117062
117063 if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){
117064 fts3EvalNextRow(pCsr, pLeft, pRc);
117065 }else if( pLeft->bEof || (pRight->bEof==0 && iCmp>0) ){
117066 fts3EvalNextRow(pCsr, pRight, pRc);
117067 }else{
117068 fts3EvalNextRow(pCsr, pLeft, pRc);
117069 fts3EvalNextRow(pCsr, pRight, pRc);
117070 }
117071
117072 pExpr->bEof = (pLeft->bEof && pRight->bEof);
117073 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid);
117074 if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){
117075 pExpr->iDocid = pLeft->iDocid;
117076 }else{
117077 pExpr->iDocid = pRight->iDocid;
117078 }
117079
117080 break;
117081 }
117082
117083 case FTSQUERY_NOT: {
117084 Fts3Expr *pLeft = pExpr->pLeft;
117085 Fts3Expr *pRight = pExpr->pRight;
117086
117087 if( pRight->bStart==0 ){
117088 fts3EvalNextRow(pCsr, pRight, pRc);
117089 assert( *pRc!=SQLITE_OK || pRight->bStart );
117090 }
117091
117092 fts3EvalNextRow(pCsr, pLeft, pRc);
117093 if( pLeft->bEof==0 ){
117094 while( !*pRc
117095 && !pRight->bEof
117096 && DOCID_CMP(pLeft->iDocid, pRight->iDocid)>0
117097 ){
117098 fts3EvalNextRow(pCsr, pRight, pRc);
117099 }
117100 }
117101 pExpr->iDocid = pLeft->iDocid;
117102 pExpr->bEof = pLeft->bEof;
117103 break;
117104 }
117105
117106 default: {
117107 Fts3Phrase *pPhrase = pExpr->pPhrase;
117108 fts3EvalInvalidatePoslist(pPhrase);
117109 *pRc = fts3EvalPhraseNext(pCsr, pPhrase, &pExpr->bEof);
117110 pExpr->iDocid = pPhrase->doclist.iDocid;
117111 break;
117112 }
117113 }
117114 }
117115 }
117116
117117 /*
117118 ** If *pRc is not SQLITE_OK, or if pExpr is not the root node of a NEAR
117119 ** cluster, then this function returns 1 immediately.
117120 **
117121 ** Otherwise, it checks if the current row really does match the NEAR
117122 ** expression, using the data currently stored in the position lists
117123 ** (Fts3Expr->pPhrase.doclist.pList/nList) for each phrase in the expression.
117124 **
117125 ** If the current row is a match, the position list associated with each
117126 ** phrase in the NEAR expression is edited in place to contain only those
117127 ** phrase instances sufficiently close to their peers to satisfy all NEAR
117128 ** constraints. In this case it returns 1. If the NEAR expression does not
117129 ** match the current row, 0 is returned. The position lists may or may not
117130 ** be edited if 0 is returned.
117131 */
117132 static int fts3EvalNearTest(Fts3Expr *pExpr, int *pRc){
117133 int res = 1;
117134
117135 /* The following block runs if pExpr is the root of a NEAR query.
117136 ** For example, the query:
117137 **
117138 ** "w" NEAR "x" NEAR "y" NEAR "z"
117139 **
117140 ** which is represented in tree form as:
117141 **
117142 ** |
117143 ** +--NEAR--+ <-- root of NEAR query
117144 ** | |
117145 ** +--NEAR--+ "z"
117146 ** | |
117147 ** +--NEAR--+ "y"
117148 ** | |
117149 ** "w" "x"
117150 **
117151 ** The right-hand child of a NEAR node is always a phrase. The
117152 ** left-hand child may be either a phrase or a NEAR node. There are
117153 ** no exceptions to this - it's the way the parser in fts3_expr.c works.
117154 */
117155 if( *pRc==SQLITE_OK
117156 && pExpr->eType==FTSQUERY_NEAR
117157 && pExpr->bEof==0
117158 && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR)
117159 ){
117160 Fts3Expr *p;
117161 int nTmp = 0; /* Bytes of temp space */
117162 char *aTmp; /* Temp space for PoslistNearMerge() */
117163
117164 /* Allocate temporary working space. */
117165 for(p=pExpr; p->pLeft; p=p->pLeft){
117166 nTmp += p->pRight->pPhrase->doclist.nList;
117167 }
117168 nTmp += p->pPhrase->doclist.nList;
117169 aTmp = sqlite3_malloc(nTmp*2);
117170 if( !aTmp ){
117171 *pRc = SQLITE_NOMEM;
117172 res = 0;
117173 }else{
117174 char *aPoslist = p->pPhrase->doclist.pList;
117175 int nToken = p->pPhrase->nToken;
117176
117177 for(p=p->pParent;res && p && p->eType==FTSQUERY_NEAR; p=p->pParent){
117178 Fts3Phrase *pPhrase = p->pRight->pPhrase;
117179 int nNear = p->nNear;
117180 res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
117181 }
117182
117183 aPoslist = pExpr->pRight->pPhrase->doclist.pList;
117184 nToken = pExpr->pRight->pPhrase->nToken;
117185 for(p=pExpr->pLeft; p && res; p=p->pLeft){
117186 int nNear = p->pParent->nNear;
117187 Fts3Phrase *pPhrase = (
117188 p->eType==FTSQUERY_NEAR ? p->pRight->pPhrase : p->pPhrase
117189 );
117190 res = fts3EvalNearTrim(nNear, aTmp, &aPoslist, &nToken, pPhrase);
117191 }
117192 }
117193
117194 sqlite3_free(aTmp);
117195 }
117196
117197 return res;
117198 }
117199
117200 /*
117201 ** This function is a helper function for fts3EvalTestDeferredAndNear().
117202 ** Assuming no error occurs or has occurred, It returns non-zero if the
117203 ** expression passed as the second argument matches the row that pCsr
117204 ** currently points to, or zero if it does not.
117205 **
117206 ** If *pRc is not SQLITE_OK when this function is called, it is a no-op.
117207 ** If an error occurs during execution of this function, *pRc is set to
117208 ** the appropriate SQLite error code. In this case the returned value is
117209 ** undefined.
117210 */
117211 static int fts3EvalTestExpr(
117212 Fts3Cursor *pCsr, /* FTS cursor handle */
117213 Fts3Expr *pExpr, /* Expr to test. May or may not be root. */
117214 int *pRc /* IN/OUT: Error code */
117215 ){
117216 int bHit = 1; /* Return value */
117217 if( *pRc==SQLITE_OK ){
117218 switch( pExpr->eType ){
117219 case FTSQUERY_NEAR:
117220 case FTSQUERY_AND:
117221 bHit = (
117222 fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc)
117223 && fts3EvalTestExpr(pCsr, pExpr->pRight, pRc)
117224 && fts3EvalNearTest(pExpr, pRc)
117225 );
117226
117227 /* If the NEAR expression does not match any rows, zero the doclist for
117228 ** all phrases involved in the NEAR. This is because the snippet(),
117229 ** offsets() and matchinfo() functions are not supposed to recognize
117230 ** any instances of phrases that are part of unmatched NEAR queries.
117231 ** For example if this expression:
117232 **
117233 ** ... MATCH 'a OR (b NEAR c)'
117234 **
117235 ** is matched against a row containing:
117236 **
117237 ** 'a b d e'
117238 **
117239 ** then any snippet() should ony highlight the "a" term, not the "b"
117240 ** (as "b" is part of a non-matching NEAR clause).
117241 */
117242 if( bHit==0
117243 && pExpr->eType==FTSQUERY_NEAR
117244 && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR)
117245 ){
117246 Fts3Expr *p;
117247 for(p=pExpr; p->pPhrase==0; p=p->pLeft){
117248 if( p->pRight->iDocid==pCsr->iPrevId ){
117249 fts3EvalInvalidatePoslist(p->pRight->pPhrase);
117250 }
117251 }
117252 if( p->iDocid==pCsr->iPrevId ){
117253 fts3EvalInvalidatePoslist(p->pPhrase);
117254 }
117255 }
117256
117257 break;
117258
117259 case FTSQUERY_OR: {
117260 int bHit1 = fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc);
117261 int bHit2 = fts3EvalTestExpr(pCsr, pExpr->pRight, pRc);
117262 bHit = bHit1 || bHit2;
117263 break;
117264 }
117265
117266 case FTSQUERY_NOT:
117267 bHit = (
117268 fts3EvalTestExpr(pCsr, pExpr->pLeft, pRc)
117269 && !fts3EvalTestExpr(pCsr, pExpr->pRight, pRc)
117270 );
117271 break;
117272
117273 default: {
117274 if( pCsr->pDeferred
117275 && (pExpr->iDocid==pCsr->iPrevId || pExpr->bDeferred)
117276 ){
117277 Fts3Phrase *pPhrase = pExpr->pPhrase;
117278 assert( pExpr->bDeferred || pPhrase->doclist.bFreeList==0 );
117279 if( pExpr->bDeferred ){
117280 fts3EvalInvalidatePoslist(pPhrase);
117281 }
117282 *pRc = fts3EvalDeferredPhrase(pCsr, pPhrase);
117283 bHit = (pPhrase->doclist.pList!=0);
117284 pExpr->iDocid = pCsr->iPrevId;
117285 }else{
117286 bHit = (pExpr->bEof==0 && pExpr->iDocid==pCsr->iPrevId);
117287 }
117288 break;
117289 }
117290 }
117291 }
117292 return bHit;
117293 }
117294
117295 /*
117296 ** This function is called as the second part of each xNext operation when
117297 ** iterating through the results of a full-text query. At this point the
117298 ** cursor points to a row that matches the query expression, with the
117299 ** following caveats:
117300 **
117301 ** * Up until this point, "NEAR" operators in the expression have been
117302 ** treated as "AND".
117303 **
117304 ** * Deferred tokens have not yet been considered.
117305 **
117306 ** If *pRc is not SQLITE_OK when this function is called, it immediately
117307 ** returns 0. Otherwise, it tests whether or not after considering NEAR
117308 ** operators and deferred tokens the current row is still a match for the
117309 ** expression. It returns 1 if both of the following are true:
117310 **
117311 ** 1. *pRc is SQLITE_OK when this function returns, and
117312 **
117313 ** 2. After scanning the current FTS table row for the deferred tokens,
117314 ** it is determined that the row does *not* match the query.
117315 **
117316 ** Or, if no error occurs and it seems the current row does match the FTS
117317 ** query, return 0.
117318 */
117319 static int fts3EvalTestDeferredAndNear(Fts3Cursor *pCsr, int *pRc){
117320 int rc = *pRc;
117321 int bMiss = 0;
117322 if( rc==SQLITE_OK ){
117323
117324 /* If there are one or more deferred tokens, load the current row into
117325 ** memory and scan it to determine the position list for each deferred
117326 ** token. Then, see if this row is really a match, considering deferred
117327 ** tokens and NEAR operators (neither of which were taken into account
117328 ** earlier, by fts3EvalNextRow()).
117329 */
117330 if( pCsr->pDeferred ){
117331 rc = fts3CursorSeek(0, pCsr);
117332 if( rc==SQLITE_OK ){
117333 rc = sqlite3Fts3CacheDeferredDoclists(pCsr);
117334 }
117335 }
117336 bMiss = (0==fts3EvalTestExpr(pCsr, pCsr->pExpr, &rc));
117337
117338 /* Free the position-lists accumulated for each deferred token above. */
117339 sqlite3Fts3FreeDeferredDoclists(pCsr);
117340 *pRc = rc;
117341 }
117342 return (rc==SQLITE_OK && bMiss);
117343 }
117344
117345 /*
117346 ** Advance to the next document that matches the FTS expression in
117347 ** Fts3Cursor.pExpr.
117348 */
117349 static int fts3EvalNext(Fts3Cursor *pCsr){
117350 int rc = SQLITE_OK; /* Return Code */
117351 Fts3Expr *pExpr = pCsr->pExpr;
117352 assert( pCsr->isEof==0 );
117353 if( pExpr==0 ){
117354 pCsr->isEof = 1;
117355 }else{
117356 do {
117357 if( pCsr->isRequireSeek==0 ){
117358 sqlite3_reset(pCsr->pStmt);
117359 }
117360 assert( sqlite3_data_count(pCsr->pStmt)==0 );
117361 fts3EvalNextRow(pCsr, pExpr, &rc);
117362 pCsr->isEof = pExpr->bEof;
117363 pCsr->isRequireSeek = 1;
117364 pCsr->isMatchinfoNeeded = 1;
117365 pCsr->iPrevId = pExpr->iDocid;
117366 }while( pCsr->isEof==0 && fts3EvalTestDeferredAndNear(pCsr, &rc) );
117367 }
117368 return rc;
117369 }
117370
117371 /*
117372 ** Restart interation for expression pExpr so that the next call to
117373 ** fts3EvalNext() visits the first row. Do not allow incremental
117374 ** loading or merging of phrase doclists for this iteration.
117375 **
117376 ** If *pRc is other than SQLITE_OK when this function is called, it is
117377 ** a no-op. If an error occurs within this function, *pRc is set to an
117378 ** SQLite error code before returning.
117379 */
117380 static void fts3EvalRestart(
117381 Fts3Cursor *pCsr,
117382 Fts3Expr *pExpr,
117383 int *pRc
117384 ){
117385 if( pExpr && *pRc==SQLITE_OK ){
117386 Fts3Phrase *pPhrase = pExpr->pPhrase;
117387
117388 if( pPhrase ){
117389 fts3EvalInvalidatePoslist(pPhrase);
117390 if( pPhrase->bIncr ){
117391 assert( pPhrase->nToken==1 );
117392 assert( pPhrase->aToken[0].pSegcsr );
117393 sqlite3Fts3MsrIncrRestart(pPhrase->aToken[0].pSegcsr);
117394 *pRc = fts3EvalPhraseStart(pCsr, 0, pPhrase);
117395 }
117396
117397 pPhrase->doclist.pNextDocid = 0;
117398 pPhrase->doclist.iDocid = 0;
117399 }
117400
117401 pExpr->iDocid = 0;
117402 pExpr->bEof = 0;
117403 pExpr->bStart = 0;
117404
117405 fts3EvalRestart(pCsr, pExpr->pLeft, pRc);
117406 fts3EvalRestart(pCsr, pExpr->pRight, pRc);
117407 }
117408 }
117409
117410 /*
117411 ** After allocating the Fts3Expr.aMI[] array for each phrase in the
117412 ** expression rooted at pExpr, the cursor iterates through all rows matched
117413 ** by pExpr, calling this function for each row. This function increments
117414 ** the values in Fts3Expr.aMI[] according to the position-list currently
117415 ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase
117416 ** expression nodes.
117417 */
117418 static void fts3EvalUpdateCounts(Fts3Expr *pExpr){
117419 if( pExpr ){
117420 Fts3Phrase *pPhrase = pExpr->pPhrase;
117421 if( pPhrase && pPhrase->doclist.pList ){
117422 int iCol = 0;
117423 char *p = pPhrase->doclist.pList;
117424
117425 assert( *p );
117426 while( 1 ){
117427 u8 c = 0;
117428 int iCnt = 0;
117429 while( 0xFE & (*p | c) ){
117430 if( (c&0x80)==0 ) iCnt++;
117431 c = *p++ & 0x80;
117432 }
117433
117434 /* aMI[iCol*3 + 1] = Number of occurrences
117435 ** aMI[iCol*3 + 2] = Number of rows containing at least one instance
117436 */
117437 pExpr->aMI[iCol*3 + 1] += iCnt;
117438 pExpr->aMI[iCol*3 + 2] += (iCnt>0);
117439 if( *p==0x00 ) break;
117440 p++;
117441 p += sqlite3Fts3GetVarint32(p, &iCol);
117442 }
117443 }
117444
117445 fts3EvalUpdateCounts(pExpr->pLeft);
117446 fts3EvalUpdateCounts(pExpr->pRight);
117447 }
117448 }
117449
117450 /*
117451 ** Expression pExpr must be of type FTSQUERY_PHRASE.
117452 **
117453 ** If it is not already allocated and populated, this function allocates and
117454 ** populates the Fts3Expr.aMI[] array for expression pExpr. If pExpr is part
117455 ** of a NEAR expression, then it also allocates and populates the same array
117456 ** for all other phrases that are part of the NEAR expression.
117457 **
117458 ** SQLITE_OK is returned if the aMI[] array is successfully allocated and
117459 ** populated. Otherwise, if an error occurs, an SQLite error code is returned.
117460 */
117461 static int fts3EvalGatherStats(
117462 Fts3Cursor *pCsr, /* Cursor object */
117463 Fts3Expr *pExpr /* FTSQUERY_PHRASE expression */
117464 ){
117465 int rc = SQLITE_OK; /* Return code */
117466
117467 assert( pExpr->eType==FTSQUERY_PHRASE );
117468 if( pExpr->aMI==0 ){
117469 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
117470 Fts3Expr *pRoot; /* Root of NEAR expression */
117471 Fts3Expr *p; /* Iterator used for several purposes */
117472
117473 sqlite3_int64 iPrevId = pCsr->iPrevId;
117474 sqlite3_int64 iDocid;
117475 u8 bEof;
117476
117477 /* Find the root of the NEAR expression */
117478 pRoot = pExpr;
117479 while( pRoot->pParent && pRoot->pParent->eType==FTSQUERY_NEAR ){
117480 pRoot = pRoot->pParent;
117481 }
117482 iDocid = pRoot->iDocid;
117483 bEof = pRoot->bEof;
117484 assert( pRoot->bStart );
117485
117486 /* Allocate space for the aMSI[] array of each FTSQUERY_PHRASE node */
117487 for(p=pRoot; p; p=p->pLeft){
117488 Fts3Expr *pE = (p->eType==FTSQUERY_PHRASE?p:p->pRight);
117489 assert( pE->aMI==0 );
117490 pE->aMI = (u32 *)sqlite3_malloc(pTab->nColumn * 3 * sizeof(u32));
117491 if( !pE->aMI ) return SQLITE_NOMEM;
117492 memset(pE->aMI, 0, pTab->nColumn * 3 * sizeof(u32));
117493 }
117494
117495 fts3EvalRestart(pCsr, pRoot, &rc);
117496
117497 while( pCsr->isEof==0 && rc==SQLITE_OK ){
117498
117499 do {
117500 /* Ensure the %_content statement is reset. */
117501 if( pCsr->isRequireSeek==0 ) sqlite3_reset(pCsr->pStmt);
117502 assert( sqlite3_data_count(pCsr->pStmt)==0 );
117503
117504 /* Advance to the next document */
117505 fts3EvalNextRow(pCsr, pRoot, &rc);
117506 pCsr->isEof = pRoot->bEof;
117507 pCsr->isRequireSeek = 1;
117508 pCsr->isMatchinfoNeeded = 1;
117509 pCsr->iPrevId = pRoot->iDocid;
117510 }while( pCsr->isEof==0
117511 && pRoot->eType==FTSQUERY_NEAR
117512 && fts3EvalTestDeferredAndNear(pCsr, &rc)
117513 );
117514
117515 if( rc==SQLITE_OK && pCsr->isEof==0 ){
117516 fts3EvalUpdateCounts(pRoot);
117517 }
117518 }
117519
117520 pCsr->isEof = 0;
117521 pCsr->iPrevId = iPrevId;
117522
117523 if( bEof ){
117524 pRoot->bEof = bEof;
117525 }else{
117526 /* Caution: pRoot may iterate through docids in ascending or descending
117527 ** order. For this reason, even though it seems more defensive, the
117528 ** do loop can not be written:
117529 **
117530 ** do {...} while( pRoot->iDocid<iDocid && rc==SQLITE_OK );
117531 */
117532 fts3EvalRestart(pCsr, pRoot, &rc);
117533 do {
117534 fts3EvalNextRow(pCsr, pRoot, &rc);
117535 assert( pRoot->bEof==0 );
117536 }while( pRoot->iDocid!=iDocid && rc==SQLITE_OK );
117537 fts3EvalTestDeferredAndNear(pCsr, &rc);
117538 }
117539 }
117540 return rc;
117541 }
117542
117543 /*
117544 ** This function is used by the matchinfo() module to query a phrase
117545 ** expression node for the following information:
117546 **
117547 ** 1. The total number of occurrences of the phrase in each column of
117548 ** the FTS table (considering all rows), and
117549 **
117550 ** 2. For each column, the number of rows in the table for which the
117551 ** column contains at least one instance of the phrase.
117552 **
117553 ** If no error occurs, SQLITE_OK is returned and the values for each column
117554 ** written into the array aiOut as follows:
117555 **
117556 ** aiOut[iCol*3 + 1] = Number of occurrences
117557 ** aiOut[iCol*3 + 2] = Number of rows containing at least one instance
117558 **
117559 ** Caveats:
117560 **
117561 ** * If a phrase consists entirely of deferred tokens, then all output
117562 ** values are set to the number of documents in the table. In other
117563 ** words we assume that very common tokens occur exactly once in each
117564 ** column of each row of the table.
117565 **
117566 ** * If a phrase contains some deferred tokens (and some non-deferred
117567 ** tokens), count the potential occurrence identified by considering
117568 ** the non-deferred tokens instead of actual phrase occurrences.
117569 **
117570 ** * If the phrase is part of a NEAR expression, then only phrase instances
117571 ** that meet the NEAR constraint are included in the counts.
117572 */
117573 SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats(
117574 Fts3Cursor *pCsr, /* FTS cursor handle */
117575 Fts3Expr *pExpr, /* Phrase expression */
117576 u32 *aiOut /* Array to write results into (see above) */
117577 ){
117578 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
117579 int rc = SQLITE_OK;
117580 int iCol;
117581
117582 if( pExpr->bDeferred && pExpr->pParent->eType!=FTSQUERY_NEAR ){
117583 assert( pCsr->nDoc>0 );
117584 for(iCol=0; iCol<pTab->nColumn; iCol++){
117585 aiOut[iCol*3 + 1] = (u32)pCsr->nDoc;
117586 aiOut[iCol*3 + 2] = (u32)pCsr->nDoc;
117587 }
117588 }else{
117589 rc = fts3EvalGatherStats(pCsr, pExpr);
117590 if( rc==SQLITE_OK ){
117591 assert( pExpr->aMI );
117592 for(iCol=0; iCol<pTab->nColumn; iCol++){
117593 aiOut[iCol*3 + 1] = pExpr->aMI[iCol*3 + 1];
117594 aiOut[iCol*3 + 2] = pExpr->aMI[iCol*3 + 2];
117595 }
117596 }
117597 }
117598
117599 return rc;
117600 }
117601
117602 /*
117603 ** The expression pExpr passed as the second argument to this function
117604 ** must be of type FTSQUERY_PHRASE.
117605 **
117606 ** The returned value is either NULL or a pointer to a buffer containing
117607 ** a position-list indicating the occurrences of the phrase in column iCol
117608 ** of the current row.
117609 **
117610 ** More specifically, the returned buffer contains 1 varint for each
117611 ** occurence of the phrase in the column, stored using the normal (delta+2)
117612 ** compression and is terminated by either an 0x01 or 0x00 byte. For example,
117613 ** if the requested column contains "a b X c d X X" and the position-list
117614 ** for 'X' is requested, the buffer returned may contain:
117615 **
117616 ** 0x04 0x05 0x03 0x01 or 0x04 0x05 0x03 0x00
117617 **
117618 ** This function works regardless of whether or not the phrase is deferred,
117619 ** incremental, or neither.
117620 */
117621 SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist(
117622 Fts3Cursor *pCsr, /* FTS3 cursor object */
117623 Fts3Expr *pExpr, /* Phrase to return doclist for */
117624 int iCol /* Column to return position list for */
117625 ){
117626 Fts3Phrase *pPhrase = pExpr->pPhrase;
117627 Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab;
117628 char *pIter = pPhrase->doclist.pList;
117629 int iThis;
117630
117631 assert( iCol>=0 && iCol<pTab->nColumn );
117632 if( !pIter
117633 || pExpr->bEof
117634 || pExpr->iDocid!=pCsr->iPrevId
117635 || (pPhrase->iColumn<pTab->nColumn && pPhrase->iColumn!=iCol)
117636 ){
117637 return 0;
117638 }
117639
117640 assert( pPhrase->doclist.nList>0 );
117641 if( *pIter==0x01 ){
117642 pIter++;
117643 pIter += sqlite3Fts3GetVarint32(pIter, &iThis);
117644 }else{
117645 iThis = 0;
117646 }
117647 while( iThis<iCol ){
117648 fts3ColumnlistCopy(0, &pIter);
117649 if( *pIter==0x00 ) return 0;
117650 pIter++;
117651 pIter += sqlite3Fts3GetVarint32(pIter, &iThis);
117652 }
117653
117654 return ((iCol==iThis)?pIter:0);
117655 }
117656
117657 /*
117658 ** Free all components of the Fts3Phrase structure that were allocated by
117659 ** the eval module. Specifically, this means to free:
117660 **
117661 ** * the contents of pPhrase->doclist, and
117662 ** * any Fts3MultiSegReader objects held by phrase tokens.
117663 */
117664 SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *pPhrase){
117665 if( pPhrase ){
117666 int i;
117667 sqlite3_free(pPhrase->doclist.aAll);
117668 fts3EvalInvalidatePoslist(pPhrase);
117669 memset(&pPhrase->doclist, 0, sizeof(Fts3Doclist));
117670 for(i=0; i<pPhrase->nToken; i++){
117671 fts3SegReaderCursorFree(pPhrase->aToken[i].pSegcsr);
117672 pPhrase->aToken[i].pSegcsr = 0;
117673 }
117674 }
117675 }
117676
117677 #if !SQLITE_CORE
117678 /*
117679 ** Initialize API pointer table, if required.
117680 */
117681 SQLITE_API int sqlite3_extension_init(
117682 sqlite3 *db,
117683 char **pzErrMsg,
117684 const sqlite3_api_routines *pApi
117685 ){
@@ -115634,11 +117703,10 @@
117703 ** May you share freely, never taking more than you give.
117704 **
117705 ******************************************************************************
117706 **
117707 */
 
117708 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
117709
117710
117711 typedef struct Fts3auxTable Fts3auxTable;
117712 typedef struct Fts3auxCursor Fts3auxCursor;
@@ -115648,11 +117716,11 @@
117716 Fts3Table *pFts3Tab;
117717 };
117718
117719 struct Fts3auxCursor {
117720 sqlite3_vtab_cursor base; /* Base class used by SQLite core */
117721 Fts3MultiSegReader csr; /* Must be right after "base" */
117722 Fts3SegFilter filter;
117723 char *zStop;
117724 int nStop; /* Byte-length of string zStop */
117725 int isEof; /* True if cursor is at EOF */
117726 sqlite3_int64 iRowid; /* Current rowid */
@@ -115716,10 +117784,11 @@
117784
117785 p->pFts3Tab = (Fts3Table *)&p[1];
117786 p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1];
117787 p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1];
117788 p->pFts3Tab->db = db;
117789 p->pFts3Tab->nIndex = 1;
117790
117791 memcpy((char *)p->pFts3Tab->zDb, zDb, nDb);
117792 memcpy((char *)p->pFts3Tab->zName, zFts3, nFts3);
117793 sqlite3Fts3Dequote((char *)p->pFts3Tab->zName);
117794
@@ -115996,11 +118065,11 @@
118065 pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iIdx]));
118066 pCsr->nStop = sqlite3_value_bytes(apVal[iIdx]);
118067 if( pCsr->zStop==0 ) return SQLITE_NOMEM;
118068 }
118069
118070 rc = sqlite3Fts3SegReaderCursor(pFts3, 0, FTS3_SEGCURSOR_ALL,
118071 pCsr->filter.zTerm, pCsr->filter.nTerm, 0, isScan, &pCsr->csr
118072 );
118073 if( rc==SQLITE_OK ){
118074 rc = sqlite3Fts3SegReaderStart(pFts3, &pCsr->csr, &pCsr->filter);
118075 }
@@ -116175,16 +118244,25 @@
118244 ** Default span for NEAR operators.
118245 */
118246 #define SQLITE_FTS3_DEFAULT_NEAR_PARAM 10
118247
118248
118249 /*
118250 ** isNot:
118251 ** This variable is used by function getNextNode(). When getNextNode() is
118252 ** called, it sets ParseContext.isNot to true if the 'next node' is a
118253 ** FTSQUERY_PHRASE with a unary "-" attached to it. i.e. "mysql" in the
118254 ** FTS3 query "sqlite -mysql". Otherwise, ParseContext.isNot is set to
118255 ** zero.
118256 */
118257 typedef struct ParseContext ParseContext;
118258 struct ParseContext {
118259 sqlite3_tokenizer *pTokenizer; /* Tokenizer module */
118260 const char **azCol; /* Array of column names for fts3 table */
118261 int nCol; /* Number of entries in azCol[] */
118262 int iDefaultCol; /* Default column to query */
118263 int isNot; /* True if getNextNode() sees a unary - */
118264 sqlite3_context *pCtx; /* Write error message here */
118265 int nNest; /* Number of nested brackets */
118266 };
118267
118268 /*
@@ -116266,11 +118344,11 @@
118344 if( iEnd<n && z[iEnd]=='*' ){
118345 pRet->pPhrase->aToken[0].isPrefix = 1;
118346 iEnd++;
118347 }
118348 if( !sqlite3_fts3_enable_parentheses && iStart>0 && z[iStart-1]=='-' ){
118349 pParse->isNot = 1;
118350 }
118351 }
118352 nConsumed = iEnd;
118353 }
118354
@@ -116318,71 +118396,86 @@
118396 Fts3Expr *p = 0;
118397 sqlite3_tokenizer_cursor *pCursor = 0;
118398 char *zTemp = 0;
118399 int nTemp = 0;
118400
118401 const int nSpace = sizeof(Fts3Expr) + sizeof(Fts3Phrase);
118402 int nToken = 0;
118403
118404 /* The final Fts3Expr data structure, including the Fts3Phrase,
118405 ** Fts3PhraseToken structures token buffers are all stored as a single
118406 ** allocation so that the expression can be freed with a single call to
118407 ** sqlite3_free(). Setting this up requires a two pass approach.
118408 **
118409 ** The first pass, in the block below, uses a tokenizer cursor to iterate
118410 ** through the tokens in the expression. This pass uses fts3ReallocOrFree()
118411 ** to assemble data in two dynamic buffers:
118412 **
118413 ** Buffer p: Points to the Fts3Expr structure, followed by the Fts3Phrase
118414 ** structure, followed by the array of Fts3PhraseToken
118415 ** structures. This pass only populates the Fts3PhraseToken array.
118416 **
118417 ** Buffer zTemp: Contains copies of all tokens.
118418 **
118419 ** The second pass, in the block that begins "if( rc==SQLITE_DONE )" below,
118420 ** appends buffer zTemp to buffer p, and fills in the Fts3Expr and Fts3Phrase
118421 ** structures.
118422 */
118423 rc = pModule->xOpen(pTokenizer, zInput, nInput, &pCursor);
118424 if( rc==SQLITE_OK ){
118425 int ii;
118426 pCursor->pTokenizer = pTokenizer;
118427 for(ii=0; rc==SQLITE_OK; ii++){
118428 const char *zByte;
118429 int nByte, iBegin, iEnd, iPos;
118430 rc = pModule->xNext(pCursor, &zByte, &nByte, &iBegin, &iEnd, &iPos);
118431 if( rc==SQLITE_OK ){
118432 Fts3PhraseToken *pToken;
118433
118434 p = fts3ReallocOrFree(p, nSpace + ii*sizeof(Fts3PhraseToken));
118435 if( !p ) goto no_mem;
118436
118437 zTemp = fts3ReallocOrFree(zTemp, nTemp + nByte);
118438 if( !zTemp ) goto no_mem;
118439
118440 assert( nToken==ii );
118441 pToken = &((Fts3Phrase *)(&p[1]))->aToken[ii];
118442 memset(pToken, 0, sizeof(Fts3PhraseToken));
118443
118444 memcpy(&zTemp[nTemp], zByte, nByte);
118445 nTemp += nByte;
118446
118447 pToken->n = nByte;
118448 pToken->isPrefix = (iEnd<nInput && zInput[iEnd]=='*');
118449 nToken = ii+1;
 
 
 
118450 }
118451 }
118452
118453 pModule->xClose(pCursor);
118454 pCursor = 0;
118455 }
118456
118457 if( rc==SQLITE_DONE ){
118458 int jj;
118459 char *zBuf = 0;
118460
118461 p = fts3ReallocOrFree(p, nSpace + nToken*sizeof(Fts3PhraseToken) + nTemp);
118462 if( !p ) goto no_mem;
118463 memset(p, 0, (char *)&(((Fts3Phrase *)&p[1])->aToken[0])-(char *)p);
118464 p->eType = FTSQUERY_PHRASE;
 
 
 
 
 
 
 
 
118465 p->pPhrase = (Fts3Phrase *)&p[1];
118466 p->pPhrase->iColumn = pParse->iDefaultCol;
118467 p->pPhrase->nToken = nToken;
118468
118469 zBuf = (char *)&p->pPhrase->aToken[nToken];
118470 memcpy(zBuf, zTemp, nTemp);
118471 sqlite3_free(zTemp);
118472
118473 for(jj=0; jj<p->pPhrase->nToken; jj++){
118474 p->pPhrase->aToken[jj].z = zBuf;
118475 zBuf += p->pPhrase->aToken[jj].n;
118476 }
 
 
 
118477 rc = SQLITE_OK;
118478 }
118479
118480 *ppExpr = p;
118481 return rc;
@@ -116434,10 +118527,12 @@
118527 int rc;
118528 Fts3Expr *pRet = 0;
118529
118530 const char *zInput = z;
118531 int nInput = n;
118532
118533 pParse->isNot = 0;
118534
118535 /* Skip over any whitespace before checking for a keyword, an open or
118536 ** close bracket, or a quoted string.
118537 */
118538 while( nInput>0 && fts3isspace(*zInput) ){
@@ -116653,11 +118748,11 @@
118748 rc = getNextNode(pParse, zIn, nIn, &p, &nByte);
118749 if( rc==SQLITE_OK ){
118750 int isPhrase;
118751
118752 if( !sqlite3_fts3_enable_parentheses
118753 && p->eType==FTSQUERY_PHRASE && pParse->isNot
118754 ){
118755 /* Create an implicit NOT operator. */
118756 Fts3Expr *pNot = fts3MallocZero(sizeof(Fts3Expr));
118757 if( !pNot ){
118758 sqlite3Fts3ExprFree(p);
@@ -116671,11 +118766,10 @@
118766 }
118767 pNotBranch = pNot;
118768 p = pPrev;
118769 }else{
118770 int eType = p->eType;
 
118771 isPhrase = (eType==FTSQUERY_PHRASE || p->pLeft);
118772
118773 /* The isRequirePhrase variable is set to true if a phrase or
118774 ** an expression contained in parenthesis is required. If a
118775 ** binary operator (AND, OR, NOT or NEAR) is encounted when
@@ -116834,13 +118928,15 @@
118928 /*
118929 ** Free a parsed fts3 query expression allocated by sqlite3Fts3ExprParse().
118930 */
118931 SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *p){
118932 if( p ){
118933 assert( p->eType==FTSQUERY_PHRASE || p->pPhrase==0 );
118934 sqlite3Fts3ExprFree(p->pLeft);
118935 sqlite3Fts3ExprFree(p->pRight);
118936 sqlite3Fts3EvalPhraseCleanup(p->pPhrase);
118937 sqlite3_free(p->aMI);
118938 sqlite3_free(p);
118939 }
118940 }
118941
118942 /****************************************************************************
@@ -116893,11 +118989,11 @@
118989 switch( pExpr->eType ){
118990 case FTSQUERY_PHRASE: {
118991 Fts3Phrase *pPhrase = pExpr->pPhrase;
118992 int i;
118993 zBuf = sqlite3_mprintf(
118994 "%zPHRASE %d 0", zBuf, pPhrase->iColumn);
118995 for(i=0; zBuf && i<pPhrase->nToken; i++){
118996 zBuf = sqlite3_mprintf("%z %.*s%s", zBuf,
118997 pPhrase->aToken[i].n, pPhrase->aToken[i].z,
118998 (pPhrase->aToken[i].isPrefix?"+":"")
118999 );
@@ -117437,11 +119533,10 @@
119533 **
119534 ** * The FTS3 module is being built into the core of
119535 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
119536 */
119537 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
 
119538
119539
119540
119541 /*
119542 ** Class derived from sqlite3_tokenizer
@@ -118082,14 +120177,10 @@
120177 ** * The FTS3 module is being built into the core of
120178 ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined).
120179 */
120180 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
120181
 
 
 
 
120182
120183 /*
120184 ** Implementation of the SQL scalar function for accessing the underlying
120185 ** hash table. This function may be called as follows:
120186 **
@@ -118209,11 +120300,11 @@
120300 sqlite3_tokenizer **ppTok, /* OUT: Tokenizer (if applicable) */
120301 char **pzErr /* OUT: Set to malloced error message */
120302 ){
120303 int rc;
120304 char *z = (char *)zArg;
120305 int n = 0;
120306 char *zCopy;
120307 char *zEnd; /* Pointer to nul-term of zCopy */
120308 sqlite3_tokenizer_module *m;
120309
120310 zCopy = sqlite3_mprintf("%s", zArg);
@@ -118574,11 +120665,10 @@
120665 */
120666 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
120667
120668
120669
 
120670 typedef struct simple_tokenizer {
120671 sqlite3_tokenizer base;
120672 char delim[128]; /* flag ASCII delimiters */
120673 } simple_tokenizer;
120674
@@ -118811,18 +120901,44 @@
120901 ** it is always safe to read up to two varints from it without risking an
120902 ** overread, even if the node data is corrupted.
120903 */
120904 #define FTS3_NODE_PADDING (FTS3_VARINT_MAX*2)
120905
120906 /*
120907 ** Under certain circumstances, b-tree nodes (doclists) can be loaded into
120908 ** memory incrementally instead of all at once. This can be a big performance
120909 ** win (reduced IO and CPU) if SQLite stops calling the virtual table xNext()
120910 ** method before retrieving all query results (as may happen, for example,
120911 ** if a query has a LIMIT clause).
120912 **
120913 ** Incremental loading is used for b-tree nodes FTS3_NODE_CHUNK_THRESHOLD
120914 ** bytes and larger. Nodes are loaded in chunks of FTS3_NODE_CHUNKSIZE bytes.
120915 ** The code is written so that the hard lower-limit for each of these values
120916 ** is 1. Clearly such small values would be inefficient, but can be useful
120917 ** for testing purposes.
120918 **
120919 ** If this module is built with SQLITE_TEST defined, these constants may
120920 ** be overridden at runtime for testing purposes. File fts3_test.c contains
120921 ** a Tcl interface to read and write the values.
120922 */
120923 #ifdef SQLITE_TEST
120924 int test_fts3_node_chunksize = (4*1024);
120925 int test_fts3_node_chunk_threshold = (4*1024)*4;
120926 # define FTS3_NODE_CHUNKSIZE test_fts3_node_chunksize
120927 # define FTS3_NODE_CHUNK_THRESHOLD test_fts3_node_chunk_threshold
120928 #else
120929 # define FTS3_NODE_CHUNKSIZE (4*1024)
120930 # define FTS3_NODE_CHUNK_THRESHOLD (FTS3_NODE_CHUNKSIZE*4)
120931 #endif
120932
120933 typedef struct PendingList PendingList;
120934 typedef struct SegmentNode SegmentNode;
120935 typedef struct SegmentWriter SegmentWriter;
120936
120937 /*
120938 ** An instance of the following data structure is used to build doclists
120939 ** incrementally. See function fts3PendingListAppend() for details.
 
120940 */
120941 struct PendingList {
120942 int nData;
120943 char *aData;
120944 int nSpace;
@@ -118849,11 +120965,10 @@
120965 ** of type Fts3SegReader* are also used by code in fts3.c to iterate through
120966 ** terms when querying the full-text index. See functions:
120967 **
120968 ** sqlite3Fts3SegReaderNew()
120969 ** sqlite3Fts3SegReaderFree()
 
120970 ** sqlite3Fts3SegReaderIterate()
120971 **
120972 ** Methods used to manipulate Fts3SegReader structures:
120973 **
120974 ** fts3SegReaderNext()
@@ -118868,10 +120983,13 @@
120983 sqlite3_int64 iEndBlock; /* Rowid of final block in segment (or 0) */
120984 sqlite3_int64 iCurrentBlock; /* Current leaf block (or 0) */
120985
120986 char *aNode; /* Pointer to node data (or NULL) */
120987 int nNode; /* Size of buffer at aNode (or 0) */
120988 int nPopulate; /* If >0, bytes of buffer aNode[] loaded */
120989 sqlite3_blob *pBlob; /* If not NULL, blob handle to read node */
120990
120991 Fts3HashElem **ppNextElem;
120992
120993 /* Variables set by fts3SegReaderNext(). These may be read directly
120994 ** by the caller. They are valid from the time SegmentReaderNew() returns
120995 ** until SegmentReaderNext() returns something other than SQLITE_OK
@@ -118881,12 +120999,15 @@
120999 char *zTerm; /* Pointer to current term */
121000 int nTermAlloc; /* Allocated size of zTerm buffer */
121001 char *aDoclist; /* Pointer to doclist of current entry */
121002 int nDoclist; /* Size of doclist in current entry */
121003
121004 /* The following variables are used by fts3SegReaderNextDocid() to iterate
121005 ** through the current doclist (aDoclist/nDoclist).
121006 */
121007 char *pOffsetList;
121008 int nOffsetList; /* For descending pending seg-readers only */
121009 sqlite3_int64 iDocid;
121010 };
121011
121012 #define fts3SegReaderIsPending(p) ((p)->ppNextElem!=0)
121013 #define fts3SegReaderIsRootOnly(p) ((p)->aNode==(char *)&(p)[1])
@@ -118920,10 +121041,18 @@
121041 ** within the fts3SegWriterXXX() family of functions described above.
121042 **
121043 ** fts3NodeAddTerm()
121044 ** fts3NodeWrite()
121045 ** fts3NodeFree()
121046 **
121047 ** When a b+tree is written to the database (either as a result of a merge
121048 ** or the pending-terms table being flushed), leaves are written into the
121049 ** database file as soon as they are completely populated. The interior of
121050 ** the tree is assembled in memory and written out only once all leaves have
121051 ** been populated and stored. This is Ok, as the b+-tree fanout is usually
121052 ** very large, meaning that the interior of the tree consumes relatively
121053 ** little memory.
121054 */
121055 struct SegmentNode {
121056 SegmentNode *pParent; /* Parent node (or NULL for root node) */
121057 SegmentNode *pRight; /* Pointer to right-sibling */
121058 SegmentNode *pLeftmost; /* Pointer to left-most node of this depth */
@@ -118950,21 +121079,26 @@
121079 #define SQL_NEXT_SEGMENT_INDEX 8
121080 #define SQL_INSERT_SEGMENTS 9
121081 #define SQL_NEXT_SEGMENTS_ID 10
121082 #define SQL_INSERT_SEGDIR 11
121083 #define SQL_SELECT_LEVEL 12
121084 #define SQL_SELECT_LEVEL_RANGE 13
121085 #define SQL_SELECT_LEVEL_COUNT 14
121086 #define SQL_SELECT_SEGDIR_MAX_LEVEL 15
121087 #define SQL_DELETE_SEGDIR_LEVEL 16
121088 #define SQL_DELETE_SEGMENTS_RANGE 17
121089 #define SQL_CONTENT_INSERT 18
121090 #define SQL_DELETE_DOCSIZE 19
121091 #define SQL_REPLACE_DOCSIZE 20
121092 #define SQL_SELECT_DOCSIZE 21
121093 #define SQL_SELECT_DOCTOTAL 22
121094 #define SQL_REPLACE_DOCTOTAL 23
121095
121096 #define SQL_SELECT_ALL_PREFIX_LEVEL 24
121097 #define SQL_DELETE_ALL_TERMS_SEGDIR 25
121098
121099 #define SQL_DELETE_SEGDIR_RANGE 26
121100
121101 /*
121102 ** This function is used to obtain an SQLite prepared statement handle
121103 ** for the statement identified by the second argument. If successful,
121104 ** *pp is set to the requested statement handle and SQLITE_OK returned.
@@ -118997,23 +121131,29 @@
121131
121132 /* Return segments in order from oldest to newest.*/
121133 /* 12 */ "SELECT idx, start_block, leaves_end_block, end_block, root "
121134 "FROM %Q.'%q_segdir' WHERE level = ? ORDER BY idx ASC",
121135 /* 13 */ "SELECT idx, start_block, leaves_end_block, end_block, root "
121136 "FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?"
121137 "ORDER BY level DESC, idx ASC",
121138
121139 /* 14 */ "SELECT count(*) FROM %Q.'%q_segdir' WHERE level = ?",
121140 /* 15 */ "SELECT max(level) FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?",
121141
121142 /* 16 */ "DELETE FROM %Q.'%q_segdir' WHERE level = ?",
121143 /* 17 */ "DELETE FROM %Q.'%q_segments' WHERE blockid BETWEEN ? AND ?",
121144 /* 18 */ "INSERT INTO %Q.'%q_content' VALUES(%s)",
121145 /* 19 */ "DELETE FROM %Q.'%q_docsize' WHERE docid = ?",
121146 /* 20 */ "REPLACE INTO %Q.'%q_docsize' VALUES(?,?)",
121147 /* 21 */ "SELECT size FROM %Q.'%q_docsize' WHERE docid=?",
121148 /* 22 */ "SELECT value FROM %Q.'%q_stat' WHERE id=0",
121149 /* 23 */ "REPLACE INTO %Q.'%q_stat' VALUES(0,?)",
121150 /* 24 */ "",
121151 /* 25 */ "",
121152
121153 /* 26 */ "DELETE FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?",
121154
121155 };
121156 int rc = SQLITE_OK;
121157 sqlite3_stmt *pStmt;
121158
121159 assert( SizeofArray(azSql)==SizeofArray(p->aStmt) );
@@ -119165,18 +121305,36 @@
121305 ** 1: start_block
121306 ** 2: leaves_end_block
121307 ** 3: end_block
121308 ** 4: root
121309 */
121310 SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(
121311 Fts3Table *p, /* FTS3 table */
121312 int iIndex, /* Index for p->aIndex[] */
121313 int iLevel, /* Level to select */
121314 sqlite3_stmt **ppStmt /* OUT: Compiled statement */
121315 ){
121316 int rc;
121317 sqlite3_stmt *pStmt = 0;
121318
121319 assert( iLevel==FTS3_SEGCURSOR_ALL || iLevel>=0 );
121320 assert( iLevel<FTS3_SEGDIR_MAXLEVEL );
121321 assert( iIndex>=0 && iIndex<p->nIndex );
121322
121323 if( iLevel<0 ){
121324 /* "SELECT * FROM %_segdir WHERE level BETWEEN ? AND ? ORDER BY ..." */
121325 rc = fts3SqlStmt(p, SQL_SELECT_LEVEL_RANGE, &pStmt, 0);
121326 if( rc==SQLITE_OK ){
121327 sqlite3_bind_int(pStmt, 1, iIndex*FTS3_SEGDIR_MAXLEVEL);
121328 sqlite3_bind_int(pStmt, 2, (iIndex+1)*FTS3_SEGDIR_MAXLEVEL-1);
121329 }
121330 }else{
121331 /* "SELECT * FROM %_segdir WHERE level = ? ORDER BY ..." */
121332 rc = fts3SqlStmt(p, SQL_SELECT_LEVEL, &pStmt, 0);
121333 if( rc==SQLITE_OK ){
121334 sqlite3_bind_int(pStmt, 1, iLevel+iIndex*FTS3_SEGDIR_MAXLEVEL);
121335 }
121336 }
121337 *ppStmt = pStmt;
121338 return rc;
121339 }
121340
@@ -119286,10 +121444,51 @@
121444 *pp = p;
121445 return 1;
121446 }
121447 return 0;
121448 }
121449
121450 /*
121451 ** Free a PendingList object allocated by fts3PendingListAppend().
121452 */
121453 static void fts3PendingListDelete(PendingList *pList){
121454 sqlite3_free(pList);
121455 }
121456
121457 /*
121458 ** Add an entry to one of the pending-terms hash tables.
121459 */
121460 static int fts3PendingTermsAddOne(
121461 Fts3Table *p,
121462 int iCol,
121463 int iPos,
121464 Fts3Hash *pHash, /* Pending terms hash table to add entry to */
121465 const char *zToken,
121466 int nToken
121467 ){
121468 PendingList *pList;
121469 int rc = SQLITE_OK;
121470
121471 pList = (PendingList *)fts3HashFind(pHash, zToken, nToken);
121472 if( pList ){
121473 p->nPendingData -= (pList->nData + nToken + sizeof(Fts3HashElem));
121474 }
121475 if( fts3PendingListAppend(&pList, p->iPrevDocid, iCol, iPos, &rc) ){
121476 if( pList==fts3HashInsert(pHash, zToken, nToken, pList) ){
121477 /* Malloc failed while inserting the new entry. This can only
121478 ** happen if there was no previous entry for this token.
121479 */
121480 assert( 0==fts3HashFind(pHash, zToken, nToken) );
121481 sqlite3_free(pList);
121482 rc = SQLITE_NOMEM;
121483 }
121484 }
121485 if( rc==SQLITE_OK ){
121486 p->nPendingData += (pList->nData + nToken + sizeof(Fts3HashElem));
121487 }
121488 return rc;
121489 }
121490
121491 /*
121492 ** Tokenize the nul-terminated string zText and add all tokens to the
121493 ** pending-terms hash-table. The docid used is that currently stored in
121494 ** p->iPrevDocid, and the column is specified by argument iCol.
@@ -119335,12 +121534,11 @@
121534
121535 xNext = pModule->xNext;
121536 while( SQLITE_OK==rc
121537 && SQLITE_OK==(rc = xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos))
121538 ){
121539 int i;
 
121540 if( iPos>=nWord ) nWord = iPos+1;
121541
121542 /* Positions cannot be negative; we use -1 as a terminator internally.
121543 ** Tokens must have a non-zero length.
121544 */
@@ -119347,26 +121545,23 @@
121545 if( iPos<0 || !zToken || nToken<=0 ){
121546 rc = SQLITE_ERROR;
121547 break;
121548 }
121549
121550 /* Add the term to the terms index */
121551 rc = fts3PendingTermsAddOne(
121552 p, iCol, iPos, &p->aIndex[0].hPending, zToken, nToken
121553 );
121554
121555 /* Add the term to each of the prefix indexes that it is not too
121556 ** short for. */
121557 for(i=1; rc==SQLITE_OK && i<p->nIndex; i++){
121558 struct Fts3Index *pIndex = &p->aIndex[i];
121559 if( nToken<pIndex->nPrefix ) continue;
121560 rc = fts3PendingTermsAddOne(
121561 p, iCol, iPos, &pIndex->hPending, zToken, pIndex->nPrefix
121562 );
 
 
 
121563 }
121564 }
121565
121566 pModule->xClose(pCsr);
121567 *pnWord = nWord;
@@ -119392,18 +121587,23 @@
121587 p->iPrevDocid = iDocid;
121588 return SQLITE_OK;
121589 }
121590
121591 /*
121592 ** Discard the contents of the pending-terms hash tables.
121593 */
121594 SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *p){
121595 int i;
121596 for(i=0; i<p->nIndex; i++){
121597 Fts3HashElem *pElem;
121598 Fts3Hash *pHash = &p->aIndex[i].hPending;
121599 for(pElem=fts3HashFirst(pHash); pElem; pElem=fts3HashNext(pElem)){
121600 PendingList *pList = (PendingList *)fts3HashData(pElem);
121601 fts3PendingListDelete(pList);
121602 }
121603 fts3HashClear(pHash);
121604 }
 
121605 p->nPendingData = 0;
121606 }
121607
121608 /*
121609 ** This function is called by the xUpdate() method as part of an INSERT
@@ -119555,11 +121755,11 @@
121755
121756 /*
121757 ** Forward declaration to account for the circular dependency between
121758 ** functions fts3SegmentMerge() and fts3AllocateSegdirIdx().
121759 */
121760 static int fts3SegmentMerge(Fts3Table *, int, int);
121761
121762 /*
121763 ** This function allocates a new level iLevel index in the segdir table.
121764 ** Usually, indexes are allocated within a level sequentially starting
121765 ** with 0, so the allocated index is one greater than the value returned
@@ -119572,19 +121772,24 @@
121772 ** allocated index is 0.
121773 **
121774 ** If successful, *piIdx is set to the allocated index slot and SQLITE_OK
121775 ** returned. Otherwise, an SQLite error code is returned.
121776 */
121777 static int fts3AllocateSegdirIdx(
121778 Fts3Table *p,
121779 int iIndex, /* Index for p->aIndex */
121780 int iLevel,
121781 int *piIdx
121782 ){
121783 int rc; /* Return Code */
121784 sqlite3_stmt *pNextIdx; /* Query for next idx at level iLevel */
121785 int iNext = 0; /* Result of query pNextIdx */
121786
121787 /* Set variable iNext to the next available segdir index at level iLevel. */
121788 rc = fts3SqlStmt(p, SQL_NEXT_SEGMENT_INDEX, &pNextIdx, 0);
121789 if( rc==SQLITE_OK ){
121790 sqlite3_bind_int(pNextIdx, 1, iIndex*FTS3_SEGDIR_MAXLEVEL + iLevel);
121791 if( SQLITE_ROW==sqlite3_step(pNextIdx) ){
121792 iNext = sqlite3_column_int(pNextIdx, 0);
121793 }
121794 rc = sqlite3_reset(pNextIdx);
121795 }
@@ -119594,11 +121799,11 @@
121799 ** full, merge all segments in level iLevel into a single iLevel+1
121800 ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise,
121801 ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext.
121802 */
121803 if( iNext>=FTS3_MERGE_COUNT ){
121804 rc = fts3SegmentMerge(p, iIndex, iLevel);
121805 *piIdx = 0;
121806 }else{
121807 *piIdx = iNext;
121808 }
121809 }
@@ -119635,11 +121840,12 @@
121840 */
121841 SQLITE_PRIVATE int sqlite3Fts3ReadBlock(
121842 Fts3Table *p, /* FTS3 table handle */
121843 sqlite3_int64 iBlockid, /* Access the row with blockid=$iBlockid */
121844 char **paBlob, /* OUT: Blob data in malloc'd buffer */
121845 int *pnBlob, /* OUT: Size of blob data */
121846 int *pnLoad /* OUT: Bytes actually loaded */
121847 ){
121848 int rc; /* Return code */
121849
121850 /* pnBlob must be non-NULL. paBlob may be NULL or non-NULL. */
121851 assert( pnBlob);
@@ -119656,25 +121862,29 @@
121862 );
121863 }
121864
121865 if( rc==SQLITE_OK ){
121866 int nByte = sqlite3_blob_bytes(p->pSegments);
121867 *pnBlob = nByte;
121868 if( paBlob ){
121869 char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING);
121870 if( !aByte ){
121871 rc = SQLITE_NOMEM;
121872 }else{
121873 if( pnLoad && nByte>(FTS3_NODE_CHUNK_THRESHOLD) ){
121874 nByte = FTS3_NODE_CHUNKSIZE;
121875 *pnLoad = nByte;
121876 }
121877 rc = sqlite3_blob_read(p->pSegments, aByte, nByte, 0);
121878 memset(&aByte[nByte], 0, FTS3_NODE_PADDING);
121879 if( rc!=SQLITE_OK ){
121880 sqlite3_free(aByte);
121881 aByte = 0;
121882 }
121883 }
121884 *paBlob = aByte;
121885 }
 
121886 }
121887
121888 return rc;
121889 }
121890
@@ -119684,17 +121894,59 @@
121894 */
121895 SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *p){
121896 sqlite3_blob_close(p->pSegments);
121897 p->pSegments = 0;
121898 }
121899
121900 static int fts3SegReaderIncrRead(Fts3SegReader *pReader){
121901 int nRead; /* Number of bytes to read */
121902 int rc; /* Return code */
121903
121904 nRead = MIN(pReader->nNode - pReader->nPopulate, FTS3_NODE_CHUNKSIZE);
121905 rc = sqlite3_blob_read(
121906 pReader->pBlob,
121907 &pReader->aNode[pReader->nPopulate],
121908 nRead,
121909 pReader->nPopulate
121910 );
121911
121912 if( rc==SQLITE_OK ){
121913 pReader->nPopulate += nRead;
121914 memset(&pReader->aNode[pReader->nPopulate], 0, FTS3_NODE_PADDING);
121915 if( pReader->nPopulate==pReader->nNode ){
121916 sqlite3_blob_close(pReader->pBlob);
121917 pReader->pBlob = 0;
121918 pReader->nPopulate = 0;
121919 }
121920 }
121921 return rc;
121922 }
121923
121924 static int fts3SegReaderRequire(Fts3SegReader *pReader, char *pFrom, int nByte){
121925 int rc = SQLITE_OK;
121926 assert( !pReader->pBlob
121927 || (pFrom>=pReader->aNode && pFrom<&pReader->aNode[pReader->nNode])
121928 );
121929 while( pReader->pBlob && rc==SQLITE_OK
121930 && (pFrom - pReader->aNode + nByte)>pReader->nPopulate
121931 ){
121932 rc = fts3SegReaderIncrRead(pReader);
121933 }
121934 return rc;
121935 }
121936
121937 /*
121938 ** Move the iterator passed as the first argument to the next term in the
121939 ** segment. If successful, SQLITE_OK is returned. If there is no next term,
121940 ** SQLITE_DONE. Otherwise, an SQLite error code.
121941 */
121942 static int fts3SegReaderNext(
121943 Fts3Table *p,
121944 Fts3SegReader *pReader,
121945 int bIncr
121946 ){
121947 int rc; /* Return code of various sub-routines */
121948 char *pNext; /* Cursor variable */
121949 int nPrefix; /* Number of bytes in term prefix */
121950 int nSuffix; /* Number of bytes in term suffix */
121951
121952 if( !pReader->aDoclist ){
@@ -119702,11 +121954,10 @@
121954 }else{
121955 pNext = &pReader->aDoclist[pReader->nDoclist];
121956 }
121957
121958 if( !pNext || pNext>=&pReader->aNode[pReader->nNode] ){
 
121959
121960 if( fts3SegReaderIsPending(pReader) ){
121961 Fts3HashElem *pElem = *(pReader->ppNextElem);
121962 if( pElem==0 ){
121963 pReader->aNode = 0;
@@ -119722,10 +121973,12 @@
121973 return SQLITE_OK;
121974 }
121975
121976 if( !fts3SegReaderIsRootOnly(pReader) ){
121977 sqlite3_free(pReader->aNode);
121978 sqlite3_blob_close(pReader->pBlob);
121979 pReader->pBlob = 0;
121980 }
121981 pReader->aNode = 0;
121982
121983 /* If iCurrentBlock>=iLeafEndBlock, this is an EOF condition. All leaf
121984 ** blocks have already been traversed. */
@@ -119733,19 +121986,29 @@
121986 if( pReader->iCurrentBlock>=pReader->iLeafEndBlock ){
121987 return SQLITE_OK;
121988 }
121989
121990 rc = sqlite3Fts3ReadBlock(
121991 p, ++pReader->iCurrentBlock, &pReader->aNode, &pReader->nNode,
121992 (bIncr ? &pReader->nPopulate : 0)
121993 );
121994 if( rc!=SQLITE_OK ) return rc;
121995 assert( pReader->pBlob==0 );
121996 if( bIncr && pReader->nPopulate<pReader->nNode ){
121997 pReader->pBlob = p->pSegments;
121998 p->pSegments = 0;
121999 }
122000 pNext = pReader->aNode;
122001 }
122002
122003 assert( !fts3SegReaderIsPending(pReader) );
122004
122005 rc = fts3SegReaderRequire(pReader, pNext, FTS3_VARINT_MAX*2);
122006 if( rc!=SQLITE_OK ) return rc;
122007
122008 /* Because of the FTS3_NODE_PADDING bytes of padding, the following is
122009 ** safe (no risk of overread) even if the node data is corrupted. */
 
122010 pNext += sqlite3Fts3GetVarint32(pNext, &nPrefix);
122011 pNext += sqlite3Fts3GetVarint32(pNext, &nSuffix);
122012 if( nPrefix<0 || nSuffix<=0
122013 || &pNext[nSuffix]>&pReader->aNode[pReader->nNode]
122014 ){
@@ -119759,10 +122022,14 @@
122022 return SQLITE_NOMEM;
122023 }
122024 pReader->zTerm = zNew;
122025 pReader->nTermAlloc = nNew;
122026 }
122027
122028 rc = fts3SegReaderRequire(pReader, pNext, nSuffix+FTS3_VARINT_MAX);
122029 if( rc!=SQLITE_OK ) return rc;
122030
122031 memcpy(&pReader->zTerm[nPrefix], pNext, nSuffix);
122032 pReader->nTerm = nPrefix+nSuffix;
122033 pNext += nSuffix;
122034 pNext += sqlite3Fts3GetVarint32(pNext, &pReader->nDoclist);
122035 pReader->aDoclist = pNext;
@@ -119771,11 +122038,11 @@
122038 /* Check that the doclist does not appear to extend past the end of the
122039 ** b-tree node. And that the final byte of the doclist is 0x00. If either
122040 ** of these statements is untrue, then the data structure is corrupt.
122041 */
122042 if( &pReader->aDoclist[pReader->nDoclist]>&pReader->aNode[pReader->nNode]
122043 || (pReader->nPopulate==0 && pReader->aDoclist[pReader->nDoclist-1])
122044 ){
122045 return SQLITE_CORRUPT_VTAB;
122046 }
122047 return SQLITE_OK;
122048 }
@@ -119782,16 +122049,30 @@
122049
122050 /*
122051 ** Set the SegReader to point to the first docid in the doclist associated
122052 ** with the current term.
122053 */
122054 static int fts3SegReaderFirstDocid(Fts3Table *pTab, Fts3SegReader *pReader){
122055 int rc = SQLITE_OK;
122056 assert( pReader->aDoclist );
122057 assert( !pReader->pOffsetList );
122058 if( pTab->bDescIdx && fts3SegReaderIsPending(pReader) ){
122059 u8 bEof = 0;
122060 pReader->iDocid = 0;
122061 pReader->nOffsetList = 0;
122062 sqlite3Fts3DoclistPrev(0,
122063 pReader->aDoclist, pReader->nDoclist, &pReader->pOffsetList,
122064 &pReader->iDocid, &pReader->nOffsetList, &bEof
122065 );
122066 }else{
122067 rc = fts3SegReaderRequire(pReader, pReader->aDoclist, FTS3_VARINT_MAX);
122068 if( rc==SQLITE_OK ){
122069 int n = sqlite3Fts3GetVarint(pReader->aDoclist, &pReader->iDocid);
122070 pReader->pOffsetList = &pReader->aDoclist[n];
122071 }
122072 }
122073 return rc;
122074 }
122075
122076 /*
122077 ** Advance the SegReader to point to the next docid in the doclist
122078 ** associated with the current term.
@@ -119800,132 +122081,129 @@
122081 ** *ppOffsetList is set to point to the first column-offset list
122082 ** in the doclist entry (i.e. immediately past the docid varint).
122083 ** *pnOffsetList is set to the length of the set of column-offset
122084 ** lists, not including the nul-terminator byte. For example:
122085 */
122086 static int fts3SegReaderNextDocid(
122087 Fts3Table *pTab,
122088 Fts3SegReader *pReader, /* Reader to advance to next docid */
122089 char **ppOffsetList, /* OUT: Pointer to current position-list */
122090 int *pnOffsetList /* OUT: Length of *ppOffsetList in bytes */
122091 ){
122092 int rc = SQLITE_OK;
122093 char *p = pReader->pOffsetList;
122094 char c = 0;
122095
122096 assert( p );
122097
122098 if( pTab->bDescIdx && fts3SegReaderIsPending(pReader) ){
122099 /* A pending-terms seg-reader for an FTS4 table that uses order=desc.
122100 ** Pending-terms doclists are always built up in ascending order, so
122101 ** we have to iterate through them backwards here. */
122102 u8 bEof = 0;
122103 if( ppOffsetList ){
122104 *ppOffsetList = pReader->pOffsetList;
122105 *pnOffsetList = pReader->nOffsetList - 1;
122106 }
122107 sqlite3Fts3DoclistPrev(0,
122108 pReader->aDoclist, pReader->nDoclist, &p, &pReader->iDocid,
122109 &pReader->nOffsetList, &bEof
122110 );
122111 if( bEof ){
122112 pReader->pOffsetList = 0;
122113 }else{
122114 pReader->pOffsetList = p;
122115 }
 
 
122116 }else{
122117 char *pEnd = &pReader->aDoclist[pReader->nDoclist];
122118
122119 /* Pointer p currently points at the first byte of an offset list. The
122120 ** following block advances it to point one byte past the end of
122121 ** the same offset list. */
122122 while( 1 ){
122123
122124 /* The following line of code (and the "p++" below the while() loop) is
122125 ** normally all that is required to move pointer p to the desired
122126 ** position. The exception is if this node is being loaded from disk
122127 ** incrementally and pointer "p" now points to the first byte passed
122128 ** the populated part of pReader->aNode[].
122129 */
122130 while( *p | c ) c = *p++ & 0x80;
122131 assert( *p==0 );
122132
122133 if( pReader->pBlob==0 || p<&pReader->aNode[pReader->nPopulate] ) break;
122134 rc = fts3SegReaderIncrRead(pReader);
122135 if( rc!=SQLITE_OK ) return rc;
122136 }
122137 p++;
122138
122139 /* If required, populate the output variables with a pointer to and the
122140 ** size of the previous offset-list.
122141 */
122142 if( ppOffsetList ){
122143 *ppOffsetList = pReader->pOffsetList;
122144 *pnOffsetList = (int)(p - pReader->pOffsetList - 1);
122145 }
122146
122147 while( p<pEnd && *p==0 ) p++;
122148
122149 /* If there are no more entries in the doclist, set pOffsetList to
122150 ** NULL. Otherwise, set Fts3SegReader.iDocid to the next docid and
122151 ** Fts3SegReader.pOffsetList to point to the next offset list before
122152 ** returning.
122153 */
122154 if( p>=pEnd ){
122155 pReader->pOffsetList = 0;
122156 }else{
122157 rc = fts3SegReaderRequire(pReader, p, FTS3_VARINT_MAX);
122158 if( rc==SQLITE_OK ){
122159 sqlite3_int64 iDelta;
122160 pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta);
122161 if( pTab->bDescIdx ){
122162 pReader->iDocid -= iDelta;
122163 }else{
122164 pReader->iDocid += iDelta;
122165 }
122166 }
122167 }
122168 }
122169
122170 return SQLITE_OK;
122171 }
122172
122173
122174 SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(
122175 Fts3Cursor *pCsr,
122176 Fts3MultiSegReader *pMsr,
122177 int *pnOvfl
122178 ){
122179 Fts3Table *p = (Fts3Table*)pCsr->base.pVtab;
122180 int nOvfl = 0;
122181 int ii;
122182 int rc = SQLITE_OK;
122183 int pgsz = p->nPgsz;
122184
122185 assert( p->bHasStat );
122186 assert( pgsz>0 );
122187
122188 for(ii=0; rc==SQLITE_OK && ii<pMsr->nSegment; ii++){
122189 Fts3SegReader *pReader = pMsr->apSegment[ii];
122190 if( !fts3SegReaderIsPending(pReader)
122191 && !fts3SegReaderIsRootOnly(pReader)
122192 ){
122193 sqlite3_int64 jj;
122194 for(jj=pReader->iStartBlock; jj<=pReader->iLeafEndBlock; jj++){
122195 int nBlob;
122196 rc = sqlite3Fts3ReadBlock(p, jj, 0, &nBlob, 0);
122197 if( rc!=SQLITE_OK ) break;
122198 if( (nBlob+35)>pgsz ){
122199 nOvfl += (nBlob + 34)/pgsz;
122200 }
122201 }
122202 }
122203 }
122204 *pnOvfl = nOvfl;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122205 return rc;
122206 }
122207
122208 /*
122209 ** Free all allocations associated with the iterator passed as the
@@ -119934,10 +122212,11 @@
122212 SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){
122213 if( pReader && !fts3SegReaderIsPending(pReader) ){
122214 sqlite3_free(pReader->zTerm);
122215 if( !fts3SegReaderIsRootOnly(pReader) ){
122216 sqlite3_free(pReader->aNode);
122217 sqlite3_blob_close(pReader->pBlob);
122218 }
122219 }
122220 sqlite3_free(pReader);
122221 }
122222
@@ -120010,28 +122289,46 @@
122289 }
122290
122291 /*
122292 ** This function is used to allocate an Fts3SegReader that iterates through
122293 ** a subset of the terms stored in the Fts3Table.pendingTerms array.
122294 **
122295 ** If the isPrefixIter parameter is zero, then the returned SegReader iterates
122296 ** through each term in the pending-terms table. Or, if isPrefixIter is
122297 ** non-zero, it iterates through each term and its prefixes. For example, if
122298 ** the pending terms hash table contains the terms "sqlite", "mysql" and
122299 ** "firebird", then the iterator visits the following 'terms' (in the order
122300 ** shown):
122301 **
122302 ** f fi fir fire fireb firebi firebir firebird
122303 ** m my mys mysq mysql
122304 ** s sq sql sqli sqlit sqlite
122305 **
122306 ** Whereas if isPrefixIter is zero, the terms visited are:
122307 **
122308 ** firebird mysql sqlite
122309 */
122310 SQLITE_PRIVATE int sqlite3Fts3SegReaderPending(
122311 Fts3Table *p, /* Virtual table handle */
122312 int iIndex, /* Index for p->aIndex */
122313 const char *zTerm, /* Term to search for */
122314 int nTerm, /* Size of buffer zTerm */
122315 int bPrefix, /* True for a prefix iterator */
122316 Fts3SegReader **ppReader /* OUT: SegReader for pending-terms */
122317 ){
122318 Fts3SegReader *pReader = 0; /* Fts3SegReader object to return */
122319 Fts3HashElem **aElem = 0; /* Array of term hash entries to scan */
122320 int nElem = 0; /* Size of array at aElem */
122321 int rc = SQLITE_OK; /* Return Code */
122322 Fts3Hash *pHash;
122323
122324 pHash = &p->aIndex[iIndex].hPending;
122325 if( bPrefix ){
122326 int nAlloc = 0; /* Size of allocated array at aElem */
122327 Fts3HashElem *pE = 0; /* Iterator variable */
122328
122329 for(pE=fts3HashFirst(pHash); pE; pE=fts3HashNext(pE)){
122330 char *zKey = (char *)fts3HashKey(pE);
122331 int nKey = fts3HashKeysize(pE);
122332 if( nTerm==0 || (nKey>=nTerm && 0==memcmp(zKey, zTerm, nTerm)) ){
122333 if( nElem==nAlloc ){
122334 Fts3HashElem **aElem2;
@@ -120044,10 +122341,11 @@
122341 nElem = 0;
122342 break;
122343 }
122344 aElem = aElem2;
122345 }
122346
122347 aElem[nElem++] = pE;
122348 }
122349 }
122350
122351 /* If more than one term matches the prefix, sort the Fts3HashElem
@@ -120057,11 +122355,13 @@
122355 if( nElem>1 ){
122356 qsort(aElem, nElem, sizeof(Fts3HashElem *), fts3CompareElemByTerm);
122357 }
122358
122359 }else{
122360 /* The query is a simple term lookup that matches at most one term in
122361 ** the index. All that is required is a straight hash-lookup. */
122362 Fts3HashElem *pE = fts3HashFindElem(pHash, zTerm, nTerm);
122363 if( pE ){
122364 aElem = &pE;
122365 nElem = 1;
122366 }
122367 }
@@ -120077,11 +122377,11 @@
122377 pReader->ppNextElem = (Fts3HashElem **)&pReader[1];
122378 memcpy(pReader->ppNextElem, aElem, nElem*sizeof(Fts3HashElem *));
122379 }
122380 }
122381
122382 if( bPrefix ){
122383 sqlite3_free(aElem);
122384 }
122385 *ppReader = pReader;
122386 return rc;
122387 }
@@ -120137,10 +122437,22 @@
122437 if( pLhs->iDocid==pRhs->iDocid ){
122438 rc = pRhs->iIdx - pLhs->iIdx;
122439 }else{
122440 rc = (pLhs->iDocid > pRhs->iDocid) ? 1 : -1;
122441 }
122442 }
122443 assert( pLhs->aNode && pRhs->aNode );
122444 return rc;
122445 }
122446 static int fts3SegReaderDoclistCmpRev(Fts3SegReader *pLhs, Fts3SegReader *pRhs){
122447 int rc = (pLhs->pOffsetList==0)-(pRhs->pOffsetList==0);
122448 if( rc==0 ){
122449 if( pLhs->iDocid==pRhs->iDocid ){
122450 rc = pRhs->iIdx - pLhs->iIdx;
122451 }else{
122452 rc = (pLhs->iDocid < pRhs->iDocid) ? 1 : -1;
122453 }
122454 }
122455 assert( pLhs->aNode && pRhs->aNode );
122456 return rc;
122457 }
122458
@@ -120689,25 +123001,34 @@
123001 }
123002 return rc;
123003 }
123004
123005 /*
123006 ** Set *pnMax to the largest segment level in the database for the index
123007 ** iIndex.
123008 **
123009 ** Segment levels are stored in the 'level' column of the %_segdir table.
123010 **
123011 ** Return SQLITE_OK if successful, or an SQLite error code if not.
123012 */
123013 static int fts3SegmentMaxLevel(Fts3Table *p, int iIndex, int *pnMax){
123014 sqlite3_stmt *pStmt;
123015 int rc;
123016 assert( iIndex>=0 && iIndex<p->nIndex );
123017
123018 /* Set pStmt to the compiled version of:
123019 **
123020 ** SELECT max(level) FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?
123021 **
123022 ** (1024 is actually the value of macro FTS3_SEGDIR_PREFIXLEVEL_STR).
123023 */
123024 rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_MAX_LEVEL, &pStmt, 0);
123025 if( rc!=SQLITE_OK ) return rc;
123026 sqlite3_bind_int(pStmt, 1, iIndex*FTS3_SEGDIR_MAXLEVEL);
123027 sqlite3_bind_int(pStmt, 2, (iIndex+1)*FTS3_SEGDIR_MAXLEVEL - 1);
123028 if( SQLITE_ROW==sqlite3_step(pStmt) ){
123029 *pnMax = sqlite3_column_int(pStmt, 0);
 
123030 }
123031 return sqlite3_reset(pStmt);
123032 }
123033
123034 /*
@@ -120724,10 +123045,11 @@
123045 **
123046 ** SQLITE_OK is returned if successful, otherwise an SQLite error code.
123047 */
123048 static int fts3DeleteSegdir(
123049 Fts3Table *p, /* Virtual table handle */
123050 int iIndex, /* Index for p->aIndex */
123051 int iLevel, /* Level of %_segdir entries to delete */
123052 Fts3SegReader **apSegment, /* Array of SegReader objects */
123053 int nReader /* Size of array apSegment */
123054 ){
123055 int rc; /* Return Code */
@@ -120746,23 +123068,28 @@
123068 }
123069 if( rc!=SQLITE_OK ){
123070 return rc;
123071 }
123072
123073 assert( iLevel>=0 || iLevel==FTS3_SEGCURSOR_ALL );
123074 if( iLevel==FTS3_SEGCURSOR_ALL ){
123075 rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_RANGE, &pDelete, 0);
123076 if( rc==SQLITE_OK ){
123077 sqlite3_bind_int(pDelete, 1, iIndex*FTS3_SEGDIR_MAXLEVEL);
123078 sqlite3_bind_int(pDelete, 2, (iIndex+1) * FTS3_SEGDIR_MAXLEVEL - 1);
123079 }
123080 }else{
123081 rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_LEVEL, &pDelete, 0);
 
123082 if( rc==SQLITE_OK ){
123083 sqlite3_bind_int(pDelete, 1, iIndex*FTS3_SEGDIR_MAXLEVEL + iLevel);
 
 
123084 }
123085 }
123086
123087 if( rc==SQLITE_OK ){
123088 sqlite3_step(pDelete);
123089 rc = sqlite3_reset(pDelete);
123090 }
123091
123092 return rc;
123093 }
123094
123095 /*
@@ -120806,44 +123133,216 @@
123133
123134 *ppList = pList;
123135 *pnList = nList;
123136 }
123137
123138 /*
123139 ** Cache data in the Fts3MultiSegReader.aBuffer[] buffer (overwriting any
123140 ** existing data). Grow the buffer if required.
123141 **
123142 ** If successful, return SQLITE_OK. Otherwise, if an OOM error is encountered
123143 ** trying to resize the buffer, return SQLITE_NOMEM.
123144 */
123145 static int fts3MsrBufferData(
123146 Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */
123147 char *pList,
123148 int nList
123149 ){
123150 if( nList>pMsr->nBuffer ){
123151 char *pNew;
123152 pMsr->nBuffer = nList*2;
123153 pNew = (char *)sqlite3_realloc(pMsr->aBuffer, pMsr->nBuffer);
123154 if( !pNew ) return SQLITE_NOMEM;
123155 pMsr->aBuffer = pNew;
123156 }
123157
123158 memcpy(pMsr->aBuffer, pList, nList);
123159 return SQLITE_OK;
123160 }
123161
123162 SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext(
123163 Fts3Table *p, /* Virtual table handle */
123164 Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */
123165 sqlite3_int64 *piDocid, /* OUT: Docid value */
123166 char **paPoslist, /* OUT: Pointer to position list */
123167 int *pnPoslist /* OUT: Size of position list in bytes */
123168 ){
123169 int nMerge = pMsr->nAdvance;
123170 Fts3SegReader **apSegment = pMsr->apSegment;
123171 int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = (
123172 p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp
123173 );
123174
123175 if( nMerge==0 ){
123176 *paPoslist = 0;
123177 return SQLITE_OK;
123178 }
123179
123180 while( 1 ){
123181 Fts3SegReader *pSeg;
123182 pSeg = pMsr->apSegment[0];
123183
123184 if( pSeg->pOffsetList==0 ){
123185 *paPoslist = 0;
123186 break;
123187 }else{
123188 int rc;
123189 char *pList;
123190 int nList;
123191 int j;
123192 sqlite3_int64 iDocid = apSegment[0]->iDocid;
123193
123194 rc = fts3SegReaderNextDocid(p, apSegment[0], &pList, &nList);
123195 j = 1;
123196 while( rc==SQLITE_OK
123197 && j<nMerge
123198 && apSegment[j]->pOffsetList
123199 && apSegment[j]->iDocid==iDocid
123200 ){
123201 rc = fts3SegReaderNextDocid(p, apSegment[j], 0, 0);
123202 j++;
123203 }
123204 if( rc!=SQLITE_OK ) return rc;
123205 fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp);
123206
123207 if( pMsr->iColFilter>=0 ){
123208 fts3ColumnFilter(pMsr->iColFilter, &pList, &nList);
123209 }
123210
123211 if( nList>0 ){
123212 if( fts3SegReaderIsPending(apSegment[0]) ){
123213 rc = fts3MsrBufferData(pMsr, pList, nList+1);
123214 if( rc!=SQLITE_OK ) return rc;
123215 *paPoslist = pMsr->aBuffer;
123216 assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 );
123217 }else{
123218 *paPoslist = pList;
123219 }
123220 *piDocid = iDocid;
123221 *pnPoslist = nList;
123222 break;
123223 }
123224 }
123225 }
123226
123227 return SQLITE_OK;
123228 }
123229
123230 static int fts3SegReaderStart(
123231 Fts3Table *p, /* Virtual table handle */
123232 Fts3MultiSegReader *pCsr, /* Cursor object */
123233 const char *zTerm, /* Term searched for (or NULL) */
123234 int nTerm /* Length of zTerm in bytes */
123235 ){
123236 int i;
123237 int nSeg = pCsr->nSegment;
 
 
123238
123239 /* If the Fts3SegFilter defines a specific term (or term prefix) to search
123240 ** for, then advance each segment iterator until it points to a term of
123241 ** equal or greater value than the specified term. This prevents many
123242 ** unnecessary merge/sort operations for the case where single segment
123243 ** b-tree leaf nodes contain more than one term.
123244 */
123245 for(i=0; pCsr->bRestart==0 && i<pCsr->nSegment; i++){
 
 
123246 Fts3SegReader *pSeg = pCsr->apSegment[i];
123247 do {
123248 int rc = fts3SegReaderNext(p, pSeg, 0);
123249 if( rc!=SQLITE_OK ) return rc;
123250 }while( zTerm && fts3SegReaderTermCmp(pSeg, zTerm, nTerm)<0 );
123251 }
123252 fts3SegReaderSort(pCsr->apSegment, nSeg, nSeg, fts3SegReaderCmp);
123253
123254 return SQLITE_OK;
123255 }
123256
123257 SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(
123258 Fts3Table *p, /* Virtual table handle */
123259 Fts3MultiSegReader *pCsr, /* Cursor object */
123260 Fts3SegFilter *pFilter /* Restrictions on range of iteration */
123261 ){
123262 pCsr->pFilter = pFilter;
123263 return fts3SegReaderStart(p, pCsr, pFilter->zTerm, pFilter->nTerm);
123264 }
123265
123266 SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart(
123267 Fts3Table *p, /* Virtual table handle */
123268 Fts3MultiSegReader *pCsr, /* Cursor object */
123269 int iCol, /* Column to match on. */
123270 const char *zTerm, /* Term to iterate through a doclist for */
123271 int nTerm /* Number of bytes in zTerm */
123272 ){
123273 int i;
123274 int rc;
123275 int nSegment = pCsr->nSegment;
123276 int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = (
123277 p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp
123278 );
123279
123280 assert( pCsr->pFilter==0 );
123281 assert( zTerm && nTerm>0 );
123282
123283 /* Advance each segment iterator until it points to the term zTerm/nTerm. */
123284 rc = fts3SegReaderStart(p, pCsr, zTerm, nTerm);
123285 if( rc!=SQLITE_OK ) return rc;
123286
123287 /* Determine how many of the segments actually point to zTerm/nTerm. */
123288 for(i=0; i<nSegment; i++){
123289 Fts3SegReader *pSeg = pCsr->apSegment[i];
123290 if( !pSeg->aNode || fts3SegReaderTermCmp(pSeg, zTerm, nTerm) ){
123291 break;
123292 }
123293 }
123294 pCsr->nAdvance = i;
123295
123296 /* Advance each of the segments to point to the first docid. */
123297 for(i=0; i<pCsr->nAdvance; i++){
123298 rc = fts3SegReaderFirstDocid(p, pCsr->apSegment[i]);
123299 if( rc!=SQLITE_OK ) return rc;
123300 }
123301 fts3SegReaderSort(pCsr->apSegment, i, i, xCmp);
123302
123303 assert( iCol<0 || iCol<p->nColumn );
123304 pCsr->iColFilter = iCol;
123305
123306 return SQLITE_OK;
123307 }
123308
123309 /*
123310 ** This function is called on a MultiSegReader that has been started using
123311 ** sqlite3Fts3MsrIncrStart(). One or more calls to MsrIncrNext() may also
123312 ** have been made. Calling this function puts the MultiSegReader in such
123313 ** a state that if the next two calls are:
123314 **
123315 ** sqlite3Fts3SegReaderStart()
123316 ** sqlite3Fts3SegReaderStep()
123317 **
123318 ** then the entire doclist for the term is available in
123319 ** MultiSegReader.aDoclist/nDoclist.
123320 */
123321 SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr){
123322 int i; /* Used to iterate through segment-readers */
123323
123324 assert( pCsr->zTerm==0 );
123325 assert( pCsr->nTerm==0 );
123326 assert( pCsr->aDoclist==0 );
123327 assert( pCsr->nDoclist==0 );
123328
123329 pCsr->nAdvance = 0;
123330 pCsr->bRestart = 1;
123331 for(i=0; i<pCsr->nSegment; i++){
123332 pCsr->apSegment[i]->pOffsetList = 0;
123333 pCsr->apSegment[i]->nOffsetList = 0;
123334 pCsr->apSegment[i]->iDocid = 0;
123335 }
123336
123337 return SQLITE_OK;
123338 }
123339
123340
123341 SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(
123342 Fts3Table *p, /* Virtual table handle */
123343 Fts3MultiSegReader *pCsr /* Cursor object */
123344 ){
123345 int rc = SQLITE_OK;
123346
123347 int isIgnoreEmpty = (pCsr->pFilter->flags & FTS3_SEGMENT_IGNORE_EMPTY);
123348 int isRequirePos = (pCsr->pFilter->flags & FTS3_SEGMENT_REQUIRE_POS);
@@ -120852,10 +123351,13 @@
123351 int isScan = (pCsr->pFilter->flags & FTS3_SEGMENT_SCAN);
123352
123353 Fts3SegReader **apSegment = pCsr->apSegment;
123354 int nSegment = pCsr->nSegment;
123355 Fts3SegFilter *pFilter = pCsr->pFilter;
123356 int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = (
123357 p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp
123358 );
123359
123360 if( pCsr->nSegment==0 ) return SQLITE_OK;
123361
123362 do {
123363 int nMerge;
@@ -120863,11 +123365,11 @@
123365
123366 /* Advance the first pCsr->nAdvance entries in the apSegment[] array
123367 ** forward. Then sort the list in order of current term again.
123368 */
123369 for(i=0; i<pCsr->nAdvance; i++){
123370 rc = fts3SegReaderNext(p, apSegment[i], 0);
123371 if( rc!=SQLITE_OK ) return rc;
123372 }
123373 fts3SegReaderSort(apSegment, nSegment, pCsr->nAdvance, fts3SegReaderCmp);
123374 pCsr->nAdvance = 0;
123375
@@ -120902,69 +123404,87 @@
123404 ){
123405 nMerge++;
123406 }
123407
123408 assert( isIgnoreEmpty || (isRequirePos && !isColFilter) );
123409 if( nMerge==1
123410 && !isIgnoreEmpty
123411 && (p->bDescIdx==0 || fts3SegReaderIsPending(apSegment[0])==0)
123412 ){
123413 pCsr->nDoclist = apSegment[0]->nDoclist;
123414 if( fts3SegReaderIsPending(apSegment[0]) ){
123415 rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, pCsr->nDoclist);
123416 pCsr->aDoclist = pCsr->aBuffer;
123417 }else{
123418 pCsr->aDoclist = apSegment[0]->aDoclist;
123419 }
123420 if( rc==SQLITE_OK ) rc = SQLITE_ROW;
123421 }else{
123422 int nDoclist = 0; /* Size of doclist */
123423 sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */
123424
123425 /* The current term of the first nMerge entries in the array
123426 ** of Fts3SegReader objects is the same. The doclists must be merged
123427 ** and a single term returned with the merged doclist.
123428 */
123429 for(i=0; i<nMerge; i++){
123430 fts3SegReaderFirstDocid(p, apSegment[i]);
123431 }
123432 fts3SegReaderSort(apSegment, nMerge, nMerge, xCmp);
123433 while( apSegment[0]->pOffsetList ){
123434 int j; /* Number of segments that share a docid */
123435 char *pList;
123436 int nList;
123437 int nByte;
123438 sqlite3_int64 iDocid = apSegment[0]->iDocid;
123439 fts3SegReaderNextDocid(p, apSegment[0], &pList, &nList);
123440 j = 1;
123441 while( j<nMerge
123442 && apSegment[j]->pOffsetList
123443 && apSegment[j]->iDocid==iDocid
123444 ){
123445 fts3SegReaderNextDocid(p, apSegment[j], 0, 0);
123446 j++;
123447 }
123448
123449 if( isColFilter ){
123450 fts3ColumnFilter(pFilter->iCol, &pList, &nList);
123451 }
123452
123453 if( !isIgnoreEmpty || nList>0 ){
123454
123455 /* Calculate the 'docid' delta value to write into the merged
123456 ** doclist. */
123457 sqlite3_int64 iDelta;
123458 if( p->bDescIdx && nDoclist>0 ){
123459 iDelta = iPrev - iDocid;
123460 }else{
123461 iDelta = iDocid - iPrev;
123462 }
123463 assert( iDelta>0 || (nDoclist==0 && iDelta==iDocid) );
123464 assert( nDoclist>0 || iDelta==iDocid );
123465
123466 nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0);
123467 if( nDoclist+nByte>pCsr->nBuffer ){
123468 char *aNew;
123469 pCsr->nBuffer = (nDoclist+nByte)*2;
123470 aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer);
123471 if( !aNew ){
123472 return SQLITE_NOMEM;
123473 }
123474 pCsr->aBuffer = aNew;
123475 }
123476 nDoclist += sqlite3Fts3PutVarint(&pCsr->aBuffer[nDoclist], iDelta);
 
 
123477 iPrev = iDocid;
123478 if( isRequirePos ){
123479 memcpy(&pCsr->aBuffer[nDoclist], pList, nList);
123480 nDoclist += nList;
123481 pCsr->aBuffer[nDoclist++] = '\0';
123482 }
123483 }
123484
123485 fts3SegReaderSort(apSegment, nMerge, j, xCmp);
123486 }
123487 if( nDoclist>0 ){
123488 pCsr->aDoclist = pCsr->aBuffer;
123489 pCsr->nDoclist = nDoclist;
123490 rc = SQLITE_ROW;
@@ -120973,13 +123493,14 @@
123493 pCsr->nAdvance = nMerge;
123494 }while( rc==SQLITE_OK );
123495
123496 return rc;
123497 }
123498
123499
123500 SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(
123501 Fts3MultiSegReader *pCsr /* Cursor object */
123502 ){
123503 if( pCsr ){
123504 int i;
123505 for(i=0; i<pCsr->nSegment; i++){
123506 sqlite3Fts3SegReaderFree(pCsr->apSegment[i]);
@@ -121002,47 +123523,60 @@
123523 ** If this function is called with iLevel<0, but there is only one
123524 ** segment in the database, SQLITE_DONE is returned immediately.
123525 ** Otherwise, if successful, SQLITE_OK is returned. If an error occurs,
123526 ** an SQLite error code is returned.
123527 */
123528 static int fts3SegmentMerge(Fts3Table *p, int iIndex, int iLevel){
123529 int rc; /* Return code */
123530 int iIdx = 0; /* Index of new segment */
123531 int iNewLevel = 0; /* Level/index to create new segment at */
123532 SegmentWriter *pWriter = 0; /* Used to write the new, merged, segment */
123533 Fts3SegFilter filter; /* Segment term filter condition */
123534 Fts3MultiSegReader csr; /* Cursor to iterate through level(s) */
123535 int bIgnoreEmpty = 0; /* True to ignore empty segments */
123536
123537 assert( iLevel==FTS3_SEGCURSOR_ALL
123538 || iLevel==FTS3_SEGCURSOR_PENDING
123539 || iLevel>=0
123540 );
123541 assert( iLevel<FTS3_SEGDIR_MAXLEVEL );
123542 assert( iIndex>=0 && iIndex<p->nIndex );
123543
123544 rc = sqlite3Fts3SegReaderCursor(p, iIndex, iLevel, 0, 0, 1, 0, &csr);
123545 if( rc!=SQLITE_OK || csr.nSegment==0 ) goto finished;
123546
123547 if( iLevel==FTS3_SEGCURSOR_ALL ){
123548 /* This call is to merge all segments in the database to a single
123549 ** segment. The level of the new segment is equal to the the numerically
123550 ** greatest segment level currently present in the database for this
123551 ** index. The idx of the new segment is always 0. */
 
123552 if( csr.nSegment==1 ){
123553 rc = SQLITE_DONE;
123554 goto finished;
123555 }
123556 rc = fts3SegmentMaxLevel(p, iIndex, &iNewLevel);
123557 bIgnoreEmpty = 1;
123558
123559 }else if( iLevel==FTS3_SEGCURSOR_PENDING ){
123560 iNewLevel = iIndex * FTS3_SEGDIR_MAXLEVEL;
123561 rc = fts3AllocateSegdirIdx(p, iIndex, 0, &iIdx);
123562 }else{
123563 /* This call is to merge all segments at level iLevel. find the next
123564 ** available segment index at level iLevel+1. The call to
123565 ** fts3AllocateSegdirIdx() will merge the segments at level iLevel+1 to
123566 ** a single iLevel+2 segment if necessary. */
123567 rc = fts3AllocateSegdirIdx(p, iIndex, iLevel+1, &iIdx);
123568 iNewLevel = iIndex * FTS3_SEGDIR_MAXLEVEL + iLevel+1;
123569 }
123570 if( rc!=SQLITE_OK ) goto finished;
123571 assert( csr.nSegment>0 );
123572 assert( iNewLevel>=(iIndex*FTS3_SEGDIR_MAXLEVEL) );
123573 assert( iNewLevel<((iIndex+1)*FTS3_SEGDIR_MAXLEVEL) );
123574
123575 memset(&filter, 0, sizeof(Fts3SegFilter));
123576 filter.flags = FTS3_SEGMENT_REQUIRE_POS;
123577 filter.flags |= (bIgnoreEmpty ? FTS3_SEGMENT_IGNORE_EMPTY : 0);
123578
123579 rc = sqlite3Fts3SegReaderStart(p, &csr, &filter);
123580 while( SQLITE_OK==rc ){
123581 rc = sqlite3Fts3SegReaderStep(p, &csr);
123582 if( rc!=SQLITE_ROW ) break;
@@ -121050,12 +123584,14 @@
123584 csr.zTerm, csr.nTerm, csr.aDoclist, csr.nDoclist);
123585 }
123586 if( rc!=SQLITE_OK ) goto finished;
123587 assert( pWriter );
123588
123589 if( iLevel!=FTS3_SEGCURSOR_PENDING ){
123590 rc = fts3DeleteSegdir(p, iIndex, iLevel, csr.apSegment, csr.nSegment);
123591 if( rc!=SQLITE_OK ) goto finished;
123592 }
123593 rc = fts3SegWriterFlush(p, pWriter, iNewLevel, iIdx);
123594
123595 finished:
123596 fts3SegWriterFree(pWriter);
123597 sqlite3Fts3SegReaderFinish(&csr);
@@ -121062,14 +123598,21 @@
123598 return rc;
123599 }
123600
123601
123602 /*
123603 ** Flush the contents of pendingTerms to level 0 segments.
123604 */
123605 SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *p){
123606 int rc = SQLITE_OK;
123607 int i;
123608 for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){
123609 rc = fts3SegmentMerge(p, i, FTS3_SEGCURSOR_PENDING);
123610 if( rc==SQLITE_DONE ) rc = SQLITE_OK;
123611 }
123612 sqlite3Fts3PendingTermsClear(p);
123613 return rc;
123614 }
123615
123616 /*
123617 ** Encode N integers as varints into a blob.
123618 */
@@ -121215,10 +123758,27 @@
123758 sqlite3_bind_blob(pStmt, 1, pBlob, nBlob, SQLITE_STATIC);
123759 sqlite3_step(pStmt);
123760 *pRC = sqlite3_reset(pStmt);
123761 sqlite3_free(a);
123762 }
123763
123764 static int fts3DoOptimize(Fts3Table *p, int bReturnDone){
123765 int i;
123766 int bSeenDone = 0;
123767 int rc = SQLITE_OK;
123768 for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){
123769 rc = fts3SegmentMerge(p, i, FTS3_SEGCURSOR_ALL);
123770 if( rc==SQLITE_DONE ){
123771 bSeenDone = 1;
123772 rc = SQLITE_OK;
123773 }
123774 }
123775 sqlite3Fts3SegmentsClose(p);
123776 sqlite3Fts3PendingTermsClear(p);
123777
123778 return (rc==SQLITE_OK && bReturnDone && bSeenDone) ? SQLITE_DONE : rc;
123779 }
123780
123781 /*
123782 ** Handle a 'special' INSERT of the form:
123783 **
123784 ** "INSERT INTO tbl(tbl) VALUES(<expr>)"
@@ -121232,16 +123792,11 @@
123792 int nVal = sqlite3_value_bytes(pVal);
123793
123794 if( !zVal ){
123795 return SQLITE_NOMEM;
123796 }else if( nVal==8 && 0==sqlite3_strnicmp(zVal, "optimize", 8) ){
123797 rc = fts3DoOptimize(p, 0);
 
 
 
 
 
123798 #ifdef SQLITE_TEST
123799 }else if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){
123800 p->nNodeSize = atoi(&zVal[9]);
123801 rc = SQLITE_OK;
123802 }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){
@@ -121250,61 +123805,23 @@
123805 #endif
123806 }else{
123807 rc = SQLITE_ERROR;
123808 }
123809
 
123810 return rc;
123811 }
123812
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
123813 /*
123814 ** Delete all cached deferred doclists. Deferred doclists are cached
123815 ** (allocated) by the sqlite3Fts3CacheDeferredDoclists() function.
123816 */
123817 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *pCsr){
123818 Fts3DeferredToken *pDef;
123819 for(pDef=pCsr->pDeferred; pDef; pDef=pDef->pNext){
123820 fts3PendingListDelete(pDef->pList);
123821 pDef->pList = 0;
123822 }
 
 
 
123823 }
123824
123825 /*
123826 ** Free all entries in the pCsr->pDeffered list. Entries are added to
123827 ** this list using sqlite3Fts3DeferToken().
@@ -121312,11 +123829,11 @@
123829 SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *pCsr){
123830 Fts3DeferredToken *pDef;
123831 Fts3DeferredToken *pNext;
123832 for(pDef=pCsr->pDeferred; pDef; pDef=pNext){
123833 pNext = pDef->pNext;
123834 fts3PendingListDelete(pDef->pList);
123835 sqlite3_free(pDef);
123836 }
123837 pCsr->pDeferred = 0;
123838 }
123839
@@ -121376,10 +123893,37 @@
123893 }
123894 }
123895
123896 return rc;
123897 }
123898
123899 SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(
123900 Fts3DeferredToken *p,
123901 char **ppData,
123902 int *pnData
123903 ){
123904 char *pRet;
123905 int nSkip;
123906 sqlite3_int64 dummy;
123907
123908 *ppData = 0;
123909 *pnData = 0;
123910
123911 if( p->pList==0 ){
123912 return SQLITE_OK;
123913 }
123914
123915 pRet = (char *)sqlite3_malloc(p->pList->nData);
123916 if( !pRet ) return SQLITE_NOMEM;
123917
123918 nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy);
123919 *pnData = p->pList->nData - nSkip;
123920 *ppData = pRet;
123921
123922 memcpy(pRet, &p->pList->aData[nSkip], *pnData);
123923 return SQLITE_OK;
123924 }
123925
123926 /*
123927 ** Add an entry for token pToken to the pCsr->pDeferred list.
123928 */
123929 SQLITE_PRIVATE int sqlite3Fts3DeferToken(
@@ -121451,11 +123995,11 @@
123995 ){
123996 Fts3Table *p = (Fts3Table *)pVtab;
123997 int rc = SQLITE_OK; /* Return Code */
123998 int isRemove = 0; /* True for an UPDATE or DELETE */
123999 sqlite3_int64 iRemove = 0; /* Rowid removed by UPDATE or DELETE */
124000 u32 *aSzIns = 0; /* Sizes of inserted documents */
124001 u32 *aSzDel; /* Sizes of deleted documents */
124002 int nChng = 0; /* Net change in number of documents */
124003 int bInsertDone = 0;
124004
124005 assert( p->pSegments==0 );
@@ -121466,16 +124010,20 @@
124010 */
124011 if( nArg>1
124012 && sqlite3_value_type(apVal[0])==SQLITE_NULL
124013 && sqlite3_value_type(apVal[p->nColumn+2])!=SQLITE_NULL
124014 ){
124015 rc = fts3SpecialInsert(p, apVal[p->nColumn+2]);
124016 goto update_out;
124017 }
124018
124019 /* Allocate space to hold the change in document sizes */
124020 aSzIns = sqlite3_malloc( sizeof(aSzIns[0])*(p->nColumn+1)*2 );
124021 if( aSzIns==0 ){
124022 rc = SQLITE_NOMEM;
124023 goto update_out;
124024 }
124025 aSzDel = &aSzIns[p->nColumn+1];
124026 memset(aSzIns, 0, sizeof(aSzIns[0])*(p->nColumn+1)*2);
124027
124028 /* If this is an INSERT operation, or an UPDATE that modifies the rowid
124029 ** value, then this operation requires constraint handling.
@@ -121521,12 +124069,11 @@
124069 bInsertDone = 1;
124070 }
124071 }
124072 }
124073 if( rc!=SQLITE_OK ){
124074 goto update_out;
 
124075 }
124076
124077 /* If this is a DELETE or UPDATE operation, remove the old record. */
124078 if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
124079 assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER );
@@ -121555,10 +124102,11 @@
124102
124103 if( p->bHasStat ){
124104 fts3UpdateDocTotals(&rc, p, aSzIns, aSzDel, nChng);
124105 }
124106
124107 update_out:
124108 sqlite3_free(aSzIns);
124109 sqlite3Fts3SegmentsClose(p);
124110 return rc;
124111 }
124112
@@ -121569,16 +124117,14 @@
124117 */
124118 SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *p){
124119 int rc;
124120 rc = sqlite3_exec(p->db, "SAVEPOINT fts3", 0, 0, 0);
124121 if( rc==SQLITE_OK ){
124122 rc = fts3DoOptimize(p, 1);
124123 if( rc==SQLITE_OK || rc==SQLITE_DONE ){
124124 int rc2 = sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
124125 if( rc2!=SQLITE_OK ) rc = rc2;
 
 
124126 }else{
124127 sqlite3_exec(p->db, "ROLLBACK TO fts3", 0, 0, 0);
124128 sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0);
124129 }
124130 }
@@ -121763,76 +124309,24 @@
124309 ){
124310 int iPhrase = 0; /* Variable used as the phrase counter */
124311 return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx);
124312 }
124313
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124314 /*
124315 ** This is an fts3ExprIterate() callback used while loading the doclists
124316 ** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
124317 ** fts3ExprLoadDoclists().
124318 */
124319 static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
124320 int rc = SQLITE_OK;
124321 Fts3Phrase *pPhrase = pExpr->pPhrase;
124322 LoadDoclistCtx *p = (LoadDoclistCtx *)ctx;
124323
124324 UNUSED_PARAMETER(iPhrase);
124325
124326 p->nPhrase++;
124327 p->nToken += pPhrase->nToken;
 
 
 
 
 
 
 
 
124328
124329 return rc;
124330 }
124331
124332 /*
@@ -122002,11 +124496,11 @@
124496 SnippetPhrase *pPhrase = &p->aPhrase[iPhrase];
124497 char *pCsr;
124498
124499 pPhrase->nToken = pExpr->pPhrase->nToken;
124500
124501 pCsr = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol);
124502 if( pCsr ){
124503 int iFirst = 0;
124504 pPhrase->pList = pCsr;
124505 fts3GetDeltaPosition(&pCsr, &iFirst);
124506 pPhrase->pHead = pCsr;
@@ -122359,30 +124853,10 @@
124853
124854 *ppCollist = pEnd;
124855 return nEntry;
124856 }
124857
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124858 /*
124859 ** fts3ExprIterate() callback used to collect the "global" matchinfo stats
124860 ** for a single query.
124861 **
124862 ** fts3ExprIterate() callback to load the 'global' elements of a
@@ -122412,52 +124886,13 @@
124886 Fts3Expr *pExpr, /* Phrase expression node */
124887 int iPhrase, /* Phrase number (numbered from zero) */
124888 void *pCtx /* Pointer to MatchInfo structure */
124889 ){
124890 MatchInfo *p = (MatchInfo *)pCtx;
124891 return sqlite3Fts3EvalPhraseStats(
124892 p->pCursor, pExpr, &p->aMatchinfo[3*iPhrase*p->nCol]
124893 );
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
124894 }
124895
124896 /*
124897 ** fts3ExprIterate() callback used to collect the "local" part of the
124898 ** FTS3_MATCHINFO_HITS array. The local stats are those elements of the
@@ -122470,18 +124905,17 @@
124905 ){
124906 MatchInfo *p = (MatchInfo *)pCtx;
124907 int iStart = iPhrase * p->nCol * 3;
124908 int i;
124909
124910 for(i=0; i<p->nCol; i++){
 
 
124911 char *pCsr;
124912 pCsr = sqlite3Fts3EvalPhrasePoslist(p->pCursor, pExpr, i);
 
124913 if( pCsr ){
124914 p->aMatchinfo[iStart+i*3] = fts3ColumnlistCount(&pCsr);
124915 }else{
124916 p->aMatchinfo[iStart+i*3] = 0;
124917 }
124918 }
124919
124920 return SQLITE_OK;
124921 }
@@ -122563,13 +124997,12 @@
124997 ** values for a matchinfo() FTS3_MATCHINFO_LCS request.
124998 */
124999 typedef struct LcsIterator LcsIterator;
125000 struct LcsIterator {
125001 Fts3Expr *pExpr; /* Pointer to phrase expression */
 
125002 int iPosOffset; /* Tokens count up to end of this phrase */
125003 char *pRead; /* Cursor used to iterate through aDoclist */
125004 int iPos; /* Current position */
125005 };
125006
125007 /*
125008 ** If LcsIterator.iCol is set to the following value, the iterator has
@@ -122596,21 +125029,14 @@
125029 char *pRead = pIter->pRead;
125030 sqlite3_int64 iRead;
125031 int rc = 0;
125032
125033 pRead += sqlite3Fts3GetVarint(pRead, &iRead);
125034 if( iRead==0 || iRead==1 ){
125035 pRead = 0;
125036 rc = 1;
125037 }else{
 
 
 
 
 
 
 
125038 pIter->iPos += (int)(iRead-2);
125039 }
125040
125041 pIter->pRead = pRead;
125042 return rc;
@@ -122638,46 +125064,38 @@
125064 **/
125065 aIter = sqlite3_malloc(sizeof(LcsIterator) * pCsr->nPhrase);
125066 if( !aIter ) return SQLITE_NOMEM;
125067 memset(aIter, 0, sizeof(LcsIterator) * pCsr->nPhrase);
125068 (void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter);
125069
125070 for(i=0; i<pInfo->nPhrase; i++){
125071 LcsIterator *pIter = &aIter[i];
125072 nToken -= pIter->pExpr->pPhrase->nToken;
125073 pIter->iPosOffset = nToken;
 
 
 
 
 
 
 
125074 }
125075
125076 for(iCol=0; iCol<pInfo->nCol; iCol++){
125077 int nLcs = 0; /* LCS value for this column */
125078 int nLive = 0; /* Number of iterators in aIter not at EOF */
125079
 
 
 
125080 for(i=0; i<pInfo->nPhrase; i++){
125081 LcsIterator *pIt = &aIter[i];
125082 pIt->pRead = sqlite3Fts3EvalPhrasePoslist(pCsr, pIt->pExpr, iCol);
125083 if( pIt->pRead ){
125084 pIt->iPos = pIt->iPosOffset;
125085 fts3LcsIteratorAdvance(&aIter[i]);
125086 nLive++;
125087 }
125088 }
125089
 
 
 
 
125090 while( nLive>0 ){
125091 LcsIterator *pAdv = 0; /* The iterator to advance by one position */
125092 int nThisLcs = 0; /* LCS for the current iterator positions */
125093
125094 for(i=0; i<pInfo->nPhrase; i++){
125095 LcsIterator *pIter = &aIter[i];
125096 if( pIter->pRead==0 ){
125097 /* This iterator is already at EOF for this column. */
125098 nThisLcs = 0;
125099 }else{
125100 if( pAdv==0 || pIter->iPos<pAdv->iPos ){
125101 pAdv = pIter;
@@ -122739,11 +125157,11 @@
125157 if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol;
125158 break;
125159
125160 case FTS3_MATCHINFO_NDOC:
125161 if( bGlobal ){
125162 sqlite3_int64 nDoc = 0;
125163 rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0);
125164 pInfo->aMatchinfo[0] = (u32)nDoc;
125165 }
125166 break;
125167
@@ -123013,11 +125431,11 @@
125431 int iTerm; /* For looping through nTerm phrase terms */
125432 char *pList; /* Pointer to position list for phrase */
125433 int iPos = 0; /* First position in position-list */
125434
125435 UNUSED_PARAMETER(iPhrase);
125436 pList = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol);
125437 nTerm = pExpr->pPhrase->nToken;
125438 if( pList ){
125439 fts3GetDeltaPosition(&pList, &iPos);
125440 assert( iPos>=0 );
125441 }
@@ -124634,11 +127052,11 @@
127052 */
127053 static float cellArea(Rtree *pRtree, RtreeCell *p){
127054 float area = 1.0;
127055 int ii;
127056 for(ii=0; ii<(pRtree->nDim*2); ii+=2){
127057 area = (float)(area * (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii])));
127058 }
127059 return area;
127060 }
127061
127062 /*
@@ -124647,11 +127065,11 @@
127065 */
127066 static float cellMargin(Rtree *pRtree, RtreeCell *p){
127067 float margin = 0.0;
127068 int ii;
127069 for(ii=0; ii<(pRtree->nDim*2); ii+=2){
127070 margin += (float)(DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]));
127071 }
127072 return margin;
127073 }
127074
127075 /*
@@ -124732,11 +127150,11 @@
127150
127151 if( x2<x1 ){
127152 o = 0.0;
127153 break;
127154 }else{
127155 o = o * (float)(x2-x1);
127156 }
127157 }
127158 overlap += o;
127159 }
127160 }
@@ -124751,16 +127169,16 @@
127169 RtreeCell *pInsert,
127170 RtreeCell *aCell,
127171 int nCell,
127172 int iExclude
127173 ){
127174 double before;
127175 double after;
127176 before = cellOverlap(pRtree, p, aCell, nCell, iExclude);
127177 cellUnion(pRtree, p, pInsert);
127178 after = cellOverlap(pRtree, p, aCell, nCell, iExclude);
127179 return (float)(after-before);
127180 }
127181 #endif
127182
127183
127184 /*
@@ -124778,15 +127196,15 @@
127196 RtreeNode *pNode;
127197 rc = nodeAcquire(pRtree, 1, 0, &pNode);
127198
127199 for(ii=0; rc==SQLITE_OK && ii<(pRtree->iDepth-iHeight); ii++){
127200 int iCell;
127201 sqlite3_int64 iBest = 0;
127202
127203 float fMinGrowth = 0.0;
127204 float fMinArea = 0.0;
127205 float fMinOverlap = 0.0;
127206
127207 int nCell = NCELL(pNode);
127208 RtreeCell cell;
127209 RtreeNode *pChild;
127210
@@ -125212,13 +127630,13 @@
127630 ){
127631 int **aaSorted;
127632 int *aSpare;
127633 int ii;
127634
127635 int iBestDim = 0;
127636 int iBestSplit = 0;
127637 float fBestMargin = 0.0;
127638
127639 int nByte = (pRtree->nDim+1)*(sizeof(int*)+nCell*sizeof(int));
127640
127641 aaSorted = (int **)sqlite3_malloc(nByte);
127642 if( !aaSorted ){
@@ -125236,13 +127654,13 @@
127654 SortByDimension(pRtree, aaSorted[ii], nCell, ii, aCell, aSpare);
127655 }
127656
127657 for(ii=0; ii<pRtree->nDim; ii++){
127658 float margin = 0.0;
127659 float fBestOverlap = 0.0;
127660 float fBestArea = 0.0;
127661 int iBestLeft = 0;
127662 int nLeft;
127663
127664 for(
127665 nLeft=RTREE_MINCELLS(pRtree);
127666 nLeft<=(nCell-RTREE_MINCELLS(pRtree));
@@ -125553,11 +127971,11 @@
127971 static int deleteCell(Rtree *, RtreeNode *, int, int);
127972
127973 static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){
127974 int rc;
127975 int rc2;
127976 RtreeNode *pParent = 0;
127977 int iCell;
127978
127979 assert( pNode->nRef==1 );
127980
127981 /* Remove the entry in the parent cell. */
@@ -125701,23 +128119,23 @@
128119 }else{
128120 nodeGetCell(pRtree, pNode, ii, &aCell[ii]);
128121 }
128122 aOrder[ii] = ii;
128123 for(iDim=0; iDim<pRtree->nDim; iDim++){
128124 aCenterCoord[iDim] += (float)DCOORD(aCell[ii].aCoord[iDim*2]);
128125 aCenterCoord[iDim] += (float)DCOORD(aCell[ii].aCoord[iDim*2+1]);
128126 }
128127 }
128128 for(iDim=0; iDim<pRtree->nDim; iDim++){
128129 aCenterCoord[iDim] = (float)(aCenterCoord[iDim]/((float)nCell*2.0));
128130 }
128131
128132 for(ii=0; ii<nCell; ii++){
128133 aDistance[ii] = 0.0;
128134 for(iDim=0; iDim<pRtree->nDim; iDim++){
128135 float coord = (float)(DCOORD(aCell[ii].aCoord[iDim*2+1]) -
128136 DCOORD(aCell[ii].aCoord[iDim*2]));
128137 aDistance[ii] += (coord-aCenterCoord[iDim])*(coord-aCenterCoord[iDim]);
128138 }
128139 }
128140
128141 SortByDistance(aOrder, nCell, aDistance, aSpare);
@@ -125812,14 +128230,14 @@
128230 nodeGetCell(pRtree, pNode, ii, &cell);
128231
128232 /* Find a node to store this cell in. pNode->iNode currently contains
128233 ** the height of the sub-tree headed by the cell.
128234 */
128235 rc = ChooseLeaf(pRtree, &cell, (int)pNode->iNode, &pInsert);
128236 if( rc==SQLITE_OK ){
128237 int rc2;
128238 rc = rtreeInsertCell(pRtree, pInsert, &cell, (int)pNode->iNode);
128239 rc2 = nodeRelease(pRtree, pInsert);
128240 if( rc==SQLITE_OK ){
128241 rc = rc2;
128242 }
128243 }
@@ -126204,11 +128622,11 @@
128622 int isCreate /* True for xCreate, false for xConnect */
128623 ){
128624 int rc;
128625 char *zSql;
128626 if( isCreate ){
128627 int iPageSize = 0;
128628 zSql = sqlite3_mprintf("PRAGMA %Q.page_size", pRtree->zDb);
128629 rc = getIntFromStmt(db, zSql, &iPageSize);
128630 if( rc==SQLITE_OK ){
128631 pRtree->iNodeSize = iPageSize-64;
128632 if( (4+pRtree->nBytesPerCell*RTREE_MAXCELLS)<pRtree->iNodeSize ){
@@ -127007,14 +129425,11 @@
129425 ** May you find forgiveness for yourself and forgive others.
129426 ** May you share freely, never taking more than you give.
129427 **
129428 *************************************************************************
129429 ** This file implements a tokenizer for fts3 based on the ICU library.
 
 
129430 */
 
129431 #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
129432 #ifdef SQLITE_ENABLE_ICU
129433
129434
129435 #include <unicode/ubrk.h>
129436
M src/sqlite3.h
+48 -15
--- src/sqlite3.h
+++ src/sqlite3.h
@@ -105,13 +105,13 @@
105105
 **
106106
 ** See also: [sqlite3_libversion()],
107107
 ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
108108
 ** [sqlite_version()] and [sqlite_source_id()].
109109
 */
110
-#define SQLITE_VERSION "3.7.7"
111
-#define SQLITE_VERSION_NUMBER 3007007
112
-#define SQLITE_SOURCE_ID "2011-06-03 14:19:10 0206bc6f87bb9393218a380fc5b18039d334a8d8"
110
+#define SQLITE_VERSION "3.7.8"
111
+#define SQLITE_VERSION_NUMBER 3007008
112
+#define SQLITE_SOURCE_ID "2011-08-16 02:07:04 9650d7962804d61f56cac944ff9bb2c7bc111957"
113113
114114
 /*
115115
 ** CAPI3REF: Run-Time Library Version Numbers
116116
 ** KEYWORDS: sqlite3_version, sqlite3_sourceid
117117
 **
@@ -308,11 +308,11 @@
308308
 ** semicolon-separate SQL statements passed into its 2nd argument,
309309
 ** in the context of the [database connection] passed in as its 1st
310310
 ** argument. ^If the callback function of the 3rd argument to
311311
 ** sqlite3_exec() is not NULL, then it is invoked for each result row
312312
 ** coming out of the evaluated SQL statements. ^The 4th argument to
313
-** to sqlite3_exec() is relayed through to the 1st argument of each
313
+** sqlite3_exec() is relayed through to the 1st argument of each
314314
 ** callback invocation. ^If the callback pointer to sqlite3_exec()
315315
 ** is NULL, then no callback is ever invoked and result rows are
316316
 ** ignored.
317317
 **
318318
 ** ^If an error occurs while evaluating the SQL statements passed into
@@ -739,20 +739,52 @@
739739
 ** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most
740740
 ** VFSes do not need this signal and should silently ignore this opcode.
741741
 ** Applications should not call [sqlite3_file_control()] with this
742742
 ** opcode as doing so may disrupt the operation of the specialized VFSes
743743
 ** that do require it.
744
+**
745
+** ^The [SQLITE_FCNTL_WIN32_AV_RETRY] opcode is used to configure automatic
746
+** retry counts and intervals for certain disk I/O operations for the
747
+** windows [VFS] in order to work to provide robustness against
748
+** anti-virus programs. By default, the windows VFS will retry file read,
749
+** file write, and file delete opertions up to 10 times, with a delay
750
+** of 25 milliseconds before the first retry and with the delay increasing
751
+** by an additional 25 milliseconds with each subsequent retry. This
752
+** opcode allows those to values (10 retries and 25 milliseconds of delay)
753
+** to be adjusted. The values are changed for all database connections
754
+** within the same process. The argument is a pointer to an array of two
755
+** integers where the first integer i the new retry count and the second
756
+** integer is the delay. If either integer is negative, then the setting
757
+** is not changed but instead the prior value of that setting is written
758
+** into the array entry, allowing the current retry settings to be
759
+** interrogated. The zDbName parameter is ignored.
760
+**
761
+** ^The [SQLITE_FCNTL_PERSIST_WAL] opcode is used to set or query the
762
+** persistent [WAL | Write AHead Log] setting. By default, the auxiliary
763
+** write ahead log and shared memory files used for transaction control
764
+** are automatically deleted when the latest connection to the database
765
+** closes. Setting persistent WAL mode causes those files to persist after
766
+** close. Persisting the files is useful when other processes that do not
767
+** have write permission on the directory containing the database file want
768
+** to read the database file, as the WAL and shared memory files must exist
769
+** in order for the database to be readable. The fourth parameter to
770
+** [sqlite3_file_control()] for this opcode should be a pointer to an integer.
771
+** That integer is 0 to disable persistent WAL mode or 1 to enable persistent
772
+** WAL mode. If the integer is -1, then it is overwritten with the current
773
+** WAL persistence setting.
774
+**
744775
 */
745776
 #define SQLITE_FCNTL_LOCKSTATE 1
746777
 #define SQLITE_GET_LOCKPROXYFILE 2
747778
 #define SQLITE_SET_LOCKPROXYFILE 3
748779
 #define SQLITE_LAST_ERRNO 4
749780
 #define SQLITE_FCNTL_SIZE_HINT 5
750781
 #define SQLITE_FCNTL_CHUNK_SIZE 6
751782
 #define SQLITE_FCNTL_FILE_POINTER 7
752783
 #define SQLITE_FCNTL_SYNC_OMITTED 8
753
-
784
+#define SQLITE_FCNTL_WIN32_AV_RETRY 9
785
+#define SQLITE_FCNTL_PERSIST_WAL 10
754786
755787
 /*
756788
 ** CAPI3REF: Mutex Handle
757789
 **
758790
 ** The mutex module within SQLite defines [sqlite3_mutex] to be an
@@ -900,11 +932,11 @@
900932
 ** The xSleep() method causes the calling thread to sleep for at
901933
 ** least the number of microseconds given. ^The xCurrentTime()
902934
 ** method returns a Julian Day Number for the current date and time as
903935
 ** a floating point value.
904936
 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian
905
-** Day Number multipled by 86400000 (the number of milliseconds in
937
+** Day Number multiplied by 86400000 (the number of milliseconds in
906938
 ** a 24-hour day).
907939
 ** ^SQLite will use the xCurrentTimeInt64() method to get the current
908940
 ** date and time if that method is available (if iVersion is 2 or
909941
 ** greater and the function pointer is not NULL) and will fall back
910942
 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable.
@@ -1338,11 +1370,11 @@
13381370
 ** scratch memory beyond what is provided by this configuration option, then
13391371
 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd>
13401372
 **
13411373
 ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt>
13421374
 ** <dd> ^This option specifies a static memory buffer that SQLite can use for
1343
-** the database page cache with the default page cache implemenation.
1375
+** the database page cache with the default page cache implementation.
13441376
 ** This configuration should not be used if an application-define page
13451377
 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option.
13461378
 ** There are three arguments to this option: A pointer to 8-byte aligned
13471379
 ** memory, the size of each page buffer (sz), and the number of pages (N).
13481380
 ** The sz argument should be the size of the largest database page
@@ -2436,16 +2468,16 @@
24362468
 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
24372469
 **
24382470
 ** ^If [URI filename] interpretation is enabled, and the filename argument
24392471
 ** begins with "file:", then the filename is interpreted as a URI. ^URI
24402472
 ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is
2441
-** is set in the fourth argument to sqlite3_open_v2(), or if it has
2473
+** set in the fourth argument to sqlite3_open_v2(), or if it has
24422474
 ** been enabled globally using the [SQLITE_CONFIG_URI] option with the
24432475
 ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.
24442476
 ** As of SQLite version 3.7.7, URI filename interpretation is turned off
24452477
 ** by default, but future releases of SQLite might enable URI filename
2446
-** intepretation by default. See "[URI filenames]" for additional
2478
+** interpretation by default. See "[URI filenames]" for additional
24472479
 ** information.
24482480
 **
24492481
 ** URI filenames are parsed according to RFC 3986. ^If the URI contains an
24502482
 ** authority, then it must be either an empty string or the string
24512483
 ** "localhost". ^If the authority is not an empty string or "localhost", an
@@ -2822,11 +2854,11 @@
28222854
 ** a schema change, on the first [sqlite3_step()] call following any change
28232855
 ** to the [sqlite3_bind_text | bindings] of that [parameter].
28242856
 ** ^The specific value of WHERE-clause [parameter] might influence the
28252857
 ** choice of query plan if the parameter is the left-hand side of a [LIKE]
28262858
 ** or [GLOB] operator or if the parameter is compared to an indexed column
2827
-** and the [SQLITE_ENABLE_STAT2] compile-time option is enabled.
2859
+** and the [SQLITE_ENABLE_STAT3] compile-time option is enabled.
28282860
 ** the
28292861
 ** </li>
28302862
 ** </ol>
28312863
 */
28322864
 SQLITE_API int sqlite3_prepare(
@@ -3260,11 +3292,11 @@
32603292
 ** [extended result codes] might be returned as well.
32613293
 **
32623294
 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the
32633295
 ** database locks it needs to do its job. ^If the statement is a [COMMIT]
32643296
 ** or occurs outside of an explicit transaction, then you can retry the
3265
-** statement. If the statement is not a [COMMIT] and occurs within a
3297
+** statement. If the statement is not a [COMMIT] and occurs within an
32663298
 ** explicit transaction then you should rollback the transaction before
32673299
 ** continuing.
32683300
 **
32693301
 ** ^[SQLITE_DONE] means that the statement has finished executing
32703302
 ** successfully. sqlite3_step() should not be called again on this virtual
@@ -3539,11 +3571,11 @@
35393571
35403572
 /*
35413573
 ** CAPI3REF: Destroy A Prepared Statement Object
35423574
 **
35433575
 ** ^The sqlite3_finalize() function is called to delete a [prepared statement].
3544
-** ^If the most recent evaluation of the statement encountered no errors or
3576
+** ^If the most recent evaluation of the statement encountered no errors
35453577
 ** or if the statement is never been evaluated, then sqlite3_finalize() returns
35463578
 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then
35473579
 ** sqlite3_finalize(S) returns the appropriate [error code] or
35483580
 ** [extended error code].
35493581
 **
@@ -5453,11 +5485,11 @@
54535485
 ** versions of these routines, it should at least provide stubs that always
54545486
 ** return true so that one does not get spurious assertion failures.
54555487
 **
54565488
 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then
54575489
 ** the routine should return 1. This seems counter-intuitive since
5458
-** clearly the mutex cannot be held if it does not exist. But the
5490
+** clearly the mutex cannot be held if it does not exist. But
54595491
 ** the reason the mutex does not exist is because the build is not
54605492
 ** using mutexes. And we do not want the assert() containing the
54615493
 ** call to sqlite3_mutex_held() to fail, so a non-zero return is
54625494
 ** the appropriate thing to do. ^The sqlite3_mutex_notheld()
54635495
 ** interface should also return 1 when given a NULL pointer.
@@ -5576,11 +5608,12 @@
55765608
 #define SQLITE_TESTCTRL_RESERVE 14
55775609
 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15
55785610
 #define SQLITE_TESTCTRL_ISKEYWORD 16
55795611
 #define SQLITE_TESTCTRL_PGHDRSZ 17
55805612
 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18
5581
-#define SQLITE_TESTCTRL_LAST 18
5613
+#define SQLITE_TESTCTRL_LOCALTIME_FAULT 19
5614
+#define SQLITE_TESTCTRL_LAST 19
55825615
55835616
 /*
55845617
 ** CAPI3REF: SQLite Runtime Status
55855618
 **
55865619
 ** ^This interface is used to retrieve runtime status information
@@ -5962,11 +5995,11 @@
59625995
 ** [[the xFetch() page cache methods]]
59635996
 ** The xFetch() method locates a page in the cache and returns a pointer to
59645997
 ** the page, or a NULL pointer.
59655998
 ** A "page", in this context, means a buffer of szPage bytes aligned at an
59665999
 ** 8-byte boundary. The page to be fetched is determined by the key. ^The
5967
-** mimimum key value is 1. After it has been retrieved using xFetch, the page
6000
+** minimum key value is 1. After it has been retrieved using xFetch, the page
59686001
 ** is considered to be "pinned".
59696002
 **
59706003
 ** If the requested page is already in the page cache, then the page cache
59716004
 ** implementation must return a pointer to the page buffer with its content
59726005
 ** intact. If the requested page is not already in the cache, then the
59736006
--- src/sqlite3.h
+++ src/sqlite3.h
@@ -105,13 +105,13 @@
105 **
106 ** See also: [sqlite3_libversion()],
107 ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
108 ** [sqlite_version()] and [sqlite_source_id()].
109 */
110 #define SQLITE_VERSION "3.7.7"
111 #define SQLITE_VERSION_NUMBER 3007007
112 #define SQLITE_SOURCE_ID "2011-06-03 14:19:10 0206bc6f87bb9393218a380fc5b18039d334a8d8"
113
114 /*
115 ** CAPI3REF: Run-Time Library Version Numbers
116 ** KEYWORDS: sqlite3_version, sqlite3_sourceid
117 **
@@ -308,11 +308,11 @@
308 ** semicolon-separate SQL statements passed into its 2nd argument,
309 ** in the context of the [database connection] passed in as its 1st
310 ** argument. ^If the callback function of the 3rd argument to
311 ** sqlite3_exec() is not NULL, then it is invoked for each result row
312 ** coming out of the evaluated SQL statements. ^The 4th argument to
313 ** to sqlite3_exec() is relayed through to the 1st argument of each
314 ** callback invocation. ^If the callback pointer to sqlite3_exec()
315 ** is NULL, then no callback is ever invoked and result rows are
316 ** ignored.
317 **
318 ** ^If an error occurs while evaluating the SQL statements passed into
@@ -739,20 +739,52 @@
739 ** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most
740 ** VFSes do not need this signal and should silently ignore this opcode.
741 ** Applications should not call [sqlite3_file_control()] with this
742 ** opcode as doing so may disrupt the operation of the specialized VFSes
743 ** that do require it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
744 */
745 #define SQLITE_FCNTL_LOCKSTATE 1
746 #define SQLITE_GET_LOCKPROXYFILE 2
747 #define SQLITE_SET_LOCKPROXYFILE 3
748 #define SQLITE_LAST_ERRNO 4
749 #define SQLITE_FCNTL_SIZE_HINT 5
750 #define SQLITE_FCNTL_CHUNK_SIZE 6
751 #define SQLITE_FCNTL_FILE_POINTER 7
752 #define SQLITE_FCNTL_SYNC_OMITTED 8
753
 
754
755 /*
756 ** CAPI3REF: Mutex Handle
757 **
758 ** The mutex module within SQLite defines [sqlite3_mutex] to be an
@@ -900,11 +932,11 @@
900 ** The xSleep() method causes the calling thread to sleep for at
901 ** least the number of microseconds given. ^The xCurrentTime()
902 ** method returns a Julian Day Number for the current date and time as
903 ** a floating point value.
904 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian
905 ** Day Number multipled by 86400000 (the number of milliseconds in
906 ** a 24-hour day).
907 ** ^SQLite will use the xCurrentTimeInt64() method to get the current
908 ** date and time if that method is available (if iVersion is 2 or
909 ** greater and the function pointer is not NULL) and will fall back
910 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable.
@@ -1338,11 +1370,11 @@
1338 ** scratch memory beyond what is provided by this configuration option, then
1339 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd>
1340 **
1341 ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt>
1342 ** <dd> ^This option specifies a static memory buffer that SQLite can use for
1343 ** the database page cache with the default page cache implemenation.
1344 ** This configuration should not be used if an application-define page
1345 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option.
1346 ** There are three arguments to this option: A pointer to 8-byte aligned
1347 ** memory, the size of each page buffer (sz), and the number of pages (N).
1348 ** The sz argument should be the size of the largest database page
@@ -2436,16 +2468,16 @@
2436 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
2437 **
2438 ** ^If [URI filename] interpretation is enabled, and the filename argument
2439 ** begins with "file:", then the filename is interpreted as a URI. ^URI
2440 ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is
2441 ** is set in the fourth argument to sqlite3_open_v2(), or if it has
2442 ** been enabled globally using the [SQLITE_CONFIG_URI] option with the
2443 ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.
2444 ** As of SQLite version 3.7.7, URI filename interpretation is turned off
2445 ** by default, but future releases of SQLite might enable URI filename
2446 ** intepretation by default. See "[URI filenames]" for additional
2447 ** information.
2448 **
2449 ** URI filenames are parsed according to RFC 3986. ^If the URI contains an
2450 ** authority, then it must be either an empty string or the string
2451 ** "localhost". ^If the authority is not an empty string or "localhost", an
@@ -2822,11 +2854,11 @@
2822 ** a schema change, on the first [sqlite3_step()] call following any change
2823 ** to the [sqlite3_bind_text | bindings] of that [parameter].
2824 ** ^The specific value of WHERE-clause [parameter] might influence the
2825 ** choice of query plan if the parameter is the left-hand side of a [LIKE]
2826 ** or [GLOB] operator or if the parameter is compared to an indexed column
2827 ** and the [SQLITE_ENABLE_STAT2] compile-time option is enabled.
2828 ** the
2829 ** </li>
2830 ** </ol>
2831 */
2832 SQLITE_API int sqlite3_prepare(
@@ -3260,11 +3292,11 @@
3260 ** [extended result codes] might be returned as well.
3261 **
3262 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the
3263 ** database locks it needs to do its job. ^If the statement is a [COMMIT]
3264 ** or occurs outside of an explicit transaction, then you can retry the
3265 ** statement. If the statement is not a [COMMIT] and occurs within a
3266 ** explicit transaction then you should rollback the transaction before
3267 ** continuing.
3268 **
3269 ** ^[SQLITE_DONE] means that the statement has finished executing
3270 ** successfully. sqlite3_step() should not be called again on this virtual
@@ -3539,11 +3571,11 @@
3539
3540 /*
3541 ** CAPI3REF: Destroy A Prepared Statement Object
3542 **
3543 ** ^The sqlite3_finalize() function is called to delete a [prepared statement].
3544 ** ^If the most recent evaluation of the statement encountered no errors or
3545 ** or if the statement is never been evaluated, then sqlite3_finalize() returns
3546 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then
3547 ** sqlite3_finalize(S) returns the appropriate [error code] or
3548 ** [extended error code].
3549 **
@@ -5453,11 +5485,11 @@
5453 ** versions of these routines, it should at least provide stubs that always
5454 ** return true so that one does not get spurious assertion failures.
5455 **
5456 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then
5457 ** the routine should return 1. This seems counter-intuitive since
5458 ** clearly the mutex cannot be held if it does not exist. But the
5459 ** the reason the mutex does not exist is because the build is not
5460 ** using mutexes. And we do not want the assert() containing the
5461 ** call to sqlite3_mutex_held() to fail, so a non-zero return is
5462 ** the appropriate thing to do. ^The sqlite3_mutex_notheld()
5463 ** interface should also return 1 when given a NULL pointer.
@@ -5576,11 +5608,12 @@
5576 #define SQLITE_TESTCTRL_RESERVE 14
5577 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15
5578 #define SQLITE_TESTCTRL_ISKEYWORD 16
5579 #define SQLITE_TESTCTRL_PGHDRSZ 17
5580 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18
5581 #define SQLITE_TESTCTRL_LAST 18
 
5582
5583 /*
5584 ** CAPI3REF: SQLite Runtime Status
5585 **
5586 ** ^This interface is used to retrieve runtime status information
@@ -5962,11 +5995,11 @@
5962 ** [[the xFetch() page cache methods]]
5963 ** The xFetch() method locates a page in the cache and returns a pointer to
5964 ** the page, or a NULL pointer.
5965 ** A "page", in this context, means a buffer of szPage bytes aligned at an
5966 ** 8-byte boundary. The page to be fetched is determined by the key. ^The
5967 ** mimimum key value is 1. After it has been retrieved using xFetch, the page
5968 ** is considered to be "pinned".
5969 **
5970 ** If the requested page is already in the page cache, then the page cache
5971 ** implementation must return a pointer to the page buffer with its content
5972 ** intact. If the requested page is not already in the cache, then the
5973
--- src/sqlite3.h
+++ src/sqlite3.h
@@ -105,13 +105,13 @@
105 **
106 ** See also: [sqlite3_libversion()],
107 ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
108 ** [sqlite_version()] and [sqlite_source_id()].
109 */
110 #define SQLITE_VERSION "3.7.8"
111 #define SQLITE_VERSION_NUMBER 3007008
112 #define SQLITE_SOURCE_ID "2011-08-16 02:07:04 9650d7962804d61f56cac944ff9bb2c7bc111957"
113
114 /*
115 ** CAPI3REF: Run-Time Library Version Numbers
116 ** KEYWORDS: sqlite3_version, sqlite3_sourceid
117 **
@@ -308,11 +308,11 @@
308 ** semicolon-separate SQL statements passed into its 2nd argument,
309 ** in the context of the [database connection] passed in as its 1st
310 ** argument. ^If the callback function of the 3rd argument to
311 ** sqlite3_exec() is not NULL, then it is invoked for each result row
312 ** coming out of the evaluated SQL statements. ^The 4th argument to
313 ** sqlite3_exec() is relayed through to the 1st argument of each
314 ** callback invocation. ^If the callback pointer to sqlite3_exec()
315 ** is NULL, then no callback is ever invoked and result rows are
316 ** ignored.
317 **
318 ** ^If an error occurs while evaluating the SQL statements passed into
@@ -739,20 +739,52 @@
739 ** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most
740 ** VFSes do not need this signal and should silently ignore this opcode.
741 ** Applications should not call [sqlite3_file_control()] with this
742 ** opcode as doing so may disrupt the operation of the specialized VFSes
743 ** that do require it.
744 **
745 ** ^The [SQLITE_FCNTL_WIN32_AV_RETRY] opcode is used to configure automatic
746 ** retry counts and intervals for certain disk I/O operations for the
747 ** windows [VFS] in order to work to provide robustness against
748 ** anti-virus programs. By default, the windows VFS will retry file read,
749 ** file write, and file delete opertions up to 10 times, with a delay
750 ** of 25 milliseconds before the first retry and with the delay increasing
751 ** by an additional 25 milliseconds with each subsequent retry. This
752 ** opcode allows those to values (10 retries and 25 milliseconds of delay)
753 ** to be adjusted. The values are changed for all database connections
754 ** within the same process. The argument is a pointer to an array of two
755 ** integers where the first integer i the new retry count and the second
756 ** integer is the delay. If either integer is negative, then the setting
757 ** is not changed but instead the prior value of that setting is written
758 ** into the array entry, allowing the current retry settings to be
759 ** interrogated. The zDbName parameter is ignored.
760 **
761 ** ^The [SQLITE_FCNTL_PERSIST_WAL] opcode is used to set or query the
762 ** persistent [WAL | Write AHead Log] setting. By default, the auxiliary
763 ** write ahead log and shared memory files used for transaction control
764 ** are automatically deleted when the latest connection to the database
765 ** closes. Setting persistent WAL mode causes those files to persist after
766 ** close. Persisting the files is useful when other processes that do not
767 ** have write permission on the directory containing the database file want
768 ** to read the database file, as the WAL and shared memory files must exist
769 ** in order for the database to be readable. The fourth parameter to
770 ** [sqlite3_file_control()] for this opcode should be a pointer to an integer.
771 ** That integer is 0 to disable persistent WAL mode or 1 to enable persistent
772 ** WAL mode. If the integer is -1, then it is overwritten with the current
773 ** WAL persistence setting.
774 **
775 */
776 #define SQLITE_FCNTL_LOCKSTATE 1
777 #define SQLITE_GET_LOCKPROXYFILE 2
778 #define SQLITE_SET_LOCKPROXYFILE 3
779 #define SQLITE_LAST_ERRNO 4
780 #define SQLITE_FCNTL_SIZE_HINT 5
781 #define SQLITE_FCNTL_CHUNK_SIZE 6
782 #define SQLITE_FCNTL_FILE_POINTER 7
783 #define SQLITE_FCNTL_SYNC_OMITTED 8
784 #define SQLITE_FCNTL_WIN32_AV_RETRY 9
785 #define SQLITE_FCNTL_PERSIST_WAL 10
786
787 /*
788 ** CAPI3REF: Mutex Handle
789 **
790 ** The mutex module within SQLite defines [sqlite3_mutex] to be an
@@ -900,11 +932,11 @@
932 ** The xSleep() method causes the calling thread to sleep for at
933 ** least the number of microseconds given. ^The xCurrentTime()
934 ** method returns a Julian Day Number for the current date and time as
935 ** a floating point value.
936 ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian
937 ** Day Number multiplied by 86400000 (the number of milliseconds in
938 ** a 24-hour day).
939 ** ^SQLite will use the xCurrentTimeInt64() method to get the current
940 ** date and time if that method is available (if iVersion is 2 or
941 ** greater and the function pointer is not NULL) and will fall back
942 ** to xCurrentTime() if xCurrentTimeInt64() is unavailable.
@@ -1338,11 +1370,11 @@
1370 ** scratch memory beyond what is provided by this configuration option, then
1371 ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd>
1372 **
1373 ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt>
1374 ** <dd> ^This option specifies a static memory buffer that SQLite can use for
1375 ** the database page cache with the default page cache implementation.
1376 ** This configuration should not be used if an application-define page
1377 ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option.
1378 ** There are three arguments to this option: A pointer to 8-byte aligned
1379 ** memory, the size of each page buffer (sz), and the number of pages (N).
1380 ** The sz argument should be the size of the largest database page
@@ -2436,16 +2468,16 @@
2468 ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3>
2469 **
2470 ** ^If [URI filename] interpretation is enabled, and the filename argument
2471 ** begins with "file:", then the filename is interpreted as a URI. ^URI
2472 ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is
2473 ** set in the fourth argument to sqlite3_open_v2(), or if it has
2474 ** been enabled globally using the [SQLITE_CONFIG_URI] option with the
2475 ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option.
2476 ** As of SQLite version 3.7.7, URI filename interpretation is turned off
2477 ** by default, but future releases of SQLite might enable URI filename
2478 ** interpretation by default. See "[URI filenames]" for additional
2479 ** information.
2480 **
2481 ** URI filenames are parsed according to RFC 3986. ^If the URI contains an
2482 ** authority, then it must be either an empty string or the string
2483 ** "localhost". ^If the authority is not an empty string or "localhost", an
@@ -2822,11 +2854,11 @@
2854 ** a schema change, on the first [sqlite3_step()] call following any change
2855 ** to the [sqlite3_bind_text | bindings] of that [parameter].
2856 ** ^The specific value of WHERE-clause [parameter] might influence the
2857 ** choice of query plan if the parameter is the left-hand side of a [LIKE]
2858 ** or [GLOB] operator or if the parameter is compared to an indexed column
2859 ** and the [SQLITE_ENABLE_STAT3] compile-time option is enabled.
2860 ** the
2861 ** </li>
2862 ** </ol>
2863 */
2864 SQLITE_API int sqlite3_prepare(
@@ -3260,11 +3292,11 @@
3292 ** [extended result codes] might be returned as well.
3293 **
3294 ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the
3295 ** database locks it needs to do its job. ^If the statement is a [COMMIT]
3296 ** or occurs outside of an explicit transaction, then you can retry the
3297 ** statement. If the statement is not a [COMMIT] and occurs within an
3298 ** explicit transaction then you should rollback the transaction before
3299 ** continuing.
3300 **
3301 ** ^[SQLITE_DONE] means that the statement has finished executing
3302 ** successfully. sqlite3_step() should not be called again on this virtual
@@ -3539,11 +3571,11 @@
3571
3572 /*
3573 ** CAPI3REF: Destroy A Prepared Statement Object
3574 **
3575 ** ^The sqlite3_finalize() function is called to delete a [prepared statement].
3576 ** ^If the most recent evaluation of the statement encountered no errors
3577 ** or if the statement is never been evaluated, then sqlite3_finalize() returns
3578 ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then
3579 ** sqlite3_finalize(S) returns the appropriate [error code] or
3580 ** [extended error code].
3581 **
@@ -5453,11 +5485,11 @@
5485 ** versions of these routines, it should at least provide stubs that always
5486 ** return true so that one does not get spurious assertion failures.
5487 **
5488 ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then
5489 ** the routine should return 1. This seems counter-intuitive since
5490 ** clearly the mutex cannot be held if it does not exist. But
5491 ** the reason the mutex does not exist is because the build is not
5492 ** using mutexes. And we do not want the assert() containing the
5493 ** call to sqlite3_mutex_held() to fail, so a non-zero return is
5494 ** the appropriate thing to do. ^The sqlite3_mutex_notheld()
5495 ** interface should also return 1 when given a NULL pointer.
@@ -5576,11 +5608,12 @@
5608 #define SQLITE_TESTCTRL_RESERVE 14
5609 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15
5610 #define SQLITE_TESTCTRL_ISKEYWORD 16
5611 #define SQLITE_TESTCTRL_PGHDRSZ 17
5612 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18
5613 #define SQLITE_TESTCTRL_LOCALTIME_FAULT 19
5614 #define SQLITE_TESTCTRL_LAST 19
5615
5616 /*
5617 ** CAPI3REF: SQLite Runtime Status
5618 **
5619 ** ^This interface is used to retrieve runtime status information
@@ -5962,11 +5995,11 @@
5995 ** [[the xFetch() page cache methods]]
5996 ** The xFetch() method locates a page in the cache and returns a pointer to
5997 ** the page, or a NULL pointer.
5998 ** A "page", in this context, means a buffer of szPage bytes aligned at an
5999 ** 8-byte boundary. The page to be fetched is determined by the key. ^The
6000 ** minimum key value is 1. After it has been retrieved using xFetch, the page
6001 ** is considered to be "pinned".
6002 **
6003 ** If the requested page is already in the page cache, then the page cache
6004 ** implementation must return a pointer to the page buffer with its content
6005 ** intact. If the requested page is not already in the cache, then the
6006
M src/stash.c
+1
--- src/stash.c
+++ src/stash.c
@@ -100,10 +100,11 @@
100100
 if( rid==0 ){
101101
 /* A new file */
102102
 blob_read_from_file(&content, zPath);
103103
 db_bind_blob(&ins, ":content", &content);
104104
 }else if( deleted ){
105
+ blob_zero(&content);
105106
 db_bind_null(&ins, ":content");
106107
 }else{
107108
 /* A modified file */
108109
 Blob orig;
109110
 Blob disk;
110111
--- src/stash.c
+++ src/stash.c
@@ -100,10 +100,11 @@
100 if( rid==0 ){
101 /* A new file */
102 blob_read_from_file(&content, zPath);
103 db_bind_blob(&ins, ":content", &content);
104 }else if( deleted ){
 
105 db_bind_null(&ins, ":content");
106 }else{
107 /* A modified file */
108 Blob orig;
109 Blob disk;
110
--- src/stash.c
+++ src/stash.c
@@ -100,10 +100,11 @@
100 if( rid==0 ){
101 /* A new file */
102 blob_read_from_file(&content, zPath);
103 db_bind_blob(&ins, ":content", &content);
104 }else if( deleted ){
105 blob_zero(&content);
106 db_bind_null(&ins, ":content");
107 }else{
108 /* A modified file */
109 Blob orig;
110 Blob disk;
111
M src/stat.c
+2 -1
--- src/stat.c
+++ src/stat.c
@@ -104,11 +104,12 @@
104104
 @ <tr><th>Server&nbsp;ID:</th><td>
105105
 @ %h(db_get("server-code",""))
106106
 @ </td></tr>
107107
108108
 @ <tr><th>Fossil&nbsp;Version:</th><td>
109
- @ %h(MANIFEST_DATE) %h(MANIFEST_VERSION) (%h(COMPILER_NAME))
109
+ @ %h(RELEASE_VERSION) %h(MANIFEST_DATE) %h(MANIFEST_VERSION)
110
+ @ (%h(COMPILER_NAME))
110111
 @ </td></tr>
111112
 @ <tr><th>SQLite&nbsp;Version:</th><td>
112113
 sqlite3_snprintf(sizeof(zBuf), zBuf, "%.19s [%.10s] (%s)",
113114
 SQLITE_SOURCE_ID, &SQLITE_SOURCE_ID[20], SQLITE_VERSION);
114115
 zDb = db_name("repository");
115116
--- src/stat.c
+++ src/stat.c
@@ -104,11 +104,12 @@
104 @ <tr><th>Server&nbsp;ID:</th><td>
105 @ %h(db_get("server-code",""))
106 @ </td></tr>
107
108 @ <tr><th>Fossil&nbsp;Version:</th><td>
109 @ %h(MANIFEST_DATE) %h(MANIFEST_VERSION) (%h(COMPILER_NAME))
 
110 @ </td></tr>
111 @ <tr><th>SQLite&nbsp;Version:</th><td>
112 sqlite3_snprintf(sizeof(zBuf), zBuf, "%.19s [%.10s] (%s)",
113 SQLITE_SOURCE_ID, &SQLITE_SOURCE_ID[20], SQLITE_VERSION);
114 zDb = db_name("repository");
115
--- src/stat.c
+++ src/stat.c
@@ -104,11 +104,12 @@
104 @ <tr><th>Server&nbsp;ID:</th><td>
105 @ %h(db_get("server-code",""))
106 @ </td></tr>
107
108 @ <tr><th>Fossil&nbsp;Version:</th><td>
109 @ %h(RELEASE_VERSION) %h(MANIFEST_DATE) %h(MANIFEST_VERSION)
110 @ (%h(COMPILER_NAME))
111 @ </td></tr>
112 @ <tr><th>SQLite&nbsp;Version:</th><td>
113 sqlite3_snprintf(sizeof(zBuf), zBuf, "%.19s [%.10s] (%s)",
114 SQLITE_SOURCE_ID, &SQLITE_SOURCE_ID[20], SQLITE_VERSION);
115 zDb = db_name("repository");
116
M src/style.c
+2 -1
--- src/style.c
+++ src/style.c
@@ -98,10 +98,11 @@
9898
 Th_Store("title", zTitle);
9999
 Th_Store("baseurl", g.zBaseURL);
100100
 Th_Store("home", g.zTop);
101101
 Th_Store("index_page", db_get("index-page","/home"));
102102
 Th_Store("current_page", g.zPath);
103
+ Th_Store("release_version", RELEASE_VERSION);
103104
 Th_Store("manifest_version", MANIFEST_VERSION);
104105
 Th_Store("manifest_date", MANIFEST_DATE);
105106
 Th_Store("compiler_name", COMPILER_NAME);
106107
 if( g.zLogin ){
107108
 Th_Store("login", g.zLogin);
@@ -245,11 +246,11 @@
245246
 /*
246247
 ** The default page footer
247248
 */
248249
 const char zDefaultFooter[] =
249250
 @ <div class="footer">
250
-@ Fossil version $manifest_version $manifest_date
251
+@ Fossil version $release_version $manifest_version $manifest_date
251252
 @ </div>
252253
 @ </body></html>
253254
 ;
254255
255256
 /*
256257
--- src/style.c
+++ src/style.c
@@ -98,10 +98,11 @@
98 Th_Store("title", zTitle);
99 Th_Store("baseurl", g.zBaseURL);
100 Th_Store("home", g.zTop);
101 Th_Store("index_page", db_get("index-page","/home"));
102 Th_Store("current_page", g.zPath);
 
103 Th_Store("manifest_version", MANIFEST_VERSION);
104 Th_Store("manifest_date", MANIFEST_DATE);
105 Th_Store("compiler_name", COMPILER_NAME);
106 if( g.zLogin ){
107 Th_Store("login", g.zLogin);
@@ -245,11 +246,11 @@
245 /*
246 ** The default page footer
247 */
248 const char zDefaultFooter[] =
249 @ <div class="footer">
250 @ Fossil version $manifest_version $manifest_date
251 @ </div>
252 @ </body></html>
253 ;
254
255 /*
256
--- src/style.c
+++ src/style.c
@@ -98,10 +98,11 @@
98 Th_Store("title", zTitle);
99 Th_Store("baseurl", g.zBaseURL);
100 Th_Store("home", g.zTop);
101 Th_Store("index_page", db_get("index-page","/home"));
102 Th_Store("current_page", g.zPath);
103 Th_Store("release_version", RELEASE_VERSION);
104 Th_Store("manifest_version", MANIFEST_VERSION);
105 Th_Store("manifest_date", MANIFEST_DATE);
106 Th_Store("compiler_name", COMPILER_NAME);
107 if( g.zLogin ){
108 Th_Store("login", g.zLogin);
@@ -245,11 +246,11 @@
246 /*
247 ** The default page footer
248 */
249 const char zDefaultFooter[] =
250 @ <div class="footer">
251 @ Fossil version $release_version $manifest_version $manifest_date
252 @ </div>
253 @ </body></html>
254 ;
255
256 /*
257
M src/tar.c
+280 -29
--- src/tar.c
+++ src/tar.c
@@ -27,31 +27,252 @@
2727
 */
2828
 static struct tarball_t {
2929
 unsigned char *aHdr; /* Space for building headers */
3030
 char *zSpaces; /* Spaces for padding */
3131
 char *zPrevDir; /* Name of directory for previous entry */
32
+ int nPrevDirAlloc; /* size of zPrevDir */
33
+ Blob pax; /* PAX data */
3234
 } tball;
35
+
36
+
37
+/*
38
+** field lengths of 'ustar' name and prefix fields.
39
+*/
40
+#define USTAR_NAME_LEN 100
41
+#define USTAR_PREFIX_LEN 155
42
+
3343
3444
 /*
3545
 ** Begin the process of generating a tarball.
3646
 **
3747
 ** Initialize the GZIP compressor and the table of directory names.
3848
 */
3949
 static void tar_begin(void){
4050
 assert( tball.aHdr==0 );
41
- tball.aHdr = fossil_malloc(512+512+256);
42
- memset(tball.aHdr, 0, 512+512+256);
51
+ tball.aHdr = fossil_malloc(512+512);
52
+ memset(tball.aHdr, 0, 512+512);
4353
 tball.zSpaces = (char*)&tball.aHdr[512];
44
- tball.zPrevDir = (char*)&tball.zSpaces[512];
54
+ /* zPrevDir init */
55
+ tball.zPrevDir = NULL;
56
+ tball.nPrevDirAlloc = 0;
57
+ /* scratch buffer init */
58
+ blob_zero(&tball.pax);
59
+
4560
 memcpy(&tball.aHdr[108], "0000000", 8); /* Owner ID */
4661
 memcpy(&tball.aHdr[116], "0000000", 8); /* Group ID */
47
- memcpy(&tball.aHdr[257], "ustar ", 7); /* Format */
62
+ memcpy(&tball.aHdr[257], "ustar\00000", 8); /* POSIX.1 format */
63
+ memcpy(&tball.aHdr[265], "nobody", 7); /* Owner name */
64
+ memcpy(&tball.aHdr[297], "nobody", 7); /* Group name */
4865
 gzip_begin();
4966
 db_multi_exec(
5067
 "CREATE TEMP TABLE dir(name UNIQUE);"
5168
 );
5269
 }
70
+
71
+
72
+/*
73
+** verify that lla characters in 'zName' are in the
74
+** ISO646 (=ASCII) character set.
75
+*/
76
+static int is_iso646_name(
77
+ const char *zName, /* file path */
78
+ int nName /* path length */
79
+){
80
+ int i;
81
+ for(i = 0; i < nName; i++){
82
+ unsigned char c = (unsigned char)zName[i];
83
+ if( c>0x7e ) return 0;
84
+ }
85
+ return 1;
86
+}
87
+
88
+
89
+/*
90
+** copy string pSrc into pDst, truncating or padding with 0 if necessary
91
+*/
92
+static void padded_copy(
93
+ char *pDest,
94
+ int nDest,
95
+ const char *pSrc,
96
+ int nSrc
97
+){
98
+ if(nSrc >= nDest){
99
+ memcpy(pDest, pSrc, nDest);
100
+ }else{
101
+ memcpy(pDest, pSrc, nSrc);
102
+ memset(&pDest[nSrc], 0, nDest - nSrc);
103
+ }
104
+}
105
+
106
+
107
+
108
+/******************************************************************************
109
+**
110
+** The 'tar' format has evolved over time. Initially the name was stored
111
+** in a 100 byte null-terminated field 'name'. File path names were
112
+** limited to 99 bytes.
113
+**
114
+** The Posix.1 'ustar' format added a 155 byte field 'prefix', allowing
115
+** for up to 255 characters to be stored. The full file path is formed by
116
+** concatenating the field 'prefix', a slash, and the field 'name'. This
117
+** gives some measure of compatibility with programs that only understand
118
+** the oldest format.
119
+**
120
+** The latest Posix extension is called the 'pax Interchange Format'.
121
+** It removes all the limitations of the previous two formats by allowing
122
+** the storage of arbitrary-length attributes in a separate object that looks
123
+** like a file to programs that do not understand this extension. So the
124
+** contents of the 'name' and 'prefix' fields should contain values that allow
125
+** versions of tar that do not understand this extension to still do
126
+** something useful.
127
+**
128
+******************************************************************************/
129
+
130
+/*
131
+** The position we use to split a file path into the 'name' and 'prefix'
132
+** fields needs to meet the following criteria:
133
+**
134
+** - not at the beginning or end of the string
135
+** - the position must contain a slash
136
+** - no more than 100 characters follow the slash
137
+** - no more than 155 characters precede it
138
+**
139
+** The routine 'find_split_pos' finds a split position. It will meet the
140
+** criteria of listed above if such a position exists. If no such
141
+** position exists it generates one that useful for generating the
142
+** values used for backward compatibility.
143
+*/
144
+static int find_split_pos(
145
+ const char *zName, /* file path */
146
+ int nName /* path length */
147
+){
148
+ int i, split = 0;
149
+ /* only search if the string needs splitting */
150
+ if(nName > USTAR_NAME_LEN){
151
+ for(i = 1; i+1 < nName; i++)
152
+ if(zName[i] == '/'){
153
+ split = i+1;
154
+ /* if the split position is within USTAR_NAME_LEN bytes from
155
+ * the end we can quit */
156
+ if(nName - split <= USTAR_NAME_LEN) break;
157
+ }
158
+ }
159
+ return split;
160
+}
161
+
162
+
163
+/*
164
+** attempt to split the file name path to meet 'ustar' header
165
+** criteria.
166
+*/
167
+static int tar_split_path(
168
+ const char *zName, /* path */
169
+ int nName, /* path length */
170
+ char *pName, /* name field */
171
+ char *pPrefix /* prefix field */
172
+){
173
+ int split = find_split_pos(zName, nName);
174
+ /* check whether both pieces fit */
175
+ if(nName - split > USTAR_NAME_LEN || split > USTAR_PREFIX_LEN+1){
176
+ return 0; /* no */
177
+ }
178
+
179
+ /* extract name */
180
+ padded_copy(pName, USTAR_NAME_LEN, &zName[split], nName - split);
181
+
182
+ /* extract prefix */
183
+ padded_copy(pPrefix, USTAR_PREFIX_LEN, zName, (split > 0 ? split - 1 : 0));
184
+
185
+ return 1; /* success */
186
+}
187
+
188
+
189
+/*
190
+** When using an extension header we still need to put something
191
+** reasonable in the name and prefix fields. This is probably as
192
+** good as it gets.
193
+*/
194
+static void approximate_split_path(
195
+ const char *zName, /* path */
196
+ int nName, /* path length */
197
+ char *pName, /* name field */
198
+ char *pPrefix, /* prefix field */
199
+ int bHeader /* is this a 'x' type tar header? */
200
+){
201
+ int split;
202
+
203
+ /* if this is a Pax Interchange header prepend "PaxHeader/"
204
+ ** so we can tell files apart from metadata */
205
+ if( bHeader ){
206
+ blob_reset(&tball.pax);
207
+ blob_appendf(&tball.pax, "PaxHeader/%*.*s", nName, nName, zName);
208
+ zName = blob_buffer(&tball.pax);
209
+ nName = blob_size(&tball.pax);
210
+ }
211
+
212
+ /* find the split position */
213
+ split = find_split_pos(zName, nName);
214
+
215
+ /* extract a name, truncate if needed */
216
+ padded_copy(pName, USTAR_NAME_LEN, &zName[split], nName - split);
217
+
218
+ /* extract a prefix field, truncate when needed */
219
+ padded_copy(pPrefix, USTAR_PREFIX_LEN, zName, (split > 0 ? split-1 : 0));
220
+}
221
+
222
+
223
+/*
224
+** add a Pax Interchange header to the scratch buffer
225
+**
226
+** format: <length> <key>=<value>\n
227
+** the tricky part is that each header contains its own
228
+** size in decimal, counting that length.
229
+*/
230
+static void add_pax_header(
231
+ const char *zField,
232
+ const char *zValue,
233
+ int nValue
234
+){
235
+ /* calculate length without length field */
236
+ int blen = strlen(zField) + nValue + 3;
237
+ /* calculate the length of the length field */
238
+ int next10 = 1;
239
+ int n;
240
+ for(n = blen; n > 0; ){
241
+ blen++; next10 *= 10;
242
+ n /= 10;
243
+ }
244
+ /* adding the length extended the length field? */
245
+ if(blen > next10){
246
+ blen++;
247
+ }
248
+ /* build the string */
249
+ blob_appendf(&tball.pax, "%d %s=%*.*s\n", blen, zField, nValue, nValue, zValue);
250
+ /* this _must_ be right */
251
+ if(blob_size(&tball.pax) != blen){
252
+ fossil_fatal("internal error: PAX tar header has bad length");
253
+ }
254
+}
255
+
256
+
257
+/*
258
+** set the header type, calculate the checksum and output
259
+** the header
260
+*/
261
+static void cksum_and_write_header(
262
+ char cType
263
+){
264
+ unsigned int cksum = 0;
265
+ int i;
266
+ memset(&tball.aHdr[148], ' ', 8);
267
+ tball.aHdr[156] = cType;
268
+ for(i=0; i<512; i++) cksum += tball.aHdr[i];
269
+ sqlite3_snprintf(8, (char*)&tball.aHdr[148], "%07o", cksum);
270
+ tball.aHdr[155] = 0;
271
+ gzip_step((char*)tball.aHdr, 512);
272
+}
273
+
53274
54275
 /*
55276
 ** Build a header for a file or directory and write that header
56277
 ** into the growing tarball.
57278
 */
@@ -59,33 +280,52 @@
59280
 const char *zName, /* Name of the object */
60281
 int nName, /* Number of characters in zName */
61282
 int iMode, /* Mode. 0644 or 0755 */
62283
 unsigned int mTime, /* File modification time */
63284
 int iSize, /* Size of the object in bytes */
64
- int iType /* Type of object. 0==file. 5==directory */
285
+ char cType /* Type of object. '0'==file. '5'==directory */
65286
 ){
66
- unsigned int cksum = 0;
67
- int i;
68
- if( nName>100 ){
69
- memcpy(&tball.aHdr[345], zName, nName-100);
70
- memcpy(tball.aHdr, &zName[nName-100], 100);
71
- memset(&tball.aHdr[245+nName], 0, 267-nName);
72
- }else{
73
- memcpy(tball.aHdr, zName, nName);
74
- memset(&tball.aHdr[nName], 0, 100-nName);
75
- memset(&tball.aHdr[345], 0, 167);
76
- }
287
+ /* set mode and modification time */
77288
 sqlite3_snprintf(8, (char*)&tball.aHdr[100], "%07o", iMode);
78
- sqlite3_snprintf(12, (char*)&tball.aHdr[124], "%011o", iSize);
79289
 sqlite3_snprintf(12, (char*)&tball.aHdr[136], "%011o", mTime);
80
- memset(&tball.aHdr[148], ' ', 8);
81
- tball.aHdr[156] = iType + '0';
82
- for(i=0; i<512; i++) cksum += tball.aHdr[i];
83
- sqlite3_snprintf(7, (char*)&tball.aHdr[148], "%06o", cksum);
84
- tball.aHdr[154] = 0;
85
- gzip_step((char*)tball.aHdr, 512);
290
+
291
+ /* see if we need to output a Pax Interchange Header */
292
+ if( !is_iso646_name(zName, nName)
293
+ || !tar_split_path(zName, nName, (char*)tball.aHdr, (char*)&tball.aHdr[345])
294
+ ){
295
+ int lastPage;
296
+ /* add a file name for interoperability with older programs */
297
+ approximate_split_path(zName, nName, (char*)tball.aHdr,
298
+ (char*)&tball.aHdr[345], 1);
299
+
300
+ /* generate the Pax Interchange path header */
301
+ blob_reset(&tball.pax);
302
+ add_pax_header("path", zName, nName);
303
+
304
+ /* set the header length, and write the header */
305
+ sqlite3_snprintf(12, (char*)&tball.aHdr[124], "%011o",
306
+ blob_size(&tball.pax));
307
+ cksum_and_write_header('x');
308
+
309
+ /* write the Pax Interchange data */
310
+ gzip_step(blob_buffer(&tball.pax), blob_size(&tball.pax));
311
+ lastPage = blob_size(&tball.pax) % 512;
312
+ if( lastPage!=0 ){
313
+ gzip_step(tball.zSpaces, 512 - lastPage);
314
+ }
315
+
316
+ /* generate an approximate path for the regular header */
317
+ approximate_split_path(zName, nName, (char*)tball.aHdr,
318
+ (char*)&tball.aHdr[345], 0);
319
+ }
320
+ /* set the size */
321
+ sqlite3_snprintf(12, (char*)&tball.aHdr[124], "%011o", iSize);
322
+
323
+ /* write the regular header */
324
+ cksum_and_write_header(cType);
86325
 }
326
+
87327
88328
 /*
89329
 ** Recursively add an directory entry for the given file if those
90330
 ** directories have not previously been seen.
91331
 */
@@ -95,18 +335,27 @@
95335
 unsigned int mTime /* Modification time */
96336
 ){
97337
 int i;
98338
 for(i=nName-1; i>0 && zName[i]!='/'; i--){}
99339
 if( i<=0 ) return;
100
- if( tball.zPrevDir[i]==0 && memcmp(tball.zPrevDir, zName, i)==0 ) return;
340
+ if( i < tball.nPrevDirAlloc && tball.zPrevDir[i]==0 &&
341
+ memcmp(tball.zPrevDir, zName, i)==0 ) return;
101342
 db_multi_exec("INSERT OR IGNORE INTO dir VALUES('%#q')", i, zName);
102343
 if( sqlite3_changes(g.db)==0 ) return;
103344
 tar_add_directory_of(zName, i-1, mTime);
104
- tar_add_header(zName, i, 0755, mTime, 0, 5);
345
+ tar_add_header(zName, i, 0755, mTime, 0, '5');
346
+ if( i >= tball.nPrevDirAlloc ){
347
+ int nsize = tball.nPrevDirAlloc * 2;
348
+ if(i+1 > nsize)
349
+ nsize = i+1;
350
+ tball.zPrevDir = fossil_realloc(tball.zPrevDir, nsize);
351
+ tball.nPrevDirAlloc = nsize;
352
+ }
105353
 memcpy(tball.zPrevDir, zName, i);
106354
 tball.zPrevDir[i] = 0;
107355
 }
356
+
108357
109358
 /*
110359
 ** Add a single file to the growing tarball.
111360
 */
112361
 static void tar_add_file(
@@ -117,15 +366,13 @@
117366
 ){
118367
 int nName = strlen(zName);
119368
 int n = blob_size(pContent);
120369
 int lastPage;
121370
122
- if( nName>=250 ){
123
- fossil_fatal("name too long for ustar format: \"%s\"", zName);
124
- }
371
+ /* length check moved to tar_split_path */
125372
 tar_add_directory_of(zName, nName, mTime);
126
- tar_add_header(zName, nName, isExe ? 0755 : 0644, mTime, n, 0);
373
+ tar_add_header(zName, nName, isExe ? 0755 : 0644, mTime, n, '0');
127374
 if( n ){
128375
 gzip_step(blob_buffer(pContent), n);
129376
 lastPage = n % 512;
130377
 if( lastPage!=0 ){
131378
 gzip_step(tball.zSpaces, 512 - lastPage);
@@ -142,10 +389,14 @@
142389
 gzip_step(tball.zSpaces, 512);
143390
 gzip_step(tball.zSpaces, 512);
144391
 gzip_finish(pOut);
145392
 fossil_free(tball.aHdr);
146393
 tball.aHdr = 0;
394
+ fossil_free(tball.zPrevDir);
395
+ tball.zPrevDir = NULL;
396
+ tball.nPrevDirAlloc = 0;
397
+ blob_reset(&tball.pax);
147398
 }
148399
149400
150401
 /*
151402
 ** COMMAND: test-tarball
152403
--- src/tar.c
+++ src/tar.c
@@ -27,31 +27,252 @@
27 */
28 static struct tarball_t {
29 unsigned char *aHdr; /* Space for building headers */
30 char *zSpaces; /* Spaces for padding */
31 char *zPrevDir; /* Name of directory for previous entry */
 
 
32 } tball;
 
 
 
 
 
 
 
 
33
34 /*
35 ** Begin the process of generating a tarball.
36 **
37 ** Initialize the GZIP compressor and the table of directory names.
38 */
39 static void tar_begin(void){
40 assert( tball.aHdr==0 );
41 tball.aHdr = fossil_malloc(512+512+256);
42 memset(tball.aHdr, 0, 512+512+256);
43 tball.zSpaces = (char*)&tball.aHdr[512];
44 tball.zPrevDir = (char*)&tball.zSpaces[512];
 
 
 
 
 
45 memcpy(&tball.aHdr[108], "0000000", 8); /* Owner ID */
46 memcpy(&tball.aHdr[116], "0000000", 8); /* Group ID */
47 memcpy(&tball.aHdr[257], "ustar ", 7); /* Format */
 
 
48 gzip_begin();
49 db_multi_exec(
50 "CREATE TEMP TABLE dir(name UNIQUE);"
51 );
52 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
54 /*
55 ** Build a header for a file or directory and write that header
56 ** into the growing tarball.
57 */
@@ -59,33 +280,52 @@
59 const char *zName, /* Name of the object */
60 int nName, /* Number of characters in zName */
61 int iMode, /* Mode. 0644 or 0755 */
62 unsigned int mTime, /* File modification time */
63 int iSize, /* Size of the object in bytes */
64 int iType /* Type of object. 0==file. 5==directory */
65 ){
66 unsigned int cksum = 0;
67 int i;
68 if( nName>100 ){
69 memcpy(&tball.aHdr[345], zName, nName-100);
70 memcpy(tball.aHdr, &zName[nName-100], 100);
71 memset(&tball.aHdr[245+nName], 0, 267-nName);
72 }else{
73 memcpy(tball.aHdr, zName, nName);
74 memset(&tball.aHdr[nName], 0, 100-nName);
75 memset(&tball.aHdr[345], 0, 167);
76 }
77 sqlite3_snprintf(8, (char*)&tball.aHdr[100], "%07o", iMode);
78 sqlite3_snprintf(12, (char*)&tball.aHdr[124], "%011o", iSize);
79 sqlite3_snprintf(12, (char*)&tball.aHdr[136], "%011o", mTime);
80 memset(&tball.aHdr[148], ' ', 8);
81 tball.aHdr[156] = iType + '0';
82 for(i=0; i<512; i++) cksum += tball.aHdr[i];
83 sqlite3_snprintf(7, (char*)&tball.aHdr[148], "%06o", cksum);
84 tball.aHdr[154] = 0;
85 gzip_step((char*)tball.aHdr, 512);
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
86 }
 
87
88 /*
89 ** Recursively add an directory entry for the given file if those
90 ** directories have not previously been seen.
91 */
@@ -95,18 +335,27 @@
95 unsigned int mTime /* Modification time */
96 ){
97 int i;
98 for(i=nName-1; i>0 && zName[i]!='/'; i--){}
99 if( i<=0 ) return;
100 if( tball.zPrevDir[i]==0 && memcmp(tball.zPrevDir, zName, i)==0 ) return;
 
101 db_multi_exec("INSERT OR IGNORE INTO dir VALUES('%#q')", i, zName);
102 if( sqlite3_changes(g.db)==0 ) return;
103 tar_add_directory_of(zName, i-1, mTime);
104 tar_add_header(zName, i, 0755, mTime, 0, 5);
 
 
 
 
 
 
 
105 memcpy(tball.zPrevDir, zName, i);
106 tball.zPrevDir[i] = 0;
107 }
 
108
109 /*
110 ** Add a single file to the growing tarball.
111 */
112 static void tar_add_file(
@@ -117,15 +366,13 @@
117 ){
118 int nName = strlen(zName);
119 int n = blob_size(pContent);
120 int lastPage;
121
122 if( nName>=250 ){
123 fossil_fatal("name too long for ustar format: \"%s\"", zName);
124 }
125 tar_add_directory_of(zName, nName, mTime);
126 tar_add_header(zName, nName, isExe ? 0755 : 0644, mTime, n, 0);
127 if( n ){
128 gzip_step(blob_buffer(pContent), n);
129 lastPage = n % 512;
130 if( lastPage!=0 ){
131 gzip_step(tball.zSpaces, 512 - lastPage);
@@ -142,10 +389,14 @@
142 gzip_step(tball.zSpaces, 512);
143 gzip_step(tball.zSpaces, 512);
144 gzip_finish(pOut);
145 fossil_free(tball.aHdr);
146 tball.aHdr = 0;
 
 
 
 
147 }
148
149
150 /*
151 ** COMMAND: test-tarball
152
--- src/tar.c
+++ src/tar.c
@@ -27,31 +27,252 @@
27 */
28 static struct tarball_t {
29 unsigned char *aHdr; /* Space for building headers */
30 char *zSpaces; /* Spaces for padding */
31 char *zPrevDir; /* Name of directory for previous entry */
32 int nPrevDirAlloc; /* size of zPrevDir */
33 Blob pax; /* PAX data */
34 } tball;
35
36
37 /*
38 ** field lengths of 'ustar' name and prefix fields.
39 */
40 #define USTAR_NAME_LEN 100
41 #define USTAR_PREFIX_LEN 155
42
43
44 /*
45 ** Begin the process of generating a tarball.
46 **
47 ** Initialize the GZIP compressor and the table of directory names.
48 */
49 static void tar_begin(void){
50 assert( tball.aHdr==0 );
51 tball.aHdr = fossil_malloc(512+512);
52 memset(tball.aHdr, 0, 512+512);
53 tball.zSpaces = (char*)&tball.aHdr[512];
54 /* zPrevDir init */
55 tball.zPrevDir = NULL;
56 tball.nPrevDirAlloc = 0;
57 /* scratch buffer init */
58 blob_zero(&tball.pax);
59
60 memcpy(&tball.aHdr[108], "0000000", 8); /* Owner ID */
61 memcpy(&tball.aHdr[116], "0000000", 8); /* Group ID */
62 memcpy(&tball.aHdr[257], "ustar\00000", 8); /* POSIX.1 format */
63 memcpy(&tball.aHdr[265], "nobody", 7); /* Owner name */
64 memcpy(&tball.aHdr[297], "nobody", 7); /* Group name */
65 gzip_begin();
66 db_multi_exec(
67 "CREATE TEMP TABLE dir(name UNIQUE);"
68 );
69 }
70
71
72 /*
73 ** verify that lla characters in 'zName' are in the
74 ** ISO646 (=ASCII) character set.
75 */
76 static int is_iso646_name(
77 const char *zName, /* file path */
78 int nName /* path length */
79 ){
80 int i;
81 for(i = 0; i < nName; i++){
82 unsigned char c = (unsigned char)zName[i];
83 if( c>0x7e ) return 0;
84 }
85 return 1;
86 }
87
88
89 /*
90 ** copy string pSrc into pDst, truncating or padding with 0 if necessary
91 */
92 static void padded_copy(
93 char *pDest,
94 int nDest,
95 const char *pSrc,
96 int nSrc
97 ){
98 if(nSrc >= nDest){
99 memcpy(pDest, pSrc, nDest);
100 }else{
101 memcpy(pDest, pSrc, nSrc);
102 memset(&pDest[nSrc], 0, nDest - nSrc);
103 }
104 }
105
106
107
108 /******************************************************************************
109 **
110 ** The 'tar' format has evolved over time. Initially the name was stored
111 ** in a 100 byte null-terminated field 'name'. File path names were
112 ** limited to 99 bytes.
113 **
114 ** The Posix.1 'ustar' format added a 155 byte field 'prefix', allowing
115 ** for up to 255 characters to be stored. The full file path is formed by
116 ** concatenating the field 'prefix', a slash, and the field 'name'. This
117 ** gives some measure of compatibility with programs that only understand
118 ** the oldest format.
119 **
120 ** The latest Posix extension is called the 'pax Interchange Format'.
121 ** It removes all the limitations of the previous two formats by allowing
122 ** the storage of arbitrary-length attributes in a separate object that looks
123 ** like a file to programs that do not understand this extension. So the
124 ** contents of the 'name' and 'prefix' fields should contain values that allow
125 ** versions of tar that do not understand this extension to still do
126 ** something useful.
127 **
128 ******************************************************************************/
129
130 /*
131 ** The position we use to split a file path into the 'name' and 'prefix'
132 ** fields needs to meet the following criteria:
133 **
134 ** - not at the beginning or end of the string
135 ** - the position must contain a slash
136 ** - no more than 100 characters follow the slash
137 ** - no more than 155 characters precede it
138 **
139 ** The routine 'find_split_pos' finds a split position. It will meet the
140 ** criteria of listed above if such a position exists. If no such
141 ** position exists it generates one that useful for generating the
142 ** values used for backward compatibility.
143 */
144 static int find_split_pos(
145 const char *zName, /* file path */
146 int nName /* path length */
147 ){
148 int i, split = 0;
149 /* only search if the string needs splitting */
150 if(nName > USTAR_NAME_LEN){
151 for(i = 1; i+1 < nName; i++)
152 if(zName[i] == '/'){
153 split = i+1;
154 /* if the split position is within USTAR_NAME_LEN bytes from
155 * the end we can quit */
156 if(nName - split <= USTAR_NAME_LEN) break;
157 }
158 }
159 return split;
160 }
161
162
163 /*
164 ** attempt to split the file name path to meet 'ustar' header
165 ** criteria.
166 */
167 static int tar_split_path(
168 const char *zName, /* path */
169 int nName, /* path length */
170 char *pName, /* name field */
171 char *pPrefix /* prefix field */
172 ){
173 int split = find_split_pos(zName, nName);
174 /* check whether both pieces fit */
175 if(nName - split > USTAR_NAME_LEN || split > USTAR_PREFIX_LEN+1){
176 return 0; /* no */
177 }
178
179 /* extract name */
180 padded_copy(pName, USTAR_NAME_LEN, &zName[split], nName - split);
181
182 /* extract prefix */
183 padded_copy(pPrefix, USTAR_PREFIX_LEN, zName, (split > 0 ? split - 1 : 0));
184
185 return 1; /* success */
186 }
187
188
189 /*
190 ** When using an extension header we still need to put something
191 ** reasonable in the name and prefix fields. This is probably as
192 ** good as it gets.
193 */
194 static void approximate_split_path(
195 const char *zName, /* path */
196 int nName, /* path length */
197 char *pName, /* name field */
198 char *pPrefix, /* prefix field */
199 int bHeader /* is this a 'x' type tar header? */
200 ){
201 int split;
202
203 /* if this is a Pax Interchange header prepend "PaxHeader/"
204 ** so we can tell files apart from metadata */
205 if( bHeader ){
206 blob_reset(&tball.pax);
207 blob_appendf(&tball.pax, "PaxHeader/%*.*s", nName, nName, zName);
208 zName = blob_buffer(&tball.pax);
209 nName = blob_size(&tball.pax);
210 }
211
212 /* find the split position */
213 split = find_split_pos(zName, nName);
214
215 /* extract a name, truncate if needed */
216 padded_copy(pName, USTAR_NAME_LEN, &zName[split], nName - split);
217
218 /* extract a prefix field, truncate when needed */
219 padded_copy(pPrefix, USTAR_PREFIX_LEN, zName, (split > 0 ? split-1 : 0));
220 }
221
222
223 /*
224 ** add a Pax Interchange header to the scratch buffer
225 **
226 ** format: <length> <key>=<value>\n
227 ** the tricky part is that each header contains its own
228 ** size in decimal, counting that length.
229 */
230 static void add_pax_header(
231 const char *zField,
232 const char *zValue,
233 int nValue
234 ){
235 /* calculate length without length field */
236 int blen = strlen(zField) + nValue + 3;
237 /* calculate the length of the length field */
238 int next10 = 1;
239 int n;
240 for(n = blen; n > 0; ){
241 blen++; next10 *= 10;
242 n /= 10;
243 }
244 /* adding the length extended the length field? */
245 if(blen > next10){
246 blen++;
247 }
248 /* build the string */
249 blob_appendf(&tball.pax, "%d %s=%*.*s\n", blen, zField, nValue, nValue, zValue);
250 /* this _must_ be right */
251 if(blob_size(&tball.pax) != blen){
252 fossil_fatal("internal error: PAX tar header has bad length");
253 }
254 }
255
256
257 /*
258 ** set the header type, calculate the checksum and output
259 ** the header
260 */
261 static void cksum_and_write_header(
262 char cType
263 ){
264 unsigned int cksum = 0;
265 int i;
266 memset(&tball.aHdr[148], ' ', 8);
267 tball.aHdr[156] = cType;
268 for(i=0; i<512; i++) cksum += tball.aHdr[i];
269 sqlite3_snprintf(8, (char*)&tball.aHdr[148], "%07o", cksum);
270 tball.aHdr[155] = 0;
271 gzip_step((char*)tball.aHdr, 512);
272 }
273
274
275 /*
276 ** Build a header for a file or directory and write that header
277 ** into the growing tarball.
278 */
@@ -59,33 +280,52 @@
280 const char *zName, /* Name of the object */
281 int nName, /* Number of characters in zName */
282 int iMode, /* Mode. 0644 or 0755 */
283 unsigned int mTime, /* File modification time */
284 int iSize, /* Size of the object in bytes */
285 char cType /* Type of object. '0'==file. '5'==directory */
286 ){
287 /* set mode and modification time */
 
 
 
 
 
 
 
 
 
 
288 sqlite3_snprintf(8, (char*)&tball.aHdr[100], "%07o", iMode);
 
289 sqlite3_snprintf(12, (char*)&tball.aHdr[136], "%011o", mTime);
290
291 /* see if we need to output a Pax Interchange Header */
292 if( !is_iso646_name(zName, nName)
293 || !tar_split_path(zName, nName, (char*)tball.aHdr, (char*)&tball.aHdr[345])
294 ){
295 int lastPage;
296 /* add a file name for interoperability with older programs */
297 approximate_split_path(zName, nName, (char*)tball.aHdr,
298 (char*)&tball.aHdr[345], 1);
299
300 /* generate the Pax Interchange path header */
301 blob_reset(&tball.pax);
302 add_pax_header("path", zName, nName);
303
304 /* set the header length, and write the header */
305 sqlite3_snprintf(12, (char*)&tball.aHdr[124], "%011o",
306 blob_size(&tball.pax));
307 cksum_and_write_header('x');
308
309 /* write the Pax Interchange data */
310 gzip_step(blob_buffer(&tball.pax), blob_size(&tball.pax));
311 lastPage = blob_size(&tball.pax) % 512;
312 if( lastPage!=0 ){
313 gzip_step(tball.zSpaces, 512 - lastPage);
314 }
315
316 /* generate an approximate path for the regular header */
317 approximate_split_path(zName, nName, (char*)tball.aHdr,
318 (char*)&tball.aHdr[345], 0);
319 }
320 /* set the size */
321 sqlite3_snprintf(12, (char*)&tball.aHdr[124], "%011o", iSize);
322
323 /* write the regular header */
324 cksum_and_write_header(cType);
325 }
326
327
328 /*
329 ** Recursively add an directory entry for the given file if those
330 ** directories have not previously been seen.
331 */
@@ -95,18 +335,27 @@
335 unsigned int mTime /* Modification time */
336 ){
337 int i;
338 for(i=nName-1; i>0 && zName[i]!='/'; i--){}
339 if( i<=0 ) return;
340 if( i < tball.nPrevDirAlloc && tball.zPrevDir[i]==0 &&
341 memcmp(tball.zPrevDir, zName, i)==0 ) return;
342 db_multi_exec("INSERT OR IGNORE INTO dir VALUES('%#q')", i, zName);
343 if( sqlite3_changes(g.db)==0 ) return;
344 tar_add_directory_of(zName, i-1, mTime);
345 tar_add_header(zName, i, 0755, mTime, 0, '5');
346 if( i >= tball.nPrevDirAlloc ){
347 int nsize = tball.nPrevDirAlloc * 2;
348 if(i+1 > nsize)
349 nsize = i+1;
350 tball.zPrevDir = fossil_realloc(tball.zPrevDir, nsize);
351 tball.nPrevDirAlloc = nsize;
352 }
353 memcpy(tball.zPrevDir, zName, i);
354 tball.zPrevDir[i] = 0;
355 }
356
357
358 /*
359 ** Add a single file to the growing tarball.
360 */
361 static void tar_add_file(
@@ -117,15 +366,13 @@
366 ){
367 int nName = strlen(zName);
368 int n = blob_size(pContent);
369 int lastPage;
370
371 /* length check moved to tar_split_path */
 
 
372 tar_add_directory_of(zName, nName, mTime);
373 tar_add_header(zName, nName, isExe ? 0755 : 0644, mTime, n, '0');
374 if( n ){
375 gzip_step(blob_buffer(pContent), n);
376 lastPage = n % 512;
377 if( lastPage!=0 ){
378 gzip_step(tball.zSpaces, 512 - lastPage);
@@ -142,10 +389,14 @@
389 gzip_step(tball.zSpaces, 512);
390 gzip_step(tball.zSpaces, 512);
391 gzip_finish(pOut);
392 fossil_free(tball.aHdr);
393 tball.aHdr = 0;
394 fossil_free(tball.zPrevDir);
395 tball.zPrevDir = NULL;
396 tball.nPrevDirAlloc = 0;
397 blob_reset(&tball.pax);
398 }
399
400
401 /*
402 ** COMMAND: test-tarball
403
M src/timeline.c
+76 -15
--- src/timeline.c
+++ src/timeline.c
@@ -107,11 +107,55 @@
107107
 #define TIMELINE_LEAFONLY 0x0002 /* Show "Leaf", but not "Merge", "Fork" etc */
108108
 #define TIMELINE_BRIEF 0x0004 /* Combine adjacent elements of same object */
109109
 #define TIMELINE_GRAPH 0x0008 /* Compute a graph */
110110
 #define TIMELINE_DISJOINT 0x0010 /* Elements are not contiguous */
111111
 #define TIMELINE_FCHANGES 0x0020 /* Detail file changes */
112
+#define TIMELINE_BRCOLOR 0x0040 /* Background color by branch name */
113
+#define TIMELINE_UCOLOR 0x0080 /* Background color by user */
112114
 #endif
115
+
116
+/*
117
+** Hash a string and use the hash to determine a background color.
118
+*/
119
+char *hash_color(const char *z){
120
+ int i; /* Loop counter */
121
+ unsigned int h = 0; /* Hash on the branch name */
122
+ int r, g, b; /* Values for red, green, and blue */
123
+ int h1, h2, h3, h4; /* Elements of the hash value */
124
+ int mx, mn; /* Components of HSV */
125
+ static char zColor[10]; /* The resulting color */
126
+ static int ix[2] = {0,0}; /* Color chooser parameters */
127
+
128
+ if( ix[0]==0 ){
129
+ if( db_get_boolean("white-foreground", 0) ){
130
+ ix[0] = 140;
131
+ ix[1] = 40;
132
+ }else{
133
+ ix[0] = 216;
134
+ ix[1] = 16;
135
+ }
136
+ }
137
+ for(i=0; z[i]; i++ ){
138
+ h = (h<<11) ^ (h<<1) ^ (h>>3) ^ z[i];
139
+ }
140
+ h1 = h % 6; h /= 6;
141
+ h3 = h % 30; h /= 30;
142
+ h4 = h % 40; h /= 40;
143
+ mx = ix[0] - h3;
144
+ mn = mx - h4 - ix[1];
145
+ h2 = (h%(mx - mn)) + mn;
146
+ switch( h1 ){
147
+ case 0: r = mx; g = h2, b = mn; break;
148
+ case 1: r = h2; g = mx, b = mn; break;
149
+ case 2: r = mn; g = mx, b = h2; break;
150
+ case 3: r = mn; g = h2, b = mx; break;
151
+ case 4: r = h2; g = mn, b = mx; break;
152
+ default: r = mx; g = mn, b = h2; break;
153
+ }
154
+ sqlite3_snprintf(8, zColor, "#%02x%02x%02x", r,g,b);
155
+ return zColor;
156
+}
113157
114158
 /*
115159
 ** Output a timeline in the web format given a query. The query
116160
 ** should return these columns:
117161
 **
@@ -142,10 +186,11 @@
142186
 char zPrevDate[20];
143187
 GraphContext *pGraph = 0;
144188
 int prevWasDivider = 0; /* True if previous output row was <hr> */
145189
 int fchngQueryInit = 0; /* True if fchngQuery is initialized */
146190
 Stmt fchngQuery; /* Query for file changes on check-ins */
191
+ static Stmt qbranch;
147192
148193
 zPrevDate[0] = 0;
149194
 mxWikiLen = db_get_int("timeline-max-comment", 0);
150195
 if( db_get_boolean("timeline-block-markup", 0) ){
151196
 wikiFlags = WIKI_INLINE;
@@ -157,10 +202,14 @@
157202
 /* style is not moved to css, because this is
158203
 ** a technical div for the timeline graph
159204
 */
160205
 @ <div id="canvas" style="position:relative;width:1px;height:1px;"></div>
161206
 }
207
+ db_static_prepare(&qbranch,
208
+ "SELECT value FROM tagxref WHERE tagid=%d AND tagtype>0 AND rid=:rid",
209
+ TAG_BRANCH
210
+ );
162211
163212
 @ <table id="timelineTable" class="timelineTable">
164213
 blob_zero(&comment);
165214
 while( db_step(pQuery)==SQLITE_ROW ){
166215
 int rid = db_column_int(pQuery, 0);
@@ -170,10 +219,11 @@
170219
 const char *zDate = db_column_text(pQuery, 2);
171220
 const char *zType = db_column_text(pQuery, 7);
172221
 const char *zUser = db_column_text(pQuery, 4);
173222
 const char *zTagList = db_column_text(pQuery, 8);
174223
 int tagid = db_column_int(pQuery, 9);
224
+ const char *zBr = 0; /* Branch */
175225
 int commentColumn = 3; /* Column containing comment text */
176226
 char zTime[8];
177227
 if( tagid ){
178228
 if( tagid==prevTagid ){
179229
 if( tmFlags & TIMELINE_BRIEF ){
@@ -208,37 +258,44 @@
208258
 memcpy(zTime, &zDate[11], 5);
209259
 zTime[5] = 0;
210260
 @ <tr>
211261
 @ <td class="timelineTime">%s(zTime)</td>
212262
 @ <td class="timelineGraph">
213
- if( pGraph && zType[0]=='c' ){
263
+ if( tmFlags & TIMELINE_UCOLOR ) zBgClr = zUser ? hash_color(zUser) : 0;
264
+ if( zType[0]=='c'
265
+ && (pGraph || zBgClr==0 || (tmFlags & TIMELINE_BRCOLOR)!=0)
266
+ ){
267
+ db_reset(&qbranch);
268
+ db_bind_int(&qbranch, ":rid", rid);
269
+ if( db_step(&qbranch)==SQLITE_ROW ){
270
+ zBr = db_column_text(&qbranch, 0);
271
+ }else{
272
+ zBr = "trunk";
273
+ }
274
+ if( zBgClr==0 || (tmFlags & TIMELINE_BRCOLOR)!=0 ){
275
+ if( zBr==0 || strcmp(zBr,"trunk")==0 ){
276
+ zBgClr = 0;
277
+ }else{
278
+ zBgClr = hash_color(zBr);
279
+ }
280
+ }
281
+ }
282
+ if( zType[0]=='c' && (pGraph || (tmFlags & TIMELINE_BRCOLOR)!=0) ){
214283
 int nParent = 0;
215284
 int aParent[32];
216
- const char *zBr;
217285
 int gidx;
218286
 static Stmt qparent;
219
- static Stmt qbranch;
220287
 db_static_prepare(&qparent,
221288
 "SELECT pid FROM plink"
222289
 " WHERE cid=:rid AND pid NOT IN phantom"
223290
 " ORDER BY isprim DESC /*sort*/"
224291
 );
225
- db_static_prepare(&qbranch,
226
- "SELECT value FROM tagxref WHERE tagid=%d AND tagtype>0 AND rid=:rid",
227
- TAG_BRANCH
228
- );
229292
 db_bind_int(&qparent, ":rid", rid);
230293
 while( db_step(&qparent)==SQLITE_ROW && nParent<32 ){
231294
 aParent[nParent++] = db_column_int(&qparent, 0);
232295
 }
233296
 db_reset(&qparent);
234
- db_bind_int(&qbranch, ":rid", rid);
235
- if( db_step(&qbranch)==SQLITE_ROW ){
236
- zBr = db_column_text(&qbranch, 0);
237
- }else{
238
- zBr = "trunk";
239
- }
240297
 gidx = graph_add_row(pGraph, rid, nParent, aParent, zBr, zBgClr, isLeaf);
241298
 db_reset(&qbranch);
242299
 @ <div id="m%d(gidx)"></div>
243300
 }
244301
 @</td>
@@ -333,11 +390,11 @@
333390
 " (SELECT name FROM filename WHERE fnid=mlink.fnid) AS name,"
334391
 " (SELECT uuid FROM blob WHERE rid=fid),"
335392
 " (SELECT uuid FROM blob WHERE rid=pid)"
336393
 " FROM mlink"
337394
 " WHERE mid=:mid AND pid!=fid"
338
- " ORDER BY 3"
395
+ " ORDER BY 3 /*sort*/"
339396
 );
340397
 fchngQueryInit = 1;
341398
 }
342399
 db_bind_int(&fchngQuery, ":mid", rid);
343400
 while( db_step(&fchngQuery)==SQLITE_ROW ){
@@ -397,11 +454,11 @@
397454
 /*
398455
 ** Generate all of the necessary javascript to generate a timeline
399456
 ** graph.
400457
 */
401458
 void timeline_output_graph_javascript(GraphContext *pGraph, int omitDescenders){
402
- if( pGraph && pGraph->nErr==0 ){
459
+ if( pGraph && pGraph->nErr==0 && pGraph->nRow>0 ){
403460
 GraphRow *pRow;
404461
 int i;
405462
 char cSep;
406463
 @ <script type="text/JavaScript">
407464
 @ /* <![CDATA[ */
@@ -768,10 +825,12 @@
768825
 ** fc Show details of files changed
769826
 ** f=RID Show family (immediate parents and children) of RID
770827
 ** from=RID Path from...
771828
 ** to=RID ... to this
772829
 ** nomerge ... avoid merge links on the path
830
+** brbg Background color from branch name
831
+** ubg Background color from user
773832
 **
774833
 ** p= and d= can appear individually or together. If either p= or d=
775834
 ** appear, then u=, y=, a=, and b= are ignored.
776835
 **
777836
 ** If a= and b= appear, only a= is used. If neither appear, the most
@@ -826,10 +885,12 @@
826885
 tmFlags = TIMELINE_GRAPH;
827886
 }
828887
 if( P("ng")!=0 || zSearch!=0 ){
829888
 tmFlags &= ~TIMELINE_GRAPH;
830889
 }
890
+ if( P("brbg")!=0 ) tmFlags |= TIMELINE_BRCOLOR;
891
+ if( P("ubg")!=0 ) tmFlags |= TIMELINE_UCOLOR;
831892
832893
 style_header("Timeline");
833894
 login_anonymous_available();
834895
 timeline_temp_table();
835896
 blob_zero(&sql);
836897
--- src/timeline.c
+++ src/timeline.c
@@ -107,11 +107,55 @@
107 #define TIMELINE_LEAFONLY 0x0002 /* Show "Leaf", but not "Merge", "Fork" etc */
108 #define TIMELINE_BRIEF 0x0004 /* Combine adjacent elements of same object */
109 #define TIMELINE_GRAPH 0x0008 /* Compute a graph */
110 #define TIMELINE_DISJOINT 0x0010 /* Elements are not contiguous */
111 #define TIMELINE_FCHANGES 0x0020 /* Detail file changes */
 
 
112 #endif
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
113
114 /*
115 ** Output a timeline in the web format given a query. The query
116 ** should return these columns:
117 **
@@ -142,10 +186,11 @@
142 char zPrevDate[20];
143 GraphContext *pGraph = 0;
144 int prevWasDivider = 0; /* True if previous output row was <hr> */
145 int fchngQueryInit = 0; /* True if fchngQuery is initialized */
146 Stmt fchngQuery; /* Query for file changes on check-ins */
 
147
148 zPrevDate[0] = 0;
149 mxWikiLen = db_get_int("timeline-max-comment", 0);
150 if( db_get_boolean("timeline-block-markup", 0) ){
151 wikiFlags = WIKI_INLINE;
@@ -157,10 +202,14 @@
157 /* style is not moved to css, because this is
158 ** a technical div for the timeline graph
159 */
160 @ <div id="canvas" style="position:relative;width:1px;height:1px;"></div>
161 }
 
 
 
 
162
163 @ <table id="timelineTable" class="timelineTable">
164 blob_zero(&comment);
165 while( db_step(pQuery)==SQLITE_ROW ){
166 int rid = db_column_int(pQuery, 0);
@@ -170,10 +219,11 @@
170 const char *zDate = db_column_text(pQuery, 2);
171 const char *zType = db_column_text(pQuery, 7);
172 const char *zUser = db_column_text(pQuery, 4);
173 const char *zTagList = db_column_text(pQuery, 8);
174 int tagid = db_column_int(pQuery, 9);
 
175 int commentColumn = 3; /* Column containing comment text */
176 char zTime[8];
177 if( tagid ){
178 if( tagid==prevTagid ){
179 if( tmFlags & TIMELINE_BRIEF ){
@@ -208,37 +258,44 @@
208 memcpy(zTime, &zDate[11], 5);
209 zTime[5] = 0;
210 @ <tr>
211 @ <td class="timelineTime">%s(zTime)</td>
212 @ <td class="timelineGraph">
213 if( pGraph && zType[0]=='c' ){
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
214 int nParent = 0;
215 int aParent[32];
216 const char *zBr;
217 int gidx;
218 static Stmt qparent;
219 static Stmt qbranch;
220 db_static_prepare(&qparent,
221 "SELECT pid FROM plink"
222 " WHERE cid=:rid AND pid NOT IN phantom"
223 " ORDER BY isprim DESC /*sort*/"
224 );
225 db_static_prepare(&qbranch,
226 "SELECT value FROM tagxref WHERE tagid=%d AND tagtype>0 AND rid=:rid",
227 TAG_BRANCH
228 );
229 db_bind_int(&qparent, ":rid", rid);
230 while( db_step(&qparent)==SQLITE_ROW && nParent<32 ){
231 aParent[nParent++] = db_column_int(&qparent, 0);
232 }
233 db_reset(&qparent);
234 db_bind_int(&qbranch, ":rid", rid);
235 if( db_step(&qbranch)==SQLITE_ROW ){
236 zBr = db_column_text(&qbranch, 0);
237 }else{
238 zBr = "trunk";
239 }
240 gidx = graph_add_row(pGraph, rid, nParent, aParent, zBr, zBgClr, isLeaf);
241 db_reset(&qbranch);
242 @ <div id="m%d(gidx)"></div>
243 }
244 @</td>
@@ -333,11 +390,11 @@
333 " (SELECT name FROM filename WHERE fnid=mlink.fnid) AS name,"
334 " (SELECT uuid FROM blob WHERE rid=fid),"
335 " (SELECT uuid FROM blob WHERE rid=pid)"
336 " FROM mlink"
337 " WHERE mid=:mid AND pid!=fid"
338 " ORDER BY 3"
339 );
340 fchngQueryInit = 1;
341 }
342 db_bind_int(&fchngQuery, ":mid", rid);
343 while( db_step(&fchngQuery)==SQLITE_ROW ){
@@ -397,11 +454,11 @@
397 /*
398 ** Generate all of the necessary javascript to generate a timeline
399 ** graph.
400 */
401 void timeline_output_graph_javascript(GraphContext *pGraph, int omitDescenders){
402 if( pGraph && pGraph->nErr==0 ){
403 GraphRow *pRow;
404 int i;
405 char cSep;
406 @ <script type="text/JavaScript">
407 @ /* <![CDATA[ */
@@ -768,10 +825,12 @@
768 ** fc Show details of files changed
769 ** f=RID Show family (immediate parents and children) of RID
770 ** from=RID Path from...
771 ** to=RID ... to this
772 ** nomerge ... avoid merge links on the path
 
 
773 **
774 ** p= and d= can appear individually or together. If either p= or d=
775 ** appear, then u=, y=, a=, and b= are ignored.
776 **
777 ** If a= and b= appear, only a= is used. If neither appear, the most
@@ -826,10 +885,12 @@
826 tmFlags = TIMELINE_GRAPH;
827 }
828 if( P("ng")!=0 || zSearch!=0 ){
829 tmFlags &= ~TIMELINE_GRAPH;
830 }
 
 
831
832 style_header("Timeline");
833 login_anonymous_available();
834 timeline_temp_table();
835 blob_zero(&sql);
836
--- src/timeline.c
+++ src/timeline.c
@@ -107,11 +107,55 @@
107 #define TIMELINE_LEAFONLY 0x0002 /* Show "Leaf", but not "Merge", "Fork" etc */
108 #define TIMELINE_BRIEF 0x0004 /* Combine adjacent elements of same object */
109 #define TIMELINE_GRAPH 0x0008 /* Compute a graph */
110 #define TIMELINE_DISJOINT 0x0010 /* Elements are not contiguous */
111 #define TIMELINE_FCHANGES 0x0020 /* Detail file changes */
112 #define TIMELINE_BRCOLOR 0x0040 /* Background color by branch name */
113 #define TIMELINE_UCOLOR 0x0080 /* Background color by user */
114 #endif
115
116 /*
117 ** Hash a string and use the hash to determine a background color.
118 */
119 char *hash_color(const char *z){
120 int i; /* Loop counter */
121 unsigned int h = 0; /* Hash on the branch name */
122 int r, g, b; /* Values for red, green, and blue */
123 int h1, h2, h3, h4; /* Elements of the hash value */
124 int mx, mn; /* Components of HSV */
125 static char zColor[10]; /* The resulting color */
126 static int ix[2] = {0,0}; /* Color chooser parameters */
127
128 if( ix[0]==0 ){
129 if( db_get_boolean("white-foreground", 0) ){
130 ix[0] = 140;
131 ix[1] = 40;
132 }else{
133 ix[0] = 216;
134 ix[1] = 16;
135 }
136 }
137 for(i=0; z[i]; i++ ){
138 h = (h<<11) ^ (h<<1) ^ (h>>3) ^ z[i];
139 }
140 h1 = h % 6; h /= 6;
141 h3 = h % 30; h /= 30;
142 h4 = h % 40; h /= 40;
143 mx = ix[0] - h3;
144 mn = mx - h4 - ix[1];
145 h2 = (h%(mx - mn)) + mn;
146 switch( h1 ){
147 case 0: r = mx; g = h2, b = mn; break;
148 case 1: r = h2; g = mx, b = mn; break;
149 case 2: r = mn; g = mx, b = h2; break;
150 case 3: r = mn; g = h2, b = mx; break;
151 case 4: r = h2; g = mn, b = mx; break;
152 default: r = mx; g = mn, b = h2; break;
153 }
154 sqlite3_snprintf(8, zColor, "#%02x%02x%02x", r,g,b);
155 return zColor;
156 }
157
158 /*
159 ** Output a timeline in the web format given a query. The query
160 ** should return these columns:
161 **
@@ -142,10 +186,11 @@
186 char zPrevDate[20];
187 GraphContext *pGraph = 0;
188 int prevWasDivider = 0; /* True if previous output row was <hr> */
189 int fchngQueryInit = 0; /* True if fchngQuery is initialized */
190 Stmt fchngQuery; /* Query for file changes on check-ins */
191 static Stmt qbranch;
192
193 zPrevDate[0] = 0;
194 mxWikiLen = db_get_int("timeline-max-comment", 0);
195 if( db_get_boolean("timeline-block-markup", 0) ){
196 wikiFlags = WIKI_INLINE;
@@ -157,10 +202,14 @@
202 /* style is not moved to css, because this is
203 ** a technical div for the timeline graph
204 */
205 @ <div id="canvas" style="position:relative;width:1px;height:1px;"></div>
206 }
207 db_static_prepare(&qbranch,
208 "SELECT value FROM tagxref WHERE tagid=%d AND tagtype>0 AND rid=:rid",
209 TAG_BRANCH
210 );
211
212 @ <table id="timelineTable" class="timelineTable">
213 blob_zero(&comment);
214 while( db_step(pQuery)==SQLITE_ROW ){
215 int rid = db_column_int(pQuery, 0);
@@ -170,10 +219,11 @@
219 const char *zDate = db_column_text(pQuery, 2);
220 const char *zType = db_column_text(pQuery, 7);
221 const char *zUser = db_column_text(pQuery, 4);
222 const char *zTagList = db_column_text(pQuery, 8);
223 int tagid = db_column_int(pQuery, 9);
224 const char *zBr = 0; /* Branch */
225 int commentColumn = 3; /* Column containing comment text */
226 char zTime[8];
227 if( tagid ){
228 if( tagid==prevTagid ){
229 if( tmFlags & TIMELINE_BRIEF ){
@@ -208,37 +258,44 @@
258 memcpy(zTime, &zDate[11], 5);
259 zTime[5] = 0;
260 @ <tr>
261 @ <td class="timelineTime">%s(zTime)</td>
262 @ <td class="timelineGraph">
263 if( tmFlags & TIMELINE_UCOLOR ) zBgClr = zUser ? hash_color(zUser) : 0;
264 if( zType[0]=='c'
265 && (pGraph || zBgClr==0 || (tmFlags & TIMELINE_BRCOLOR)!=0)
266 ){
267 db_reset(&qbranch);
268 db_bind_int(&qbranch, ":rid", rid);
269 if( db_step(&qbranch)==SQLITE_ROW ){
270 zBr = db_column_text(&qbranch, 0);
271 }else{
272 zBr = "trunk";
273 }
274 if( zBgClr==0 || (tmFlags & TIMELINE_BRCOLOR)!=0 ){
275 if( zBr==0 || strcmp(zBr,"trunk")==0 ){
276 zBgClr = 0;
277 }else{
278 zBgClr = hash_color(zBr);
279 }
280 }
281 }
282 if( zType[0]=='c' && (pGraph || (tmFlags & TIMELINE_BRCOLOR)!=0) ){
283 int nParent = 0;
284 int aParent[32];
 
285 int gidx;
286 static Stmt qparent;
 
287 db_static_prepare(&qparent,
288 "SELECT pid FROM plink"
289 " WHERE cid=:rid AND pid NOT IN phantom"
290 " ORDER BY isprim DESC /*sort*/"
291 );
 
 
 
 
292 db_bind_int(&qparent, ":rid", rid);
293 while( db_step(&qparent)==SQLITE_ROW && nParent<32 ){
294 aParent[nParent++] = db_column_int(&qparent, 0);
295 }
296 db_reset(&qparent);
 
 
 
 
 
 
297 gidx = graph_add_row(pGraph, rid, nParent, aParent, zBr, zBgClr, isLeaf);
298 db_reset(&qbranch);
299 @ <div id="m%d(gidx)"></div>
300 }
301 @</td>
@@ -333,11 +390,11 @@
390 " (SELECT name FROM filename WHERE fnid=mlink.fnid) AS name,"
391 " (SELECT uuid FROM blob WHERE rid=fid),"
392 " (SELECT uuid FROM blob WHERE rid=pid)"
393 " FROM mlink"
394 " WHERE mid=:mid AND pid!=fid"
395 " ORDER BY 3 /*sort*/"
396 );
397 fchngQueryInit = 1;
398 }
399 db_bind_int(&fchngQuery, ":mid", rid);
400 while( db_step(&fchngQuery)==SQLITE_ROW ){
@@ -397,11 +454,11 @@
454 /*
455 ** Generate all of the necessary javascript to generate a timeline
456 ** graph.
457 */
458 void timeline_output_graph_javascript(GraphContext *pGraph, int omitDescenders){
459 if( pGraph && pGraph->nErr==0 && pGraph->nRow>0 ){
460 GraphRow *pRow;
461 int i;
462 char cSep;
463 @ <script type="text/JavaScript">
464 @ /* <![CDATA[ */
@@ -768,10 +825,12 @@
825 ** fc Show details of files changed
826 ** f=RID Show family (immediate parents and children) of RID
827 ** from=RID Path from...
828 ** to=RID ... to this
829 ** nomerge ... avoid merge links on the path
830 ** brbg Background color from branch name
831 ** ubg Background color from user
832 **
833 ** p= and d= can appear individually or together. If either p= or d=
834 ** appear, then u=, y=, a=, and b= are ignored.
835 **
836 ** If a= and b= appear, only a= is used. If neither appear, the most
@@ -826,10 +885,12 @@
885 tmFlags = TIMELINE_GRAPH;
886 }
887 if( P("ng")!=0 || zSearch!=0 ){
888 tmFlags &= ~TIMELINE_GRAPH;
889 }
890 if( P("brbg")!=0 ) tmFlags |= TIMELINE_BRCOLOR;
891 if( P("ubg")!=0 ) tmFlags |= TIMELINE_UCOLOR;
892
893 style_header("Timeline");
894 login_anonymous_available();
895 timeline_temp_table();
896 blob_zero(&sql);
897
M src/tkt.c
+1 -1
--- src/tkt.c
+++ src/tkt.c
@@ -731,11 +731,11 @@
731731
 zUuid = PD("name","");
732732
 zTitle = mprintf("History Of Ticket %h", zUuid);
733733
 style_submenu_element("Status", "Status",
734734
 "%s/info/%s", g.zTop, zUuid);
735735
 style_submenu_element("Check-ins", "Check-ins",
736
- "%s/tkttimeline?name=%s?y=ci", g.zTop, zUuid);
736
+ "%s/tkttimeline?name=%s&amp;y=ci", g.zTop, zUuid);
737737
 style_submenu_element("Timeline", "Timeline",
738738
 "%s/tkttimeline?name=%s", g.zTop, zUuid);
739739
 style_header(zTitle);
740740
 free(zTitle);
741741
742742
--- src/tkt.c
+++ src/tkt.c
@@ -731,11 +731,11 @@
731 zUuid = PD("name","");
732 zTitle = mprintf("History Of Ticket %h", zUuid);
733 style_submenu_element("Status", "Status",
734 "%s/info/%s", g.zTop, zUuid);
735 style_submenu_element("Check-ins", "Check-ins",
736 "%s/tkttimeline?name=%s?y=ci", g.zTop, zUuid);
737 style_submenu_element("Timeline", "Timeline",
738 "%s/tkttimeline?name=%s", g.zTop, zUuid);
739 style_header(zTitle);
740 free(zTitle);
741
742
--- src/tkt.c
+++ src/tkt.c
@@ -731,11 +731,11 @@
731 zUuid = PD("name","");
732 zTitle = mprintf("History Of Ticket %h", zUuid);
733 style_submenu_element("Status", "Status",
734 "%s/info/%s", g.zTop, zUuid);
735 style_submenu_element("Check-ins", "Check-ins",
736 "%s/tkttimeline?name=%s&amp;y=ci", g.zTop, zUuid);
737 style_submenu_element("Timeline", "Timeline",
738 "%s/tkttimeline?name=%s", g.zTop, zUuid);
739 style_header(zTitle);
740 free(zTitle);
741
742
M src/update.c
+57
--- src/update.c
+++ src/update.c
@@ -115,10 +115,13 @@
115115
 }
116116
 if( !nochangeFlag && db_exists("SELECT 1 FROM vmerge") ){
117117
 fossil_fatal("cannot update an uncommitted merge");
118118
 }
119119
 if( !nochangeFlag && !internalUpdate ) autosync(AUTOSYNC_PULL);
120
+
121
+ /* Create any empty directories now, as well as after the update, so changes in settings are reflected now */
122
+ ensure_empty_dirs_created();
120123
121124
 if( internalUpdate ){
122125
 tid = internalUpdate;
123126
 }else if( g.argc>=3 ){
124127
 if( fossil_strcmp(g.argv[2], "current")==0 ){
@@ -441,10 +444,11 @@
441444
 ** Clean up the mid and pid VFILE entries. Then commit the changes.
442445
 */
443446
 if( nochangeFlag ){
444447
 db_end_transaction(1); /* With --nochange, rollback changes */
445448
 }else{
449
+ ensure_empty_dirs_created();
446450
 if( g.argc<=3 ){
447451
 /* All files updated. Shift the current checkout to the target. */
448452
 db_multi_exec("DELETE FROM vfile WHERE vid!=%d", tid);
449453
 checkout_set_all_exe(tid);
450454
 manifest_to_disk(tid);
@@ -456,10 +460,63 @@
456460
 }
457461
 if( !internalUpdate ) undo_finish();
458462
 db_end_transaction(0);
459463
 }
460464
 }
465
+
466
+/*
467
+** Make sure empty directories are created
468
+*/
469
+void ensure_empty_dirs_created(void){
470
+ /* Make empty directories? */
471
+ char *zEmptyDirs = db_get("empty-dirs", 0);
472
+ if( zEmptyDirs!=0 ){
473
+ char *bc;
474
+ Blob dirName;
475
+ Blob dirsList;
476
+
477
+ blob_zero(&dirsList);
478
+ blob_init(&dirsList, zEmptyDirs, strlen(zEmptyDirs));
479
+ /* Replace commas by spaces */
480
+ bc = blob_str(&dirsList);
481
+ while( (*bc)!='\0' ){
482
+ if( (*bc)==',' ) { *bc = ' '; }
483
+ ++bc;
484
+ }
485
+ /* Make directories */
486
+ blob_zero(&dirName);
487
+ while( blob_token(&dirsList, &dirName) ){
488
+ const char *zDir = blob_str(&dirName);
489
+ /* Make full pathname of the directory */
490
+ Blob path;
491
+ const char *zPath;
492
+
493
+ blob_zero(&path);
494
+ blob_appendf(&path, "%s/%s", g.zLocalRoot, zDir);
495
+ zPath = blob_str(&path);
496
+ /* Handle various cases of existence of the directory */
497
+ switch( file_isdir(zPath) ){
498
+ case 0: { /* doesn't exist */
499
+ if( file_mkdir(zPath, 0)!=0 ) {
500
+ fossil_warning("couldn't create directory %s as "
501
+ "required by empty-dirs setting", zDir);
502
+ }
503
+ break;
504
+ }
505
+ case 1: { /* exists, and is a directory */
506
+ /* do nothing - required directory exists already */
507
+ break;
508
+ }
509
+ case 2: { /* exists, but isn't a directory */
510
+ fossil_warning("file %s found, but a directory is required "
511
+ "by empty-dirs setting", zDir);
512
+ }
513
+ }
514
+ blob_reset(&path);
515
+ }
516
+ }
517
+}
461518
462519
463520
 /*
464521
 ** Get the contents of a file within the checking "revision". If
465522
 ** revision==NULL then get the file content for the current checkout.
466523
--- src/update.c
+++ src/update.c
@@ -115,10 +115,13 @@
115 }
116 if( !nochangeFlag && db_exists("SELECT 1 FROM vmerge") ){
117 fossil_fatal("cannot update an uncommitted merge");
118 }
119 if( !nochangeFlag && !internalUpdate ) autosync(AUTOSYNC_PULL);
 
 
 
120
121 if( internalUpdate ){
122 tid = internalUpdate;
123 }else if( g.argc>=3 ){
124 if( fossil_strcmp(g.argv[2], "current")==0 ){
@@ -441,10 +444,11 @@
441 ** Clean up the mid and pid VFILE entries. Then commit the changes.
442 */
443 if( nochangeFlag ){
444 db_end_transaction(1); /* With --nochange, rollback changes */
445 }else{
 
446 if( g.argc<=3 ){
447 /* All files updated. Shift the current checkout to the target. */
448 db_multi_exec("DELETE FROM vfile WHERE vid!=%d", tid);
449 checkout_set_all_exe(tid);
450 manifest_to_disk(tid);
@@ -456,10 +460,63 @@
456 }
457 if( !internalUpdate ) undo_finish();
458 db_end_transaction(0);
459 }
460 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
461
462
463 /*
464 ** Get the contents of a file within the checking "revision". If
465 ** revision==NULL then get the file content for the current checkout.
466
--- src/update.c
+++ src/update.c
@@ -115,10 +115,13 @@
115 }
116 if( !nochangeFlag && db_exists("SELECT 1 FROM vmerge") ){
117 fossil_fatal("cannot update an uncommitted merge");
118 }
119 if( !nochangeFlag && !internalUpdate ) autosync(AUTOSYNC_PULL);
120
121 /* Create any empty directories now, as well as after the update, so changes in settings are reflected now */
122 ensure_empty_dirs_created();
123
124 if( internalUpdate ){
125 tid = internalUpdate;
126 }else if( g.argc>=3 ){
127 if( fossil_strcmp(g.argv[2], "current")==0 ){
@@ -441,10 +444,11 @@
444 ** Clean up the mid and pid VFILE entries. Then commit the changes.
445 */
446 if( nochangeFlag ){
447 db_end_transaction(1); /* With --nochange, rollback changes */
448 }else{
449 ensure_empty_dirs_created();
450 if( g.argc<=3 ){
451 /* All files updated. Shift the current checkout to the target. */
452 db_multi_exec("DELETE FROM vfile WHERE vid!=%d", tid);
453 checkout_set_all_exe(tid);
454 manifest_to_disk(tid);
@@ -456,10 +460,63 @@
460 }
461 if( !internalUpdate ) undo_finish();
462 db_end_transaction(0);
463 }
464 }
465
466 /*
467 ** Make sure empty directories are created
468 */
469 void ensure_empty_dirs_created(void){
470 /* Make empty directories? */
471 char *zEmptyDirs = db_get("empty-dirs", 0);
472 if( zEmptyDirs!=0 ){
473 char *bc;
474 Blob dirName;
475 Blob dirsList;
476
477 blob_zero(&dirsList);
478 blob_init(&dirsList, zEmptyDirs, strlen(zEmptyDirs));
479 /* Replace commas by spaces */
480 bc = blob_str(&dirsList);
481 while( (*bc)!='\0' ){
482 if( (*bc)==',' ) { *bc = ' '; }
483 ++bc;
484 }
485 /* Make directories */
486 blob_zero(&dirName);
487 while( blob_token(&dirsList, &dirName) ){
488 const char *zDir = blob_str(&dirName);
489 /* Make full pathname of the directory */
490 Blob path;
491 const char *zPath;
492
493 blob_zero(&path);
494 blob_appendf(&path, "%s/%s", g.zLocalRoot, zDir);
495 zPath = blob_str(&path);
496 /* Handle various cases of existence of the directory */
497 switch( file_isdir(zPath) ){
498 case 0: { /* doesn't exist */
499 if( file_mkdir(zPath, 0)!=0 ) {
500 fossil_warning("couldn't create directory %s as "
501 "required by empty-dirs setting", zDir);
502 }
503 break;
504 }
505 case 1: { /* exists, and is a directory */
506 /* do nothing - required directory exists already */
507 break;
508 }
509 case 2: { /* exists, but isn't a directory */
510 fossil_warning("file %s found, but a directory is required "
511 "by empty-dirs setting", zDir);
512 }
513 }
514 blob_reset(&path);
515 }
516 }
517 }
518
519
520 /*
521 ** Get the contents of a file within the checking "revision". If
522 ** revision==NULL then get the file content for the current checkout.
523
M src/wiki.c
+8 -20
--- src/wiki.c
+++ src/wiki.c
@@ -182,12 +182,11 @@
182182
 " ORDER BY mtime DESC", zTag
183183
 );
184184
 free(zTag);
185185
 pWiki = manifest_get(rid, CFTYPE_WIKI);
186186
 if( pWiki ){
187
- while( fossil_isspace(pWiki->zWiki[0]) ) pWiki->zWiki++;
188
- if( pWiki->zWiki[0] ) zBody = pWiki->zWiki;
187
+ zBody = pWiki->zWiki;
189188
 }
190189
 }
191190
 if( !g.isHome ){
192191
 if( (rid && g.okWrWiki) || (!rid && g.okNewWiki) ){
193192
 style_submenu_element("Edit", "Edit Wiki Page", "%s/wikiedit?name=%T",
@@ -891,10 +890,11 @@
891890
 char const *zPageName; /* Name of the wiki page to export */
892891
 char const *zFile; /* Name of the output file (0=stdout) */
893892
 int rid; /* Artifact ID of the wiki page */
894893
 int i; /* Loop counter */
895894
 char *zBody = 0; /* Wiki page content */
895
+ Blob body; /* Wiki page content */
896896
 Manifest *pWiki = 0; /* Parsed wiki page content */
897897
898898
 if( (g.argc!=4) && (g.argc!=5) ){
899899
 usage("export PAGENAME ?FILE?");
900900
 }
@@ -909,28 +909,16 @@
909909
 }
910910
 if( zBody==0 ){
911911
 fossil_fatal("wiki page [%s] not found",zPageName);
912912
 }
913913
 for(i=strlen(zBody); i>0 && fossil_isspace(zBody[i-1]); i--){}
914
- zFile = (g.argc==4) ? 0 : g.argv[4];
915
- if( zFile ){
916
- FILE * zF;
917
- short doClose = 0;
918
- if( (1 == strlen(zFile)) && ('-'==zFile[0]) ){
919
- zF = stdout;
920
- }else{
921
- zF = fossil_fopen( zFile, "w" );
922
- doClose = zF ? 1 : 0;
923
- }
924
- if( ! zF ){
925
- fossil_fatal("wiki export could not open output file for writing.");
926
- }
927
- fprintf(zF,"%.*s\n", i, zBody);
928
- if( doClose ) fclose(zF);
929
- }else{
930
- fossil_print("%.*s\n", i, zBody);
931
- }
914
+ zBody[i] = 0;
915
+ zFile = (g.argc==4) ? "-" : g.argv[4];
916
+ blob_init(&body, zBody, -1);
917
+ blob_append(&body, "\n", 1);
918
+ blob_write_to_file(&body, zFile);
919
+ blob_reset(&body);
932920
 manifest_destroy(pWiki);
933921
 return;
934922
 }else
935923
 if( strncmp(g.argv[2],"commit",n)==0
936924
 || strncmp(g.argv[2],"create",n)==0 ){
937925
--- src/wiki.c
+++ src/wiki.c
@@ -182,12 +182,11 @@
182 " ORDER BY mtime DESC", zTag
183 );
184 free(zTag);
185 pWiki = manifest_get(rid, CFTYPE_WIKI);
186 if( pWiki ){
187 while( fossil_isspace(pWiki->zWiki[0]) ) pWiki->zWiki++;
188 if( pWiki->zWiki[0] ) zBody = pWiki->zWiki;
189 }
190 }
191 if( !g.isHome ){
192 if( (rid && g.okWrWiki) || (!rid && g.okNewWiki) ){
193 style_submenu_element("Edit", "Edit Wiki Page", "%s/wikiedit?name=%T",
@@ -891,10 +890,11 @@
891 char const *zPageName; /* Name of the wiki page to export */
892 char const *zFile; /* Name of the output file (0=stdout) */
893 int rid; /* Artifact ID of the wiki page */
894 int i; /* Loop counter */
895 char *zBody = 0; /* Wiki page content */
 
896 Manifest *pWiki = 0; /* Parsed wiki page content */
897
898 if( (g.argc!=4) && (g.argc!=5) ){
899 usage("export PAGENAME ?FILE?");
900 }
@@ -909,28 +909,16 @@
909 }
910 if( zBody==0 ){
911 fossil_fatal("wiki page [%s] not found",zPageName);
912 }
913 for(i=strlen(zBody); i>0 && fossil_isspace(zBody[i-1]); i--){}
914 zFile = (g.argc==4) ? 0 : g.argv[4];
915 if( zFile ){
916 FILE * zF;
917 short doClose = 0;
918 if( (1 == strlen(zFile)) && ('-'==zFile[0]) ){
919 zF = stdout;
920 }else{
921 zF = fossil_fopen( zFile, "w" );
922 doClose = zF ? 1 : 0;
923 }
924 if( ! zF ){
925 fossil_fatal("wiki export could not open output file for writing.");
926 }
927 fprintf(zF,"%.*s\n", i, zBody);
928 if( doClose ) fclose(zF);
929 }else{
930 fossil_print("%.*s\n", i, zBody);
931 }
932 manifest_destroy(pWiki);
933 return;
934 }else
935 if( strncmp(g.argv[2],"commit",n)==0
936 || strncmp(g.argv[2],"create",n)==0 ){
937
--- src/wiki.c
+++ src/wiki.c
@@ -182,12 +182,11 @@
182 " ORDER BY mtime DESC", zTag
183 );
184 free(zTag);
185 pWiki = manifest_get(rid, CFTYPE_WIKI);
186 if( pWiki ){
187 zBody = pWiki->zWiki;
 
188 }
189 }
190 if( !g.isHome ){
191 if( (rid && g.okWrWiki) || (!rid && g.okNewWiki) ){
192 style_submenu_element("Edit", "Edit Wiki Page", "%s/wikiedit?name=%T",
@@ -891,10 +890,11 @@
890 char const *zPageName; /* Name of the wiki page to export */
891 char const *zFile; /* Name of the output file (0=stdout) */
892 int rid; /* Artifact ID of the wiki page */
893 int i; /* Loop counter */
894 char *zBody = 0; /* Wiki page content */
895 Blob body; /* Wiki page content */
896 Manifest *pWiki = 0; /* Parsed wiki page content */
897
898 if( (g.argc!=4) && (g.argc!=5) ){
899 usage("export PAGENAME ?FILE?");
900 }
@@ -909,28 +909,16 @@
909 }
910 if( zBody==0 ){
911 fossil_fatal("wiki page [%s] not found",zPageName);
912 }
913 for(i=strlen(zBody); i>0 && fossil_isspace(zBody[i-1]); i--){}
914 zBody[i] = 0;
915 zFile = (g.argc==4) ? "-" : g.argv[4];
916 blob_init(&body, zBody, -1);
917 blob_append(&body, "\n", 1);
918 blob_write_to_file(&body, zFile);
919 blob_reset(&body);
 
 
 
 
 
 
 
 
 
 
 
 
920 manifest_destroy(pWiki);
921 return;
922 }else
923 if( strncmp(g.argv[2],"commit",n)==0
924 || strncmp(g.argv[2],"create",n)==0 ){
925
M src/winhttp.c
+644 -9
--- src/winhttp.c
+++ src/winhttp.c
@@ -14,17 +14,18 @@
1414
 ** http://www.hwaci.com/drh/
1515
 **
1616
 *******************************************************************************
1717
 **
1818
 ** This file implements a very simple (and low-performance) HTTP server
19
-** for windows.
19
+** for windows. It also implements a Windows Service which allows the HTTP
20
+** server to be run without any user logged on.
2021
 */
2122
 #include "config.h"
2223
 #ifdef _WIN32
2324
 /* This code is for win32 only */
24
-#include "winhttp.h"
2525
 #include <windows.h>
26
+#include "winhttp.h"
2627
2728
 /*
2829
 ** The HttpRequest structure holds information about each incoming
2930
 ** HTTP request.
3031
 */
@@ -60,11 +61,11 @@
6061
 }
6162
6263
 /*
6364
 ** Process a single incoming HTTP request.
6465
 */
65
-void win32_process_one_http_request(void *pAppData){
66
+static void win32_process_one_http_request(void *pAppData){
6667
 HttpRequest *p = (HttpRequest*)pAppData;
6768
 FILE *in = 0, *out = 0;
6869
 int amt, got;
6970
 int wanted = 0;
7071
 char *z;
@@ -145,10 +146,11 @@
145146
 SOCKET s = INVALID_SOCKET;
146147
 SOCKADDR_IN addr;
147148
 int idCnt = 0;
148149
 int iPort = mnPort;
149150
 Blob options;
151
+ char zTmpPath[MAX_PATH];
150152
151153
 if( zStopper ) file_delete(zStopper);
152154
 blob_zero(&options);
153155
 if( zNotFound ){
154156
 blob_appendf(&options, " --notfound %s", zNotFound);
@@ -189,31 +191,46 @@
189191
 }else{
190192
 fossil_fatal("unable to open listening socket on any"
191193
 " port in the range %d..%d", mnPort, mxPort);
192194
 }
193195
 }
194
- zTempPrefix = mprintf("fossil_server_P%d_", iPort);
196
+ if( !GetTempPath(sizeof(zTmpPath), zTmpPath) ){
197
+ fossil_fatal("unable to get path to the temporary directory.");
198
+ }
199
+ zTempPrefix = mprintf("%sfossil_server_P%d_", zTmpPath, iPort);
195200
 fossil_print("Listening for HTTP requests on TCP port %d\n", iPort);
196201
 if( zBrowser ){
197202
 zBrowser = mprintf(zBrowser, iPort);
198203
 fossil_print("Launch webbrowser: %s\n", zBrowser);
199204
 fossil_system(zBrowser);
200205
 }
201206
 fossil_print("Type Ctrl-C to stop the HTTP server\n");
207
+ /* Set the service status to running and pass the listener socket to the
208
+ ** service handling procedures. */
209
+ win32_http_service_running(s);
202210
 for(;;){
203211
 SOCKET client;
204212
 SOCKADDR_IN client_addr;
205213
 HttpRequest *p;
206214
 int len = sizeof(client_addr);
215
+ int wsaError;
207216
208217
 client = accept(s, (struct sockaddr*)&client_addr, &len);
209
- if( zStopper && file_size(zStopper)>=0 ){
210
- break;
211
- }
212218
 if( client==INVALID_SOCKET ){
213
- closesocket(s);
214
- fossil_fatal("error from accept()");
219
+ /* If the service control handler has closed the listener socket,
220
+ ** cleanup and return, otherwise report a fatal error. */
221
+ wsaError = WSAGetLastError();
222
+ if( (wsaError==WSAEINTR) || (wsaError==WSAENOTSOCK) ){
223
+ WSACleanup();
224
+ return;
225
+ }else{
226
+ closesocket(s);
227
+ WSACleanup();
228
+ fossil_fatal("error from accept()");
229
+ }
230
+ }else if( zStopper && file_size(zStopper)>=0 ){
231
+ break;
215232
 }
216233
 p = fossil_malloc( sizeof(*p) );
217234
 p->id = ++idCnt;
218235
 p->s = client;
219236
 p->addr = client_addr;
@@ -221,7 +238,625 @@
221238
 _beginthread(win32_process_one_http_request, 0, (void*)p);
222239
 }
223240
 closesocket(s);
224241
 WSACleanup();
225242
 }
243
+
244
+/*
245
+** The HttpService structure is used to pass information to the service main
246
+** function and to the service control handler function.
247
+*/
248
+typedef struct HttpService HttpService;
249
+struct HttpService {
250
+ int port; /* Port on which the http server should run */
251
+ const char *zNotFound; /* The --notfound option, or NULL */
252
+ int flags; /* One or more HTTP_SERVER_ flags */
253
+ int isRunningAsService; /* Are we running as a service ? */
254
+ const char *zServiceName; /* Name of the service */
255
+ SOCKET s; /* Socket on which the http server listens */
256
+};
257
+
258
+/*
259
+** Variables used for running as windows service.
260
+*/
261
+static HttpService hsData = {8080, NULL, 0, 0, NULL, INVALID_SOCKET};
262
+static SERVICE_STATUS ssStatus;
263
+static SERVICE_STATUS_HANDLE sshStatusHandle;
264
+
265
+/*
266
+** Get message string of the last system error. Return a pointer to the
267
+** message string. Call fossil_mbcs_free() to deallocate any memory used
268
+** to store the message string when done.
269
+*/
270
+static char *win32_get_last_errmsg(void){
271
+ DWORD nMsg;
272
+ LPTSTR tmp = NULL;
273
+ char *zMsg = NULL;
274
+
275
+ nMsg = FormatMessage(
276
+ FORMAT_MESSAGE_ALLOCATE_BUFFER |
277
+ FORMAT_MESSAGE_FROM_SYSTEM |
278
+ FORMAT_MESSAGE_IGNORE_INSERTS,
279
+ NULL,
280
+ GetLastError(),
281
+ MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US),
282
+ (LPTSTR) &tmp,
283
+ 0,
284
+ NULL
285
+ );
286
+ if( nMsg ){
287
+ zMsg = fossil_mbcs_to_utf8(tmp);
288
+ }else{
289
+ fossil_fatal("unable to get system error message.");
290
+ }
291
+ if( tmp ){
292
+ LocalFree((HLOCAL) tmp);
293
+ }
294
+ return zMsg;
295
+}
296
+
297
+/*
298
+** Report the current status of the service to the service control manager.
299
+** Make sure that during service startup no control codes are accepted.
300
+*/
301
+static void win32_report_service_status(
302
+ DWORD dwCurrentState, /* The current state of the service */
303
+ DWORD dwWin32ExitCode, /* The error code to report */
304
+ DWORD dwWaitHint /* The estimated time for a pending operation */
305
+){
306
+ if( dwCurrentState==SERVICE_START_PENDING) {
307
+ ssStatus.dwControlsAccepted = 0;
308
+ }else{
309
+ ssStatus.dwControlsAccepted = SERVICE_ACCEPT_STOP;
310
+ }
311
+ ssStatus.dwCurrentState = dwCurrentState;
312
+ ssStatus.dwWin32ExitCode = dwWin32ExitCode;
313
+ ssStatus.dwWaitHint = dwWaitHint;
314
+
315
+ if( (dwCurrentState==SERVICE_RUNNING) ||
316
+ (dwCurrentState==SERVICE_STOPPED) ){
317
+ ssStatus.dwCheckPoint = 0;
318
+ }else{
319
+ ssStatus.dwCheckPoint++;
320
+ }
321
+ SetServiceStatus(sshStatusHandle, &ssStatus);
322
+ return ;
323
+}
324
+
325
+/*
326
+** Handle control codes sent from the service control manager.
327
+** The control dispatcher in the main thread of the service process invokes
328
+** this function whenever it receives a control request from the service
329
+** control manager.
330
+*/
331
+static void WINAPI win32_http_service_ctrl(
332
+ DWORD dwCtrlCode
333
+){
334
+ switch( dwCtrlCode ){
335
+ case SERVICE_CONTROL_STOP: {
336
+ win32_report_service_status(SERVICE_STOP_PENDING, NO_ERROR, 0);
337
+ if( hsData.s != INVALID_SOCKET ){
338
+ closesocket(hsData.s);
339
+ }
340
+ win32_report_service_status(ssStatus.dwCurrentState, NO_ERROR, 0);
341
+ break;
342
+ }
343
+ default: {
344
+ break;
345
+ }
346
+ }
347
+ return;
348
+}
349
+
350
+/*
351
+** This is the main entry point for the service.
352
+** When the service control manager receives a request to start the service,
353
+** it starts the service process (if it is not already running). The main
354
+** thread of the service process calls the StartServiceCtrlDispatcher
355
+** function with a pointer to an array of SERVICE_TABLE_ENTRY structures.
356
+** Then the service control manager sends a start request to the service
357
+** control dispatcher for this service process. The service control dispatcher
358
+** creates a new thread to execute the ServiceMain function (this function)
359
+** of the service being started.
360
+*/
361
+static void WINAPI win32_http_service_main(
362
+ DWORD argc, /* Number of arguments in argv */
363
+ LPTSTR *argv /* Arguments passed */
364
+){
365
+
366
+ /* Update the service information. */
367
+ hsData.isRunningAsService = 1;
368
+ if( argc>0 ){
369
+ hsData.zServiceName = argv[0];
370
+ }
371
+
372
+ /* Register the service control handler function */
373
+ sshStatusHandle = RegisterServiceCtrlHandler("", win32_http_service_ctrl);
374
+ if( !sshStatusHandle ){
375
+ win32_report_service_status(SERVICE_STOPPED, NO_ERROR, 0);
376
+ return;
377
+ }
378
+
379
+ /* Set service specific data and report that the service is starting. */
380
+ ssStatus.dwServiceType = SERVICE_WIN32_OWN_PROCESS;
381
+ ssStatus.dwServiceSpecificExitCode = 0;
382
+ win32_report_service_status(SERVICE_START_PENDING, NO_ERROR, 3000);
383
+
384
+ /* Execute the http server */
385
+ win32_http_server(hsData.port, hsData.port,
386
+ NULL, NULL, hsData.zNotFound, hsData.flags);
387
+
388
+ /* Service has stopped now. */
389
+ win32_report_service_status(SERVICE_STOPPED, NO_ERROR, 0);
390
+ return;
391
+}
392
+
393
+/*
394
+** When running as service, update the HttpService structure with the
395
+** listener socket and update the service status. This procedure must be
396
+** called from the http server when he is ready to accept connections.
397
+*/
398
+LOCAL void win32_http_service_running(SOCKET s){
399
+ if( hsData.isRunningAsService ){
400
+ hsData.s = s;
401
+ win32_report_service_status(SERVICE_RUNNING, NO_ERROR, 0);
402
+ }
403
+}
404
+
405
+/*
406
+** Try to start the http server as a windows service. If we are running in
407
+** a interactive console session, this routine fails and returns a non zero
408
+** integer value. When running as service, this routine does not return until
409
+** the service is stopped. In this case, the return value is zero.
410
+*/
411
+int win32_http_service(
412
+ int nPort, /* TCP port number */
413
+ const char *zNotFound, /* The --notfound option, or NULL */
414
+ int flags /* One or more HTTP_SERVER_ flags */
415
+){
416
+ /* Define the service table. */
417
+ SERVICE_TABLE_ENTRY ServiceTable[] =
418
+ {{"", (LPSERVICE_MAIN_FUNCTION)win32_http_service_main}, {NULL, NULL}};
419
+
420
+ /* Initialize the HttpService structure. */
421
+ hsData.port = nPort;
422
+ hsData.zNotFound = zNotFound;
423
+ hsData.flags = flags;
424
+
425
+ /* Try to start the control dispatcher thread for the service. */
426
+ if( !StartServiceCtrlDispatcher(ServiceTable) ){
427
+ if( GetLastError()==ERROR_FAILED_SERVICE_CONTROLLER_CONNECT ){
428
+ return 1;
429
+ }else{
430
+ fossil_fatal("error from StartServiceCtrlDispatcher()");
431
+ }
432
+ }
433
+ return 0;
434
+}
435
+
436
+/*
437
+** COMMAND: winsrv
438
+** Usage: fossil winsrv METHOD ?SERVICE-NAME? ?OPTIONS?
439
+**
440
+** Where METHOD is one of: create delete show start stop.
441
+**
442
+** The winsrv command manages Fossil as a Windows service. This allows
443
+** (for example) Fossil to be running in the background when no user
444
+** is logged in.
445
+**
446
+** In the following description of the methods, "Fossil-DSCM" will be
447
+** used as the default SERVICE-NAME:
448
+**
449
+** fossil winsrv create ?SERVICE-NAME? ?OPTIONS?
450
+**
451
+** Creates a service. Available options include:
452
+**
453
+** -D|--display DISPLAY-NAME
454
+**
455
+** Sets the display name of the service. This name is shown
456
+** by graphical interface programs. By default, the display name
457
+** equals to the service name.
458
+**
459
+** -S|--start TYPE
460
+**
461
+** Sets the start type of the service. TYPE can be "manual",
462
+** which means you need to start the service yourself with the
463
+** 'fossil winsrv start' command or with the "net start" command
464
+** from the operating system. If TYPE is set to "auto", the service
465
+** will be started automatically by the system during startup.
466
+**
467
+** -U|--username USERNAME
468
+**
469
+** Specifies the user account which will be used to run the
470
+** service. The account needs the "Logon as a service" right
471
+** enabled in its profile. Specify local accounts as follows:
472
+** ".\\USERNAME". By default, the "LocalSystem" account will be
473
+** used.
474
+**
475
+** -W|--password PASSWORD
476
+**
477
+** Password for the user account.
478
+**
479
+** The following options are more or less the same as for the "server"
480
+** command and influence the behaviour of the http server:
481
+**
482
+** -p|--port TCPPORT
483
+**
484
+** Specifies the TCP port (default port is 8080) on which the
485
+** server should listen.
486
+**
487
+** -R|--repository REPOSITORY
488
+**
489
+** Specifies the name of the repository to be served.
490
+** The repository option may be omitted if the working directory
491
+** is within an open checkout.
492
+** The REPOSITORY can be a directory (aka folder) that contains
493
+** one or more respositories with names ending in ".fossil".
494
+** In that case, the first element of the URL is used to select
495
+** among the various repositories.
496
+**
497
+** --notfound URL
498
+**
499
+** If REPOSITORY is a directory that contains one or more
500
+** respositories with names of the form "*.fossil" then the
501
+** first element of the URL pathname selects among the various
502
+** repositories. If the pathname does not select a valid
503
+** repository and the --notfound option is available,
504
+** then the server redirects (HTTP code 302) to the URL of
505
+** --notfound.
506
+**
507
+** --localauth
508
+**
509
+** Enables automatic login if the --localauth option is present
510
+** and the "localauth" setting is off and the connection is from
511
+** localhost.
512
+**
513
+**
514
+** fossil winsrv delete ?SERVICE-NAME?
515
+**
516
+** Deletes a service. If the service is currently running, it will be
517
+** stopped first and then deleted.
518
+**
519
+**
520
+** fossil winsrv show ?SERVICE-NAME?
521
+**
522
+** Shows how the service is configured and its current state.
523
+**
524
+**
525
+** fossil winsrv start ?SERVICE-NAME?
526
+**
527
+** Start the service.
528
+**
529
+**
530
+** fossil winsrv stop ?SERVICE-NAME?
531
+**
532
+** Stop the service.
533
+**
534
+**
535
+** NOTE: This command is available on Windows operating systems only and
536
+** requires administrative rights on the machine executed.
537
+**
538
+*/
539
+void cmd_win32_service(void){
540
+ int n;
541
+ const char *zMethod;
542
+ const char *zSvcName = "Fossil-DSCM"; /* Default service name */
543
+
544
+ if( g.argc<3 ){
545
+ usage("create|delete|show|start|stop ...");
546
+ }
547
+ zMethod = g.argv[2];
548
+ n = strlen(zMethod);
549
+ if( g.argc==4 ){
550
+ zSvcName = g.argv[3];
551
+ }
552
+
553
+ if( strncmp(zMethod, "create", n)==0 ){
554
+ SC_HANDLE hScm;
555
+ SC_HANDLE hSvc;
556
+ SERVICE_DESCRIPTION
557
+ svcDescr = {"Fossil - Distributed Software Configuration Management"};
558
+ char *zErrFmt = "unable to create service '%s': %s";
559
+ DWORD dwStartType = SERVICE_DEMAND_START;
560
+ const char *zDisplay;
561
+ const char *zStart;
562
+ const char *zUsername;
563
+ const char *zPassword;
564
+ const char *zPort;
565
+ const char *zNotFound;
566
+ const char *zLocalAuth;
567
+ const char *zRepository;
568
+ Blob binPath;
569
+
570
+ /* Process service creation specific options. */
571
+ zDisplay = find_option("display", "D", 1);
572
+ if( !zDisplay ){
573
+ zDisplay = zSvcName;
574
+ }
575
+ zStart = find_option("start", "S", 1);
576
+ if( zStart ){
577
+ if( strncmp(zStart, "auto", strlen(zStart))==0 ){
578
+ dwStartType = SERVICE_AUTO_START;
579
+ }else if( strncmp(zStart, "manual", strlen(zStart))==0 ){
580
+ dwStartType = SERVICE_DEMAND_START;
581
+ }else{
582
+ fossil_fatal(zErrFmt, zSvcName,
583
+ "specify 'auto' or 'manual' for the '-S|--start' option");
584
+ }
585
+ }
586
+ zUsername = find_option("username", "U", 1);
587
+ zPassword = find_option("password", "W", 1);
588
+ /* Process options for Fossil running as server. */
589
+ zPort = find_option("port", "P", 1);
590
+ if( zPort && (atoi(zPort)<=0) ){
591
+ fossil_fatal(zErrFmt, zSvcName,
592
+ "port number must be in the range 1 - 65535.");
593
+ }
594
+ zNotFound = find_option("notfound", 0, 1);
595
+ zLocalAuth = find_option("localauth", 0, 0);
596
+ zRepository = find_option("repository", "R", 1);
597
+ if( !zRepository ){
598
+ db_must_be_within_tree();
599
+ }else if( file_isdir(zRepository)==1 ){
600
+ g.zRepositoryName = mprintf("%s", zRepository);
601
+ file_simplify_name(g.zRepositoryName, -1);
602
+ }else{
603
+ db_open_repository(zRepository);
604
+ }
605
+ db_close(0);
606
+ verify_all_options();
607
+ if( g.argc>4 ) fossil_fatal("to much arguments for create method.");
608
+ /* Build the fully-qualified path to the service binary file. */
609
+ blob_zero(&binPath);
610
+ blob_appendf(&binPath, "\"%s\" server", fossil_nameofexe());
611
+ if( zPort ) blob_appendf(&binPath, " --port %s", zPort);
612
+ if( zNotFound ) blob_appendf(&binPath, " --notfound \"%s\"", zNotFound);
613
+ if( zLocalAuth ) blob_append(&binPath, " --localauth", -1);
614
+ blob_appendf(&binPath, " \"%s\"", g.zRepositoryName);
615
+ /* Create the service. */
616
+ hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
617
+ if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
618
+ hSvc = CreateService(
619
+ hScm, /* Handle to the SCM */
620
+ fossil_utf8_to_mbcs(zSvcName), /* Name of the service */
621
+ fossil_utf8_to_mbcs(zDisplay), /* Display name */
622
+ SERVICE_ALL_ACCESS, /* Desired access */
623
+ SERVICE_WIN32_OWN_PROCESS, /* Service type */
624
+ dwStartType, /* Start type */
625
+ SERVICE_ERROR_NORMAL, /* Error control */
626
+ fossil_utf8_to_mbcs(blob_str(&binPath)), /* Binary path */
627
+ NULL, /* Load ordering group */
628
+ NULL, /* Tag value */
629
+ NULL, /* Service dependencies */
630
+ fossil_utf8_to_mbcs(zUsername), /* Service account */
631
+ fossil_utf8_to_mbcs(zPassword) /* Account password */
632
+ );
633
+ if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
634
+ /* Set the service description. */
635
+ ChangeServiceConfig2(hSvc, SERVICE_CONFIG_DESCRIPTION, &svcDescr);
636
+ fossil_print("Service '%s' successfully created.\n", zSvcName);
637
+ CloseServiceHandle(hSvc);
638
+ CloseServiceHandle(hScm);
639
+ }else
640
+ if( strncmp(zMethod, "delete", n)==0 ){
641
+ SC_HANDLE hScm;
642
+ SC_HANDLE hSvc;
643
+ SERVICE_STATUS sstat;
644
+ char *zErrFmt = "unable to delete service '%s': %s";
645
+
646
+ verify_all_options();
647
+ if( g.argc>4 ) fossil_fatal("to much arguments for delete method.");
648
+ hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
649
+ if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
650
+ hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), SERVICE_ALL_ACCESS);
651
+ if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
652
+ QueryServiceStatus(hSvc, &sstat);
653
+ if( sstat.dwCurrentState!=SERVICE_STOPPED ){
654
+ fossil_print("Stopping service '%s'", zSvcName);
655
+ if( sstat.dwCurrentState!=SERVICE_STOP_PENDING ){
656
+ if( !ControlService(hSvc, SERVICE_CONTROL_STOP, &sstat) ){
657
+ fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
658
+ }
659
+ }
660
+ while( sstat.dwCurrentState!=SERVICE_STOPPED ){
661
+ Sleep(100);
662
+ fossil_print(".");
663
+ QueryServiceStatus(hSvc, &sstat);
664
+ }
665
+ fossil_print("\nService '%s' stopped.\n", zSvcName);
666
+ }
667
+ if( !DeleteService(hSvc) ){
668
+ if( GetLastError()==ERROR_SERVICE_MARKED_FOR_DELETE ){
669
+ fossil_warning("Service '%s' already marked for delete.\n", zSvcName);
670
+ }else{
671
+ fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
672
+ }
673
+ }else{
674
+ fossil_print("Service '%s' successfully deleted.\n", zSvcName);
675
+ }
676
+ CloseServiceHandle(hSvc);
677
+ CloseServiceHandle(hScm);
678
+ }else
679
+ if( strncmp(zMethod, "show", n)==0 ){
680
+ SC_HANDLE hScm;
681
+ SC_HANDLE hSvc;
682
+ SERVICE_STATUS sstat;
683
+ LPQUERY_SERVICE_CONFIG pSvcConfig;
684
+ LPSERVICE_DESCRIPTION pSvcDescr;
685
+ BOOL bStatus;
686
+ DWORD nRequired;
687
+ char *zErrFmt = "unable to show service '%s': %s";
688
+ static const char *zSvcTypes[] = {
689
+ "Driver service",
690
+ "File system driver service",
691
+ "Service runs in its own process",
692
+ "Service shares a process with other services",
693
+ "Service can interact with the desktop"
694
+ };
695
+ const char *zSvcType = "";
696
+ static char *zSvcStartTypes[] = {
697
+ "Started by the system loader",
698
+ "Started by the IoInitSystem function",
699
+ "Started automatically by the service control manager",
700
+ "Started manually",
701
+ "Service cannot be started"
702
+ };
703
+ const char *zSvcStartType = "";
704
+ static const char *zSvcStates[] = {
705
+ "Stopped", "Starting", "Stopping", "Running",
706
+ "Continue pending", "Pause pending", "Paused"
707
+ };
708
+ const char *zSvcState = "";
709
+
710
+ verify_all_options();
711
+ if( g.argc>4 ) fossil_fatal("to much arguments for show method.");
712
+ hScm = OpenSCManager(NULL, NULL, GENERIC_READ);
713
+ if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
714
+ hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), GENERIC_READ);
715
+ if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
716
+ /* Get the service configuration */
717
+ bStatus = QueryServiceConfig(hSvc, NULL, 0, &nRequired);
718
+ if( !bStatus && GetLastError()!=ERROR_INSUFFICIENT_BUFFER ){
719
+ fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
720
+ }
721
+ pSvcConfig = fossil_malloc(nRequired);
722
+ bStatus = QueryServiceConfig(hSvc, pSvcConfig, nRequired, &nRequired);
723
+ if( !bStatus ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
724
+ /* Translate the service type */
725
+ switch( pSvcConfig->dwServiceType ){
726
+ case SERVICE_KERNEL_DRIVER: zSvcType = zSvcTypes[0]; break;
727
+ case SERVICE_FILE_SYSTEM_DRIVER: zSvcType = zSvcTypes[1]; break;
728
+ case SERVICE_WIN32_OWN_PROCESS: zSvcType = zSvcTypes[2]; break;
729
+ case SERVICE_WIN32_SHARE_PROCESS: zSvcType = zSvcTypes[3]; break;
730
+ case SERVICE_INTERACTIVE_PROCESS: zSvcType = zSvcTypes[4]; break;
731
+ }
732
+ /* Translate the service start type */
733
+ switch( pSvcConfig->dwStartType ){
734
+ case SERVICE_BOOT_START: zSvcStartType = zSvcStartTypes[0]; break;
735
+ case SERVICE_SYSTEM_START: zSvcStartType = zSvcStartTypes[1]; break;
736
+ case SERVICE_AUTO_START: zSvcStartType = zSvcStartTypes[2]; break;
737
+ case SERVICE_DEMAND_START: zSvcStartType = zSvcStartTypes[3]; break;
738
+ case SERVICE_DISABLED: zSvcStartType = zSvcStartTypes[4]; break;
739
+ }
740
+ /* Get the service description. */
741
+ bStatus = QueryServiceConfig2(hSvc, SERVICE_CONFIG_DESCRIPTION,
742
+ NULL, 0, &nRequired);
743
+ if( !bStatus && GetLastError()!=ERROR_INSUFFICIENT_BUFFER ){
744
+ fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
745
+ }
746
+ pSvcDescr = fossil_malloc(nRequired);
747
+ bStatus = QueryServiceConfig2(hSvc, SERVICE_CONFIG_DESCRIPTION,
748
+ (LPBYTE)pSvcDescr, nRequired, &nRequired);
749
+ if( !bStatus ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
750
+ /* Retrieves the current status of the specified service. */
751
+ bStatus = QueryServiceStatus(hSvc, &sstat);
752
+ if( !bStatus ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
753
+ /* Translate the current state. */
754
+ switch( sstat.dwCurrentState ){
755
+ case SERVICE_STOPPED: zSvcState = zSvcStates[0]; break;
756
+ case SERVICE_START_PENDING: zSvcState = zSvcStates[1]; break;
757
+ case SERVICE_STOP_PENDING: zSvcState = zSvcStates[2]; break;
758
+ case SERVICE_RUNNING: zSvcState = zSvcStates[3]; break;
759
+ case SERVICE_CONTINUE_PENDING: zSvcState = zSvcStates[4]; break;
760
+ case SERVICE_PAUSE_PENDING: zSvcState = zSvcStates[5]; break;
761
+ case SERVICE_PAUSED: zSvcState = zSvcStates[6]; break;
762
+ }
763
+ /* Print service information to terminal */
764
+ fossil_print("Service name .......: %s\n", zSvcName);
765
+ fossil_print("Display name .......: %s\n",
766
+ fossil_mbcs_to_utf8(pSvcConfig->lpDisplayName));
767
+ fossil_print("Service description : %s\n",
768
+ fossil_mbcs_to_utf8(pSvcDescr->lpDescription));
769
+ fossil_print("Service type .......: %s.\n", zSvcType);
770
+ fossil_print("Service start type .: %s.\n", zSvcStartType);
771
+ fossil_print("Binary path name ...: %s\n",
772
+ fossil_mbcs_to_utf8(pSvcConfig->lpBinaryPathName));
773
+ fossil_print("Service username ...: %s\n",
774
+ fossil_mbcs_to_utf8(pSvcConfig->lpServiceStartName));
775
+ fossil_print("Current state ......: %s.\n", zSvcState);
776
+ /* Cleanup */
777
+ fossil_free(pSvcConfig);
778
+ fossil_free(pSvcDescr);
779
+ CloseServiceHandle(hSvc);
780
+ CloseServiceHandle(hScm);
781
+ }else
782
+ if( strncmp(zMethod, "start", n)==0 ){
783
+ SC_HANDLE hScm;
784
+ SC_HANDLE hSvc;
785
+ SERVICE_STATUS sstat;
786
+ char *zErrFmt = "unable to start service '%s': %s";
787
+
788
+ verify_all_options();
789
+ if( g.argc>4 ) fossil_fatal("to much arguments for start method.");
790
+ hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
791
+ if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
792
+ hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), SERVICE_ALL_ACCESS);
793
+ if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
794
+ QueryServiceStatus(hSvc, &sstat);
795
+ if( sstat.dwCurrentState!=SERVICE_RUNNING ){
796
+ fossil_print("Starting service '%s'", zSvcName);
797
+ if( sstat.dwCurrentState!=SERVICE_START_PENDING ){
798
+ if( !StartService(hSvc, 0, NULL) ){
799
+ fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
800
+ }
801
+ }
802
+ while( sstat.dwCurrentState!=SERVICE_RUNNING ){
803
+ Sleep(100);
804
+ fossil_print(".");
805
+ QueryServiceStatus(hSvc, &sstat);
806
+ }
807
+ fossil_print("\nService '%s' started.\n", zSvcName);
808
+ }else{
809
+ fossil_print("Service '%s' is already started.\n", zSvcName);
810
+ }
811
+ CloseServiceHandle(hSvc);
812
+ CloseServiceHandle(hScm);
813
+ }else
814
+ if( strncmp(zMethod, "stop", n)==0 ){
815
+ SC_HANDLE hScm;
816
+ SC_HANDLE hSvc;
817
+ SERVICE_STATUS sstat;
818
+ char *zErrFmt = "unable to stop service '%s': %s";
819
+
820
+ verify_all_options();
821
+ if( g.argc>4 ) fossil_fatal("to much arguments for stop method.");
822
+ hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
823
+ if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
824
+ hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), SERVICE_ALL_ACCESS);
825
+ if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
826
+ QueryServiceStatus(hSvc, &sstat);
827
+ if( sstat.dwCurrentState!=SERVICE_STOPPED ){
828
+ fossil_print("Stopping service '%s'", zSvcName);
829
+ if( sstat.dwCurrentState!=SERVICE_STOP_PENDING ){
830
+ if( !ControlService(hSvc, SERVICE_CONTROL_STOP, &sstat) ){
831
+ fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
832
+ }
833
+ }
834
+ while( sstat.dwCurrentState!=SERVICE_STOPPED ){
835
+ Sleep(100);
836
+ fossil_print(".");
837
+ QueryServiceStatus(hSvc, &sstat);
838
+ }
839
+ fossil_print("\nService '%s' stopped.\n", zSvcName);
840
+ }else{
841
+ fossil_print("Service '%s' is already stopped.\n", zSvcName);
842
+ }
843
+ CloseServiceHandle(hSvc);
844
+ CloseServiceHandle(hScm);
845
+ }else
846
+ {
847
+ fossil_fatal("METHOD should be one of:"
848
+ " create delete show start stop");
849
+ }
850
+ return;
851
+}
852
+
853
+#else /* _WIN32 -- This code is for win32 only */
854
+#include "winhttp.h"
855
+
856
+void cmd_win32_service(void){
857
+ fossil_fatal("The winsrv command is platform specific "
858
+ "and not available on this platform.");
859
+ return;
860
+}
226861
227862
 #endif /* _WIN32 -- This code is for win32 only */
228863
--- src/winhttp.c
+++ src/winhttp.c
@@ -14,17 +14,18 @@
14 ** http://www.hwaci.com/drh/
15 **
16 *******************************************************************************
17 **
18 ** This file implements a very simple (and low-performance) HTTP server
19 ** for windows.
 
20 */
21 #include "config.h"
22 #ifdef _WIN32
23 /* This code is for win32 only */
24 #include "winhttp.h"
25 #include <windows.h>
 
26
27 /*
28 ** The HttpRequest structure holds information about each incoming
29 ** HTTP request.
30 */
@@ -60,11 +61,11 @@
60 }
61
62 /*
63 ** Process a single incoming HTTP request.
64 */
65 void win32_process_one_http_request(void *pAppData){
66 HttpRequest *p = (HttpRequest*)pAppData;
67 FILE *in = 0, *out = 0;
68 int amt, got;
69 int wanted = 0;
70 char *z;
@@ -145,10 +146,11 @@
145 SOCKET s = INVALID_SOCKET;
146 SOCKADDR_IN addr;
147 int idCnt = 0;
148 int iPort = mnPort;
149 Blob options;
 
150
151 if( zStopper ) file_delete(zStopper);
152 blob_zero(&options);
153 if( zNotFound ){
154 blob_appendf(&options, " --notfound %s", zNotFound);
@@ -189,31 +191,46 @@
189 }else{
190 fossil_fatal("unable to open listening socket on any"
191 " port in the range %d..%d", mnPort, mxPort);
192 }
193 }
194 zTempPrefix = mprintf("fossil_server_P%d_", iPort);
 
 
 
195 fossil_print("Listening for HTTP requests on TCP port %d\n", iPort);
196 if( zBrowser ){
197 zBrowser = mprintf(zBrowser, iPort);
198 fossil_print("Launch webbrowser: %s\n", zBrowser);
199 fossil_system(zBrowser);
200 }
201 fossil_print("Type Ctrl-C to stop the HTTP server\n");
 
 
 
202 for(;;){
203 SOCKET client;
204 SOCKADDR_IN client_addr;
205 HttpRequest *p;
206 int len = sizeof(client_addr);
 
207
208 client = accept(s, (struct sockaddr*)&client_addr, &len);
209 if( zStopper && file_size(zStopper)>=0 ){
210 break;
211 }
212 if( client==INVALID_SOCKET ){
213 closesocket(s);
214 fossil_fatal("error from accept()");
 
 
 
 
 
 
 
 
 
 
 
215 }
216 p = fossil_malloc( sizeof(*p) );
217 p->id = ++idCnt;
218 p->s = client;
219 p->addr = client_addr;
@@ -221,7 +238,625 @@
221 _beginthread(win32_process_one_http_request, 0, (void*)p);
222 }
223 closesocket(s);
224 WSACleanup();
225 }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
226
227 #endif /* _WIN32 -- This code is for win32 only */
228
--- src/winhttp.c
+++ src/winhttp.c
@@ -14,17 +14,18 @@
14 ** http://www.hwaci.com/drh/
15 **
16 *******************************************************************************
17 **
18 ** This file implements a very simple (and low-performance) HTTP server
19 ** for windows. It also implements a Windows Service which allows the HTTP
20 ** server to be run without any user logged on.
21 */
22 #include "config.h"
23 #ifdef _WIN32
24 /* This code is for win32 only */
 
25 #include <windows.h>
26 #include "winhttp.h"
27
28 /*
29 ** The HttpRequest structure holds information about each incoming
30 ** HTTP request.
31 */
@@ -60,11 +61,11 @@
61 }
62
63 /*
64 ** Process a single incoming HTTP request.
65 */
66 static void win32_process_one_http_request(void *pAppData){
67 HttpRequest *p = (HttpRequest*)pAppData;
68 FILE *in = 0, *out = 0;
69 int amt, got;
70 int wanted = 0;
71 char *z;
@@ -145,10 +146,11 @@
146 SOCKET s = INVALID_SOCKET;
147 SOCKADDR_IN addr;
148 int idCnt = 0;
149 int iPort = mnPort;
150 Blob options;
151 char zTmpPath[MAX_PATH];
152
153 if( zStopper ) file_delete(zStopper);
154 blob_zero(&options);
155 if( zNotFound ){
156 blob_appendf(&options, " --notfound %s", zNotFound);
@@ -189,31 +191,46 @@
191 }else{
192 fossil_fatal("unable to open listening socket on any"
193 " port in the range %d..%d", mnPort, mxPort);
194 }
195 }
196 if( !GetTempPath(sizeof(zTmpPath), zTmpPath) ){
197 fossil_fatal("unable to get path to the temporary directory.");
198 }
199 zTempPrefix = mprintf("%sfossil_server_P%d_", zTmpPath, iPort);
200 fossil_print("Listening for HTTP requests on TCP port %d\n", iPort);
201 if( zBrowser ){
202 zBrowser = mprintf(zBrowser, iPort);
203 fossil_print("Launch webbrowser: %s\n", zBrowser);
204 fossil_system(zBrowser);
205 }
206 fossil_print("Type Ctrl-C to stop the HTTP server\n");
207 /* Set the service status to running and pass the listener socket to the
208 ** service handling procedures. */
209 win32_http_service_running(s);
210 for(;;){
211 SOCKET client;
212 SOCKADDR_IN client_addr;
213 HttpRequest *p;
214 int len = sizeof(client_addr);
215 int wsaError;
216
217 client = accept(s, (struct sockaddr*)&client_addr, &len);
 
 
 
218 if( client==INVALID_SOCKET ){
219 /* If the service control handler has closed the listener socket,
220 ** cleanup and return, otherwise report a fatal error. */
221 wsaError = WSAGetLastError();
222 if( (wsaError==WSAEINTR) || (wsaError==WSAENOTSOCK) ){
223 WSACleanup();
224 return;
225 }else{
226 closesocket(s);
227 WSACleanup();
228 fossil_fatal("error from accept()");
229 }
230 }else if( zStopper && file_size(zStopper)>=0 ){
231 break;
232 }
233 p = fossil_malloc( sizeof(*p) );
234 p->id = ++idCnt;
235 p->s = client;
236 p->addr = client_addr;
@@ -221,7 +238,625 @@
238 _beginthread(win32_process_one_http_request, 0, (void*)p);
239 }
240 closesocket(s);
241 WSACleanup();
242 }
243
244 /*
245 ** The HttpService structure is used to pass information to the service main
246 ** function and to the service control handler function.
247 */
248 typedef struct HttpService HttpService;
249 struct HttpService {
250 int port; /* Port on which the http server should run */
251 const char *zNotFound; /* The --notfound option, or NULL */
252 int flags; /* One or more HTTP_SERVER_ flags */
253 int isRunningAsService; /* Are we running as a service ? */
254 const char *zServiceName; /* Name of the service */
255 SOCKET s; /* Socket on which the http server listens */
256 };
257
258 /*
259 ** Variables used for running as windows service.
260 */
261 static HttpService hsData = {8080, NULL, 0, 0, NULL, INVALID_SOCKET};
262 static SERVICE_STATUS ssStatus;
263 static SERVICE_STATUS_HANDLE sshStatusHandle;
264
265 /*
266 ** Get message string of the last system error. Return a pointer to the
267 ** message string. Call fossil_mbcs_free() to deallocate any memory used
268 ** to store the message string when done.
269 */
270 static char *win32_get_last_errmsg(void){
271 DWORD nMsg;
272 LPTSTR tmp = NULL;
273 char *zMsg = NULL;
274
275 nMsg = FormatMessage(
276 FORMAT_MESSAGE_ALLOCATE_BUFFER |
277 FORMAT_MESSAGE_FROM_SYSTEM |
278 FORMAT_MESSAGE_IGNORE_INSERTS,
279 NULL,
280 GetLastError(),
281 MAKELANGID(LANG_ENGLISH, SUBLANG_ENGLISH_US),
282 (LPTSTR) &tmp,
283 0,
284 NULL
285 );
286 if( nMsg ){
287 zMsg = fossil_mbcs_to_utf8(tmp);
288 }else{
289 fossil_fatal("unable to get system error message.");
290 }
291 if( tmp ){
292 LocalFree((HLOCAL) tmp);
293 }
294 return zMsg;
295 }
296
297 /*
298 ** Report the current status of the service to the service control manager.
299 ** Make sure that during service startup no control codes are accepted.
300 */
301 static void win32_report_service_status(
302 DWORD dwCurrentState, /* The current state of the service */
303 DWORD dwWin32ExitCode, /* The error code to report */
304 DWORD dwWaitHint /* The estimated time for a pending operation */
305 ){
306 if( dwCurrentState==SERVICE_START_PENDING) {
307 ssStatus.dwControlsAccepted = 0;
308 }else{
309 ssStatus.dwControlsAccepted = SERVICE_ACCEPT_STOP;
310 }
311 ssStatus.dwCurrentState = dwCurrentState;
312 ssStatus.dwWin32ExitCode = dwWin32ExitCode;
313 ssStatus.dwWaitHint = dwWaitHint;
314
315 if( (dwCurrentState==SERVICE_RUNNING) ||
316 (dwCurrentState==SERVICE_STOPPED) ){
317 ssStatus.dwCheckPoint = 0;
318 }else{
319 ssStatus.dwCheckPoint++;
320 }
321 SetServiceStatus(sshStatusHandle, &ssStatus);
322 return ;
323 }
324
325 /*
326 ** Handle control codes sent from the service control manager.
327 ** The control dispatcher in the main thread of the service process invokes
328 ** this function whenever it receives a control request from the service
329 ** control manager.
330 */
331 static void WINAPI win32_http_service_ctrl(
332 DWORD dwCtrlCode
333 ){
334 switch( dwCtrlCode ){
335 case SERVICE_CONTROL_STOP: {
336 win32_report_service_status(SERVICE_STOP_PENDING, NO_ERROR, 0);
337 if( hsData.s != INVALID_SOCKET ){
338 closesocket(hsData.s);
339 }
340 win32_report_service_status(ssStatus.dwCurrentState, NO_ERROR, 0);
341 break;
342 }
343 default: {
344 break;
345 }
346 }
347 return;
348 }
349
350 /*
351 ** This is the main entry point for the service.
352 ** When the service control manager receives a request to start the service,
353 ** it starts the service process (if it is not already running). The main
354 ** thread of the service process calls the StartServiceCtrlDispatcher
355 ** function with a pointer to an array of SERVICE_TABLE_ENTRY structures.
356 ** Then the service control manager sends a start request to the service
357 ** control dispatcher for this service process. The service control dispatcher
358 ** creates a new thread to execute the ServiceMain function (this function)
359 ** of the service being started.
360 */
361 static void WINAPI win32_http_service_main(
362 DWORD argc, /* Number of arguments in argv */
363 LPTSTR *argv /* Arguments passed */
364 ){
365
366 /* Update the service information. */
367 hsData.isRunningAsService = 1;
368 if( argc>0 ){
369 hsData.zServiceName = argv[0];
370 }
371
372 /* Register the service control handler function */
373 sshStatusHandle = RegisterServiceCtrlHandler("", win32_http_service_ctrl);
374 if( !sshStatusHandle ){
375 win32_report_service_status(SERVICE_STOPPED, NO_ERROR, 0);
376 return;
377 }
378
379 /* Set service specific data and report that the service is starting. */
380 ssStatus.dwServiceType = SERVICE_WIN32_OWN_PROCESS;
381 ssStatus.dwServiceSpecificExitCode = 0;
382 win32_report_service_status(SERVICE_START_PENDING, NO_ERROR, 3000);
383
384 /* Execute the http server */
385 win32_http_server(hsData.port, hsData.port,
386 NULL, NULL, hsData.zNotFound, hsData.flags);
387
388 /* Service has stopped now. */
389 win32_report_service_status(SERVICE_STOPPED, NO_ERROR, 0);
390 return;
391 }
392
393 /*
394 ** When running as service, update the HttpService structure with the
395 ** listener socket and update the service status. This procedure must be
396 ** called from the http server when he is ready to accept connections.
397 */
398 LOCAL void win32_http_service_running(SOCKET s){
399 if( hsData.isRunningAsService ){
400 hsData.s = s;
401 win32_report_service_status(SERVICE_RUNNING, NO_ERROR, 0);
402 }
403 }
404
405 /*
406 ** Try to start the http server as a windows service. If we are running in
407 ** a interactive console session, this routine fails and returns a non zero
408 ** integer value. When running as service, this routine does not return until
409 ** the service is stopped. In this case, the return value is zero.
410 */
411 int win32_http_service(
412 int nPort, /* TCP port number */
413 const char *zNotFound, /* The --notfound option, or NULL */
414 int flags /* One or more HTTP_SERVER_ flags */
415 ){
416 /* Define the service table. */
417 SERVICE_TABLE_ENTRY ServiceTable[] =
418 {{"", (LPSERVICE_MAIN_FUNCTION)win32_http_service_main}, {NULL, NULL}};
419
420 /* Initialize the HttpService structure. */
421 hsData.port = nPort;
422 hsData.zNotFound = zNotFound;
423 hsData.flags = flags;
424
425 /* Try to start the control dispatcher thread for the service. */
426 if( !StartServiceCtrlDispatcher(ServiceTable) ){
427 if( GetLastError()==ERROR_FAILED_SERVICE_CONTROLLER_CONNECT ){
428 return 1;
429 }else{
430 fossil_fatal("error from StartServiceCtrlDispatcher()");
431 }
432 }
433 return 0;
434 }
435
436 /*
437 ** COMMAND: winsrv
438 ** Usage: fossil winsrv METHOD ?SERVICE-NAME? ?OPTIONS?
439 **
440 ** Where METHOD is one of: create delete show start stop.
441 **
442 ** The winsrv command manages Fossil as a Windows service. This allows
443 ** (for example) Fossil to be running in the background when no user
444 ** is logged in.
445 **
446 ** In the following description of the methods, "Fossil-DSCM" will be
447 ** used as the default SERVICE-NAME:
448 **
449 ** fossil winsrv create ?SERVICE-NAME? ?OPTIONS?
450 **
451 ** Creates a service. Available options include:
452 **
453 ** -D|--display DISPLAY-NAME
454 **
455 ** Sets the display name of the service. This name is shown
456 ** by graphical interface programs. By default, the display name
457 ** equals to the service name.
458 **
459 ** -S|--start TYPE
460 **
461 ** Sets the start type of the service. TYPE can be "manual",
462 ** which means you need to start the service yourself with the
463 ** 'fossil winsrv start' command or with the "net start" command
464 ** from the operating system. If TYPE is set to "auto", the service
465 ** will be started automatically by the system during startup.
466 **
467 ** -U|--username USERNAME
468 **
469 ** Specifies the user account which will be used to run the
470 ** service. The account needs the "Logon as a service" right
471 ** enabled in its profile. Specify local accounts as follows:
472 ** ".\\USERNAME". By default, the "LocalSystem" account will be
473 ** used.
474 **
475 ** -W|--password PASSWORD
476 **
477 ** Password for the user account.
478 **
479 ** The following options are more or less the same as for the "server"
480 ** command and influence the behaviour of the http server:
481 **
482 ** -p|--port TCPPORT
483 **
484 ** Specifies the TCP port (default port is 8080) on which the
485 ** server should listen.
486 **
487 ** -R|--repository REPOSITORY
488 **
489 ** Specifies the name of the repository to be served.
490 ** The repository option may be omitted if the working directory
491 ** is within an open checkout.
492 ** The REPOSITORY can be a directory (aka folder) that contains
493 ** one or more respositories with names ending in ".fossil".
494 ** In that case, the first element of the URL is used to select
495 ** among the various repositories.
496 **
497 ** --notfound URL
498 **
499 ** If REPOSITORY is a directory that contains one or more
500 ** respositories with names of the form "*.fossil" then the
501 ** first element of the URL pathname selects among the various
502 ** repositories. If the pathname does not select a valid
503 ** repository and the --notfound option is available,
504 ** then the server redirects (HTTP code 302) to the URL of
505 ** --notfound.
506 **
507 ** --localauth
508 **
509 ** Enables automatic login if the --localauth option is present
510 ** and the "localauth" setting is off and the connection is from
511 ** localhost.
512 **
513 **
514 ** fossil winsrv delete ?SERVICE-NAME?
515 **
516 ** Deletes a service. If the service is currently running, it will be
517 ** stopped first and then deleted.
518 **
519 **
520 ** fossil winsrv show ?SERVICE-NAME?
521 **
522 ** Shows how the service is configured and its current state.
523 **
524 **
525 ** fossil winsrv start ?SERVICE-NAME?
526 **
527 ** Start the service.
528 **
529 **
530 ** fossil winsrv stop ?SERVICE-NAME?
531 **
532 ** Stop the service.
533 **
534 **
535 ** NOTE: This command is available on Windows operating systems only and
536 ** requires administrative rights on the machine executed.
537 **
538 */
539 void cmd_win32_service(void){
540 int n;
541 const char *zMethod;
542 const char *zSvcName = "Fossil-DSCM"; /* Default service name */
543
544 if( g.argc<3 ){
545 usage("create|delete|show|start|stop ...");
546 }
547 zMethod = g.argv[2];
548 n = strlen(zMethod);
549 if( g.argc==4 ){
550 zSvcName = g.argv[3];
551 }
552
553 if( strncmp(zMethod, "create", n)==0 ){
554 SC_HANDLE hScm;
555 SC_HANDLE hSvc;
556 SERVICE_DESCRIPTION
557 svcDescr = {"Fossil - Distributed Software Configuration Management"};
558 char *zErrFmt = "unable to create service '%s': %s";
559 DWORD dwStartType = SERVICE_DEMAND_START;
560 const char *zDisplay;
561 const char *zStart;
562 const char *zUsername;
563 const char *zPassword;
564 const char *zPort;
565 const char *zNotFound;
566 const char *zLocalAuth;
567 const char *zRepository;
568 Blob binPath;
569
570 /* Process service creation specific options. */
571 zDisplay = find_option("display", "D", 1);
572 if( !zDisplay ){
573 zDisplay = zSvcName;
574 }
575 zStart = find_option("start", "S", 1);
576 if( zStart ){
577 if( strncmp(zStart, "auto", strlen(zStart))==0 ){
578 dwStartType = SERVICE_AUTO_START;
579 }else if( strncmp(zStart, "manual", strlen(zStart))==0 ){
580 dwStartType = SERVICE_DEMAND_START;
581 }else{
582 fossil_fatal(zErrFmt, zSvcName,
583 "specify 'auto' or 'manual' for the '-S|--start' option");
584 }
585 }
586 zUsername = find_option("username", "U", 1);
587 zPassword = find_option("password", "W", 1);
588 /* Process options for Fossil running as server. */
589 zPort = find_option("port", "P", 1);
590 if( zPort && (atoi(zPort)<=0) ){
591 fossil_fatal(zErrFmt, zSvcName,
592 "port number must be in the range 1 - 65535.");
593 }
594 zNotFound = find_option("notfound", 0, 1);
595 zLocalAuth = find_option("localauth", 0, 0);
596 zRepository = find_option("repository", "R", 1);
597 if( !zRepository ){
598 db_must_be_within_tree();
599 }else if( file_isdir(zRepository)==1 ){
600 g.zRepositoryName = mprintf("%s", zRepository);
601 file_simplify_name(g.zRepositoryName, -1);
602 }else{
603 db_open_repository(zRepository);
604 }
605 db_close(0);
606 verify_all_options();
607 if( g.argc>4 ) fossil_fatal("to much arguments for create method.");
608 /* Build the fully-qualified path to the service binary file. */
609 blob_zero(&binPath);
610 blob_appendf(&binPath, "\"%s\" server", fossil_nameofexe());
611 if( zPort ) blob_appendf(&binPath, " --port %s", zPort);
612 if( zNotFound ) blob_appendf(&binPath, " --notfound \"%s\"", zNotFound);
613 if( zLocalAuth ) blob_append(&binPath, " --localauth", -1);
614 blob_appendf(&binPath, " \"%s\"", g.zRepositoryName);
615 /* Create the service. */
616 hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
617 if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
618 hSvc = CreateService(
619 hScm, /* Handle to the SCM */
620 fossil_utf8_to_mbcs(zSvcName), /* Name of the service */
621 fossil_utf8_to_mbcs(zDisplay), /* Display name */
622 SERVICE_ALL_ACCESS, /* Desired access */
623 SERVICE_WIN32_OWN_PROCESS, /* Service type */
624 dwStartType, /* Start type */
625 SERVICE_ERROR_NORMAL, /* Error control */
626 fossil_utf8_to_mbcs(blob_str(&binPath)), /* Binary path */
627 NULL, /* Load ordering group */
628 NULL, /* Tag value */
629 NULL, /* Service dependencies */
630 fossil_utf8_to_mbcs(zUsername), /* Service account */
631 fossil_utf8_to_mbcs(zPassword) /* Account password */
632 );
633 if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
634 /* Set the service description. */
635 ChangeServiceConfig2(hSvc, SERVICE_CONFIG_DESCRIPTION, &svcDescr);
636 fossil_print("Service '%s' successfully created.\n", zSvcName);
637 CloseServiceHandle(hSvc);
638 CloseServiceHandle(hScm);
639 }else
640 if( strncmp(zMethod, "delete", n)==0 ){
641 SC_HANDLE hScm;
642 SC_HANDLE hSvc;
643 SERVICE_STATUS sstat;
644 char *zErrFmt = "unable to delete service '%s': %s";
645
646 verify_all_options();
647 if( g.argc>4 ) fossil_fatal("to much arguments for delete method.");
648 hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
649 if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
650 hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), SERVICE_ALL_ACCESS);
651 if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
652 QueryServiceStatus(hSvc, &sstat);
653 if( sstat.dwCurrentState!=SERVICE_STOPPED ){
654 fossil_print("Stopping service '%s'", zSvcName);
655 if( sstat.dwCurrentState!=SERVICE_STOP_PENDING ){
656 if( !ControlService(hSvc, SERVICE_CONTROL_STOP, &sstat) ){
657 fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
658 }
659 }
660 while( sstat.dwCurrentState!=SERVICE_STOPPED ){
661 Sleep(100);
662 fossil_print(".");
663 QueryServiceStatus(hSvc, &sstat);
664 }
665 fossil_print("\nService '%s' stopped.\n", zSvcName);
666 }
667 if( !DeleteService(hSvc) ){
668 if( GetLastError()==ERROR_SERVICE_MARKED_FOR_DELETE ){
669 fossil_warning("Service '%s' already marked for delete.\n", zSvcName);
670 }else{
671 fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
672 }
673 }else{
674 fossil_print("Service '%s' successfully deleted.\n", zSvcName);
675 }
676 CloseServiceHandle(hSvc);
677 CloseServiceHandle(hScm);
678 }else
679 if( strncmp(zMethod, "show", n)==0 ){
680 SC_HANDLE hScm;
681 SC_HANDLE hSvc;
682 SERVICE_STATUS sstat;
683 LPQUERY_SERVICE_CONFIG pSvcConfig;
684 LPSERVICE_DESCRIPTION pSvcDescr;
685 BOOL bStatus;
686 DWORD nRequired;
687 char *zErrFmt = "unable to show service '%s': %s";
688 static const char *zSvcTypes[] = {
689 "Driver service",
690 "File system driver service",
691 "Service runs in its own process",
692 "Service shares a process with other services",
693 "Service can interact with the desktop"
694 };
695 const char *zSvcType = "";
696 static char *zSvcStartTypes[] = {
697 "Started by the system loader",
698 "Started by the IoInitSystem function",
699 "Started automatically by the service control manager",
700 "Started manually",
701 "Service cannot be started"
702 };
703 const char *zSvcStartType = "";
704 static const char *zSvcStates[] = {
705 "Stopped", "Starting", "Stopping", "Running",
706 "Continue pending", "Pause pending", "Paused"
707 };
708 const char *zSvcState = "";
709
710 verify_all_options();
711 if( g.argc>4 ) fossil_fatal("to much arguments for show method.");
712 hScm = OpenSCManager(NULL, NULL, GENERIC_READ);
713 if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
714 hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), GENERIC_READ);
715 if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
716 /* Get the service configuration */
717 bStatus = QueryServiceConfig(hSvc, NULL, 0, &nRequired);
718 if( !bStatus && GetLastError()!=ERROR_INSUFFICIENT_BUFFER ){
719 fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
720 }
721 pSvcConfig = fossil_malloc(nRequired);
722 bStatus = QueryServiceConfig(hSvc, pSvcConfig, nRequired, &nRequired);
723 if( !bStatus ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
724 /* Translate the service type */
725 switch( pSvcConfig->dwServiceType ){
726 case SERVICE_KERNEL_DRIVER: zSvcType = zSvcTypes[0]; break;
727 case SERVICE_FILE_SYSTEM_DRIVER: zSvcType = zSvcTypes[1]; break;
728 case SERVICE_WIN32_OWN_PROCESS: zSvcType = zSvcTypes[2]; break;
729 case SERVICE_WIN32_SHARE_PROCESS: zSvcType = zSvcTypes[3]; break;
730 case SERVICE_INTERACTIVE_PROCESS: zSvcType = zSvcTypes[4]; break;
731 }
732 /* Translate the service start type */
733 switch( pSvcConfig->dwStartType ){
734 case SERVICE_BOOT_START: zSvcStartType = zSvcStartTypes[0]; break;
735 case SERVICE_SYSTEM_START: zSvcStartType = zSvcStartTypes[1]; break;
736 case SERVICE_AUTO_START: zSvcStartType = zSvcStartTypes[2]; break;
737 case SERVICE_DEMAND_START: zSvcStartType = zSvcStartTypes[3]; break;
738 case SERVICE_DISABLED: zSvcStartType = zSvcStartTypes[4]; break;
739 }
740 /* Get the service description. */
741 bStatus = QueryServiceConfig2(hSvc, SERVICE_CONFIG_DESCRIPTION,
742 NULL, 0, &nRequired);
743 if( !bStatus && GetLastError()!=ERROR_INSUFFICIENT_BUFFER ){
744 fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
745 }
746 pSvcDescr = fossil_malloc(nRequired);
747 bStatus = QueryServiceConfig2(hSvc, SERVICE_CONFIG_DESCRIPTION,
748 (LPBYTE)pSvcDescr, nRequired, &nRequired);
749 if( !bStatus ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
750 /* Retrieves the current status of the specified service. */
751 bStatus = QueryServiceStatus(hSvc, &sstat);
752 if( !bStatus ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
753 /* Translate the current state. */
754 switch( sstat.dwCurrentState ){
755 case SERVICE_STOPPED: zSvcState = zSvcStates[0]; break;
756 case SERVICE_START_PENDING: zSvcState = zSvcStates[1]; break;
757 case SERVICE_STOP_PENDING: zSvcState = zSvcStates[2]; break;
758 case SERVICE_RUNNING: zSvcState = zSvcStates[3]; break;
759 case SERVICE_CONTINUE_PENDING: zSvcState = zSvcStates[4]; break;
760 case SERVICE_PAUSE_PENDING: zSvcState = zSvcStates[5]; break;
761 case SERVICE_PAUSED: zSvcState = zSvcStates[6]; break;
762 }
763 /* Print service information to terminal */
764 fossil_print("Service name .......: %s\n", zSvcName);
765 fossil_print("Display name .......: %s\n",
766 fossil_mbcs_to_utf8(pSvcConfig->lpDisplayName));
767 fossil_print("Service description : %s\n",
768 fossil_mbcs_to_utf8(pSvcDescr->lpDescription));
769 fossil_print("Service type .......: %s.\n", zSvcType);
770 fossil_print("Service start type .: %s.\n", zSvcStartType);
771 fossil_print("Binary path name ...: %s\n",
772 fossil_mbcs_to_utf8(pSvcConfig->lpBinaryPathName));
773 fossil_print("Service username ...: %s\n",
774 fossil_mbcs_to_utf8(pSvcConfig->lpServiceStartName));
775 fossil_print("Current state ......: %s.\n", zSvcState);
776 /* Cleanup */
777 fossil_free(pSvcConfig);
778 fossil_free(pSvcDescr);
779 CloseServiceHandle(hSvc);
780 CloseServiceHandle(hScm);
781 }else
782 if( strncmp(zMethod, "start", n)==0 ){
783 SC_HANDLE hScm;
784 SC_HANDLE hSvc;
785 SERVICE_STATUS sstat;
786 char *zErrFmt = "unable to start service '%s': %s";
787
788 verify_all_options();
789 if( g.argc>4 ) fossil_fatal("to much arguments for start method.");
790 hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
791 if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
792 hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), SERVICE_ALL_ACCESS);
793 if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
794 QueryServiceStatus(hSvc, &sstat);
795 if( sstat.dwCurrentState!=SERVICE_RUNNING ){
796 fossil_print("Starting service '%s'", zSvcName);
797 if( sstat.dwCurrentState!=SERVICE_START_PENDING ){
798 if( !StartService(hSvc, 0, NULL) ){
799 fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
800 }
801 }
802 while( sstat.dwCurrentState!=SERVICE_RUNNING ){
803 Sleep(100);
804 fossil_print(".");
805 QueryServiceStatus(hSvc, &sstat);
806 }
807 fossil_print("\nService '%s' started.\n", zSvcName);
808 }else{
809 fossil_print("Service '%s' is already started.\n", zSvcName);
810 }
811 CloseServiceHandle(hSvc);
812 CloseServiceHandle(hScm);
813 }else
814 if( strncmp(zMethod, "stop", n)==0 ){
815 SC_HANDLE hScm;
816 SC_HANDLE hSvc;
817 SERVICE_STATUS sstat;
818 char *zErrFmt = "unable to stop service '%s': %s";
819
820 verify_all_options();
821 if( g.argc>4 ) fossil_fatal("to much arguments for stop method.");
822 hScm = OpenSCManager(NULL, NULL, SC_MANAGER_ALL_ACCESS);
823 if( !hScm ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
824 hSvc = OpenService(hScm, fossil_utf8_to_mbcs(zSvcName), SERVICE_ALL_ACCESS);
825 if( !hSvc ) fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
826 QueryServiceStatus(hSvc, &sstat);
827 if( sstat.dwCurrentState!=SERVICE_STOPPED ){
828 fossil_print("Stopping service '%s'", zSvcName);
829 if( sstat.dwCurrentState!=SERVICE_STOP_PENDING ){
830 if( !ControlService(hSvc, SERVICE_CONTROL_STOP, &sstat) ){
831 fossil_fatal(zErrFmt, zSvcName, win32_get_last_errmsg());
832 }
833 }
834 while( sstat.dwCurrentState!=SERVICE_STOPPED ){
835 Sleep(100);
836 fossil_print(".");
837 QueryServiceStatus(hSvc, &sstat);
838 }
839 fossil_print("\nService '%s' stopped.\n", zSvcName);
840 }else{
841 fossil_print("Service '%s' is already stopped.\n", zSvcName);
842 }
843 CloseServiceHandle(hSvc);
844 CloseServiceHandle(hScm);
845 }else
846 {
847 fossil_fatal("METHOD should be one of:"
848 " create delete show start stop");
849 }
850 return;
851 }
852
853 #else /* _WIN32 -- This code is for win32 only */
854 #include "winhttp.h"
855
856 void cmd_win32_service(void){
857 fossil_fatal("The winsrv command is platform specific "
858 "and not available on this platform.");
859 return;
860 }
861
862 #endif /* _WIN32 -- This code is for win32 only */
863
M test/release-checklist.wiki
+7 -1
--- test/release-checklist.wiki
+++ test/release-checklist.wiki
@@ -21,10 +21,11 @@
2121
 <li> Linux x86
2222
 <li> Linux x86_64
2323
 <li> Mac x86
2424
 <li> Mac x86_64
2525
 <li> Windows (mingw)
26
+<li> Windows (vc++)
2627
 <li> OpenBSD
2728
 </ol>
2829
2930
 <li><p>
3031
 Run at least one occurrence of the the following commands on every
@@ -56,11 +57,16 @@
5657
5758
 <li><p>
5859
 Use the release candidate version of fossil in production on the
5960
 [http://www.fossil-scm.org/] website for at least 48 hours (without
6061
 incident) prior to making the release official.
62
+
63
+<li><p>
64
+Verify that the [../www/changes.wiki | Change Log] is correct and
65
+up-to-date.
6166
 </ol>
6267
6368
 <hr>
6469
6570
 Upon successful completion of all tests above, tag the release candidate
66
-with the "release" tag and set its background color to "#d0c0ff".
71
+with the "release" tag and set its background color to "#d0c0ff". Update
72
+the www/changes.wiki file to show the date of the release.
6773
--- test/release-checklist.wiki
+++ test/release-checklist.wiki
@@ -21,10 +21,11 @@
21 <li> Linux x86
22 <li> Linux x86_64
23 <li> Mac x86
24 <li> Mac x86_64
25 <li> Windows (mingw)
 
26 <li> OpenBSD
27 </ol>
28
29 <li><p>
30 Run at least one occurrence of the the following commands on every
@@ -56,11 +57,16 @@
56
57 <li><p>
58 Use the release candidate version of fossil in production on the
59 [http://www.fossil-scm.org/] website for at least 48 hours (without
60 incident) prior to making the release official.
 
 
 
 
61 </ol>
62
63 <hr>
64
65 Upon successful completion of all tests above, tag the release candidate
66 with the "release" tag and set its background color to "#d0c0ff".
 
67
--- test/release-checklist.wiki
+++ test/release-checklist.wiki
@@ -21,10 +21,11 @@
21 <li> Linux x86
22 <li> Linux x86_64
23 <li> Mac x86
24 <li> Mac x86_64
25 <li> Windows (mingw)
26 <li> Windows (vc++)
27 <li> OpenBSD
28 </ol>
29
30 <li><p>
31 Run at least one occurrence of the the following commands on every
@@ -56,11 +57,16 @@
57
58 <li><p>
59 Use the release candidate version of fossil in production on the
60 [http://www.fossil-scm.org/] website for at least 48 hours (without
61 incident) prior to making the release official.
62
63 <li><p>
64 Verify that the [../www/changes.wiki | Change Log] is correct and
65 up-to-date.
66 </ol>
67
68 <hr>
69
70 Upon successful completion of all tests above, tag the release candidate
71 with the "release" tag and set its background color to "#d0c0ff". Update
72 the www/changes.wiki file to show the date of the release.
73
M win/Makefile.PellesCGMake
+3 -3
--- win/Makefile.PellesCGMake
+++ win/Makefile.PellesCGMake
@@ -126,11 +126,11 @@
126126
 # compiling standard fossil utils
127127
 $(UTILS_OBJ): %.obj: $(SRCDIR)%.c
128128
 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
129129
130130
 # compile special windows utils
131
-version.obj: $(WINDIR)version.c
131
+version.obj: $(SRCDIR)mkversion.c
132132
 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
133133
134134
 # generate the translated c-source files
135135
 $(TRANSLATEDSRC): %_.c: $(SRCDIR)%.c translate.exe
136136
 translate.exe $< >$@
@@ -138,12 +138,12 @@
138138
 # generate the index source, containing all web references,..
139139
 page_index.h: $(TRANSLATEDSRC) mkindex.exe
140140
 mkindex.exe $(TRANSLATEDSRC) >$@
141141
142142
 # extracting version info from manifest
143
-VERSION.h: version.exe ..\manifest.uuid ..\manifest
144
- version.exe ..\manifest.uuid ..\manifest > $@
143
+VERSION.h: version.exe ..\manifest.uuid ..\manifest ..\VERSION
144
+ version.exe ..\manifest.uuid ..\manifest ..\VERSION > $@
145145
146146
 # generate the simplified headers
147147
 headers: makeheaders.exe page_index.h VERSION.h ../src/sqlite3.h ../src/th.h VERSION.h
148148
 makeheaders.exe $(foreach ts,$(TRANSLATEDSRC),$(ts):$(ts:_.c=.h)) ../src/sqlite3.h ../src/th.h VERSION.h
149149
 echo Done >$@
150150
--- win/Makefile.PellesCGMake
+++ win/Makefile.PellesCGMake
@@ -126,11 +126,11 @@
126 # compiling standard fossil utils
127 $(UTILS_OBJ): %.obj: $(SRCDIR)%.c
128 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
129
130 # compile special windows utils
131 version.obj: $(WINDIR)version.c
132 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
133
134 # generate the translated c-source files
135 $(TRANSLATEDSRC): %_.c: $(SRCDIR)%.c translate.exe
136 translate.exe $< >$@
@@ -138,12 +138,12 @@
138 # generate the index source, containing all web references,..
139 page_index.h: $(TRANSLATEDSRC) mkindex.exe
140 mkindex.exe $(TRANSLATEDSRC) >$@
141
142 # extracting version info from manifest
143 VERSION.h: version.exe ..\manifest.uuid ..\manifest
144 version.exe ..\manifest.uuid ..\manifest > $@
145
146 # generate the simplified headers
147 headers: makeheaders.exe page_index.h VERSION.h ../src/sqlite3.h ../src/th.h VERSION.h
148 makeheaders.exe $(foreach ts,$(TRANSLATEDSRC),$(ts):$(ts:_.c=.h)) ../src/sqlite3.h ../src/th.h VERSION.h
149 echo Done >$@
150
--- win/Makefile.PellesCGMake
+++ win/Makefile.PellesCGMake
@@ -126,11 +126,11 @@
126 # compiling standard fossil utils
127 $(UTILS_OBJ): %.obj: $(SRCDIR)%.c
128 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
129
130 # compile special windows utils
131 version.obj: $(SRCDIR)mkversion.c
132 $(CC) $(CCFLAGS) $(INCLUDE) "$<" -Fo"$@"
133
134 # generate the translated c-source files
135 $(TRANSLATEDSRC): %_.c: $(SRCDIR)%.c translate.exe
136 translate.exe $< >$@
@@ -138,12 +138,12 @@
138 # generate the index source, containing all web references,..
139 page_index.h: $(TRANSLATEDSRC) mkindex.exe
140 mkindex.exe $(TRANSLATEDSRC) >$@
141
142 # extracting version info from manifest
143 VERSION.h: version.exe ..\manifest.uuid ..\manifest ..\VERSION
144 version.exe ..\manifest.uuid ..\manifest ..\VERSION > $@
145
146 # generate the simplified headers
147 headers: makeheaders.exe page_index.h VERSION.h ../src/sqlite3.h ../src/th.h VERSION.h
148 makeheaders.exe $(foreach ts,$(TRANSLATEDSRC),$(ts):$(ts:_.c=.h)) ../src/sqlite3.h ../src/th.h VERSION.h
149 echo Done >$@
150
M win/Makefile.dmc
+4 -4
--- win/Makefile.dmc
+++ win/Makefile.dmc
@@ -20,11 +20,11 @@
2020
 CFLAGS = -o
2121
 BCC = $(DMDIR)\bin\dmc $(CFLAGS)
2222
 TCC = $(DMDIR)\bin\dmc $(CFLAGS) $(DMCDEF) $(SSL) $(INCL)
2323
 LIBS = $(DMDIR)\extra\lib\ zlib wsock32
2424
25
-SQLITE_OPTIONS = -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT2 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0
25
+SQLITE_OPTIONS = -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT3 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0
2626
2727
 SRC = add_.c allrepo_.c attach_.c bag_.c bisect_.c blob_.c branch_.c browse_.c captcha_.c cgi_.c checkin_.c checkout_.c clearsign_.c clone_.c comformat_.c configure_.c content_.c db_.c delta_.c deltacmd_.c descendants_.c diff_.c diffcmd_.c doc_.c encode_.c event_.c export_.c file_.c finfo_.c glob_.c graph_.c gzip_.c http_.c http_socket_.c http_ssl_.c http_transport_.c import_.c info_.c leaf_.c login_.c main_.c manifest_.c md5_.c merge_.c merge3_.c name_.c path_.c pivot_.c popen_.c pqueue_.c printf_.c rebuild_.c report_.c rss_.c schema_.c search_.c setup_.c sha1_.c shun_.c skins_.c sqlcmd_.c stash_.c stat_.c style_.c sync_.c tag_.c tar_.c th_main_.c timeline_.c tkt_.c tktsetup_.c undo_.c update_.c url_.c user_.c verify_.c vfile_.c wiki_.c wikiformat_.c winhttp_.c xfer_.c zip_.c
2828
2929
 OBJ = $(OBJDIR)\add$O $(OBJDIR)\allrepo$O $(OBJDIR)\attach$O $(OBJDIR)\bag$O $(OBJDIR)\bisect$O $(OBJDIR)\blob$O $(OBJDIR)\branch$O $(OBJDIR)\browse$O $(OBJDIR)\captcha$O $(OBJDIR)\cgi$O $(OBJDIR)\checkin$O $(OBJDIR)\checkout$O $(OBJDIR)\clearsign$O $(OBJDIR)\clone$O $(OBJDIR)\comformat$O $(OBJDIR)\configure$O $(OBJDIR)\content$O $(OBJDIR)\db$O $(OBJDIR)\delta$O $(OBJDIR)\deltacmd$O $(OBJDIR)\descendants$O $(OBJDIR)\diff$O $(OBJDIR)\diffcmd$O $(OBJDIR)\doc$O $(OBJDIR)\encode$O $(OBJDIR)\event$O $(OBJDIR)\export$O $(OBJDIR)\file$O $(OBJDIR)\finfo$O $(OBJDIR)\glob$O $(OBJDIR)\graph$O $(OBJDIR)\gzip$O $(OBJDIR)\http$O $(OBJDIR)\http_socket$O $(OBJDIR)\http_ssl$O $(OBJDIR)\http_transport$O $(OBJDIR)\import$O $(OBJDIR)\info$O $(OBJDIR)\leaf$O $(OBJDIR)\login$O $(OBJDIR)\main$O $(OBJDIR)\manifest$O $(OBJDIR)\md5$O $(OBJDIR)\merge$O $(OBJDIR)\merge3$O $(OBJDIR)\name$O $(OBJDIR)\path$O $(OBJDIR)\pivot$O $(OBJDIR)\popen$O $(OBJDIR)\pqueue$O $(OBJDIR)\printf$O $(OBJDIR)\rebuild$O $(OBJDIR)\report$O $(OBJDIR)\rss$O $(OBJDIR)\schema$O $(OBJDIR)\search$O $(OBJDIR)\setup$O $(OBJDIR)\sha1$O $(OBJDIR)\shun$O $(OBJDIR)\skins$O $(OBJDIR)\sqlcmd$O $(OBJDIR)\stash$O $(OBJDIR)\stat$O $(OBJDIR)\style$O $(OBJDIR)\sync$O $(OBJDIR)\tag$O $(OBJDIR)\tar$O $(OBJDIR)\th_main$O $(OBJDIR)\timeline$O $(OBJDIR)\tkt$O $(OBJDIR)\tktsetup$O $(OBJDIR)\undo$O $(OBJDIR)\update$O $(OBJDIR)\url$O $(OBJDIR)\user$O $(OBJDIR)\verify$O $(OBJDIR)\vfile$O $(OBJDIR)\wiki$O $(OBJDIR)\wikiformat$O $(OBJDIR)\winhttp$O $(OBJDIR)\xfer$O $(OBJDIR)\zip$O $(OBJDIR)\shell$O $(OBJDIR)\sqlite3$O $(OBJDIR)\th$O $(OBJDIR)\th_lang$O
3030
@@ -58,11 +58,11 @@
5858
 $(BCC) -o$@ $**
5959
6060
 mkindex$E: $(SRCDIR)\mkindex.c
6161
 $(BCC) -o$@ $**
6262
63
-version$E: $B\win\version.c
63
+version$E: $B\src\mkversion.c
6464
 $(BCC) -o$@ $**
6565
6666
 $(OBJDIR)\shell$O : $(SRCDIR)\shell.c
6767
 $(TCC) -o$@ -c -Dmain=sqlite3_shell $(SQLITE_OPTIONS) $**
6868
@@ -73,11 +73,11 @@
7373
 $(TCC) -o$@ -c $**
7474
7575
 $(OBJDIR)\th_lang$O : $(SRCDIR)\th_lang.c
7676
 $(TCC) -o$@ -c $**
7777
78
-VERSION.h : version$E $B\manifest.uuid $B\manifest
78
+VERSION.h : version$E $B\manifest.uuid $B\manifest $B\VERSION
7979
 +$** > $@
8080
8181
 page_index.h: mkindex$E $(SRC)
8282
 +$** > $@
8383
@@ -84,11 +84,11 @@
8484
 clean:
8585
 -del $(OBJDIR)\*.obj
8686
 -del *.obj *_.c *.h *.map
8787
8888
 realclean:
89
- -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
89
+ -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
9090
9191
9292
 $(OBJDIR)\add$O : add_.c add.h
9393
 $(TCC) -o$@ -c add_.c
9494
9595
--- win/Makefile.dmc
+++ win/Makefile.dmc
@@ -20,11 +20,11 @@
20 CFLAGS = -o
21 BCC = $(DMDIR)\bin\dmc $(CFLAGS)
22 TCC = $(DMDIR)\bin\dmc $(CFLAGS) $(DMCDEF) $(SSL) $(INCL)
23 LIBS = $(DMDIR)\extra\lib\ zlib wsock32
24
25 SQLITE_OPTIONS = -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT2 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0
26
27 SRC = add_.c allrepo_.c attach_.c bag_.c bisect_.c blob_.c branch_.c browse_.c captcha_.c cgi_.c checkin_.c checkout_.c clearsign_.c clone_.c comformat_.c configure_.c content_.c db_.c delta_.c deltacmd_.c descendants_.c diff_.c diffcmd_.c doc_.c encode_.c event_.c export_.c file_.c finfo_.c glob_.c graph_.c gzip_.c http_.c http_socket_.c http_ssl_.c http_transport_.c import_.c info_.c leaf_.c login_.c main_.c manifest_.c md5_.c merge_.c merge3_.c name_.c path_.c pivot_.c popen_.c pqueue_.c printf_.c rebuild_.c report_.c rss_.c schema_.c search_.c setup_.c sha1_.c shun_.c skins_.c sqlcmd_.c stash_.c stat_.c style_.c sync_.c tag_.c tar_.c th_main_.c timeline_.c tkt_.c tktsetup_.c undo_.c update_.c url_.c user_.c verify_.c vfile_.c wiki_.c wikiformat_.c winhttp_.c xfer_.c zip_.c
28
29 OBJ = $(OBJDIR)\add$O $(OBJDIR)\allrepo$O $(OBJDIR)\attach$O $(OBJDIR)\bag$O $(OBJDIR)\bisect$O $(OBJDIR)\blob$O $(OBJDIR)\branch$O $(OBJDIR)\browse$O $(OBJDIR)\captcha$O $(OBJDIR)\cgi$O $(OBJDIR)\checkin$O $(OBJDIR)\checkout$O $(OBJDIR)\clearsign$O $(OBJDIR)\clone$O $(OBJDIR)\comformat$O $(OBJDIR)\configure$O $(OBJDIR)\content$O $(OBJDIR)\db$O $(OBJDIR)\delta$O $(OBJDIR)\deltacmd$O $(OBJDIR)\descendants$O $(OBJDIR)\diff$O $(OBJDIR)\diffcmd$O $(OBJDIR)\doc$O $(OBJDIR)\encode$O $(OBJDIR)\event$O $(OBJDIR)\export$O $(OBJDIR)\file$O $(OBJDIR)\finfo$O $(OBJDIR)\glob$O $(OBJDIR)\graph$O $(OBJDIR)\gzip$O $(OBJDIR)\http$O $(OBJDIR)\http_socket$O $(OBJDIR)\http_ssl$O $(OBJDIR)\http_transport$O $(OBJDIR)\import$O $(OBJDIR)\info$O $(OBJDIR)\leaf$O $(OBJDIR)\login$O $(OBJDIR)\main$O $(OBJDIR)\manifest$O $(OBJDIR)\md5$O $(OBJDIR)\merge$O $(OBJDIR)\merge3$O $(OBJDIR)\name$O $(OBJDIR)\path$O $(OBJDIR)\pivot$O $(OBJDIR)\popen$O $(OBJDIR)\pqueue$O $(OBJDIR)\printf$O $(OBJDIR)\rebuild$O $(OBJDIR)\report$O $(OBJDIR)\rss$O $(OBJDIR)\schema$O $(OBJDIR)\search$O $(OBJDIR)\setup$O $(OBJDIR)\sha1$O $(OBJDIR)\shun$O $(OBJDIR)\skins$O $(OBJDIR)\sqlcmd$O $(OBJDIR)\stash$O $(OBJDIR)\stat$O $(OBJDIR)\style$O $(OBJDIR)\sync$O $(OBJDIR)\tag$O $(OBJDIR)\tar$O $(OBJDIR)\th_main$O $(OBJDIR)\timeline$O $(OBJDIR)\tkt$O $(OBJDIR)\tktsetup$O $(OBJDIR)\undo$O $(OBJDIR)\update$O $(OBJDIR)\url$O $(OBJDIR)\user$O $(OBJDIR)\verify$O $(OBJDIR)\vfile$O $(OBJDIR)\wiki$O $(OBJDIR)\wikiformat$O $(OBJDIR)\winhttp$O $(OBJDIR)\xfer$O $(OBJDIR)\zip$O $(OBJDIR)\shell$O $(OBJDIR)\sqlite3$O $(OBJDIR)\th$O $(OBJDIR)\th_lang$O
30
@@ -58,11 +58,11 @@
58 $(BCC) -o$@ $**
59
60 mkindex$E: $(SRCDIR)\mkindex.c
61 $(BCC) -o$@ $**
62
63 version$E: $B\win\version.c
64 $(BCC) -o$@ $**
65
66 $(OBJDIR)\shell$O : $(SRCDIR)\shell.c
67 $(TCC) -o$@ -c -Dmain=sqlite3_shell $(SQLITE_OPTIONS) $**
68
@@ -73,11 +73,11 @@
73 $(TCC) -o$@ -c $**
74
75 $(OBJDIR)\th_lang$O : $(SRCDIR)\th_lang.c
76 $(TCC) -o$@ -c $**
77
78 VERSION.h : version$E $B\manifest.uuid $B\manifest
79 +$** > $@
80
81 page_index.h: mkindex$E $(SRC)
82 +$** > $@
83
@@ -84,11 +84,11 @@
84 clean:
85 -del $(OBJDIR)\*.obj
86 -del *.obj *_.c *.h *.map
87
88 realclean:
89 -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
90
91
92 $(OBJDIR)\add$O : add_.c add.h
93 $(TCC) -o$@ -c add_.c
94
95
--- win/Makefile.dmc
+++ win/Makefile.dmc
@@ -20,11 +20,11 @@
20 CFLAGS = -o
21 BCC = $(DMDIR)\bin\dmc $(CFLAGS)
22 TCC = $(DMDIR)\bin\dmc $(CFLAGS) $(DMCDEF) $(SSL) $(INCL)
23 LIBS = $(DMDIR)\extra\lib\ zlib wsock32
24
25 SQLITE_OPTIONS = -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT3 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0
26
27 SRC = add_.c allrepo_.c attach_.c bag_.c bisect_.c blob_.c branch_.c browse_.c captcha_.c cgi_.c checkin_.c checkout_.c clearsign_.c clone_.c comformat_.c configure_.c content_.c db_.c delta_.c deltacmd_.c descendants_.c diff_.c diffcmd_.c doc_.c encode_.c event_.c export_.c file_.c finfo_.c glob_.c graph_.c gzip_.c http_.c http_socket_.c http_ssl_.c http_transport_.c import_.c info_.c leaf_.c login_.c main_.c manifest_.c md5_.c merge_.c merge3_.c name_.c path_.c pivot_.c popen_.c pqueue_.c printf_.c rebuild_.c report_.c rss_.c schema_.c search_.c setup_.c sha1_.c shun_.c skins_.c sqlcmd_.c stash_.c stat_.c style_.c sync_.c tag_.c tar_.c th_main_.c timeline_.c tkt_.c tktsetup_.c undo_.c update_.c url_.c user_.c verify_.c vfile_.c wiki_.c wikiformat_.c winhttp_.c xfer_.c zip_.c
28
29 OBJ = $(OBJDIR)\add$O $(OBJDIR)\allrepo$O $(OBJDIR)\attach$O $(OBJDIR)\bag$O $(OBJDIR)\bisect$O $(OBJDIR)\blob$O $(OBJDIR)\branch$O $(OBJDIR)\browse$O $(OBJDIR)\captcha$O $(OBJDIR)\cgi$O $(OBJDIR)\checkin$O $(OBJDIR)\checkout$O $(OBJDIR)\clearsign$O $(OBJDIR)\clone$O $(OBJDIR)\comformat$O $(OBJDIR)\configure$O $(OBJDIR)\content$O $(OBJDIR)\db$O $(OBJDIR)\delta$O $(OBJDIR)\deltacmd$O $(OBJDIR)\descendants$O $(OBJDIR)\diff$O $(OBJDIR)\diffcmd$O $(OBJDIR)\doc$O $(OBJDIR)\encode$O $(OBJDIR)\event$O $(OBJDIR)\export$O $(OBJDIR)\file$O $(OBJDIR)\finfo$O $(OBJDIR)\glob$O $(OBJDIR)\graph$O $(OBJDIR)\gzip$O $(OBJDIR)\http$O $(OBJDIR)\http_socket$O $(OBJDIR)\http_ssl$O $(OBJDIR)\http_transport$O $(OBJDIR)\import$O $(OBJDIR)\info$O $(OBJDIR)\leaf$O $(OBJDIR)\login$O $(OBJDIR)\main$O $(OBJDIR)\manifest$O $(OBJDIR)\md5$O $(OBJDIR)\merge$O $(OBJDIR)\merge3$O $(OBJDIR)\name$O $(OBJDIR)\path$O $(OBJDIR)\pivot$O $(OBJDIR)\popen$O $(OBJDIR)\pqueue$O $(OBJDIR)\printf$O $(OBJDIR)\rebuild$O $(OBJDIR)\report$O $(OBJDIR)\rss$O $(OBJDIR)\schema$O $(OBJDIR)\search$O $(OBJDIR)\setup$O $(OBJDIR)\sha1$O $(OBJDIR)\shun$O $(OBJDIR)\skins$O $(OBJDIR)\sqlcmd$O $(OBJDIR)\stash$O $(OBJDIR)\stat$O $(OBJDIR)\style$O $(OBJDIR)\sync$O $(OBJDIR)\tag$O $(OBJDIR)\tar$O $(OBJDIR)\th_main$O $(OBJDIR)\timeline$O $(OBJDIR)\tkt$O $(OBJDIR)\tktsetup$O $(OBJDIR)\undo$O $(OBJDIR)\update$O $(OBJDIR)\url$O $(OBJDIR)\user$O $(OBJDIR)\verify$O $(OBJDIR)\vfile$O $(OBJDIR)\wiki$O $(OBJDIR)\wikiformat$O $(OBJDIR)\winhttp$O $(OBJDIR)\xfer$O $(OBJDIR)\zip$O $(OBJDIR)\shell$O $(OBJDIR)\sqlite3$O $(OBJDIR)\th$O $(OBJDIR)\th_lang$O
30
@@ -58,11 +58,11 @@
58 $(BCC) -o$@ $**
59
60 mkindex$E: $(SRCDIR)\mkindex.c
61 $(BCC) -o$@ $**
62
63 version$E: $B\src\mkversion.c
64 $(BCC) -o$@ $**
65
66 $(OBJDIR)\shell$O : $(SRCDIR)\shell.c
67 $(TCC) -o$@ -c -Dmain=sqlite3_shell $(SQLITE_OPTIONS) $**
68
@@ -73,11 +73,11 @@
73 $(TCC) -o$@ -c $**
74
75 $(OBJDIR)\th_lang$O : $(SRCDIR)\th_lang.c
76 $(TCC) -o$@ -c $**
77
78 VERSION.h : version$E $B\manifest.uuid $B\manifest $B\VERSION
79 +$** > $@
80
81 page_index.h: mkindex$E $(SRC)
82 +$** > $@
83
@@ -84,11 +84,11 @@
84 clean:
85 -del $(OBJDIR)\*.obj
86 -del *.obj *_.c *.h *.map
87
88 realclean:
89 -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
90
91
92 $(OBJDIR)\add$O : add_.c add.h
93 $(TCC) -o$@ -c add_.c
94
95
M win/Makefile.mingw
+5 -5
--- win/Makefile.mingw
+++ win/Makefile.mingw
@@ -351,21 +351,21 @@
351351
 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
352352
353353
 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
354354
 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
355355
356
-$(VERSION): $(SRCDIR)/../win/version.c
357
- $(BCC) -o $(OBJDIR)/version $(SRCDIR)/../win/version.c
356
+$(VERSION): $(SRCDIR)/mkversion.c
357
+ $(BCC) -o $(OBJDIR)/version $(SRCDIR)/mkversion.c
358358
359359
 # WARNING. DANGER. Running the testsuite modifies the repository the
360360
 # build is done from, i.e. the checkout belongs to. Do not sync/push
361361
 # the repository after running the tests.
362362
 test: $(APPNAME)
363363
 $(TCLSH) test/tester.tcl $(APPNAME)
364364
365365
 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(VERSION)
366
- $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest >$(OBJDIR)/VERSION.h
366
+ $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
367367
368368
 EXTRAOBJ = $(OBJDIR)/sqlite3.o $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
369369
370370
 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ) $(OBJDIR)/icon.o
371371
 $(TCC) -o $(APPNAME) $(OBJ) $(EXTRAOBJ) $(LIB) $(OBJDIR)/icon.o
@@ -968,16 +968,16 @@
968968
 $(OBJDIR)/zip.o: $(OBJDIR)/zip_.c $(OBJDIR)/zip.h $(SRCDIR)/config.h
969969
 $(XTCC) -o $(OBJDIR)/zip.o -c $(OBJDIR)/zip_.c
970970
971971
 zip.h: $(OBJDIR)/headers
972972
 $(OBJDIR)/sqlite3.o: $(SRCDIR)/sqlite3.c
973
- $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT2 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
973
+ $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT3 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
974974
975
-$(OBJDIR)/shell.o: $(SRCDIR)/shell.c
975
+$(OBJDIR)/shell.o: $(SRCDIR)/shell.c $(SRCDIR)/sqlite3.h
976976
 $(XTCC) -Dmain=sqlite3_shell -DSQLITE_OMIT_LOAD_EXTENSION=1 -c $(SRCDIR)/shell.c -o $(OBJDIR)/shell.o
977977
978978
 $(OBJDIR)/th.o: $(SRCDIR)/th.c
979979
 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th.c -o $(OBJDIR)/th.o
980980
981981
 $(OBJDIR)/th_lang.o: $(SRCDIR)/th_lang.c
982982
 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th_lang.c -o $(OBJDIR)/th_lang.o
983983
984984
--- win/Makefile.mingw
+++ win/Makefile.mingw
@@ -351,21 +351,21 @@
351 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
352
353 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
354 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
355
356 $(VERSION): $(SRCDIR)/../win/version.c
357 $(BCC) -o $(OBJDIR)/version $(SRCDIR)/../win/version.c
358
359 # WARNING. DANGER. Running the testsuite modifies the repository the
360 # build is done from, i.e. the checkout belongs to. Do not sync/push
361 # the repository after running the tests.
362 test: $(APPNAME)
363 $(TCLSH) test/tester.tcl $(APPNAME)
364
365 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(VERSION)
366 $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest >$(OBJDIR)/VERSION.h
367
368 EXTRAOBJ = $(OBJDIR)/sqlite3.o $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
369
370 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ) $(OBJDIR)/icon.o
371 $(TCC) -o $(APPNAME) $(OBJ) $(EXTRAOBJ) $(LIB) $(OBJDIR)/icon.o
@@ -968,16 +968,16 @@
968 $(OBJDIR)/zip.o: $(OBJDIR)/zip_.c $(OBJDIR)/zip.h $(SRCDIR)/config.h
969 $(XTCC) -o $(OBJDIR)/zip.o -c $(OBJDIR)/zip_.c
970
971 zip.h: $(OBJDIR)/headers
972 $(OBJDIR)/sqlite3.o: $(SRCDIR)/sqlite3.c
973 $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT2 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
974
975 $(OBJDIR)/shell.o: $(SRCDIR)/shell.c
976 $(XTCC) -Dmain=sqlite3_shell -DSQLITE_OMIT_LOAD_EXTENSION=1 -c $(SRCDIR)/shell.c -o $(OBJDIR)/shell.o
977
978 $(OBJDIR)/th.o: $(SRCDIR)/th.c
979 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th.c -o $(OBJDIR)/th.o
980
981 $(OBJDIR)/th_lang.o: $(SRCDIR)/th_lang.c
982 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th_lang.c -o $(OBJDIR)/th_lang.o
983
984
--- win/Makefile.mingw
+++ win/Makefile.mingw
@@ -351,21 +351,21 @@
351 $(BCC) -o $(OBJDIR)/makeheaders $(SRCDIR)/makeheaders.c
352
353 $(OBJDIR)/mkindex: $(SRCDIR)/mkindex.c
354 $(BCC) -o $(OBJDIR)/mkindex $(SRCDIR)/mkindex.c
355
356 $(VERSION): $(SRCDIR)/mkversion.c
357 $(BCC) -o $(OBJDIR)/version $(SRCDIR)/mkversion.c
358
359 # WARNING. DANGER. Running the testsuite modifies the repository the
360 # build is done from, i.e. the checkout belongs to. Do not sync/push
361 # the repository after running the tests.
362 test: $(APPNAME)
363 $(TCLSH) test/tester.tcl $(APPNAME)
364
365 $(OBJDIR)/VERSION.h: $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(VERSION)
366 $(VERSION) $(SRCDIR)/../manifest.uuid $(SRCDIR)/../manifest $(SRCDIR)/../VERSION >$(OBJDIR)/VERSION.h
367
368 EXTRAOBJ = $(OBJDIR)/sqlite3.o $(OBJDIR)/shell.o $(OBJDIR)/th.o $(OBJDIR)/th_lang.o
369
370 $(APPNAME): $(OBJDIR)/headers $(OBJ) $(EXTRAOBJ) $(OBJDIR)/icon.o
371 $(TCC) -o $(APPNAME) $(OBJ) $(EXTRAOBJ) $(LIB) $(OBJDIR)/icon.o
@@ -968,16 +968,16 @@
968 $(OBJDIR)/zip.o: $(OBJDIR)/zip_.c $(OBJDIR)/zip.h $(SRCDIR)/config.h
969 $(XTCC) -o $(OBJDIR)/zip.o -c $(OBJDIR)/zip_.c
970
971 zip.h: $(OBJDIR)/headers
972 $(OBJDIR)/sqlite3.o: $(SRCDIR)/sqlite3.c
973 $(XTCC) -DSQLITE_OMIT_LOAD_EXTENSION=1 -DSQLITE_THREADSAFE=0 -DSQLITE_DEFAULT_FILE_FORMAT=4 -DSQLITE_ENABLE_STAT3 -Dlocaltime=fossil_localtime -DSQLITE_ENABLE_LOCKING_STYLE=0 -c $(SRCDIR)/sqlite3.c -o $(OBJDIR)/sqlite3.o
974
975 $(OBJDIR)/shell.o: $(SRCDIR)/shell.c $(SRCDIR)/sqlite3.h
976 $(XTCC) -Dmain=sqlite3_shell -DSQLITE_OMIT_LOAD_EXTENSION=1 -c $(SRCDIR)/shell.c -o $(OBJDIR)/shell.o
977
978 $(OBJDIR)/th.o: $(SRCDIR)/th.c
979 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th.c -o $(OBJDIR)/th.o
980
981 $(OBJDIR)/th_lang.o: $(SRCDIR)/th_lang.c
982 $(XTCC) -I$(SRCDIR) -c $(SRCDIR)/th_lang.c -o $(OBJDIR)/th_lang.o
983
984
M win/Makefile.msc
+93 -8
--- win/Makefile.msc
+++ win/Makefile.msc
@@ -31,14 +31,14 @@
3131
 INCL = -I. -I$(SRCDIR) -I$B\win\include -I$(MSCDIR)\extra\include -I$(ZINCDIR)
3232
3333
 CFLAGS = -nologo -MT -O2
3434
 BCC = $(CC) $(CFLAGS)
3535
 TCC = $(CC) -c $(CFLAGS) $(MSCDEF) $(SSL) $(INCL)
36
-LIBS = $(ZLIB) ws2_32.lib $(SSLLIB)
36
+LIBS = $(ZLIB) ws2_32.lib advapi32.lib $(SSLLIB)
3737
 LIBDIR = -LIBPATH:$(MSCDIR)\extra\lib -LIBPATH:$(ZLIBDIR)
3838
39
-SQLITE_OPTIONS = /DSQLITE_OMIT_LOAD_EXTENSION=1 /DSQLITE_THREADSAFE=0 /DSQLITE_DEFAULT_FILE_FORMAT=4 /DSQLITE_ENABLE_STAT2 /Dlocaltime=fossil_localtime /DSQLITE_ENABLE_LOCKING_STYLE=0
39
+SQLITE_OPTIONS = /DSQLITE_OMIT_LOAD_EXTENSION=1 /DSQLITE_THREADSAFE=0 /DSQLITE_DEFAULT_FILE_FORMAT=4 /DSQLITE_ENABLE_STAT3 /Dlocaltime=fossil_localtime /DSQLITE_ENABLE_LOCKING_STYLE=0
4040
4141
 SRC = add_.c allrepo_.c attach_.c bag_.c bisect_.c blob_.c branch_.c browse_.c captcha_.c cgi_.c checkin_.c checkout_.c clearsign_.c clone_.c comformat_.c configure_.c content_.c db_.c delta_.c deltacmd_.c descendants_.c diff_.c diffcmd_.c doc_.c encode_.c event_.c export_.c file_.c finfo_.c glob_.c graph_.c gzip_.c http_.c http_socket_.c http_ssl_.c http_transport_.c import_.c info_.c leaf_.c login_.c main_.c manifest_.c md5_.c merge_.c merge3_.c name_.c path_.c pivot_.c popen_.c pqueue_.c printf_.c rebuild_.c report_.c rss_.c schema_.c search_.c setup_.c sha1_.c shun_.c skins_.c sqlcmd_.c stash_.c stat_.c style_.c sync_.c tag_.c tar_.c th_main_.c timeline_.c tkt_.c tktsetup_.c undo_.c update_.c url_.c user_.c verify_.c vfile_.c wiki_.c wikiformat_.c winhttp_.c xfer_.c zip_.c
4242
4343
 OBJ = $(OX)\add$O $(OX)\allrepo$O $(OX)\attach$O $(OX)\bag$O $(OX)\bisect$O $(OX)\blob$O $(OX)\branch$O $(OX)\browse$O $(OX)\captcha$O $(OX)\cgi$O $(OX)\checkin$O $(OX)\checkout$O $(OX)\clearsign$O $(OX)\clone$O $(OX)\comformat$O $(OX)\configure$O $(OX)\content$O $(OX)\db$O $(OX)\delta$O $(OX)\deltacmd$O $(OX)\descendants$O $(OX)\diff$O $(OX)\diffcmd$O $(OX)\doc$O $(OX)\encode$O $(OX)\event$O $(OX)\export$O $(OX)\file$O $(OX)\finfo$O $(OX)\glob$O $(OX)\graph$O $(OX)\gzip$O $(OX)\http$O $(OX)\http_socket$O $(OX)\http_ssl$O $(OX)\http_transport$O $(OX)\import$O $(OX)\info$O $(OX)\leaf$O $(OX)\login$O $(OX)\main$O $(OX)\manifest$O $(OX)\md5$O $(OX)\merge$O $(OX)\merge3$O $(OX)\name$O $(OX)\path$O $(OX)\pivot$O $(OX)\popen$O $(OX)\pqueue$O $(OX)\printf$O $(OX)\rebuild$O $(OX)\report$O $(OX)\rss$O $(OX)\schema$O $(OX)\search$O $(OX)\setup$O $(OX)\sha1$O $(OX)\shun$O $(OX)\skins$O $(OX)\sqlcmd$O $(OX)\stash$O $(OX)\stat$O $(OX)\style$O $(OX)\sync$O $(OX)\tag$O $(OX)\tar$O $(OX)\th_main$O $(OX)\timeline$O $(OX)\tkt$O $(OX)\tktsetup$O $(OX)\undo$O $(OX)\update$O $(OX)\url$O $(OX)\user$O $(OX)\verify$O $(OX)\vfile$O $(OX)\wiki$O $(OX)\wikiformat$O $(OX)\winhttp$O $(OX)\xfer$O $(OX)\zip$O $(OX)\shell$O $(OX)\sqlite3$O $(OX)\th$O $(OX)\th_lang$O
4444
@@ -47,14 +47,99 @@
4747
4848
 all: $(OX) $(APPNAME)
4949
5050
 $(APPNAME) : translate$E mkindex$E headers $(OBJ) $(OX)\linkopts
5151
 cd $(OX)
52
- link -LINK -OUT:$@ $(LIBDIR) @linkopts
52
+ link /NODEFAULTLIB:msvcrt -OUT:$@ $(LIBDIR) @linkopts
5353
5454
 $(OX)\linkopts: $B\win\Makefile.msc
55
- echo add allrepo attach bag bisect blob branch browse captcha cgi checkin checkout clearsign clone comformat configure content db delta deltacmd descendants diff diffcmd doc encode event export file finfo glob graph gzip http http_socket http_ssl http_transport import info leaf login main manifest md5 merge merge3 name path pivot popen pqueue printf rebuild report rss schema search setup sha1 shun skins sqlcmd stash stat style sync tag tar th_main timeline tkt tktsetup undo update url user verify vfile wiki wikiformat winhttp xfer zip sqlite3 th th_lang > $@
55
+ echo $(OX)\add.obj > $@
56
+ echo $(OX)\allrepo.obj >> $@
57
+ echo $(OX)\attach.obj >> $@
58
+ echo $(OX)\bag.obj >> $@
59
+ echo $(OX)\bisect.obj >> $@
60
+ echo $(OX)\blob.obj >> $@
61
+ echo $(OX)\branch.obj >> $@
62
+ echo $(OX)\browse.obj >> $@
63
+ echo $(OX)\captcha.obj >> $@
64
+ echo $(OX)\cgi.obj >> $@
65
+ echo $(OX)\checkin.obj >> $@
66
+ echo $(OX)\checkout.obj >> $@
67
+ echo $(OX)\clearsign.obj >> $@
68
+ echo $(OX)\clone.obj >> $@
69
+ echo $(OX)\comformat.obj >> $@
70
+ echo $(OX)\configure.obj >> $@
71
+ echo $(OX)\content.obj >> $@
72
+ echo $(OX)\db.obj >> $@
73
+ echo $(OX)\delta.obj >> $@
74
+ echo $(OX)\deltacmd.obj >> $@
75
+ echo $(OX)\descendants.obj >> $@
76
+ echo $(OX)\diff.obj >> $@
77
+ echo $(OX)\diffcmd.obj >> $@
78
+ echo $(OX)\doc.obj >> $@
79
+ echo $(OX)\encode.obj >> $@
80
+ echo $(OX)\event.obj >> $@
81
+ echo $(OX)\export.obj >> $@
82
+ echo $(OX)\file.obj >> $@
83
+ echo $(OX)\finfo.obj >> $@
84
+ echo $(OX)\glob.obj >> $@
85
+ echo $(OX)\graph.obj >> $@
86
+ echo $(OX)\gzip.obj >> $@
87
+ echo $(OX)\http.obj >> $@
88
+ echo $(OX)\http_socket.obj >> $@
89
+ echo $(OX)\http_ssl.obj >> $@
90
+ echo $(OX)\http_transport.obj >> $@
91
+ echo $(OX)\import.obj >> $@
92
+ echo $(OX)\info.obj >> $@
93
+ echo $(OX)\leaf.obj >> $@
94
+ echo $(OX)\login.obj >> $@
95
+ echo $(OX)\main.obj >> $@
96
+ echo $(OX)\manifest.obj >> $@
97
+ echo $(OX)\md5.obj >> $@
98
+ echo $(OX)\merge.obj >> $@
99
+ echo $(OX)\merge3.obj >> $@
100
+ echo $(OX)\name.obj >> $@
101
+ echo $(OX)\path.obj >> $@
102
+ echo $(OX)\pivot.obj >> $@
103
+ echo $(OX)\popen.obj >> $@
104
+ echo $(OX)\pqueue.obj >> $@
105
+ echo $(OX)\printf.obj >> $@
106
+ echo $(OX)\rebuild.obj >> $@
107
+ echo $(OX)\report.obj >> $@
108
+ echo $(OX)\rss.obj >> $@
109
+ echo $(OX)\schema.obj >> $@
110
+ echo $(OX)\search.obj >> $@
111
+ echo $(OX)\setup.obj >> $@
112
+ echo $(OX)\sha1.obj >> $@
113
+ echo $(OX)\shell.obj >> $@
114
+ echo $(OX)\shun.obj >> $@
115
+ echo $(OX)\skins.obj >> $@
116
+ echo $(OX)\sqlcmd.obj >> $@
117
+ echo $(OX)\sqlite3.obj >> $@
118
+ echo $(OX)\stash.obj >> $@
119
+ echo $(OX)\stat.obj >> $@
120
+ echo $(OX)\style.obj >> $@
121
+ echo $(OX)\sync.obj >> $@
122
+ echo $(OX)\tag.obj >> $@
123
+ echo $(OX)\tar.obj >> $@
124
+ echo $(OX)\th.obj >> $@
125
+ echo $(OX)\th_lang.obj >> $@
126
+ echo $(OX)\th_main.obj >> $@
127
+ echo $(OX)\timeline.obj >> $@
128
+ echo $(OX)\tkt.obj >> $@
129
+ echo $(OX)\tktsetup.obj >> $@
130
+ echo $(OX)\undo.obj >> $@
131
+ echo $(OX)\update.obj >> $@
132
+ echo $(OX)\url.obj >> $@
133
+ echo $(OX)\user.obj >> $@
134
+ echo $(OX)\verify.obj >> $@
135
+ echo $(OX)\vfile.obj >> $@
136
+ echo $(OX)\wiki.obj >> $@
137
+ echo $(OX)\wikiformat.obj >> $@
138
+ echo $(OX)\winhttp.obj >> $@
139
+ echo $(OX)\xfer.obj >> $@
140
+ echo $(OX)\zip.obj >> $@
56141
 echo $(LIBS) >> $@
57142
58143
59144
60145
@@ -68,15 +153,15 @@
68153
 $(BCC) $**
69154
70155
 mkindex$E: $(SRCDIR)\mkindex.c
71156
 $(BCC) $**
72157
73
-version$E: $B\win\version.c
158
+mkversion$E: $B\src\mkversion.c
74159
 $(BCC) $**
75160
76161
 $(OX)\shell$O : $(SRCDIR)\shell.c
77
- $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c shell_.c
162
+ $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c $(SRCDIR)\shell.c
78163
79164
 $(OX)\sqlite3$O : $(SRCDIR)\sqlite3.c
80165
 $(TCC) /Fo$@ -c $(SQLITE_OPTIONS) $**
81166
82167
 $(OX)\th$O : $(SRCDIR)\th.c
@@ -83,11 +168,11 @@
83168
 $(TCC) /Fo$@ -c $**
84169
85170
 $(OX)\th_lang$O : $(SRCDIR)\th_lang.c
86171
 $(TCC) /Fo$@ -c $**
87172
88
-VERSION.h : version$E $B\manifest.uuid $B\manifest
173
+VERSION.h : mkversion$E $B\manifest.uuid $B\manifest $B\VERSION
89174
 $** > $@
90175
91176
 page_index.h: mkindex$E $(SRC)
92177
 $** > $@
93178
@@ -95,11 +180,11 @@
95180
 -del $(OX)\*.obj
96181
 -del *.obj *_.c *.h *.map
97182
 -del headers linkopts
98183
99184
 realclean:
100
- -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
185
+ -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
101186
102187
103188
 $(OX)\add$O : add_.c add.h
104189
 $(TCC) /Fo$@ -c add_.c
105190
106191
107192
DELETED win/version.c
--- win/Makefile.msc
+++ win/Makefile.msc
@@ -31,14 +31,14 @@
31 INCL = -I. -I$(SRCDIR) -I$B\win\include -I$(MSCDIR)\extra\include -I$(ZINCDIR)
32
33 CFLAGS = -nologo -MT -O2
34 BCC = $(CC) $(CFLAGS)
35 TCC = $(CC) -c $(CFLAGS) $(MSCDEF) $(SSL) $(INCL)
36 LIBS = $(ZLIB) ws2_32.lib $(SSLLIB)
37 LIBDIR = -LIBPATH:$(MSCDIR)\extra\lib -LIBPATH:$(ZLIBDIR)
38
39 SQLITE_OPTIONS = /DSQLITE_OMIT_LOAD_EXTENSION=1 /DSQLITE_THREADSAFE=0 /DSQLITE_DEFAULT_FILE_FORMAT=4 /DSQLITE_ENABLE_STAT2 /Dlocaltime=fossil_localtime /DSQLITE_ENABLE_LOCKING_STYLE=0
40
41 SRC = add_.c allrepo_.c attach_.c bag_.c bisect_.c blob_.c branch_.c browse_.c captcha_.c cgi_.c checkin_.c checkout_.c clearsign_.c clone_.c comformat_.c configure_.c content_.c db_.c delta_.c deltacmd_.c descendants_.c diff_.c diffcmd_.c doc_.c encode_.c event_.c export_.c file_.c finfo_.c glob_.c graph_.c gzip_.c http_.c http_socket_.c http_ssl_.c http_transport_.c import_.c info_.c leaf_.c login_.c main_.c manifest_.c md5_.c merge_.c merge3_.c name_.c path_.c pivot_.c popen_.c pqueue_.c printf_.c rebuild_.c report_.c rss_.c schema_.c search_.c setup_.c sha1_.c shun_.c skins_.c sqlcmd_.c stash_.c stat_.c style_.c sync_.c tag_.c tar_.c th_main_.c timeline_.c tkt_.c tktsetup_.c undo_.c update_.c url_.c user_.c verify_.c vfile_.c wiki_.c wikiformat_.c winhttp_.c xfer_.c zip_.c
42
43 OBJ = $(OX)\add$O $(OX)\allrepo$O $(OX)\attach$O $(OX)\bag$O $(OX)\bisect$O $(OX)\blob$O $(OX)\branch$O $(OX)\browse$O $(OX)\captcha$O $(OX)\cgi$O $(OX)\checkin$O $(OX)\checkout$O $(OX)\clearsign$O $(OX)\clone$O $(OX)\comformat$O $(OX)\configure$O $(OX)\content$O $(OX)\db$O $(OX)\delta$O $(OX)\deltacmd$O $(OX)\descendants$O $(OX)\diff$O $(OX)\diffcmd$O $(OX)\doc$O $(OX)\encode$O $(OX)\event$O $(OX)\export$O $(OX)\file$O $(OX)\finfo$O $(OX)\glob$O $(OX)\graph$O $(OX)\gzip$O $(OX)\http$O $(OX)\http_socket$O $(OX)\http_ssl$O $(OX)\http_transport$O $(OX)\import$O $(OX)\info$O $(OX)\leaf$O $(OX)\login$O $(OX)\main$O $(OX)\manifest$O $(OX)\md5$O $(OX)\merge$O $(OX)\merge3$O $(OX)\name$O $(OX)\path$O $(OX)\pivot$O $(OX)\popen$O $(OX)\pqueue$O $(OX)\printf$O $(OX)\rebuild$O $(OX)\report$O $(OX)\rss$O $(OX)\schema$O $(OX)\search$O $(OX)\setup$O $(OX)\sha1$O $(OX)\shun$O $(OX)\skins$O $(OX)\sqlcmd$O $(OX)\stash$O $(OX)\stat$O $(OX)\style$O $(OX)\sync$O $(OX)\tag$O $(OX)\tar$O $(OX)\th_main$O $(OX)\timeline$O $(OX)\tkt$O $(OX)\tktsetup$O $(OX)\undo$O $(OX)\update$O $(OX)\url$O $(OX)\user$O $(OX)\verify$O $(OX)\vfile$O $(OX)\wiki$O $(OX)\wikiformat$O $(OX)\winhttp$O $(OX)\xfer$O $(OX)\zip$O $(OX)\shell$O $(OX)\sqlite3$O $(OX)\th$O $(OX)\th_lang$O
44
@@ -47,14 +47,99 @@
47
48 all: $(OX) $(APPNAME)
49
50 $(APPNAME) : translate$E mkindex$E headers $(OBJ) $(OX)\linkopts
51 cd $(OX)
52 link -LINK -OUT:$@ $(LIBDIR) @linkopts
53
54 $(OX)\linkopts: $B\win\Makefile.msc
55 echo add allrepo attach bag bisect blob branch browse captcha cgi checkin checkout clearsign clone comformat configure content db delta deltacmd descendants diff diffcmd doc encode event export file finfo glob graph gzip http http_socket http_ssl http_transport import info leaf login main manifest md5 merge merge3 name path pivot popen pqueue printf rebuild report rss schema search setup sha1 shun skins sqlcmd stash stat style sync tag tar th_main timeline tkt tktsetup undo update url user verify vfile wiki wikiformat winhttp xfer zip sqlite3 th th_lang > $@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
56 echo $(LIBS) >> $@
57
58
59
60
@@ -68,15 +153,15 @@
68 $(BCC) $**
69
70 mkindex$E: $(SRCDIR)\mkindex.c
71 $(BCC) $**
72
73 version$E: $B\win\version.c
74 $(BCC) $**
75
76 $(OX)\shell$O : $(SRCDIR)\shell.c
77 $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c shell_.c
78
79 $(OX)\sqlite3$O : $(SRCDIR)\sqlite3.c
80 $(TCC) /Fo$@ -c $(SQLITE_OPTIONS) $**
81
82 $(OX)\th$O : $(SRCDIR)\th.c
@@ -83,11 +168,11 @@
83 $(TCC) /Fo$@ -c $**
84
85 $(OX)\th_lang$O : $(SRCDIR)\th_lang.c
86 $(TCC) /Fo$@ -c $**
87
88 VERSION.h : version$E $B\manifest.uuid $B\manifest
89 $** > $@
90
91 page_index.h: mkindex$E $(SRC)
92 $** > $@
93
@@ -95,11 +180,11 @@
95 -del $(OX)\*.obj
96 -del *.obj *_.c *.h *.map
97 -del headers linkopts
98
99 realclean:
100 -del $(APPNAME) translate$E mkindex$E makeheaders$E version$E
101
102
103 $(OX)\add$O : add_.c add.h
104 $(TCC) /Fo$@ -c add_.c
105
106
107 ELETED win/version.c
--- win/Makefile.msc
+++ win/Makefile.msc
@@ -31,14 +31,14 @@
31 INCL = -I. -I$(SRCDIR) -I$B\win\include -I$(MSCDIR)\extra\include -I$(ZINCDIR)
32
33 CFLAGS = -nologo -MT -O2
34 BCC = $(CC) $(CFLAGS)
35 TCC = $(CC) -c $(CFLAGS) $(MSCDEF) $(SSL) $(INCL)
36 LIBS = $(ZLIB) ws2_32.lib advapi32.lib $(SSLLIB)
37 LIBDIR = -LIBPATH:$(MSCDIR)\extra\lib -LIBPATH:$(ZLIBDIR)
38
39 SQLITE_OPTIONS = /DSQLITE_OMIT_LOAD_EXTENSION=1 /DSQLITE_THREADSAFE=0 /DSQLITE_DEFAULT_FILE_FORMAT=4 /DSQLITE_ENABLE_STAT3 /Dlocaltime=fossil_localtime /DSQLITE_ENABLE_LOCKING_STYLE=0
40
41 SRC = add_.c allrepo_.c attach_.c bag_.c bisect_.c blob_.c branch_.c browse_.c captcha_.c cgi_.c checkin_.c checkout_.c clearsign_.c clone_.c comformat_.c configure_.c content_.c db_.c delta_.c deltacmd_.c descendants_.c diff_.c diffcmd_.c doc_.c encode_.c event_.c export_.c file_.c finfo_.c glob_.c graph_.c gzip_.c http_.c http_socket_.c http_ssl_.c http_transport_.c import_.c info_.c leaf_.c login_.c main_.c manifest_.c md5_.c merge_.c merge3_.c name_.c path_.c pivot_.c popen_.c pqueue_.c printf_.c rebuild_.c report_.c rss_.c schema_.c search_.c setup_.c sha1_.c shun_.c skins_.c sqlcmd_.c stash_.c stat_.c style_.c sync_.c tag_.c tar_.c th_main_.c timeline_.c tkt_.c tktsetup_.c undo_.c update_.c url_.c user_.c verify_.c vfile_.c wiki_.c wikiformat_.c winhttp_.c xfer_.c zip_.c
42
43 OBJ = $(OX)\add$O $(OX)\allrepo$O $(OX)\attach$O $(OX)\bag$O $(OX)\bisect$O $(OX)\blob$O $(OX)\branch$O $(OX)\browse$O $(OX)\captcha$O $(OX)\cgi$O $(OX)\checkin$O $(OX)\checkout$O $(OX)\clearsign$O $(OX)\clone$O $(OX)\comformat$O $(OX)\configure$O $(OX)\content$O $(OX)\db$O $(OX)\delta$O $(OX)\deltacmd$O $(OX)\descendants$O $(OX)\diff$O $(OX)\diffcmd$O $(OX)\doc$O $(OX)\encode$O $(OX)\event$O $(OX)\export$O $(OX)\file$O $(OX)\finfo$O $(OX)\glob$O $(OX)\graph$O $(OX)\gzip$O $(OX)\http$O $(OX)\http_socket$O $(OX)\http_ssl$O $(OX)\http_transport$O $(OX)\import$O $(OX)\info$O $(OX)\leaf$O $(OX)\login$O $(OX)\main$O $(OX)\manifest$O $(OX)\md5$O $(OX)\merge$O $(OX)\merge3$O $(OX)\name$O $(OX)\path$O $(OX)\pivot$O $(OX)\popen$O $(OX)\pqueue$O $(OX)\printf$O $(OX)\rebuild$O $(OX)\report$O $(OX)\rss$O $(OX)\schema$O $(OX)\search$O $(OX)\setup$O $(OX)\sha1$O $(OX)\shun$O $(OX)\skins$O $(OX)\sqlcmd$O $(OX)\stash$O $(OX)\stat$O $(OX)\style$O $(OX)\sync$O $(OX)\tag$O $(OX)\tar$O $(OX)\th_main$O $(OX)\timeline$O $(OX)\tkt$O $(OX)\tktsetup$O $(OX)\undo$O $(OX)\update$O $(OX)\url$O $(OX)\user$O $(OX)\verify$O $(OX)\vfile$O $(OX)\wiki$O $(OX)\wikiformat$O $(OX)\winhttp$O $(OX)\xfer$O $(OX)\zip$O $(OX)\shell$O $(OX)\sqlite3$O $(OX)\th$O $(OX)\th_lang$O
44
@@ -47,14 +47,99 @@
47
48 all: $(OX) $(APPNAME)
49
50 $(APPNAME) : translate$E mkindex$E headers $(OBJ) $(OX)\linkopts
51 cd $(OX)
52 link /NODEFAULTLIB:msvcrt -OUT:$@ $(LIBDIR) @linkopts
53
54 $(OX)\linkopts: $B\win\Makefile.msc
55 echo $(OX)\add.obj > $@
56 echo $(OX)\allrepo.obj >> $@
57 echo $(OX)\attach.obj >> $@
58 echo $(OX)\bag.obj >> $@
59 echo $(OX)\bisect.obj >> $@
60 echo $(OX)\blob.obj >> $@
61 echo $(OX)\branch.obj >> $@
62 echo $(OX)\browse.obj >> $@
63 echo $(OX)\captcha.obj >> $@
64 echo $(OX)\cgi.obj >> $@
65 echo $(OX)\checkin.obj >> $@
66 echo $(OX)\checkout.obj >> $@
67 echo $(OX)\clearsign.obj >> $@
68 echo $(OX)\clone.obj >> $@
69 echo $(OX)\comformat.obj >> $@
70 echo $(OX)\configure.obj >> $@
71 echo $(OX)\content.obj >> $@
72 echo $(OX)\db.obj >> $@
73 echo $(OX)\delta.obj >> $@
74 echo $(OX)\deltacmd.obj >> $@
75 echo $(OX)\descendants.obj >> $@
76 echo $(OX)\diff.obj >> $@
77 echo $(OX)\diffcmd.obj >> $@
78 echo $(OX)\doc.obj >> $@
79 echo $(OX)\encode.obj >> $@
80 echo $(OX)\event.obj >> $@
81 echo $(OX)\export.obj >> $@
82 echo $(OX)\file.obj >> $@
83 echo $(OX)\finfo.obj >> $@
84 echo $(OX)\glob.obj >> $@
85 echo $(OX)\graph.obj >> $@
86 echo $(OX)\gzip.obj >> $@
87 echo $(OX)\http.obj >> $@
88 echo $(OX)\http_socket.obj >> $@
89 echo $(OX)\http_ssl.obj >> $@
90 echo $(OX)\http_transport.obj >> $@
91 echo $(OX)\import.obj >> $@
92 echo $(OX)\info.obj >> $@
93 echo $(OX)\leaf.obj >> $@
94 echo $(OX)\login.obj >> $@
95 echo $(OX)\main.obj >> $@
96 echo $(OX)\manifest.obj >> $@
97 echo $(OX)\md5.obj >> $@
98 echo $(OX)\merge.obj >> $@
99 echo $(OX)\merge3.obj >> $@
100 echo $(OX)\name.obj >> $@
101 echo $(OX)\path.obj >> $@
102 echo $(OX)\pivot.obj >> $@
103 echo $(OX)\popen.obj >> $@
104 echo $(OX)\pqueue.obj >> $@
105 echo $(OX)\printf.obj >> $@
106 echo $(OX)\rebuild.obj >> $@
107 echo $(OX)\report.obj >> $@
108 echo $(OX)\rss.obj >> $@
109 echo $(OX)\schema.obj >> $@
110 echo $(OX)\search.obj >> $@
111 echo $(OX)\setup.obj >> $@
112 echo $(OX)\sha1.obj >> $@
113 echo $(OX)\shell.obj >> $@
114 echo $(OX)\shun.obj >> $@
115 echo $(OX)\skins.obj >> $@
116 echo $(OX)\sqlcmd.obj >> $@
117 echo $(OX)\sqlite3.obj >> $@
118 echo $(OX)\stash.obj >> $@
119 echo $(OX)\stat.obj >> $@
120 echo $(OX)\style.obj >> $@
121 echo $(OX)\sync.obj >> $@
122 echo $(OX)\tag.obj >> $@
123 echo $(OX)\tar.obj >> $@
124 echo $(OX)\th.obj >> $@
125 echo $(OX)\th_lang.obj >> $@
126 echo $(OX)\th_main.obj >> $@
127 echo $(OX)\timeline.obj >> $@
128 echo $(OX)\tkt.obj >> $@
129 echo $(OX)\tktsetup.obj >> $@
130 echo $(OX)\undo.obj >> $@
131 echo $(OX)\update.obj >> $@
132 echo $(OX)\url.obj >> $@
133 echo $(OX)\user.obj >> $@
134 echo $(OX)\verify.obj >> $@
135 echo $(OX)\vfile.obj >> $@
136 echo $(OX)\wiki.obj >> $@
137 echo $(OX)\wikiformat.obj >> $@
138 echo $(OX)\winhttp.obj >> $@
139 echo $(OX)\xfer.obj >> $@
140 echo $(OX)\zip.obj >> $@
141 echo $(LIBS) >> $@
142
143
144
145
@@ -68,15 +153,15 @@
153 $(BCC) $**
154
155 mkindex$E: $(SRCDIR)\mkindex.c
156 $(BCC) $**
157
158 mkversion$E: $B\src\mkversion.c
159 $(BCC) $**
160
161 $(OX)\shell$O : $(SRCDIR)\shell.c
162 $(TCC) /Fo$@ /Dmain=sqlite3_shell $(SQLITE_OPTIONS) -c $(SRCDIR)\shell.c
163
164 $(OX)\sqlite3$O : $(SRCDIR)\sqlite3.c
165 $(TCC) /Fo$@ -c $(SQLITE_OPTIONS) $**
166
167 $(OX)\th$O : $(SRCDIR)\th.c
@@ -83,11 +168,11 @@
168 $(TCC) /Fo$@ -c $**
169
170 $(OX)\th_lang$O : $(SRCDIR)\th_lang.c
171 $(TCC) /Fo$@ -c $**
172
173 VERSION.h : mkversion$E $B\manifest.uuid $B\manifest $B\VERSION
174 $** > $@
175
176 page_index.h: mkindex$E $(SRC)
177 $** > $@
178
@@ -95,11 +180,11 @@
180 -del $(OX)\*.obj
181 -del *.obj *_.c *.h *.map
182 -del headers linkopts
183
184 realclean:
185 -del $(APPNAME) translate$E mkindex$E makeheaders$E mkversion$E
186
187
188 $(OX)\add$O : add_.c add.h
189 $(TCC) /Fo$@ -c add_.c
190
191
192 ELETED win/version.c
D win/version.c
-10
--- a/win/version.c
+++ b/win/version.c
@@ -1,10 +0,0 @@
1
-fopen(argv[1],"gv[1],exists to do the job that AWK would do for the unix
2
-** makefile - to extract information from the "mainfest" and "manifest.uuid"
3
-** files for this p)
4
- d = _MSC_VER / 1 return 0;
5
-}
6
-b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
7
-}
8
-;
9
- char b[1024b[strlen(b)-1] =0; return 0"gv[1],"fMSC_VER)
10
- fopen(argv[1
--- a/win/version.c
+++ b/win/version.c
@@ -1,10 +0,0 @@
1 fopen(argv[1],"gv[1],exists to do the job that AWK would do for the unix
2 ** makefile - to extract information from the "mainfest" and "manifest.uuid"
3 ** files for this p)
4 d = _MSC_VER / 1 return 0;
5 }
6 b2b2,0,sizeof(b2)strcpy(b2b2b2vx[1000]return 0;
7 }
8 ;
9 char b[1024b[strlen(b)-1] =0; return 0"gv[1],"fMSC_VER)
10 fopen(argv[1
--- a/win/version.c
+++ b/win/version.c
@@ -1,10 +0,0 @@
 
 
 
 
 
 
 
 
 
 
M www/build.wiki
+63 -26
--- www/build.wiki
+++ www/build.wiki
@@ -1,16 +1,29 @@
11
 <title>Building and Installing Fossil</title>
22
3
-<p>This page describes how to build and install Fossil. The
4
-whole process is designed to be very easy.</p>
5
-
63
 <h2>0.0 Using A Pre-compiled Binary</h2>
74
8
-<p>You can skip steps 1.0 and 2.0 below by downloading
5
+<p>You can skip all of the following by downloading
96
 a <a href="http://www.fossil-scm.org/download.html">pre-compiled binary</a>
10
-appropriate for your platform. If you use a pre-compiled binary
11
-jump immediate to step 3.0.</p>
7
+appropriate for your platform and putting that self-contained binary
8
+someplace on your $PATH.
9
+To uninstall, simply delete the binary.
10
+To upgrade from an older release, just overwrite the older binary with
11
+the newer one.</p>
12
+
13
+<h2>0.1 Executive Summary</h2>
14
+
15
+<p>Building and installing is very simple. Three steps:</p>
16
+
17
+<ol>
18
+<li> Download and unpack a source tarball or ZIP.
19
+<li> <b>./configure; make</b>
20
+<li> Move or copy the resulting "fossil" executable to someplace
21
+ on your $PATH.
22
+</ol>
23
+
24
+<p><hr>
1225
1326
 <h2>1.0 Obtaining The Source Code</h2>
1427
1528
 <p>Fossil is self-hosting, so you can obtain a ZIP archive containing
1629
 a snapshot of the latest version directly from fossil's own fossil
@@ -33,42 +46,66 @@
3346
3447
 <li><p>Select a version of of fossil you want to download. Click on its
3548
 link. Note that you must successfully log in as "anonymous" in step 1
3649
 above in order to see the link to the detailed version information.</p></li>
3750
38
-<li><p>Finally, click on the
39
-"Zip Archive" link. This link will build a ZIP archive of the
51
+<li><p>Finally, click on one of the
52
+"Zip Archive" or "Tarball" links, according to your preference.
53
+These link will build a ZIP archive or a gzip-compressed tarball of the
4054
 complete source code and download it to your browser.
4155
 </ol>
4256
4357
 <h2>2.0 Compiling</h2>
4458
45
-<p>Follow these steps to compile on a unix platform
46
-(Linux, *BSD, MacOS, etc):</p>
47
-
4859
 <ol>
4960
 <li value="6">
50
-<p>Create a directory to hold the source code. Then unzip the
51
-ZIP archive you downloaded into that directory. You should be
61
+<p>Create a directory to hold the source code. Then unarchive the
62
+ZIP or tarball you downloaded into that directory. You should be
5263
 in the top-level folder of that directory</p></li>
5364
54
-<li><p><b>(Optional:)</b>
55
-Edit the Makefile to set it up like you want. You probably do not
56
-need to do anything. Do not be intimidated: There are less than 10
57
-variables in the makefile that can be changed. The whole Makefile
58
-is only a few dozen lines long and most of those lines are comments.</p>
65
+<li><i>(Optional, unix only)</i>
66
+Run <b>./configure</b> to construct a makefile.
67
+
68
+<ol type="a">
69
+<li><p>
70
+If you do not have the OpenSSL library installed on your system, then
71
+add <b>--with-openssl=none</b> to omit the https functionality.
72
+
73
+<li><p>
74
+To build a statically linked binary (suitable for use inside a chroot
75
+jail) add the <b>--static</b> option.
76
+
77
+<li><p>
78
+Other configuration options can be seen by running
79
+<b>./configure --help</b>
80
+</ol>
81
+
82
+<li><p>Run make to build the "fossil" or "fossil.exe" executable. The
83
+details depend on your platform and compiler.
84
+
85
+<ol type="a">
86
+<li><p><i>Unix</i> &rarr; the configure-generated Makefile should work on
87
+all unix and unix-like systems. Simply type "<b>make</b>".
88
+
89
+<li><p><i>Unix without running "configure"</i> &rarr; if you prefer to avoid running configure, you
90
+can also use: <b>make -f Makefile.classic</b>. You may want to make minor
91
+edits to Makefile.classic to configure the build for your system.
92
+
93
+<li><p><i>Msys/MinGW</i> &rarr; Use the
94
+mingw makefile: "<b>make -f win/Makefile.mingw</b>"
5995
60
-<li><p>Type "<b>make</b>"
96
+<li><p><i>VC++</i> &rarr; Use the msc makefile. First
97
+change to the "win/" subdirectory ("<b>cd win</b>") then run
98
+"<b>nmake /f Makefile.msc</b>".
99
+</ol>
61100
 </ol>
62101
63
-<p>To build on windows, use an alternative makefile suitable for your
64
-particular build environment. The alternative windows makefiles are
65
-all found in the win/ subdirectory of the source tree. So, for example,
66
-if you want build using the
67
-[http://www.mingw.org/ | mingw/msys compiler package] for windows, then
68
-run "<b>make -f win/Makefile.mingw</b>" instead of just "<b>make</b>"
69
-in step 8 above.</p>
102
+<p>Note that Fossil requires the "zlib" compression library. This library
103
+is available by default on most unix systems, but it will typically have to
104
+be installed separately on windows systems. For windows builds, you may
105
+need to edit the makefile to tell it exactly where zlib is located on your
106
+system.</p>
70107
71108
 <h2>3.0 Installing</h2>
72109
73110
 <ol>
74111
 <li value="9">
75112
76113
ADDED www/changes.wiki
--- www/build.wiki
+++ www/build.wiki
@@ -1,16 +1,29 @@
1 <title>Building and Installing Fossil</title>
2
3 <p>This page describes how to build and install Fossil. The
4 whole process is designed to be very easy.</p>
5
6 <h2>0.0 Using A Pre-compiled Binary</h2>
7
8 <p>You can skip steps 1.0 and 2.0 below by downloading
9 a <a href="http://www.fossil-scm.org/download.html">pre-compiled binary</a>
10 appropriate for your platform. If you use a pre-compiled binary
11 jump immediate to step 3.0.</p>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
13 <h2>1.0 Obtaining The Source Code</h2>
14
15 <p>Fossil is self-hosting, so you can obtain a ZIP archive containing
16 a snapshot of the latest version directly from fossil's own fossil
@@ -33,42 +46,66 @@
33
34 <li><p>Select a version of of fossil you want to download. Click on its
35 link. Note that you must successfully log in as "anonymous" in step 1
36 above in order to see the link to the detailed version information.</p></li>
37
38 <li><p>Finally, click on the
39 "Zip Archive" link. This link will build a ZIP archive of the
 
40 complete source code and download it to your browser.
41 </ol>
42
43 <h2>2.0 Compiling</h2>
44
45 <p>Follow these steps to compile on a unix platform
46 (Linux, *BSD, MacOS, etc):</p>
47
48 <ol>
49 <li value="6">
50 <p>Create a directory to hold the source code. Then unzip the
51 ZIP archive you downloaded into that directory. You should be
52 in the top-level folder of that directory</p></li>
53
54 <li><p><b>(Optional:)</b>
55 Edit the Makefile to set it up like you want. You probably do not
56 need to do anything. Do not be intimidated: There are less than 10
57 variables in the makefile that can be changed. The whole Makefile
58 is only a few dozen lines long and most of those lines are comments.</p>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
60 <li><p>Type "<b>make</b>"
 
 
 
61 </ol>
62
63 <p>To build on windows, use an alternative makefile suitable for your
64 particular build environment. The alternative windows makefiles are
65 all found in the win/ subdirectory of the source tree. So, for example,
66 if you want build using the
67 [http://www.mingw.org/ | mingw/msys compiler package] for windows, then
68 run "<b>make -f win/Makefile.mingw</b>" instead of just "<b>make</b>"
69 in step 8 above.</p>
70
71 <h2>3.0 Installing</h2>
72
73 <ol>
74 <li value="9">
75
76 DDED www/changes.wiki
--- www/build.wiki
+++ www/build.wiki
@@ -1,16 +1,29 @@
1 <title>Building and Installing Fossil</title>
2
 
 
 
3 <h2>0.0 Using A Pre-compiled Binary</h2>
4
5 <p>You can skip all of the following by downloading
6 a <a href="http://www.fossil-scm.org/download.html">pre-compiled binary</a>
7 appropriate for your platform and putting that self-contained binary
8 someplace on your $PATH.
9 To uninstall, simply delete the binary.
10 To upgrade from an older release, just overwrite the older binary with
11 the newer one.</p>
12
13 <h2>0.1 Executive Summary</h2>
14
15 <p>Building and installing is very simple. Three steps:</p>
16
17 <ol>
18 <li> Download and unpack a source tarball or ZIP.
19 <li> <b>./configure; make</b>
20 <li> Move or copy the resulting "fossil" executable to someplace
21 on your $PATH.
22 </ol>
23
24 <p><hr>
25
26 <h2>1.0 Obtaining The Source Code</h2>
27
28 <p>Fossil is self-hosting, so you can obtain a ZIP archive containing
29 a snapshot of the latest version directly from fossil's own fossil
@@ -33,42 +46,66 @@
46
47 <li><p>Select a version of of fossil you want to download. Click on its
48 link. Note that you must successfully log in as "anonymous" in step 1
49 above in order to see the link to the detailed version information.</p></li>
50
51 <li><p>Finally, click on one of the
52 "Zip Archive" or "Tarball" links, according to your preference.
53 These link will build a ZIP archive or a gzip-compressed tarball of the
54 complete source code and download it to your browser.
55 </ol>
56
57 <h2>2.0 Compiling</h2>
58
 
 
 
59 <ol>
60 <li value="6">
61 <p>Create a directory to hold the source code. Then unarchive the
62 ZIP or tarball you downloaded into that directory. You should be
63 in the top-level folder of that directory</p></li>
64
65 <li><i>(Optional, unix only)</i>
66 Run <b>./configure</b> to construct a makefile.
67
68 <ol type="a">
69 <li><p>
70 If you do not have the OpenSSL library installed on your system, then
71 add <b>--with-openssl=none</b> to omit the https functionality.
72
73 <li><p>
74 To build a statically linked binary (suitable for use inside a chroot
75 jail) add the <b>--static</b> option.
76
77 <li><p>
78 Other configuration options can be seen by running
79 <b>./configure --help</b>
80 </ol>
81
82 <li><p>Run make to build the "fossil" or "fossil.exe" executable. The
83 details depend on your platform and compiler.
84
85 <ol type="a">
86 <li><p><i>Unix</i> &rarr; the configure-generated Makefile should work on
87 all unix and unix-like systems. Simply type "<b>make</b>".
88
89 <li><p><i>Unix without running "configure"</i> &rarr; if you prefer to avoid running configure, you
90 can also use: <b>make -f Makefile.classic</b>. You may want to make minor
91 edits to Makefile.classic to configure the build for your system.
92
93 <li><p><i>Msys/MinGW</i> &rarr; Use the
94 mingw makefile: "<b>make -f win/Makefile.mingw</b>"
95
96 <li><p><i>VC++</i> &rarr; Use the msc makefile. First
97 change to the "win/" subdirectory ("<b>cd win</b>") then run
98 "<b>nmake /f Makefile.msc</b>".
99 </ol>
100 </ol>
101
102 <p>Note that Fossil requires the "zlib" compression library. This library
103 is available by default on most unix systems, but it will typically have to
104 be installed separately on windows systems. For windows builds, you may
105 need to edit the makefile to tell it exactly where zlib is located on your
106 system.</p>
 
 
107
108 <h2>3.0 Installing</h2>
109
110 <ol>
111 <li value="9">
112
113 DDED www/changes.wiki
M www/changes.wiki
+1
--- a/www/changes.wiki
+++ b/www/changes.wiki
@@ -0,0 +1 @@
1
+2-xxVarious minor bug fixepending
--- a/www/changes.wiki
+++ b/www/changes.wiki
@@ -0,0 +1 @@
 
--- a/www/changes.wiki
+++ b/www/changes.wiki
@@ -0,0 +1 @@
1 2-xxVarious minor bug fixepending
M www/checkin.wiki
+73 -16
--- www/checkin.wiki
+++ www/checkin.wiki
@@ -1,21 +1,78 @@
1
-<title>Checkin Checklist</title>
1
+<title>Check-in Checklist</title>
2
+
3
+<h2><u>Always</u> run the following checklist prior to <u>every</u>
4
+check-in or commit to the Fossil repository:</h2>
25
3
-Before every checkins:
6
+Before every check-in:
47
5
- 1. <b>fossil diff</b> &rarr; no stray changes
8
+ 1. <b>fossil diff</b> &rarr;
9
+ <ol type="a">
10
+ <li> No stray changes
11
+ <li> All changes comply with the license
12
+ <li> All inputs are scrubbed before use
13
+ <li> No injection attacks via %s formats
14
+ </ol>
615
716
 2. <b>fossil extra</b> &rarr; no unmanaged files need to be added.
817
9
- 3. The checkin will go onto the desired branch.
10
-
11
- 4. "Autosync" is enabled &rarr;
12
- <ol>
13
- <li> The checkin will not cause a unintentional fork.
14
- <li> The local system clock is set correctly.
15
- </ol>
16
-
17
-Before every checkin to <b>trunk</b>:
18
-
19
- 5. No compiler warnings on the development machine.
20
-
21
- 6. Changes will not cause problems on a future <b>bisect</b>.
18
+ 3. The check-in will go onto the desired branch.
19
+ &rarr; Do <u>not</u> check into trunk without prior approval from
20
+ the lead programmer (drh)!
21
+
22
+ 4. auto-sync is on, or the system clock is verified
23
+
24
+ 5. If sources files have been added or removed, ensure all makefiles
25
+ and configure scripts have been updated accordingly.
26
+
27
+Before every check-in to <b>trunk</b>:
28
+
29
+ 6. No compiler warnings on the development machine.
30
+
31
+ 7. The fossil executable that results from a build actually works.
32
+
33
+
34
+<hr>
35
+<h2>Commentary</h2>
36
+
37
+Item 1 is the most important step. Consider using <b>gdiff</b>
38
+instead of <b>diff</b> if you have a graphical differ configured. Or,
39
+pipe the output of "<b>fossil diff</b>" into "<b>open -f</b>" (on a mac) to
40
+get the diff output in a separate text window for easier viewing.
41
+Look carefully at every changed line in item 1.
42
+Make sure that you are not about to commit unrelated changes.
43
+If there are two or more unrelated changes present, consider
44
+breaking up the commit into two or more separate commits.
45
+Always make 100% sure that all changes are compatible with the
46
+BSD license, that you have the authority to commit the code in accordance
47
+with the [/doc/trunk/www/copyright-release.html | CLA] that you have
48
+signed and have on file, and that
49
+no NDAs, copyrights, patents, or trademarks are infringed by the check-in.
50
+Also check very carefully to make sure that
51
+you are not introducing security vulnerabilities. Pay particular attention
52
+to the possibility of SQL or HTML injection attacks.
53
+
54
+Item 2 verifies that you have not added source files but failed to
55
+do the necessary "<b>fossil add</b>" to manage them. If you commit
56
+without bringing the new file under source control, the check-in will
57
+be broken. That, in turn, can cause complications far in the future
58
+when we are bisecting for a bug.
59
+
60
+For item 3, Run "<b>fossil status</b>" or the equivalent to
61
+make sure your changes are going into the branch you think they are.
62
+Also double-check the branch name when entering change comments.
63
+Never check into trunk unless you are expressly authorized to do so.
64
+
65
+For Item 4, if you are on-network, make sure auto-sync is enabled. This
66
+will minimize the risk of an unintended fork. It will also give you a
67
+warning if you system clock is set incorrectly. If you are off-network,
68
+make sure that your system clock is correct or at least close to correct
69
+so that your check-in does not appear out-of-sequence on timelines.
70
+On-network commits are preferred, especially for trunk commits.
71
+
72
+Items 6 and 7 help to ensure that check-ins on the trunk always work.
73
+Knowing that the trunk always works makes bisecting much easier. Items
74
+6 and 7 are recommended for all check-ins, even those that are on a branch.
75
+But they are especially important for trunk check-ins. We desire that
76
+all trunk check-ins work at all times. Any experimental, unstable, or
77
+questionable changes should go on a branch and be merged into trunk after
78
+further testing.
2279
2380
ADDED www/foss-cklist.wiki
--- www/checkin.wiki
+++ www/checkin.wiki
@@ -1,21 +1,78 @@
1 <title>Checkin Checklist</title>
 
 
 
2
3 Before every checkins:
4
5 1. <b>fossil diff</b> &rarr; no stray changes
 
 
 
 
 
 
6
7 2. <b>fossil extra</b> &rarr; no unmanaged files need to be added.
8
9 3. The checkin will go onto the desired branch.
10
11 4. "Autosync" is enabled &rarr;
12 <ol>
13 <li> The checkin will not cause a unintentional fork.
14 <li> The local system clock is set correctly.
15 </ol>
16
17 Before every checkin to <b>trunk</b>:
18
19 5. No compiler warnings on the development machine.
20
21 6. Changes will not cause problems on a future <b>bisect</b>.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
23 DDED www/foss-cklist.wiki
--- www/checkin.wiki
+++ www/checkin.wiki
@@ -1,21 +1,78 @@
1 <title>Check-in Checklist</title>
2
3 <h2><u>Always</u> run the following checklist prior to <u>every</u>
4 check-in or commit to the Fossil repository:</h2>
5
6 Before every check-in:
7
8 1. <b>fossil diff</b> &rarr;
9 <ol type="a">
10 <li> No stray changes
11 <li> All changes comply with the license
12 <li> All inputs are scrubbed before use
13 <li> No injection attacks via %s formats
14 </ol>
15
16 2. <b>fossil extra</b> &rarr; no unmanaged files need to be added.
17
18 3. The check-in will go onto the desired branch.
19 &rarr; Do <u>not</u> check into trunk without prior approval from
20 the lead programmer (drh)!
21
22 4. auto-sync is on, or the system clock is verified
23
24 5. If sources files have been added or removed, ensure all makefiles
25 and configure scripts have been updated accordingly.
26
27 Before every check-in to <b>trunk</b>:
28
29 6. No compiler warnings on the development machine.
30
31 7. The fossil executable that results from a build actually works.
32
33
34 <hr>
35 <h2>Commentary</h2>
36
37 Item 1 is the most important step. Consider using <b>gdiff</b>
38 instead of <b>diff</b> if you have a graphical differ configured. Or,
39 pipe the output of "<b>fossil diff</b>" into "<b>open -f</b>" (on a mac) to
40 get the diff output in a separate text window for easier viewing.
41 Look carefully at every changed line in item 1.
42 Make sure that you are not about to commit unrelated changes.
43 If there are two or more unrelated changes present, consider
44 breaking up the commit into two or more separate commits.
45 Always make 100% sure that all changes are compatible with the
46 BSD license, that you have the authority to commit the code in accordance
47 with the [/doc/trunk/www/copyright-release.html | CLA] that you have
48 signed and have on file, and that
49 no NDAs, copyrights, patents, or trademarks are infringed by the check-in.
50 Also check very carefully to make sure that
51 you are not introducing security vulnerabilities. Pay particular attention
52 to the possibility of SQL or HTML injection attacks.
53
54 Item 2 verifies that you have not added source files but failed to
55 do the necessary "<b>fossil add</b>" to manage them. If you commit
56 without bringing the new file under source control, the check-in will
57 be broken. That, in turn, can cause complications far in the future
58 when we are bisecting for a bug.
59
60 For item 3, Run "<b>fossil status</b>" or the equivalent to
61 make sure your changes are going into the branch you think they are.
62 Also double-check the branch name when entering change comments.
63 Never check into trunk unless you are expressly authorized to do so.
64
65 For Item 4, if you are on-network, make sure auto-sync is enabled. This
66 will minimize the risk of an unintended fork. It will also give you a
67 warning if you system clock is set incorrectly. If you are off-network,
68 make sure that your system clock is correct or at least close to correct
69 so that your check-in does not appear out-of-sequence on timelines.
70 On-network commits are preferred, especially for trunk commits.
71
72 Items 6 and 7 help to ensure that check-ins on the trunk always work.
73 Knowing that the trunk always works makes bisecting much easier. Items
74 6 and 7 are recommended for all check-ins, even those that are on a branch.
75 But they are especially important for trunk check-ins. We desire that
76 all trunk check-ins work at all times. Any experimental, unstable, or
77 questionable changes should go on a branch and be merged into trunk after
78 further testing.
79
80 DDED www/foss-cklist.wiki
M www/foss-cklist.wiki
+60
--- a/www/foss-cklist.wiki
+++ b/www/foss-cklist.wiki
@@ -0,0 +1,60 @@
1
+<title>Checklist For Successful Open-Source Projects</title>
2
+<nowiki>
3
+
4
+<p>This checklist is loosely derived from Tom "Spot" Callaway's Fail Score
5
+blog post <a href="http://spot.livejournal.com/308370.html">
6
+http://spot.livejournal.com/308370.html</a> (see also
7
+<a href="http://www.theopensourceway.org/book/The_Open_Source_Way-How_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL.html">[1]</a> and
8
+<a href="https://wwwHow_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL.html">[1]</a>).
9
+Tom's original post assigned point scores to the various elements and
10
+by adding together the individual points, the reader is suposed to be able
11
+to judge the likelihood that the project will fail.
12
+The point scores, and the items on the list, clearly reflect Tom's
13
+biases and are not necessarily those of the larger open-source community.
14
+Nevertheless, the policy of the Fossil shall be to strive for a perfect
15
+score.</p>
16
+
17
+<p>rive for a perfect
18
+score.
19
+
20
+This checklist is an inversion of Tom's original post in that it strives to
21
+say what the project should do in order to succeed rather than what it
22
+should not do to avoid failure. Th</p>
23
+
24
+<p>point values are omitted.
25
+
26
+See also:
27
+<ul>
28
+<li><a href="http://offog.org/articles/packaging/">
29
+http://offog.org/articles/packaging/</a>
30
+<li>
31
+<a href="http://www.gnu.org/prep/standards/standards.html#Managing-Releases">
32
+http://www.gnu.org/prep/standards/standards.html#Managing-Releases<p>The source code size is less than <p>The project uses a Version Control System (VCS).
33
+<ol type="a">
34
+<li>The VCS has a working web interface.
35
+<li>There is documentation on how to use the VCS.
36
+<li>The VCS is general-purpose, not something hacked together for this
37
+ <p>The project comes with documentation on how to build from source
38
+and that documentation is lucid, correct, and up-to-date.
39
+
40
+<li><p>The project is configurable using an autoconf-generated configure
41
+script, or the equivalent, and does not require:
42
+<ol type="a">
43
+<li>Manually editing flat files
44
+<li>Editing code header files
45
+</ol>
46
+
47
+<li><pes a Version les
48
+</ol>
49
+
50
+<li>The project should be buildable using commonly available and
51
+ standa<p Open-Source Projects</title>
52
+<nowiki>
53
+
54
+This checklist is loosely derive<t>The project is configurable using an autoconf-generated configure
55
+script, or the equivalent, and does not requir essful Open-Source Projects</title>
56
+<nowiki>
57
+
58
+<p>This checklist is loosely derived from Tom "Spot" Callaway's Fail Score
59
+blog post <a href="http://spot.livejournal.com/308370.html">
60
+http://sp
--- a/www/foss-cklist.wiki
+++ b/www/foss-cklist.wiki
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/www/foss-cklist.wiki
+++ b/www/foss-cklist.wiki
@@ -0,0 +1,60 @@
1 <title>Checklist For Successful Open-Source Projects</title>
2 <nowiki>
3
4 <p>This checklist is loosely derived from Tom "Spot" Callaway's Fail Score
5 blog post <a href="http://spot.livejournal.com/308370.html">
6 http://spot.livejournal.com/308370.html</a> (see also
7 <a href="http://www.theopensourceway.org/book/The_Open_Source_Way-How_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL.html">[1]</a> and
8 <a href="https://wwwHow_to_tell_if_a_FLOSS_project_is_doomed_to_FAIL.html">[1]</a>).
9 Tom's original post assigned point scores to the various elements and
10 by adding together the individual points, the reader is suposed to be able
11 to judge the likelihood that the project will fail.
12 The point scores, and the items on the list, clearly reflect Tom's
13 biases and are not necessarily those of the larger open-source community.
14 Nevertheless, the policy of the Fossil shall be to strive for a perfect
15 score.</p>
16
17 <p>rive for a perfect
18 score.
19
20 This checklist is an inversion of Tom's original post in that it strives to
21 say what the project should do in order to succeed rather than what it
22 should not do to avoid failure. Th</p>
23
24 <p>point values are omitted.
25
26 See also:
27 <ul>
28 <li><a href="http://offog.org/articles/packaging/">
29 http://offog.org/articles/packaging/</a>
30 <li>
31 <a href="http://www.gnu.org/prep/standards/standards.html#Managing-Releases">
32 http://www.gnu.org/prep/standards/standards.html#Managing-Releases<p>The source code size is less than <p>The project uses a Version Control System (VCS).
33 <ol type="a">
34 <li>The VCS has a working web interface.
35 <li>There is documentation on how to use the VCS.
36 <li>The VCS is general-purpose, not something hacked together for this
37 <p>The project comes with documentation on how to build from source
38 and that documentation is lucid, correct, and up-to-date.
39
40 <li><p>The project is configurable using an autoconf-generated configure
41 script, or the equivalent, and does not require:
42 <ol type="a">
43 <li>Manually editing flat files
44 <li>Editing code header files
45 </ol>
46
47 <li><pes a Version les
48 </ol>
49
50 <li>The project should be buildable using commonly available and
51 standa<p Open-Source Projects</title>
52 <nowiki>
53
54 This checklist is loosely derive<t>The project is configurable using an autoconf-generated configure
55 script, or the equivalent, and does not requir essful Open-Source Projects</title>
56 <nowiki>
57
58 <p>This checklist is loosely derived from Tom "Spot" Callaway's Fail Score
59 blog post <a href="http://spot.livejournal.com/308370.html">
60 http://sp
M www/index.wiki
+4
--- www/index.wiki
+++ www/index.wiki
@@ -20,10 +20,12 @@
2020
 <li> [./quickstart.wiki | Quick Start]
2121
 <li> [./build.wiki | Install]
2222
 <li> [../COPYRIGHT-BSD2.txt | License]
2323
 <li> [/timeline | Recent changes]
2424
 <li> [./faq.wiki | FAQ]
25
+<li> [./contribute.wiki | Contributing]
26
+<li> [./changes.wiki | Change Log]
2527
 <li> [./permutedindex.wiki | Doc Index]
2628
 <li> Mailing list
2729
 <ul>
2830
 <li> [http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users | sign-up]
2931
 <li> [http://www.mail-archive.com/[email protected] | archives]
@@ -125,10 +127,12 @@
125127
 * The [./selfcheck.wiki | automatic self-check] mechanism
126128
 helps insure project integrity.
127129
 * Fossil contains a [./wikitheory.wiki | built-in wiki].
128130
 * An [./event.wiki | Event] is a special kind of wiki page associated
129131
 with a point in time rather than a name.
132
+ * [./settings.wiki | Settings] control the behaviour of fossil.
133
+ * [./ssl.wiki | Use SSL] to encrypt communication with the server.
130134
 * There is a
131135
 [http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users | mailing list] (with publicly readable
132136
 [http://www.mail-archive.com/[email protected] | archives]
133137
 available for discussing fossil issues.
134138
 * [./stats.wiki | Performance statistics] taken from real-world projects
135139
--- www/index.wiki
+++ www/index.wiki
@@ -20,10 +20,12 @@
20 <li> [./quickstart.wiki | Quick Start]
21 <li> [./build.wiki | Install]
22 <li> [../COPYRIGHT-BSD2.txt | License]
23 <li> [/timeline | Recent changes]
24 <li> [./faq.wiki | FAQ]
 
 
25 <li> [./permutedindex.wiki | Doc Index]
26 <li> Mailing list
27 <ul>
28 <li> [http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users | sign-up]
29 <li> [http://www.mail-archive.com/[email protected] | archives]
@@ -125,10 +127,12 @@
125 * The [./selfcheck.wiki | automatic self-check] mechanism
126 helps insure project integrity.
127 * Fossil contains a [./wikitheory.wiki | built-in wiki].
128 * An [./event.wiki | Event] is a special kind of wiki page associated
129 with a point in time rather than a name.
 
 
130 * There is a
131 [http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users | mailing list] (with publicly readable
132 [http://www.mail-archive.com/[email protected] | archives]
133 available for discussing fossil issues.
134 * [./stats.wiki | Performance statistics] taken from real-world projects
135
--- www/index.wiki
+++ www/index.wiki
@@ -20,10 +20,12 @@
20 <li> [./quickstart.wiki | Quick Start]
21 <li> [./build.wiki | Install]
22 <li> [../COPYRIGHT-BSD2.txt | License]
23 <li> [/timeline | Recent changes]
24 <li> [./faq.wiki | FAQ]
25 <li> [./contribute.wiki | Contributing]
26 <li> [./changes.wiki | Change Log]
27 <li> [./permutedindex.wiki | Doc Index]
28 <li> Mailing list
29 <ul>
30 <li> [http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users | sign-up]
31 <li> [http://www.mail-archive.com/[email protected] | archives]
@@ -125,10 +127,12 @@
127 * The [./selfcheck.wiki | automatic self-check] mechanism
128 helps insure project integrity.
129 * Fossil contains a [./wikitheory.wiki | built-in wiki].
130 * An [./event.wiki | Event] is a special kind of wiki page associated
131 with a point in time rather than a name.
132 * [./settings.wiki | Settings] control the behaviour of fossil.
133 * [./ssl.wiki | Use SSL] to encrypt communication with the server.
134 * There is a
135 [http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users | mailing list] (with publicly readable
136 [http://www.mail-archive.com/[email protected] | archives]
137 available for discussing fossil issues.
138 * [./stats.wiki | Performance statistics] taken from real-world projects
139
M www/makefile.wiki
+21 -15
--- www/makefile.wiki
+++ www/makefile.wiki
@@ -52,29 +52,34 @@
5252
 7. th.c
5353
 8. th.h
5454
5555
 These two files are imports like the SQLite source files,
5656
 and so are not preprocessed.
57
+
58
+The VERSION.h header file is generated from other information sources
59
+using a small program called:
60
+
61
+ 9. mkversion.c
5762
5863
 The src/ subdirectory also contains documentation about the
5964
 makeheaders preprocessor program:
6065
61
- 9. [../src/makeheaders.html | makeheaders.html]
66
+ 10. [../src/makeheaders.html | makeheaders.html]
6267
6368
 Click on the link to read this documentation. In addition there is
6469
 a [http://www.tcl.tk/ | Tcl] script used to build the various makefiles:
6570
66
- 10. makemake.tcl
71
+ 11. makemake.tcl
6772
6873
 Running this Tcl script will automatically regenerate all makefiles.
6974
 In order to add a new source file to the Fossil implementation, simply
7075
 edit makemake.tcl to add the new filename, then rerun the script, and
7176
 all of the makefiles for all targets will be rebuild.
7277
7378
 Finally, there is one of the makefiles generated by makemake.tcl:
7479
75
- 11. main.mk
80
+ 12. main.mk
7681
7782
 The main.mk makefile is invoked from the Makefile in the top-level
7883
 directory. The main.mk is generated by makemake.tcl and should not
7984
 be hand edited. Other makefiles generated by makemake.tcl are in
8085
 other subdirectories (currently all in the win/ subdirectory).
@@ -89,31 +94,32 @@
8994
9095
 <h1>3.0 Automatically generated files</h1>
9196
9297
 The "VERSION.h" header file contains some C preprocessor macros that
9398
 identify the version of Fossil that is to be build. The VERSION.h file is
94
-generated automatically from information extracted from the "manifest"
95
-and "manifest.uuid" source files in the root directory of the source tree.
99
+generated automatically from information extracted from the "manifest",
100
+"manifest.uuid", and "VERSION" source files in the root directory of the
101
+source tree.
96102
 (The "manifest" and "manifest.uuid" files are automatically generated and
97103
 updated by Fossil itself. See the [/help/setting | fossil set manifest]
98104
 command for additional information.)
99105
100
-Under unix, there is an AWK script that converts manifest and manifest.uuid
101
-into VERSION.h. For windows, there is a C program: win/version.c.
102
-(Unix builds could also used the version.c convert, if desired, but AWK
103
-seems easier there.) To run the VERSION.h generator, first compile
104
-the win/version.c source file into a command-line program (named "version.exe")
106
+The VERSION.h header file is generated by
107
+a C program: src/mkversion.c.
108
+To run the VERSION.h generator, first compile the src/mkversion.c
109
+ source file into a command-line program (named "mkversion.exe")
105110
 than run:
106111
107112
 <blockquote><pre>
108
-version.exe manifest.uuid manifest VERSION.h
113
+mkversion.exe manifest.uuid manifest VERSION &gt;VERSION.h
109114
 </pre></blockquote>
110115
111116
 The pathnames in the above command might need to be adjusted to get the
112
-directories right. The point is that the manifest.uuid and manifest files
113
-in the root of the source tree are the first two arguments and the name of
114
-the generated VERSION.h file is the third and final argument.
117
+directories right. The point is that the manifest.uuid, manifest, and
118
+VERSION files
119
+in the root of the source tree are the three arguments and
120
+the generated VERSION.h file appears on standard output.
115121
116122
 <h1>4.0 Preprocessing</h1>
117123
118124
 There are three preprocessors for the Fossil sources. The mkindex
119125
 and translate preprocessors can be run in any order. The makeheaders
@@ -120,11 +126,11 @@
120126
 preprocessor must be run after translate.
121127
122128
 <h2>4.1 The mkindex preprocessor</h2>
123129
124130
 The mkindex program scans the "src.c" source files looking for special
125
-comments that identify routines that implement of various Fossil commands,
131
+comments that identify routines that implement various Fossil commands,
126132
 web interface methods, and help text comments. The mkindex program
127133
 generates some C code that Fossil uses in order to dispatch commands and
128134
 HTTP requests and to show on-line help. Compile the mkindex program
129135
 from the mkindex.c source file. Then run:
130136
131137
--- www/makefile.wiki
+++ www/makefile.wiki
@@ -52,29 +52,34 @@
52 7. th.c
53 8. th.h
54
55 These two files are imports like the SQLite source files,
56 and so are not preprocessed.
 
 
 
 
 
57
58 The src/ subdirectory also contains documentation about the
59 makeheaders preprocessor program:
60
61 9. [../src/makeheaders.html | makeheaders.html]
62
63 Click on the link to read this documentation. In addition there is
64 a [http://www.tcl.tk/ | Tcl] script used to build the various makefiles:
65
66 10. makemake.tcl
67
68 Running this Tcl script will automatically regenerate all makefiles.
69 In order to add a new source file to the Fossil implementation, simply
70 edit makemake.tcl to add the new filename, then rerun the script, and
71 all of the makefiles for all targets will be rebuild.
72
73 Finally, there is one of the makefiles generated by makemake.tcl:
74
75 11. main.mk
76
77 The main.mk makefile is invoked from the Makefile in the top-level
78 directory. The main.mk is generated by makemake.tcl and should not
79 be hand edited. Other makefiles generated by makemake.tcl are in
80 other subdirectories (currently all in the win/ subdirectory).
@@ -89,31 +94,32 @@
89
90 <h1>3.0 Automatically generated files</h1>
91
92 The "VERSION.h" header file contains some C preprocessor macros that
93 identify the version of Fossil that is to be build. The VERSION.h file is
94 generated automatically from information extracted from the "manifest"
95 and "manifest.uuid" source files in the root directory of the source tree.
 
96 (The "manifest" and "manifest.uuid" files are automatically generated and
97 updated by Fossil itself. See the [/help/setting | fossil set manifest]
98 command for additional information.)
99
100 Under unix, there is an AWK script that converts manifest and manifest.uuid
101 into VERSION.h. For windows, there is a C program: win/version.c.
102 (Unix builds could also used the version.c convert, if desired, but AWK
103 seems easier there.) To run the VERSION.h generator, first compile
104 the win/version.c source file into a command-line program (named "version.exe")
105 than run:
106
107 <blockquote><pre>
108 version.exe manifest.uuid manifest VERSION.h
109 </pre></blockquote>
110
111 The pathnames in the above command might need to be adjusted to get the
112 directories right. The point is that the manifest.uuid and manifest files
113 in the root of the source tree are the first two arguments and the name of
114 the generated VERSION.h file is the third and final argument.
 
115
116 <h1>4.0 Preprocessing</h1>
117
118 There are three preprocessors for the Fossil sources. The mkindex
119 and translate preprocessors can be run in any order. The makeheaders
@@ -120,11 +126,11 @@
120 preprocessor must be run after translate.
121
122 <h2>4.1 The mkindex preprocessor</h2>
123
124 The mkindex program scans the "src.c" source files looking for special
125 comments that identify routines that implement of various Fossil commands,
126 web interface methods, and help text comments. The mkindex program
127 generates some C code that Fossil uses in order to dispatch commands and
128 HTTP requests and to show on-line help. Compile the mkindex program
129 from the mkindex.c source file. Then run:
130
131
--- www/makefile.wiki
+++ www/makefile.wiki
@@ -52,29 +52,34 @@
52 7. th.c
53 8. th.h
54
55 These two files are imports like the SQLite source files,
56 and so are not preprocessed.
57
58 The VERSION.h header file is generated from other information sources
59 using a small program called:
60
61 9. mkversion.c
62
63 The src/ subdirectory also contains documentation about the
64 makeheaders preprocessor program:
65
66 10. [../src/makeheaders.html | makeheaders.html]
67
68 Click on the link to read this documentation. In addition there is
69 a [http://www.tcl.tk/ | Tcl] script used to build the various makefiles:
70
71 11. makemake.tcl
72
73 Running this Tcl script will automatically regenerate all makefiles.
74 In order to add a new source file to the Fossil implementation, simply
75 edit makemake.tcl to add the new filename, then rerun the script, and
76 all of the makefiles for all targets will be rebuild.
77
78 Finally, there is one of the makefiles generated by makemake.tcl:
79
80 12. main.mk
81
82 The main.mk makefile is invoked from the Makefile in the top-level
83 directory. The main.mk is generated by makemake.tcl and should not
84 be hand edited. Other makefiles generated by makemake.tcl are in
85 other subdirectories (currently all in the win/ subdirectory).
@@ -89,31 +94,32 @@
94
95 <h1>3.0 Automatically generated files</h1>
96
97 The "VERSION.h" header file contains some C preprocessor macros that
98 identify the version of Fossil that is to be build. The VERSION.h file is
99 generated automatically from information extracted from the "manifest",
100 "manifest.uuid", and "VERSION" source files in the root directory of the
101 source tree.
102 (The "manifest" and "manifest.uuid" files are automatically generated and
103 updated by Fossil itself. See the [/help/setting | fossil set manifest]
104 command for additional information.)
105
106 The VERSION.h header file is generated by
107 a C program: src/mkversion.c.
108 To run the VERSION.h generator, first compile the src/mkversion.c
109 source file into a command-line program (named "mkversion.exe")
 
110 than run:
111
112 <blockquote><pre>
113 mkversion.exe manifest.uuid manifest VERSION &gt;VERSION.h
114 </pre></blockquote>
115
116 The pathnames in the above command might need to be adjusted to get the
117 directories right. The point is that the manifest.uuid, manifest, and
118 VERSION files
119 in the root of the source tree are the three arguments and
120 the generated VERSION.h file appears on standard output.
121
122 <h1>4.0 Preprocessing</h1>
123
124 There are three preprocessors for the Fossil sources. The mkindex
125 and translate preprocessors can be run in any order. The makeheaders
@@ -120,11 +126,11 @@
126 preprocessor must be run after translate.
127
128 <h2>4.1 The mkindex preprocessor</h2>
129
130 The mkindex program scans the "src.c" source files looking for special
131 comments that identify routines that implement various Fossil commands,
132 web interface methods, and help text comments. The mkindex program
133 generates some C code that Fossil uses in order to dispatch commands and
134 HTTP requests and to show on-line help. Compile the mkindex program
135 from the mkindex.c source file. Then run:
136
137
M www/mkindex.tcl
+11 -6
--- www/mkindex.tcl
+++ www/mkindex.tcl
@@ -9,10 +9,11 @@
99
 set doclist {
1010
 bugtheory.wiki {Bug Tracking In Fossil}
1111
 branching.wiki {Branching, Forking, Merging, and Tagging}
1212
 build.wiki {Building and Installing Fossil}
1313
 checkin_names.wiki {Checkin And Version Names}
14
+ changes.wiki {Fossil Changelog}
1415
 copyright-release.html {Contributor License Agreement}
1516
 concepts.wiki {Fossil Core Concepts}
1617
 contribute.wiki {Contributing Code or Documentation To The Fossil Project}
1718
 custom_ticket.wiki {Customizing The Ticket System}
1819
 delta_encoder_algorithm.wiki {Fossil Delta Encoding Algorithm}
@@ -19,10 +20,11 @@
1920
 delta_format.wiki {Fossil Delta Format}
2021
 embeddeddoc.wiki {Embedded Project Documentation}
2122
 event.wiki {Events}
2223
 faq.wiki {Frequently Asked Questions}
2324
 fileformat.wiki {Fossil File Format}
25
+ foss-cklist.wiki {Checklist For Successful Open-Source Projects}
2426
 fossil-v-git.wiki {Fossil Versus Git}
2527
 index.wiki {Home Page}
2628
 inout.wiki {Import And Export To And From Git}
2729
 makefile.wiki {The Fossil Build Process}
2830
 password.wiki {Password Management And Authentication}
@@ -34,13 +36,15 @@
3436
 {Quotes: What People Are Saying About Fossil, Git, and DVCSes in General}
3537
 ../test/release-checklist.wiki {Pre-Release Testing Checklist}
3638
 selfcheck.wiki {Fossil Repository Integrity Self Checks}
3739
 selfhost.wiki {Fossil Self Hosting Repositories}
3840
 server.wiki {How To Configure A Fossil Server}
41
+ settings.wiki {Fossil Settings}
3942
 shunning.wiki {Shunning: Deleting Content From Fossil}
4043
 stats.wiki {Performance Statistics}
4144
 style.wiki {Source Code Style Guidelines}
45
+ ssl.wiki {Using SSL with Fossil}
4246
 sync.wiki {The Fossil Sync Protocol}
4347
 tech_overview.wiki {A Technical Overview Of The Design And Implementation
4448
 Of Fossil}
4549
 tech_overview.wiki {SQLite Databases Used By Fossil}
4650
 theory1.wiki {Thoughts On The Design Of The Fossil DVCS}
@@ -47,11 +51,11 @@
4751
 webui.wiki {The Fossil Web Interface}
4852
 wikitheory.wiki {Wiki In Fossil}
4953
 }
5054
5155
 set permindex {}
52
-set stopwords {fossil and a in of on the to are about used by}
56
+set stopwords {fossil and a in of on the to are about used by for or}
5357
 foreach {file title} $doclist {
5458
 set n [llength $title]
5559
 lappend permindex [list $title $file]
5660
 for {set i 0} {$i<$n-1} {incr i} {
5761
 set prefix [lrange $title 0 $i]
@@ -61,13 +65,14 @@
6165
 lappend permindex [list "$suffix &#151; $prefix" $file]
6266
 }
6367
 }
6468
 }
6569
 set permindex [lsort -dict $permindex]
66
-puts "<title>Permuted Index Of Fossil Documentation</title>"
67
-puts "<nowiki>"
68
-puts "<ul>"
70
+set out [open permutedindex.wiki w]
71
+puts $out "<title>Permuted Index Of Fossil Documentation</title>"
72
+puts $out "<nowiki>"
73
+puts $out "<ul>"
6974
 foreach entry $permindex {
7075
 foreach {title file} $entry break
71
- puts "<li><a href=\"$file\">$title</a></li>"
76
+ puts $out "<li><a href=\"$file\">$title</a></li>"
7277
 }
73
-puts "</ul>"
78
+puts $out "</ul>"
7479
--- www/mkindex.tcl
+++ www/mkindex.tcl
@@ -9,10 +9,11 @@
9 set doclist {
10 bugtheory.wiki {Bug Tracking In Fossil}
11 branching.wiki {Branching, Forking, Merging, and Tagging}
12 build.wiki {Building and Installing Fossil}
13 checkin_names.wiki {Checkin And Version Names}
 
14 copyright-release.html {Contributor License Agreement}
15 concepts.wiki {Fossil Core Concepts}
16 contribute.wiki {Contributing Code or Documentation To The Fossil Project}
17 custom_ticket.wiki {Customizing The Ticket System}
18 delta_encoder_algorithm.wiki {Fossil Delta Encoding Algorithm}
@@ -19,10 +20,11 @@
19 delta_format.wiki {Fossil Delta Format}
20 embeddeddoc.wiki {Embedded Project Documentation}
21 event.wiki {Events}
22 faq.wiki {Frequently Asked Questions}
23 fileformat.wiki {Fossil File Format}
 
24 fossil-v-git.wiki {Fossil Versus Git}
25 index.wiki {Home Page}
26 inout.wiki {Import And Export To And From Git}
27 makefile.wiki {The Fossil Build Process}
28 password.wiki {Password Management And Authentication}
@@ -34,13 +36,15 @@
34 {Quotes: What People Are Saying About Fossil, Git, and DVCSes in General}
35 ../test/release-checklist.wiki {Pre-Release Testing Checklist}
36 selfcheck.wiki {Fossil Repository Integrity Self Checks}
37 selfhost.wiki {Fossil Self Hosting Repositories}
38 server.wiki {How To Configure A Fossil Server}
 
39 shunning.wiki {Shunning: Deleting Content From Fossil}
40 stats.wiki {Performance Statistics}
41 style.wiki {Source Code Style Guidelines}
 
42 sync.wiki {The Fossil Sync Protocol}
43 tech_overview.wiki {A Technical Overview Of The Design And Implementation
44 Of Fossil}
45 tech_overview.wiki {SQLite Databases Used By Fossil}
46 theory1.wiki {Thoughts On The Design Of The Fossil DVCS}
@@ -47,11 +51,11 @@
47 webui.wiki {The Fossil Web Interface}
48 wikitheory.wiki {Wiki In Fossil}
49 }
50
51 set permindex {}
52 set stopwords {fossil and a in of on the to are about used by}
53 foreach {file title} $doclist {
54 set n [llength $title]
55 lappend permindex [list $title $file]
56 for {set i 0} {$i<$n-1} {incr i} {
57 set prefix [lrange $title 0 $i]
@@ -61,13 +65,14 @@
61 lappend permindex [list "$suffix &#151; $prefix" $file]
62 }
63 }
64 }
65 set permindex [lsort -dict $permindex]
66 puts "<title>Permuted Index Of Fossil Documentation</title>"
67 puts "<nowiki>"
68 puts "<ul>"
 
69 foreach entry $permindex {
70 foreach {title file} $entry break
71 puts "<li><a href=\"$file\">$title</a></li>"
72 }
73 puts "</ul>"
74
--- www/mkindex.tcl
+++ www/mkindex.tcl
@@ -9,10 +9,11 @@
9 set doclist {
10 bugtheory.wiki {Bug Tracking In Fossil}
11 branching.wiki {Branching, Forking, Merging, and Tagging}
12 build.wiki {Building and Installing Fossil}
13 checkin_names.wiki {Checkin And Version Names}
14 changes.wiki {Fossil Changelog}
15 copyright-release.html {Contributor License Agreement}
16 concepts.wiki {Fossil Core Concepts}
17 contribute.wiki {Contributing Code or Documentation To The Fossil Project}
18 custom_ticket.wiki {Customizing The Ticket System}
19 delta_encoder_algorithm.wiki {Fossil Delta Encoding Algorithm}
@@ -19,10 +20,11 @@
20 delta_format.wiki {Fossil Delta Format}
21 embeddeddoc.wiki {Embedded Project Documentation}
22 event.wiki {Events}
23 faq.wiki {Frequently Asked Questions}
24 fileformat.wiki {Fossil File Format}
25 foss-cklist.wiki {Checklist For Successful Open-Source Projects}
26 fossil-v-git.wiki {Fossil Versus Git}
27 index.wiki {Home Page}
28 inout.wiki {Import And Export To And From Git}
29 makefile.wiki {The Fossil Build Process}
30 password.wiki {Password Management And Authentication}
@@ -34,13 +36,15 @@
36 {Quotes: What People Are Saying About Fossil, Git, and DVCSes in General}
37 ../test/release-checklist.wiki {Pre-Release Testing Checklist}
38 selfcheck.wiki {Fossil Repository Integrity Self Checks}
39 selfhost.wiki {Fossil Self Hosting Repositories}
40 server.wiki {How To Configure A Fossil Server}
41 settings.wiki {Fossil Settings}
42 shunning.wiki {Shunning: Deleting Content From Fossil}
43 stats.wiki {Performance Statistics}
44 style.wiki {Source Code Style Guidelines}
45 ssl.wiki {Using SSL with Fossil}
46 sync.wiki {The Fossil Sync Protocol}
47 tech_overview.wiki {A Technical Overview Of The Design And Implementation
48 Of Fossil}
49 tech_overview.wiki {SQLite Databases Used By Fossil}
50 theory1.wiki {Thoughts On The Design Of The Fossil DVCS}
@@ -47,11 +51,11 @@
51 webui.wiki {The Fossil Web Interface}
52 wikitheory.wiki {Wiki In Fossil}
53 }
54
55 set permindex {}
56 set stopwords {fossil and a in of on the to are about used by for or}
57 foreach {file title} $doclist {
58 set n [llength $title]
59 lappend permindex [list $title $file]
60 for {set i 0} {$i<$n-1} {incr i} {
61 set prefix [lrange $title 0 $i]
@@ -61,13 +65,14 @@
65 lappend permindex [list "$suffix &#151; $prefix" $file]
66 }
67 }
68 }
69 set permindex [lsort -dict $permindex]
70 set out [open permutedindex.wiki w]
71 puts $out "<title>Permuted Index Of Fossil Documentation</title>"
72 puts $out "<nowiki>"
73 puts $out "<ul>"
74 foreach entry $permindex {
75 foreach {title file} $entry break
76 puts $out "<li><a href=\"$file\">$title</a></li>"
77 }
78 puts $out "</ul>"
79
M www/permutedindex.wiki
+11 -1
--- www/permutedindex.wiki
+++ www/permutedindex.wiki
@@ -11,12 +11,14 @@
1111
 <li><a href="private.wiki">Branches &#151; Creating, Syncing, and Deleting Private</a></li>
1212
 <li><a href="branching.wiki">Branching, Forking, Merging, and Tagging</a></li>
1313
 <li><a href="bugtheory.wiki">Bug Tracking In Fossil</a></li>
1414
 <li><a href="makefile.wiki">Build Process &#151; The Fossil</a></li>
1515
 <li><a href="build.wiki">Building and Installing Fossil</a></li>
16
+<li><a href="changes.wiki">Changelog &#151; Fossil</a></li>
1617
 <li><a href="checkin_names.wiki">Checkin And Version Names</a></li>
1718
 <li><a href="../test/release-checklist.wiki">Checklist &#151; Pre-Release Testing</a></li>
19
+<li><a href="foss-cklist.wiki">Checklist For Successful Open-Source Projects</a></li>
1820
 <li><a href="selfcheck.wiki">Checks &#151; Fossil Repository Integrity Self</a></li>
1921
 <li><a href="contribute.wiki">Code or Documentation To The Fossil Project &#151; Contributing</a></li>
2022
 <li><a href="style.wiki">Code Style Guidelines &#151; Source</a></li>
2123
 <li><a href="concepts.wiki">Concepts &#151; Fossil Core</a></li>
2224
 <li><a href="server.wiki">Configure A Fossil Server &#151; How To</a></li>
@@ -43,17 +45,19 @@
4345
 <li><a href="inout.wiki">Export To And From Git &#151; Import And</a></li>
4446
 <li><a href="fileformat.wiki">File Format &#151; Fossil</a></li>
4547
 <li><a href="branching.wiki">Forking, Merging, and Tagging &#151; Branching,</a></li>
4648
 <li><a href="delta_format.wiki">Format &#151; Fossil Delta</a></li>
4749
 <li><a href="fileformat.wiki">Format &#151; Fossil File</a></li>
50
+<li><a href="changes.wiki">Fossil Changelog</a></li>
4851
 <li><a href="concepts.wiki">Fossil Core Concepts</a></li>
4952
 <li><a href="delta_encoder_algorithm.wiki">Fossil Delta Encoding Algorithm</a></li>
5053
 <li><a href="delta_format.wiki">Fossil Delta Format</a></li>
5154
 <li><a href="fileformat.wiki">Fossil File Format</a></li>
5255
 <li><a href="quickstart.wiki">Fossil Quick Start Guide</a></li>
5356
 <li><a href="selfcheck.wiki">Fossil Repository Integrity Self Checks</a></li>
5457
 <li><a href="selfhost.wiki">Fossil Self Hosting Repositories</a></li>
58
+<li><a href="settings.wiki">Fossil Settings</a></li>
5559
 <li><a href="fossil-v-git.wiki">Fossil Versus Git</a></li>
5660
 <li><a href="quotes.wiki">Fossil, Git, and DVCSes in General &#151; Quotes: What People Are Saying About</a></li>
5761
 <li><a href="faq.wiki">Frequently Asked Questions</a></li>
5862
 <li><a href="shunning.wiki">From Fossil &#151; Shunning: Deleting Content</a></li>
5963
 <li><a href="inout.wiki">From Git &#151; Import And Export To And</a></li>
@@ -73,12 +77,12 @@
7377
 <li><a href="webui.wiki">Interface &#151; The Fossil Web</a></li>
7478
 <li><a href="copyright-release.html">License Agreement &#151; Contributor</a></li>
7579
 <li><a href="password.wiki">Management And Authentication &#151; Password</a></li>
7680
 <li><a href="branching.wiki">Merging, and Tagging &#151; Branching, Forking,</a></li>
7781
 <li><a href="checkin_names.wiki">Names &#151; Checkin And Version</a></li>
82
+<li><a href="foss-cklist.wiki">Open-Source Projects &#151; Checklist For Successful</a></li>
7883
 <li><a href="pop.wiki">Operations &#151; Principles Of</a></li>
79
-<li><a href="contribute.wiki">or Documentation To The Fossil Project &#151; Contributing Code</a></li>
8084
 <li><a href="tech_overview.wiki">Overview Of The Design And Implementation Of Fossil &#151; A Technical</a></li>
8185
 <li><a href="index.wiki">Page &#151; Home</a></li>
8286
 <li><a href="password.wiki">Password Management And Authentication</a></li>
8387
 <li><a href="quotes.wiki">People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What</a></li>
8488
 <li><a href="stats.wiki">Performance Statistics</a></li>
@@ -86,10 +90,11 @@
8690
 <li><a href="pop.wiki">Principles Of Operations</a></li>
8791
 <li><a href="private.wiki">Private Branches &#151; Creating, Syncing, and Deleting</a></li>
8892
 <li><a href="makefile.wiki">Process &#151; The Fossil Build</a></li>
8993
 <li><a href="contribute.wiki">Project &#151; Contributing Code or Documentation To The Fossil</a></li>
9094
 <li><a href="embeddeddoc.wiki">Project Documentation &#151; Embedded</a></li>
95
+<li><a href="foss-cklist.wiki">Projects &#151; Checklist For Successful Open-Source</a></li>
9196
 <li><a href="sync.wiki">Protocol &#151; The Fossil Sync</a></li>
9297
 <li><a href="faq.wiki">Questions &#151; Frequently Asked</a></li>
9398
 <li><a href="qandc.wiki">Questions And Criticisms</a></li>
9499
 <li><a href="quickstart.wiki">Quick Start Guide &#151; Fossil</a></li>
95100
 <li><a href="quotes.wiki">Quotes: What People Are Saying About Fossil, Git, and DVCSes in General</a></li>
@@ -97,16 +102,19 @@
97102
 <li><a href="selfcheck.wiki">Repository Integrity Self Checks &#151; Fossil</a></li>
98103
 <li><a href="quotes.wiki">Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What People Are</a></li>
99104
 <li><a href="selfcheck.wiki">Self Checks &#151; Fossil Repository Integrity</a></li>
100105
 <li><a href="selfhost.wiki">Self Hosting Repositories &#151; Fossil</a></li>
101106
 <li><a href="server.wiki">Server &#151; How To Configure A Fossil</a></li>
107
+<li><a href="settings.wiki">Settings &#151; Fossil</a></li>
102108
 <li><a href="shunning.wiki">Shunning: Deleting Content From Fossil</a></li>
103109
 <li><a href="style.wiki">Source Code Style Guidelines</a></li>
104110
 <li><a href="tech_overview.wiki">SQLite Databases Used By Fossil</a></li>
111
+<li><a href="ssl.wiki">SSL with Fossil &#151; Using</a></li>
105112
 <li><a href="quickstart.wiki">Start Guide &#151; Fossil Quick</a></li>
106113
 <li><a href="stats.wiki">Statistics &#151; Performance</a></li>
107114
 <li><a href="style.wiki">Style Guidelines &#151; Source Code</a></li>
115
+<li><a href="foss-cklist.wiki">Successful Open-Source Projects &#151; Checklist For</a></li>
108116
 <li><a href="sync.wiki">Sync Protocol &#151; The Fossil</a></li>
109117
 <li><a href="private.wiki">Syncing, and Deleting Private Branches &#151; Creating,</a></li>
110118
 <li><a href="custom_ticket.wiki">System &#151; Customizing The Ticket</a></li>
111119
 <li><a href="branching.wiki">Tagging &#151; Branching, Forking, Merging, and</a></li>
112120
 <li><a href="tech_overview.wiki">Technical Overview Of The Design And Implementation Of Fossil &#151; A</a></li>
@@ -115,11 +123,13 @@
115123
 <li><a href="sync.wiki">The Fossil Sync Protocol</a></li>
116124
 <li><a href="webui.wiki">The Fossil Web Interface</a></li>
117125
 <li><a href="theory1.wiki">Thoughts On The Design Of The Fossil DVCS</a></li>
118126
 <li><a href="custom_ticket.wiki">Ticket System &#151; Customizing The</a></li>
119127
 <li><a href="bugtheory.wiki">Tracking In Fossil &#151; Bug</a></li>
128
+<li><a href="ssl.wiki">Using SSL with Fossil</a></li>
120129
 <li><a href="checkin_names.wiki">Version Names &#151; Checkin And</a></li>
121130
 <li><a href="fossil-v-git.wiki">Versus Git &#151; Fossil</a></li>
122131
 <li><a href="webui.wiki">Web Interface &#151; The Fossil</a></li>
123132
 <li><a href="quotes.wiki">What People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes:</a></li>
124133
 <li><a href="wikitheory.wiki">Wiki In Fossil</a></li>
134
+<li><a href="ssl.wiki">with Fossil &#151; Using SSL</a></li>
125135
 </ul>
126136
--- www/permutedindex.wiki
+++ www/permutedindex.wiki
@@ -11,12 +11,14 @@
11 <li><a href="private.wiki">Branches &#151; Creating, Syncing, and Deleting Private</a></li>
12 <li><a href="branching.wiki">Branching, Forking, Merging, and Tagging</a></li>
13 <li><a href="bugtheory.wiki">Bug Tracking In Fossil</a></li>
14 <li><a href="makefile.wiki">Build Process &#151; The Fossil</a></li>
15 <li><a href="build.wiki">Building and Installing Fossil</a></li>
 
16 <li><a href="checkin_names.wiki">Checkin And Version Names</a></li>
17 <li><a href="../test/release-checklist.wiki">Checklist &#151; Pre-Release Testing</a></li>
 
18 <li><a href="selfcheck.wiki">Checks &#151; Fossil Repository Integrity Self</a></li>
19 <li><a href="contribute.wiki">Code or Documentation To The Fossil Project &#151; Contributing</a></li>
20 <li><a href="style.wiki">Code Style Guidelines &#151; Source</a></li>
21 <li><a href="concepts.wiki">Concepts &#151; Fossil Core</a></li>
22 <li><a href="server.wiki">Configure A Fossil Server &#151; How To</a></li>
@@ -43,17 +45,19 @@
43 <li><a href="inout.wiki">Export To And From Git &#151; Import And</a></li>
44 <li><a href="fileformat.wiki">File Format &#151; Fossil</a></li>
45 <li><a href="branching.wiki">Forking, Merging, and Tagging &#151; Branching,</a></li>
46 <li><a href="delta_format.wiki">Format &#151; Fossil Delta</a></li>
47 <li><a href="fileformat.wiki">Format &#151; Fossil File</a></li>
 
48 <li><a href="concepts.wiki">Fossil Core Concepts</a></li>
49 <li><a href="delta_encoder_algorithm.wiki">Fossil Delta Encoding Algorithm</a></li>
50 <li><a href="delta_format.wiki">Fossil Delta Format</a></li>
51 <li><a href="fileformat.wiki">Fossil File Format</a></li>
52 <li><a href="quickstart.wiki">Fossil Quick Start Guide</a></li>
53 <li><a href="selfcheck.wiki">Fossil Repository Integrity Self Checks</a></li>
54 <li><a href="selfhost.wiki">Fossil Self Hosting Repositories</a></li>
 
55 <li><a href="fossil-v-git.wiki">Fossil Versus Git</a></li>
56 <li><a href="quotes.wiki">Fossil, Git, and DVCSes in General &#151; Quotes: What People Are Saying About</a></li>
57 <li><a href="faq.wiki">Frequently Asked Questions</a></li>
58 <li><a href="shunning.wiki">From Fossil &#151; Shunning: Deleting Content</a></li>
59 <li><a href="inout.wiki">From Git &#151; Import And Export To And</a></li>
@@ -73,12 +77,12 @@
73 <li><a href="webui.wiki">Interface &#151; The Fossil Web</a></li>
74 <li><a href="copyright-release.html">License Agreement &#151; Contributor</a></li>
75 <li><a href="password.wiki">Management And Authentication &#151; Password</a></li>
76 <li><a href="branching.wiki">Merging, and Tagging &#151; Branching, Forking,</a></li>
77 <li><a href="checkin_names.wiki">Names &#151; Checkin And Version</a></li>
 
78 <li><a href="pop.wiki">Operations &#151; Principles Of</a></li>
79 <li><a href="contribute.wiki">or Documentation To The Fossil Project &#151; Contributing Code</a></li>
80 <li><a href="tech_overview.wiki">Overview Of The Design And Implementation Of Fossil &#151; A Technical</a></li>
81 <li><a href="index.wiki">Page &#151; Home</a></li>
82 <li><a href="password.wiki">Password Management And Authentication</a></li>
83 <li><a href="quotes.wiki">People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What</a></li>
84 <li><a href="stats.wiki">Performance Statistics</a></li>
@@ -86,10 +90,11 @@
86 <li><a href="pop.wiki">Principles Of Operations</a></li>
87 <li><a href="private.wiki">Private Branches &#151; Creating, Syncing, and Deleting</a></li>
88 <li><a href="makefile.wiki">Process &#151; The Fossil Build</a></li>
89 <li><a href="contribute.wiki">Project &#151; Contributing Code or Documentation To The Fossil</a></li>
90 <li><a href="embeddeddoc.wiki">Project Documentation &#151; Embedded</a></li>
 
91 <li><a href="sync.wiki">Protocol &#151; The Fossil Sync</a></li>
92 <li><a href="faq.wiki">Questions &#151; Frequently Asked</a></li>
93 <li><a href="qandc.wiki">Questions And Criticisms</a></li>
94 <li><a href="quickstart.wiki">Quick Start Guide &#151; Fossil</a></li>
95 <li><a href="quotes.wiki">Quotes: What People Are Saying About Fossil, Git, and DVCSes in General</a></li>
@@ -97,16 +102,19 @@
97 <li><a href="selfcheck.wiki">Repository Integrity Self Checks &#151; Fossil</a></li>
98 <li><a href="quotes.wiki">Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What People Are</a></li>
99 <li><a href="selfcheck.wiki">Self Checks &#151; Fossil Repository Integrity</a></li>
100 <li><a href="selfhost.wiki">Self Hosting Repositories &#151; Fossil</a></li>
101 <li><a href="server.wiki">Server &#151; How To Configure A Fossil</a></li>
 
102 <li><a href="shunning.wiki">Shunning: Deleting Content From Fossil</a></li>
103 <li><a href="style.wiki">Source Code Style Guidelines</a></li>
104 <li><a href="tech_overview.wiki">SQLite Databases Used By Fossil</a></li>
 
105 <li><a href="quickstart.wiki">Start Guide &#151; Fossil Quick</a></li>
106 <li><a href="stats.wiki">Statistics &#151; Performance</a></li>
107 <li><a href="style.wiki">Style Guidelines &#151; Source Code</a></li>
 
108 <li><a href="sync.wiki">Sync Protocol &#151; The Fossil</a></li>
109 <li><a href="private.wiki">Syncing, and Deleting Private Branches &#151; Creating,</a></li>
110 <li><a href="custom_ticket.wiki">System &#151; Customizing The Ticket</a></li>
111 <li><a href="branching.wiki">Tagging &#151; Branching, Forking, Merging, and</a></li>
112 <li><a href="tech_overview.wiki">Technical Overview Of The Design And Implementation Of Fossil &#151; A</a></li>
@@ -115,11 +123,13 @@
115 <li><a href="sync.wiki">The Fossil Sync Protocol</a></li>
116 <li><a href="webui.wiki">The Fossil Web Interface</a></li>
117 <li><a href="theory1.wiki">Thoughts On The Design Of The Fossil DVCS</a></li>
118 <li><a href="custom_ticket.wiki">Ticket System &#151; Customizing The</a></li>
119 <li><a href="bugtheory.wiki">Tracking In Fossil &#151; Bug</a></li>
 
120 <li><a href="checkin_names.wiki">Version Names &#151; Checkin And</a></li>
121 <li><a href="fossil-v-git.wiki">Versus Git &#151; Fossil</a></li>
122 <li><a href="webui.wiki">Web Interface &#151; The Fossil</a></li>
123 <li><a href="quotes.wiki">What People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes:</a></li>
124 <li><a href="wikitheory.wiki">Wiki In Fossil</a></li>
 
125 </ul>
126
--- www/permutedindex.wiki
+++ www/permutedindex.wiki
@@ -11,12 +11,14 @@
11 <li><a href="private.wiki">Branches &#151; Creating, Syncing, and Deleting Private</a></li>
12 <li><a href="branching.wiki">Branching, Forking, Merging, and Tagging</a></li>
13 <li><a href="bugtheory.wiki">Bug Tracking In Fossil</a></li>
14 <li><a href="makefile.wiki">Build Process &#151; The Fossil</a></li>
15 <li><a href="build.wiki">Building and Installing Fossil</a></li>
16 <li><a href="changes.wiki">Changelog &#151; Fossil</a></li>
17 <li><a href="checkin_names.wiki">Checkin And Version Names</a></li>
18 <li><a href="../test/release-checklist.wiki">Checklist &#151; Pre-Release Testing</a></li>
19 <li><a href="foss-cklist.wiki">Checklist For Successful Open-Source Projects</a></li>
20 <li><a href="selfcheck.wiki">Checks &#151; Fossil Repository Integrity Self</a></li>
21 <li><a href="contribute.wiki">Code or Documentation To The Fossil Project &#151; Contributing</a></li>
22 <li><a href="style.wiki">Code Style Guidelines &#151; Source</a></li>
23 <li><a href="concepts.wiki">Concepts &#151; Fossil Core</a></li>
24 <li><a href="server.wiki">Configure A Fossil Server &#151; How To</a></li>
@@ -43,17 +45,19 @@
45 <li><a href="inout.wiki">Export To And From Git &#151; Import And</a></li>
46 <li><a href="fileformat.wiki">File Format &#151; Fossil</a></li>
47 <li><a href="branching.wiki">Forking, Merging, and Tagging &#151; Branching,</a></li>
48 <li><a href="delta_format.wiki">Format &#151; Fossil Delta</a></li>
49 <li><a href="fileformat.wiki">Format &#151; Fossil File</a></li>
50 <li><a href="changes.wiki">Fossil Changelog</a></li>
51 <li><a href="concepts.wiki">Fossil Core Concepts</a></li>
52 <li><a href="delta_encoder_algorithm.wiki">Fossil Delta Encoding Algorithm</a></li>
53 <li><a href="delta_format.wiki">Fossil Delta Format</a></li>
54 <li><a href="fileformat.wiki">Fossil File Format</a></li>
55 <li><a href="quickstart.wiki">Fossil Quick Start Guide</a></li>
56 <li><a href="selfcheck.wiki">Fossil Repository Integrity Self Checks</a></li>
57 <li><a href="selfhost.wiki">Fossil Self Hosting Repositories</a></li>
58 <li><a href="settings.wiki">Fossil Settings</a></li>
59 <li><a href="fossil-v-git.wiki">Fossil Versus Git</a></li>
60 <li><a href="quotes.wiki">Fossil, Git, and DVCSes in General &#151; Quotes: What People Are Saying About</a></li>
61 <li><a href="faq.wiki">Frequently Asked Questions</a></li>
62 <li><a href="shunning.wiki">From Fossil &#151; Shunning: Deleting Content</a></li>
63 <li><a href="inout.wiki">From Git &#151; Import And Export To And</a></li>
@@ -73,12 +77,12 @@
77 <li><a href="webui.wiki">Interface &#151; The Fossil Web</a></li>
78 <li><a href="copyright-release.html">License Agreement &#151; Contributor</a></li>
79 <li><a href="password.wiki">Management And Authentication &#151; Password</a></li>
80 <li><a href="branching.wiki">Merging, and Tagging &#151; Branching, Forking,</a></li>
81 <li><a href="checkin_names.wiki">Names &#151; Checkin And Version</a></li>
82 <li><a href="foss-cklist.wiki">Open-Source Projects &#151; Checklist For Successful</a></li>
83 <li><a href="pop.wiki">Operations &#151; Principles Of</a></li>
 
84 <li><a href="tech_overview.wiki">Overview Of The Design And Implementation Of Fossil &#151; A Technical</a></li>
85 <li><a href="index.wiki">Page &#151; Home</a></li>
86 <li><a href="password.wiki">Password Management And Authentication</a></li>
87 <li><a href="quotes.wiki">People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What</a></li>
88 <li><a href="stats.wiki">Performance Statistics</a></li>
@@ -86,10 +90,11 @@
90 <li><a href="pop.wiki">Principles Of Operations</a></li>
91 <li><a href="private.wiki">Private Branches &#151; Creating, Syncing, and Deleting</a></li>
92 <li><a href="makefile.wiki">Process &#151; The Fossil Build</a></li>
93 <li><a href="contribute.wiki">Project &#151; Contributing Code or Documentation To The Fossil</a></li>
94 <li><a href="embeddeddoc.wiki">Project Documentation &#151; Embedded</a></li>
95 <li><a href="foss-cklist.wiki">Projects &#151; Checklist For Successful Open-Source</a></li>
96 <li><a href="sync.wiki">Protocol &#151; The Fossil Sync</a></li>
97 <li><a href="faq.wiki">Questions &#151; Frequently Asked</a></li>
98 <li><a href="qandc.wiki">Questions And Criticisms</a></li>
99 <li><a href="quickstart.wiki">Quick Start Guide &#151; Fossil</a></li>
100 <li><a href="quotes.wiki">Quotes: What People Are Saying About Fossil, Git, and DVCSes in General</a></li>
@@ -97,16 +102,19 @@
102 <li><a href="selfcheck.wiki">Repository Integrity Self Checks &#151; Fossil</a></li>
103 <li><a href="quotes.wiki">Saying About Fossil, Git, and DVCSes in General &#151; Quotes: What People Are</a></li>
104 <li><a href="selfcheck.wiki">Self Checks &#151; Fossil Repository Integrity</a></li>
105 <li><a href="selfhost.wiki">Self Hosting Repositories &#151; Fossil</a></li>
106 <li><a href="server.wiki">Server &#151; How To Configure A Fossil</a></li>
107 <li><a href="settings.wiki">Settings &#151; Fossil</a></li>
108 <li><a href="shunning.wiki">Shunning: Deleting Content From Fossil</a></li>
109 <li><a href="style.wiki">Source Code Style Guidelines</a></li>
110 <li><a href="tech_overview.wiki">SQLite Databases Used By Fossil</a></li>
111 <li><a href="ssl.wiki">SSL with Fossil &#151; Using</a></li>
112 <li><a href="quickstart.wiki">Start Guide &#151; Fossil Quick</a></li>
113 <li><a href="stats.wiki">Statistics &#151; Performance</a></li>
114 <li><a href="style.wiki">Style Guidelines &#151; Source Code</a></li>
115 <li><a href="foss-cklist.wiki">Successful Open-Source Projects &#151; Checklist For</a></li>
116 <li><a href="sync.wiki">Sync Protocol &#151; The Fossil</a></li>
117 <li><a href="private.wiki">Syncing, and Deleting Private Branches &#151; Creating,</a></li>
118 <li><a href="custom_ticket.wiki">System &#151; Customizing The Ticket</a></li>
119 <li><a href="branching.wiki">Tagging &#151; Branching, Forking, Merging, and</a></li>
120 <li><a href="tech_overview.wiki">Technical Overview Of The Design And Implementation Of Fossil &#151; A</a></li>
@@ -115,11 +123,13 @@
123 <li><a href="sync.wiki">The Fossil Sync Protocol</a></li>
124 <li><a href="webui.wiki">The Fossil Web Interface</a></li>
125 <li><a href="theory1.wiki">Thoughts On The Design Of The Fossil DVCS</a></li>
126 <li><a href="custom_ticket.wiki">Ticket System &#151; Customizing The</a></li>
127 <li><a href="bugtheory.wiki">Tracking In Fossil &#151; Bug</a></li>
128 <li><a href="ssl.wiki">Using SSL with Fossil</a></li>
129 <li><a href="checkin_names.wiki">Version Names &#151; Checkin And</a></li>
130 <li><a href="fossil-v-git.wiki">Versus Git &#151; Fossil</a></li>
131 <li><a href="webui.wiki">Web Interface &#151; The Fossil</a></li>
132 <li><a href="quotes.wiki">What People Are Saying About Fossil, Git, and DVCSes in General &#151; Quotes:</a></li>
133 <li><a href="wikitheory.wiki">Wiki In Fossil</a></li>
134 <li><a href="ssl.wiki">with Fossil &#151; Using SSL</a></li>
135 </ul>
136
M www/quickstart.wiki
+15 -7
--- www/quickstart.wiki
+++ www/quickstart.wiki
@@ -9,11 +9,11 @@
99
 <p>Fossil is a single self-contained C program. You need to
1010
 either download a
1111
 <a href="http://www.fossil-scm.org/download.html">precompiled binary</a>
1212
 or <a href="build.wiki">build it yourself</a> from sources.
1313
 Install fossil by putting the fossil binary
14
- someplace on your PATH environment variable.</p>
14
+ someplace on your $PATH.</p>
1515
1616
 <a name="fslclone"></a>
1717
 <h2>General Work Flow</h2>
1818
1919
 <p>Fossil works with repository files (a database with the project's
@@ -20,29 +20,27 @@
2020
 complete history) and with checked-out local trees (the working directory
2121
 you use to do your work).
2222
 The workflow looks like this:</p>
2323
2424
 <ul>
25
- <li>Create or clone a repository file. ([/help/new|fossil new] or
25
+ <li>Create or clone a repository file. ([/help/init|fossil init] or
2626
 [/help/clone | fossil clone])
2727
 <li>Check out a local tree. ([/help/open | fossil open])
2828
 <li>Perform operations on the repository (including repository
2929
 configuration).
30
- <li><em>Optionally</em> close the local tree.
31
- ([/help/close | fossil close], but this is rarely used.)
3230
 </ul>
3331
3432
 <p>The following sections will give you a brief overview of these
3533
 operations.</p>
3634
3735
 <h2>Starting A New Project</h2>
3836
3937
 <p>To start a new project with fossil, create a new empty repository
40
- this way: ([/help/new | more info]) </p>
38
+ this way: ([/help/init | more info]) </p>
4139
4240
 <blockquote>
43
- <b>fossil new </b><i> repository-filename</i>
41
+ <b>fossil init </b><i> repository-filename</i>
4442
 </blockquote>
4543
4644
 <h2>Cloning An Existing Repository</h2>
4745
4846
 <p>Most fossil operations interact with a repository that is on the
@@ -73,10 +71,19 @@
7371
 You can name your repositories anything you want. The ".fossil" suffix
7472
 is not required.</p>
7573
7674
 <p>Note: If you are behind a restrictive firewall, you might need
7775
 to <a href="#proxy">specify an HTTP proxy</a> to use.</p>
76
+
77
+ <p>A Fossil repository is a single disk file. Instead of cloning,
78
+ you can just make a copy of the repository file (for example, using
79
+ "scp"). Note, however, that the repository file contains auxiliary
80
+ information above and beyond the versioned files, including some
81
+ sensitive information such as passwords and email addresses. If you
82
+ want to share Fossil repositories directly, consider running the
83
+ [/help/scrub|fossil scrub] command to remove sensitive information
84
+ before transmitting the file.
7885
7986
 <h2>Importing From Another Version Control System</h2>
8087
8188
 <p>Rather than start a new project, or clone an existing Fossil project,
8289
 you might prefer to
@@ -147,11 +154,12 @@
147154
 <p>To add new files to your project, or remove old files, use these
148155
 commands:</p>
149156
150157
 <blockquote>
151158
 <b>[/help/add | fossil add]</b> <i>file...</i><br>
152
- <b>[/help/rm | fossil rm]</b> <i>file...</i>
159
+ <b>[/help/rm | fossil rm]</b> <i>file...</i><br>
160
+ <b>[/help/addremove | fossil addremove]</b> <i>file...</i><br>
153161
 </blockquote>
154162
155163
 <p>You can also edit files freely. Once you are ready to commit
156164
 your changes, type:</p>
157165
158166
--- www/quickstart.wiki
+++ www/quickstart.wiki
@@ -9,11 +9,11 @@
9 <p>Fossil is a single self-contained C program. You need to
10 either download a
11 <a href="http://www.fossil-scm.org/download.html">precompiled binary</a>
12 or <a href="build.wiki">build it yourself</a> from sources.
13 Install fossil by putting the fossil binary
14 someplace on your PATH environment variable.</p>
15
16 <a name="fslclone"></a>
17 <h2>General Work Flow</h2>
18
19 <p>Fossil works with repository files (a database with the project's
@@ -20,29 +20,27 @@
20 complete history) and with checked-out local trees (the working directory
21 you use to do your work).
22 The workflow looks like this:</p>
23
24 <ul>
25 <li>Create or clone a repository file. ([/help/new|fossil new] or
26 [/help/clone | fossil clone])
27 <li>Check out a local tree. ([/help/open | fossil open])
28 <li>Perform operations on the repository (including repository
29 configuration).
30 <li><em>Optionally</em> close the local tree.
31 ([/help/close | fossil close], but this is rarely used.)
32 </ul>
33
34 <p>The following sections will give you a brief overview of these
35 operations.</p>
36
37 <h2>Starting A New Project</h2>
38
39 <p>To start a new project with fossil, create a new empty repository
40 this way: ([/help/new | more info]) </p>
41
42 <blockquote>
43 <b>fossil new </b><i> repository-filename</i>
44 </blockquote>
45
46 <h2>Cloning An Existing Repository</h2>
47
48 <p>Most fossil operations interact with a repository that is on the
@@ -73,10 +71,19 @@
73 You can name your repositories anything you want. The ".fossil" suffix
74 is not required.</p>
75
76 <p>Note: If you are behind a restrictive firewall, you might need
77 to <a href="#proxy">specify an HTTP proxy</a> to use.</p>
 
 
 
 
 
 
 
 
 
78
79 <h2>Importing From Another Version Control System</h2>
80
81 <p>Rather than start a new project, or clone an existing Fossil project,
82 you might prefer to
@@ -147,11 +154,12 @@
147 <p>To add new files to your project, or remove old files, use these
148 commands:</p>
149
150 <blockquote>
151 <b>[/help/add | fossil add]</b> <i>file...</i><br>
152 <b>[/help/rm | fossil rm]</b> <i>file...</i>
 
153 </blockquote>
154
155 <p>You can also edit files freely. Once you are ready to commit
156 your changes, type:</p>
157
158
--- www/quickstart.wiki
+++ www/quickstart.wiki
@@ -9,11 +9,11 @@
9 <p>Fossil is a single self-contained C program. You need to
10 either download a
11 <a href="http://www.fossil-scm.org/download.html">precompiled binary</a>
12 or <a href="build.wiki">build it yourself</a> from sources.
13 Install fossil by putting the fossil binary
14 someplace on your $PATH.</p>
15
16 <a name="fslclone"></a>
17 <h2>General Work Flow</h2>
18
19 <p>Fossil works with repository files (a database with the project's
@@ -20,29 +20,27 @@
20 complete history) and with checked-out local trees (the working directory
21 you use to do your work).
22 The workflow looks like this:</p>
23
24 <ul>
25 <li>Create or clone a repository file. ([/help/init|fossil init] or
26 [/help/clone | fossil clone])
27 <li>Check out a local tree. ([/help/open | fossil open])
28 <li>Perform operations on the repository (including repository
29 configuration).
 
 
30 </ul>
31
32 <p>The following sections will give you a brief overview of these
33 operations.</p>
34
35 <h2>Starting A New Project</h2>
36
37 <p>To start a new project with fossil, create a new empty repository
38 this way: ([/help/init | more info]) </p>
39
40 <blockquote>
41 <b>fossil init </b><i> repository-filename</i>
42 </blockquote>
43
44 <h2>Cloning An Existing Repository</h2>
45
46 <p>Most fossil operations interact with a repository that is on the
@@ -73,10 +71,19 @@
71 You can name your repositories anything you want. The ".fossil" suffix
72 is not required.</p>
73
74 <p>Note: If you are behind a restrictive firewall, you might need
75 to <a href="#proxy">specify an HTTP proxy</a> to use.</p>
76
77 <p>A Fossil repository is a single disk file. Instead of cloning,
78 you can just make a copy of the repository file (for example, using
79 "scp"). Note, however, that the repository file contains auxiliary
80 information above and beyond the versioned files, including some
81 sensitive information such as passwords and email addresses. If you
82 want to share Fossil repositories directly, consider running the
83 [/help/scrub|fossil scrub] command to remove sensitive information
84 before transmitting the file.
85
86 <h2>Importing From Another Version Control System</h2>
87
88 <p>Rather than start a new project, or clone an existing Fossil project,
89 you might prefer to
@@ -147,11 +154,12 @@
154 <p>To add new files to your project, or remove old files, use these
155 commands:</p>
156
157 <blockquote>
158 <b>[/help/add | fossil add]</b> <i>file...</i><br>
159 <b>[/help/rm | fossil rm]</b> <i>file...</i><br>
160 <b>[/help/addremove | fossil addremove]</b> <i>file...</i><br>
161 </blockquote>
162
163 <p>You can also edit files freely. Once you are ready to commit
164 your changes, type:</p>
165
166
M www/server.wiki
+3
--- www/server.wiki
+++ www/server.wiki
@@ -97,10 +97,13 @@
9797
 If you are using "inetd" to serve your repository, then you simply need to add "/usr/bin/stunnel" (perhaps on a different path, depending on your setup) before the command line to launch Fossil.
9898
 </p>
9999
 <p>
100100
 At this stage, the standalone server (e.g. "fossil server") does not support SSL.
101101
 </p>
102
+<p>
103
+For more information, see <a href="./ssl.wiki">Using SSL with Fossil</a>.
104
+</p>
102105
 </blockquote>
103106
104107
 <h2>Various security concerns with hosted repositories</h2><blockquote>
105108
 <p>
106109
 There are two main concerns relating to usage of Fossil for sharing sensitive information (source or any other data):
107110
108111
ADDED www/settings.wiki
109112
ADDED www/ssl.wiki
--- www/server.wiki
+++ www/server.wiki
@@ -97,10 +97,13 @@
97 If you are using "inetd" to serve your repository, then you simply need to add "/usr/bin/stunnel" (perhaps on a different path, depending on your setup) before the command line to launch Fossil.
98 </p>
99 <p>
100 At this stage, the standalone server (e.g. "fossil server") does not support SSL.
101 </p>
 
 
 
102 </blockquote>
103
104 <h2>Various security concerns with hosted repositories</h2><blockquote>
105 <p>
106 There are two main concerns relating to usage of Fossil for sharing sensitive information (source or any other data):
107
108 DDED www/settings.wiki
109 DDED www/ssl.wiki
--- www/server.wiki
+++ www/server.wiki
@@ -97,10 +97,13 @@
97 If you are using "inetd" to serve your repository, then you simply need to add "/usr/bin/stunnel" (perhaps on a different path, depending on your setup) before the command line to launch Fossil.
98 </p>
99 <p>
100 At this stage, the standalone server (e.g. "fossil server") does not support SSL.
101 </p>
102 <p>
103 For more information, see <a href="./ssl.wiki">Using SSL with Fossil</a>.
104 </p>
105 </blockquote>
106
107 <h2>Various security concerns with hosted repositories</h2><blockquote>
108 <p>
109 There are two main concerns relating to usage of Fossil for sharing sensitive information (source or any other data):
110
111 DDED www/settings.wiki
112 DDED www/ssl.wiki
M www/settings.wiki
+6
--- a/www/settings.wiki
+++ b/www/settings.wiki
@@ -0,0 +1,6 @@
1
+ command.
2
+ a repository on the web. b</tt>,
3
+ � �
4
+ddirectory < <title>Fossil S nl-glob</tt>,
5
+ � �
6
+ddirectory t ignorealso
--- a/www/settings.wiki
+++ b/www/settings.wiki
@@ -0,0 +1,6 @@
 
 
 
 
 
 
--- a/www/settings.wiki
+++ b/www/settings.wiki
@@ -0,0 +1,6 @@
1 command.
2 a repository on the web. b</tt>,
3 � �
4 ddirectory < <title>Fossil S nl-glob</tt>,
5 � �
6 ddirectory t ignorealso
M www/ssl.wiki
+21
--- a/www/ssl.wiki
+++ b/www/ssl.wiki
@@ -0,0 +1,21 @@
1
+<title>SSL and Fossiluring a ReposiUsing SSLyour repositge." This setting is not . This will protectessed via HTTP,
2
+sas well preventing eavesdropping of the contents of your repository.
3
+
4
+To host a repository with SSL, you need to use an web server which supports SSL in front of the Fossil server. You can host it using the CGI option or by proxying Fossil's built in HTTP server.
5
+
6
+Your fossil client must system Make sure scheme to ensure you're using SSL. from http as w important S front end proxy up again for access, which
7
+causes Fossil to see that it's being accessed via HTTP, so it redirects
8
+the client to detects
9
+
10
+
11
+If you wishto a Fossil web server, iayer, not by use of Fossil-
12
+shows one way to`ssl`SSL uses certificates.certificates you trustyou wishto a Fossil web server, iayer, not by use of Fossil-
13
+shows one way e of Fossil-
14
+shows one way to`ssl` using a certificate signed by a certificate authority, you need to specifyil-
15
+shows one was you trust with the <tt>ssl-ca-location</tt> setting. Set this globally with the <tt>-global</tt> option for convenience.
16
+
17
+This should be set to the location of a file containing all the PEM encoded certificates you trust. You can obtain a certificate using a web browser, for example, Firefox, or just refer to your system's trusted CA roots which are usually stored somewhere in <tt>/etc</tt>.
18
+
19
+
20
+<h2>C
21
+If you are storing sensitive information in a repository accessible T <title title>TLS and Fossiluring a ReposiUsing TLS-EncryptedTLS
--- a/www/ssl.wiki
+++ b/www/ssl.wiki
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
--- a/www/ssl.wiki
+++ b/www/ssl.wiki
@@ -0,0 +1,21 @@
1 <title>SSL and Fossiluring a ReposiUsing SSLyour repositge." This setting is not . This will protectessed via HTTP,
2 sas well preventing eavesdropping of the contents of your repository.
3
4 To host a repository with SSL, you need to use an web server which supports SSL in front of the Fossil server. You can host it using the CGI option or by proxying Fossil's built in HTTP server.
5
6 Your fossil client must system Make sure scheme to ensure you're using SSL. from http as w important S front end proxy up again for access, which
7 causes Fossil to see that it's being accessed via HTTP, so it redirects
8 the client to detects
9
10
11 If you wishto a Fossil web server, iayer, not by use of Fossil-
12 shows one way to`ssl`SSL uses certificates.certificates you trustyou wishto a Fossil web server, iayer, not by use of Fossil-
13 shows one way e of Fossil-
14 shows one way to`ssl` using a certificate signed by a certificate authority, you need to specifyil-
15 shows one was you trust with the <tt>ssl-ca-location</tt> setting. Set this globally with the <tt>-global</tt> option for convenience.
16
17 This should be set to the location of a file containing all the PEM encoded certificates you trust. You can obtain a certificate using a web browser, for example, Firefox, or just refer to your system's trusted CA roots which are usually stored somewhere in <tt>/etc</tt>.
18
19
20 <h2>C
21 If you are storing sensitive information in a repository accessible T <title title>TLS and Fossiluring a ReposiUsing TLS-EncryptedTLS

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button