10

I would like to query an SQLite table that contains directory paths to find all the paths under some hierarchy. Here's an example of the contents of the column:

/alpha/papa/
/alpha/papa/tango/
/alpha/quebec/
/bravo/papa/
/bravo/papa/uniform/
/charlie/quebec/tango/

If I search for everything under /bravo/papa/, I would like to get:

/bravo/papa/
/bravo/papa/uniform/

I am currently trying to do this like so (see below for the long story of why I can't use more simple methods):

SELECT * FROM Files WHERE Path >= '/bravo/papa/' AND Path < '/bravo/papa0';

This works. It looks a bit weird, but it works for this example. '0' is the unicode code point 1 greater than '/'. When ordered lexicographically, all the paths starting with '/bravo/papa/' compare greater than it and less than 'bravo/papa0'. However, in my tests, I find that this breaks down when we try this:

SELECT * FROM Files WHERE Path >= '/' AND Path < '0';

This returns no results, but it should return every row. As far as I can tell, the problem is that SQLite is treating '0' as a number, not a string. If I use '0Z' instead of '0', for example, I do get results, but I introduce a risk of getting false positives. (For example, if there actually was an entry '0'.)

The simple version of my question is: is there some way to get SQLite to treat '0' in such a query as the length-1 string containing the unicode character '0' (which should sort strings such as '!', '*' and '/', but before '1', '=' and 'A') instead of the integer 0 (which SQLite sorts before all strings)?

I think in this case I can actually get away with special-casing a search for everything under '/', since all my entries will always start with '/', but I'd really like to know how to avoid this sort of thing in general, as it's unpleasantly surprising in all the same ways as Javascript's "==" operator.

First approach

A more natural approach would be to use the LIKE or GLOB operator. For example:

SELECT * FROM Files WHERE Path LIKE @prefix || '%';

But I want to support all valid path characters, so I would need to use ESCAPE for the '_' and '%' symbols. Apparently this prevents SQLite from using an index on Path. (See http://www.sqlite.org/optoverview.html#like_opt ) I really want to be able to benefit from an index here, and it sounds like that's impossible using either LIKE or GLOB unless I can guarantee that none of their special characters will occur in the directory name, and POSIX allows anything other than NUL and '/', even GLOB's '*' and '?' characters.

I'm providing this for context. I'm interested in other approaches to solve the underlying problem, but I'd prefer to accept an answer that directly addresses the ambiguity of strings-that-look-like-numbers in SQLite.

Similar questions

How do I prevent sqlite from evaluating a string as a math expression?

In that question, the values weren't quoted. I get these results even when the values are quoted or passed in as parameters.


EDIT - See my answer below. The column was created with the invalid type "STRING", which SQLite treated as NUMERIC.

5
  • 1
    Works for me. How is the Path column defined? Commented Mar 13, 2013 at 17:42
  • @CL. I'm pretty sure it was defined as a TEXT column, but it's not in front of me right now. I'll check tomorrow and update the question with than and the SQLite version number. Commented Mar 13, 2013 at 18:24
  • I'm not sure if this would work, but what about something like WHERE 'X' || Path > 'X' || '/bravo/papa/' AND 'X' || '/bravo/papa0'? Or if possible, assigning a variable to the value of 'X' || Path first. Commented Mar 13, 2013 at 21:30
  • I'll try that, though I'm not confident it will still manage to use an index. However, I've tried it out the queries on my Linux laptop and it's working as I'd hoped in the first place as CL. reports, so there must be something else going on with my original database on the Windows machine. I'll update tomorrow. Commented Mar 13, 2013 at 23:22
  • I also had this problem. All my string columns were defined as STRING rather than TEXT. Worked fine until I tried to insert something which looked like "1.4e-1", which sqlite blithely turned into 0.14 . I wish the sqlite engine would raise some sort of exception when declaring affinity for a type which does not exist (like STRING). Commented Jun 3, 2015 at 13:26

1 Answer 1

9

* Groan *. The column had NUMERIC affinity because it had accidentally been specified as "STRING" instead of "TEXT". Since SQLite didn't recognize the type name, it made it NUMERIC, and because SQLite doesn't enforce column types, everything else worked as expected, except that any time a number-like string is inserted into that column it is converted into a numeric type.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.