Querying with Cypher¶

There are two ways to run Cypher, and most code wants the first:

The examples on this page share this setup:

import pyarrow as pa
from lance_csr import CypherEngine, GraphConfig

config = (
    GraphConfig.builder()
    .node_table("Person", "id")
    .node_table("City", "id")
    .rel_table("KNOWS", "Person", "Person", "src", "dst")
    .rel_table("LIVES_IN", "Person", "City", "src", "dst")
    .build()
)
datasets = {
    "Person": pa.table({
        "id": [1, 2, 3, 4],
        "name": ["Alice", "Bob", "Carol", "David"],
        "age": [28, 34, 29, 42],
    }),
    "City": pa.table({"id": [10, 20], "name": ["London", "Sydney"]}),
    "KNOWS": pa.table({"src": [1, 1, 2, 3], "dst": [2, 3, 4, 4]}),
    "LIVES_IN": pa.table({"src": [1, 2, 3, 4], "dst": [10, 10, 20, 20]}),
}
engine = CypherEngine(config, datasets, use_csr=True)

CypherEngine¶

CypherEngine(config, datasets, use_csr=False)

The constructor validates the datasets against the config, registers them with a DataFusion session, and, with use_csr=True, builds the CSR index. After that, every query reuses the same catalog and session.

execute(query, params=None)¶

Runs the query and returns the whole result as a pyarrow.Table:

table = engine.execute("""
    MATCH (p:Person)-[:LIVES_IN]->(c:City)
    RETURN c.name AS city, collect(p.name) AS people, avg(p.age) AS avg_age
    ORDER BY city
""")
print(table.to_pylist())
# [{'city': 'London', 'people': ['Alice', 'Bob'], 'avg_age': 31.0},
#  {'city': 'Sydney', 'people': ['Carol', 'David'], 'avg_age': 35.5}]

Output columns are named by their AS alias, or by the expression text (p.name) when there is none.

Parameters¶

Write $name in the query and pass a dict as the second argument. Parameter values can be numbers, strings, booleans, None, lists, string-keyed maps, and datetime.datetime or datetime.date values:

import datetime

engine.execute(
    "MATCH (p:Person) WHERE p.age >= $min_age AND p.name <> $skip RETURN p.name AS name",
    {"min_age": 30, "skip": "Bob"},
).to_pylist()  # [{'name': 'David'}]

Lists retain integer IDs, and maps can contain nested lists/maps and nulls:

engine.execute(
    "MATCH (p:Person) WHERE p.id IN $ids RETURN p.name AS name ORDER BY name",
    {"ids": [1, 3]},
).to_pylist()  # [{'name': 'Alice'}, {'name': 'Carol'}]

Pagination also accepts parameters, for example RETURN 1 AS n LIMIT $take with {"take": 1}. WITH ... LIMIT $take uses the same rules. Missing, null, boolean, fractional, negative and overflowing counts raise ValueError; arbitrary expressions and WITH SKIP remain unsupported.

Map keys must be strings, integer parameters must fit signed 64-bit values, and nesting is limited to 128 levels. Cyclic containers and unsupported Python objects raise ValueError. Lists follow literal-list coercion: mixed string/numeric values become strings because Arrow lists have one element type. Containers are expression values; inline node/relationship property filters still require scalar parameters.

Datetimes compare against timestamp columns, and dates against date columns. Parameters are substituted before planning, so they are also allowed in inline property maps: (p:Person {id: $id}).

execute_batches(query, params=None)¶

Streams the result as a pyarrow.RecordBatchReader instead of building one table:

reader = engine.execute_batches("MATCH (p:Person) RETURN p.id AS id, p.name AS name")
rows = 0
for batch in reader:  # each batch is a pyarrow.RecordBatch
    rows += batch.num_rows
print(rows)  # 4

• Batches are computed as you read them. Scans, filters and projections stream with bounded memory; joins, sorts and aggregations may still buffer their inputs.
• Planning errors (a syntax error, an unknown label) raise immediately. Execution errors raise while reading.
• The reader keeps its snapshot alive, so a concurrent refresh() doesn't affect it.
• Dropping the reader cancels the remaining work.

explain(query, params=None)¶

Returns the plan as text: the graph logical plan, DataFusion's logical plan and the physical plan. It's the quickest way to check whether a hop used the CSR (CsrExpandExec) or a join (HashJoinExec), and whether filters reached the Lance scan:

plan = engine.explain("MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN b.name")
print("CsrExpandExec" in plan)  # True

Other methods¶

CypherQuery¶

CypherQuery parses a query up front and runs it against whatever datasets you hand it. Each builder method returns a new query:

from lance_csr import CypherQuery

query = (
    CypherQuery("MATCH (p:Person) WHERE p.age > $age RETURN p.name AS name ORDER BY name")
    .with_config(config)
    .with_parameter("age", 30)
)
print(query.execute(datasets).to_pydict())  # {'name': ['Bob', 'David']}

strategy takes an ExecutionStrategy. ExecutionStrategy.DataFusion is the default and the only one implemented; ExecutionStrategy.LanceNative is reserved and raises an error.

Supported Cypher¶

The Graph Data Council (GDC), formerly known as the Linked Data Benchmark Council (LDBC), maintains the GDC SNB Interactive complex queries. lance-csr implements a large read-only subset of openCypher, enough to run all 14 (IC14 without reduce()).

Clauses¶

Integer list literals retain signed 64-bit values: RETURN [1, 2, 3] AS ids returns integers, and UNWIND [1131, 19791209300775] AS id RETURN id preserves both IDs exactly. Numeric literals with a decimal point or exponent, such as [0.1, 1e-3], form float32 vectors. Whole-number lists can also be used as query vectors; conversion to float32 happens inside the vector operation. A configured engine is still required for standalone RETURN queries.

Patterns¶

// Direction: outgoing, incoming or either
MATCH (a:Person)-[:KNOWS]->(b)     // outgoing
MATCH (a:Person)<-[:KNOWS]-(b)     // incoming
MATCH (a:Person)-[:KNOWS]-(b)      // either direction

// Variable length: *, *n, *n..m, *..m, *n..
MATCH (a:Person {id: 1})-[:KNOWS*1..3]->(b:Person)

// Alternation, also with variable length
MATCH (p:Person)-[:LIKES_POST|LIKES_COMMENT]->(m)

// Relationship variables and properties
MATCH (a:Person)-[k:KNOWS]->(b:Person) WHERE k.since > 2020

// Shortest paths (one relationship type, both ends bound)
MATCH p = shortestPath((a:Person {id: 1})-[:KNOWS*]-(b:Person {id: 4}))
RETURN length(p), nodes(p)

• Variable length defaults to a minimum of 1 and a maximum of 20 hops; a maximum above 20 is rejected, and zero-length paths (*0..) are not supported. Each length becomes one branch of a union.
• Undirected variable-length paths skip immediate backtracking (a-b-a).
• shortestPath and allShortestPaths need exactly one relationship type, no relationship variable, and named end nodes. Their length is unbounded unless you give a maximum. nodes(p) returns the node keys. With no path, MATCH drops the row and OPTIONAL MATCH returns null.
• Named paths (p = ...) are only supported with shortestPath and allShortestPaths.

Expressions and predicates¶

Functions¶

Any other function raises NotImplementedError: Cypher function 'name' is not implemented. Notably missing: math functions (abs, round, sqrt, ...), most string and list functions, reduce, the list predicates any / all / none / single, and relationships(p).

When size() consumes an unmapped list comprehension, such as size([x IN values WHERE x > 0]), the planner counts retained elements directly. Each duplicate and each retained null element still contributes one; an empty or null input list produces zero, as for the existing comprehension. A mapped comprehension (| expression) or a pattern comprehension keeps the ordinary list-building path. The count path still unnests and groups rows; explain() shows a filtered count in place of ordered list aggregation. A direct count output is marked non-nullable in its Arrow schema; the previous list-size expression was marked nullable.

Example: a little of everything¶

result = engine.execute("""
    MATCH (p:Person)
    OPTIONAL MATCH (p)-[:KNOWS]->(f:Person)
    WITH p, count(f) AS friends
    WHERE EXISTS { MATCH (p)-[:LIVES_IN]->(:City {name: 'London'}) }
    RETURN p.name AS name,
           friends,
           CASE WHEN p.age >= 30 THEN 'senior' ELSE 'junior' END AS band
    ORDER BY name
""")
print(result.to_pylist())
# [{'name': 'Alice', 'friends': 2, 'band': 'junior'},
#  {'name': 'Bob', 'friends': 1, 'band': 'senior'}]

Case sensitivity¶

Keywords, function names, variables, labels, relationship types and property names are all case-insensitive. match (P:person) return p.NAME works, and the output column keeps the spelling you wrote.

An output alias may reuse an input variable's name. Expressions in the same RETURN still use the input variable; the following ORDER BY uses the output alias. For example, RETURN p.name AS p, p.id AS id ORDER BY p returns names and IDs, sorted by name. This also works with aggregation and OPTIONAL MATCH.

Errors¶