rdflib.plugins.stores package

Submodules

rdflib.plugins.stores.auditable module

This wrapper intercepts calls through the store interface and implements thread-safe logging of destructive operations (adds / removes) in reverse. This is persisted on the store instance and the reverse operations are executed In order to return the store to the state it was when the transaction began Since the reverse operations are persisted on the store, the store itself acts as a transaction.

Calls to commit or rollback, flush the list of reverse operations This provides thread-safe atomicity and isolation (assuming concurrent operations occur with different store instances), but no durability (transactions are persisted in memory and wont be available to reverse operations after the system fails): A and I out of ACID.

class rdflib.plugins.stores.auditable.AuditableStore(store)

Bases: rdflib.store.Store

__init__(store)

identifier: URIRef of the Store. Defaults to CWD configuration: string containing infomation open can use to connect to datastore.

__len__(context=None)

Number of statements in the store. This should only account for non- quoted (asserted) statements if the context is not specified, otherwise it should return the number of statements in the formula or context given.

Parameters

context – a graph instance to query or None

__module__ = 'rdflib.plugins.stores.auditable'

add(triple, context, quoted=False)

Adds the given statement to a specific context or to the model. The quoted argument is interpreted by formula-aware stores to indicate this statement is quoted/hypothetical It should be an error to not specify a context and have the quoted argument be True. It should also be an error for the quoted argument to be True when the store is not formula-aware.

bind(prefix, namespace)

close(commit_pending_transaction=False)

This closes the database connection. The commit_pending_transaction parameter specifies whether to commit all pending transactions before closing (if the store is transactional).

commit()

contexts(triple=None)

Generator over all contexts in the graph. If triple is specified, a generator over all contexts the triple is in.

if store is graph_aware, may also return empty contexts

Returns

a generator over Nodes

destroy(configuration)

This destroys the instance of the store identified by the configuration string.

namespace(prefix)

namespaces()

open(configuration, create=True)

Opens the store specified by the configuration string. If create is True a store will be created if it does not already exist. If create is False and a store does not already exist an exception is raised. An exception is also raised if a store exists, but there is insufficient permissions to open the store. This should return one of: VALID_STORE, CORRUPTED_STORE, or NO_STORE

prefix(namespace)

query(*args, **kw)

If stores provide their own SPARQL implementation, override this.

queryGraph is None, a URIRef or ‘__UNION__’ If None the graph is specified in the query-string/object If URIRef it specifies the graph to query, If ‘__UNION__’ the union of all named graphs should be queried (This is used by ConjunctiveGraphs Values other than None obviously only makes sense for context-aware stores.)

remove(spo, context=None)

Remove the set of triples matching the pattern from the store

rollback()

triples(triple, context=None)

A generator over all the triples matching the pattern. Pattern can include any objects for used for comparing against nodes in the store, for example, REGEXTerm, URIRef, Literal, BNode, Variable, Graph, QuotedGraph, Date? DateRange?

Parameters

context – A conjunctive query can be indicated by either providing a value of None, or a specific context can be queries by passing a Graph instance (if store is context aware).

rdflib.plugins.stores.concurrent module

class rdflib.plugins.stores.concurrent.ConcurrentStore(store)

Bases: object

__dict__ = mappingproxy({'__module__': 'rdflib.plugins.stores.concurrent', '__init__': <function ConcurrentStore.__init__>, 'add': <function ConcurrentStore.add>, 'remove': <function ConcurrentStore.remove>, 'triples': <function ConcurrentStore.triples>, '__len__': <function ConcurrentStore.__len__>, '_ConcurrentStore__begin_read': <function ConcurrentStore.__begin_read>, '_ConcurrentStore__end_read': <function ConcurrentStore.__end_read>, '__dict__': <attribute '__dict__' of 'ConcurrentStore' objects>, '__weakref__': <attribute '__weakref__' of 'ConcurrentStore' objects>, '__doc__': None})

__init__(store)

Initialize self. See help(type(self)) for accurate signature.

__len__()

__module__ = 'rdflib.plugins.stores.concurrent'

__weakref__

list of weak references to the object (if defined)

add(triple)

remove(triple)

triples(triple)

class rdflib.plugins.stores.concurrent.ResponsibleGenerator(gen, cleanup)

Bases: object

A generator that will help clean up when it is done being used.

__del__()

__init__(gen, cleanup)

Initialize self. See help(type(self)) for accurate signature.

__iter__()

__module__ = 'rdflib.plugins.stores.concurrent'

__next__()

__slots__ = ['cleanup', 'gen']

cleanup

gen

rdflib.plugins.stores.regexmatching module

This wrapper intercepts calls through the store interface which make use of the REGEXTerm class to represent matches by REGEX instead of literal comparison.

Implemented for stores that don’t support this and essentially provides the support by replacing the REGEXTerms by wildcards (None) and matching against the results from the store it’s wrapping.

class rdflib.plugins.stores.regexmatching.REGEXMatching(storage)

Bases: rdflib.store.Store

__init__(storage)

identifier: URIRef of the Store. Defaults to CWD configuration: string containing infomation open can use to connect to datastore.

__len__(context=None)

Number of statements in the store. This should only account for non- quoted (asserted) statements if the context is not specified, otherwise it should return the number of statements in the formula or context given.

Parameters

context – a graph instance to query or None

__module__ = 'rdflib.plugins.stores.regexmatching'

add(triple, context, quoted=False)

Adds the given statement to a specific context or to the model. The quoted argument is interpreted by formula-aware stores to indicate this statement is quoted/hypothetical It should be an error to not specify a context and have the quoted argument be True. It should also be an error for the quoted argument to be True when the store is not formula-aware.

bind(prefix, namespace)

close(commit_pending_transaction=False)

This closes the database connection. The commit_pending_transaction parameter specifies whether to commit all pending transactions before closing (if the store is transactional).

commit()

contexts(triple=None)

Generator over all contexts in the graph. If triple is specified, a generator over all contexts the triple is in.

if store is graph_aware, may also return empty contexts

Returns

a generator over Nodes

destroy(configuration)

This destroys the instance of the store identified by the configuration string.

namespace(prefix)

namespaces()

open(configuration, create=True)

Opens the store specified by the configuration string. If create is True a store will be created if it does not already exist. If create is False and a store does not already exist an exception is raised. An exception is also raised if a store exists, but there is insufficient permissions to open the store. This should return one of: VALID_STORE, CORRUPTED_STORE, or NO_STORE

prefix(namespace)

remove(triple, context=None)

Remove the set of triples matching the pattern from the store

remove_context(identifier)

rollback()

triples(triple, context=None)

A generator over all the triples matching the pattern. Pattern can include any objects for used for comparing against nodes in the store, for example, REGEXTerm, URIRef, Literal, BNode, Variable, Graph, QuotedGraph, Date? DateRange?

Parameters

context – A conjunctive query can be indicated by either providing a value of None, or a specific context can be queries by passing a Graph instance (if store is context aware).

class rdflib.plugins.stores.regexmatching.REGEXTerm(expr)

Bases: str

REGEXTerm can be used in any term slot and is interpreted as a request to perform a REGEX match (not a string comparison) using the value (pre-compiled) for checking rdf:type matches

__dict__ = mappingproxy({'__module__': 'rdflib.plugins.stores.regexmatching', '__doc__': '\n REGEXTerm can be used in any term slot and is interpreted as a request to\n perform a REGEX match (not a string comparison) using the value\n (pre-compiled) for checking rdf:type matches\n ', '__init__': <function REGEXTerm.__init__>, '__reduce__': <function REGEXTerm.__reduce__>, '__dict__': <attribute '__dict__' of 'REGEXTerm' objects>, '__weakref__': <attribute '__weakref__' of 'REGEXTerm' objects>})

__init__(expr)

Initialize self. See help(type(self)) for accurate signature.

__module__ = 'rdflib.plugins.stores.regexmatching'

__reduce__()

Helper for pickle.

__weakref__

list of weak references to the object (if defined)

rdflib.plugins.stores.regexmatching.regexCompareQuad(quad, regexQuad)

rdflib.plugins.stores.sparqlconnector module

class rdflib.plugins.stores.sparqlconnector.SPARQLConnector(query_endpoint=None, update_endpoint=None, returnFormat='xml', method='GET', **kwargs)

Bases: object

this class deals with nitty gritty details of talking to a SPARQL server

__dict__ = mappingproxy({'__module__': 'rdflib.plugins.stores.sparqlconnector', '__doc__': '\n this class deals with nitty gritty details of talking to a SPARQL server\n ', '__init__': <function SPARQLConnector.__init__>, 'session': <property object>, 'method': <property object>, 'query': <function SPARQLConnector.query>, 'update': <function SPARQLConnector.update>, 'close': <function SPARQLConnector.close>, '__dict__': <attribute '__dict__' of 'SPARQLConnector' objects>, '__weakref__': <attribute '__weakref__' of 'SPARQLConnector' objects>})

__init__(query_endpoint=None, update_endpoint=None, returnFormat='xml', method='GET', **kwargs)

Any additional keyword arguments will be passed to requests, and can be used to setup timesouts, basic auth, etc.

__module__ = 'rdflib.plugins.stores.sparqlconnector'

__weakref__

list of weak references to the object (if defined)

close()

property method

query(query, default_graph=None)

property session

update(update, default_graph=None)

exception rdflib.plugins.stores.sparqlconnector.SPARQLConnectorException

Bases: Exception

__module__ = 'rdflib.plugins.stores.sparqlconnector'

__weakref__

list of weak references to the object (if defined)

rdflib.plugins.stores.sparqlstore module

This is an RDFLib store around Ivan Herman et al.’s SPARQL service wrapper. This was first done in layer-cake, and then ported to RDFLib

class rdflib.plugins.stores.sparqlstore.SPARQLStore(endpoint=None, sparql11=True, context_aware=True, node_to_sparql=<function _node_to_sparql>, returnFormat='xml', **sparqlconnector_kwargs)

Bases: rdflib.plugins.stores.sparqlconnector.SPARQLConnector, rdflib.store.Store

An RDFLib store around a SPARQL endpoint

This is context-aware and should work as expected when a context is specified.

For ConjunctiveGraphs, reading is done from the “default graph”. Exactly what this means depends on your endpoint, because SPARQL does not offer a simple way to query the union of all graphs as it would be expected for a ConjuntiveGraph. This is why we recommend using Dataset instead, which is motivated by the SPARQL 1.1.

Fuseki/TDB has a flag for specifying that the default graph is the union of all graphs (tdb:unionDefaultGraph in the Fuseki config).

Warning

By default the SPARQL Store does not support blank-nodes!

As blank-nodes act as variables in SPARQL queries, there is no way to query for a particular blank node without using non-standard SPARQL extensions.

See http://www.w3.org/TR/sparql11-query/#BGPsparqlBNodes

You can make use of such extensions through the node_to_sparql argument. For example if you want to transform BNode(‘0001’) into “<bnode:b0001>”, you can use a function like this:

>>> def my_bnode_ext(node):
...    if isinstance(node, BNode):
...        return '<bnode:b%s>' % node
...    return _node_to_sparql(node)
>>> store = SPARQLStore('http://dbpedia.org/sparql',
...                     node_to_sparql=my_bnode_ext)

You can request a particular result serialization with the returnFormat parameter. This is a string that must have a matching plugin registered. Built in is support for xml, json, csv, tsv and application/rdf+xml.

The underlying SPARQLConnector builds in the requests library. Any extra kwargs passed to the SPARQLStore connector are passed to requests when doing HTTP calls. I.e. you have full control of cookies/auth/headers.

Form example:

>>> store = SPARQLStore('...my endpoint ...', auth=('user','pass'))

will use HTTP basic auth.

__init__(endpoint=None, sparql11=True, context_aware=True, node_to_sparql=<function _node_to_sparql>, returnFormat='xml', **sparqlconnector_kwargs)

__len__(context=None)

Number of statements in the store. This should only account for non- quoted (asserted) statements if the context is not specified, otherwise it should return the number of statements in the formula or context given.

Parameters

context – a graph instance to query or None

__module__ = 'rdflib.plugins.stores.sparqlstore'

add(_, context=None, quoted=False)

Adds the given statement to a specific context or to the model. The quoted argument is interpreted by formula-aware stores to indicate this statement is quoted/hypothetical It should be an error to not specify a context and have the quoted argument be True. It should also be an error for the quoted argument to be True when the store is not formula-aware.

addN(quads)

Adds each item in the list of statements to a specific context. The quoted argument is interpreted by formula-aware stores to indicate this statement is quoted/hypothetical. Note that the default implementation is a redirect to add

add_graph(graph)

Add a graph to the store, no effect if the graph already exists. :param graph: a Graph instance

bind(prefix, namespace)

close(commit_pending_transaction=None)

This closes the database connection. The commit_pending_transaction parameter specifies whether to commit all pending transactions before closing (if the store is transactional).

commit()

contexts(triple=None)

Iterates over results to “SELECT ?NAME { GRAPH ?NAME { ?s ?p ?o } }” or “SELECT ?NAME { GRAPH ?NAME {} }” if triple is None.

Returns instances of this store with the SPARQL wrapper object updated via addNamedGraph(?NAME).

This causes a named-graph-uri key / value pair to be sent over the protocol.

Please note that some SPARQL endpoints are not able to find empty named graphs.

create(configuration)

destroy(configuration)

This destroys the instance of the store identified by the configuration string.

formula_aware = False

graph_aware = True

namespace(prefix)

namespaces()

open(configuration, create=False)

sets the endpoint URL for this SPARQLStore if create==True an exception is thrown.

prefix(namespace)

query(query, initNs={}, initBindings={}, queryGraph=None, DEBUG=False)

If stores provide their own SPARQL implementation, override this.

queryGraph is None, a URIRef or ‘__UNION__’ If None the graph is specified in the query-string/object If URIRef it specifies the graph to query, If ‘__UNION__’ the union of all named graphs should be queried (This is used by ConjunctiveGraphs Values other than None obviously only makes sense for context-aware stores.)

regex_matching = 0

remove(_, context)

Remove the set of triples matching the pattern from the store

remove_graph(graph)

Remove a graph from the store, this shoud also remove all triples in the graph

Parameters

graphid – a Graph instance

rollback()

transaction_aware = False

triples(spo, context=None)

• tuple (s, o, p)

  the triple used as filter for the SPARQL select. (None, None, None) means anything.

• context context

  the graph effectively calling this method.

Returns a tuple of triples executing essentially a SPARQL like SELECT ?subj ?pred ?obj WHERE { ?subj ?pred ?obj }

context may include three parameter to refine the underlying query:

• LIMIT: an integer to limit the number of results

• OFFSET: an integer to enable paging of results

• ORDERBY: an instance of Variable(‘s’), Variable(‘o’) or Variable(‘p’)

or, by default, the first ‘None’ from the given triple

• Using LIMIT or OFFSET automatically include ORDERBY otherwise this is

because the results are retrieved in a not deterministic way (depends on the walking path on the graph) - Using OFFSET without defining LIMIT will discard the first OFFSET - 1 results

`` a_graph.LIMIT = limit a_graph.OFFSET = offset triple_generator = a_graph.triples(mytriple):

#do something

#Removes LIMIT and OFFSET if not required for the next triple() calls del a_graph.LIMIT del a_graph.OFFSET ``

triples_choices(_, context=None)

A variant of triples that can take a list of terms instead of a single term in any slot. Stores can implement this to optimize the response time from the import default ‘fallback’ implementation, which will iterate over each term in the list and dispatch to triples.

class rdflib.plugins.stores.sparqlstore.SPARQLUpdateStore(queryEndpoint=None, update_endpoint=None, sparql11=True, context_aware=True, postAsEncoded=True, autocommit=True, dirty_reads=False, **kwds)

Bases: rdflib.plugins.stores.sparqlstore.SPARQLStore

A store using SPARQL queries for reading and SPARQL Update for changes.

This can be context-aware, if so, any changes will be to the given named graph only.

In favor of the SPARQL 1.1 motivated Dataset, we advise against using this with ConjunctiveGraphs, as it reads and writes from and to the “default graph”. Exactly what this means depends on the endpoint and can result in confusion.

For Graph objects, everything works as expected.

See the SPARQLStore base class for more information.

BLOCK_END = '}'

BLOCK_FINDING_PATTERN = re.compile('(?P<block_start>{)|(?P<block_end>})|(?P<block_content>((\'([^\'\\\\]|\\\\.)*\')|("([^"\\\\]|\\\\.)*")|(\'\'\'((\'|\'\')?([^\'\\\\]|\\\\.))*\'\'\')|("""(("|"")?([^"\\\\]|\\\\.))*"""))|(<([^<>"{}|^`\\])

BLOCK_START = '{'

BlockContent = '((\'([^\'\\\\]|\\\\.)*\')|("([^"\\\\]|\\\\.)*")|(\'\'\'((\'|\'\')?([^\'\\\\]|\\\\.))*\'\'\')|("""(("|"")?([^"\\\\]|\\\\.))*"""))|(<([^<>"{}|^`\\]\\\\\\[\\x00-\\x20])*>)|(#[^\\x0D\\x0A]*([\\x0D\\x0A]|\\Z))|(\\\\.)'

BlockFinding = '(?P<block_start>{)|(?P<block_end>})|(?P<block_content>((\'([^\'\\\\]|\\\\.)*\')|("([^"\\\\]|\\\\.)*")|(\'\'\'((\'|\'\')?([^\'\\\\]|\\\\.))*\'\'\')|("""(("|"")?([^"\\\\]|\\\\.))*"""))|(<([^<>"{}|^`\\]\\\\\\[\\x00-\\x20])*>)|(#[^\\x0D\\x0A]*([\\x0D\\x0A]|\\Z))|(\\\\.))'

COMMENT = '#[^\\x0D\\x0A]*([\\x0D\\x0A]|\\Z)'

ESCAPED = '\\\\.'

IRIREF = '<([^<>"{}|^`\\]\\\\\\[\\x00-\\x20])*>'

STRING_LITERAL1 = "'([^'\\\\]|\\\\.)*'"

STRING_LITERAL2 = '"([^"\\\\]|\\\\.)*"'

STRING_LITERAL_LONG1 = "'''(('|'')?([^'\\\\]|\\\\.))*'''"

STRING_LITERAL_LONG2 = '"""(("|"")?([^"\\\\]|\\\\.))*"""'

String = '(\'([^\'\\\\]|\\\\.)*\')|("([^"\\\\]|\\\\.)*")|(\'\'\'((\'|\'\')?([^\'\\\\]|\\\\.))*\'\'\')|("""(("|"")?([^"\\\\]|\\\\.))*""")'

__init__(queryEndpoint=None, update_endpoint=None, sparql11=True, context_aware=True, postAsEncoded=True, autocommit=True, dirty_reads=False, **kwds)

:param autocommit if set, the store will commit after every writing operations. If False, we only make queries on the server once commit is called.

:param dirty_reads if set, we do not commit before reading. So you cannot read what you wrote before manually calling commit.

__len__(*args, **kwargs)

Number of statements in the store. This should only account for non- quoted (asserted) statements if the context is not specified, otherwise it should return the number of statements in the formula or context given.

Parameters

context – a graph instance to query or None

__module__ = 'rdflib.plugins.stores.sparqlstore'

add(spo, context=None, quoted=False)

Add a triple to the store of triples.

addN(quads)

Add a list of quads to the store.

add_graph(graph)

Add a graph to the store, no effect if the graph already exists. :param graph: a Graph instance

close(commit_pending_transaction=False)

This closes the database connection. The commit_pending_transaction parameter specifies whether to commit all pending transactions before closing (if the store is transactional).

commit()

add(), addN(), and remove() are transactional to reduce overhead of many small edits. Read and update() calls will automatically commit any outstanding edits. This should behave as expected most of the time, except that alternating writes and reads can degenerate to the original call-per-triple situation that originally existed.

contexts(*args, **kwargs)

Iterates over results to “SELECT ?NAME { GRAPH ?NAME { ?s ?p ?o } }” or “SELECT ?NAME { GRAPH ?NAME {} }” if triple is None.

Returns instances of this store with the SPARQL wrapper object updated via addNamedGraph(?NAME).

This causes a named-graph-uri key / value pair to be sent over the protocol.

Please note that some SPARQL endpoints are not able to find empty named graphs.

open(configuration, create=False)

sets the endpoint URLs for this SPARQLStore :param configuration: either a tuple of (queryEndpoint, update_endpoint),

or a string with the query endpoint

Parameters

create – if True an exception is thrown.

query(*args, **kwargs)

If stores provide their own SPARQL implementation, override this.

queryGraph is None, a URIRef or ‘__UNION__’ If None the graph is specified in the query-string/object If URIRef it specifies the graph to query, If ‘__UNION__’ the union of all named graphs should be queried (This is used by ConjunctiveGraphs Values other than None obviously only makes sense for context-aware stores.)

remove(spo, context)

Remove a triple from the store

remove_graph(graph)

Remove a graph from the store, this shoud also remove all triples in the graph

Parameters

graphid – a Graph instance

rollback()

setTimeout(timeout)

triples(*args, **kwargs)

• tuple (s, o, p)

  the triple used as filter for the SPARQL select. (None, None, None) means anything.

• context context

  the graph effectively calling this method.

Returns a tuple of triples executing essentially a SPARQL like SELECT ?subj ?pred ?obj WHERE { ?subj ?pred ?obj }

context may include three parameter to refine the underlying query:

• LIMIT: an integer to limit the number of results

• OFFSET: an integer to enable paging of results

• ORDERBY: an instance of Variable(‘s’), Variable(‘o’) or Variable(‘p’)

or, by default, the first ‘None’ from the given triple

• Using LIMIT or OFFSET automatically include ORDERBY otherwise this is

because the results are retrieved in a not deterministic way (depends on the walking path on the graph) - Using OFFSET without defining LIMIT will discard the first OFFSET - 1 results

`` a_graph.LIMIT = limit a_graph.OFFSET = offset triple_generator = a_graph.triples(mytriple):

#do something

#Removes LIMIT and OFFSET if not required for the next triple() calls del a_graph.LIMIT del a_graph.OFFSET ``

update(query, initNs={}, initBindings={}, queryGraph=None, DEBUG=False)

Perform a SPARQL Update Query against the endpoint, INSERT, LOAD, DELETE etc. Setting initNs adds PREFIX declarations to the beginning of the update. Setting initBindings adds inline VALUEs to the beginning of every WHERE clause. By the SPARQL grammar, all operations that support variables (namely INSERT and DELETE) require a WHERE clause. Important: initBindings fails if the update contains the substring ‘WHERE {’ which does not denote a WHERE clause, e.g. if it is part of a literal.

Context-aware query rewriting

• When: If context-awareness is enabled and the graph is not the default graph of the store.

• Why: To ensure consistency with the IOMemory store. The graph must except “local” SPARQL requests (requests with no GRAPH keyword) like if it was the default graph.

• What is done: These “local” queries are rewritten by this store. The content of each block of a SPARQL Update operation is wrapped in a GRAPH block except if the block is empty. This basically causes INSERT, INSERT DATA, DELETE, DELETE DATA and WHERE to operate only on the context.

• Example: “INSERT DATA { <urn:michel> <urn:likes> <urn:pizza> }” is converted into “INSERT DATA { GRAPH <urn:graph> { <urn:michel> <urn:likes> <urn:pizza> } }”.

• Warning: Queries are presumed to be “local” but this assumption is not checked. For instance, if the query already contains GRAPH blocks, the latter will be wrapped in new GRAPH blocks.

• Warning: A simplified grammar is used that should tolerate extensions of the SPARQL grammar. Still, the process may fail in uncommon situations and produce invalid output.

where_pattern = re.compile('(?P<where>WHERE\\s*\\{)', re.IGNORECASE)

Module contents

This package contains modules for additional RDFLib stores