VectorStore/Q&A, quickstart¶

NOTE: this uses Cassandra's "Vector Similarity Search" capability. Make sure you are connecting to a vector-enabled database for this demo.

In [1]:

from langchain.indexes import VectorstoreIndexCreator
from langchain.text_splitter import (
    CharacterTextSplitter,
    RecursiveCharacterTextSplitter,
)
from langchain.docstore.document import Document
from langchain.document_loaders import TextLoader

from langchain.indexes import VectorstoreIndexCreator from langchain.text_splitter import ( CharacterTextSplitter, RecursiveCharacterTextSplitter, ) from langchain.docstore.document import Document from langchain.document_loaders import TextLoader

The following line imports the Cassandra flavor of a LangChain vector store:

In [2]:

from langchain.vectorstores.cassandra import Cassandra

from langchain.vectorstores.cassandra import Cassandra

As usual, a database connection is needed to access Cassandra. The following assumes that a vector-search-capable Astra DB instance is available. Adjust as needed.

In [3]:

from cqlsession import getCQLSession, getCQLKeyspace
cqlMode = 'astra_db' # 'astra_db'/'local'
session = getCQLSession(mode=cqlMode)
keyspace = getCQLKeyspace(mode=cqlMode)

from cqlsession import getCQLSession, getCQLKeyspace cqlMode = 'astra_db' # 'astra_db'/'local' session = getCQLSession(mode=cqlMode) keyspace = getCQLKeyspace(mode=cqlMode)

Both an LLM and an embedding function are required.

Below is the logic to instantiate the LLM and embeddings of choice. We choose to leave it in the notebooks for clarity.

In [6]:

from llm_choice import suggestLLMProvider

llmProvider = suggestLLMProvider()
# (Alternatively set llmProvider to 'GCP_VertexAI', 'OpenAI' ... manually if you have credentials)

if llmProvider == 'GCP_VertexAI':
    from langchain.llms import VertexAI
    from langchain.embeddings import VertexAIEmbeddings
    llm = VertexAI()
    myEmbedding = VertexAIEmbeddings()
    print('LLM+embeddings from VertexAI')
elif llmProvider == 'OpenAI':
    from langchain.llms import OpenAI
    from langchain.embeddings import OpenAIEmbeddings
    llm = OpenAI(temperature=0)
    myEmbedding = OpenAIEmbeddings()
    print('LLM+embeddings from OpenAI')
else:
    raise ValueError('Unknown LLM provider.')

from llm_choice import suggestLLMProvider llmProvider = suggestLLMProvider() # (Alternatively set llmProvider to 'GCP_VertexAI', 'OpenAI' ... manually if you have credentials) if llmProvider == 'GCP_VertexAI': from langchain.llms import VertexAI from langchain.embeddings import VertexAIEmbeddings llm = VertexAI() myEmbedding = VertexAIEmbeddings() print('LLM+embeddings from VertexAI') elif llmProvider == 'OpenAI': from langchain.llms import OpenAI from langchain.embeddings import OpenAIEmbeddings llm = OpenAI(temperature=0) myEmbedding = OpenAIEmbeddings() print('LLM+embeddings from OpenAI') else: raise ValueError('Unknown LLM provider.')

LLM+embeddings from VertexAI

A minimal example¶

The following is a minimal usage of the Cassandra vector store. The store is created and filled at once, and is then queried to retrieve relevant parts of the indexed text, which are then stuffed into a prompt finally used to answer a question.

The following creates an "index creator", which knows about the type of vector store, the embedding to use and how to preprocess the input text:

(Note: stores built with different embedding functions will need different tables. This is why we append the llmProvider name to the table name in the next cell.)

In [7]:

table_name = 'vs_test1_' + llmProvider

index_creator = VectorstoreIndexCreator(
    vectorstore_cls=Cassandra,
    embedding=myEmbedding,
    text_splitter=CharacterTextSplitter(
        chunk_size=400,
        chunk_overlap=0,
    ),
    vectorstore_kwargs={
        'session': session,
        'keyspace': keyspace,
        'table_name': table_name,
    },
)

table_name = 'vs_test1_' + llmProvider index_creator = VectorstoreIndexCreator( vectorstore_cls=Cassandra, embedding=myEmbedding, text_splitter=CharacterTextSplitter( chunk_size=400, chunk_overlap=0, ), vectorstore_kwargs={ 'session': session, 'keyspace': keyspace, 'table_name': table_name, }, )

Loading a local text (a short story by E. A. Poe will do)

In [8]:

loader = TextLoader('texts/amontillado.txt', encoding='utf8')

loader = TextLoader('texts/amontillado.txt', encoding='utf8')

This takes a few seconds to run, as it must calculate embedding vectors for a number of chunks of the input text:

In [9]:

# Note: this is a temporary replacement for next cell, pending resolution of this error from VertexAI:
#    "InvalidArgument: 400 5 instance(s) is allowed per prediction. Actual: <N>"
# (i.e. one cannot request embedding of more than 5 strings in a single call)
if llmProvider == 'GCP_VertexAI':
    from langchain.indexes.vectorstore import VectorStoreIndexWrapper
    docs = loader.load()
    subdocs = index_creator.text_splitter.split_documents(docs)
    #
    print(f'subdocument {0} ...', end=' ')
    vs = index_creator.vectorstore_cls.from_documents(
        subdocs[:1],
        index_creator.embedding,
        **index_creator.vectorstore_kwargs,
    )
    print('done.')
    for sdi, sd in enumerate(subdocs[1:]):
        print(f'subdocument {sdi+1} ...', end=' ')
        vs.add_texts(texts=[sd.page_content], metadata=[sd.metadata])
        print('done.')
    #
    index = VectorStoreIndexWrapper(vectorstore=vs)

# Note: this is a temporary replacement for next cell, pending resolution of this error from VertexAI: # "InvalidArgument: 400 5 instance(s) is allowed per prediction. Actual: " # (i.e. one cannot request embedding of more than 5 strings in a single call) if llmProvider == 'GCP_VertexAI': from langchain.indexes.vectorstore import VectorStoreIndexWrapper docs = loader.load() subdocs = index_creator.text_splitter.split_documents(docs) # print(f'subdocument {0} ...', end=' ') vs = index_creator.vectorstore_cls.from_documents( subdocs[:1], index_creator.embedding, **index_creator.vectorstore_kwargs, ) print('done.') for sdi, sd in enumerate(subdocs[1:]): print(f'subdocument {sdi+1} ...', end=' ') vs.add_texts(texts=[sd.page_content], metadata=[sd.metadata]) print('done.') # index = VectorStoreIndexWrapper(vectorstore=vs)

subdocument 0 ... done.
subdocument 1 ... done.
subdocument 2 ... done.
subdocument 3 ... done.
subdocument 4 ... done.
subdocument 5 ... done.
subdocument 6 ... done.
subdocument 7 ... done.
subdocument 8 ... done.
subdocument 9 ... done.
subdocument 10 ... done.
subdocument 11 ... done.
subdocument 12 ... done.
subdocument 13 ... done.
subdocument 14 ... done.
subdocument 15 ... done.
subdocument 16 ... done.
subdocument 17 ... done.
subdocument 18 ... done.
subdocument 19 ... done.
subdocument 20 ... done.
subdocument 21 ... done.
subdocument 22 ... done.
subdocument 23 ... done.
subdocument 24 ... done.
subdocument 25 ... done.
subdocument 26 ... done.
subdocument 27 ... done.
subdocument 28 ... done.
subdocument 29 ... done.
subdocument 30 ... done.
subdocument 31 ... done.
subdocument 32 ... done.

In [10]:

if llmProvider != 'GCP_VertexAI':
    index = index_creator.from_loaders([loader])

if llmProvider != 'GCP_VertexAI': index = index_creator.from_loaders([loader])

Check what's on DB¶

By way of demonstration, if you were to directly read the rows stored in your database table, this is what you would now find there (not that you'll ever have to, for LangChain and CassIO provide an abstraction on top of that):

In [11]:

cqlSelect = f'SELECT * FROM {keyspace}.{table_name} LIMIT 3;'  # (Not a production-optimized query ...)
rows = session.execute(cqlSelect)
for row_i, row in enumerate(rows):
    print(f'\nRow {row_i}:')
    print(f'    document_id:      {row.document_id}')
    print(f'    embedding_vector: {str(row.embedding_vector)[:64]} ...')
    print(f'    document:         {row.document[:64]} ...')
    print(f'    metadata_blob:    {row.metadata_blob}')

print('\n...')

cqlSelect = f'SELECT * FROM {keyspace}.{table_name} LIMIT 3;' # (Not a production-optimized query ...) rows = session.execute(cqlSelect) for row_i, row in enumerate(rows): print(f'\nRow {row_i}:') print(f' document_id: {row.document_id}') print(f' embedding_vector: {str(row.embedding_vector)[:64]} ...') print(f' document: {row.document[:64]} ...') print(f' metadata_blob: {row.metadata_blob}') print('\n...')

Row 0:
    document_id:      21fbd9985564f7f12ac51f4c20232d75
    embedding_vector: [-0.011631837114691734, -0.01850522682070732, 0.0115448292344808 ...
    document:         "Pass your hand," I said, "over the wall; you cannot help feelin ...
    metadata_blob:    {}

Row 1:
    document_id:      f5020721820969b3fbf6b12691818508
    embedding_vector: [0.011497375555336475, -0.007067505735903978, -0.007191576063632 ...
    document:         No answer still.  I thrust a torch through the remaining apertur ...
    metadata_blob:    {}

Row 2:
    document_id:      d2ff9ab96b181d455481c67f84558091
    embedding_vector: [-0.005730615463107824, -0.002260348992422223, 0.049317024648189 ...
    document:         I said to him--"My dear Fortunato, you are luckily met.  How rem ...
    metadata_blob:    {}

...

Ask a question, get an answer¶

In [12]:

query = "Who is Luchesi?"
index.query(query, llm=llm)

query = "Who is Luchesi?" index.query(query, llm=llm)

Out[12]:

'Luchesi is a wine critic.'

Spawning a "retriever" from the index¶

In [13]:

retriever = index.vectorstore.as_retriever(search_kwargs={
    'k': 2,
})

retriever = index.vectorstore.as_retriever(search_kwargs={ 'k': 2, })

In [14]:

retriever.get_relevant_documents(
    "Check the motto of the Montresors"
)

retriever.get_relevant_documents( "Check the motto of the Montresors" )

Out[14]:

[Document(page_content='"Yes," I said, "let us be gone."\n\n"_For the love of God, Montresor!_"\n\n"Yes," I said, "for the love of God!"\n\nBut to these words I hearkened in vain for a reply.  I grew impatient.\nI called aloud--\n\n"Fortunato!"\n\nNo answer.  I called again--\n\n"Fortunato--"', metadata={}),
 Document(page_content='"You are not of the masons."\n\n"Yes, yes," I said; "yes, yes."\n\n"You?  Impossible!  A mason?"\n\n"A mason," I replied.\n\n"A sign," he said, "a sign."\n\n"It is this," I answered, producing a trowel from beneath the folds of\nmy _roquelaire_.\n\n"You jest," he exclaimed, recoiling a few paces.  "But let us proceed\nto the Amontillado."', metadata={})]