aQbjzHdZddlZddlmZmZmZmZmZmZddl Z ddl m Z ddl mZddlmZddlmZddlmZdd lmZmZmZmZmZmZmZmZmZmZdd l m!Z!d Z"d Z#ed Z$edZ%edZ&edZ'edZ(dZ)GddeZ*y)zMySQL-backed vector store for embeddings and semantic document retrieval. Provides a VectorStore implementation persisting documents, metadata, and embeddings in MySQL, plus similarity search utilities. N)AnyIterableListOptionalSequenceUnion)Document) Embeddings) VectorStore) PrivateAttr) MyEmbeddings) VAR_NAME_SPACEatomic_transactiondelete_sql_table execute_sqlextend_sql_tableformat_value_sqlget_random_nameis_table_empty source_schema table_exists)MySQLConnectionAbstractz Hello world!external_sourcez .embeddingz.contextz .context_mapz.retrieval_infoz.optionsinternal_ai_id_c (eZdZUdZeZeed<eZe ed<eZ e ed<eZ e e ed<eZeed<eZeed< d#d ed e e d dffd Zd ed ee fdZd$dZd#de ee ded dfdZd$dZ d%dee de eede ee ded ee f dZe d%dee d e de eed ed ef dZ d#dee dee d ee fdZ! d&de deded ee fdZ"d'dZ#de$e%dfd e$e&dfd!e$e'dfd dfd"Z(xZ)S)( MyVectorStorea MySQL-backed vector store for handling embeddings and semantic document retrieval. Supports adding, deleting, and searching high-dimensional vector representations of documents using efficient storage and HeatWave ML similarity search procedures. Supports use as a context manager: when used in a `with` statement, all backing tables/data are deleted automatically when the block exits (even on exception). Attributes: db_connection (MySQLConnectionAbstract): Active MySQL database connection. embedder (Embeddings): Embeddings generator for computing vector representations. schema_name (str): SQL schema for table storage. table_name (Optional[str]): Name of the active table backing the store (or None until created). embedding_dimension (int): Size of embedding vectors stored. next_id (int): Internal counter for unique document ID generation. _db_connection _embedder _schema_name _table_name_embedding_dimension_next_idN db_connectionembedderreturnct|d|_t||_|xs t ||_||_d|_t|j jt|_ y)a Initialize a MyVectorStore with a database connection and embedding generator. Args: db_connection: MySQL database connection for all vector operations. embedder: Embeddings generator used for creating and querying embeddings. Raises: ValueError: If the schema name is not valid DatabaseError: If a database connection issue occurs. If an operational error occurs during execution. rN) super__init__r"rrr rrr len embed_queryBASIC_EMBEDDING_QUERYr!)selfr#r$ __class__s N/var/www/html/venv/lib/python3.12/site-packages/mysql/ai/genai/vector_store.pyr(zMyVectorStore.__init__dsg$  )-8!@\-%@+*.%( NN & &'< =% !num_idsct|j|j|zDcgc]}d| }}|xj|z c_|Scc}w)z Generate a batch of unique internal document IDs for vector storage. Args: num_ids: Number of IDs to create. Returns: List of sequentially numbered internal string IDs. r)ranger")r,r0iidss r._get_idszMyVectorStore._get_idssT,1 PW@W+X &'oaS !      s A cjatj5tfd}djd|d}t |j f|_dddyy#1swYyxYw)a Create a backing SQL table for storing vectors if not already created. Returns: None Raises: DatabaseError: If a database connection issue occurs. If an operational error occurs during execution. Notes: The table name is randomized to avoid collisions. Schema includes content, metadata, and embedding vector. Nc4tj| SN)rr) table_namecursorr,s r.z2MyVectorStore._make_vector_store..s< 1 1:,(r/z CREATE TABLE .a  ( `id` VARCHAR(128) NOT NULL, `content` TEXT, `metadata` JSON DEFAULT NULL, `embed` vector(%s), PRIMARY KEY (`id`) ) ENGINE=InnoDB; params)r rrrrrr!)r,r9create_table_stmtr:s` @r._make_vector_storez MyVectorStore._make_vector_stores    ##D$7$78 .F, )"//0*>%!-t7P7P6R$. ) . . $ . .s AA11A:r4_c t|j5}|r1|D],}t|d|jd|jd|f.t ||j|jr|j dddy#1swYyxYw)a# Delete documents by ID. Optionally deletes the vector table if empty after deletions. Args: ids: Optional sequence of document IDs to delete. If None, no action is taken. Returns: None Raises: DatabaseError: If a database connection issue occurs. If an operational error occurs during execution. Notes: If the backing table is empty after deletions, the table is dropped and table_name is set to None. z DELETE FROM r< WHERE id = %sr=N)rrrrr r delete_all)r,r4rAr:_ids r.deletezMyVectorStore.deletes&  3 3 4 "C&t'8'8&94;K;K:LN[ #vfd&7&79I9IJ! " " "s A%BB c|jGt|j5}t||j|jd|_dddyy#1swYyxYw)zc Delete and drop the entire vector store table. Returns: None N)r rrrr)r,r:s r.rDzMyVectorStore.delete_alls_    '#D$7$78 (F ):):DA   # # %I ;--I/C/893##99..009 \\^49 7 2;<3#,,<:  3 3 4  !!  079O/P    %:=  sD7D<'EE querykkwargsc |jgS|jj|}t|j5}t |dt dt|g|jdd}|jdd|jd d |jd d d }t|\}} dt d|jd|jd|d|dtdtdtd} t || |g| t |dtg} tj|j!d } | D]~} t |d|jd|jd| df|j!\}}}||d}|tj||d<t#di|}| j%|| cdddS#1swYyxYw)a Search for and return the most similar documents in the store to the given query. Args: query: String query to embed and use for similarity search. k: Number of top documents to return. kwargs: options to pass to ML_SIMILARITY_SEARCH. Currently supports distance_metric, max_distance, percentage_distance, and segment_overlap Returns: List of Document objects, ordered from most to least similar. Raises: DatabaseError: If provided kwargs are invalid or unsupported. If a database connection issue occurs. If an operational error occurs during execution. Implementation Notes: - Calls ML similarity search within MySQL using stored procedures. - Retrieves IDs, content, and metadata for search matches. - Parsing and retrieval for context results are handled via intermediate JSONs. NzSET @z = string_to_vector(%s)r=distance_metricCOSINE max_distanceg333333?percentage_distanceg4@segment_overlapr)rkrlrmz= CALL sys.ML_SIMILARITY_SEARCH( @z3, JSON_ARRAY( 'r<z' ), JSON_OBJECT( "segment", "content", "segment_embedding", "embed", "document_name", "id" ), zR, %s, NULL, NULL, z, @z ) zSELECT @z"SELECT id, content, metadata FROM rC document_name)rZrKrL)r rr*rrr VAR_EMBEDDINGstrgetrr VAR_CONTEXTVAR_CONTEXT_MAPVAR_RETRIEVAL_INFOjsonloadsfetchoner append)r,rerfrg embeddingr:riretrieval_optionsretrieval_options_placeholderretrieval_options_paramssimilarity_search_queryresults context_mapscontextdoc_idr[rLdoc_argsrbs r.similarity_searchzMyVectorStore.similarity_searchgs?:    #INN..u5  3 3 4H   &=>I'  %jj):HEO & >3 ?'-zz2G'N#)::.?#C!  GW!G C )+C+!''($*:*:);<//0!"#$%& %' #, ''C*BC  (?*; < =G::foo&7&:;L' $ $ 1 12!D4D4D3EF(($O46-3OO,=)!$+'+/::h+?HZ(**s#) $,QH H H s FGGc|S)a Enter the runtime context related to this vector store instance. Returns: The current MyVectorStore object, allowing use within a `with` statement block. Usage Notes: - Intended for use in a `with` statement to ensure automatic cleanup of resources. - No special initialization occurs during context entry, but enables proper context-managed lifecycle. Example: with MyVectorStore(db_connection, embedder) as vectorstore: vectorstore.add_texts([...]) # Vector store is active within this block. # All storage and resources are now cleaned up. ro)r,s r. __enter__zMyVectorStore.__enter__s & r/exc_typeexc_valexc_tbc$|jy)a Exit the runtime context for the vector store, ensuring all storage resources are cleaned up. Args: exc_type: The exception type, if any exception occurred in the context block. exc_val: The exception value, if any exception occurred in the context block. exc_tb: The traceback object, if any exception occurred in the context block. Returns: None: Indicates that exceptions are never suppressed; they will propagate as normal. Implementation Notes: - Automatically deletes all vector store data and backing tables via `delete_all()` upon exiting the context. - This cleanup occurs whether the block exits normally or due to an exception. - Does not suppress exceptions; errors in the context block will continue to propagate. - Use when the vector store lifecycle is intended to be temporary or scoped. Example: with MyVectorStore(db_connection, embedder) as vectorstore: vectorstore.add_texts([...]) # Vector store is active within this block. # All storage and resources are now cleaned up. N)rD)r,rrrs r.__exit__zMyVectorStore.__exit__s> r/r8)r%N)NN))r%r )*__name__ __module__ __qualname____doc__r rr__annotations__rr rrqr rr!intr"r(rMr5r@rrrFrDrdictrrS classmethodr rXr rOrrrtype BaseExceptionobjectr __classcell__)r-s@r.rrIs0&/:mN+;'MIz)# L#%!,K#. + #-MHc! *. . :&    @S  %.N"(8C=1"s"t"> (+/#' 6} 6DJ' 6d3i 6  6 c 6D +/15 #}##DJ' # / #  ##L;?7h7.23i7 c7xjj j j h jX*d #}d*+fdl#   r/r)+rrvtypingrrrrrrpandasr_langchain_core.documentsr langchain_core.embeddingsr langchain_core.vectorstoresr pydanticr mysql.ai.genai.embeddingr mysql.ai.utilsrrrrrrrrrrmysql.connector.abstractsrr+EMBEDDING_SOURCErprsrtru VAR_OPTIONSID_SPACError/r.rs:  AA-03 1   >&$!"*-  ) #$L1&'7 ) ~K~r/