src.dackar.RCA.doc_extraction.store =================================== .. py:module:: src.dackar.RCA.doc_extraction.store Attributes ---------- .. autoapisummary:: src.dackar.RCA.doc_extraction.store.logger src.dackar.RCA.doc_extraction.store._ID_SEP Exceptions ---------- .. autoapisummary:: src.dackar.RCA.doc_extraction.store.DocExtractionStoreError src.dackar.RCA.doc_extraction.store.EmbeddingModelVersionError Classes ------- .. autoapisummary:: src.dackar.RCA.doc_extraction.store.SemanticMatch src.dackar.RCA.doc_extraction.store.DocExtractionStore Functions --------- .. autoapisummary:: src.dackar.RCA.doc_extraction.store._make_record_id src.dackar.RCA.doc_extraction.store._cosine_similarity src.dackar.RCA.doc_extraction.store._build_where_clause src.dackar.RCA.doc_extraction.store._meta_to_semantic_match Module Contents --------------- .. py:data:: logger .. py:data:: _ID_SEP :value: '::chain::' .. py:function:: _make_record_id(doc_id, chain_index) .. py:function:: _cosine_similarity(a, b) .. py:class:: SemanticMatch One deduplicated result from a DocExtractionStore.query() call. .. py:attribute:: record_id :type: str .. py:attribute:: doc_id :type: str .. py:attribute:: chain_index :type: int .. py:attribute:: identified_effect :type: Optional[str] .. py:attribute:: assessed_cause :type: Optional[str] .. py:attribute:: inferred_fm_label :type: Optional[str] .. py:attribute:: fm_id_candidate :type: Optional[str] .. py:attribute:: confidence :type: src.dackar.RCA.doc_extraction.schema.ConfidenceLevel .. py:attribute:: cause_is_symptom :type: bool .. py:attribute:: similarity_score :type: float .. py:attribute:: fm_resolution_status :type: Optional[str] :value: None .. py:attribute:: doc_type :type: str :value: '' .. py:attribute:: finding_status :type: Optional[str] :value: None .. py:attribute:: authority_level :type: Optional[str] :value: None .. py:attribute:: epistemic_class :type: Optional[str] :value: None .. py:attribute:: classification_resolution_level :type: Optional[str] :value: None .. py:attribute:: degraded_classification :type: bool :value: False .. py:property:: confidence_weight :type: float Numeric weight for this match's confidence level (HIGH=1.0, MEDIUM=0.7, LOW=0.3). Used as a multiplier in semantic_contribution (§4.3) so lower-confidence extractions contribute proportionally less to effective_recurrence_count. .. py:property:: cause_is_symptom_factor :type: float Down-weight factor when the assessed cause is itself a symptom (0.5) vs. a mechanism (1.0). A symptom-as-cause is a weaker recurrence signal than a true failure mechanism, so it halves this match's semantic_contribution. .. py:property:: semantic_contribution :type: float Fractional recurrence contribution for effective_recurrence_count (§4.3). .. py:exception:: DocExtractionStoreError Bases: :py:obj:`RuntimeError` Base error for DocExtractionStore backend failures (Chroma / embedding backend). query() and resolve_fm_candidates() deliberately degrade to empty/zero results on backend failure rather than raising, recording each event via _record_degradation() so it surfaces in the run manifest (see store_health_summary()). Callers that prefer fail-loud semantics can inspect store_health_summary()["degraded"] and escalate. .. py:exception:: EmbeddingModelVersionError Bases: :py:obj:`DocExtractionStoreError` Raised when the query-time embedding model does not match the collection's stored model. .. py:class:: DocExtractionStore(persist_directory, embed_model = 'nomic-embed-text-v1.5', ollama_base_url = None, fm_resolution_threshold = 0.88, epistemics_classifier = None) Chroma-backed store for DocExtractionRecord objects. One collection (``"doc_extractions"``) stores all extraction records across all document types. Each record's embed_text (§4.1) is computed and stored at upsert time; similarity queries run against these pre-computed vectors. Key guarantees: - Embedding model version is written into every record's metadata. A mismatch between query-time model and stored model raises EmbeddingModelVersionError. - fm_id_candidate is always null at ingestion; resolve_fm_candidates() writes it back in batch at the start of an RCA run (§3.3 Step C). - query() deduplicates by doc_id, returning only the best-scoring chain per document. .. py:attribute:: COLLECTION_NAME :value: 'doc_extractions' .. py:attribute:: _HNSW_SPACE :value: 'cosine' .. py:attribute:: persist_directory .. py:attribute:: embed_model :value: 'nomic-embed-text-v1.5' .. py:attribute:: ollama_base_url .. py:attribute:: fm_resolution_threshold :value: 0.88 .. py:attribute:: epistemics_classifier :value: None .. py:attribute:: _collection :value: None .. py:attribute:: _embedder :value: None .. py:attribute:: _degraded_operations :type: List[Dict[str, Any]] :value: [] .. py:property:: embedding_model_version :type: str .. py:method:: _get_embedder() .. py:method:: _get_collection() .. py:method:: _embed_texts(texts) .. py:method:: _embed_query(text) .. py:method:: _chroma_collection() .. py:method:: upsert(record) Embed and store one extraction record. Returns the Chroma record_id. Records with no embed_text are stored with a single-space document to avoid Chroma rejecting empty strings; they are retrievable by metadata but will not surface in similarity queries. .. py:method:: upsert_batch(records) Embed and store multiple extraction records in one batch call. .. py:method:: query(query_text, *, top_k = 5, similarity_threshold = 0.75, near_match_window = 0.1, filter_meta = None, exact_doc_ids = None) Query for semantically similar extraction records. The embedding model used at query time must match the collection's stored ``embedding_model_version``. A mismatch raises EmbeddingModelVersionError. Deduplication: only the highest-scoring chain per doc_id is returned. :param query_text: The query string (e.g. ``fm.name | fm.expected_symptoms | event.symptom_description``). :param top_k: Maximum number of doc_id-deduplicated results to return. :param similarity_threshold: Minimum cosine similarity for inclusion in the main result set. :param near_match_window: Width of the soft zone below threshold that populates near_matches. :param filter_meta: Optional Chroma metadata pre-filter (e.g. ``{"doc_type": "CR"}``). :param exact_doc_ids: Set of doc_ids already counted via exact-match recurrence (kg_context.past_events). Matching records are excluded from both matches and near_matches to prevent double-counting. None or empty set disables the guard. :returns: - matches: similarity >= similarity_threshold, deduplicated, top_k max, exact_doc_ids excluded - near_matches: similarity in [similarity_threshold - near_match_window, similarity_threshold) :rtype: (matches, near_matches) where .. py:method:: _assert_model_version() Raise EmbeddingModelVersionError if stored records use a different model. .. py:method:: resolve_fm_candidates(fm_list, *, resolution_threshold = None) Resolve fm_id_candidate for unresolved extraction records. Called once per RCA run before Step 3 / Step 2d. For each record with fm_id_candidate == "" (unresolved), embeds the stored inferred_fm_label and compares against the KG FM list for the current asset neighborhood. Writes fm_id_candidate (and fm_id_candidate_alt) back to the collection if cosine similarity >= resolution_threshold. :param fm_list: List of (fm_id, fm_label) tuples from the KG. :param resolution_threshold: Override default; defaults to self.fm_resolution_threshold. :returns: Number of records updated. .. py:method:: count() Return total number of extraction records in the collection. .. py:method:: delete_by_doc_id(doc_id) Delete all extraction records for a given source document (re-ingestion path). .. py:method:: _record_degradation(operation, exc) Record a degraded (error-swallowed) operation for run-manifest surfacing. query() and resolve_fm_candidates() degrade to empty/zero results on backend failure rather than crashing a batch run; each such event is captured here so store_health_summary() can report it and the failure is not misread downstream as a legitimate "no semantic recurrence". .. py:property:: degraded :type: bool True when any query/resolve operation swallowed a backend error this run. .. py:method:: store_health_summary() Degradation summary for run_manifest (store_health section). Returns ``{"degraded": bool, "degraded_operation_count": int, "events": [...]}``. The orchestrator stamps this onto the run manifest so a swallowed Chroma/embedding error surfaces as an explicit degraded-run signal instead of a silent under-count. .. py:function:: _build_where_clause(filter_meta) .. py:function:: _meta_to_semantic_match(record_id, similarity, meta)