src.dackar.RCA.ner.causal_condition_adapter =========================================== .. py:module:: src.dackar.RCA.ner.causal_condition_adapter Attributes ---------- .. autoapisummary:: src.dackar.RCA.ner.causal_condition_adapter.logger src.dackar.RCA.ner.causal_condition_adapter._ALLOWED_CONDITION_STATES src.dackar.RCA.ner.causal_condition_adapter._DATA_DIR src.dackar.RCA.ner.causal_condition_adapter._SPEECH_ACT_VERBS src.dackar.RCA.ner.causal_condition_adapter._DEP_CAUSAL_VERB_LEMMAS_FALLBACK src.dackar.RCA.ner.causal_condition_adapter._DEP_CAUSAL_VERB_LEMMAS src.dackar.RCA.ner.causal_condition_adapter._CONJECTURE_PAT src.dackar.RCA.ner.causal_condition_adapter._HEALTH_FAILED_FRAGMENTS src.dackar.RCA.ner.causal_condition_adapter._HEALTH_CONDITION_FALLBACK src.dackar.RCA.ner.causal_condition_adapter._HEALTH_CONDITION_TERMS src.dackar.RCA.ner.causal_condition_adapter._HEALTH_FAILED_ROOTS src.dackar.RCA.ner.causal_condition_adapter._HEALTH_DEGRADED_ROOTS src.dackar.RCA.ner.causal_condition_adapter._DEMONSTRATIVES src.dackar.RCA.ner.causal_condition_adapter._CONJ_BACKWARD_PAT src.dackar.RCA.ner.causal_condition_adapter._TEMPORAL_SINCE_PAT src.dackar.RCA.ner.causal_condition_adapter._FORWARD_PREP_CONNECTORS src.dackar.RCA.ner.causal_condition_adapter._PREP_CONNECTOR_PAT src.dackar.RCA.ner.causal_condition_adapter._NEGATION_PAT src.dackar.RCA.ner.causal_condition_adapter._CHAIN_JACCARD_THRESHOLD src.dackar.RCA.ner.causal_condition_adapter._CHAIN_EMBED_THRESHOLD src.dackar.RCA.ner.causal_condition_adapter._PARTICIPIAL_DEPS src.dackar.RCA.ner.causal_condition_adapter._CLAUSE_DEPS_NP src.dackar.RCA.ner.causal_condition_adapter.STAGE5_OUTPUT_SCHEMA Functions --------- .. autoapisummary:: src.dackar.RCA.ner.causal_condition_adapter._load_causal_verb_lemmas src.dackar.RCA.ner.causal_condition_adapter._build_conjecture_pattern src.dackar.RCA.ner.causal_condition_adapter._load_health_condition_terms src.dackar.RCA.ner.causal_condition_adapter._build_embed_fn src.dackar.RCA.ner.causal_condition_adapter.calibrate_chain_threshold src.dackar.RCA.ner.causal_condition_adapter._chain_causal_statements src.dackar.RCA.ner.causal_condition_adapter.empty_stage5_output src.dackar.RCA.ner.causal_condition_adapter._call_llm_json src.dackar.RCA.ner.causal_condition_adapter._llm_causal_fallback src.dackar.RCA.ner.causal_condition_adapter._llm_condition_state_fallback src.dackar.RCA.ner.causal_condition_adapter._format_few_shot_block src.dackar.RCA.ner.causal_condition_adapter._llm_extract_all_relations src.dackar.RCA.ner.causal_condition_adapter._merge_llm_statements src.dackar.RCA.ner.causal_condition_adapter._maybe_trigger_llm_on_weak_fallback src.dackar.RCA.ner.causal_condition_adapter._apply_llm_extract_all src.dackar.RCA.ner.causal_condition_adapter.extract_stage5_causal_condition src.dackar.RCA.ner.causal_condition_adapter._instantiate_extractor src.dackar.RCA.ner.causal_condition_adapter._run_extractor_if_needed src.dackar.RCA.ner.causal_condition_adapter._raise_ctor_skip src.dackar.RCA.ner.causal_condition_adapter._normalize_from_causal_sentence src.dackar.RCA.ner.causal_condition_adapter._normalize_from_causal_simple src.dackar.RCA.ner.causal_condition_adapter._extract_native_stage5 src.dackar.RCA.ner.causal_condition_adapter._extract_causal_rows src.dackar.RCA.ner.causal_condition_adapter._tuple_to_causal_dict src.dackar.RCA.ner.causal_condition_adapter._extract_status_mentions src.dackar.RCA.ner.causal_condition_adapter._safe_text src.dackar.RCA.ner.causal_condition_adapter._pick_first src.dackar.RCA.ner.causal_condition_adapter._build_causal_statement src.dackar.RCA.ner.causal_condition_adapter._score_causal_statement src.dackar.RCA.ner.causal_condition_adapter._dep_causal_fallback src.dackar.RCA.ner.causal_condition_adapter._find_all_causal_tokens src.dackar.RCA.ner.causal_condition_adapter._causal_span_text src.dackar.RCA.ner.causal_condition_adapter._dedup_span_overlap src.dackar.RCA.ner.causal_condition_adapter._np_subtree_text src.dackar.RCA.ner.causal_condition_adapter._head_phrase src.dackar.RCA.ner.causal_condition_adapter._extract_passive_agent src.dackar.RCA.ner.causal_condition_adapter._get_prev_sent_subject_np src.dackar.RCA.ner.causal_condition_adapter._infer_participial_cause src.dackar.RCA.ner.causal_condition_adapter._llm_repair_weak_statements src.dackar.RCA.ner.causal_condition_adapter._derive_condition_state src.dackar.RCA.ner.causal_condition_adapter._extract_labeled_condition src.dackar.RCA.ner.causal_condition_adapter._infer_condition_from_mentions src.dackar.RCA.ner.causal_condition_adapter._infer_condition_from_text src.dackar.RCA.ner.causal_condition_adapter._normalize_health_state src.dackar.RCA.ner.causal_condition_adapter._detect_procedural_deviation src.dackar.RCA.ner.causal_condition_adapter._route_negated_statements src.dackar.RCA.ner.causal_condition_adapter._fill_summary_flags src.dackar.RCA.ner.causal_condition_adapter._has_useful_stage5_signal src.dackar.RCA.ner.causal_condition_adapter._dedupe_dicts Module Contents --------------- .. py:data:: logger .. py:data:: _ALLOWED_CONDITION_STATES .. py:data:: _DATA_DIR .. py:data:: _SPEECH_ACT_VERBS .. py:data:: _DEP_CAUSAL_VERB_LEMMAS_FALLBACK .. py:function:: _load_causal_verb_lemmas() Load single-word causal verb lemmas from cause_effect_keywords_full.csv. Falls back to the minimal hardcoded set when the file is unavailable so the adapter remains functional in stripped deployment environments. .. py:function:: _build_conjecture_pattern() Build the conjecture regex from conjecture_keywords.csv. Falls back to the hardcoded pattern when the file is unavailable. The file is the same source used by ConjectureEntity, so both components stay in sync automatically. .. py:data:: _DEP_CAUSAL_VERB_LEMMAS :type: frozenset .. py:data:: _CONJECTURE_PAT :type: re.Pattern .. py:data:: _HEALTH_FAILED_FRAGMENTS .. py:data:: _HEALTH_CONDITION_FALLBACK :type: Dict[str, frozenset] .. py:function:: _load_health_condition_terms() Load health-state terms from the project's health status CSV files. Negative-file terms are split: terms containing a fragment from _HEALTH_FAILED_FRAGMENTS map to "failed"; the remainder map to "degraded" (conservative — impaired function is safer to under-state than over-state). Positive-file terms map to "acceptable". Falls back to _HEALTH_CONDITION_FALLBACK when files are unavailable. .. py:data:: _HEALTH_CONDITION_TERMS :type: Dict[str, frozenset] .. py:data:: _HEALTH_FAILED_ROOTS .. py:data:: _HEALTH_DEGRADED_ROOTS .. py:data:: _DEMONSTRATIVES .. py:data:: _CONJ_BACKWARD_PAT .. py:data:: _TEMPORAL_SINCE_PAT .. py:data:: _FORWARD_PREP_CONNECTORS .. py:data:: _PREP_CONNECTOR_PAT .. py:data:: _NEGATION_PAT .. py:data:: _CHAIN_JACCARD_THRESHOLD :value: 0.35 .. py:data:: _CHAIN_EMBED_THRESHOLD :value: 0.75 .. py:data:: _PARTICIPIAL_DEPS .. py:data:: _CLAUSE_DEPS_NP .. py:function:: _build_embed_fn(nlp = None, llm_cfg = None) Return an embedding similarity callable for chain linking (Improvement D). Priority: 1. ``llm_cfg["embedding_fn"]`` — a user-injected callable(text_a, text_b) → float. Suitable for sentence-transformer models or any custom encoder. 2. spaCy ``nlp(a).similarity(nlp(b))`` — only when the loaded model carries word vectors (en_core_web_md/lg/trf). en_core_web_sm has no vectors and is skipped automatically. 3. None — chain linking falls back to Jaccard-only. The returned callable accepts two strings and returns a float in [0, 1]. .. py:function:: calibrate_chain_threshold(annotated_chains, extracted_statements, thresholds = None, embed_fn = None) Sweep Jaccard thresholds to find the value maximising chain F1. Intended to be called from the test notebook (Improvement B). Pass in the gold-standard chain node lists and the extracted statements produced by ``_dep_causal_fallback`` or the full pipeline; returns a table of precision / recall / F1 per threshold and the best threshold. When embed_fn is provided (see _build_embed_fn), the calibration also tests the embedding fallback path introduced by Improvement D, so the returned best_threshold applies to the same code path used at run time. :param annotated_chains: list of gold chains, each chain is an ordered list of node text strings (as stored in the dataset). :param extracted_statements: list of statement dicts from the adapter. :param thresholds: thresholds to evaluate; defaults to 0.10 … 0.90 step 0.05. :param embed_fn: optional embedding callable — same as passed to _chain_causal_statements. :returns: dict with keys "results" (list of per-threshold dicts) and "best_threshold" (float). .. py:function:: _chain_causal_statements(stmts, embed_fn = None) Link individual cause-effect statements into multi-hop causal chains. Builds a directed graph where edge i→j exists when effect_text[i] and cause_text[j] are sufficiently similar. Two similarity passes: Pass 1 — Jaccard token overlap ≥ _CHAIN_JACCARD_THRESHOLD (lexical match). Pass 2 — embedding cosine similarity ≥ _CHAIN_EMBED_THRESHOLD (Improvement D). Only runs when Jaccard fails and embed_fn is not None. This catches semantically equivalent but lexically distinct phrases (e.g. "coolant inventory loss" ↔ "reactor coolant system leakage"). embed_fn: callable(text_a: str, text_b: str) → float. Build with ``_build_embed_fn(nlp, llm_cfg)`` — returns spaCy similarity when the loaded model has word vectors (md/lg/trf), a user-injected sentence-transformer callable, or None (Jaccard-only) when neither is available. Returns a list of chain dicts sorted by length descending, or [] when fewer than two statements are provided or no links are found. .. py:data:: STAGE5_OUTPUT_SCHEMA .. py:function:: empty_stage5_output() .. py:function:: _call_llm_json(prompt, llm_cfg) Make a single LLM call and return the parsed JSON response body. Uses the OpenAI-compatible chat/completions endpoint by default. Returns None on any error so callers can degrade gracefully. .. py:function:: _llm_causal_fallback(*, doc_id, chunk_index, chunk_text, doc_type, section_role, llm_cfg) Ask the LLM to extract implicit causal relationships that rule-based methods missed. Fires only when both CausalSentence and CausalSimple return no statements. Returns a list of normalised causal-statement dicts ready for ``extracted_causal_statements``, or [] on failure. .. py:function:: _llm_condition_state_fallback(*, chunk_text, doc_type, section_role, llm_cfg) Classify equipment condition when keyword heuristics return None. Returns one of ``"acceptable"``, ``"degraded"``, ``"failed"``, ``"unknown"``, or None if the LLM call fails. .. py:function:: _format_few_shot_block(examples) Format llm_cfg['few_shot_examples'] into a numbered example block for injection. Each example dict should have keys: cause_text, effect_text, connector, relation_type (optional). Missing keys are rendered as empty strings. .. py:function:: _llm_extract_all_relations(*, doc_id, chunk_index, chunk_text, doc_type, section_role, llm_cfg) Ask the LLM to extract ALL causal relations (explicit and implicit) from the text. Used in two modes: 1. Supplement — fires after rule-based extraction when llm_cfg["extract_all"] is True; novel relations are merged in via _merge_llm_statements. 2. Replacement — fires instead of _llm_causal_fallback when both rule-based paths return empty AND llm_cfg["extract_all"] is True. 3. Weak-fallback supplement — fires via _maybe_trigger_llm_on_weak_fallback (Improvement H) when dep_fallback returns only low-confidence results. Prompt covers: - Explicit connectors ("caused by", "due to", "resulting in") - Implicit causation (no connector; domain-knowledge inference) - Reversed causal order (effect stated before cause in a "because" clause) - Multi-cause convergence (multiple causes sharing one effect) Few-shot examples injected when llm_cfg["few_shot_examples"] is present (14.3). Returns a normalised statement list tagged source="LLM_extract_all", or [] on failure. .. py:function:: _merge_llm_statements(existing, llm_stmts, threshold = 0.6) Append LLM statements not already covered by rule-based extractions. A LLM statement is treated as a near-duplicate of an existing one when Jaccard(cause_tokens) >= threshold AND Jaccard(effect_tokens) >= threshold. Only novel statements are appended; existing statement order is preserved. .. py:function:: _maybe_trigger_llm_on_weak_fallback(result, text, doc_id, chunk_index, doc_type, section_role, llm_cfg, embed_fn = None) Supplement dep_fallback with LLM when all existing statements are weak (Improvement H). Trigger condition: every statement in extracted_causal_statements has an empty effect_text or a confidence score at or below weak_confidence_threshold (default 0.60). When triggered, calls _apply_llm_extract_all which merges novel LLM relations and re-chains the combined set. Activation requires all three flags to be set in llm_cfg: - enabled: True - extract_all: True - trigger_on_weak_fallback: True (opt-in; False by default — no regression on deployments that do not configure an LLM) Optional tuning key: - weak_confidence_threshold: float (default 0.60) — statements above this with non-empty effect_text are considered "strong enough" to skip the LLM supplement. .. py:function:: _apply_llm_extract_all(result, text, doc_id, chunk_index, doc_type, section_role, llm_cfg, embed_fn = None) Supplement existing rule-based statements with LLM-extracted relations (Fix 6). Fires only when llm_cfg["enabled"] is True AND llm_cfg["extract_all"] is True. Novel LLM relations are merged in, the result is re-chained, and summary flags are updated. Mutates result in place; no-ops when nothing new is found. .. py:function:: extract_stage5_causal_condition(*, doc_id, chunk_index, chunk_text, doc_type, section_role, nlp = None, causal_sentence_factory = None, causal_simple_factory = None, llm_cfg = None) Primary: CausalSentence (dep-tree, explicit connectors) Fallback: CausalSimple (simpler dep-tree, fewer constraints) LLM tiers: - repair : _llm_repair_weak_statements — fills empty sides on weak statements - implicit : _llm_causal_fallback — fires on complete rule-based miss (implicit only) - extract_all (Fix 6): _llm_extract_all_relations — supplements or replaces rule-based extraction when llm_cfg["extract_all"] is True; covers explicit + implicit relations and multi-hop links the dep-tree misses. Chain linking (Improvement D): embedding fallback runs after Jaccard when embed_fn is available — builds from _build_embed_fn(nlp, llm_cfg). Returns normalized Stage 5 payload. .. py:function:: _instantiate_extractor(*, cls, text, nlp = None, factory = None) Robust extractor construction. Preferred options for the current causal stack: 1) explicit factory(text=..., nlp=...) 2) cls(nlp) 3) cls(nlp=nlp) Legacy fallbacks retained after that. .. py:function:: _run_extractor_if_needed(*, obj, text) Best-effort execution hook for legacy extractor classes that require an explicit parse/run call after construction. .. py:function:: _raise_ctor_skip() .. py:function:: _normalize_from_causal_sentence(*, doc_id, chunk_index, chunk_text, doc_type, section_role, extractor_obj, llm_cfg = None) .. py:function:: _normalize_from_causal_simple(*, doc_id, chunk_index, chunk_text, doc_type, section_role, extractor_obj, llm_cfg = None) .. py:function:: _extract_native_stage5(extractor_obj) Preferred normalized interface from the causal classes themselves. .. py:function:: _extract_causal_rows(extractor_obj, native = None) Supports CausalSentence / CausalSimple objects that expose internal causal outputs as lists or dataframes. .. py:function:: _tuple_to_causal_dict(item) .. py:function:: _extract_status_mentions(extractor_obj, native = None) .. py:function:: _safe_text(value) .. py:function:: _pick_first(d, keys, default = None) .. py:function:: _build_causal_statement(*, doc_id, chunk_index, i, row, source) .. py:function:: _score_causal_statement(*, connector, cause_text, effect_text, negated, conjectural) .. py:function:: _dep_causal_fallback(text, nlp, doc_id, chunk_index) Dep-tree + regex causal extraction that works without SSC entity patterns. Fires when CausalSentence / CausalSimple return no statements. Pass 1 — dep-tree verb scan: walk every sentence for causal verb lemmas; extract nsubj (cause) and dobj/pobj/xcomp/ccomp (effect) spans. Improvements: passive agent (3a), participial inference (3b), cross-statement inheritance (3c), VP-complement effects (G), demonstrative coreference resolution (I). Pass 2 — regex prepositional connectors: "due to", "because of", "resulted from", "led to", etc. Direction resolved via _FORWARD_PREP_CONNECTORS. Pass 3 — conjunctive backward connectors (14.1): "because", "since", "given that". Handles both leading form ("Because X, Y") and trailing form ("Y because X"). Temporal "since" filtered by _TEMPORAL_SINCE_PAT. .. py:function:: _find_all_causal_tokens(sent) Return all tokens in a spaCy Span whose lemma is a causal verb. .. py:function:: _causal_span_text(causal_tok) Return the text of causal_tok's argument window, excluding embedded clauses. Collects causal_tok itself and the subtrees of its direct causal-argument children (nsubj, dobj, etc.) while blocking clause expansions (relcl, advcl, ccomp, …). Used to scope negation/conjecture matching to the local causal span rather than the full sentence, preventing false positives from hedged clauses in multi-relation sentences (Improvement C). .. py:function:: _dedup_span_overlap(stmts, threshold = 0.8) Remove near-duplicate statements with highly-overlapping cause+effect spans. When two statements share a sentence and have Jaccard ≥ threshold on both cause_text and effect_text, keep only the higher-confidence one. Used to prune redundant extractions produced by multiple causal tokens in the same sentence that govern overlapping subtrees. .. py:function:: _np_subtree_text(tok) Return the full NP text rooted at tok, excluding embedded clause subtrees. Walks the dependency subtree but blocks any sub-tree whose root dep_ is in _CLAUSE_DEPS_NP (relative clauses, adverbial clauses, complement clauses). This captures prepositional extensions — "Erosion of the turbine blade leading edges" — without pulling in entire relative-clause sentences like "the valve that failed last year" (Fix 4). .. py:function:: _head_phrase(head_tok, target_deps) Return the PP-extended NP text of the first dep child matching target_deps. Uses the full dependency subtree of the matched child (Fix 4), which correctly captures "Erosion of the turbine blade leading edges" where noun_chunks would return only "Erosion". Embedded clauses (relcl/acl/ advcl) are filtered out via _np_subtree_text so relative-clause content is not included in the span. .. py:function:: _extract_passive_agent(causal_tok) Return the by-phrase agent of a passive causal verb, or None. Handles "Y was caused/triggered by X" where X is the real cause. Looks for an ``agent`` dep child (the ``by`` PP that spaCy labels explicitly). .. py:function:: _get_prev_sent_subject_np(doc, sent_idx, sent_bounds) Return the antecedent NP for a demonstrative subject from sentence sent_idx-1. Looks for the nsubj of the ROOT verb in the previous sentence. This covers the dominant demonstrative reference pattern in nuclear maintenance text: "The bearing wore out. [This] caused shaft vibration." Returns None when sent_idx == 0 or no suitable antecedent is found. .. py:function:: _infer_participial_cause(causal_tok) Infer cause_text for a participial/adverbial causal verb that has no nsubj. Participial verbs (dep_ in _PARTICIPIAL_DEPS) inherit their grammatical subject from the governing clause. Walk up the dependency tree to find the nearest ancestor verb that does have an nsubj/nsubjpass child, and return that noun phrase as the implicit cause. Example: "Erosion of the leading edges, triggering accelerated seal wear" → "triggering".dep_ = advcl, head = "Erosion" → cause = "Erosion of the leading edges" .. py:function:: _llm_repair_weak_statements(stmts, weak, chunk_text, llm_cfg) LLM-targeted repair of incomplete causal statements. Accepts statements that have at least one empty side (cause_text or effect_text). Statements with both sides filled and confidence ≥ 0.60 are never sent to LLM. Call budget is controlled by llm_cfg["max_repair_calls"] (default 3). Priority order: most-missing sides first, then lowest confidence. Repaired statements are tagged source = "dep_fallback+llm_repair" and re-scored via _score_causal_statement. Dicts are modified in place (the same objects exist in `stmts`). .. py:function:: _derive_condition_state(*, chunk_text, doc_type, section_role, status_mentions, llm_cfg = None) .. py:function:: _extract_labeled_condition(text, label) .. py:function:: _infer_condition_from_mentions(status_mentions) .. py:function:: _infer_condition_from_text(text) Classify equipment condition from free text using the curated vocabulary. Two-pass check (Improvement F): Pass 1 — exact substring match against _HEALTH_CONDITION_TERMS (loaded from data/health_status_keywords_*.csv at import time). Pass 2 — root-fragment fallback (_HEALTH_FAILED_ROOTS / _HEALTH_DEGRADED_ROOTS) to catch morphological variants not present in the CSV (e.g. "leakage" via "leak", "worn" via "wear"). Priority: failed > degraded > acceptable. .. py:function:: _normalize_health_state(status) Map a raw status string to one of failed/degraded/acceptable/unknown. Uses _HEALTH_CONDITION_TERMS (Improvement F). Checks exact set membership first, then substring fallback for compound status strings, then root fragments for morphological variants. .. py:function:: _detect_procedural_deviation(*, chunk_text, doc_type, section_role) Placeholder heuristic. Good enough for v1, especially for SOP/WO. .. py:function:: _route_negated_statements(result) Move negated causal statements to ruled_out_mechanisms (14.4). Negated statements ("did not cause", "was not caused by") are ruled-out failure hypotheses, not confirmed causal links. Routing them separately lets the RCA workflow surface them as eliminated mechanisms rather than noise in the main causal stream. .. py:function:: _fill_summary_flags(out, chunk_text = '') .. py:function:: _has_useful_stage5_signal(out) .. py:function:: _dedupe_dicts(items)