src.dackar.RCA.doc_parsers.mdParser
===================================

.. py:module:: src.dackar.RCA.doc_parsers.mdParser


Attributes
----------

.. autoapisummary::

   src.dackar.RCA.doc_parsers.mdParser.LOGGER
   src.dackar.RCA.doc_parsers.mdParser.ch
   src.dackar.RCA.doc_parsers.mdParser.FIELD_LABEL_MAP
   src.dackar.RCA.doc_parsers.mdParser.HIGH_SIGNAL_ROLES
   src.dackar.RCA.doc_parsers.mdParser.STOPWORDS
   src.dackar.RCA.doc_parsers.mdParser.HEADING_RE
   src.dackar.RCA.doc_parsers.mdParser.IMAGE_RE
   src.dackar.RCA.doc_parsers.mdParser.STANDARD_PATTERNS
   src.dackar.RCA.doc_parsers.mdParser.TAG_PATTERNS
   src.dackar.RCA.doc_parsers.mdParser.DOC_REF_PREFIXES
   src.dackar.RCA.doc_parsers.mdParser.HTML_TAG_RE
   src.dackar.RCA.doc_parsers.mdParser.MD_IMAGE_RE
   src.dackar.RCA.doc_parsers.mdParser.MD_LINK_RE
   src.dackar.RCA.doc_parsers.mdParser.MD_EMPH_RE
   src.dackar.RCA.doc_parsers.mdParser.doc_index_path


Functions
---------

.. autoapisummary::

   src.dackar.RCA.doc_parsers.mdParser.canonicalize_section_role
   src.dackar.RCA.doc_parsers.mdParser._now_iso
   src.dackar.RCA.doc_parsers.mdParser.read_text
   src.dackar.RCA.doc_parsers.mdParser.write_json
   src.dackar.RCA.doc_parsers.mdParser.write_jsonl
   src.dackar.RCA.doc_parsers.mdParser.parse_markdown_sections
   src.dackar.RCA.doc_parsers.mdParser.build_section_paths
   src.dackar.RCA.doc_parsers.mdParser.load_tables_from_json
   src.dackar.RCA.doc_parsers.mdParser._clean_table_cell
   src.dackar.RCA.doc_parsers.mdParser.clean_table_object
   src.dackar.RCA.doc_parsers.mdParser._repair_wrapped_action_rows
   src.dackar.RCA.doc_parsers.mdParser.detect_mbse_mentions
   src.dackar.RCA.doc_parsers.mdParser.detect_equipment_tags
   src.dackar.RCA.doc_parsers.mdParser.detect_document_refs
   src.dackar.RCA.doc_parsers.mdParser.detect_standard_refs
   src.dackar.RCA.doc_parsers.mdParser.extract_keywords
   src.dackar.RCA.doc_parsers.mdParser.assign_tables_to_sections
   src.dackar.RCA.doc_parsers.mdParser.assign_figures_to_sections
   src.dackar.RCA.doc_parsers.mdParser._section_to_chunk
   src.dackar.RCA.doc_parsers.mdParser._paragraph_subchunks
   src.dackar.RCA.doc_parsers.mdParser.build_chunks_for_doc_type
   src.dackar.RCA.doc_parsers.mdParser.md_parser
   src.dackar.RCA.doc_parsers.mdParser.strip_markup_noise
   src.dackar.RCA.doc_parsers.mdParser.normalize_raw_text
   src.dackar.RCA.doc_parsers.mdParser.split_into_paragraphs
   src.dackar.RCA.doc_parsers.mdParser.chunk_paragraphs
   src.dackar.RCA.doc_parsers.mdParser.textify_table


Module Contents
---------------

.. py:data:: LOGGER

.. py:data:: ch

.. py:data:: FIELD_LABEL_MAP
   :type:  dict[str, dict[str, str]]

.. py:data:: HIGH_SIGNAL_ROLES

.. py:function:: canonicalize_section_role(title, doc_type)

   Map a raw section heading to a canonical role name using FIELD_LABEL_MAP.
   Falls back to 'body' if no pattern matches.


.. py:data:: STOPWORDS

.. py:function:: _now_iso()

.. py:function:: read_text(path)

.. py:function:: write_json(path, obj)

.. py:function:: write_jsonl(path, records)

.. py:data:: HEADING_RE

.. py:data:: IMAGE_RE

.. py:function:: parse_markdown_sections(md_text)

   Parse headings and build a hierarchical section list.
   Each element: {"title", "level", "text", "figures": [...], "tables": [...]}


.. py:function:: build_section_paths(sections)

   Adds a hierarchical section_path based on heading nesting.


.. py:function:: load_tables_from_json(paths)

.. py:function:: _clean_table_cell(x)

.. py:function:: clean_table_object(t)

.. py:function:: _repair_wrapped_action_rows(columns, rows)

   Light repair for common corrective-action tables where wrapped lines corrupt
   owner/status columns. Only applies when columns look action-like.


.. py:function:: detect_mbse_mentions(text, mbse_entities)

.. py:data:: STANDARD_PATTERNS
   :value: ['\\bASME\\b.*\\bBPVC\\b.*\\b(Section|Sec\\.?|III|NCA|NC|NB|NH)\\b',...


.. py:data:: TAG_PATTERNS
   :value: ['\\b[A-Z]{1,4}-\\d{2,5}[A-Z]?\\b', '\\b[A-Z]{2,6}\\d{2,5}[A-Z]?\\b']


.. py:data:: DOC_REF_PREFIXES

.. py:function:: detect_equipment_tags(text)

.. py:function:: detect_document_refs(text)

.. py:function:: detect_standard_refs(text)

.. py:function:: extract_keywords(text, top_k = 12)

.. py:function:: assign_tables_to_sections(tables, sections, doc_index)

.. py:function:: assign_figures_to_sections(figures_in_text, sections, doc_figures = None)

   Build a figure list with section assignments.

   Sources (merged, deduplicated by path):
     1. Inline Markdown image references parsed per section (s["figures"]).
     2. `doc_figures`: figures list from document_index (Marker path-indexed).


.. py:function:: _section_to_chunk(s, idx, doc_id, doc_type, doc_name, source_path, ingest_id, classification, content_hash)

   Build a single section-level TextChunk with canonical role.


.. py:function:: _paragraph_subchunks(s, sec_idx, doc_id, doc_type, doc_name, source_path, ingest_id, classification, content_hash, mbse_entities)

   Paragraph sub-chunks for a section. Only produced for high-signal roles.


.. py:function:: build_chunks_for_doc_type(sections, doc_id, doc_type, doc_name, source_path, ingest_id, classification, content_hash, mbse_entities)

   Dispatch to the correct chunking strategy based on doc_type.
   All strategies produce section-level chunks with canonical roles.
   SOP additionally produces one chunk per step group.


.. py:function:: md_parser(document_index, destination_folder, mbse_entities = None, nureg_section_ids = None)

   Parse a document's extracted Markdown (produced by ``pdfParser``) into
   hierarchical sections/chunks enriched with MBSE mentions, standards/document
   references, and keywords, then persist ``structured_output.json`` and
   ``chunks.jsonl``.

   :param document_index: The index dict returned by ``pdfParser`` (must include ``text_md_path``).
   :type document_index: Dict[str, Any]
   :param destination_folder: Root destination for parsed outputs. If None, it is inferred from the
                              parent of the Markdown text file's directory.
   :type destination_folder: Optional[str]
   :param mbse_entities: Optional MBSE entity dictionary used for dictionary-based NER of
                         component mentions.
   :type mbse_entities: Optional[List[Dict[str, Any]]]
   :param nureg_section_ids: Optional NUREG section identifiers to attach to every section.
   :type nureg_section_ids: Optional[List[str]]

   :returns: The structured output dict (also written to ``structured_output.json``),
             mirroring the section/table/figure chunks emitted to ``chunks.jsonl``.
   :rtype: Dict[str, Any]

   :raises ValueError: If ``document_index`` is not a dict.
   :raises FileNotFoundError: If ``text_md_path`` is missing or does not exist on disk.


.. py:data:: HTML_TAG_RE

.. py:data:: MD_IMAGE_RE

.. py:data:: MD_LINK_RE

.. py:data:: MD_EMPH_RE

.. py:function:: strip_markup_noise(text)

.. py:function:: normalize_raw_text(text)

.. py:function:: split_into_paragraphs(text)

.. py:function:: chunk_paragraphs(paragraphs, max_chars = 1400, overlap_chars = 200)

   Packs paragraphs into chunks up to max_chars. Overlap is applied between consecutive chunks.
   Char-based is OK since you're storing raw text and embedding later.


.. py:function:: textify_table(caption, headers, rows, head_rows = 4, tail_rows = 2)

   Compact textual representation for embedding/indexing.

   Always includes:
     - caption
     - column headers
     - first `head_rows` rows  (typical parameter definitions)
     - last  `tail_rows` rows  (often contain limit/summary values)
     - a note when rows are omitted


.. py:data:: doc_index_path