src.dackar.RCA.ner.hybrid_ner.generators

Candidate generators produce CandidateSpan objects from a Document.

Multiple generators are intended: gazetteer, regex/patterns, noun phrases, heuristics, etc.

Submodules

• src.dackar.RCA.ner.hybrid_ner.generators.anchored_np_generator
• src.dackar.RCA.ner.hybrid_ner.generators.base
• src.dackar.RCA.ner.hybrid_ner.generators.candidate_utils
• src.dackar.RCA.ner.hybrid_ner.generators.description_embed_generator
• src.dackar.RCA.ner.hybrid_ner.generators.gazetteer_generator
• src.dackar.RCA.ner.hybrid_ner.generators.nounphrase_generator
• src.dackar.RCA.ner.hybrid_ner.generators.regex_generator

Classes

CandidateGenerator	Interface for candidate generation.
RegexCandidateGenerator	Simple candidate generator using regex patterns.
GazetteerGenerator	Gazetteer-based candidate generator reading an Excel file of labeled term lists.
GazetteerConfig	Configuration for gazetteer matching.
NounPhraseGenerator	Noun phrase candidate generator.

Package Contents

class src.dackar.RCA.ner.hybrid_ner.generators.CandidateGenerator

Bases: abc.ABC

Interface for candidate generation.

Candidate generation is the high-recall stage of Hybrid NER. Each generator proposes spans (with provenance and optional label hypotheses).

abstractmethod generate(doc)

Return a list of CandidateSpan objects for the given document.

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

class src.dackar.RCA.ner.hybrid_ner.generators.RegexCandidateGenerator(patterns)

Bases: src.dackar.RCA.ner.hybrid_ner.generators.base.CandidateGenerator

Simple candidate generator using regex patterns.

Intended as a starter generator for v0.1 (before gazetteer + noun chunks are added). You can use this to propose common reliability phrases like:

• “failed to start”

• “trip occurred”

• “corrosion-induced failure”

Each regex can optionally attach an initial label hypothesis.

Parameters:

patterns (Sequence[Tuple[str, str, str]])

patterns

generate(doc)

Return a list of CandidateSpan objects for the given document.

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

class src.dackar.RCA.ner.hybrid_ner.generators.GazetteerGenerator(excel_path, sheet_names=None, config=None)

Bases: src.dackar.RCA.ner.hybrid_ner.generators.base.CandidateGenerator

Gazetteer-based candidate generator reading an Excel file of labeled term lists.

Expected Excel convention:

• Each sheet contains one or more columns of terms.

• Column header should contain the label in square brackets, e.g.:

  “Degradation mechanisms [deg_mech]”

  If brackets are missing, the entire header is treated as the label.

For each term:

• exact_phrase mode: compiled, case-insensitive word-boundary regex

• fuzzy_tokens mode: approximate phrase matching by token overlap

Produces CandidateSpan with a LabelHypothesis(label=<label>).

Parameters:

• excel_path (str)

• sheet_names (Optional[List[str]])

• config (Optional[GazetteerConfig])

excel_path

config

label_terms

_compiled_exact: List[Tuple[str, str, re.Pattern]] | None = None

_load_gazetteer(path, sheet_names)

Parameters:

• path (str)

• sheet_names (Optional[List[str]])

Return type:

Dict[str, Set[str]]

_compile_exact(label_terms)

Parameters:

label_terms (Dict[str, Set[str]])

Return type:

List[Tuple[str, str, re.Pattern]]

_tokenize(text)

Parameters:

text (str)

Return type:

List[Tuple[str, int, int]]

_term_tokens(term)

Parameters:

term (str)

Return type:

List[str]

generate(doc)

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

_generate_exact(doc)

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

_generate_fuzzy(doc)

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

get_token_evidence(schema, min_len=3)

Return role-aware token evidence derived from the gazetteer.

Output schema:

{

“exclusive_by_group”: { “G1_PHYSICAL_COMPONENT”: {tokens…}, “G4_MECHANISM_PROCESS”: {tokens…}, … }, “token_to_groups”: { “token”: {“G1_PHYSICAL_COMPONENT”,”G4_MECHANISM_PROCESS”,…}, … }

}

A token is exclusive to a group if it only appears in gazetteer terms whose labels map to that group. This is used downstream to reduce false multi-label acceptance.

Parameters:

• schema (SchemaIndex)

• min_len (int)

Return type:

dict

class src.dackar.RCA.ner.hybrid_ner.generators.GazetteerConfig

Configuration for gazetteer matching.

match_mode:

• “exact_phrase”: match whole phrase with word boundaries (case-insensitive)

• “fuzzy_tokens”: slide a token window and match by token overlap (Jaccard)

fuzzy_jaccard_threshold:

Minimum Jaccard similarity (0..1) to accept a fuzzy match.

max_window_tokens:

Maximum tokens for sliding window when fuzzy matching is enabled.

emit_overlapping:

If False, stops after the first fuzzy match for each term (reduces duplicates).

match_mode: str = 'exact_phrase'

fuzzy_jaccard_threshold: float = 0.8

max_window_tokens: int = 8

emit_overlapping: bool = True

class src.dackar.RCA.ner.hybrid_ner.generators.NounPhraseGenerator(max_tokens=6, min_tokens=1)

Bases: src.dackar.RCA.ner.hybrid_ner.generators.base.CandidateGenerator

Noun phrase candidate generator.

Primary mode:

• Uses spaCy noun chunks if spaCy and an English model are available.

Fallback mode (regex-free):

• Tokenize with a simple whitespace/punctuation tokenizer

• (Optionally) POS-tag with NLTK if available

• Chunk noun-phrase-like spans using lightweight rules:

  • sequences of (ADJ|NOUN|PROPN|NUM|HYPHENATED) tokens

  • length 1..max_tokens tokens

  • must contain at least one alphabetic token

Produces CandidateSpan without proposed labels (labeling handled later by gazetteer/rules/ML).

Parameters:

• max_tokens (int)

• min_tokens (int)

max_tokens = 6

min_tokens = 1

_nlp = None

_nltk_available = False

generate(doc)

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

_generate_spacy(doc)

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

_tokenize_with_offsets(text)

Return (token, start, end) offsets with simple punctuation stripping.

Parameters:

text (str)

Return type:

List[Tuple[str, int, int]]

_pos_tags(tokens)

Parameters:

tokens (List[str])

Return type:

Optional[List[str]]

_is_np_token(tok, pos)

Parameters:

• tok (str)

• pos (Optional[str])

Return type:

bool

_generate_fallback(doc)

Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]