src.dackar.RCA.ner.hybrid_ner.generators.nounphrase_generator

Classes

NounPhraseGenerator

Noun phrase candidate generator.

Module Contents

class src.dackar.RCA.ner.hybrid_ner.generators.nounphrase_generator.NounPhraseGenerator(max_tokens=6, min_tokens=1)[source]

Bases: src.dackar.RCA.ner.hybrid_ner.generators.base.CandidateGenerator

Noun phrase candidate generator.

Primary mode:

  • Uses spaCy noun chunks if spaCy and an English model are available.

Fallback mode (regex-free):

  • Tokenize with a simple whitespace/punctuation tokenizer

  • (Optionally) POS-tag with NLTK if available

  • Chunk noun-phrase-like spans using lightweight rules:

    • sequences of (ADJ|NOUN|PROPN|NUM|HYPHENATED) tokens

    • length 1..max_tokens tokens

    • must contain at least one alphabetic token

Produces CandidateSpan without proposed labels (labeling handled later by gazetteer/rules/ML).

Parameters:

  • max_tokens (int)

  • min_tokens (int)

max_tokens = 6[source]
min_tokens = 1[source]
_nlp = None[source]
_nltk_available = False[source]
generate(doc)[source]
Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

_generate_spacy(doc)[source]
Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]

_tokenize_with_offsets(text)[source]

Return (token, start, end) offsets with simple punctuation stripping.

Parameters:

text (str)

Return type:

List[Tuple[str, int, int]]

_pos_tags(tokens)[source]
Parameters:

tokens (List[str])

Return type:

Optional[List[str]]

_is_np_token(tok, pos)[source]
Parameters:

  • tok (str)

  • pos (Optional[str])

Return type:

bool

_generate_fallback(doc)[source]
Parameters:

doc (src.dackar.RCA.ner.hybrid_ner.models.Document)

Return type:

List[src.dackar.RCA.ner.hybrid_ner.models.CandidateSpan]