2b:null 2c:["$","title",null,{"children":"Compromise - match-syntax | spencer kelly | Observable"}] 31:[] 2f:{"id":"b462285307e24faa","slug":"compromise-match-syntax","title":"Compromise - match-syntax","thumbnail":"1a27670d5dd4125356c079eb38962f47259a3b16d797c750f1516bfd5eb8d9c2","access_level":"public","create_time":"$D2018-06-05T18:16:10.846Z","update_time":"$D2022-11-09T21:20:45.138Z","view_time":null,"first_public_time":"$D2018-06-05T18:52:56.544Z","fork":false,"stars":3,"starred":false,"owner":{"type":"team","id":"178b998367cf1e11","login":"spencermountain","name":"spencer kelly","avatar_url":"https://avatars-next.observableusercontent.com/006044842415892d25db450d275d493d3e5629ec1c2a2dfad0167421e0f026f9","tier":"starter_2024"},"version":384,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"25825fd15d797d3f","login":"spencermountain","name":"spencer kelly","avatar_url":"https://avatars-next.observableusercontent.com/006044842415892d25db450d275d493d3e5629ec1c2a2dfad0167421e0f026f9","tier":null}],"members":[{"type":"individual","id":"25825fd15d797d3f","login":"spencermountain","name":"spencer kelly","avatar_url":"https://avatars-next.observableusercontent.com/006044842415892d25db450d275d493d3e5629ec1c2a2dfad0167421e0f026f9","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":6,"collection":{"id":"e53dcf6ebdb6a393","slug":"nlp-compromise","title":"nlp-compromise","owner":{"id":"178b998367cf1e11","login":"spencermountain","name":"spencer kelly"}},"type":"notebook","body":{"title":"Compromise - match-syntax","cells":[{"id":0,"value":"md`# Compromise - match-syntax\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"md`[compromise](https://beta.observablehq.com/@spencermountain/nlp-compromise) provides a handy way to lookup/grab parts of a text, based on their interpreted metadata- as opposed to just their characters.\n\n\n Since we've parsed-up and tagged a document, writing a bunch of custom \\'forEach\\` loops would be a drag. The goal of .match() is to provide a quick way to describe any grammatical pattern, or match condition, using a human-readable, and mostly-reasonable style.\n\n\nWhen writing match-syntax statements, you can [test them out here](https://observablehq.com/@spencermountain/compromise-match-test).\n\nFor ease of use, it superficially resembles REGEX, and was also built to resemble the [TOKENSREGEX syntax](https://nlp.stanford.edu/pubs/tokensregex-tr-2014.pdf) in the Stanford Tagger.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":15,"value":"html`\n

Word-matches

\nterm-term matches use normalised & non-normalised text as a direct lookup:\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":131,"value":"nlp('John eats glue!').match('john').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":125,"value":"html`

Tag-matches - `#`

\nyou can loosen a search by any matching part-of-speech, allowing you to find all the things john eats, for example:\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":127,"value":"nlp('John eats glue').match('john eats #Noun').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":108,"value":"html`\n

Wildcard matches - `.`

\nThe \\`.\\` character means any one term.\n

\n`\n\n\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":110,"value":"nlp('John eats glue').match('john . glue').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":113,"value":"html`

\nThe * means all terms until - it may be 0.\n

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":115,"value":"nlp('John always ravenously eats his glue').match('john * eats').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":92,"value":"html`\n

Greedy-match - `+`

\nthe\\`+\\` character at the end of a tag (or \\`.\\`) implies the match will continue with repeated consecutive matches, as far as it can:\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":94,"value":"nlp('he is really, really good').match('#Adverb+ good').terms().text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":102,"value":"html`

Optional matching - `?`

\nThe \\`?\\` character at the end of a word means it is nice to have, but not necessary.\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":104,"value":"nlp('John eats glue').match('john always? eats glue').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":86,"value":"html`\n

Options-list - `()`

\n\\`(word1|word2)\\` parentheses allow listing possible matches for the word. - ( 'OR' logic )\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":88,"value":" nlp('John eats glue').match('john (eats|sniffs|wears) .').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":76,"value":"html`\n

Regex-match - `/reg/`

\nyou can run a javascript regular-expression on every word in your document, if you wish, using the \\`/myregex/\\` syntax.\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":82,"value":"nlp('it is raining and had rained').match('#Verb /rain[ing|ed]/').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":78,"value":"html`

\nnote that this will not match multiple-word patterns, and will be slower than other lookups, like \\`(#Verb raining|#Verb rained)\\`, for example.\n\n

Note: if you are letting users create custom matches themselves, this can open-up a security vulnerability that could allow them to bring-down a server with a very-slow regular expression.

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":70,"value":"html`\n

Capture-groups - `[]`

\nyou can name and grab a part of a match, using \\`[]\\` brackets around any group. Using this pattern you can effectively to 'look-arounds', to add conditions to a match statement.\n

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":290,"value":"nlp('five maids a-milking.').match('[#Value] maids a milking').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":72,"value":"nlp('i saw ralf eat the glue, Mrs Hoover.').match('[#Verb the #Noun]').groups()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":283,"value":"md`if you use a 'bare' capture group, it can be referred to by its index:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":284,"value":"{\n let doc = nlp(`If you can't find metal stucco lath... Use carbon-fiber stucco lath!`)\n let m = doc.match('#Verb [.{1,2}] stucco lath')\n return m.groups(0).out('array')\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":296,"value":"`calling \\`.groups()\\` without a parameter will return an object with all named groups:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":299,"value":"{\n let doc = nlp('that leaves 5 in the bank, and 80 hundred we need')\n let m = doc.match('[#Value+] in the bank and [#Value+] we need')\n return m.groups()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":288,"value":"md`\nif you want to quickly return the group, and ignore the rest, throw the name in the 2nd param: \n`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":293,"value":"nlp('turn left and slay the dragon').match('turn [.]',0).text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":56,"value":"html`\n

First/Last - `^` `$`

\nyou can specify that something should be in the start, or end of a match\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":58,"value":"nlp('John eats glue').match('^john #Verb').out('text')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"nlp('John eats glue').match('eats glue$').terms().out('array')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":49,"value":"html`

Negative - `!`

\nyou can specify that something not-match with a leading \\`!\\` character:\n

\n`\n\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":51,"value":"nlp('Homer Simpson and Homer Adkins').match('homer !simpson').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":301,"value":"md`On it's own, a negative match is a stand-in for a token.\nSometimes it's more intuitive for negative matches to be paired with the \\`?\\` (optional) syntax:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":305,"value":"nlp('one two three').match('one !foo? two').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":319,"value":"nlp('this is tricky.').match('this is !not? #Adjective').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":43,"value":"html`

Range - `{min,max}`

\nyou can specify a max, min number of wildcard terms, like this:\n

\n`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":45,"value":"nlp('homer j j j j simpson').match('homer #Acronym{2,6} simpson').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":32,"value":"html`

Prefix, suffix, infix - `_`

\nyou can look for sub-word matches, with the regex \\`/ /\\` characters:\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"nlp(`funny story`).match('/nny$/').out() //suffix lookup","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":151,"value":"nlp('Why all the pearls? Why all the hair?').match('/ear/').text() //'infix' lookup","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":157,"value":"html`

AND logic

\nit's sometimes a little awkward, but you can perform an AND comparison using two match statements:\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":159,"value":"nlp(`Homer J. Simpson`).match('#Person').match('#Acronym').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":182,"value":"md`in a limited way, we also support AND match statements with a \\`&&\\` syntax:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":186,"value":"nlp(`Homer J. Simpson`).match('(#Person && #Acronym)').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":196,"value":"html`

@ methods

\nyou can also query some term information with the \\`@\\` character:\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":194,"value":"md`\n- **@hasQuote** - does it have a quotation symbol?\n- **@hasComma** - does it have a comma?\n- **@hasPeriod** - does it end in a period?\n- **@hasExclamation** - does it end in an exclamation\n- **@hasQuestionMark** - does it end with a question mark?\n- **@hasEllipses** - is there a ... at the end?\n- **@hasSemicolon** - is there a semicolon after this word?\n- **@hasColon** - does the word end in with a colon?\n- **@hasSlash** - is there a slash '/' in this word?\n- **@hasHyphen** - a hyphen connects words *'like-this'*, with no space\n- **@hasDash** - a dash separates words *'like - this'*, with a space (not @hasHyphen)\n- **@hasContraction** - is it multiple words combined?\n- **@isAcronym** - does this term look like an acronym?\n- **@isKnown** - does the term have at least one good tag?\n- **@isUpperCase** - are all the (main) letters upper-case?\n- **@isTitleCase** - is the first letter upper-case, and the remaining not?\n- **@isUpperCase** - are all letters upper-case?\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":202,"value":"nlp(`we're not gonna take it. No! We're not gonna take it.`).match('@hasExclamation').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":211,"value":"md`term methods work-well with any other syntax variant, like so:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":206,"value":"nlp(`we're not gonna take it. No!`).match('(@hasContraction|@isTitleCase)+').out('array')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":225,"value":"html`

{ Root-matches }

\nif you run .compute('root') first, you can do a soft-match using \\`{}\\` characters.\n

the root form of a term is like a 'stemming, or lemmatization'

\nthe root form of a #Verb is an #Infinitive form.\n

\nthe root form of a #Noun is a #Singular form.\n\n

there are other, minor normalizations of Acronyms, and Abbreviations.

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":223,"value":"{\n let doc = nlp(\"I'm looking for Amanda Hugginkiss\") \n // cache the root form of each word\n doc.compute('root')\n // use a 'soft' lookup:\n let m = doc.match('{look} for')\n // print our result:\n return m.text()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":340,"value":"html`

~ Fuzzy-matches ~

\nyou can soften-up text-matching using \\`~\\` characters. Compromise uses Damerau–Levenshtein distance to compute approximate string matches.\n
\nSee details on this feature here\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":356,"value":"nlp('Albuquerque Holiday Inn').match('~albequerque~').text()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":362,"value":"md`you can specify how soft the matching can be, using a 3rd param. The default is pretty safe, at 0.8.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":346,"value":"{\n let doc = nlp('my son is also named bort')\n return doc.match('~bart~', null, {fuzzy:0.7}).text()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":244,"value":"md`---\n### JSON match syntax\nYou can skip all the string-stuff and query your document using straight json, if you prefer.\n\nThis is especially helpful if you have some funny encoded characters or something.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":276,"value":"{\n let json=[\n {choices:[{word:'bort'}, {tag:'MaleName'}]},\n {word:'license'},\n {word:'plates', optional:true}\n ]\n return nlp(`Repeat, we are sold out of Bort license plates.`).match(json).text()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":327,"value":"md`you can see the json version of a given match, using \\`nlp.parseMatch()\\`:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":331,"value":"nlp.parseMatch('we (walked|talked) in? #Month')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":245,"value":"md`\nthe options available are as follows:\n\\`\\`\\`js\n{\n word:'bart',\n tag:'Person',\n choices:[{word:'bort'}, {tag:'MaleName'}],\n regex:/^b[ao]rt$/, //an actual regex object\n start:false, //^\n end:false, //$\n negative:false, //!\n anything:false, //.\n greedy:false, //+\n optional:false, //?\n\n named:'foo', // [ #Tag] \n}\n\n\\`\\`\\`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"style = html`

Word-matches

Tag-matches - #

Wildcard matches - .

Greedy-match - +

Optional matching - ?

Options-list - ()

Regex-match - /reg/

Capture-groups - []

First/Last - ^ $

Negative - !

Range - {min,max}

Prefix, suffix, infix - _