28:null 29:["$","title",null,{"children":"Tutorial: Generating with RiTa Grammars | Daniel Howe | Observable"}] 2e:T7d1,html`If you're not yet familiar with RiTa, check out the website here ...

Other Resources

The RiTa reference in observable
A tutorial on RiScript, RiTa's micro scripting language
Tools for RiTa in observable, including the rs template
Using RiScript and RiTa Grammars within Twine
RiTa's Lexicon and Letter-to-Sound engine
Generating text with Markov chains/n-grams

Grammars

Grammar comes from the Greek syntaxis, meaning 'structure' or 'arrangement', and refers to the way items can be combined together. And while I'll use natural language in the examples below, grammars can pertain to sounds, images, drawing images, or even code (the rules for programming languages are good examples of grammars). We can even use grammars as a means of defining other grammars...

${await FileAttachment( "divider2.png" ).image()}

So what is a grammar? In essence, it is simply a set of rules which defines a valid sequence of items. For example purposes I'll refer to this sequence as a 'sentence', but remember that it can be nearly any type of item.

If we're given a 'sentence', a grammar can tell us whether it is grammatical correct or not. But more importantly for our purposes, once we have a grammar, we can use it to generate new 'sentences' according to its rules.

`2f:T433,md`-------- ## Appendix When [Noam Chomsky](https://en.wikipedia.org/wiki/Noam_Chomsky) first formalized generative grammars in 1956, he classified them into types now known as the [Chomsky hierarchy](https://en.wikipedia.org/wiki/Chomsky_hierarchy). The difference between these types is that they have increasingly strict production rules and can express fewer formal languages. Some important grammar types include [context-sensitive grammars](https://en.wikipedia.org/Context_sensitive_grammar), [context-free grammars](https://en.wikipedia.org/Context_free_grammar) (like RiGrammar) and [regular grammars]((https://en.wikipedia.org/Regular_grammar). The languages that can be described with such grammars are called context-sensitive, context-free, and regular languages, respectively. The latter two are the types of grammars most often used in [natural language processing](https://en.wikipedia.org/natural_language_processing) because, as opposed to context-sensitive grammars like English or Chinese, they can be efficiently implemented in software. ------------ `30:[{"id":"84ecd35263989fddcfc74fd88a670e15332070c139eb907015270e41d77babe8be5fa5ad934556e0384f94ad832d9c3ce6a4971b8aca095017dc0adedcde43cb","href":"https://static.observableusercontent.com/files/84ecd35263989fddcfc74fd88a670e15332070c139eb907015270e41d77babe8be5fa5ad934556e0384f94ad832d9c3ce6a4971b8aca095017dc0adedcde43cb","name":"divider2.png","type":"image/png","lastModified":1631462978383.337,"size":23927},{"id":"cc53347d22bb8280ee0380e2c03525c17f7bcbc7aaec6f6318e13cb4b80705b949f5e7f7546b1333d003f8598a15e003f5ec0553e269f1a28d78ee18b333e857","href":"https://static.observableusercontent.com/files/cc53347d22bb8280ee0380e2c03525c17f7bcbc7aaec6f6318e13cb4b80705b949f5e7f7546b1333d003f8598a15e003f5ec0553e269f1a28d78ee18b333e857","name":"divider@2.jpg","type":"image/jpeg","lastModified":1631459214353.389,"size":18556}] 2c:{"id":"8a44c408fa3ec0eb","slug":"tut-rita-grammars","title":"Tutorial: Generating with RiTa Grammars","thumbnail":"2d45b2a24cfa0e2440ca4d8b99a3588f6372c4f27dbb99c9421bfe37557dc0db","access_level":"public","create_time":"$D2021-03-06T13:42:16.476Z","update_time":"$D2026-02-15T19:36:54.853Z","view_time":null,"first_public_time":"$D2021-03-06T15:22:47.155Z","has_unpublished_edits":null,"fork":{"id":"bff22427fd7e3e47","slug":"generating-with-rita-grammars","title":"Tutorial on Generating with RiTa's Grammars","owner":{"id":"b2e9840be7b1b607","login":"real-john-cheung","name":"John C"}},"stars":5,"starred":false,"owner":{"type":"team","id":"636cf123a07db681","login":"dhowe","name":"Daniel Howe","avatar_url":"https://avatars-next.observableusercontent.com/a5081aaa56ae9990aa27642040dc61de3ce8cea67f6f778a5b864e41111ba7a3","tier":"starter_2024"},"version":1316,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"32869872eb6f2f82","login":"dhowe","name":"Daniel Howe","avatar_url":"https://avatars-next.observableusercontent.com/a5081aaa56ae9990aa27642040dc61de3ce8cea67f6f778a5b864e41111ba7a3","tier":null}],"members":[{"type":"individual","id":"32869872eb6f2f82","login":"dhowe","name":"Daniel Howe","avatar_url":"https://avatars-next.observableusercontent.com/a5081aaa56ae9990aa27642040dc61de3ce8cea67f6f778a5b864e41111ba7a3","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":11,"collection":{"id":"3c04483074fcc200","slug":"rita","title":"RiTa","owner":{"id":"636cf123a07db681","login":"dhowe","name":"Daniel Howe"}},"type":"notebook","body":{"title":"Tutorial: Generating with RiTa Grammars","cells":[{"id":0,"value":"md`# Tutorial: Generating with RiTa Grammars#`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"$2e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1169,"value":"html`\n

RiGrammar

\n\n

We can experiment with grammars in RiTa using the RiGrammar object, which is an example of a probabalistic context-free grammar. This is a rather formal definition, but for our purposes it just means that it is a grammar we can use to generate text according to simple rules that we define. \n

\n\n

For example, if we wanted to generate some very simple sentences with a subject-verb-object pattern, our grammar might look like this:

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":18,"value":"md`\n~~~js\n\n start: $subject $verb $object.\n subject: I | You | They\n object: coffee | bread | milk\n verb: want | hate | like | love \n ...\n~~~ \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":24,"value":"html`\n

One generation from this grammar would be I hate milk. Another would be They want bread.

\n\n

The start line above defines the whole structure of your grammar. In this case, it will generate a subject, followed by a verb, and then an object and a period. For each part, RiTa looks up the rule in the grammar and then randomly chooses a string from the options listed (separated by the | character).

\n\n

In the next cell you can see the above grammar as a JavaScript object
Click the ▸ to view it:

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":252,"value":"rules = ({\n start: \"$subject $verb $object.\",\n subject: \"I | You | They\",\n object: \"coffee | bread | milk\",\n verb: \"want | hate | like | love\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1298,"value":"html`

Note: if you want to experiment with your own grammar, you can use this template in the p5.js online editor.

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"html`

Expanding Grammars

To load our rules, we use RiTa's grammar() function which returns a grammar object. Then we can use the grammar's expand() function to run it.

\n${md`\n~~~ js\n\n let rg = RiTa.grammar(rules); // load our grammar\n let result = rg.expand(); // and then run it!\n‍\n~~~\n`}\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":293,"value":"{\n let el = html`

or, do it all with one line:

⮕ ${a}
click me ☝

`\n el.onclick = () => mutable a = RiTa.grammar(rules).expand();\n return el;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":58,"value":"html`\n\n

By default, RiTa will assign equal weights to all choices in a rule. However, one can adjust the weights by adding 'multipliers' in parentheses, as follows:

\n${md`\n~~~ js\n{\n start: \"coffee (2) | bread | milk\"\n}\n~~~\n`}\n\n

This says that the word “coffee” should be chosen, on average, twice as often as either “bread” or “milk”.

\n\n

We can also make the grammar more complex by adding rules within rules (or even recursive rules):

\n${md`\n~~~ js\n {\n start: \"$subject $verb $object.\",\n subject: \"I | You | They\",\n object: \"$food | $animals\",\n verb: \"want | hate | like | love\",\n food: \"coffee | bread | milk\",\n animals: \"cats | dogs | fish\"\n }\n~~~\n`}`\n\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":331,"value":"{\n let el = html` ⮕ ${b}

Now you might get You love coffee or They want fish ...`\n el.onclick = () => mutable b = RiTa.grammar({\n \"start\": \"$subject $verb $object.\",\n \"subject\": \"I | You | They\",\n \"object\": \"$food | $animals\",\n \"verb\": \"want | hate | like | love\",\n \"food\": \"coffee | bread | milk\",\n \"animals\": \"cats | dogs | fish\"\n }).expand();\n return el;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":924,"value":"md`

By default, the expand() function will start with the start symbol, but you can instead specify your own starting point:

\n~~~ js\n\n grammar.expand('subject'); // start from the \"subject\" rule\n‍\n~~~\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1048,"value":"html`

Aside: In our example grammar here we are using individual words, but it can often be more expressive to use longer fragments in grammars, e.g., phrases or even sentences.

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":479,"value":"reuse = md`## Reusing Rules\n\nThe most useful grammar rules are often those we can reuse. For example, if we define a rule to be a list of colors, we'd like to be able to re-use that rule when we need colors in a later part of the grammar. But this is not always so straightforward. \n\nIn fact there are (at least) *three* different behaviors we might want for our rule:\n\n1. If we are writing about unrelated colorful things, we are probably ok with a simple *random* selection from our color rule\n2. But if we are talking about the color of the main character's hat, we probably want to make sure it stays *the same* throughout the story\n3. And if we are listing colors (“Jane's favorite colors were blue and pink”), we want to make sure that the colors are *different*\n\nLet's consider each case...\n\n**Case 1:** This is the default behavior for grammar rules, that is, random (or weighted random) selection. We provide some options and one is chosen at random.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":687,"value":"statics=md`\n**Case 2:** Consider the common scenario where we need a name for our main character (or *hero*) that we will reuse throughout the story. In RiTa grammars we can handle this case by prepending a # to the rule name. This tells the grammar not to re-evaluate that rule when it appears again (see the #hero rule below).\n\n~~~ json\n${JSON.stringify(hero1,0,2)}\n~~~\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":511,"value":"{\n let el = html` ⮕ ${c}

Notice how hero's name varies across expansions, but is reused in both sentences.`\n el.onclick = () => mutable c = RiTa.grammar(hero1).expand();\n return el;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":576,"value":"md`\n**Case 3:** Finally, in our third scenario, we want to ensure that two evaluations of a rule have *different* values. Here we can use the norepeat(), or nr transform. \n\n~~~ json\n${JSON.stringify(hero2,0,2)}\n~~~\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":605,"value":"{\n let el = html` ⮕ ${d}

Note that the two drinks change values, but are never the same.

For more info on transforms, see this introduction to RiScript.`\n el.onclick = () => mutable d = RiTa.grammar(hero2).expand();\n return el;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":71,"value":"html`
\n
To get a better sense of how it all works, check out this example, or experiment with this template in the p5.js online editor
\n
HINT: All valid RiScript expressions (assignments, transforms, sequences, etc.) are fair game within RiTa grammars. Just include them and they will be handled automatically when you expand your grammar. 💡
\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1131,"value":"html`
NEXT: RiTa+Observable ➡️
`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":37,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":580,"value":"hero1 = ({\n start: \"$line1 $line2\",\n line1: \"$hero $verbs $drink.\",\n line2: \"That's just how $hero is.\",\n '#hero': \"Dave | Wing | Teri\",\n verbs: \"wants | likes | loves\",\n drink: \"coffee | tea | milk\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":601,"value":"hero2 = ({\n start: \"$line1 $line2\",\n line1: \"$hero loves $drink, but hates $drink.norepeat().\",\n line2: \"That's $hero.\",\n '#hero': \"Dave | Wing | Teri\",\n drink: \"coffee | tea | milk\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":262,"value":"import {RiTa} from \"@dhowe/rita\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":263,"value":"mutable a = ''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":560,"value":"mutable b = ''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":562,"value":"mutable c = ''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":564,"value":"mutable d = ''","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":27,"value":"html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W30","stdlib":"1","titling":"auto"}} 2d:"$2c:body"