31:T5a3,However, looking at pretty much any sample paper and paragraph, we can see immediately that sentiment analysis completely fails at capturing modality. This makes sense because the way an author might assign positive or negative judgment to a previous work _isn't sentimental_; it's much more of a calculated affair. Here are some salient examples, some of which show why sentiment analysis is bad here: - Paper 1, Para 0 (Gene Therapy): Cites 6 sources on functional cationic polymers, and this sentences has a relatively high sentiment score (0.50), but the next sentence says "However, these non-viral gene carriers still have shortcomings, including cytotoxicity, low transfection efficiency and lack of multifunction." This is a problem about recognizing anaphora. So it's not necessarily a negative modality, but the authors are also trying to point out a serious gap. - Paper 12, Para 20 (Chinese Universities): neutral reviews of related work are scored quite differently, possibly because one sentences uses the phrase "lack of exchange." - Paper 50, Para 3 (Orthogonal frequency division multiplexing): Sentence 0 references the Nyquist-Shannon sampling theorem without a citation, showing that it is a solidified black box. Sentence 1 says that "the field has exploded" and points to 2 papers as "pioneering work". The associated sentiment score is 0.60, which is good because this should be regarded as a highly positive modality. 32:Tcb6,## Redefining How We Might Look for Modality So, off-the-shelf sentiment analysis doesn't work well at all for approximating citation modality. Moreover, segmenting text by periods also frequently fails (e.g. if "e.g." or "et al." is used) and there's not a one-size-fits-all alternative delimiter I can use. In addition, I see that negative modality in particular applies across sentences (c.f. [anaphora](https://en.wikipedia.org/wiki/Anaphora_(linguistics))). As an alternative, I could consider a _window_ of text around a citation. Unfortunately, because I'm out of time, I won't be able to implement this concept, but I'll describe what I would to. In terms of gauging modality, I can opt for a very rough heuristic-based classifier. Latour's discussion of modality in _Science in Action_ traces the various ways that parties can add modality that aren't immediately apparent. For example, the sentence "the only way to quickly produce efficient fuel cells is to focus on the behaviour of electrodes" in the context of a debate around fuel cells is a means of favoring scientists who study metallurgy or materials science over physicists. These nuances are not easy to capture even by state-of-the-art NLP tools. Brainstorming a bit on what I might look for, that could be positive or negative: - Author names mentioned - "However" - "build", "add", "improve", "motivate" - "[no] existing work" - Citating many references at once, or citing a highly popular references, suggests an appeal to authority, a means of differentiating the current work, or, more rarely, an attempt to open a closed black box. - "Proposed", "interpretation" suggests that the authors are going to critique or reinterpret past work in some way Scrubbing more examples using the view, I started to notice one pattern: several citations occuring in one or two sentences, followed by a lack of citations in the following sentences. I'll invent three classes of these instances: - **Pivot**: a "however" or some flavor of "we are better" - **Appeal**: some form of appeal to the authority of all of those citations. In particular, I often see citations for the same paper appearing multiple times e.g. (paper 57 paragraph 28): "contributes to the symptoms of schizophrenia [45] , while too much stability contributes to the symptoms of obsessive-compulsive disorder [45] . ... favorable combinations of treatments for particular disorders [45] ." - **Incorporation**: a some discussion of the citations, or incorporating it into the authors' argument. For example (paper 7, paragraph 5): "The notion of a +adequate link is a common generalization of positive and alternating links 4 , introduced in [6] (see also [12] )." In the future, I'd like to scan the corpus for these instances. Overall, the goal of this investigation is to show what it would look like to trace discussions over text and time to see how statements evolve into fact. From my very initial investigation, I learned that this isn't a simple task, and many contingencies from text delimiting to the difficulty of determining modality make the possibility of computational Latourian analysis difficult. Nonetheless, it could offer a promising line of work for making sense of research literature. 33:T61c,{ const container = d3 .create("svg") .attr("width", width + margin.left + margin.right) .attr("height", height + margin.top + margin.bottom); const tooltip = d3 .select("body") .append("div") .attr("class", "tooltip") .style("opacity", 0); const chart = container .append("g") .attr("id", "chart") .attr("transform", `translate(${margin.left}, ${margin.top})`); chart .append("g") .attr("class", "axis axis--x") .attr("transform", `translate(0, ${height})`) .call(d3.axisBottom(xScale)); const yScale = d3.scaleLinear().domain([0, 2]).rangeRound([height, 0]); const data = metasWithBt.map((meta) => meta.year); const stacks = generateDotplotStacks(data, binwidth); const bins = chart .selectAll(".gBin") .data(stacks) .join((enter) => { const g = enter .append("g") .attr("class", "gBin") .attr("transform", (d) => `translate(${xScale(d.x)}, ${height})`); g.selectAll("circle") .data((d) => d.values.map((p, i) => { return { idx: i, value: p, radius: 6 }; }) ) .join((enter) => enter .append("circle") .attr("cx", 0) .attr("cy", (d) => -d.idx * 4 * d.radius - d.radius * 1.5) .attr("r", (d) => d.radius) .attr("stroke", "black") ); return g; }); // chart.append("path").attr( // "d", // d3.line([ // [0, 0], // [200, 200] // ]) // ); return container.node(); }34:[{"id":"5f414e7b7b5d4ca12aae49d0b2bf55474aec76f044ed98936c1b96c83dc210401b0ba48484a1d7a3f8c12ffb3dc2ab9a70c44e0e677ba2be417cfd7763de0040","href":"https://static.observableusercontent.com/files/5f414e7b7b5d4ca12aae49d0b2bf55474aec76f044ed98936c1b96c83dc210401b0ba48484a1d7a3f8c12ffb3dc2ab9a70c44e0e677ba2be417cfd7763de0040","name":"cse599d-paper-data.json","type":"application/json","lastModified":1666627691878.786,"size":8768940},{"id":"4cffb7e1c38bb2f3ae4042c39cdeac8d2c8035389b05f1d4a187393693883f60ea7e0c04a3cd306d50d85688dfd1b56ec02843b8f4ba459e059d2abf7716fe90","href":"https://static.observableusercontent.com/files/4cffb7e1c38bb2f3ae4042c39cdeac8d2c8035389b05f1d4a187393693883f60ea7e0c04a3cd306d50d85688dfd1b56ec02843b8f4ba459e059d2abf7716fe90","name":"s2orc_sample.jsonl","type":"application/octet-stream","lastModified":1666629662494.442,"size":4750764},{"id":"32808ff27dbce318d743aed298de3c86fdfb13a49ad72c402c073658b08678fc380c44a86fab4278541ec9c9c957b39ddf7c055c30780ec6c960bb006e851d30","href":"https://static.observableusercontent.com/files/32808ff27dbce318d743aed298de3c86fdfb13a49ad72c402c073658b08678fc380c44a86fab4278541ec9c9c957b39ddf7c055c30780ec6c960bb006e851d30","name":"metadata.jsonl","type":"application/octet-stream","lastModified":1667076041219.066,"size":1393195},{"id":"97f45d71d2113614de9b41415c842072ea3e42042b47fdf51b7e2c7ced62861eaea4602fbe3185226c080856df88406c76813a69ce8e980b1fbef4d4dea63d86","href":"https://static.observableusercontent.com/files/97f45d71d2113614de9b41415c842072ea3e42042b47fdf51b7e2c7ced62861eaea4602fbe3185226c080856df88406c76813a69ce8e980b1fbef4d4dea63d86","name":"citation-generations.jpg","type":"image/jpeg","lastModified":1667192514517.66,"size":1338744},{"id":"1bf5860a39eb5a34a0406113f69a4b4c0468bdbc863e66c479e93ed0e1b6843d81e73d05e6e2275272739bf05715f59c48203455623c568243d79f404984c8c9","href":"https://static.observableusercontent.com/files/1bf5860a39eb5a34a0406113f69a4b4c0468bdbc863e66c479e93ed0e1b6843d81e73d05e6e2275272739bf05715f59c48203455623c568243d79f404984c8c9","name":"modality.jpg","type":"image/jpeg","lastModified":1667192476744.622,"size":1592609}] 2f:{"title":"Latourian Literature Analysis","cells":[{"id":0,"value":"# Latourian Literature Analysis","pinned":false,"hidden":false,"mode":"md"},{"id":947,"value":"## _An Embyronic Investigation_","pinned":false,"hidden":false,"mode":"md"},{"id":28,"value":"**Jasper Tran O'Leary**","pinned":false,"hidden":false,"mode":"md"},{"id":618,"value":"In this exercise, my intention was to explore some of the ideas of Bruno Latour, who was a philosopher of science who passed away earlier this month. I have been reading _Science in Action_, which is a study of what he terms \"science in the making\"—how things actually get done in science and in scholarly communication—rather than popular conceptions of \"ready-made science.\"\n\nI chose to focus on Latour's notion of _modality_ which is how scientists comment on and reconceptualize cited works and concepts. Latour argues that through this process, papers are shaped not in and of their own claims, but instead by what future authors say or don't say about them, either becoming fact or forgotten. A consequence of this assertion is that scientific writing is the _most_ persuasive form of communication because it enlists so many tools to withstand potential falsification by future scientists.","pinned":false,"hidden":false,"mode":"md"},{"id":626,"value":"citationGenerations = FileAttachment(\"citation-generations.jpg\").image({\n width: 600\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":632,"value":"To analyze modality, I needed to use a corpus that included structured text around citations, which almost every API for processing scientific papers lacks. To achieve this, I used a sample of the Allen Institute for AI's [s2orc dataset](https://github.com/allenai/s2orc) since the full dataset, after requesting and being granted a download link, was over 800GB.","pinned":false,"hidden":false,"mode":"md"},{"id":638,"value":"After parsing the .JSONL files, we get items (papers) with body texts (paragraphs) where body texts (sometimes) have associated citation start and end points in the paragraph alongside a link to that paper's bibliographic data, which we can use to link back to other papers. However, the actual paper title, authors, and other data are stored in a separate collection.","pinned":false,"hidden":false,"mode":"md"},{"id":99,"value":"items = s2orc_sample\n .split(\"\\n\")\n .filter((line) => !!line)\n .map(JSON.parse)","pinned":false,"hidden":false,"mode":"ojs"},{"id":194,"value":"metadata = s2orc_metadata.text().then((text) => {\n return text\n .split(\"\\n\")\n .filter((line) => !!line)\n .map(JSON.parse);\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":640,"value":"Excluding papers without body texts, and thinning the metadata collections down to just include the metadata for these papers, we end up with our effective corpus of 92 papers.","pinned":false,"hidden":false,"mode":"md"},{"id":102,"value":"itemsWithBt = items.filter((item) => item.body_text.length > 0)","pinned":false,"hidden":false,"mode":"ojs"},{"id":199,"value":"metasWithBt = itemsWithBt.map((item) => {\n let itemId = item.paper_id;\n let metas = metadata.find((meta) => meta.paper_id === itemId);\n return metas;\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":653,"value":"For fun, a quick histogram of papers by year since 1990 (before that, only 3 papers or so).","pinned":false,"hidden":false,"mode":"md"},{"id":223,"value":"Plot.plot({\n x: {\n domain: [1990, 2022]\n },\n marks: [\n Plot.rectY(\n metasWithBt,\n Plot.binX({ y: \"count\" }, { x: \"year\", thresholds: 2022 - 1990 })\n ),\n Plot.ruleY([0])\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":912,"value":"## Interactively Browsing Papers, Paragraphs, and (Poorly Estimated) Modality","pinned":false,"hidden":false,"mode":"md"},{"id":910,"value":"After wrangling with the data, I created a simple exploratory tool that lets us view paragraphs, sentences, and citations associated with sentences.","pinned":false,"hidden":false,"mode":"md"},{"id":473,"value":"viewof paperIdx = Inputs.range([0, itemsWithBt.length - 1], {\n label: \"Paper Index\",\n step: 1,\n value: 1\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":465,"value":"viewof paragraphIdx = Inputs.range([0, someItemBtTexts.length - 1], {\n label: \"Paragraph Index\",\n step: 1\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":426,"value":"exampleBt = someItemBtTexts[paragraphIdx]","pinned":false,"hidden":false,"mode":"ojs"},{"id":434,"value":"htl.html`

${exampleBt.replaceAll(\n \".\",\n \".\\n\"\n)}

`","pinned":false,"hidden":false,"mode":"ojs"},{"id":914,"value":"As an initial proxy for modality, I used the [WinkNLP API's sentiment analysis](https://winkjs.org/) to assign sentiment scores to sentences. Theoretically, because I can link a sentence with its citations, I could assign those scores to citations and see which citations are most \"bullied\" versus most \"celebrated,\" so to speak.\n","pinned":false,"hidden":false,"mode":"md"},{"id":410,"value":"doc = nlp.readDoc(exampleBt)","pinned":false,"hidden":false,"mode":"ojs"},{"id":918,"value":"

Sentence-wise Sentiments for paper ${paperIdx} paragraph ${paragraphIdx}

","pinned":false,"hidden":false,"mode":"html"},{"id":925,"value":"_Some of this code is adapted from [WinkNLP's example notebook](https://observablehq.com/@winkjs/how-to-perform-sentiment-analysis)_","pinned":false,"hidden":false,"mode":"md"},{"id":437,"value":"{\n doc.sentences().each((s) => s.markup(beginTag(s), endTag(s)));\n return html`${\n \"\" + doc.out(its.markedUpText) + \"

Sentence	Score

\"\n }`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":923,"value":"

Associated Citations for paper ${paperIdx} paragraph ${paragraphIdx}

","pinned":false,"hidden":false,"mode":"html"},{"id":928,"value":"_Sometimes the citation data is absent even with clear citations in the sentence._","pinned":false,"hidden":false,"mode":"md"},{"id":470,"value":"{\n let rows = [];\n doc.sentences().each((s, sIdx) => {\n let sentiment = s.out(its.sentiment);\n let citations = getCitations(someItem, paragraphIdx, sIdx);\n let links = citations.map((c) => c.link);\n let names = citations.map((c) => c.title);\n rows.push(\n `${names.join(\". \")}${links.join(\n \". \"\n )}${sentiment}`\n );\n //s.markup(beginTag(s), endTag(s))\n });\n return html`${\n \"\" +\n rows.join(\"\") +\n \"

Paper Titles	Link IDs	Score

\"\n }`;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":935,"value":"## Problems","pinned":false,"hidden":false,"mode":"md"},{"id":930,"value":"$31","pinned":false,"hidden":false,"mode":"md"},{"id":623,"value":"modality = FileAttachment(\"modality.jpg\").image({ width: 600 })","pinned":false,"hidden":false,"mode":"ojs"},{"id":939,"value":"$32","pinned":false,"hidden":false,"mode":"md"},{"id":69,"value":"## Testing - Safe to Ignore Code Below this Line","pinned":false,"hidden":false,"mode":"md"},{"id":7,"value":"cse599dPaperData = FileAttachment(\"cse599d-paper-data.json\").json()","pinned":true,"hidden":false,"mode":"ojs"},{"id":107,"value":"cse599dPaperData[2].s2data.abstract.length","pinned":true,"hidden":false,"mode":"ojs"},{"id":83,"value":"doi = \"10.1145/3411764.3445648\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":71,"value":"doiResponse = fetch(`https://doi.org/${doi}`, {\n checkContentType: true,\n method: 'GET',\n headers: { Accept: 'application/vnd.citationstyles.csl+json' } // request in CSL-JSON format\n}).then(res => {\n if (res.status === 200) {\n return res.json();\n } else {\n return { status: res.status, message: 'DOI service responded with non-OK status code.' };\n }\n})","pinned":true,"hidden":false,"mode":"ojs"},{"id":85,"value":"pdfUrl = doiResponse.link[0].URL","pinned":true,"hidden":false,"mode":"ojs"},{"id":93,"value":"fetch(pdfUrl, { method: \"GET\" })","pinned":true,"hidden":false,"mode":"ojs"},{"id":67,"value":"### s2orc corpus Search","pinned":false,"hidden":false,"mode":"md"},{"id":32,"value":"s2orc_sample = FileAttachment(\"s2orc_sample.jsonl\").text()","pinned":true,"hidden":false,"mode":"ojs"},{"id":192,"value":"s2orc_metadata = FileAttachment(\"metadata.jsonl\")","pinned":true,"hidden":false,"mode":"ojs"},{"id":113,"value":"itemsWithBt[1]","pinned":true,"hidden":false,"mode":"ojs"},{"id":119,"value":"itemsWithBt[1].body_text[0]","pinned":true,"hidden":false,"mode":"ojs"},{"id":149,"value":"itemsWithBt[1].body_text[0].text","pinned":true,"hidden":false,"mode":"ojs"},{"id":138,"value":"itemsWithBt[1].bib_entries","pinned":true,"hidden":false,"mode":"ojs"},{"id":153,"value":"itemsWithBt[1].bib_entries.BIBREF1","pinned":true,"hidden":false,"mode":"ojs"},{"id":164,"value":"itemsWithBt[1]","pinned":true,"hidden":false,"mode":"ojs"},{"id":166,"value":"paperIds = itemsWithBt.map((item) => item.paper_id)","pinned":true,"hidden":false,"mode":"ojs"},{"id":169,"value":"citationIds = {\n let bibEntryTables = itemsWithBt.map((item) => item.bib_entries);\n let flatBibEntries = bibEntryTables\n .map((table) => Object.values(table))\n .flat();\n let ids = flatBibEntries.map((entry) => entry.link).filter((id) => !!id);\n return ids;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":179,"value":"citationIdsInDataset = paperIds.map((id) => paperIds.includes(id))","pinned":true,"hidden":false,"mode":"ojs"},{"id":155,"value":"## Todo visualizations\n- histogram of paper by publication year\n- Schally graph of citation directed edges by year, no modalities\n- might need to fill in the gaps by querying some api since we will have citations outside of the dataset\n- some sort of visualization to explain modality (positive, neutral, negative, tacit positive (reach))\n- modalized schally graph\n- visualize a single paper's modality graph\n- scrape modal schally graph paths—a path with many negative modalities over time suggests a controversial topic. many positive into one paper, or in a path, suggest black boxing","pinned":false,"hidden":false,"mode":"md"},{"id":190,"value":"items[0]","pinned":true,"hidden":false,"mode":"ojs"},{"id":211,"value":"years = metasWithBt.map((meta) => meta.year)","pinned":true,"hidden":false,"mode":"ojs"},{"id":269,"value":"margin = {\n return {\n top: 20,\n right: 50,\n bottom: 50,\n left: 20\n };\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":272,"value":"width = 800","pinned":false,"hidden":false,"mode":"ojs"},{"id":273,"value":"height = 400","pinned":false,"hidden":false,"mode":"ojs"},{"id":276,"value":"xScale = d3\n .scaleLinear()\n // .domain(d3.extent(metasWithBt, (d) => d.year))\n .domain([1990, 2022])\n .rangeRound([0, width])\n .nice()","pinned":true,"hidden":false,"mode":"ojs"},{"id":282,"value":"function generateDotplotStacks(data, binwidth) {\n data = data.sort((a, b) => a - b);\n const stacks = [];\n for (let i = 0; i < data.length; ) {\n const threshold = data[i] + binwidth;\n\n const stack = [data[i]];\n let j = i + 1;\n\n while (data[j] < threshold) {\n stack.push(data[j++]);\n }\n let v = (stack[stack.length - 1] - stack[0]) / 2;\n const diff = data[i] - data[i - 1];\n // if (diff > binwidth || i == 0) { // X_0 = -inf in the original algorithm\n // v = (stack[stack.length - 1] - stack[0]) / 2;\n // }\n\n stacks.push({\n values: stack,\n // \"x\": jStat.median(stack),\n // \"x\": (stack[0] + stack[stack.length - 1]) / 2\n x: stack[0] + v,\n v: v,\n diff: diff,\n threshold: threshold\n });\n i = j;\n }\n return stacks;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":284,"value":"binwidth = 1.25 * Math.sqrt(20 / metasWithBt.length)","pinned":true,"hidden":false,"mode":"ojs"},{"id":353,"value":"someItemRefs = Object.values(itemsWithBt[1].bib_entries)\n .map((entry) => entry.link)\n .filter((link) => !!link)","pinned":true,"hidden":false,"mode":"ojs"},{"id":541,"value":"someItem = itemsWithBt[paperIdx]","pinned":true,"hidden":false,"mode":"ojs"},{"id":265,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":367,"value":"someItemBts = someItem.body_text","pinned":true,"hidden":false,"mode":"ojs"},{"id":371,"value":"someItemBtTexts = someItemBts.map((bt) => bt.text)","pinned":true,"hidden":false,"mode":"ojs"},{"id":375,"value":"someItemBtCitations = someItemBts.map((bt) => bt.cite_spans)","pinned":true,"hidden":false,"mode":"ojs"},{"id":415,"value":"{\n let sentences = [];\n doc.sentences().each((s) => sentences.push(s.out(its.sentiment)));\n return sentences;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":505,"value":"function getBibsForSentence(sIdx, bt) {\n let sentences = bt.text.split(\".\");\n let sentence = sentences[sIdx];\n if (!sentence) {\n return [];\n }\n let beforeSentences = sentences.splice(0, sIdx);\n let beforeChars = beforeSentences.reduce((a, b) => a + b.length, 0);\n let afterChars = beforeChars + sentence.length;\n let citeSpans = bt.cite_spans;\n let citeSpansInBound = citeSpans.filter(\n (span) => span.start >= beforeChars && span.end <= afterChars\n );\n return citeSpansInBound;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":510,"value":"sentenceBibs = getBibsForSentence(5, someItemBts[paragraphIdx])","pinned":true,"hidden":false,"mode":"ojs"},{"id":507,"value":"someItemBts[paragraphIdx]","pinned":true,"hidden":false,"mode":"ojs"},{"id":553,"value":"function getCitations(item, pIdx, sIdx) {\n let sentenceBibs = getBibsForSentence(sIdx, item.body_text[pIdx]);\n let sentenceCitations = sentenceBibs.map(\n (bibEntry) => item.bib_entries[bibEntry.ref_id]\n );\n return sentenceCitations;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":567,"value":"getCitations(someItem, paragraphIdx, 5)","pinned":true,"hidden":false,"mode":"ojs"},{"id":559,"value":"someItem","pinned":true,"hidden":false,"mode":"ojs"},{"id":478,"value":"someItemBtCitations[paragraphIdx]","pinned":true,"hidden":false,"mode":"ojs"},{"id":485,"value":"itemsWithBt[paperIdx].bib_entries[\"BIBREF1\"]","pinned":true,"hidden":false,"mode":"ojs"},{"id":491,"value":"selectedPaper = itemsWithBt[paperIdx]","pinned":true,"hidden":false,"mode":"ojs"},{"id":382,"value":"## Imports","pinned":false,"hidden":false,"mode":"md"},{"id":450,"value":"### WinkNLP (using their tutorial examples)","pinned":false,"hidden":false,"mode":"md"},{"id":452,"value":"[Source.](https://observablehq.com/@winkjs/how-to-perform-sentiment-analysis)","pinned":false,"hidden":false,"mode":"md"},{"id":384,"value":"winkNLP = (await import(\"https://cdn.skypack.dev/wink-nlp\")).default","pinned":false,"hidden":false,"mode":"ojs"},{"id":386,"value":"model = (await import(\"https://cdn.skypack.dev/wink-eng-lite-web-model\"))\n .default","pinned":false,"hidden":false,"mode":"ojs"},{"id":396,"value":"nlp = winkNLP(model)","pinned":false,"hidden":false,"mode":"ojs"},{"id":399,"value":"its = nlp.its","pinned":false,"hidden":false,"mode":"ojs"},{"id":445,"value":"beginTag = function (s) {\n return ``;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":447,"value":"endTag = function (s) {\n return `${s\n .out(its.sentiment)\n .toFixed(2)}`;\n}","pinned":true,"hidden":false,"mode":"ojs"}],"files":"$W34","stdlib":"1","titling":"auto"}