2f:T919,#### Example fine tune 2 Would be nice but expensive: - `classification_context_10:` "Yang et al. (2018) also argued that rank(A ) is as high as vocabulary size V based on the following two assumptions: 1. Natural language is highly context-dependent. In addition, since we can imagine many kinds of contexts, it is difficult to assume a basis that represents a conditional probability distribution for any contexts. In other words, compressing U is difficult. 2. Since we also have many kinds of semantic meanings, it is difficult to assume basic meanings that can create all other semantic meanings by such simple operations as addition and subtraction; compressing V is difficult. In summary, Yang et al. (2018) indicated that D h N is much smaller than rank(A) because its scale is usually 10 2 and vocabulary size V is at least 10 4 . ---------------------------------- **PROPOSED METHOD: DIRECT OUTPUT CONNECTION** To construct a high-rank matrix, Yang et al. (2018) proposed Mixture of Softmaxes (MoS). MoS computes multiple probability distributions from the hidden state of final RNN layer h N and regards the weighted average of the probability distributions as the final distribution." - `gold_context:` "Yang et al. (2018) also argued that rank(A ) is as high as vocabulary size V based on the following two assumptions: 1. Natural language is highly context-dependent. In addition, since we can imagine many kinds of contexts, it is difficult to assume a basis that represents a conditional probability distribution for any contexts. 2. Since we also have many kinds of semantic meanings, it is difficult to assume basic meanings that can create all other semantic meanings by such simple operations as addition and subtraction; compressing V is difficult. In summary, Yang et al. (2018) indicated that D h N is much smaller than rank(A) because its scale is usually 10 2 and vocabulary size V is at least 10 4." Cheaper version - Only train `gpt3.5` to detect ["differences background", "background differences", "differences", "differences motivation", "differences similarities motivation", "uses differences similarities", "extends differences background", "extends differences motivation"] (n=687; mean average token count=41)30:T465,

His[ERRINGTON] work is only casually mentioned by W.-E. On the whole, considering the enormous size of the book and the many rather peripheral topics treated, it is surprising how incompletely the really pertinent points are reviewed.{'neg': 0.0, 'neu': 0.899, 'pos': 0.101, 'compound': 0.4215}
I think it may be said of the book, with a certain amount of truth, that most of what is sound in it is not new, and most of which is new is not sound.{'neg': 0.0, 'neu': 0.882, 'pos': 0.118, 'compound': 0.5267}
This in itself might not have been a very serious objection.{'neg': 0.137, 'neu': 0.863, 'pos': 0.0, 'compound': -0.1513}
If it had contained a good, well-digested, and balanced summary of facts and views on animals density regulation in relation to social behaviour, together with a clear statement of the problems (and on this foundation a statement of the more extreme theories of the author), it might have been a highly useful book.{'neg': 0.045, 'neu': 0.811, 'pos': 0.144, 'compound': 0.7178}

31:[{"id":"d92bf93eb74551b11d35b56fea6643c9a3b72a182cd45ba6dbcf51c630600bfa65fd3480bf98d7e67f8651711245ab989c8a9cc295f140618b9abe4f3973b3fb","href":"https://static.observableusercontent.com/files/d92bf93eb74551b11d35b56fea6643c9a3b72a182cd45ba6dbcf51c630600bfa65fd3480bf98d7e67f8651711245ab989c8a9cc295f140618b9abe4f3973b3fb","name":"image@2.png","type":"image/png","lastModified":1700569088849.984,"size":17883},{"id":"e15d2404aff89e1e578e686aad5e0265ae8451481adf8d9ece1d95b4221778fc2d5ca181ab0ba9e530ff7f84ec60ea1f58302e8d29e8d2e65f2f774cd510cecc","href":"https://static.observableusercontent.com/files/e15d2404aff89e1e578e686aad5e0265ae8451481adf8d9ece1d95b4221778fc2d5ca181ab0ba9e530ff7f84ec60ea1f58302e8d29e8d2e65f2f774cd510cecc","name":"image@4.png","type":"image/png","lastModified":1700569088849.984,"size":59913},{"id":"c4a0230c74dc9aec86ca35e96c8005ef10654f93b06b138b0d889b8fa24233a0473eeb164ffc925f1dff5944cef21ed4df54e30bfb9152da7d03bbd2a12de1f7","href":"https://static.observableusercontent.com/files/c4a0230c74dc9aec86ca35e96c8005ef10654f93b06b138b0d889b8fa24233a0473eeb164ffc925f1dff5944cef21ed4df54e30bfb9152da7d03bbd2a12de1f7","name":"image@3.png","type":"image/png","lastModified":1700569088849.984,"size":22973},{"id":"6afbc4ef17e85bdb750fc707d3c05dceb96b51b3d975d8e5cb32fa247118e45b8493677a0e61d73d2bf44ff4e980d94484d9a06272862fdf7fca8b55bfef9a95","href":"https://static.observableusercontent.com/files/6afbc4ef17e85bdb750fc707d3c05dceb96b51b3d975d8e5cb32fa247118e45b8493677a0e61d73d2bf44ff4e980d94484d9a06272862fdf7fca8b55bfef9a95","name":"image@6.png","type":"image/png","lastModified":1700569088849.984,"size":31794},{"id":"4319fe43b8b4c7f84bb4304516676aaf2bf6d09175d0577c5ba216e714dbabf550272ad5522807133f46aba4e9d44c4851effeded6c7d080ac1fd61d947d50c3","href":"https://static.observableusercontent.com/files/4319fe43b8b4c7f84bb4304516676aaf2bf6d09175d0577c5ba216e714dbabf550272ad5522807133f46aba4e9d44c4851effeded6c7d080ac1fd61d947d50c3","name":"image@5.png","type":"image/png","lastModified":1700569088849.984,"size":99602}] 2d:{"title":"Data pipeline course catalogue","cells":[{"id":0,"value":"# Data pipeline course catalogue","pinned":false,"hidden":false,"mode":"md"},{"id":4,"value":"mermaid`graph LR\nsubgraph Data-Collection\n direction TB\n a[\"START\"] -->|Collect UVM PDFs|b[Raw PDFs]\nend\nsubgraph Get boxes\n direction TB\n b -->|OCR + bounding boxes\\nusing\\nmask_rcnn_X_101_32x8d_FPN_3x|c(Boxed OCR)\n c -->|Annotate Boxes|d(More\\nannotated\\nBoxes)\n d -->|Get better boxes\\nusing\\nfast_rcnn_R_50_FPN_3x|c\n d -->|Train\\nusing\\nlayoutlmv3-base|e(Predict\\nbox categories)\nend\nsubgraph Get course entries NER\n direction TB \n b -->|OCR|f(OCRed data)\n f -->|Annotate text|g(Annotated\\ncourse\\nentries)\n g -->|fine-tune\\nLLAMA2 7B|h(Fine-tuned llama2)\nend\nsubgraph Bayesian-Model\n direction TB \n e -->i(Clean\\ncourse data)\n h -->i\nend\n`\n// classDef Textclass fill:#a162","pinned":true,"hidden":false,"mode":"ojs"},{"id":251,"value":"Fine-tuning gpt3.5\n\n![image@6.png](${await FileAttachment(\"image@6.png\").url()})","pinned":false,"hidden":false,"mode":"md"},{"id":255,"value":"What if we follow the following pipeline:\n - Fine tune #1: Fine tune on stance detection using dataset from Beese et al (2023;l n=1265 if we only show (sent < -.1 | sent > 0.1)). \n - Fine tune #2: Contextual citation analysis (CCA) on my hand-labelled dataset, without sentiments (since we use dataset in #1). The idea we'll be to extract something similar than discussed in Lauscher et al. (2022). Or we use gold context of Lauscher et al. 2022 to reverse engineer the CCA.","pinned":false,"hidden":false,"mode":"md"},{"id":264,"value":"$2f","pinned":false,"hidden":false,"mode":"md"},{"id":222,"value":"## VADER example","pinned":false,"hidden":false,"mode":"md"},{"id":215,"value":"$30","pinned":false,"hidden":false,"mode":"html"},{"id":232,"value":"## More examples","pinned":false,"hidden":false,"mode":"md"},{"id":234,"value":"image5 = FileAttachment(\"image@5.png\").image()","pinned":true,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}