28:null 29:["$","title",null,{"children":"Entry 4: Data Arrangers | Lila Conlee | Observable"}] 2e:T2cc7,

Topic No.	Topic Label	Keywords	Theme Category
1	Tinnitus Symptoms	ringing, buzzing, tinnitus, sound, hear, constant, continue, hissing, severe, ear	Tinnitus
2	Loud Music	loud, music, earphone, listen, eardrum, affect, damage, cause, headset, longtime	Noise-induced Hearing Loss
3	Steroids for Sudden Deafness	steroid, injection, treatment, drug, pill, prescription, shot, sudden, deafness, effect	Sudden Hearing Loss
4	Symptoms of Otitis Media	symptoms, otitis media, pain, headache, sore throat, infection, deafen, today, morning,	Otitis Media
5	Hearing Test Results	diagnosis, treatment, cure, curious, communication, hospital, learning, disability, quality, doubt	General Inquiry
6	Tinnitus Symptoms	tinnitus, loud, noise, hear, sleep, night, worse, stress, ear, feel, tired	Tinnitus
7	Complications	complication, cause, chronic, diabetes, anemia, cancer, thyroid, hepatitis, kidney, disease	Complications
8	Degree of Disability	handicap, disability, degree, disabled, support, experience, hearing, aids, cochlear, implant	Hearing Aids
9	Dizziness	dizzy, cause, awake, morning, severe, pain, balance, nausea, sudden	Dizziness
10	Otitis Media Treatments	surgery, hospital, medication, otitis media, medicine, doctor, condition, after, hearing, test	Otitis Media
11	Cochlear Implant	cochlear, implant, surgery, cost, normal, restore, wear, test, support, wonder	Hearing Aids
12	Health Screening	health, screening, increase, urine, test, radiation, surgery, complication, infection, hospital,	Complications
13	Diagnosis of Sudden Deafness	sudden, loss, diagnosis, cause, hospital, test, results, nose, ear, infection, damage	Sudden Hearing Loss
14	Headache	headache, stress, frequent, dizziness, symptom, discomfort, ménière, disease, nerve, brain	Dizziness
15	Tinnitus Treatments	tinnitus, treatment, hospital, recommend, food, symptoms, effects, audiologist, pain, doctor	Tinnitus
16	Hearing Aids Recommendation	Hearing, aids, recommend, brand, place, good, hospital, uncomfortable, price, impaired	Hearing aids
17	Treatment and Side Effects	treatment, sudden deafness, medicine, right, ear, left, test, normal, restore	Sudden Hearing Loss
18	Shooting	shooting, military, gun, explosion, damage, hospital, degree, diagnosis, protection, accident	Noise-induced Hearing Loss
19	Causes of Tinnitus	tinnitus, infection, drug, earwax, loud, noise, stress, medication, cause, sick	Tinnitus
20	General Inquiry	stress, headache, serious, nerve, exercise, job, worry, cause, diet, decibel	General Inquiry
21	Treatment and Side Effects	side effect, steroid, drug, virus, hours, memory, worry, recovery	Sudden Hearing Loss

2f:T4d0,### [Visualization topics and keywords in a conversation](https://app.lvn.org/conversation/487) I'd previously worked with topic modeling in conversation transcripts with the Local Voices Network. This was built before my time, so I'm not tooting my own horn here. To assist in navigating the themes and highlights of a conversation, there's a visualization at the top of each transcript showing key words and where they appear in the transcript and a play btn to listen to those snippets of the conversation. You can also filter down those keywords by topic to see a subset of keywords. I think these visualizations are a interesting way to visualize topics over time. I thought about this a lot during our time entry and this visualization, which I've spent a lot of time with, never ever occurred to me. Now that I'm thinking about topic modeling I see that it's an interesting way to see topics over time and then zoom in deeper to specific topics in time you're interested in. I do feel like the interactivity was quite hidden away. Maybe the idea was if you're working with this visualization, you've worked closely with LVN and have gone through a training but still not very obvious the keywords have any interaction. 30:T6b4,## Visualizations To programmatically find "categories" in my search history, I used [BERTopic](https://maartengr.github.io/BERTopic/index.html) for topic modeling. I fed BERTopic a list of my search queries and optimized a few settings/parameters for how it should handle my specific data (I wanted something optimized for short text rather than full paragraphs or longer bodies of text, I specified that I was working with primarily English text). This is my first real attempt at this sort of thing. The methods I used are better suited for longer pieces of text, but thought I'd try it out. I'd be interested in how to tweak the model or if another method would be better for this type of data. I looked around and this seemed like the most approachable for me. I initially generated topics without any intervention, creating "good but not great" topics. Many unrelated terms such as proper nouns were grouped together which wasn't surprising. Many different topics emerged for what I consider the same topic such as data visualization or multiple sclerosis. To improve on this I did attempt creating a model with [seed topics](https://maartengr.github.io/BERTopic/getting_started/guided/guided.html) that I defined, but didn't see any meaningful improvement so I definitely have some work to do there. I ended up initially making a model with a few tweaks, then [merging similar topics](https://maartengr.github.io/BERTopic/api/bertopic.html#bertopic._bertopic.BERTopic.merge_topics). There were quite a few topics, so some remain un-merged. This produced the best result from my attempts, but still lots of room for improvement. The following visualizations are generated by BERTopic (which uses plotly):31:T5ef,### Topic bar charts From [`visualize_barchart()`](https://maartengr.github.io/BERTopic/api/bertopic.html#bertopic._bertopic.BERTopic.visualize_barchart) you can generate bar charts of selected topics and the words with the highest probability to have that topic. In this case, it shows the 8 most frequent topics in my search queries. Think this is a good summary of topics and their keywords, but plenty of room for improvement. Without additional information, it's not at all clear that topic probabilities for the keywords is what's represented on the x axis. The charts all have different x axes which is easy to miss. That choice feels ok if you're giving a very general view of the topics in the data set, but it's not effective for evaluating most used keywords throughout a data set. Some notes on the topics: - Topic 3: An exciting "proper nouns" topic. It feels like removing proper nouns could be the right thing if this is how BERTopic is going to handle them, but it does feel like you could miss out key names associated with topics. Honestly don't know when I searched for `greg`. The topic probabilities are quite low for this topic as well. - Topic 5: Really unclear why `astro` is a top keyword for what is a "programming" topic. I used the static site framework quite a bit and presumably did a bunch of "how to do X in astro" queries, but wouldn't have expected it to be higher than something like "react". Curious if the fact that "react" has other meanings could mean "astro" rises to the top.32:T492,### Topic distance From [`visualize_topics`](https://maartengr.github.io/BERTopic/api/bertopic.html#bertopic._bertopic.BERTopic.visualize_topics) you can see the distance between topics as well as their size and key words for each topic on hover. It generally did a good job with clustering, thought I don't understand some of the relationships. It did a good job clustering programming and data viz topics as well as food related topics. It grouped MS, clothes, and house goods topics which make a lot of sense to me based on when I was querying for them (all topics I would've searched heavily in spring 22) but curious how the order of the text comes into play in this analysis. An example of kind of confusing cluster is IIT, cats, and health insurance. I can make some guesses (I did a lot of searching about student health insurance, many of my cat queries were related to cat health) but that cluster isn't too close to my other health cluster. I think I could develop a better understanding of how distance/similarity works in these systems. I have a sense in my head, but don't feel like I could explain it well to another person and I'd like to get there. 33:T765,## Reflections - I feel pretty uncomfortable writing about topic modeling. It's something I've heard a lot about but this is my first attempt at topic modeling myself so I feel like I don't have the language or concepts quite right. This is kind of a downside to it being so easy to get running, I'm not exactly sure how it works but wow does it generate topics! - For identifying topics in a small dataset once, this is probably more trouble than it's worth. But if you're working with a lot of data or consistently adding, identifying, and re-evaluating topics it seems like this would be great. - It could be cool to try to maintain a topic model for your own search history. I've really enjoyed this project as it's allowed me to reflect on a year I don't have a perfect memory of, it would be kinda cool to have a summary of the big points without having to actually read through embarrassing search history. - I feel like often the reason for topic modeling is discovery, but in this case I didn't feel I discovered something but reflected/processed. - I wonder how this process is working for text including many languages, last time I heard it seemed difficult but I haven't checked back. It's not something I ran into while working on these visualizations. ### Technical - It's been a second since I've written Python and spent most of my time trying to get BERTopic installed and running. I needed to downgrade from the latest Python to 3.10 and run it on my newer Windows computer. On my older Mac it seemed to run into [memory issues](https://maartengr.github.io/BERTopic/faq.html#i-am-facing-memory-issues-help), which doesn't surprise me as my Mac is a mess. - Once it was running, really exciting to play around with! - This feels like something you could tweak for years. I'd like to get some more background knowledge to make smarter adjustments. Especially for short text. 34:[{"id":"b10ff369247280e75b73b9b1c7363a1c8cc10d402f63f1f1b7ee367fc39e8d25f51139b9b7125409e4c63c12238852bba78fa08a1f67548643cb2c2e0c1db26f","href":"https://static.observableusercontent.com/files/b10ff369247280e75b73b9b1c7363a1c8cc10d402f63f1f1b7ee367fc39e8d25f51139b9b7125409e4c63c12238852bba78fa08a1f67548643cb2c2e0c1db26f","name":"topics@1.tsv","type":"text/tab-separated-values","lastModified":1670453299417.952,"size":4820},{"id":"ab41159f637fa3f15e16aa21ec0fdb7191606e7d7ca1fe96fd92a7f5d35209252ea397d6c1b2a02020a0d86ea631802b90196e4cacdd30fc03a2f1c3beb406b0","href":"https://static.observableusercontent.com/files/ab41159f637fa3f15e16aa21ec0fdb7191606e7d7ca1fe96fd92a7f5d35209252ea397d6c1b2a02020a0d86ea631802b90196e4cacdd30fc03a2f1c3beb406b0","name":"Screen Shot 2022-12-11 at 12.32.39 PM.png","type":"image/png","lastModified":1670783663852.63,"size":175637},{"id":"84ff22afedc9058d200f4451b9779b376af57ed0e0a4bbe302399beddf30a790892db110a6749055d7aa7d678df5b3ddf565a8063ed705529fb6326e9c97f85e","href":"https://static.observableusercontent.com/files/84ff22afedc9058d200f4451b9779b376af57ed0e0a4bbe302399beddf30a790892db110a6749055d7aa7d678df5b3ddf565a8063ed705529fb6326e9c97f85e","name":"Screen Shot 2022-12-11 at 12.26.22 PM.png","type":"image/png","lastModified":1670783675942.347,"size":44592}] 2c:{"id":"f4eb55f90d521218","slug":null,"title":"Entry 4: Data Arrangers","thumbnail":"96fbfb7e3c040ae46ce9fc637516332475e3bb63671c211916cb924f4cee84bd","access_level":"public","create_time":"$D2022-12-06T17:40:44.833Z","update_time":"$D2022-12-11T18:41:02.297Z","view_time":null,"first_public_time":"$D2022-12-06T17:40:44.833Z","has_unpublished_edits":null,"fork":true,"stars":0,"starred":false,"owner":{"type":"team","id":"ece51d3b54f8848b","login":"lilaconlee-id","name":"Lila Conlee","avatar_url":"https://avatars-next.observableusercontent.com/f70c8419e599320522f0703aab6aa594bee34f70ef45f1b7dd509b427f4e5c35","tier":"starter_2024"},"version":157,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"c319b963ebd22b41","login":"lilaconlee-id","name":"Lila Conlee","avatar_url":"https://avatars-next.observableusercontent.com/f70c8419e599320522f0703aab6aa594bee34f70ef45f1b7dd509b427f4e5c35","tier":null}],"members":[{"type":"individual","id":"c319b963ebd22b41","login":"lilaconlee-id","name":"Lila Conlee","avatar_url":"https://avatars-next.observableusercontent.com/f70c8419e599320522f0703aab6aa594bee34f70ef45f1b7dd509b427f4e5c35","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":{"id":"b9d94c444f3e158f","slug":"data-journal","title":"📔 Data Journal: Visualizing Browser History","owner":{"id":"ece51d3b54f8848b","login":"lilaconlee-id","name":"Lila Conlee"}},"type":"notebook","body":{"titling":"auto","title":"Entry 4: Data Arrangers","cells":[{"id":0,"value":"# [Entry 4: Data Arrangers](https://observablehq.com/d/682c41056f5c49db?collection=@zachpino/id-fall-2022-communication-systems-workshop)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":5,"value":"## Contents\n\n1. [Questions](https://observablehq.com/d/aa15645aeb414e02#cell-8)\n3. [Relevant visualizations](https://observablehq.com/d/f4eb55f90d521218#cell-11)\n4. [Data](https://observablehq.com/d/f4eb55f90d521218#cell-13)\n5. [Visualizations](https://observablehq.com/d/f4eb55f90d521218#cell-15)\n6. [Resources](https://observablehq.com/d/f4eb55f90d521218#cell-17)\n7. [Reflections](https://observablehq.com/d/f4eb55f90d521218#cell-19)\n8. [Hypothesis](https://observablehq.com/d/f4eb55f90d521218#cell-21)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":8,"value":"## Questions\n- Can I create \"categories\" or groupings to see broader themes of queries without doing the manual categorization/coding I did for domains in previous entries?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":11,"value":"## Relevant \"visualizations\"\n### [Topic Modeling for Analyzing Patients’ Perceptions and Concerns of Hearing Loss on Social Q&A Sites: Incorporating Patients’ Perspective](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7503893/)\nMaybe not a visualization, but this was a cool find so I'm including it. Researchers identified topics from patient-authored questions about hearing loss on a Q&A site. This is basically what I was interested in doing, but for my own search queries. The goal of this work was to develop \"a thorough understanding of patients' perspectives, thereby leading to better development of the patient-centered intervention\". \n\nI'm not in love with the idea of generated topics as a primary way of understanding patient perspectives, but it certainly seems like a good tool to have in the toolbox. (Note the authors didn't suggest it should be the primary way, just something I was thinking about.)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":50,"value":"#### Topicis resulting from the LDA","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":43,"value":"$2e","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":144,"value":"$2f","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":148,"value":"screenShot20221211At123239Pm = FileAttachment(\"Screen Shot 2022-12-11 at 12.32.39 PM.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":147,"value":"screenShot20221211At122622Pm = FileAttachment(\"Screen Shot 2022-12-11 at 12.26.22 PM.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":156,"value":"_Screenshots of the [Local Voices Network website](https://app.lvn.org/conversation/487) showing a timeline of keywords in a conversation and the same visualization filtered by topic_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":13,"value":"## Data\nGetting the data was easy this week! When I updated my previous entries to include my [mobile Safari browser history](https://observablehq.com/d/8895398a42abce25?collection=@lilaconlee-id/data-journal) I did a bit of cleaning to get the datasets in the same structure. I needed a list of both browser's search queries in a txt or csv and I had that read to go.\n\nI think had I done a bit more data cleaning before I shoveled things into BERTopic I might've been better off. Potentially removing single word queries or queries that were entirely numbers would've produced better results. I did a little bit of this as weird queries had high probabilities for a topic, but not consistently.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":15,"value":"$30","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":81,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":83,"value":"