21:null 22:["$","title",null,{"children":"Building a SMART FARM EP06: Productionization (custom domain, error monitoring) | Tom Larkworthy | Observable"}] 27:T542,So far our SMART FARM has data storage provided by Firebase ([EP01](https://observablehq.com/@tomlarkworthy/livecoding-2022-06-07)), and it is actively ingesting data from real farms in Colombia using Blues Wireless cellular connection ([EP02](https://observablehq.com/@tomlarkworthy/blueswireless-2022-06-14)). We are able to send dashboards to field workers on-demand using WhatsApp ([EP03](https://observablehq.com/@tomlarkworthy/dashboards-over-whatsapp)). Last time in ([EP04](https://observablehq.com/@tomlarkworthy/firebase-to-duckdb?collection=@tomlarkworthy/livecoding)) we calculated the *Growing Degree Days* (GDD) using DuckDB SQL and optimized it with a lambda architecture in ([EP05](https://observablehq.com/@tomlarkworthy/lambda-architecture)) In this episode, we cover some generic topics required to take an Observable based application into production. From scratch, we register a custom domain with [Netlify](https://netlify.com/) and serve a custom interactive dashboard of our production data in under 30 minutes! Later we walkthrough how to monitor our user sessions for unexpected errors using [Sentry](https://sentry.io/) and then how to continuously monitor a notebook for errors using [healthcheck](https://observablehq.com/@endpointservices/healthcheck) and [uptimerobot](https://uptimerobot.com/?rid=ea2c825277fe40).28:[] 25:{"id":"87bcb9c5d7171ab6","slug":"productionization","title":"Building a SMART FARM EP06: Productionization (custom domain, error monitoring)","thumbnail":"8853a56389c6d1676cd7d14fed1811cb88c00f6a2652f63152f7a043181f5023","access_level":"public","create_time":"$D2022-07-26T16:11:46.326Z","update_time":"$D2022-08-02T10:18:15.439Z","view_time":null,"first_public_time":"$D2022-07-27T11:23:30.297Z","has_unpublished_edits":null,"fork":{"id":"88880374fa37e852","slug":"lambda-architecture","title":"Building a SMART FARM EP05: Lambda Architecture","owner":{"id":"7db5ed2b0697d645","login":"tomlarkworthy","name":"Tom Larkworthy"}},"stars":5,"starred":false,"owner":{"type":"team","id":"7db5ed2b0697d645","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":"starter_2024"},"version":2766,"publish_version":2766,"role":null,"authored":false,"authors":[{"type":"individual","id":"5215f6ec4a999d40","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":null}],"members":[{"type":"individual","id":"7ff6bd0b10289728","login":"gcoch","name":"Gabriel Coch","avatar_url":"https://avatars-next.observableusercontent.com/9dd1ebc4f82239794f8c01fe1de0ea2d82a40676208a72285db42c35cfaac423","tier":null,"role":"editor"},{"type":"individual","id":"5215f6ec4a999d40","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":2,"collection":{"id":"2fcd61930356b007","slug":"livecoding","title":"Building a SMART FARM","owner":{"id":"7db5ed2b0697d645","login":"tomlarkworthy","name":"Tom Larkworthy"}},"type":"notebook","body":{"titling":"auto","title":"Building a SMART FARM EP06: Productionization (custom domain, error monitoring)","cells":[{"id":0,"value":"# Building a SMART FARM EP06: Productionization (custom domain, error monitoring)\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":365,"value":"$27","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1092,"value":"## Full episode\nSorry the video stream lags in the last half, the audio is good though, I am working on a fix for the video long term!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":343,"value":"","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2758,"value":"videoStartAt","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":643,"value":"viewof videoStartAt = Inputs.select(\n new Map([\n [\"Start\", 1],\n [\"Updates from TechHub Live (AgTech conference)\", 30],\n [\"Simple embedding\", 550],\n [\"Serving a website\", 670],\n [\"Whitelabelled Custom embedding\", 828],\n [\"Custom Domain (Netlify)\", 1165],\n [\"Performance tips\", 1738],\n [\"Error Monitoring (Sentry)\", 2044],\n [\"Active Monitoring (healthcheck + uptimerobot)\", 2558],\n [\"Next steps\", 3323]\n ]),\n {\n label: \"Jump to a location of interest\"\n }\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":85,"value":"## (EP01) Firebase Database\nSetup in [EP01](https://observablehq.com/@tomlarkworthy/livecoding-2022-06-07)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":61,"value":"import {\n firebase,\n viewof user,\n listen\n} with { firebaseConfig as firebaseConfig } from \"@tomlarkworthy/firebase\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":57,"value":"firebaseConfig = ({\n apiKey: \"AIzaSyA9OQ--0Sn8Gm6C1B4M-SbnxRMtKHgnHzs\",\n authDomain: \"agropatterns-iot.firebaseapp.com\",\n databaseURL: \"https://agropatterns-iot-default-rtdb.firebaseio.com\",\n projectId: \"agropatterns-iot\",\n appId: \"1:650132950601:web:115b1ba65750d2b0aaf998\",\n uiConfig: {\n // https://github.com/firebase/firebaseui-web#configuration\n signInOptions: [\"anonymous\"]\n }\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":87,"value":"db = firebase.database()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":176,"value":"## (EP02) Live Temperature Dashboard\n\nSetup in [EP02](https://observablehq.com/@tomlarkworthy/blueswireless-2022-06-14). This is 24 hour window of latest sensor readings. The green band represents the growth zone for the plant. Most sensors are sending data every 15 minutes which will be reflected in the dashboard automatically.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":157,"value":"### (EP02) streaming query (realtime updates)","pinned":true,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":159,"value":"liveViewRaw = Generators.observe((notify) => {\n db.ref(`history/sensors/${querySensorName}`)\n .orderByKey()\n .startAt(Math.floor(Date.now() / 1000 - 60 * 60 * 24) + \"\") // reduce data returned\n .on(\"value\", (snapshot) => {\n const data = snapshot.val();\n if (data) notify(data);\n });\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":166,"value":"liveView = Object.entries(liveViewRaw).map(([k, v]) => v)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":372,"value":"latest = Generators.observe((notify) => {\n db.ref(`current/sensors`).once(\"value\", (snapshot) => {\n const data = snapshot.val();\n // clear synthetic data from EP01 testings\n delete data[\"sensor2\"];\n delete data[\"defaultSensorName\"];\n notify(data);\n });\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2696,"value":"### (EP02) Interactive live data Dashboard","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":140,"value":"viewof querySensorName = Inputs.select(Object.keys(latest), {\n label: \"Choose sensor to chart\",\n value: \"dev:864475046458393\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":164,"value":"dashboard = Plot.plot({\n x: { type: \"time\" },\n y: { domain: [0, 40], label: \"Temperature\" },\n color: {\n domain: [0, 40],\n scheme: \"turbo\"\n },\n marks: [\n Plot.rect([{}], { fill: \"#F4BFA6\", y1: 25, y2: 40 }),\n Plot.rect([{}], { fill: \"#A7D3A6\", y1: 5, y2: 25 }),\n Plot.rect([{}], { fill: \"#A4B5E3\", y1: 0, y2: 5 }),\n Plot.ruleY([20], { stroke: \"#0003\", strokeDasharray: [5] }),\n Plot.lineY(\n liveView.filter((d) => d.body.temperature), // Fix for unfiltering data serverside\n {\n stroke: \"#0003\",\n x: (d) => new Date(d.when * 1000),\n y: (d) => d.body.temperature\n }\n ),\n Plot.dot(\n liveView.filter((d) => d.body.temperature), // Fix for unfiltering data serverside\n {\n r: 5,\n x: (d) => new Date(d.when * 1000),\n y: (d) => d.body.temperature,\n fill: (d) => d.body.temperature\n }\n )\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2500,"value":"## Custom Webpage Server","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2504,"value":"import { endpoint } from \"@endpointservices/webcode\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2502,"value":"endpoint(\"website\", async (req, res) => {\n res.send(sensor_page.outerHTML);\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2700,"value":"### Custom webpage source (in a HTML cell)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2756,"value":"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2508,"value":"\n\n \n

Agropatterns V2

Sensor page

\n\n","pinned":true,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"sensor_page","database":"$undefined"},{"id":2523,"value":"## Custom Domain\n\nWe registered a domain through netlify, the created a site that simply redirects to our custom webserver endpoint. We use Netlify's GitOps workflow to update the site, our only file is the *netlify.toml* that configures Netlify as a reverse proxy, find ours [here](https://github.com/tomlarkworthy/QAFaLWyJ/blob/main/netlify.toml)\n\nAfter configuring netlify, our custom domain points to our custom dashboard: [https://qafalwyj.com/](https://qafalwyj.com/)\n\nLinks to further resources\n- [Advanced Observablehq embedding](https://observablehq.com/@observablehq/advanced-embeds?collection=@observablehq/embed-trail)\n- [Custom domain in 4 lines-of-code](https://dev.to/tomlarkworthy/custom-domain-in-4loc-l6f)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2527,"value":"## Monitoring with Sentry\n\nAfter Sentry is installed, any errors thrown will be reported to the Sentry dashboard. It is important you test the integration! You can do this by executing `throw new Error(\"test\")` in the console. Remeber if you are using a custom domain, it needs to be allowlisted in the `'namespaces'` configuration.\n\nYou find the DSN number after creating a sentry Javascript project.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2529,"value":"import { sentry } from \"@endpointservices/sentry\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2531,"value":"mySentry = {\n console.log(\"sentry has been setup\");\n return sentry({\n DSN:\n \"https://a6079bd15b7c405bb6346e95a44f3ee8@o1335620.ingest.sentry.io/6603442\",\n namespaces: [\"tomlarkworthy\", /* IMPORTANT */ \"qafalwyj.com\"]\n });\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2553,"value":"## Active Monitoring with Healthcheck + Uptimerobot\n\nWith [`healthceck`](https://observablehq.com/@endpointservices/healthcheck) notebook, you can ask it to run a different notebook and detect errors. When you have that setup the check can be exported as a URL. You can then monitor that URL using off-the-shelf monitoring software like [uptimerobot](https://uptimerobot.com/?rid=ea2c825277fe40).\n\nSo if we have a known error:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2555,"value":"errorCell = {\n throw new Error(\"Known error\");\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2559,"value":"Calling the URL ['@endpointservices/healthcheck?target=%40tomlarkworthy%2Fproductionization&wait=5'](https://webcode.run/observablehq.com/@endpointservices/healthcheck?target=%40tomlarkworthy%2Fproductionization&wait=5) will detect the error and return a 503 status check.\n\nBy plugging that URL into [uptimerobot.com](https://uptimerobot.com/?rid=ea2c825277fe40) we can check for errors every 5 minutes or so and be alerted if errors are raised. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":585,"value":"## Next episode","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":616,"value":"upcoming","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":453,"value":"## Dependancies","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":592,"value":"import { upcoming } from \"@endpointservices/meetups\";","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":206,"value":"import { footer } from \"@endpointservices/footer\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":227,"value":"footer","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W28","stdlib":"1"}} 26:"$25:body"