2b:null 2c:["$","title",null,{"children":"Enterprise Data Centric Digitization | datagenous | Observable"}] 31:T6fc,__datageno.us__ and its engine, __dydra__, were originally adopted by a front line business unit to solve a customer facing problem. The Product Marketing department needed to streamline and digitize the process which generated technical product sheets for clients. The often manual process took over two weeks and was subject to human error. They understood that the faster they could generate accurate up to date marketing collateral, the faster they could field more orders and drive revenues for new products. So our client set out to solve this problem by creating a __"data hub"__ based on a semantic graph data to access [Product Lifecycle Managament](https://en.wikipedia.org/wiki/Product_lifecycle) data in a declarative low code environment. Using __dydra__, and supported by expertise to deliver its capabilities as a fully managed continuous integration service with expert support for data programming, they ingested data to rapidly transform and [link data](http://www.bobdc.com/blog/linkingkgs/), then expose data graphs as new data services and APIs ingestible into any application. This __"github for data"__ approach enabled our client to reduce the generation time for product sheets from weeks to minutes, and to do so based on data from the system of record thereby increasing accuracy and eliminating human errors. For this work the our client's Enterprise Data Hub, [was selected for European Linked Data Award](http://www.nxp.com/news/news-archive/2015/NXP-wins-the-first-European-Enterprise-Linked-Data-Award.html) amongst 55 submissions across 22 countries. This was only the beginning of the journey. 32:T4f1, This practice, known as [shadow IT](https://en.wikipedia.org/wiki/Shadow_IT) or [skunk works](https://en.wikipedia.org/wiki/Skunk_Works) has emerged in the context of front line business units feeling both: 1. empowered to innovate based on the capabilities enabled by the cloud 2. frustrated with traditional centralized IT departments which they feel struggle to keep up with their rapid and dynamic requirements For technology providers, the rise of shadow IT validates an approach that provides low cost, value-driven capabilities to users who need them and can't or don't want to wait. However, this new environment creates challenges for CIOs and traditional IT departments which: - have governance, security, and quality burdens that front line business units do not have insight into, - are under budgetary pressures to control IT costs, - need to carefully manage technical debt, - are already overburdened with requirements and - may be pushed outside their comfort zones For our client and __datageno.us__ this meant finding a way to transition shadow IT success to core enterprise capability, an issue driven by two additional data centric projects which demonstrated the value of data centric design to an increasing number of internal users. 33:T929,Businesses exist in a dynamic and competitive environment filled with unanticipated changes that require agile and deliberate responses. For our client such changes were driven by a couple of key business events for which __datageno.us__ provided unique capabilities. Our client acquired another semiconductor firm at the end of 2015 and began integrating business operations as a result of the merger. While both organizations had complementary product portfolios, they also had preexisting, heterogeneous IT systems and processes. This included their PLM systems. Driven by a requirement to provide unified views of the newly acquired product portfolios in an efficient manner, efforts were undertaken to integrate and merge data from these systems. Having previously succeeded in complex integration of PLM data for business processes in support of the Product Marketing Organization, the "data hub" approach was suggested as a path that could address the complexity of merging two different running PLM systems. - Within six weeks, two domain experts working with __dydra__ were able to create the data services to have a common, consolidated view of the PLM data and answer questions related to the merger - Compared to the classic approach where an 8-10 person consulting team would take ~9-24 months to deliver top down integrated data solution In the middle of this integration process our client began preparing for yet another merger and concurrent spinoff. So not only did the integration of PLMs need to happen, but at the same time a subsection of PLM data needed to be spun out for a new organization. Again using the data centric approach supported by __dydra__ Our client was able to: - Execute a sprint out in 3 months that overlapped with acquisition and integration of initial merger - Spin out organization, including 11,000+ employees, 85B+ devices with 10,000+ parts, for $2.75B to investors. As a result of these and other successes, __dydra__ was subjected to a full technical and security audit, asked to address core features such as SSO support, provide ISO certified private cloud with full redundancy, expand its ITIL based service desk, and conform to a defined MSA. After due diligence, __datageno.us__ was incorporated into the approved core architecture and today powers more than 20 applications. 34:T5e8,The __datageno.us__ perspective is based on the idea that _defining a process in terms of data is digitization_. __dydra__ is built to address the 85% or more of data-centric effort cost that goes into the black hole of wrangling. The __datageno.us__ managed service addresses this systemic need by exposing key data programming capabilities on a case by case basis while allowing organizations to avoid risky infrastructural investments in anticipation of data and control flow requirements. Those capabilities are the ability to: - identify, acquire, and unify data resources and streams - compose data workflows without affecting systems of record (logical data read layer) - iterate on data service development by providing out of the box version control and automated regression for schema and data payloads (e.g. github for data) - apply rules for data quality and augmentation (e.g. inference, mapping) - apply analytic feedback loops (e.g. ML, AI) - consolidate, federate, or distribute services (e.g. service/microservice segmentation) Furthermore, to make it easy to absorb these capabilities into an enterprise __datageno.us__ employs __dydra__ to support: - master data management automation though orchestration of data flow dependencies assisted by AI techniques - data governance infrastructure by default And because __dydra__ is delivered as part of the __datageno.us__ fully managed service, new capabilities and optimizations become continuously available based on client requirements. 35:T4ad,__datageno.us__ was developed by an experienced team familiar with the requirements of mission critical systems and enterprise governance, and was consciously developed to support both the edge requirements of business units working at speed, and the core requirements of centralized IT. As a result, __datageno.us__ powered services are designed to compound in value as they become composible and reusable parts of interconnected data networks and as the skill and maturity of your data practice increases. Because __datageno.us__ is built from the ground up to provide data mutability based on open standards, your data will never be locked in by anything other than utility. This means fitting into and complimenting existing investments in legacy systems for data warehousing, business intelligence, and ETL buses. Our motivation is simple, after decades of seeing the IT industry work to lock-in clients we wanted to give organizations the freedom to evolve autonomously. Organizations that can break free of their technical debt and reduce switching costs between systems will be happier, more competitive firms that can immediately generate recognizable business value for their efforts. 36:Ta95,The introduction of new design patterns and methods powered by __datageno.us__ enabled our client to tackle complex problems at lower cost and with more flexibility than had seemed possible. Part of this transition was based in the realization that digital twins are emergent reflections of lifecycles. They begin as conversations in R&D departments, get reflected in specifications, codified as tests, manufactured by machines in changing conditions, associated with identities lasered into physical products, subjected to changing environmental conditions, delivered though varied logistical chains, applied in varying scenarios. For our client the digital twin extended beyond a digital copy of a final product, but needed to be something more - a part of a genealogy, and an evolutionary product ontogeny. In this context waiting for a new ERP system, data warehouse, or SAP module, is secondary to the ability to begin incremental creation of digital twins based on insitu data across systems and locations in its supply chain. Instead of beginning with a system, this process began with a question: _A self-driving vehicle fails. What does that mean to your business if you are the leading manufacturer of intelligent sensors, transponders, controllers, vehicle network products and semiconductors, that enable the next generation of autonomous and connected driving solutions?_ Using __datageno.us__ approaches as a core component of their enterprise integration architecture, our client worked to define and trace quality issues across both complex, dynamic processes and legacy systems and the global supply chain to perform root cause analysis in real time. At each step new challenges were discovered that had to do with issues such as granularity, rate, frequency, and scale. __dategano.us__ empowered approaches allowed for identification and segmentation of the necessary services to produce the right automations at feasible levels of granularity. This led to smarter investments in the right kind of technology an infrastructure at the right point in time. It also allowed for the introduction of new concepts for predictive analytics, inference, index optimizations, and data pipeline management. Today the product traceability portal has identified its chosen technologies and serves as a vital resource that has accelerated what used to take several weeks to process into automated analytic queries executable in minutes. Moreover, the logical, evolutionary process powered by __datageno.us__ means that new extensions and levels of richness to digital twins can be rapidly absorbed into working models for which the right infrastructure can be identified and provisioned. 37:T416,Currently __datageno.us__ serves as an expert resource on hand to investigate and make possible new data centric design patterns. This includes projects to use: 1. AI techniques to [discover and adapt Master Data Management](https://observablehq.com/@datagenous/smithsonian-probes) 2. Bitemporal logic (logical streams) and optimizations to support Business Intelligence units in [publishing time series data services](https://observablehq.com/@lomoramic/extended-temporal-datatype-support) 3. Complex migrations for unification of PLM data across multiple systems 4. Extension and optimization of digital twin architectures 6. [ML feature development and data pipeline management](https://observablehq.com/d/3e7d17675a61834e) (private) 7. Operational predictive analytics for manufacturing processes All of these capabilities are enabled by a core symbolic representation engine developed by an expert team that continues to make available leading edge distributed data programming capabilities and practices through a fully managed service. 38:[{"id":"6340804a3417b3c9a96a085144e13c06571869531605ecb10a2eb4e0a7329d00eb4faa13f7837a80f8314cc76221b6bf4ec9ba937e6ad65dcec2a94cefdc4205","href":"https://static.observableusercontent.com/files/6340804a3417b3c9a96a085144e13c06571869531605ecb10a2eb4e0a7329d00eb4faa13f7837a80f8314cc76221b6bf4ec9ba937e6ad65dcec2a94cefdc4205","name":"Diego-Rivera-s-mural-of-factory-workers-at-Ford-s-River-Rouge-assembly-plant.png","type":"image/png","lastModified":1628770783800.49,"size":243153}] 2f:{"id":"fd4c51478a6e1def","slug":"enterprise-data-centric-digitization","title":"Enterprise Data Centric Digitization","thumbnail":"b51561949815b51e142eb590b37fae9388ecf2a65bb95568594b206d1a641a81","access_level":"public","create_time":"$D2021-08-12T12:19:43.800Z","update_time":"$D2022-09-14T11:48:28.369Z","view_time":null,"first_public_time":"$D2021-08-12T13:49:56.696Z","fork":true,"stars":1,"starred":false,"owner":{"type":"team","id":"ad42203b295b2120","login":"datagenous","name":"datagenous","avatar_url":"https://avatars-next.observableusercontent.com/c8136cc7228bf2ad689462f6a332e51dc9984c845619ac0f34ca730875bf0c7a","tier":"pro"},"version":324,"publish_version":324,"role":null,"authored":false,"authors":[{"type":"individual","id":"c769c086dc13ecf8","login":"ilabra","name":"Ivan Labra","avatar_url":"https://avatars-next.observableusercontent.com/8987cd5cf963a88d75b064ff67e941a7c2177a1b95421a786776c355403f8bcb","tier":null}],"members":[{"type":"individual","id":"c769c086dc13ecf8","login":"ilabra","name":"Ivan Labra","avatar_url":"https://avatars-next.observableusercontent.com/8987cd5cf963a88d75b064ff67e941a7c2177a1b95421a786776c355403f8bcb","tier":null,"role":"editor"},{"type":"individual","id":"987c90ad70c449c1","login":"lomoramic","name":"James Anderson","avatar_url":"https://avatars-next.observableusercontent.com/dc520bbc733259f2913522cf1d50607777b98a3083f290a6f182534561896497","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"subscribed":null,"forks":1,"collection":null,"type":"notebook","body":{"title":"Enterprise Data Centric Digitization","cells":[{"id":0,"value":"# Enterprise Data Centric Digitization \n## In which we describe how datageno.us machines helped a global leader in the semiconductor industry to evolve their data architectures","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":4,"value":"\n

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined"},{"id":246,"value":"---\n### Introduction","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":250,"value":"This paper discusses the adoption and evolution of our core data engine, __dydra__, and new practices and technologies developed as, __datageno.us__, with a client, a global leader in the semiconductor industry, over several years of evolving support. It outlines:\n- the shadow IT onboarding of the __dydra__ data engine and __datageno.us__ managed services by a front line business unit\n- the transition to being incorporated into the core Enterprise Architecture.\n- the evolving business conditions that led to new approaches to complex integrations\n- the factors for success in making the Enterprise core transition\n- the balance between the edge and the center in the enterprise\n- an overview of __dategeno.us__ support for the evolution digital twins and traceability \n- current and future areas of support and development for our client.\n\nIt reports a case study in how data-centric design and semantic representation technologies function in the context of a global enterprise.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":41,"value":"---\n### Shadow IT Sucess","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":8,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":102,"value":"---\n### Transition to a core part of Enterpise Architecture","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":35,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":110,"value":"---\n### Complex Integration","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":116,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":126,"value":"---\n### What made such success and transition possible?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":131,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":211,"value":"---\n### Empowering the Edge, Insulating the Center","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":215,"value":"$35","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":140,"value":"---\n### Datacentricity, Traceability, and Digitial Twins","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":154,"value":"$36","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":169,"value":"---\n### Future Directions","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":173,"value":"$37","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":266,"value":"","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"}],"files":"$W38","stdlib":"1","titling":"auto"}} 30:"$2f:body"