How I began my data science journey
My Journey into AI started earlier than I expected, essentially while doing my undergraduate in India, in 2010. My Project was to recognize faces. It appeared quite fascinating back then, being aware of the hours I spent watching Hollywood movies. I held the anticipation of creating something similar, but the reality was far from it. During that period, I engaged in image smoothing, followed by Principal Component Analysis, and ultimately implemented a standard prediction algorithm, all using C programming. It was a source of significant frustration, as the model I constructed exhibited substantial bias. Moving ahead to 2018, when I commenced my Masters in Business Analytics, I came to realize the remarkable strides that had been made in Python programming. It became evident how effortlessly one could construct random forests and deep learning models. What had taken me around 10 lines of code to accomplish in my undergraduate project, now required at least 10,000+ lines of code in C. At that moment, I recognized the onset of a digital transformation.
Data is the foundation
My post here will cover what we as CIOs should be looking toward, and yes it involves us needing to invest in data science teams. But some of the fundamentals do need to be looked into before us as companies dive into the journey. It all starts from the data we are collecting. The more diligently we construct systems, the greater their rigidity, and the more boundaries we implement to capture precise data, the more effectively we position ourselves for success in the data science and AI journey.
A data lake is important
Acquiring good data, as highlighted earlier, holds paramount importance. However, the challenge lies in transitioning from a landscape of hundreds of tables with obscured column names to a framework where data engineers can excel. This predicament is effectively addressed through the utilization of data lakes. When new analyst joins the organization, with all the energy to make their name, they get disappointed/buried under the massive breadth of data a company may possess all over the place with various databases. NoSQL databases are a programmer’s delight but might not be as friendly to data scientists who require well-defined data that can be queried and seamlessly integrated into a data frame. The data we possess should be organized and managed akin to a menu card in a restaurant.
Math is complicated but needed to gauge an AI project
My bias here may raise some eyebrows, but it is true, that if you want to lead an AI/data science team, you need to know the fundamental math behind it. Chat GPT/other apps are blurring the lines of the skillset needed, but when the model/algorithm does not perform to your organization’s liking, it is the math that will help you discover what went wrong.
Your first few projects may not work
It is hard, unlike a programming project to guarantee success specifically living in the promotional products space. Unlike other giant organizations, the promo industry does not have the number of engineers needed in some cases to make these projects work from the get-go. Even large organizations fail/overestimate the success of these projects, and the reason is not incompetence, I believe the reason is all these algorithms are complex and compounded by data, hard to predict success. I would always go back to what my professors said, which is baselining projects. If we think a prediction of an outcome can be estimated by the average findings and or a coin toss, we can baseline what our algorithm needs to beat. In the case of a coin toss, a model with a 55% accuracy may be a win, and the success depends on the appetite to take risks and to be able to accept failures in certain cases. My bias will be toward bad data and or jumping into a project with a baseline difficult to beat, in those cases, AI may not be the right answer.
Putting it all together
We need to try somewhere as engineers, data science leaders, and or CIOs. The AI revolution is only going to get fiercer and we need to educate ourselves on how to use AI on our side. Sometimes AI can make us more efficient and cannot be measured by how many jobs we replaced but by how much more scalable we made our employees, so they can solve more complicated problems. There are plenty of resources online to learn and hone our skills. A CIO needs to understand finance to succeed, is a well-known statement. Similarly, I believe AI can thrive in the organization only with people with knowledge in the field vs. using Generative AI apps randomly. AI is as precise as a chef preparing a special meal, where each ingredient is crucial in the dish. Similarly, in AI algorithms, parameter tuning holds equal importance.
Read Rishiraj Mukherjee’s CXO Ladder story here: https://ciotechworld.com/a-journey-to-the-opposite-side-of-the-globe-a-climb-to-the-top-of-cxo-ladder/
I am a dynamic technology executive with over ten years of experience driving digital transformations, defining organizational strategies, and modernizing processes and solutions. My expertise encompasses machine learning, ERP implementations, IT management, business Intelligence, manufacturing, and supply chain management.
In addition, my background in software development provides me with the knowledge of end-to-end application development, from concept to delivery, and proficiency in data science, core programming, building user stories, and making data-driven decisions. Recognized as a trusted advisor with resilient and empathetic communication skills, I am uniquely able to gain buy-in and align diverse corporate functions under large-scale priorities.
