computeritblog

Artificial intelligence and data science are big news in 2023. The rise of generative artificial intelligence has, of course, led to this dramatic increase in visibility. So what could happen to this area in 2024 to keep it on the front page? And how will these trends impact business?

Over the past few months, we’ve conducted three data and technology leaders surveys. Two of them attended the MIT Director of Data and Information Quality Symposium—one sponsored by Amazon Web Services (AWS) and the other by Thoughtworks. The third survey was conducted by Wavestone, formerly NewVantage Partners, whose annual surveys we have previously written about. More than 500 senior executives took part in the new surveys, with perhaps some overlap in participants.
Surveys don’t predict the future but suggest what people closest to data science and companies’ artificial intelligence strategies and projects are thinking and doing. Based on this data, here are five major emerging issues that deserve your close attention:
Generative AI sparkles but needs to deliver value.
As mentioned, generative artificial intelligence has attracted enormous attention from businesses and consumers. But does it bring economic value to the organizations that use it? The survey results show that while interest in this technology is high, its value has not yet been realized. Many respondents believe that generative AI has transformational potential; 80% of AWS survey respondents said it will transform their organizations, and 64% of Wavestone survey respondents said it is next-generation technology. The vast majority of respondents are also increasing investment in technology. However, most companies are simply experimenting at the individual or departmental level. Only 6% of companies in the AWS survey have any production generative AI applications, and only 5% in the Wavestone survey have large-scale production deployments.
Production adoption of generative AI will, of course, require more investment and organizational change, not just experimentation. Business processes must be redesigned, and employees retrained (or perhaps, in some cases only, replaced by generative AI systems). New AI capabilities need to be integrated into existing technology infrastructure.
Perhaps the most significant changes are related to data – managing unstructured content, improving data quality, and integrating different sources. In an AWS survey, 93% of respondents agreed that data strategy is critical to benefiting from generative AI, but 57% have yet to make any changes to their data.
Data science is shifting from artisanal to industrial.
Companies are feeling the need to accelerate the creation of data science models. What were previously craft activities are becoming increasingly industrialized. Companies are investing in platforms, processes and methods, feature stores, machine learning operations systems (MLOps), and other tools to improve productivity and speed of deployment. MLOps systems monitor the health of machine learning models and determine whether they are still making accurate predictions. Otherwise, models may need to be retrained using new data.
Most of these capabilities are provided by external providers, but some organizations are now building their platforms. While automation (including automated machine learning tools, which we’ll discuss below) helps improve productivity and enable greater participation in data science, data science’s most tremendous boon to productivity is arguably the reuse of existing data sets, functions, variables, and even the entire model. READ MORE. computeritblog