7 min read
|
Saved February 14, 2026
|
Copied!
Do you care about this?
This article discusses the evolving role of data engineers in the age of AI, emphasizing the need to adapt data preparation strategies. It highlights the shift from traditional data workflows to flexible, context-aware systems that prioritize data curation over mere collection.
If you do, here's more
Lak Lakshmanan's keynote at the upcoming "Data Engineering in the Age of AI" conference highlights the impact of Agentic AI on the role of data engineers. The core of the disruption stems from non-technical users increasingly engaging with data directly, using tools that allow them to create applications without needing deep technical knowledge. This shift has led to a new approach where AI models, particularly large language models (LLMs), serve as the central processing unit for both understanding user intent and carrying out tasks.
A significant change in data preparation involves rethinking traditional ETL (Extract, Transform, Load) processes. AI agents can interpret data within its context rather than relying solely on rigid schemas. For instance, a "loan amount" might mean different things in various tables, depending on whether it refers to a requested amount or the disbursed principal. This context-driven understanding calls for data engineers to question the necessity of every normalization step and to consider whether providing rich, contextual information could enhance AI performance.
The focus has shifted from sheer data collection to curation. In the past, the emphasis was on amassing large datasets to improve machine learning models. Now, the quality of examples provided to LLMs during in-context learning is more critical. Data engineers should prioritize maintaining high-quality, representative data examples that evolve with changing standards. This curation process may even involve implementing new storage solutions like graph and vector databases to support diverse data types.
Accessibility of data formats also plays a crucial role in how effectively AI agents can function. Formats that maintain semantic meaning and require little preprocessing allow for smoother interactions. Ultimately, data engineers must adapt their strategies not only to support these AI capabilities but also to enable a new persona: the data curator.
Questions about this article
No questions yet.