How to Become a Data AND AI Engineer

Open original

This article discusses a live session with Alejandro Aboy, a Senior Data and AI Engineer, about the evolving role of data engineers in the context of AI. Alejandro emphasizes that strong data engineering skills, particularly data modeling and orchestration, are crucial for successful AI work and shares practical steps for integrating AI into existing data workflows.

  • Data modeling accounts for roughly 80% of the skill transfer from data engineering to AI engineering, according to Aboy.
  • Good orchestration/workflow design can handle most AI tasks without needing complex agents.
  • Precise column/model descriptions function as prompts for AI, so metadata and documentation quality directly determine AI performance.
  • AI still lacks common-sense judgment (illustrated by a debugging agent missing context), so critically reviewing outputs rather than accepting them blindly remains essential.

Data engineering and AI are increasingly intertwined. Alejandro Aboy, a Senior Data Engineer, emphasizes that you can't effectively work with AI without a solid foundation in data engineering. Roughly 80–90% of his work relates to AI, but he insists that understanding data systems is crucial for developing robust AI use cases. He describes AI engineering as integrating various systems to provide the context AI needs to function, highlighting that poor data practices lead to weak AI performance. Aboy pinpoints data modeling as the key skill that transfers to AI work, accounting for about 80% of its impact. Good data models require careful planning and long-term thinking, which aligns with what he calls "context engineering." He also stresses the importance of orchestration skills, noting that a well-designed workflow can handle most AI tasks without needing complex agents. In his day-to-day work, he utilizes Claude Code for more than just coding; he refines prompts and iterates on rule files to optimize performance. When it comes to practical steps for integrating AI into data engineering, Aboy recommends identifying relevant AI use cases within your company and mapping out the data you control. Writing precise column and model descriptions can act as prompts for AI, and setting up metadata connection points (MCPs) with the tools you use is essential. He advises against accepting AI's first output blindly, emphasizing the importance of critical review to leverage your expertise effectively. One significant challenge in AI development remains the lack of common-sense judgment, as illustrated by Aboy's experience with a debugging agent that missed context in its suggestions. He also highlights using AI to clarify messy, unstructured requirements rather than waiting for perfect input. By feeding vague requests into AI, he identifies gaps and iterates until he has enough clarity to build. Senior engineers face the risk of losing their edge if they stop applying critical thinking, which remains vital in distinguishing between useful AI outputs and flawed ones.

Ask about this article