Click any tag below to further narrow down your results
+ data-engineering
(5)
+ ai
(4)
+ business-intelligence
(3)
+ semantics
(2)
+ analytics
(2)
+ relational-databases
(2)
+ agile
(1)
+ iteration
(1)
+ conways-law
(1)
+ just-in-time
(1)
+ organizational-structure
(1)
+ communication-patterns
(1)
+ artificial-intelligence
(1)
+ challenges
(1)
+ technology
(1)
Links
This article reruns a 2023 benchmark with the latest LLMs, comparing direct SQL generation against querying through a structured dbt Semantic Layer. It finds that while text-to-SQL accuracy has jumped, a modeled Semantic Layer still delivers near-perfect, deterministic results for covered queries, making it ideal for complex or critical use cases.
This article discusses a live session with Alejandro Aboy, a Senior Data and AI Engineer, about the evolving role of data engineers in the context of AI. Alejandro emphasizes that strong data engineering skills, particularly data modeling and orchestration, are crucial for successful AI work and shares practical steps for integrating AI into existing data workflows.
The article discusses the construction and business value of knowledge graphs, emphasizing their role in data organization and relational modeling. It explains how knowledge graphs differ from traditional databases, particularly in handling complex relationships and metadata. The piece also touches on the integration of knowledge graphs with AI, especially in enhancing large language models.
This article discusses effective data modeling techniques for private markets, emphasizing the need for clarity and adaptability in data structures. It highlights common challenges in managing unique schemas and offers practical solutions for organizing complex data relationships.
This article discusses how Whatnot implemented an AI-powered Slack bot to streamline data inquiries for their data scientists. It highlights key lessons learned about balancing flexibility and trustworthiness, the importance of context engineering, and the ongoing role of data scientists in this evolving landscape.
This article explores a shift in data modeling from rigid orthodoxies to a more pragmatic approach. It emphasizes starting with simple structures, adding complexity only when necessary, and leveraging semantic clarity for flexibility across different modeling techniques.
The article explores how AI will transform business processes, domains, and data models, potentially creating systems that feel alien to humans. It discusses the concept of AI agents developing their own knowledge and processes, shifting the focus of data modeling from "what" happened to "why" and "how." The author speculates on the emergence of new data formats and structures driven by AI.
This article explores the evolving role of data engineers over the past 50 years, highlighting their often unnoticed contributions to data infrastructure. It discusses the challenges they face, such as managing dependencies and schema changes, while emphasizing that the core problems remain unchanged despite new tools and technologies.
Since the inception of SQL in 1974, there has been a recurring dream to replace data analytics developers with tools that simplify the querying process. Each decade has seen innovations that aim to democratize data access, yet the complex intellectual work of understanding business needs and making informed decisions remains essential. Advances like AI can enhance efficiency but do not eliminate the crucial human expertise required in data analytics.
Real-time analytics solutions enable querying vast datasets, such as weather records, with rapid response times. The article outlines how to effectively model data in ClickHouse for optimized real-time analytics, covering techniques from ingestion to advanced strategies like materialized views and denormalization, while emphasizing the importance of efficient data flow and trade-offs between data freshness and accuracy.
The article discusses effective strategies for significantly reducing the size of Power BI data models, potentially achieving a reduction of up to 90%. It focuses on various techniques such as optimizing data types, removing unnecessary columns, and implementing aggregation to improve performance and efficiency in data analysis.
The article discusses the challenges and strategies of agentic data modeling in analytics, emphasizing the need for three key pillars: semantics for understanding, speed for rapid verification, and stewardship for governance. By integrating these elements, businesses can effectively leverage AI agents to enhance data insights while maintaining accuracy and trust.
Double-entry ledger modeling is underutilized in modern software development, despite its potential to simplify tracking financial transactions and other amount changes. By implementing a ledger system, developers can create a more robust and auditable way to manage various accounts, payments, and even user points, reducing complexity in their applications. Using a ledger can streamline data handling and improve error-checking across different use cases.
The content appears to be corrupted or unreadable, making it impossible to extract any meaningful information or insights from the article. As a result, there is no summary available for this piece.
The author critiques the Medallion Architecture promoted by Databricks, arguing that it is merely marketing jargon that confuses data modeling concepts. They believe it misleads new data engineers and pushes unnecessary complexity, advocating instead for traditional data modeling practices that have proven effective over decades.
The article discusses the limitations of artificial intelligence in addressing data modeling challenges, emphasizing that reliance on AI alone will not resolve fundamental issues within data structures. It argues that without a solid understanding of data modeling principles, organizations may struggle despite adopting advanced AI technologies.
The article discusses Conway's Law, which posits that the structure of a system reflects the communication structure of the organization that designed it. It explores how this principle applies to data modeling and the implications it has for the design and organization of data systems. By understanding Conway's Law, data professionals can create models that better align with the communication patterns of their teams.
Data modeling is considered "dead" by the author due to the shift in focus towards modern data architectures like Data Lakes and Lake Houses, which prioritize flexibility over traditional modeling techniques. The article criticizes the lack of clarity and guidance in contemporary data modeling practices, contrasting it with the structured approaches of the past, particularly those advocated by Kimball. The author expresses a longing for a definitive framework or authority to restore the importance of data modeling in the industry.
The article discusses the concept of just-in-time data modeling, emphasizing its importance in adapting data structures to meet immediate project needs rather than trying to predict future requirements. It advocates for a flexible and iterative approach to data modeling that allows teams to respond more effectively to changes and challenges as they arise.