Get out and vote

Tomorrow will be South Africa’s 6th general election since 1994’s first democratic election. Is the economy working for you? Are you getting the services you need? Are you being treated fairly? Let your voice be heard! Irrespective of your political affiliation – voting is the only practical way to have your voice be heard. I…

Data Governance versus Data Quality

Data quality and data governance both strive to optimise data and information to meet business needs. Simplistically, however, where Data Governance deals with the definition of, and responsibility for, data management standards, Data Quality deals with the practical implementation, monitoring and enforcement of these data management standards for individual platforms and systems. Both data governance…

true cost of ELT

Big data use case: Offloading the data warehouse to Hadoop

The true cost of ELT Today’s business world is demanding more from the data warehouse, because more than ever an organisation’s survival depends on its ability to transform data into actionable insights. However, ELT data integration workloads are now consuming up to 80% of database capacity, resulting in: Rising infrastructure costs Increasing batch windows Longer development cycles Slower…

dmx gui

What is data integration? (building a single view of the truth)

Data integration defined Data integration is a common industry term referring to the requirement to combine data from multiple separate business systems into a single unified view, often called a single view of the truth. This unified view is typically stored in a central data repository known as a data warehouse. For example, customer data integration involves…

Is data engineering overtaking data science

When Harvard Business Review first touted the data scientist as sexiest job of the 21st century  back in 2012 the role was still in its infancy. The promise of advanced analytics and the insights that business could gain – about their customers, their interactions, their products and everything else were rightly identified as potential gold. Data…

What is the role of the data engineer?

Data engineering is the term that has emerged to describe the tasks related to delivering useful data for analytics – particularly in relation to data science. With between 60% and 90% of the effort of most big data project allocated to data engineering tasks, the role has matured as organisations found that traditional data scientists…